A lane line processing method and apparatus

Abstract

This application provides a lane line processing method, including: obtaining a current positioning point, a reference line point set, and a lane line point set; determining a target search window based on the current positioning point and the reference line point set, matching the lane line point set with the reference line point set within the target search window to obtain the number of matching points and a lateral offset sequence; calculating the average lateral offset of the actual lane lines based on the lateral offset sequence, selecting the actual lane line with the most matching points as the reference lane line, and calculating the relative lateral offset between the reference lane line and the other lane lines; if the relative lateral offset meets the lane line width condition, generating a fitted lane line corresponding to the actual lane line using the lateral offset sequence of the actual lane lines and the reference line point set; marking all fitted lane lines using the lane width based on the sorted average lateral offset to obtain marking information; if a missing lane line is determined based on the marking information, determining the number of lane intervals between the missing lane line and the reference lane line based on the average lateral offset between all fitted lane lines, generating the average lateral offset of the missing lane line based on the number of lane intervals, the average lateral offset of the reference lane line, and the lane width; and generating a fitted lane line to complete the missing lane line based on the average lateral offset of the missing lane line and the reference line.

Description

Technical Field

[0001] This application relates to the field of data processing technology, and in particular to a lane line processing method and device. Background Technology

[0002] In autonomous driving systems, lane detection and processing are fundamental to achieving functions such as vehicle centering and lane keeping. Accurately identifying and completing lane information is crucial for improving driving safety and the driving experience. With the development of intelligent driving technology, obtaining stable, reliable, and complete lane information in complex driving scenarios has become a key technical challenge.

[0003] The existing lane line processing methods involve extracting lane line segments from a single frame image and then using geometric features such as angle and distance to merge these segments and fit curves, thereby generating continuous lane lines. However, in real-world driving environments, lane lines are often difficult for cameras to continuously and stably perceive completely due to wear, occlusion, shadows, or changes in lighting. When lane line quality is poor or partially missing, this method can easily lead to a degraded or even disengaged autonomous driving system, severely impacting user experience and driving safety. Summary of the Invention

[0004] One objective of this application is to provide a lane line processing method to solve the problem of inaccurate parking space detection in related technologies; another objective is to provide a lane line processing device; a third objective is to provide a lane line processing equipment; a fourth objective is to provide a vehicle; a fifth objective is to provide a computer-readable storage medium; and a sixth objective is to provide a computer program product.

[0005] To achieve the above objectives, the technical solution of this application embodiment is implemented as follows: This application provides a lane line processing method, the method including: Obtain the vehicle's current location point, the reference line point set of the reference line, and the lane line point set of at least one actual lane line identified by the on-board camera; Based on the current location point and reference line point set, determine the target search window of the reference line, and match each lane line point set with the reference line point set in the target search window to obtain the number of matching points for the corresponding actual lane line, as well as the lateral offset sequence of the actual lane line relative to the reference line. Based on each lateral offset sequence, the average lateral offset of each actual lane line is calculated, and the actual lane line with the most matching points is selected as the reference lane line from at least one actual lane line. Based on the average lateral offset of each actual lane line, the relative lateral offset between the reference lane line and the other lane lines is calculated; wherein, at least one actual lane line includes the reference lane line and the other lane lines. If the relative lateral offset meets the lane width condition, the fitted lane line corresponding to the actual lane line is generated using the lateral offset sequence of the actual lane line and the reference line point set. Based on the sorted average lateral offset, all fitted lane lines are marked using the lane width to obtain marking information. If it is determined that there is a missing lane line based on the marking information, the number of lane intervals between the missing lane line and the reference lane line is determined based on the average lateral offset between all fitted lane lines. Based on the number of lane intervals, the average lateral offset of the reference lane line, and the lane width, the average lateral offset of the missing lane line is generated. Based on the average lateral offset of the missing lane line and the reference line, a fitted lane line is generated to complete the missing lane line.

[0006] Based on the aforementioned technical means, the following steps are first taken: First, the vehicle's current positioning point, the reference line point set of the reference line, and the lane line point set of at least one actual lane line identified by the onboard camera are obtained. Next, a target search window for the reference line is determined based on the current positioning point and the reference line point set. Each lane line point set is then matched with the reference line point set within the target search window to obtain the number of matching points for each actual lane line and a sequence of lateral offsets relative to the reference line. Then, the average lateral offset of each actual lane line is calculated based on the lateral offset sequence, and the lane line with the most matching points is selected as the baseline lane line. The relative lateral offset between the baseline lane line and other lane lines is calculated. When the relative lateral offset meets the lane line width condition, a fitted lane line is generated using the lateral offset sequence and the reference line point set. The fitted lane line is then marked based on the sorted average lateral offset and the lane width. If a missing lane line is found, the number of lane intervals between the missing lane line and the baseline lane line is calculated based on the average lateral offset of the existing fitted lane lines, and the average lateral offset of the missing lane line is generated accordingly. Finally, lane line completion is achieved based on this average lateral offset and the reference line. By introducing mechanisms such as reference line guidance, adaptive search window, baseline selection, relative offset judgment, width constraint verification, and missing lane line estimation, the accuracy of lane line matching, generation reliability, and structural integrity are effectively improved, and the functional degradation problem caused by lane line occlusion, wear, or incomplete detection in the existing technology is solved.

[0007] This application provides a lane marking processing device, the device comprising: The module is used to obtain the vehicle's current positioning point, the reference line point set of the reference line, and the lane line point set of at least one actual lane line identified by the on-board camera. The determination module is used to determine the target search window of the reference line based on the current positioning point and the reference line point set, and match each lane line point set with the reference line point set in the target search window to obtain the number of matching points of the corresponding actual lane line, as well as the lateral offset sequence of the actual lane line relative to the reference line. The processing module is used to calculate the average lateral offset of each actual lane line based on each lateral offset sequence, and select the actual lane line with the most matching points from at least one actual lane line as the reference lane line, and calculate the relative lateral offset between the reference lane line and the other lane lines based on the average lateral offset of each actual lane line; wherein, at least one actual lane line includes the reference lane line and the other lane lines. The processing module is also used to generate a fitted lane line corresponding to the actual lane line by using the actual lane line's lateral offset sequence and the reference line point set if the relative lateral offset satisfies the lane line width condition. The processing module is also used to mark all fitted lane lines based on the sorted average lateral offset and the lane width to obtain marking information. If it is determined that there is a missing lane line based on the marking information, the number of lane intervals between the missing lane line and the reference lane line is determined based on the average lateral offset between all fitted lane lines. Based on the number of lane intervals, the average lateral offset of the reference lane line and the lane width, the average lateral offset of the missing lane line is generated. The processing module is also used to generate a fitted lane line corresponding to the missing lane line to complete the missing lane line based on the average lateral offset of the missing lane line and the reference line.

[0008] This application provides a lane marking processing device, including: Memory is used to store executable instructions or computer programs. When a processor executes computer-executable instructions or computer programs stored in the memory, it implements the lane line processing method provided in the embodiments of this application.

[0009] This application provides a vehicle that includes the lane marking device described above.

[0010] This application provides a computer-readable storage medium storing a computer program or computer-executable instructions for implementing the lane line processing method provided in this application when executed by a processor.

[0011] This application provides a computer program product, including a computer program or computer executable instructions. When the computer program or computer executable instructions are executed by a processor, they implement the lane line processing method provided in this application.

[0012] The beneficial effects of the embodiments of this application are as follows: (1) The offset calculation method based on reference lines is adopted, combined with the curvature adaptive window, which improves the accuracy and efficiency of lane line matching; (2) Lane merging is performed by introducing relative offset calculation based on the baseline, which avoids the error accumulation caused by simple merging and improves the quality of lane lines; (3) Combining Gaussian filtering and smoothing processing ensures that the generated lane lines are smooth and continuous, thus improving the reliability of the lane lines; (4) By using width verification and constraints, the generated lane lines are ensured to conform to the actual road conditions, which improves the accuracy of subsequent path planning; (5) A time-series filtering strategy is adopted to optimize lane lines using historical offset information, which improves the stability of the system in dynamic scenarios. Attached Figure Description

[0013] Figure 1 This is a schematic flowchart of the lane line processing method provided in the embodiments of this application; Figure 2 This is a schematic diagram of the lane line scenario provided in the embodiments of this application; Figure 3 This is a schematic diagram of the lane line processing device provided in the embodiments of this application; Figure 4 This is a schematic diagram of the lane line processing device provided in the embodiments of this application; Figure 5 This is a schematic diagram of the vehicle structure provided in the embodiments of this application.

[0014] It should be noted that the terms "first" and "second" mentioned above are only used to distinguish between different options and do not represent the degree of superiority or inferiority of the options or their priority in the implementation process. Detailed Implementation

[0015] To make the objectives, technical solutions, and advantages of this application clearer, the application will be further described in detail below with reference to the accompanying drawings. The described embodiments should not be regarded as limitations on this application. All other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0016] This application provides a lane line processing method applied to a lane line processing device. This method enables accurate detection of parking spaces in parking scenarios. The lane line processing device may include mobile phones, tablets, laptops, handheld computers, in-vehicle electronic devices, wearable devices, and ultra-mobile personal computers (Ultra). Mobile personal computers (UMPCs), netbooks or personal digital assistants (PDAs), servers, network attached storage (NAS), personal computers (PCs), etc.

[0017] Reference Figure 1 As shown, Figure 1 A schematic diagram of the implementation process of a lane line processing method provided in this application embodiment. Figure 1 The method may include the following steps: Step 101: Obtain the vehicle's current location point, the reference line point set of the reference line, and the lane line point set of at least one actual lane line identified by the vehicle camera.

[0018] In this embodiment of the application, the current location of the vehicle can be the coordinates of the vehicle's real-time position in the global Cartesian coordinate system obtained through the vehicle positioning system.

[0019] In this embodiment, the reference line can be a pre-stored driving route within a driving scenario containing lane lines (such as a road or garage), used as the route during actual lane line processing. This reference line is as follows: Figure 2 201 in the middle.

[0020] In this embodiment, the reference line point set is an ordered set of points obtained by sampling the reference line at a first equal interval. The reference line point set can be represented as... ,in, ,here, Reference line point Coordinates in the global Cartesian coordinate system Let be the heading angle of the i-th center point. It should be noted that the reference line point set is the set of reference line center points, that is, each reference line point in the reference line point set corresponds to the geometric center position of the reference line.

[0021] It is understandable that the sensing area of ​​a vehicle-mounted camera can be an area within a certain distance of the vehicle's location, such as a range of 5-10 meters (m) around the vehicle, so that the vehicle-mounted camera can collect images within the sensing area.

[0022] In this embodiment, the lane line point set of at least one actual lane line identified by the vehicle-mounted camera can be a set of lane line pixels extracted from the current frame image by an image recognition algorithm of the vehicle-mounted camera (such as semantic segmentation or edge detection based on deep learning), and mapped to a global Cartesian coordinate system after coordinate transformation. Here, each actual lane line corresponds to a physical lane line, and each lane line point set contains several consecutive lane line points, such as... Figure 2 202 in the middle.

[0023] It is understandable that, due to factors such as lane line wear, obstruction, shadows, changes in lighting, or the camera's sensing range, at least one actual lane line identified by the vehicle-mounted camera may be at least a portion of the physical lane lines in the current driving scenario. For example, on a two-lane road, the vehicle-mounted camera recognition system identifies three lane lines: the left lane line and the right lane line, forming two lane line point sets, each containing several consecutive points.

[0024] In some embodiments, after obtaining the reference line point set, the number of reference line points in the reference line point set is counted. If the number of reference line points in the reference line point set is greater than a preset number threshold, the reference line point set is sampled according to a second equal interval to obtain the final reference line point set. The final reference line point set is then used in the subsequent lane line processing flow. Here, the number of reference line points in the final reference line point set is less than or equal to the preset number threshold, and the second equal interval is an integer multiple of the first equal interval, thus reducing computational complexity.

[0025] In some embodiments, after obtaining the reference line point set, the number of reference line points in the reference line point set is counted. If the number of reference line points in the reference line point set is greater than a preset number threshold, the reference lines are resampled according to the third equal interval to obtain the final reference line point set. The final reference line point set is used in the subsequent lane line processing process. The number of reference line points in the final reference line point set is less than or equal to the preset number threshold, and the third equal interval is greater than the first equal interval, thus reducing the computational complexity.

[0026] In this embodiment, the lane line processing device obtains the vehicle's current positioning point through the vehicle positioning system, and simultaneously obtains the reference line point set corresponding to the reference line. Then, it calls the image processing module to perform lane line detection on the image captured by the vehicle camera, and outputs the lane line point set corresponding to at least one actual lane line.

[0027] Step 102: Based on the current positioning point and the reference line point set, determine the target search window for the reference line, and match each lane line point set with the reference line point set in the target search window to obtain the number of matching points for the corresponding actual lane line, as well as the lateral offset sequence of the actual lane line relative to the reference line.

[0028] In this embodiment, the target search window can be a range of points set around the vehicle's current location on a reference line, which is used to limit the range of reference line points for matching calculation and avoid increased computational complexity caused by global search.

[0029] In this embodiment, the matching point can be the reference line point in the reference line point set within the target search window that is closest to the lane line point in the lane line point set. It should be noted that different lane line points in the same lane line point set can be matched with the same reference line point in the reference line point set.

[0030] In this embodiment, the number of matching points reflects the visibility and quality of the lane lines in the current image.

[0031] In this embodiment, the lateral offset sequence can be the offset of a point in the lane line point set corresponding to each actual lane line within the target search window relative to its corresponding point on the reference line in the vertical direction (normal direction), used to characterize the positional relationship of the actual lane line relative to the reference line. The lateral offset sequence includes multiple lateral offsets, each calculated based on the Frenet coordinate system.

[0032] It can be understood that the Frenet coordinate system is a local coordinate system. This system uses the arc length *s* of the reference line as the vertical axis and the normal to the reference line as the horizontal axis to describe the geometric offset of the actual lane line relative to the reference line. For example, on a straight road, the *s*-axis of the reference line is along the direction of travel, and the *l*-axis points to the left side of the road. At curves, the *s*-axis of the Frenet coordinate system remains along the tangent to the reference line, and the *l*-axis is always perpendicular to the tangent, ensuring the continuity and consistency of the lateral offset sequence calculation in the Frenet coordinate system.

[0033] In this embodiment, the lane line processing device determines the size of the matching target search window based on the vehicle's current positioning point and the reference line point set. Then, within the target search window, it performs nearest neighbor matching on the points in each lane line point set and calculates the lateral offset of each point in the lane line point set relative to the reference line, forming a lateral offset sequence. Thus, by introducing the target search window and the Frenet coordinate system, the matching process significantly improves the accuracy and efficiency of lane line matching, especially in complex road sections such as curves, effectively reducing the introduction of mismatched points and enhancing the robustness of the algorithm.

[0034] Step 103: Based on each lateral offset sequence, calculate the average lateral offset of each actual lane line, and select the actual lane line with the most matching points from at least one actual lane line as the reference lane line. Based on the average lateral offset of each actual lane line, calculate the relative lateral offset between the reference lane line and the other lane lines.

[0035] At least one of the actual lane lines includes the base lane line and the remaining lane lines.

[0036] In this embodiment of the application, the average lateral offset is used to characterize the overall offset trend of the actual lane line relative to the reference line. The average lateral offset can be obtained based on the lateral offset sequence of the actual lane line.

[0037] In some embodiments, the average lateral offset of the actual lane line can be obtained by averaging the sequence of lateral offsets of the actual lane line.

[0038] In some embodiments, the lateral offset sequence of the actual lane line can be filtered, and the mean of the filtered lateral offset sequence can be calculated to obtain the average lateral offset of the actual lane line.

[0039] In some embodiments, after obtaining the average lateral offset of the actual lane line, the current average lateral offset can be time-series filtered based on the historical average lateral offset of the actual lane line to obtain the filtered average lateral offset of the actual lane line, which can be used for subsequent lane line processing. The filtered average lateral offset of the actual lane line can be obtained by the following formula (1).

[0040]

[0041] in, This represents the average lateral offset after filtering the actual lane lines. This represents the historical average lateral offset of the actual lane lines, such as the average lateral offset collected at the previous moment. This represents the average lateral offset at the current moment. This is the weighting coefficient, which can be adjusted according to actual needs, such as 0.98.

[0042] It is understandable that reference lines are routes formed by manually driven vehicles in a driving scenario. These routes are not necessarily parallel to lane lines, which can significantly interfere with the offset results. Since actual lane lines have good parallelism and good visibility and stability, in order to achieve accurate lane line processing, one actual lane line can be selected as the reference lane line from the identified lane lines to participate in the subsequent lane line processing process.

[0043] In this embodiment, the reference lane line can be the actual lane line with the most matching points among the identified actual lane lines, used as a reference for calculating the relative positions of other actual lane lines. It should be noted that the number of matching points reflects the visibility and quality of the actual lane line in the current frame. Selecting the actual lane line with the most matching points as the reference lane line ensures the reliability of filling in missing lane lines in the event of missing lane lines.

[0044] In this embodiment, the relative lateral offset can be the difference in lateral offset between the other actual lane lines and the reference lane line, used to determine whether lane merging is necessary. The relative lateral offset can be obtained by the following formula (2).

[0045]

[0046] in, This represents the relative lateral offset of the remaining actual lane lines relative to the reference lane lines. This represents the average lateral offset of the remaining actual lane lines. This represents the average lateral offset of the baseline lane line.

[0047] In this embodiment, the lane line processing device calculates the average lateral offset of each actual lane line based on its lateral offset sequence; then, it counts the number of matching points for each actual lane line and selects the lane line with the largest number of matching points using formula (3); finally, it calculates the relative lateral offset of the remaining actual lane lines relative to the reference lane line. Thus, by introducing a reference lane line selection mechanism and relative offset calculation, the error accumulation caused by simple merging is effectively avoided, improving the quality and reliability of lane line merging.

[0048]

[0049] in, For the identification ID of the baseline lane line, The identifier ID of the kth actual lane line, where K is the number of actual lane lines detected by the camera.

[0050] Step 104: If the relative lateral offset satisfies the lane width condition, use the lateral offset sequence of the actual lane line and the reference line point set to generate the fitted lane line corresponding to the actual lane line.

[0051] In this embodiment, the lane width condition is used to verify whether adjacent actual lane lines may overlap or belong to the same lane, thereby determining whether to merge them. The lane width condition includes: the relative lateral offset is less than the minimum lane width threshold, which can be 1.5m. The lane width condition can be expressed by the following formula (4).

[0052]

[0053] in, It is the absolute value of the relative lateral offset. The minimum width threshold for lane lines is defined by the fact that the physical width of adjacent lane lines in actual roads is generally between 0.15 and 0.3 meters, and the standard spacing between adjacent lane lines is 3.5 to 3.75 meters. When the relative lateral offset is less than 1.5 meters, it means that the two actual lane lines may belong to the same lane or have overlapping areas, and they need to be merged.

[0054] In this embodiment, the fitted lane line is a complete and smooth lane line trajectory generated by interpolation or curve fitting based on the lateral offset sequence of the actual lane line and the reference line point set.

[0055] In this embodiment, the lane line processing device determines whether the lane line width condition is met based on the relative lateral offset. If it is met, a fitted lane line is generated using the lateral offset sequence and the reference line point set. Thus, by introducing a width constraint verification mechanism, the generated fitted lane line is ensured to conform to the actual road conditions, avoiding incorrect lane line merging or omissions due to misjudgment, thereby improving the accuracy and safety of subsequent path planning.

[0056] In some embodiments, if the relative lateral offset does not meet the lane width condition, the lateral offset sequence of the reference lane line is merged with the lateral offset sequence of the actual lane line corresponding to the relative lateral offset that does not meet the lane width condition, and the merged lateral offset sequence is used as the lateral offset sequence of the reference lane line.

[0057] It is understandable that in actual road environments, the physical width of lane lines is typically between 0.15 meters and 0.3 meters, and the standard spacing between adjacent lane lines is between 3.5 meters and 3.75 meters. When the calculated relative lateral offset is less than the minimum width threshold, it indicates that the relative lateral offset does not meet the lane line width condition, suggesting that the two lane lines may belong to the same lane or have overlapping areas. In this case, merging is required to avoid generating duplicate or incorrect lane lines. Therefore, the lateral offset sequence of the reference lane line is merged with the lateral offset sequence of the actual lane line corresponding to the relative lateral offset that does not meet the lane line width condition, and the merged lateral offset sequence is used as the lateral offset sequence of the reference lane line. At the same time, the lane line point set of the reference lane line is merged with the lane line point set of the actual lane line corresponding to the relative lateral offset that does not meet the lane line width condition, and the merged lane line point set is used as the lane line point set of the reference lane line. Furthermore, based on the current lateral offset sequence of the reference lane lines, the average lateral offset of the reference lane lines is calculated. Furthermore, based on the average lateral offset of the reference lane lines and the average lateral offset of the remaining actual lane lines, the relative lateral offset between the reference lane lines and the remaining actual lane lines is calculated. The remaining actual lane lines include all lane lines other than those already merged. If the relative lateral offset meets the lane width condition, a fitted lane line corresponding to the actual lane line is generated using the lateral offset sequence of the actual lane lines and the reference line point set. Thus, when the relative offset of adjacent lane lines exceeds the lane width threshold, merging their lateral offset sequences allows for the fusion of information from multiple lane lines, improving the completeness and reliability of the reference lane line data.

[0058] Step 105: Based on the sorted average lateral offset, mark all fitted lane lines using the lane width to obtain marking information. If it is determined that there is a missing lane line based on the marking information, determine the number of lane intervals between the missing lane line and the reference lane line based on the average lateral offset between all fitted lane lines. Based on the number of lane intervals, the average lateral offset of the reference lane line, and the lane width, generate the average lateral offset of the missing lane line.

[0059] In this embodiment, the labeling information can be derived by sorting all fitted lane lines according to their average lateral offset, and then logically classifying their relative positions in the lateral arrangement based on the lane width. The labeling information includes, but is not limited to, types such as left-left, left, center, right, and right-right. Here, left represents the fitted lane line furthest to the left of the reference line, center represents the fitted lane line in the middle position, and right represents the fitted lane line furthest to the right of the reference line. It should be noted that the labeling information of the fitted lane lines not only facilitates the subsequent inference of missing lane lines but also helps the path planning module understand the lane structure.

[0060] In this embodiment, based on the sorted average lateral offset, if the difference between two adjacent average lateral offsets is greater than the lane width, it is determined that a middle lane line is missing between the fitted lane lines corresponding to the two adjacent average lateral offsets. For example, on a two-lane road, the fitted lane line with the largest offset is marked as left, the middle one as middle, and the smallest as right. If only two fitted lane lines are detected, and their offsets are +1.2 meters and +3.8 meters respectively, with a lane width of 3.5 meters, then there should be a missing lane line between the two fitted lane lines.

[0061] In this embodiment, the number of lane intervals can be the number of intervals between the missing lane line and the reference lane line in the lateral arrangement, used to estimate the offset of the reference lane line. For example, if the reference lane line is the middle lane line and the missing lane line is located to the left of the reference lane line, the number of lane intervals is 1; if it is located in the second position from the left, the number of intervals is 2. The number of lane intervals is estimated based on the offset distribution of the detected fitted lane lines and the lane width.

[0062] In this embodiment of the application, the average lateral offset of the missing lane line can be obtained by the following formula (5).

[0063]

[0064] in, This represents the average lateral offset of the missing lane lines in the Frenet coordinate system based on the reference line. This represents the average lateral offset of the baseline lane line in the Frenet coordinate system. This refers to the number of lane intervals. The lane width W is determined using historical statistics or prior map width.

[0065] For example, if the average lateral offset of the baseline lane line is 1.2 meters, the lane width is 3.5 meters, and the missing lane line is located one interval to the left of the missing lane line, then the offset of the missing lane line is 1.2 - 1 × 3.5 = -2.3 meters.

[0066] In this embodiment, after obtaining the average lateral offset of all fitted lane lines, the lane line processing device sorts the average lateral offset of all fitted lane lines in ascending or descending order, and marks the fitted lane lines as left, center, and right based on the lane width; then, it checks whether the three marks of left, center, and right exist in the sorted average lateral offset to determine whether there is a missing lane line. Here, if no reasonable marks of these three can be found in the sorted average lateral offset, it indicates that there is a missing lane line. At this time, the number of lane intervals between the missing lane line and the reference lane line is determined based on the distribution of the average lateral offset of the detected valid fitted lane lines; finally, the average lateral offset of the missing lane line is calculated using the above formula (5). In this way, by introducing a priori lane width and interval estimation mechanism, intelligent completion of missing lane lines is realized, effectively restoring the road structure and improving the robustness of the system in scenarios with partial lane line loss.

[0067] Step 106: Based on the average lateral offset of the missing lane line and the reference line, generate the fitted lane line corresponding to the missing lane line to complete the missing lane line.

[0068] In this embodiment, the fitted lane line corresponding to the missing lane line is a complete lane line trajectory calculated based on the average lateral offset of the missing lane line and the reference line point set of the reference line.

[0069] In this embodiment, the lane line processing device calculates the fitting points of the missing lane line point by point based on the calculated average lateral offset of the missing lane line and the reference point set of the reference line, and connects them to form a complete fitted lane line. Thus, by using the known lane line offset relationships for calculation, the road structure can be restored when lane lines are missing, ensuring that the autonomous driving system obtains complete and continuous road boundary information, thereby supporting safe and reliable path planning and lane keeping functions.

[0070] The lane line processing method provided in this application embodiment obtains the current positioning point of the vehicle, the reference line point set of the reference line, and the lane line point set of at least one actual lane line identified by the vehicle camera. Then, based on the current positioning point and the reference line point set, a target search window for the reference line is determined, and each lane line point set is matched with the reference line point set within the target search window to obtain the number of matching points for each actual lane line and a sequence of lateral offsets relative to the reference line. Next, based on the lateral offset sequence, the average lateral offset of each actual lane line is calculated, and the lane line with the most matching points is selected as the reference lane line. The relative lateral offset between the reference lane line and other lane lines is calculated. When the relative lateral offset meets the lane line width condition, a fitted lane line is generated using the lateral offset sequence and the reference line point set. The fitted lane line is then marked based on the sorted average lateral offset combined with the lane width. If a missing lane line is found, the number of lane intervals between the missing lane line and the reference lane line is calculated based on the average lateral offset of the existing fitted lane lines, and the average lateral offset of the missing lane line is generated accordingly. Finally, lane line completion is achieved based on the average lateral offset and the reference line. By introducing mechanisms such as reference line guidance, adaptive search window, baseline selection, relative offset judgment, width constraint verification, and missing lane line estimation, the accuracy of lane line matching, generation reliability, and structural integrity are effectively improved, and the functional degradation problem caused by lane line occlusion, wear, or incomplete detection in the existing technology is solved.

[0071] In some embodiments, determining the target search window for the reference line in step 102 based on the current positioning point and the reference line point set can be achieved through the following steps.

[0072] Step 121: Determine the candidate reference point that is closest to the current positioning point from the set of reference line points.

[0073] In this embodiment of the application, the candidate reference point is the reference line point that is closest to the current positioning point in the reference line point set.

[0074] In some implementations, the Euclidean distance between the current positioning point and any reference point in the reference point set can be calculated using the following formula (6), and the reference point corresponding to the minimum distance can be selected as the candidate reference point.

[0075]

[0076] in, For the current location point and reference line point The Euclidean distance between them Let i be the coordinates of the i-th reference line point in the global Cartesian coordinate system. The coordinates of the current location point in the global Cartesian coordinate system.

[0077] In other implementations, to improve search efficiency, a binary search + difference acceleration method can be used to search for the candidate reference point closest to the current positioning point in the reference line point set.

[0078] In some other implementations, to improve search efficiency, a reference line spatial index can be established based on the reference line point set, and the candidate reference point closest to the current positioning point can be found in the reference line spatial index.

[0079] In this embodiment of the application, NanoFlann can be used. Data structure, establishing reference lines based on reference line points. Spatial index. Reference line. Spatial indexes can reduce the time complexity of nearest neighbor queries from O(n) to O(logn), significantly improving the processing performance of large-scale reference line data.

[0080] In this embodiment of the application, after establishing the reference line After spatial indexing, based on the current location point, on the reference line... Find the nearest neighbor point and obtain the matching index of the candidate reference point on the reference line that is closest to the current positioning point. The spatial index calculation of the candidate reference point can be expressed by the following formula (7).

[0081]

[0082] in, For the matching index of candidate reference points, These are the coordinates of the current location point. Let i be the coordinates of the reference line point.

[0083] Step 122: Obtain the curvature of the candidate reference points, and determine the target search window for the reference line based on the curvature of the candidate reference points.

[0084] In this embodiment, the curvature of a candidate reference point refers to the degree of local bending of the reference line segment where the candidate reference point is located, used to characterize the geometric shape changes of the road. A larger curvature value indicates a more curved road near the candidate reference point; conversely, a smaller curvature value indicates a straighter road. In this application, the curvature information of the candidate reference point is used to adaptively adjust the size of the target search window to achieve accurate matching in different road scenarios.

[0085] In this embodiment, the target search window is a range defined by the candidate reference points on the reference line, used to limit the range of reference line points for matching calculations and avoid increased computational complexity caused by global search. The size of the target search window is not fixed, but dynamically adjusted according to the curvature of the candidate reference points.

[0086] In this embodiment of the application, after obtaining the curvature of the candidate reference point, the size of the target search window of the reference line can be adaptively adjusted according to the curvature of the candidate reference point.

[0087] In this embodiment of the application, the target search window of the reference line is determined based on the curvature of the candidate reference point, which can be achieved through the following process: if the curvature of the candidate reference point is greater than the curvature threshold, the preset search window of the candidate reference point is narrowed to obtain the target search window of the candidate reference point; if the curvature of the candidate reference point is less than or equal to the curvature threshold, the preset search window is determined as the target search window; wherein, the search window includes a forward search window and a backward search window, and the forward search window is larger than the backward search window.

[0088] In this embodiment of the application, the adjustment logic of the target search window can be represented by the following formula (8).

[0089]

[0090]

[0091] in, The curvature of the candidate reference point, The curvature threshold can be set to 0.12. PRE_WINDOW_SIZE is the preset size of the backward search window, and POST_WINDOW_SIZE is the preset size of the forward search window.

[0092] As described above, in this embodiment, the target search window for the reference line is determined based on the curvature of the candidate reference point. Specifically, when the curvature of the candidate reference point is detected to be greater than the curvature threshold, it indicates that the current road segment is in a high curvature region (such as a sharp bend or an S-shaped curve). In this case, the forward search window and the backward search window starting from the candidate reference point are reduced to decrease the introduction of mismatched points and improve matching accuracy. When the curvature of the candidate reference point is less than or equal to the curvature threshold, it indicates that the current road segment is in a low curvature region (such as a straight road or a low curvature road segment). In this case, the default forward search window size and the backward search window size are used to ensure that there is sufficient search range on straight roads or gentle curves, thereby improving the stability of matching. In this way, the matching range can be flexibly adjusted under different road curvature conditions, thereby improving the accuracy of the lane offset calculation and enhancing the robustness and safety of the autonomous driving system under complex road conditions.

[0093] In some embodiments, the matching of each lane line point set with the reference line point set in the target search window in step 102 to obtain the number of matching points for the corresponding actual lane line and the lateral offset sequence of the actual lane line relative to the reference line can be achieved through the following steps.

[0094] Step 123: Match each lane line point in the set of lane line points with the reference line points on the reference line in the target search window to obtain the matching point corresponding to each lane line point on the reference line, thereby obtaining the number of matching points for the corresponding actual lane line.

[0095] In practical applications, to facilitate the description of the lateral offset of the actual lane line relative to the reference line, the reference line is established as a Frenet coordinate system. In this system, the vertical axis (s-axis) is the tangent direction along the road centerline, where the s-value represents the cumulative distance the vehicle has traveled along the road from its starting point. The horizontal axis (l-axis) is the normal direction perpendicular to the road centerline, where the l-value represents the distance the vehicle deviates from the centerline. Values ​​are positive on the left and negative on the right. Therefore, the state of a reference point on the reference line in the Frenet coordinate system can be represented by (s, l).

[0096] In this embodiment, the coordinates of each reference point in the reference point set in the Cartesian coordinate system, and the coordinates of each lane point in each lane point set in the Cartesian coordinate system, can be converted to coordinates in the Frenet coordinate system. Then, the reference point set in the Frenet coordinate system can be represented as follows: ,in, , Reference line point The coordinates in the Frenet coordinate system; the lane line point set in the Frenet coordinate system can be represented as... ,in, , Lane line points Coordinates in the Frenet coordinate system.

[0097] In the Frenet coordinate system, 's' represents the arc length along the reference line, and 'l' represents the lateral offset perpendicular to the tangential direction of the reference line. For any point on the lane line point set... Find the nearest line segment or reference point on the reference line within the target search window to obtain the lane line point. Matching point on the reference line .

[0098] Here, the lane line points are determined. Matching point on the reference line This can be achieved through the following process: If the i-th segment of the reference line is Then the lane line point The projection coefficient on the line segment can be calculated using the following formula (9).

[0099]

[0100] in, Lane line points The projection coefficients on the i-th segment, As the starting point of the i-th segment, This is the endpoint of the i-th line segment. It should be noted that... ,when or When the time is right, it is truncated to the interval [0,1] to ensure that the matching point (also known as the projection point) is located on the i-th segment of the reference line.

[0101] Furthermore, after obtaining the projection coefficients, based on the projection coefficients and the i-th segment... The lane line point is determined using formula (10). Matching point on the reference line .

[0102]

[0103] In this embodiment, each lane line point in the lane line point set is matched with a reference line point on a reference line within the target search window to obtain the matching point corresponding to each lane line point on the reference line. The number of matching points corresponding to each actual lane line is counted, thus obtaining the total number of matching points corresponding to all actual lane line points. In this way, by accurately matching the lane line point set with the reference line point set within the target search window, the efficiency and accuracy of matching are improved. This matching method effectively reduces false matching caused by an excessively large search range, thereby improving the robustness of lane line recognition and providing a more reliable data foundation for subsequent offset calculation and lane line generation.

[0104] Step 124: Determine the lateral offset between each lane line point and its corresponding matching point to obtain a lateral offset sequence.

[0105] In this embodiment of the application, the lateral offset between each lane line point and its corresponding matching point can be determined through the following process: After obtaining the lane line point set, each lane line point Matching point on the reference line Then, the reference line at the matching point is determined using the following formula (11). Unit tangent vector at the location The unit tangent vector is calculated using the following formula (12). The corresponding unit normal vector Finally, based on the lane line point The lane line point Matching point on the reference line and unit normal vector The lane line points are calculated using formula (13). Signed lateral offset relative to the reference line For each lane line point set The corresponding horizontal offset sequence can be obtained. .

[0106]

[0107]

[0108]

[0109] in, Lane line points Located to the left of the reference line, Lane line points Located to the right of the reference line, Use the reference line at the matching point unit tangent vector at point Two components in the x and y directions of the global coordinate system.

[0110] In this embodiment, by calculating the lateral offset between lane point sets and matching points in each lane point set and forming a lateral offset sequence, a precise quantitative description of the lane point set relative to the reference line is achieved. This provides continuous and accurate geometric relationship data for subsequent offset filtering, temporal smoothing, and lane line generation, thereby improving the accuracy and consistency of lane line processing and ultimately enhancing the perception capability and path planning reliability of the autonomous driving system in complex road environments.

[0111] In some embodiments, the calculation of the average lateral offset of each actual lane line based on each lateral offset sequence in step 103 can be achieved through the following process: averaging the lateral offset sequences of each actual lane line to obtain an initial average lateral offset; calculating the offset difference between each lateral offset in the lateral offset sequence and the initial average lateral offset, and removing lateral offsets in the lateral offset sequence whose offset difference is greater than a threshold; averaging the lateral offset sequences again to obtain the average lateral offset of the actual lane line.

[0112] In this embodiment of the application, for each actual lane line's lateral offset sequence, the following formula (14) is used to average the lateral offset sequence to obtain the initial average lateral offset corresponding to the actual lane line; the offset difference between each lateral offset in the lateral offset sequence and the initial average lateral offset is calculated, and lane line points in the lateral offset sequence whose offset difference is greater than the difference threshold are regarded as abnormal points, and the lane line points are removed from the set of lane line points, and the lateral offset corresponding to the abnormal points is removed from the lateral offset sequence; further, the lateral offset sequence obtained after removal is averaged again to obtain the average lateral offset of the actual lane line, thereby obtaining the average lateral offset of all actual lane lines, and finally, the average lateral offset of all actual lane lines is sorted to prepare for subsequent marking work.

[0113]

[0114] in, This represents the average lateral offset corresponding to the actual lane lines.

[0115] As described above, this embodiment of the application first calculates the initial average lateral offset for each actual lane offset sequence, then calculates the offset difference between each lateral offset in the lateral offset sequence and the initial average lateral offset, removes lateral offsets in the lateral offset sequence whose offset difference is greater than a threshold, and finally re-averages the remaining effective lateral offsets to obtain a more accurate average lateral offset. This effectively suppresses the influence of anomalies caused by sensor noise, occlusion, and changes in lighting, thereby improving the robustness and accuracy of lane position estimation, and ultimately enhancing the reliability and safety of path planning in complex road environments for autonomous driving systems.

[0116] In some embodiments, the process of generating a fitted lane line corresponding to the actual lane line in step 104 using the lateral offset sequence of the actual lane line and the reference line point set can be achieved through the following steps: For each actual lane line, a target lane line point is determined based on at least a portion of each reference line point on the reference line, the lateral offset between the reference line point and the corresponding lane line point, the average lateral offset of the actual lane line, and a first heading angle; wherein the first heading angle is the angle between the line connecting the reference line point and the corresponding lane line point and the reference line; the target lane line point of each actual lane line is fitted to generate a fitted lane line corresponding to the actual lane line.

[0117] In this embodiment, for each valid actual lane line, if a reference point on the reference line has a corresponding lane line point on the actual lane line through the aforementioned matching process, the coordinates of the target lane line point of the valid actual lane line are determined using the following formula (15) based on the coordinates of the reference point, the lateral offset between the reference point and the corresponding lane line point, and the first heading angle between the line connecting the reference point and the corresponding lane line point and the reference line. If a reference point on the reference line does not have a corresponding lane line point on the actual lane line through the aforementioned matching process, the coordinates of the target lane line point of the valid actual lane line are determined using the following formula (15) based on the coordinates of the reference point, the average lateral offset of the actual lane line, and the first heading angle corresponding to the reference point. Further, the target lane line point of each actual lane line is fitted to generate a fitted lane line corresponding to the actual lane line.

[0119] It should be noted that if there is no matching point between the reference line point and the target lane line point, the position of the reference line point corresponding to the matching point of the effective actual lane line can be determined based on the average lateral offset of the reference line point and the effective actual lane line. The coordinates of the target lane line point of the effective actual lane line can then be obtained. The target lane line point of each actual lane line can then be fitted to generate the fitted lane line corresponding to the actual lane line.

[0120] To suppress discrete point noise in the fitted lane line corresponding to the actual lane line and improve the continuity and smoothness of the lane line, in some embodiments, after generating the fitted lane line corresponding to the actual lane line using the lateral offset sequence of the actual lane line and the reference line point set in step 104, the following steps can also be performed: Step 141: For each actual lane line, obtain the curvature of the lane line point to be smoothed in the fitted lane line, and the arc length coordinates of the target lane line point in the local window along the lane curve in the local Frenet coordinate system of the actual lane line. The local window is a sliding window determined based on the position of the lane line point to be smoothed. The target lane line point in the local window includes the lane line point to be smoothed and the neighboring lane line points of the lane line point to be smoothed.

[0121] Here, the local window can be a sliding window consisting of l adjacent points before and after any point in the lane line to be smoothed in the fitted lane line.

[0122] Here, the target lane line point within the local window in the fitted lane line can be any lane line point to be smoothed in the fitted lane line, as well as the l target lane line points adjacent to the lane line point to be smoothed.

[0123] Here, for each valid actual lane line, the fitted lane line can be defined as follows: .

[0124] Step 142: Adjust the Gaussian kernel parameters according to the curvature of the lane line points to be smoothed.

[0125] In this embodiment of the application, in order to avoid the area being overly smoothed, the Gaussian kernel parameter can be dynamically adjusted according to the curvature of the lane line points to be smoothed by the following formula (16).

[0126]

[0127] in, For the lane line points to be smoothed The corresponding Gaussian kernel parameters, Based on the core width, For the lane line points to be smoothed curvature, This is an adjustment coefficient. It decreases the smoothing range when the curvature increases and enhances the smoothing effect when the curvature is small.

[0128] Step 143: Based on the Gaussian kernel parameters, the arc length coordinates of the lane line points to be smoothed, and the arc length coordinates of the neighboring lane line points, determine the Gaussian weighted coefficients of the neighboring lane line points to be smoothed.

[0129] In this embodiment, based on the Gaussian kernel parameters, the arc length coordinates of the lane line points to be smoothed, and the arc length coordinates of the neighboring lane line points, the Gaussian weighted coefficient of the neighboring lane line points to be smoothed can be determined by the following formula (17).

[0130]

[0131] in, The Gaussian weighting coefficients for the neighborhood lane line points to be smoothed lane line points. For adjacent lane line points The coordinates of the arc length on the lane line, For the lane line points to be smoothed The coordinates of the arc length on the lane line.

[0132] Step 144: Based on the Gaussian weighted coefficients corresponding to the neighboring lane line points and the planar coordinates of the neighboring lane line points in the Cartesian coordinate system, smooth the lane line points to be smoothed to obtain the smoothed planar coordinates of the lane line points to be smoothed.

[0133] In this embodiment, based on the Gaussian weighted coefficients corresponding to the neighboring lane line points and the plane coordinates of the neighboring lane line points in the Cartesian coordinate system, the smoothed plane coordinates of the lane line points to be smoothed can be obtained by formula (18).

[0134]

[0135] in, Let [il, i+l] be the smoothed planar coordinates of the lane line points to be smoothed, and let [il, i+l] be the local window. These are the neighboring lane line points within the local window.

[0136] As described above, in this embodiment, by introducing a curvature-adaptive Gaussian kernel parameter adjustment mechanism, combined with arc length coordinate calculation in the local Frenet coordinate system and Gaussian weighted smoothing, refined optimization of the fitted lane lines is achieved. This allows for dynamic response to changes in road geometry, precise control of the smoothing range, and the generation of high-quality lane lines that are both smooth and continuous while retaining key features, significantly improving the perception accuracy and driving safety of the autonomous driving system.

[0137] In some embodiments, step 106, which generates a fitted lane line to complete the missing lane line based on the average lateral offset of the missing lane line and the reference line, can be achieved through the following process: determining the plane coordinates of the reference line at the arc length coordinates and the unit normal vector of the tangent of the reference line at the arc length coordinates; performing a translation transformation along the direction of the unit normal vector based on the plane coordinates of the reference line at the arc length coordinates and the average lateral offset of the missing lane line to obtain the plane coordinates of the missing lane line at the arc length coordinates; and fitting the missing lane line based on all the plane coordinates corresponding to the missing lane line to generate the fitted lane line of the missing lane line.

[0138] In this embodiment, for each arc length coordinate on the reference line, the plane coordinate vector corresponding to the reference line at that arc length coordinate and the unit normal vector perpendicular to the tangent of the reference line at that arc length coordinate are obtained. Based on the plane coordinate corresponding to the reference line at that arc length coordinate, the unit normal vector perpendicular to the tangent of the reference line at that arc length coordinate, and the average lateral offset of the missing lane line to be filled, the plane coordinate corresponding to the missing lane line on the arc length coordinate s can be obtained using the following formula (19), thereby obtaining the plane coordinate corresponding to the missing lane line at all arc length coordinates on the reference line. Based on the plane coordinate of the missing lane line, the missing lane line points of the missing lane line are fitted to generate the fitted lane line of the missing lane line.

[0139]

[0140] in, For reference, the plane coordinates corresponding to the missing lane line at the arc length coordinate 's' on the reference line. Let s be the plane coordinates corresponding to the arc length coordinate s on the reference line. This represents the average lateral offset of the missing lane line. Let be the unit normal vector perpendicular to the tangent of the reference line at the arc length coordinate s.

[0141] As described above, by utilizing the geometric information of the reference line and the average lateral offset of the missing lane line, a translation transformation is performed along the normal vector direction to generate the planar coordinates of the missing lane line. This coordinates are then fitted to the missing lane line, thus achieving reasonable completion of the missing lane line. This method utilizes the relative positional relationship between lanes and prior width information, without relying on additional sensors or historical data, to restore the road structure, significantly improving the robustness and usability of the system in cases of partial lane line loss.

[0142] This application provides a lane line processing device, with reference to... Figure 3 As shown, Figure 3 This is a schematic diagram of a lane line processing device provided in an embodiment of this application. The lane line processing device 3 includes: The module 301 is used to obtain the vehicle's current positioning point, the reference line point set of the reference line, and the lane line point set of at least one actual lane line identified by the vehicle-mounted camera. The determination module 302 is used to determine the target search window of the reference line based on the current positioning point and the reference line point set, and match each lane line point set with the reference line point set in the target search window to obtain the number of matching points of the corresponding actual lane line and the lateral offset sequence of the actual lane line relative to the reference line. The processing module 303 is used to calculate the average lateral offset of each actual lane line based on each lateral offset sequence, and select the actual lane line with the most matching points from at least one actual lane line as the reference lane line, and calculate the relative lateral offset between the reference lane line and the other lane lines based on the average lateral offset of each actual lane line; wherein, at least one actual lane line includes the reference lane line and the other lane lines. The processing module 303 is also used to generate a fitted lane line corresponding to the actual lane line by using the lateral offset sequence of the actual lane line and the reference line point set if the relative lateral offset satisfies the lane line width condition. The processing module 303 is also used to mark all fitted lane lines according to the sorted average lateral offset and the lane width to obtain marking information. If it is determined that there is a missing lane line according to the marking information, the number of lane intervals between the missing lane line and the reference lane line is determined based on the average lateral offset between all fitted lane lines. Based on the number of lane intervals, the average lateral offset of the reference lane line and the lane width, the average lateral offset of the missing lane line is generated. The processing module 303 is also used to generate a fitted lane line corresponding to the missing lane line to complete the missing lane line based on the average lateral offset of the missing lane line and the reference line.

[0143] This application provides a hardware entity diagram of a lane line processing device, such as... Figure 4 As shown, the hardware entity of the lane line processing device 4 includes: a processor 401 and a memory 402, wherein the memory 402 stores a computer program that can run on the processor 401, and the processor 401 executes the computer program to implement some or all of the steps in the lane line processing method as described in the above embodiments.

[0144] The memory 402 stores computer programs that can run on the processor. The memory 402 is configured to store instructions and applications that can be executed by the processor 401. It can also cache data to be processed or already processed by the processor 401 and the various modules in the lane line processing device 4 (e.g., image data, audio data, voice communication data and video communication data). It can be implemented by flash memory or random access memory (RAM).

[0145] The processor 401 executes the program to implement the steps of the lane line processing execution method described above. The processor 401 typically controls the overall operation of the lane line processing device 4.

[0146] This application provides a vehicle, referring to... Figure 5 As shown, Figure 5 This is a schematic diagram of the structure of a vehicle provided in an embodiment of this application. The vehicle 5 includes the lane line processing device 4 described above.

[0147] This application provides a computer-readable storage medium storing one or more computer programs, which can be executed by one or more processors to implement some or all of the steps in the above-described method. The storage medium can be transient or non-transient.

[0148] This application provides a computer program including computer-readable code, wherein when the computer-readable code is run in a vehicle, a processor in the vehicle executes some or all of the steps in the above-described method.

[0149] This application provides a computer program product, which includes a non-transitory computer-readable storage medium storing a computer program. When the computer program is read and executed by a computer, it implements some or all of the steps in the above-described method. This computer program product can be implemented specifically through hardware, software, or a combination thereof. In some embodiments, the computer program product is specifically embodied as a computer storage medium; in other embodiments, the computer program product is specifically embodied as a software product, such as a software development kit (SDK), etc.

[0150] It should be noted that the descriptions of the various embodiments above tend to emphasize the differences between them, while their similarities or commonalities can be referred to interchangeably. The descriptions of the above embodiments of the device, storage medium, computer program, and computer program product are similar to the descriptions of the above method embodiments and have similar beneficial effects. For technical details not disclosed in the embodiments of the device, storage medium, computer program, and computer program product of this application, please refer to the descriptions of the method embodiments of this application for understanding.

[0151] The aforementioned processor can be at least one of the following: Application Specific Integrated Circuit (ASIC), Digital Signal Processor (DSP), Digital Signal Processing Device (DSPD), Programmable Logic Device (PLD), Field Programmable Gate Array (FPGA), Central Processing Unit (CPU), Controller, Microcontroller, and Microprocessor. It is understood that other electronic devices can also implement the functions of the aforementioned processor, and this application does not specifically limit the specific implementation.

[0152] The aforementioned computer storage media / memory can be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), magnetic random access memory (FRAM), flash memory, magnetic surface memory, optical disc, or compact disc read-only memory (CD-ROM), etc.; or it can be various terminals that include one or any combination of the above-mentioned memories, such as mobile phones, computers, tablet devices, personal digital assistants, etc.

[0153] It should be understood that the phrase "one embodiment" or "an embodiment" throughout the specification means that a specific feature, structure, or characteristic related to the embodiment is included in at least one embodiment of this application. Therefore, "in one embodiment" or "in an embodiment" appearing throughout the specification do not necessarily refer to the same embodiment. Furthermore, these specific features, structures, or characteristics can be combined in any suitable manner in one or more embodiments. It should be understood that in the various embodiments of this application, the sequence numbers of the above steps / processes do not imply a sequential order of execution; the execution order of each step / process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application. The sequence numbers of the above embodiments of this application are merely descriptive and do not represent the superiority or inferiority of the embodiments.

[0154] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0155] In the several embodiments provided in this application, it should be understood that the disclosed devices and methods can be implemented in other ways. The device embodiments described above are merely illustrative. For example, the division of units is only a logical functional division, and in actual implementation, there may be other division methods, such as: multiple units or components can be combined, or integrated into another system, or some features can be ignored or not executed. In addition, the coupling, direct coupling, or communication connection between the various components shown or discussed can be through some interfaces, and the indirect coupling or communication connection between devices or units can be electrical, mechanical, or other forms.

[0156] The units described above as separate components may or may not be physically separate. The components shown as units may or may not be physical units. They may be located in one place or distributed across multiple network units. Some or all of the units may be selected to achieve the purpose of this embodiment according to actual needs.

[0157] In addition, each functional unit in the various embodiments of this application can be integrated into one processing unit, or each unit can be a separate unit, or two or more units can be integrated into one unit; the integrated unit can be implemented in hardware or in the form of hardware plus software functional units.

[0158] Those skilled in the art will understand that all or part of the steps of the above method embodiments can be implemented by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When the program is executed, it performs the steps of the above method embodiments. The aforementioned storage medium includes various media that can store program code, such as mobile storage devices, read-only memory (ROM), magnetic disks, or optical disks.

[0159] Alternatively, if the integrated units described above are implemented as software functional modules and sold or used as independent products, they can also be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence or the part that contributes to related technologies, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause an in-vehicle terminal (which may be a personal computer, server, or network device, etc.) to execute all or part of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as mobile storage devices, ROMs, magnetic disks, or optical disks.

[0160] The above description is merely an embodiment of this application and is not intended to limit the scope of protection of this application. Any modifications, equivalent substitutions, and improvements made within the spirit and scope of this application are included within the scope of protection of this application.

Claims

1. A lane marking processing method, characterized in that, The method includes: Obtain the vehicle's current location point, the reference line point set of the reference line, and the lane line point set of at least one actual lane line identified by the on-board camera; Based on the current positioning point and the reference line point set, the target search window of the reference line is determined, and each lane line point set is matched with the reference line point set in the target search window to obtain the number of matching points for the corresponding actual lane line, as well as the lateral offset sequence of the actual lane line relative to the reference line. Based on each of the lateral offset sequences, the average lateral offset of each of the actual lane lines is calculated, and the actual lane line with the most matching points is selected from the at least one actual lane line as the reference lane line. Based on the average lateral offset of each of the actual lane lines, the relative lateral offset between the reference lane line and the other lane lines is calculated; wherein, the at least one actual lane line includes the reference lane line and the other lane lines. If the relative lateral offset satisfies the lane width condition, the fitted lane line corresponding to the actual lane line is generated using the lateral offset sequence of the actual lane line and the reference line point set. Based on the sorted average lateral offset, all fitted lane lines are marked using the lane width to obtain marking information. If it is determined that there is a missing lane line based on the marking information, the number of lane intervals between the missing lane line and the reference lane line is determined based on the average lateral offset between all fitted lane lines. Based on the number of lane intervals, the average lateral offset of the reference lane line, and the lane width, the average lateral offset of the missing lane line is generated. Based on the average lateral offset of the missing lane line and the reference line, a fitted lane line is generated to complete the missing lane line.

2. The method according to claim 1, characterized in that, The step of determining the target search window for the reference line based on the current positioning point and the reference line point set includes: Determine the candidate reference point that is closest to the current positioning point from the set of reference line points; Obtain the curvature of the candidate reference points, and determine the target search window for the reference line based on the curvature of the candidate reference points.

3. The method according to claim 2, characterized in that, The step of determining the target search window for the reference line based on the curvature of the candidate reference points includes: If the curvature of the candidate reference point is greater than the curvature threshold, the preset search window of the candidate reference point is reduced to obtain the target search window of the candidate reference point; If the curvature of the candidate reference point is less than or equal to the curvature threshold, the preset search window is determined as the target search window; wherein, the search window includes a forward search window and a backward search window, and the forward search window is larger than the backward search window.

4. The method according to any one of claims 1 to 3, characterized in that, The step of matching each set of lane line points with the set of reference line points within the target search window to obtain the number of matching points for the corresponding actual lane line, and the sequence of lateral offsets of the actual lane line relative to the reference line, includes: Each lane line point in each set of lane line points is matched with a reference line point on a reference line within the target search window to obtain the matching point corresponding to each lane line point on the reference line, thereby obtaining the number of matching points for the corresponding actual lane line. The lateral offset between each lane line point and its corresponding matching point is determined to obtain the lateral offset sequence.

5. The method according to any one of claims 1 to 3, characterized in that, The step of calculating the average lateral offset of each actual lane line based on each of the lateral offset sequences includes: For each actual lane line's lateral offset sequence, the average of the lateral offset sequence is calculated to obtain the initial average lateral offset; Calculate the offset difference between each horizontal offset in the horizontal offset sequence and the initial average horizontal offset; Remove horizontal offsets from the horizontal offset sequence whose offset difference is greater than a difference threshold; The average of the lateral offset sequence is calculated again to obtain the average lateral offset of the actual lane line.

6. The method according to any one of claims 1 to 3, characterized in that, The method further includes: If the relative lateral offset does not meet the lane width condition, the lateral offset sequence of the reference lane line is merged with the lateral offset sequence of the actual lane line corresponding to the relative lateral offset that does not meet the lane width condition, and the merged lateral offset sequence is used as the lateral offset sequence of the reference lane line.

7. The method according to any one of claims 1 to 3, characterized in that, The step of generating a fitted lane line corresponding to the actual lane line using the lateral offset sequence of the actual lane line and the reference line point set includes: For each actual lane line, a target lane line point for each actual lane line is determined based on at least a portion of each reference line point on the reference line, the lateral offset between the reference line point and the corresponding lane line point, the average lateral offset of the actual lane line, and a first heading angle; wherein, the first heading angle is the angle between the line connecting the reference line point and the corresponding lane line point and the reference line. For each actual lane line, the target lane line points are fitted to generate the fitted lane line corresponding to the actual lane line.

8. The method according to claim 7, characterized in that, After fitting the target lane line points of each actual lane line to generate the fitted lane line corresponding to the actual lane line, the method includes: For each actual lane line, the curvature of the lane line point to be smoothed of the fitted lane line is obtained, as well as the arc length coordinates of the target lane line point within the local window along the lane curve in the local Frenet coordinate system of the actual lane line. The local window is a sliding window determined based on the position of the lane line point to be smoothed, and the target lane line point within the local window includes the lane line point to be smoothed and the neighboring lane line points of the lane line point to be smoothed. Adjust the Gaussian kernel parameters based on the curvature of the lane line points to be smoothed; Based on the Gaussian kernel parameters, the arc length coordinates of the lane line point to be smoothed, and the arc length coordinates of the neighboring lane line points, the Gaussian weighting coefficient of the neighboring lane line points to the lane line point to be smoothed is determined. Based on the Gaussian weighted coefficients corresponding to the neighboring lane line points and the planar coordinates of the neighboring lane line points in the Cartesian coordinate system, the lane line points to be smoothed are smoothed to obtain the smoothed planar coordinates of the lane line points to be smoothed.

9. The method according to any one of claims 1 to 3, characterized in that, The step of generating a fitted lane line corresponding to the missing lane line to complete the missing lane line based on the average lateral offset of the missing lane line and the reference line includes: Determine the plane coordinates of the reference line at the arc length coordinates and the unit normal vector of the tangent line of the reference line at the arc length coordinates; Based on the plane coordinates corresponding to the reference line in the arc length coordinates and the average lateral offset of the missing lane line, a translation transformation is performed along the unit normal vector direction to obtain the plane coordinates corresponding to the missing lane line at the arc length coordinates. Based on the planar coordinates of the missing lane line points, the missing lane line is fitted to generate a fitted lane line for the missing lane line.

10. A lane marking processing device, characterized in that, include: Memory is used to store executable instructions or computer programs. A processor, when executing computer-executable instructions or computer programs stored in the memory, implements the lane line processing method according to any one of claims 1 to 9.

Images