Methods and systems for constructing a lane line map

Abstract

A system includes a control module in communication with one or more sensors of a plurality of vehicles. The control module is configured to receive a bitmap having a plurality of pixels representing a plurality of lane lines of a roadway detected by the one or more sensors, extract a plurality of line components of the plurality of lane lines, detect whether a branch exists in a line component of the plurality of line components, in response to detecting the branch in the line component, split the line component into two or more subcomponents, generate a plurality of line points for each of the subcomponents using a regression model, generate at least one line based on the plurality of line points, and generate a map of the roadway including the at least one line. Other example systems and example methods for generating maps of roadways are also disclosed.

Description

INTRODUCTION

[0001] The present invention relates to a system according to the preamble of claim 1 for constructing a lane line map, as is essentially known from US 2024 / 0 068 836 A1.

[0002] Further state of the art can be found in the article entitled "Deep learning in lane marking detection: A survey" by ZHANG, Youcheng [et al.], published in IEEE transactions on intelligent transportation systems, Vol. 23, 2022, No. 7, pp. 5976-5992, ISSN 1558-0016; as well as in the article entitled "Using edit distance and junction feature to detect and recognize arrow road marking" by HE, Yuhang [et al.], published in IEEE 17th international conference on intelligent transportation systems (ITSC), October 8-11, 2014, Qingdao, China. IEEE, 2014. pp. 2317-2323, ISBN 9781-1-4799-6078-1.

[0003] A roadway map can be created using various images and / or data. For example, the roadway map can be created based on aerial and / or satellite imaging. Such methods require data labeling. In other examples, the roadway map can be created based on received collectively compiled data. For example, sensor data (e.g., global positioning system (GPS) data, detected lane line data, speed data, yaw data, etc.) from vehicles can be collected and uploaded to a cloud-based system. The cloud-based system can then run aggregation algorithms to combine various observations from the vehicles into map content, such as lane lines. SUMMARY

[0004] According to the invention, a system for creating a map of a roadway is presented, which is characterized by the features of claim 1.

[0005] According to other features, the control module is configured to identify connected components of the plurality of lane lines in the bitmap and categorize each connected component as a line component of the plurality of line components.

[0006] According to other features, the control module is configured to implement a plurality of masks to extract the plurality of line components.

[0007] According to other features, the control module is configured to skeletonize each line component of the plurality of line components to generate a skeletonized image and detect the branch based on the skeletonized image.

[0008] According to other features, the control module is configured to sample the skeletonized image with a kernel, determine a value associated with the kernel, and compare the value to a threshold to detect the branch in the line component.

[0009] According to other features, the control module is configured to determine an angle between each pair of the plurality of vectors and to combine one or more pairs of the plurality of vectors based on the angle and a defined threshold to divide the component into the two or more subcomponents.

[0010] According to other features, the regression model includes a B-spline regression model.

[0011] According to other features, the control module is configured to determine a length of each of the subcomponents and generate the plurality of line points for each of the subcomponents using the regression model based on the determined length for the subcomponent.

[0012] According to other features, the control module is configured, in response to non-detection of the branch in the line component, to generate a plurality of line points for the line component using the regression model and to create a line based on the plurality of line points for the line component.

[0013] Further described is a method for creating a map of a roadway. The method includes receiving a multi-layer probability density bitmap having a plurality of pixels representing a plurality of lane lines of the roadway detected by one or more sensors of a plurality of vehicles, extracting a plurality of line components of the plurality of lane lines, detecting whether a branch exists in a line component of the plurality of line components, in response to detecting the branch in the line component, splitting the line component into two or more subcomponents, generating a plurality of line points for each of the subcomponents using a regression model, creating at least one line based on the plurality of line points, and generating a map of the roadway having the at least one line.

[0014] In other features, the method further comprises identifying connected components of the plurality of lane lines in the bitmap and categorizing each connected component as a line component of the plurality of line components.

[0015] According to other features, extracting the plurality of line components of the plurality of lane lines includes implementing a plurality of masks to extract the plurality of line components.

[0016] According to other features, the method further comprises skeletonizing each line component of the plurality of line components to generate a skeletonized image.

[0017] According to other features, detecting whether the branch exists in the line component includes detecting the branch based on the skeletonized image.

[0018] According to other features, detecting the branch based on the skeletonized image comprises sampling the skeletonized image with a kernel, calculating a value associated with the kernel, and comparing the value with a threshold to detect the branch in the line component.

[0019] According to other features, the method further comprises, in response to detecting the branch in the line component, removing a branch point representing the detected branch in the line component to create a plurality of vectors.

[0020] According to other features, splitting the line component into two or more subcomponents comprises determining an angle between each pair of the plurality of vectors and bundling one or more pairs of the plurality of vectors based on the angle and a defined threshold to split the component into the two or more subcomponents.

[0021] According to other features, generating the plurality of line points for each of the subcomponents includes determining a length of each of the subcomponents and generating the plurality of line points for each of the subcomponents using the regression model based on the determined length for the subcomponent.

[0022] According to other features, the regression model includes a B-spline regression model.

[0023] A non-transitory computer-readable medium storing instructions that, when executed by a control module, cause the control module to extract a plurality of line components from a plurality of lane lines in a multi-layer probability density bitmap, detect whether a branch exists in a line component of the plurality of line components, in response to detecting the branch in the line component, split the line component into two or more subcomponents, generate a plurality of line points for each of the subcomponents using a regression model, create at least one line based on the plurality of line points, and generate a map of a roadway including the at least one line.

[0024] Further areas of applicability of the present invention will become apparent from the detailed description, claims, and drawings. The detailed description and specific examples are intended for illustrative purposes only and are not intended to limit the scope of the invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] The present invention will be more fully understood from the detailed description and the accompanying drawings, in which: Fig. 1 is a block diagram of an example system for lane line map construction using probability density bitmaps in accordance with the present invention; Fig. Figure 2 is a side view of a vehicle with sensors according to the present invention; Fig. 3 a schematic diagram of observed lane lines of vehicles on a roadway and a map of the roadway generated by the system of Fig. 1 is created according to the present invention; Fig. 4-5 are flowcharts of example methods for creating lane line maps using probability density bitmaps in accordance with the present invention; Fig. 6 is a flowchart of an example method for splitting a line component with a detected branch according to the present invention; Fig. 7 is a schematic diagram of a method for extracting a line component and skeletonizing the extracted line component according to the present invention; Fig. 8 is a schematic diagram of an example kernel with a 3x3 configuration according to the present invention; and Fig. 9 is a schematic diagram of a method for splitting a line component with a detected branch according to the present invention.

[0026] In the drawings, reference symbols may be used multiple times to identify similar and / or identical elements. DETAILED DESCRIPTION

[0027] Lane maps can be created based on collectively collected data received from vehicles. For example, sensor data such as GPS data, detected lane line data, speed data, yaw data, etc. from vehicles can be collected and processed. Algorithms can then be used to generate lane lines and other map content for a lane map based on the vehicle sensor data. However, in such examples, GPS positions of an individual vehicle are not consistently accurate due to GPS errors, including errors caused by gravitational effects, solar radiation, timing inaccuracies in satellite clocks and / or reception clocks, etc. Furthermore, detected lane line data may be inconsistent across multiple vehicle passes due to camera occlusions, perception errors, and / or other limitations.Furthermore, conventional algorithms for generating lane lines and other map content are inefficient (e.g., slow processing speed, inaccurate, etc.).

[0028] As an example, multiple vehicles may travel in the same lane (e.g., a left-hand lane) of a roadway and provide vehicle sensor data to generate lane lines on a map. However, due to GPS errors, the vehicles' GPS trajectories may appear as if the vehicles are in different lanes. Furthermore, due to perceptual errors associated with the detected lane line data, detected lanes may be reported as erroneous types; detected lane line types for a lane edge (e.g., a right-hand lane line) may be different (e.g., a dashed lane line, a solid lane line, etc.); the detected lane line color may be inconsistent (e.g., some runs report a white color while other runs report a yellow color), etc.This leads to difficulties in aligning the geometries and associated attributes of the collected collective data to produce a representative result.

[0029] The systems and methods according to the present invention provide solutions for creating lane maps by utilizing collectively collected vehicle sensor data and bitmaps to construct lane lines. Lane lines may be constructed, for example, by receiving multi-layer probability density bitmaps representing aggregated lane line data, lane line clustering based on processing the multi-layer probability density bitmaps, and regression analysis for each separate lane line. Using this method, lane lines can be constructed accurately and faster than conventional lane line construction methods. For example, using the systems and methods herein, lane lines can be constructed approximately four times faster than conventional methods while maintaining, and in some cases increasing, the accuracy of the constructed lane lines.

[0030] With reference now to Fig. 1 shows a block diagram of an example system 100 for creating a map of a roadway. In the example of Fig. 1, the system 100 may be a cloud-based system and generally includes a control module 102 (e.g., a system control module) and a memory circuit 104. As in Fig. 1, the control module 102 is generally in communication with multiple vehicles 106, 108, 110. For example, the control module 102 may be in wireless communication with sensors in each vehicle 106, 108, 110 and / or an intermediate control module in each vehicle 106, 108, 110. Although Fig. 1 shows the system 100 as comprising three vehicles 106, 108, 110 in communication with the control module 102, it should be appreciated that the control module 102 may be in communication with more vehicles, such as thousands of vehicles.

[0031] In the example of Fig. 1, the control module 102 receives collectively collected data from the vehicles 106, 108, 110 on a roadway 112 having lane lines 114, 116, 118. In such examples, the vehicle sensors may collect information and generate sensor data indicative of the collected information, and then transmit the data to the control module 102. As further explained herein, the control module 102 creates lane lines representing the lane lines 114, 116, 118 for the roadway 112 and creates a map of the roadway 112 with the created lane lines.

[0032] In various embodiments, the vehicles 106, 108, 110 and / or any other vehicle may be in communication with the control module 102 of Fig. 1 may be any suitable vehicle, such as an electric vehicle (e.g., a pure electric vehicle, a plug-in hybrid electric vehicle, etc.), an internal combustion engine vehicle, etc. In addition, the vehicles may be autonomous vehicles, semi-autonomous vehicles, etc. By way of example only, the vehicles 106, 108, 110 may be trucks, sedans, coupes, sport utility vehicles (SUVs), recreational vehicles (RVs), etc.

[0033] Fig. 2, for example, illustrates a vehicle 200 that may be any of the vehicles 106, 108, 110 of Fig. 1 can represent. As in Fig. 2, the vehicle 200 includes a control module 202, a display module 240 and sensors 242, 244. Although Fig. 2 shows the vehicle 200 as including two sensors 242, 244, it should be appreciated that the vehicle 200 (or any other vehicle herein) may include more than two sensors.

[0034] In the example of Fig. 2, the sensors 242, 244 can be connected to the control module 102 of Fig. 1 are in wireless communication (e.g., directly and / or via the control module 202). The sensors 242, 244 generally collect information and generate sensor data indicative of the collected information. The sensors 242, 244 may include, for example, GPS transceivers, yaw sensors, speed sensors, cameras, etc. In such examples, the GPS transceivers detect the vehicle location, the speed sensors detect the vehicle speed, the yaw sensors determine the vehicle heading, and the cameras capture images of a roadway (e.g., the roadway 112 of Fig. 1) relative to the vehicle (e.g., in front of, behind, etc. the vehicle). In particular, the cameras may capture images of the lane lines 114, 116, 118 of the roadway 112 and the control module 202 (and / or the control module 102 of Fig. 1) may detect the lane lines 114, 116, 118 partially based on the captured images and / or other sensor data.

[0035] In the example of Fig. 2, the display module 240 may be a device in communication with the control module 202. In such examples, the display module 240 may receive data from the control module 202 and / or output data to the control module 202. For example, the display module 240 may display a lane map generated by the control module 202 (and / or the control module 102 of Fig. 1) and is visible to a user (e.g., a driver, passenger, etc.) in the vehicle 200.

[0036] With reference to Fig. 1-2, the control module 102 may receive the sensor data (e.g., acquired lane line data and vehicle GPS data) from the vehicles 106, 108, 110. The sensors (e.g., the sensors 242, 244 of Fig. 2) The vehicles 106, 108, 110 may, for example, transmit the sensor data to the control module 102 via one or more transceivers. The sensor data may, for example, include GPS data and lane line data. In such examples, the GPS data indicates the location of their corresponding vehicle 106, 108, 110. Additionally, the lane line data may include lane line geometry data and lane line attribute data detected by one or more cameras of the vehicles 106, 108, 110. In some examples, the lane line data may correspond to lane lines in the form of polynomial curves. In this example, the lane lines may be processed data (e.g., polynomial curves). In various embodiments, the lane line data may include information about the lane lines 114, 116, 118 observed by the one or more cameras, such as: B. Lane line color, lane line type (e.g.a solid line, a dashed line, etc.), lane line geometry, etc.

[0037] The control module 102 constructs a lane line map using probability density bitmaps. For example, and as further explained herein, the control module 102 uses received collectively gathered vehicle sensor data to create bitmaps including the lane lines 114, 116, 118. The control module 102 then generates and outputs a lane map including details about the lane lines 114, 116, 118 of the roadway 112. In various embodiments, the lane map may be a high-definition (HD) map, including precise details (e.g., centimeter-level details) and usable in autonomous driving applications.

[0038] The vehicles 106, 108, 110 may, for example, be in the same lane of the roadway 112, as in Fig. 1. However, due to errors, the sensor data from the vehicles 106, 108, 110 may be perceived to indicate that the vehicles 106, 108, 110 are traveling along different lanes and / or lanes with different lane line types. Fig. 3 represents, for example, the vehicles 106, 108, 110 of Fig. 1 in the same lane of roadway 112 (e.g., between lane lines 114, 116), but their sensor data indicates that vehicles 106, 108, 110 are in different lanes and / or in lanes with different lane line types. Specifically, although vehicles 106, 108, 110 are in the same lane, the sensor data from vehicles 106, 108, 110 indicate that vehicle 106 is in a lane between lane lines 314, 316, vehicle 108 is in a lane between lane lines 314', 316', and vehicle 110 is in a lane between lane lines 314', 316'. In addition, the sensor data of the vehicles 106, 108, 110 indicate that the lane lines 318, 318', 318" (which correspond to the lane line 118 of Fig. 1) are of different types (e.g., a dashed lane line, a solid lane line, etc.). As further explained herein, the control module 102 uses bitmaps corresponding to the vehicle sensor data (which represent the different lane lines of Fig. 3) to construct lane lines 114, 116, 118 as shown in Fig. 3 shown.

[0039] In various embodiments, the control module 102 may be Fig. 1 may be programmed to execute instructions, such as any one or more of the methods described herein. In such examples, the memory circuit 104 of Fig. 1 and / or another suitable computer-readable medium storing the instructions for execution by the control module 102.

[0040] Fig. 4-6 represent example processes 400, 500, 600, which are implemented by the system 100 of Fig. 1. In particular, and as further explained below, methods 400, 500, 600 of Fig. 4-6 to creating or otherwise constructing lane lines using probability density bitmaps and generating maps (e.g., HD maps, etc.) with the constructed lane lines. Although the example methods 400, 500, 600 with respect to the system 100 of Fig. 1 with the control module 102, any of the methods 400, 500, 600 may be usable by any other suitable system.

[0041] As in Fig. 4, method 400 begins at 402, where control module 102 receives collectively collected sensor data from vehicles 106, 108, 110 (among other vehicles) about lane lines of a roadway (e.g., lane lines 114, 116, 118 of roadway 112). As explained above, the sensor data may be collected by sensors (e.g., sensors 242, 244 of Fig. 2) of the vehicles 106, 108, 110. In such examples, the sensor data may include GPS data, lane line data, etc. The method 400 then proceeds to 404.

[0042] At 404, the control module 102 performs a GPS distortion correction process. For example, the GPS data received from any one or more of the vehicles 106, 108, 110 may include GPS distortion specific to a GPS transceiver within the vehicle. In such examples, the control module 102 may correct or otherwise account for any distortion associated with the GPS transceiver of the vehicle 106, 108, 110 that is providing the GPS data. Method 400 then proceeds to 406.

[0043] At 406, the control module 102 performs a GPS random noise reduction process. In such examples, the control module 102 may reduce noise from the GPS transceiver (e.g., one of the sensors 242, 244 of Fig. 2) using one or more filters. The method 400 then proceeds to 408.

[0044] At 408, the control module 102 receives a bitmap-based lane line map using the received sensor data from the vehicles 106, 108, 110. For example, the control module 102 may rely on the received sensor data, such as the GPS data, the lane line data, heading data, and speed data of the vehicles 106, 108, 110, to create the bitmap-based lane line map. In such examples, the control module 102 may create multiple multi-layered bitmaps for each of the vehicles 106, 108, 110 using the sensor data. Then, the control module 102 may aggregate or merge the multi-layered bitmaps of each vehicle 106, 108, 110 to create a multi-layered probability density bitmap to represent the observed lane lines of the roadway. With this configuration, the created bitmap includes several pixels representing the observed lane lines of the roadway.The method 400 then proceeds to 410.

[0045] At 410, the control module 102 performs a process to create or otherwise construct lane lines using the received multi-layer probability density bitmap. For example, and as further explained below, the control module 102 may detect candidate pixels in the multi-layer probability density bitmap that represent branches in a line component of the bitmap, split the line component into two or more subcomponents, generate lane line points for each of the subcomponents using a regression model, and then create a lane line based on the lane line points. Method 400 then proceeds to 412.

[0046] At 412, the control module 102 may generate a map of a roadway (e.g., the roadway 112 of Fig. 1) generate and output. In such examples, the lane map includes the created lane line, which may represent one of the lane lines 114, 116, 118 for the roadway 112. In various embodiments, the control module 102 may transmit the map to a control module of a vehicle, such as the control module 202 of the vehicle 200 of Fig. 2. The control module 202 may then command the display module 240 to display the lane map and / or autonomously control the movement of the vehicle 200 based on the lane map.

[0047] As in Fig. 5, the method 500 begins at 502, the control module 102 receives or otherwise constructs a multi-layer probability density bitmap based on received sensor data from the vehicles 106, 108, 110. The method 500 then proceeds to 504.

[0048] At 504, the control module 102 may detect edges of lane lines in the bitmap or other locations of the lane lines. For example, the control module 102 may initially filter noise from the bitmap (e.g., a kernel density estimate (kde) image) using a global brightness threshold. Then, the control module 102 may detect edges of the lane lines or another location of the lane lines, such as a central portion of the lane lines. In such examples, a gradient-based or Gaussian kernel may be implemented to sample the bitmap to detect edges of the lane lines. In other examples, a custom kernel may be implemented to sample the bitmap to detect a central portion of the lane lines. In such examples, a 5x5 kernel may move along the bitmap sample pixels to detect the central portion. The kernel includes values ​​(e.g.,Brightness values) to indicate a central portion of a lane line. For example, a large value may indicate a brighter pixel representing a central portion of the lane line, whereas a small value (e.g., zero) may indicate a darker pixel (or a black pixel) representing an outer portion of the lane line (or an area that is not part of the lane line). Method 500 then proceeds to 506.

[0049] At 506, the control module 102 may identify and categorize lane lines as different line components in the bitmap. For example, the control module 102 may first identify any connected components in the bitmap and then categorize each set of connected components and each unconnected component as different line components in the bitmap. As an example, Fig. 7 illustrates a process 700 with steps 702, 704, 706. In step 702, multiple observed lane lines 708, 710, 712, 714, 716, 718, 720 of a bitmap are shown. In this example, the control module 102 identifies only the observed lane lines 718, 720 as connected components. Then, the control module 102 categorizes the connected lane lines 718, 720 (e.g., connected components) as an individual line component 722 and categorizes the remaining other unconnected lane lines 708, 710, 712, 714, 716 as individual line components. The method 500 then proceeds to 508.

[0050] At 508, the control module 102 extracts each line component based on its categorization. For example, the control module 102 may implement different masks to extract the different line components. In such examples, a mask layer may be used with the bitmap to isolate one of the line components. This may occur for each categorized line component in the bitmap. For example, and with continued reference to Fig. 7, a mask is applied in step 704 to isolate the categorized line component 722 (with the observed lane lines 718, 720). The method 500 then proceeds to 510.

[0051] At 510, the control module 102 skeletonizes each extracted line component to generate a skeletonized image. In such examples, each line component is represented by a single-pixel-wide representation that follows the skeletonization of that line component. For example, and with continued reference to Fig. 7, the line component 722 from step 704 is skeletonized in step 706 to produce a single-pixel wide representation, as shown by a skeletonized line component 722'. The method 500 then proceeds to 512.

[0052] At 512, the control module 102 detects whether a branch exists in each extracted line component. In various embodiments, the control module 102 detects the presence of a branch (e.g., an intersection of two or more observed lane lines) in the extracted line component based on the skeletonized image of that line component. In such examples, the branch may represent a lane merging into another lane. If a branch is detected, the method 500 then proceeds to 514. Otherwise, if no branch is detected, the method 500 then proceeds to 516.

[0053] In various embodiments, the control module 102 may detect a branch in any suitable manner. For example, the control module 102 may use the skeletonized image of an extracted line component, such as the skeletonized line component 722' of Fig. 7. In such examples, a kernel can be implemented to sample the skeletonized image. With this configuration, the kernel can be a 3x3 kernel with all ones (e.g., every value in the kernel is one). Fig. 8 illustrates an example kernel 800 with a 3x3 all-one configuration. As the kernel moves around the skeletonized image, values ​​are determined. For example, a black pixel in the skeletonized image may be zero, and a non-black pixel in the skeletonized image may be one. In such examples, as the all-one kernel moves over a block of pixels in the skeletonized image, the kernel value (1) is multiplied by the pixel value (0 or 1). The control module 102 then determines a total value associated with the kernel at that point in the skeletonized image by calculating the sum of the multiplied values. The control module 102 then compares the total value to a threshold to detect the branch in the extracted line component. For example, if the total value is greater than or equal to the threshold (e.g., 3, etc.), the kernel detects a branch.), the control module 102 detects the presence of a branch in the extracted line component. In such examples, the location of the detected branch may be stored, if desired. Otherwise, if the total value is less than the threshold, the control module 102 does not detect a branch.

[0054] At 514, the control module 102 splits the line component with the detected branch into two or more subcomponents. As further explained below, the splitting of the line component may be performed by removing a branch point representing the detected branch in the line component. Once the line component is split, the method 500 then proceeds to 516.

[0055] In various embodiments, the control module 102 may use a stretching process before proceeding to 516. For example, the control module 102 may use a stretch on the skeletonized lines of extracted line components (e.g., the line component without detected branches and the subcomponents resulting from detected branches). Then, the control module 102 may use a resulting stretched image to extract the original lane line from the bitmap. As an example, with respect to each grouped line component (with subcomponents), the control module 102 may use a mask to select a subcomponent (e.g., subcomponent 926 of Fig. 9), implement a stretch on this extracted subcomponent, and then use the resulting stretched image and the original received bitmap to extract its corresponding lane line.

[0056] At 516, the control module 102 implements a regression model to generate line points for each of the subcomponents of any line component with a detected branch and for any line component without a detected branch. In various embodiments, the control module 102 may use a B-spline regression model and / or another suitable regression model to interpolate a smooth lane line curve from candidate pixels. As an example, the control module 102 may find peaks in a signal to obtain the highest brightness points (e.g., pixels) at each subcomponent of the bitmap image. Then, the control module 102 may determine the length of that subcomponent and generate interpolation points with the regression model based on the length of the subcomponent. The control module 102 may perform the same functions (e.g.,(Identify the highest brightness, determine the length, and generate interpolation points) for each line component without a detected branch. Method 500 then proceeds to 518.

[0057] At 518, the control module 102 generates new lane lines based on the generated line points for the subcomponents and line components. Then, the method 500 proceeds to 520, where the control module 102 generates a map of a roadway (e.g., the roadway 112 of Fig. 1) with the new lane lines. In such examples, the control module 102 may transmit the map to a control module of a vehicle for display and / or control purposes, as explained above.

[0058] The procedure 600 of Fig. Figure 6 illustrates an example of the division of a line component with a detected branch into two or more subcomponents as described here. As in Fig. 6, the method 600 begins at 602, where the control module 102 detects the presence of a branch in the extracted line component (e.g., where two or more observed lane lines intersect), as explained above. Fig. For example, Figure 9 illustrates a process 900 with steps 902, 904, 906, 908, 910, 912 for splitting a line component. In step 902, two observed lane lines 916, 918 are shown as connected components and categorized as an individual line component 914, as explained above. In this example, the lane lines 916, 918 are connected at a branch point 920 (e.g., the detected branch of the line component 914). The method 600 of Fig. 6 then continues to 604.

[0059] At 604, the control module 102 removes a branch point representing the detected branch in the line component. In doing so, multiple vectors are created or formed with the line component. For example, and with continued reference to Fig. 9, the branch point 920 (from step 902) is removed in step 904, creating vectors 916' from the observed lane line 916 and vectors 918', 918" from the observed lane 918. The method 600 of Fig. 6 then continues to 606, 608.

[0060] At 606, the control module 102 identifies and categorizes each vector associated with the remote branch point to separate the vectors into different components. In such examples, each vector may be referred to as a different component. At 608, the control module 102 selects an area around the remote branch point. In various embodiments, the area may be a small rectangle (e.g., a 12-pixel by 12-pixel area, etc.). For example, and with continued reference to Fig. 9, the vectors 916', 918', 918" are categorized as components 1, 2, 3, respectively, in step 906. Then, in step 908, an area around the removed branch point is enclosed by a rectangle 922. The method 600 of Fig. 6 then continues to 610, 612.

[0061] At 610, the control module 102 determines endpoints for each component in the selected area. For example, one endpoint for each component (e.g., each categorized vector) may be located at an intersection of the component and a rectangle representing the selected area, and the other endpoint for each component may be located at the end of the component near the remote branch point. Additionally, in some examples, the control module 102 may determine the vector direction based on the determined endpoints. As shown in Fig. 9, each component 1, 2, 3 (e.g., each categorized vector) is shown as having endpoints in step 908.

[0062] At 612, the control module 102 determines an angle between each pair of components (e.g., each pair of categorized vectors). In step 908 of Fig. 9, the control module 102 may, for example, determine an angle between components 1, 2, an angle between components 1, 3, and an angle between components 2, 3. The method 600 of Fig. 6 then continues to 614.

[0063] At 614, the control module 102 compares the determined angle between each pair of components and a defined threshold. In various embodiments, the defined threshold may be any suitable value, such as 5 degrees, 7 degrees, 10 degrees, 12 degrees, 15 degrees, etc. If the determined angle is less than the defined threshold, the method 600 proceeds Fig. 6 to 616. If the determined angle is greater than or equal to the defined threshold, the method 600 proceeds from Fig. 6 otherwise continue to 618.

[0064] At 616, if the determined angle between a pair of components is less than the defined threshold, the control module 102 merges this pair of components (e.g., this pair of categorized vectors) together. In doing so, the pair of components is treated as the same lane line. For example, and with continued reference to Fig. 9, the angle between components 1, 2 at 908 may be approximately 30 degrees and greater than a defined threshold (e.g., 10 degrees, etc.), the angle between components 1, 3 at 908 may be approximately 30 degrees and greater than the defined threshold (e.g., 10 degrees, etc.), and the angle between components 2, 3 at 908 may be approximately 5 degrees and less than the defined threshold (e.g., 10 degrees, etc.). In such examples, components 2, 3 are clustered together and treated as the same lane line. The method 600 of Fig. 6 then continues to 618.

[0065] At 618, the control module 102 splits the extracted line component with the detected branch into subcomponents. For example, the control module 102 may categorize each grouping of components, thereby creating the subcomponents. In such examples, each set of components that are bundled together may be categorized as a subcomponent, and each component that does not meet the bundling requirements may be categorized as another subcomponent. Fig. 9, for example, component 1 is categorized as subcomponent 926 and components 2, 3 are bundled together and categorized collectively as subcomponent 928. As an example, control module 102 may determine the corresponding components in a generated skeletonized image, as described above with respect to 510 of Fig. 5. Once the corresponding components are found, the control module 102 can divide the components into different lane lines. In step 912 in Fig. 9, for example, the control module 102 locates the components in a generated skeleton image that correspond to the subcomponents 926, 928 and treats the component corresponding to the subcomponent 926 as one lane line and the components corresponding to the subcomponent 928 as another lane line.

Claims

[1] A system for creating a map of a roadway, the system comprising: a control module in communication with one or more sensors of multiple vehicles, the control module being configured to: receive a multi-layer probability density bitmap having a plurality of pixels representing a plurality of lane lines of the roadway detected by the one or more sensors of the plurality of vehicles; and extract multiple line components of the multiple lane lines; characterized by , that the control module is further configured to: detect whether a branch exists in a line component of the plurality of line components; remove a branch point representing the detected branch in the line component to create a plurality of vectors in response to the detection of the branch in the line component; in response to the detection of the branch in the line component, split the line component into two or more subcomponents; generate multiple line points for each of the subcomponents using a regression model; to create at least one line based on the plurality of line points; and to create a map of the roadway with at least one line. [2] The system of claim 1, wherein the control module is configured: to identify connected components of the multiple lane lines in the bitmap; and to categorize each connected component as a line component of the multiple line components. [3] The system of claim 2, wherein the control module is configured to implement a plurality of masks to extract the plurality of line components. [4] The system of claim 1, wherein the control module is configured: skeletonize each line component of the plurality of line components to generate a skeletonized image; and to detect the branching based on the skeletonized image. [5] The system of claim 4, wherein the control module is configured: to sample the skeletonized image with a kernel; to determine a value associated with the kernel; and compare the value with a threshold to detect the branch in the line component. [6] The system of claim 1, wherein the control module is configured: determine an angle between each pair of the plurality of vectors; and to combine one or more pairs of the plurality of vectors based on the angle and a defined threshold to split the component into the two or more subcomponents. [7] The system of claim 1, wherein the regression model comprises a B-spline regression model. [8] The system of claim 1, wherein the control module is configured: to determine a length of each of the subcomponents; and generate the multiple line points for each of the subcomponents using the regression model based on the determined length for the subcomponent. [9] The system of claim 1, wherein the control module is configured, in response to the non-detection of the branch in the line component, to generate a plurality of line points for the line component using the regression model and to create a line based on the plurality of line points for the line component.

Images