Multi-lane line fusion method, device, storage medium and unmanned vehicle
By constructing a spatial index R-tree and directed acyclic graph, lane lines are integrated, inter-frame continuity problems in the existing technology are solved, lane lines matching and merging in multi-vector line scenarios are realized, and BEV graphing is improved Vectorization processing capability.
Patent Information
- Application Number
- CN202411415832.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-10-11
- Publication Date
- 2024-12-03
- Estimated Expiration
- 2044-10-11
AI Technical Summary
The existing automated graphing scheme is based on a time series composed of a single frame, and it is impossible to consider the continuity of high-precision map elements between frames, and it is difficult to output reliable continuous lane lines, especially in multi-vector line scenarios, which is not conducive to the post-vectorization of BEV graphing. deal with.
By obtaining multi-lane lines and unifying specifications, a spatial index R-tree is constructed, a neighboring lane line is searched, a directional acyclic map is constructed, a connected sub-graph is solved, lane line fusion is performed, the original lane line is deleted, and the fusion results are collected.
The unified matching and line merging of multi-frame, multiple times, and multi-source lane lines are realized, providing effective fusion effect in multi-vector line scenarios, and promoting the post-vectorization processing of BEV graphs.
Smart Images

Figure CN118918559B_ABST
Abstract
Description
Technical Field
[0001] The present application belongs to the field of autonomous driving technology, and specifically relates to a multi-lane line fusion method, device, storage medium and unmanned vehicle. Background Art
[0002] With the continuous iteration of the Bird's Eye View (BEV) technology in autonomous driving, the automatic generation capability of high-precision maps has begun to change qualitatively, and more and more automatic mapping solutions have emerged, such as Horizon's VMA (Vectorized Map Annotation) based on lidar, Fudan University's MV-Map (Multi-View Consistency Map) based on pure vision, and Huawei's NeMO (Neural Map Growing System) based on pure vision. These solutions have all made fruitful explorations in the automatic extraction of lane lines. In particular, Horizon's VMA technology built on MapTR can retain the directional information of lane lines through encoding learning when identifying lane lines, and has stronger practicality in the current automatic mapping technical solutions. Based on these solutions, the automatic processing of lane lines has also had greater room for development.
[0003] However, the current BEV technologies all rely on time series composed of a single frame. In the process of generating lane lines, the above-mentioned machine learning-based methods rarely consider the fusion of multi-frame lane lines. As a result, the current automatic mapping solutions cannot consider the continuity of high-precision map elements between frames, which is not conducive to the output of reliable continuous lane lines, and is difficult to apply to scenarios with multiple vector lines, which is not conducive to the post-vectorization processing of BEV mapping. Summary of the invention
[0004] The purpose of this application is to provide a multi-lane line fusion method, device, storage medium and unmanned vehicle to solve the technical problems that the automated mapping solution in the prior art is based on a time series composed of a single frame, cannot consider the continuity of high-precision map elements between frames, is not conducive to outputting reliable continuous lane lines, is difficult to apply to scenarios with multiple vector lines, and is not conducive to the post-vectorization processing of BEV mapping.
[0005] In order to achieve the above-mentioned object, the first aspect of the present application provides a method for fusing multiple lanes, including:
[0006] Get multiple lane lines and unify their specifications to obtain a lane line vector set;
[0007] Constructing a spatial index R-tree for the lane line vector set;
[0008] Traversing the lane line vector set, searching for neighboring lane lines of each lane line through the spatial index R-tree, and obtaining a neighboring lane line subset;
[0009] Based on the neighborhood lane line subset, a directed acyclic graph is constructed, and a connected subgraph of the directed acyclic graph is solved to obtain a connected subgraph set;
[0010] Traversing the connected sub-atlas, fusing all lane lines in the connected sub-atlas to obtain fused lane lines;
[0011] All the fused lane lines in the lane line vector set are deleted, the fused lane lines are collected into the lane line vector set, and the lane line fusion result is output.
[0012] In one or more implementations, the step of constructing a spatial index R-tree for the lane line vector set is specifically:
[0013] A spatial index R-tree is constructed using the bounding box of each lane line in the lane line vector set.
[0014] In one or more embodiments, the bounding box is an AABB bounding box.
[0015] In one or more implementations, the step of searching the neighboring lane lines of each lane line through the spatial index R-tree to obtain a subset of neighboring lane lines is specifically as follows:
[0016] All neighboring lane lines that intersect with the bounding box of the target lane line are searched through the spatial index R-tree, and the target lane line and its neighboring lane lines are collected into a set to obtain a neighboring lane line subset.
[0017] In one or more implementations, the step of constructing a directed acyclic graph based on the neighborhood lane line subset is specifically as follows:
[0018] Taking each lane line in the neighborhood lane line subset as a vertex of a directed acyclic graph;
[0019] Calculating the angle between any two lane lines in the neighborhood lane lines, and determining whether the angle is less than a first threshold;
[0020] If so, adding an edge connecting the two lane lines in the directed acyclic graph;
[0021] The distance between the two lane lines is calculated as the weight of the edge.
[0022] In one or more implementations, the step of calculating the distance between the two lane lines is specifically:
[0023] The average of the shortest distances from all points on the short line of the two lane lines to the long line is calculated to obtain the distance between the two lane lines.
[0024] In one or more implementations, the step of solving the connected subgraph of the directed acyclic graph is specifically:
[0025] The edges in the directed acyclic graph are traversed, and all connected subgraphs formed by the edges whose weights are less than a second threshold are calculated to obtain a connected subgraph set.
[0026] In one or more implementations, the step of traversing the connected sub-graph set and fusing all lane lines in the connected sub-graph set to obtain the fused lane line is specifically:
[0027] Merging each connected subgraph in the connected subgraph set in turn to obtain corresponding merged geometry, collecting the merged geometry into a geometry set, and synchronously maintaining a bijective function of the lane line and the merged geometry;
[0028] After all the connected subgraphs are merged, the merged geometry in the geometry set is used as the fused lane line.
[0029] In one or more implementations, the step of merging each connected subgraph in the connected subgraph set is specifically:
[0030] Based on the bijective function, determine whether there is a corresponding merge geometry for the lane line in the target connected subgraph; if so,
[0031] Replacing the corresponding lane line with the merged geometry to obtain the updated target connected subgraph;
[0032] Based on the bijective function, determine whether the updated target connected subgraph includes lane lines without corresponding merged geometry; if so,
[0033] The updated target connected subgraph is merged to obtain a target merged geometry, and the geometry set and the bijective function are updated based on the target merged geometry.
[0034] In one or more implementations, the steps of the merging process are specifically as follows:
[0035] Calculate the nearest neighboring points of two target lane lines in the connected subgraph, and merge the neighboring points of the two target lane lines into one;
[0036] Determine whether the two target lane lines both include direction information; if not,
[0037] Selecting a longer segment of the two target lane lines located on the same side of the adjacent point, and merging the longer segments located on both sides of the adjacent point to obtain a merged lane line;
[0038] The merged lane line replaces the two target lane lines in the connected subgraph, and the above steps are repeated until all lane lines in the connected subgraph are merged to obtain a merged geometry.
[0039] In one or more embodiments, it further includes:
[0040] If both of the two target lane lines include direction information, determine whether the directions of the line segments of the two target lane lines located on the same side of the adjacent point are the same; if they are the same,
[0041] The longer segment of the two target lane lines located on the same side of the adjacent point is selected, and the longer segments located on both sides of the adjacent point are merged to obtain a merged lane line.
[0042] In order to achieve the above-mentioned object, the second aspect of the present application provides a multi-lane line fusion device, characterized in that it includes:
[0043] An acquisition module is used to acquire multiple lane lines and unify their specifications to obtain a lane line vector set;
[0044] A spatial index module, used to construct a spatial index R-tree for the lane line vector set;
[0045] A neighborhood search module, configured to traverse the lane line vector set, search for the neighboring lane lines of each lane line through the spatial index R-tree, and obtain a neighboring lane line subset;
[0046] A directed acyclic graph construction module, used to construct a directed acyclic graph based on the neighborhood lane line subset, and solve the connected subgraph of the directed acyclic graph to obtain a connected subgraph set;
[0047] A fusion module, used for traversing the connected sub-atlas, fusing all lane lines in the connected sub-atlas, and obtaining a fused lane line;
[0048] The output module is used to delete all the fused lane lines in the lane line vector set, collect the fused lane lines into the lane line vector set, and output the lane line fusion result.
[0049] In order to achieve the above-mentioned purpose, the third aspect of the present application provides an unmanned vehicle, comprising:
[0050] at least one processor; and
[0051] A memory storing instructions, which, when executed by the at least one processor, enables the at least one processor to execute the multi-lane line fusion method as described in any of the above embodiments.
[0052] In order to achieve the above-mentioned purpose, the fourth aspect of the present application provides a machine-readable storage medium, which stores executable instructions, and when the instructions are executed, the machine executes the multi-lane line fusion method as described in any of the above-mentioned embodiments.
[0053] Different from the prior art, the beneficial effects of this application are:
[0054] The present application can perform unified lane line matching on lane lines of multiple frames, multiple times, and multiple sources, and perform a line-by-line merging operation on the matched lane lines based on the nearest neighbor points; it can provide an effective and stable fusion effect in scenes with multiple vector lines, and has a significant positive effect on the later vectorization processing of BEV mapping. BRIEF DESCRIPTION OF THE DRAWINGS
[0055] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.
[0056] Figure 1 It is a flowchart of an implementation method of a multi-lane line fusion method of the present application;
[0057] Figure 2 yes Figure 1 A schematic diagram of a flow chart of an implementation method corresponding to S400;
[0058] Figure 3 This application is a directed acyclic Figure 1 A schematic diagram of the structure of an implementation method;
[0059] Figure 4 yes Figure 1 A schematic flow chart of an implementation method corresponding to S500;
[0060] Figure 5 yes Figure 4 A schematic flow chart of an implementation method corresponding to S501;
[0061] Figure 6 It is a structural diagram of an implementation method of a bijective function and a geometric set of the present application;
[0062] Figure 7 It is a flowchart of an implementation method of the combined processing of the present application;
[0063] Figure 8 It is a schematic diagram of an implementation method of the combined processing of the present application;
[0064] Fig. 9 It is a structural schematic diagram of an implementation method of a multi-lane line fusion device of the present application;
[0065] Fig.10 It is a structural schematic diagram of an implementation method of an unmanned vehicle of the present application. DETAILED DESCRIPTION
[0066] In order to enable those skilled in the art to better understand the technical solutions in the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in the field without creative work should fall within the scope of protection of the present application.
[0067] The present application provides a novel multi-lane line fusion method, which can be applied to the fusion of lane line perception results processed by multiple bird's-eye view BEVs, thereby facilitating the output of reliable continuous lane lines. The method can be applied to scenarios with multiple vector lines, and facilitates the post-vectorization processing of BEV mapping.
[0068] Specifically, see Figure 1 , Figure 1 It is a flowchart of an implementation method of a multi-lane line fusion method of the present application.
[0069] like Figure 1 As shown, the fusion method includes:
[0070] S100, obtaining multiple lane lines and unifying their specifications to obtain a lane line vector set.
[0071] Among them, multiple lane lines can be obtained by processing and perceiving bird's-eye views of different frames captured by the same image acquisition module through BEV; they can also be obtained by processing and perceiving bird's-eye views captured by the same image acquisition module multiple times through BEV; they can also be obtained by processing and perceiving bird's-eye views captured by different image acquisition modules through BEV.
[0072] Based on the position features and image features of the image acquisition module, etc., the specifications of the multiple lane lines mentioned above can be unified so that the multiple lane lines are located in the same coordinate system and collected into a set to obtain a lane line vector set.
[0073] S200: Construct a spatial index R-tree for the lane line vector set.
[0074] The spatial index R-tree is a balanced tree data structure used for multi-dimensional spatial indexing. It helps to speed up the indexing of lane lines by constructing a spatial index R-tree for a lane line vector set.
[0075] Specifically, in one embodiment, the method for constructing the spatial index R-tree may be: constructing the spatial index R-tree with the bounding box of each lane line.
[0076] The bounding box of the lane line is used to limit the range of the lane line in space, so as to help determine the proximity relationship of the lane line later. Specifically, the bounding box of the lane line can be an AABB bounding box, that is, a hexahedral bounding box, or other types of bounding boxes, such as a bounding sphere, an OBB bounding box, an FDH fixed direction convex hull, etc., all of which can achieve the effect of this embodiment.
[0077] S300, traverse the lane line vector set, search the neighboring lane lines of each lane line through the spatial index R tree, and obtain the neighboring lane line subset.
[0078] Specifically, in one embodiment, the spatial index R-tree is constructed based on the bounding box of the lane line. The spatial index R-tree can be used to search for all lane lines that intersect with the bounding box of the target lane line as neighboring lane lines of the target lane line, and the target lane line and its neighboring lane lines are collected into a set to obtain a subset of neighboring lane lines.
[0079] S400: construct a directed acyclic graph based on the neighborhood lane line subset, and solve the connected subgraph of the directed acyclic graph to obtain a connected subgraph set.
[0080] It can be understood that the lane lines in the domain lane line subset are adjacent and there is a possibility of matching and fusion. Further processing based on the neighborhood lane line subset can screen out the lane lines that can be matched.
[0081] In order to solve the problem of matching neighborhood lane lines, the subset of neighborhood lane lines can be converted into a directed acyclic graph. The vertices in the directed acyclic graph are used to represent the lane lines, and the edges between two vertices in the directed acyclic graph are used to represent the distance between the two lane lines. The problem of matching neighborhood lane lines is then converted into the problem of solving a connected subgraph in a directed acyclic graph.
[0082] Specifically, see Figure 2 , Figure 2 yes Figure 1 A flow chart of an implementation method corresponding to S400 in FIG.
[0083] like Figure 2 As shown, the method of constructing a directed acyclic graph and solving the connected subgraph includes:
[0084] S401. Take each lane line in the neighborhood lane line subset as a vertex of a directed acyclic graph.
[0085] S402: Calculate the angle between any two lane lines in the neighborhood lane lines, and determine whether the angle is less than a first threshold.
[0086] The first threshold may be preset based on actual working conditions, which means that when the angle between two lane lines exceeds the first threshold, the two lane lines lose the possibility of matching and fusion.
[0087] It can be understood that the two lane lines that can be matched and fused should be close in distance and have a small angle. If the angle exceeds the first threshold, such as 30°, it can be determined that the two lane lines are not the same lane lines. At this time, the edge between the two lane lines in the directed acyclic graph can be removed to avoid the two lane lines appearing in the same connected subgraph.
[0088] If the angle is less than the first threshold, then:
[0089] S403: Add an edge connecting two lane lines in the directed acyclic graph.
[0090] If the angle between two lane lines is less than the first threshold, it means that the two lane lines have the possibility of matching and fusion. At this time, an edge connecting the two lane lines is added to the directed acyclic graph so that the two lane lines may appear in the same connected subgraph for subsequent matching and fusion.
[0091] S404: Calculate the distance between two lane lines as the weight of the edge.
[0092] It should be understood that the physical meaning of the edge in the directed acyclic graph is the distance between two lane lines. Therefore, the distance between the two lane lines can be used as the weight of the edge, which is helpful for the subsequent screening of the edges.
[0093] Specifically, in order to characterize the distance between two lane lines in space, the Hausdorff short line median distance SMDF is used in this embodiment, that is, the distance between the two lane lines is obtained by calculating the average of the shortest distances from all points of the short line to the long line of the two lane lines. The specific calculation formula is as follows:
[0094]
[0095] In the formula, L1 and L2 represent two lane lines, where L1 is the short line and L2 is the long line, and x and y represent the coordinates of the points in L1 and L2 respectively.
[0096] The distance between two lane lines is defined by calculating the average distance of all points on the short line to the long line, and can be used as the weight of the edge connecting the two lane lines in the directed acyclic graph.
[0097] For example, see Figure 3 , Figure 3 This application is a directed acyclic Figure 1 A schematic structural diagram of an implementation method, in which a is the four lane lines included in the neighborhood lane line subset, and b is a directed acyclic graph constructed based on a.
[0098] As shown in the figure, the neighborhood lane line subset shown in a includes five lane lines numbered 0 to 4, and five vertices corresponding to the five lane lines are constructed in the directed acyclic graph.
[0099] First, calculate the angle between any two lane lines. The angle between lane line 4 and lane lines 0-3 is greater than the threshold. Therefore, there is no edge connection between lane line 4 and other lane lines in the directed acyclic graph. Add the edge between lane lines 0-3.
[0100] Furthermore, the Hausdorff short line median distance SMDF between lane lines 0~3 is calculated. After calculation, the SMDF between lane lines 0 and 1 is 1, so the weight of the edge connecting vertices 0 and 1 in the directed acyclic graph is 1. Based on this method, the distance of each edge is calculated.
[0101] Based on the above S401-404, a directed acyclic graph constructed based on a neighborhood lane line subset is obtained. Further, by solving a connected subgraph, multiple lane lines with the greatest matching fusion possibility can be collected into a connected subgraph.
[0102] Specifically, the steps of solving the connected subgraph include:
[0103] S405 , traverse the edges in the directed acyclic graph, calculate all connected subgraphs formed by the edges whose weights are less than the second threshold, and obtain a connected subgraph set.
[0104] The second threshold may be preset based on actual working conditions, and represents the maximum distance at which two lane lines can be matched and fused.
[0105] When the distance between two lane lines is greater than or equal to the second threshold, that is, when the weight of the edge connecting the two lane lines is greater than or equal to the second threshold, it can be determined that the two lane lines are not the same lane lines and cannot be matched and fused.
[0106] By traversing the edges in the directed acyclic graph, we can first filter out the edges whose weights are less than the second threshold, and further, we can obtain all connected subgraphs that can be combined with these edges, collect them into a set, and obtain a connected subset graph.
[0107] For example, please refer to Figure 3 , when the second threshold is 5, after screening Figure 3 In the directed acyclic graph, only edges 01, 12, and 02 meet the requirements. Further, we can get the connected subgraphs that can be formed by these three edges include .
[0108] In one embodiment, the present application can implement the solution of connected subgraphs based on the depth-first search algorithm DFS. The depth-first search algorithm is a commonly used traversal search algorithm in this field. It obtains all connected subgraphs by exploring all edges that meet the requirements in a directed acyclic graph. The specific algorithm process will not be repeated here.
[0109] S500 , traverse the connected sub-graph set, fuse all lane lines in the connected sub-graph set, and obtain a fused lane line.
[0110] Based on the above S400, a connected subgraph set including a plurality of connected subgraphs is obtained, each connected subgraph includes at least two lane lines that may be matched and fused, and the lane lines in the connected subgraphs can be further fused.
[0111] Specifically, see Figure 4 , Figure 4 yes Figure 1 A flowchart of an implementation method corresponding to S500 in FIG. Figure 4 As shown, the fusion method of lane lines in the connected sub-graph set includes:
[0112] S501, merge each connected subgraph in the connected subgraph set in turn to obtain corresponding merged geometry, collect the merged geometry into a geometry set, and synchronously maintain a bijective function of the lane line and the merged geometry.
[0113] By merging multiple lanes in each connected subgraph, a merged geometry can be obtained, and at the same time, a mapping relationship between lanes in the connected subgraph and the merged geometry can be constructed, that is, a bijective function can be constructed. In the process of merging connected subgraphs in the connected subgraph set one by one, the merged geometry can be updated and the bijective function can be updated synchronously.
[0114] See below for detailed instructions. Figure 5 , Figure 5 yes Figure 4 A flow chart of an implementation method corresponding to S501.
[0115] like Figure 5 As shown, the method for sequentially merging each connected subgraph in the connected subgraph set includes:
[0116] S5011. Based on the bijective function, determine whether there is a corresponding merged geometry for the lane line in the target connected subgraph.
[0117] When performing merging processing on a new connected subgraph, that is, when performing merging processing on a target connected subgraph, it can first be determined based on the bijective function whether the lane lines included therein already have corresponding merging geometry.
[0118] Of course, when merging the first connected subgraph, there is no relevant mapping relationship in the bijective function, and at this time, there is no corresponding merged geometry for its lane lines.
[0119] If the lane lines in the target connected subgraph have corresponding merged geometry, then:
[0120] S5012: Replace the corresponding lane line with the merged geometry to obtain an updated target connected subgraph.
[0121] If a lane line with a corresponding merged geometry already exists in the target connected subgraph, that is, the lane line has been merged with other lane lines in other connected subgraphs, the lane line can be replaced with its merged geometry, so that the merged geometry and other lane lines in the target connected subgraph can be further merged, so that lane lines distributed in different connected subgraphs can be merged uniformly.
[0122] S5013. Based on the bijective function, determine whether the updated target connected subgraph includes lane lines without corresponding merged geometry.
[0123] After the lane lines in the target connected subgraph are replaced by their previous merged geometry, if the target connected subgraph does not include lane lines without corresponding merged geometry, it means that all lane lines in the target connected subgraph have been merged. At this time, the target connected subgraph does not need to be further merged, and there is no need to update the geometry set and bijective function.
[0124] If the target connected subgraph also includes lane lines without corresponding merged geometry, then:
[0125] S5014. Merge the updated target connected subgraphs to obtain target merged geometry, and update the geometry set and bijective function based on the target merged geometry.
[0126] Through further merging processing, the merged geometry of further merged lane lines is obtained, thereby updating the geometry set and the bijective function.
[0127] It can be understood that in S5011, if there is no corresponding merged geometry for the lane lines in the target connected subgraph, the lane lines included therein can be directly merged to obtain the merged geometry.
[0128] S502: After all connected subgraphs are merged, the merged geometry in the geometry set is used as the fused lane line.
[0129] The following is a specific example to further introduce the method for merging connected sub-graphs in this application. Figure 6 , Figure 6This is a schematic diagram of the structure of an implementation method of the bijective function and geometric set of the present application. The connected subgraph set may include [2,0,1], [2,1], [4,5], [6,1]. First, we construct a bijective function from a connected line to a geometric set. .
[0130] The connected subgraph [2,0,1] is merged to obtain the merged geometry g1, which is collected into the geometry set geometry. The mapping relationship obtained at this time is: .
[0131] Furthermore, the connected subgraph [2,1] is merged. Since there is a mapping relationship , you can skip the merging process at this time.
[0132] Furthermore, the connected subgraph [4,5] is merged. Since there is no mapping relationship between lane lines 4 and 5, lane lines 4 and 5 are merged to obtain the merged geometry g2, which is collected into the geometry set geometry. The mapping relationship obtained at this time is: .
[0133] Furthermore, the connected subgraph [6,1] is merged. Since lane line 1 has a corresponding merged geometry g1, lane line 6 and merged geometry g1 can be merged to obtain updated g1, and the updated g1 is used to update the bijective function and geometric set.
[0134] After the processing is completed, the two merged geometries in the geometry collection geometry are used as the fused lane line.
[0135] In the above method, the method for merging multiple lane lines can be any merging method commonly used in the art, such as a fitting algorithm, etc. The following introduces a feasible merging algorithm for this application, please refer to Figure 7 , Figure 7 It is a flowchart of an implementation method of the combined processing of the present application.
[0136] like Figure 7 As shown, the method includes:
[0137] S1000: Calculate the nearest neighboring points of two target lane lines in the connected subgraph, and merge the neighboring points of the two target lane lines into one.
[0138] First, the nearest neighboring points of the two target lane lines to be merged can be calculated; the nearest neighboring points are the two points corresponding to the shortest distance between the two target lane lines.
[0139] S2000: Determine whether both target lane lines include direction information.
[0140] If not, then:
[0141] S3000: Select the longer segment of the two target lane lines located on the same side of the adjacent point, and merge the longer segments located on both sides of the adjacent point to obtain a merged lane line.
[0142] S4000: Replace the two target lane lines in the connected subgraph with the merged lane line, and repeat the above steps until all lane lines in the connected subgraph are merged to obtain a merged geometry.
[0143] In one embodiment, the two target lane lines may be lines that do not include direction information or only one lane line includes direction information. The line segments of the two target lane lines located on the same side of the adjacent points may be directly merged, and the longest end thereof may be taken as the merged line segment.
[0144] Of course, it is understandable that if both target lane lines include direction information, before S3000, the following steps are also included:
[0145] S3000a, determining whether the directions of the line segments of the two target lane lines located on the same side of the adjacent points are the same.
[0146] In another embodiment, the two target lane lines may be lines that both include direction information. If the line segments of the two target lane lines located on the same side of the adjacent points are in opposite directions, the two cannot be merged. If the directions are the same, they can be merged and S3000 is executed to take the longest segment for merging.
[0147] The above merging method is further described in detail below with a specific embodiment. Figure 8 , Figure 8 It is a schematic diagram of an implementation method of the combined processing of the present application.
[0148] like Figure 8 As shown, for merging lane lines L1 and L2, first calculate the nearest neighbor points of the two lane lines, namely P3 (L1) and P4 (L2), and then determine whether the directions of the line segments of lane lines L1 and lane lines L2 on the same side of the nearest neighbor points are the same, as shown in Figure 8 The directions shown are the same, so the longer segment of the same side of the two lane lines is selected, and the merged segments on both sides of the nearest neighbor point are merged to obtain the final merged geometry P1 (L2) - P2 (L2) - P3 (L2) - P4 (L2) - P4 (L1) - P5 (L1).
[0149] S600, deleting all the fused lane lines in the lane line vector set, collecting the fused lane lines into the lane line vector set, and outputting the lane line fusion result.
[0150] Based on the above steps, lane lines from multiple frames, multiple times, and multiple sources can be matched uniformly, and the matched lane lines can be merged based on the nearest neighbor point. This solution can provide an effective and stable fusion effect in the scene of multiple vector lines, and has a significant positive effect on the later vectorization processing of BEV mapping.
[0151] This application also provides a multi-lane line fusion device, see Fig. 9 , Fig. 9 It is a structural schematic diagram of an implementation method of a multi-lane line fusion device of the present application.
[0152] like Fig. 9 As shown, the device includes an acquisition module 21, a spatial index module 22, a neighborhood search module 23, a directed acyclic graph construction module 24, a fusion module 25 and an output module 26.
[0153] The acquisition module 21 is used to acquire multiple lane lines and unify their specifications to obtain a lane line vector set;
[0154] The spatial index module 22 is used to construct a spatial index R tree for the lane line vector set;
[0155] The neighborhood search module 23 is used to traverse the lane line vector set, search the neighborhood lane lines of each lane line through the spatial index R tree, and obtain the neighborhood lane line subset;
[0156] The directed acyclic graph construction module 24 is used to construct a directed acyclic graph based on the neighborhood lane line subset, and solve the connected subgraph of the directed acyclic graph to obtain a connected subgraph set;
[0157] The fusion module 25 is used to traverse the connected sub-atlas, fuse all lane lines in the connected sub-atlas, and obtain a fused lane line;
[0158] The output module 26 is used to delete all the fused lane lines in the lane line vector set, collect the fused lane lines into the lane line vector set, and output the lane line fusion result.
[0159] In one embodiment, the directed acyclic graph construction module 24 is also used to use each lane line in the neighborhood lane line subset as a vertex of the directed acyclic graph; calculate the angle between any two lane lines in the neighborhood lane lines, and determine whether the angle is less than a first threshold; if so, add an edge connecting the two lane lines in the directed acyclic graph; calculate the distance between the two lane lines as the weight of the edge.
[0160] In one embodiment, the directed acyclic graph construction module 24 is further used to calculate the average of the shortest distances from all points on the short line of the two lane lines to the long line to obtain the distance between the two lane lines.
[0161] In one embodiment, the fusion module 25 is also used to merge each connected subgraph in the connected subgraph set in turn to obtain the corresponding merged geometry, collect the merged geometry into a geometry set, and synchronously maintain the bijective function of the lane line and the merged geometry; after the merging of all the connected subgraphs is completed, the merged geometry in the geometry set is used as the fused lane line.
[0162] In one embodiment, the fusion module 25 is also used to determine whether there is a corresponding merged geometry for the lane line in the target connected subgraph based on the bijective function; when there is a lane line with a corresponding merged geometry, replace the corresponding lane line with the merged geometry to obtain an updated target connected subgraph; based on the bijective function, determine whether the updated target connected subgraph includes a lane line without a corresponding merged geometry; if so, merge the updated target connected subgraph to obtain the target merged geometry, and update the geometry set and the bijective function based on the target merged geometry.
[0163] In one embodiment, the fusion module 25 is also used to calculate the nearest neighboring points of two target lane lines in the connected subgraph, and fuse the neighboring points of the two target lane lines into one; determine whether the two target lane lines both include direction information; when the two target lane lines do not both include direction information, select the longer segment of the line segments of the two target lane lines on the same side of the neighboring points, merge the longer segments on both sides of the neighboring points to obtain a merged lane line; replace the two target lane lines in the connected subgraph with the merged lane line, and repeat the above steps until all lane lines of the connected subgraph are merged to obtain a merged geometry.
[0164] In one embodiment, the fusion module 25 is also used to determine whether the directions of the line segments of the two target lane lines located on the same side of the adjacent point are the same when both target lane lines include direction information; if they are the same, select the longer segment of the line segments of the two target lane lines located on the same side of the adjacent point, merge the longer segments located on both sides of the adjacent point, and obtain a merged lane line.
[0165] As above Figures 1 to 8 , a method for fusing multiple lanes according to an embodiment of this specification is described. The details mentioned in the above description of the method embodiment are also applicable to the multi-lane fusion device of the embodiment of this specification. The above multi-lane fusion device can be implemented by hardware, or by software, or by a combination of hardware and software.
[0166] Fig.10 FIG. 1 shows a hardware structure diagram of an unmanned vehicle according to an embodiment of the present specification. Fig.10As shown, the unmanned vehicle 30 may include at least one processor 31, a storage 32 (e.g., a non-volatile memory), a memory 33, and a communication interface 34, and the at least one processor 31, the storage 32, the memory 33, and the communication interface 34 are connected together via an internal bus 35. At least one processor 31 executes at least one computer-readable instruction stored or encoded in the memory 32.
[0167] It should be understood that the computer executable instructions stored in the memory 32, when executed, cause at least one processor 31 to perform the above combined operations in various embodiments of the present specification. Figure 1-Figure 8 Describes the various operations and functions.
[0168] In the embodiments of the present specification, the unmanned vehicle 30 can be configured with a functional terminal to carry the above-mentioned hardware structure, and the terminal may include but is not limited to: a personal computer, a server computer, a workstation, a desktop computer, a laptop computer, a notebook computer, a mobile electronic device, a smart phone, a tablet computer, a cellular phone, a personal digital assistant (PDA), a handheld device, a messaging device, a wearable electronic device, a consumer electronic device, and the like.
[0169] According to one embodiment, a program product such as a machine-readable medium is provided. The machine-readable medium may have instructions (i.e., the above-mentioned elements implemented in the form of software), which, when executed by a machine, causes the machine to perform the above-mentioned combination of various embodiments of this specification. Figure 1-Figure 8 Specifically, a system or device equipped with a readable storage medium may be provided, on which a software program code implementing the functions of any of the above-mentioned embodiments is stored, and a computer or processor of the system or device reads and executes the instructions stored in the readable storage medium.
[0170] In this case, the program code itself read from the machine-readable medium can realize the function of any one of the above embodiments, and thus the machine-readable code and the machine-readable storage medium storing the machine-readable code constitute part of this specification.
[0171] Examples of readable storage media include floppy disks, hard disks, magneto-optical disks, optical disks (such as CD-ROM, CD-R, CD-RW, DVD-ROM, DVD-RAM, DVD-RW, DVD-RW), magnetic tapes, non-volatile memory cards, and ROMs. Alternatively, the program code may be downloaded from a server computer or a cloud via a communication network.
[0172] Those skilled in the art should understand that the various embodiments disclosed above can be modified and altered in various ways without departing from the essence of the invention. Therefore, the protection scope of this specification should be defined by the appended claims.
[0173] It should be noted that not all steps and units in the above-mentioned processes and system structure diagrams are necessary, and some steps or units can be ignored according to actual needs. The execution order of each step is not fixed and can be determined as needed. The device structure described in the above-mentioned embodiments can be a physical structure or a logical structure, that is, some units may be implemented by the same physical client, or some units may be implemented by multiple physical clients, or some components in multiple independent devices may be implemented together.
[0174] In the above embodiments, the hardware unit or module can be realized by mechanical or electrical means. For example, a hardware unit, module or processor can include permanent dedicated circuit or logic (such as special processor, FPGA or ASIC) to complete the corresponding operation. The hardware unit or processor can also include programmable logic or circuit (such as general-purpose processor or other programmable processor), which can be temporarily set by software to complete the corresponding operation. Specific implementation (mechanical method or dedicated permanent circuit or temporary circuit) can be determined based on cost and time consideration.
[0175] The specific embodiments described above in conjunction with the accompanying drawings describe exemplary embodiments, but do not represent all embodiments that can be implemented or fall within the scope of protection of the claims. The term "exemplary" used throughout this specification means "used as an example, instance or illustration" and does not mean "preferred" or "having advantages" over other embodiments. For the purpose of providing an understanding of the described technology, the specific embodiments include specific details. However, these technologies can be implemented without these specific details. In some instances, in order to avoid making the concepts of the described embodiments difficult to understand, well-known structures and devices are shown in block diagram form.
[0176] The above description of the present disclosure is provided to enable any person of ordinary skill in the art to implement or use the present disclosure. Various modifications to the present disclosure will be apparent to those of ordinary skill in the art, and the general principles corresponding to the present disclosure may be applied to other variations without departing from the scope of protection of the present disclosure. Therefore, the present disclosure is not limited to the examples and designs described herein, but is consistent with the widest range of principles and novel features disclosed herein.
Claims
1. A multi-lane line fusion method, It is characterized in that include: Get multiple lane lines and unify their specifications to obtain a lane line vector set; Constructing a spatial index R-tree for the lane line vector set; Traversing the lane line vector set, searching for neighboring lane lines of each lane line through the spatial index R-tree, and obtaining a neighboring lane line subset; Based on the neighborhood lane line subset, a directed acyclic graph is constructed, and a connected subgraph of the directed acyclic graph is solved to obtain a connected subgraph set; Traversing the connected sub-atlas, fusing all lane lines in the connected sub-atlas to obtain fused lane lines; All the fused lane lines in the lane line vector set are deleted, the fused lane lines are collected into the lane line vector set, and the lane line fusion result is output.
2. The fusion method according to claim 1, It is characterized in that The step of constructing a spatial index R-tree for the lane line vector set is specifically as follows: A spatial index R-tree is constructed using the bounding box of each lane line in the lane line vector set.
3. The fusion method according to claim 2, It is characterized in that The step of searching the neighboring lane lines of each lane line through the spatial index R-tree to obtain a subset of the neighboring lane lines is specifically as follows: All neighboring lane lines that intersect with the bounding box of the target lane line are searched through the spatial index R-tree, and the target lane line and its neighboring lane lines are collected into a set to obtain a neighboring lane line subset.
4. The fusion method according to claim 3, It is characterized in that The bounding box is an AABB bounding box.
5. The fusion method according to claim 1, It is characterized in that The step of constructing a directed acyclic graph based on the neighborhood lane line subset is specifically as follows: Taking each lane line in the neighborhood lane line subset as a vertex of a directed acyclic graph; Calculating the angle between any two lane lines in the neighborhood lane lines, and determining whether the angle is less than a first threshold; If so, adding an edge connecting the two lane lines in the directed acyclic graph; The distance between the two lane lines is calculated as the weight of the edge.
6. The fusion method according to claim 5, It is characterized in that The step of calculating the distance between the two lane lines is specifically as follows: The average of the shortest distances from all points on the short line of the two lane lines to the long line is calculated to obtain the distance between the two lane lines.
7. The fusion method according to claim 5, It is characterized in that The step of solving the connected subgraph of the directed acyclic graph is specifically: The edges in the directed acyclic graph are traversed, and all connected subgraphs formed by the edges whose weights are less than a second threshold are calculated to obtain a connected subgraph set.
8. The fusion method according to claim 1, It is characterized in that The step of traversing the connected sub-graph set and fusing all lane lines in the connected sub-graph set to obtain the fused lane line is specifically as follows: Merging each connected subgraph in the connected subgraph set in turn to obtain corresponding merged geometry, collecting the merged geometry into a geometry set, and synchronously maintaining bijective functions of lane lines and merged geometry; After all the connected subgraphs are merged, the merged geometry in the geometry set is used as the fused lane line.
9. The fusion method according to claim 8, It is characterized in that The step of merging each connected subgraph in the connected subgraph set is specifically: Based on the bijective function, determine whether there is a corresponding merge geometry for the lane line in the target connected subgraph; if so, Replacing the corresponding lane line with the merged geometry to obtain the updated target connected subgraph; Based on the bijective function, determining whether the updated target connected subgraph includes lane lines without corresponding merged geometry; if, The updated target connected subgraph is merged to obtain a target merged geometry, and the geometry set and the bijective function are updated based on the target merged geometry.
10. The fusion method according to claim 8, It is characterized in that The steps of the merging process are specifically as follows: Calculate the nearest neighboring points of two target lane lines in the connected subgraph, and merge the neighboring points of the two target lane lines into one; Determine whether the two target lane lines both include direction information; if not, Selecting a longer segment of the two target lane lines located on the same side of the adjacent point, and merging the longer segments located on both sides of the adjacent point to obtain a merged lane line; The merged lane line replaces the two target lane lines in the connected subgraph, and the above steps are repeated until all lane lines in the connected subgraph are merged to obtain a merged geometry.
11. The fusion method according to claim 10, It is characterized in that Also includes: If both of the two target lane lines include direction information, determine whether the directions of the line segments of the two target lane lines located on the same side of the adjacent point are the same; if they are the same, The longer segment of the two target lane lines located on the same side of the adjacent point is selected, and the longer segments located on both sides of the adjacent point are merged to obtain a merged lane line.
12. A multi-lane fusion device, It is characterized in that include: An acquisition module is used to acquire multiple lane lines and unify their specifications to obtain a lane line vector set; A spatial index module, used to construct a spatial index R-tree for the lane line vector set; A neighborhood search module, configured to traverse the lane line vector set, search for the neighboring lane lines of each lane line through the spatial index R-tree, and obtain a neighboring lane line subset; A directed acyclic graph construction module, used to construct a directed acyclic graph based on the neighborhood lane line subset, and solve the connected subgraph of the directed acyclic graph to obtain a connected subgraph set; A fusion module, used for traversing the connected sub-atlas, fusing all lane lines in the connected sub-atlas, and obtaining a fused lane line; The output module is used to delete all the fused lane lines in the lane line vector set, collect the fused lane lines into the lane line vector set, and output the lane line fusion result.
13. An unmanned vehicle, include: at least one processor; as well as A memory storing instructions, which, when executed by the at least one processor, enables the at least one processor to execute the multi-lane line fusion method as described in any one of claims 1 to 11.
14. A machine-readable storage medium storing executable instructions, wherein when the instructions are executed, the machine executes the multi-lane line fusion method according to any one of claims 1 to 11.
