A lane line validity prediction method and system based on environment coding

Abstract

The present application relates to a kind of lane line validity prediction method and system based on environment coding, the validity prediction method includes: each element in the environment of the region to be measured and its basic characteristics are extracted, each element is represented by each basic vector, and each dimension of basic vector represents each basic characteristic of element;Each element is encoded based on the characteristics of basic vector, and the high-dimensional characteristics of each element are obtained;The global level interaction modeling of the high-dimensional characteristics of element is carried out by self-attention mechanism, and high-dimensional interaction characteristics are obtained;After high-dimensional interaction characteristics are converted into one-dimensional feature, the element with attribute of lane line in the environment of the region to be measured is judged based on its one-dimensional feature;More sufficient determination to lane line validity is realized.

Description

Technical Field

[0001] The present invention relates to the technical field of crowdsourced high-precision maps, and in particular to a method and system for predicting lane line validity based on environment encoding. Background Art

[0002] Currently in the field of constructing crowdsourced high-precision maps, a main source of data is to extract road features such as lane markings and curbs through vehicle-mounted sensing devices. However, due to the influence of factors such as road wear, road text, road cracks and illumination, some wrong, non-existent invalid lane lines are extracted. For example, illumination shadows of guardrails on the road surface, the middle parts of fonts such as "xiao" and "dong" in road text, and repaired cracks are identified as lane lines. All of the above are invalid lane lines, which will cause uncertain interference to the subsequent construction of high-precision maps and seriously reduce the accuracy of high-precision maps.

[0003] For these misperceived invalid lane lines, the common solution is to first obtain features such as the lateral distance and longitudinal distance between lane lines, and even the curvature change trend of lane lines based on geometric calculation, and then judge whether the lane line is reasonable and valid according to the set parameter thresholds. The obvious defects of these solutions are: first, it is difficult to accurately represent the distance between line features. In lane change scenarios, neither Frechet distance, Closest-Pair Distance (CPD), nor Sum-of-Pairs Distance (SPD) can uniformly and sufficiently describe the distance between lines; second, various parameter thresholds set based on empirical values limit the robustness of the algorithm, making it difficult to adapt to a large number of unknown scenarios. Summary of Invention

[0004] Aiming at the technical problems existing in the prior art, the present invention provides a method and system for predicting lane line validity based on environment encoding, which solves the problem that the validity of lane lines cannot be sufficiently determined.

[0005] According to a first aspect of the present invention, there is provided a method for predicting lane line validity based on environment encoding, comprising: step 1, extracting each element and its basic features in an area to be measured, representing each element with a respective basic vector, and representing each basic feature of the element with each dimension of the basic vector;

[0006] step 2, encoding each element separately based on the basic vector features to obtain high-dimensional features of each element;

[0007] step 3, performing global-level interactive modeling on the high-dimensional features of the elements through a self-attention mechanism to obtain high-dimensional interactive features;

[0008] Step 4: After converting the high-dimensional interactive features into one-dimensional features, the validity of the lane line attributes within the test area environment is judged based on their one-dimensional features.

[0009] Based on the above technical solution, the present invention can also be improved as follows.

[0010] Optionally, the process of constructing the basic vector based on each element in step 1 includes:

[0011] Each element is represented by a polyline segment, and the basic vector is constructed by taking two adjacent points of each polyline segment.

[0012] The dimensions of the basic vector include: the first and last points of the polyline, the element ID to which the polyline belongs, the element type to which the polyline belongs, color, solid / dark, polyline length, and the road level to which the environment of the area to be measured belongs.

[0013] Optionally, step 2 includes:

[0014] Each of the aforementioned basic vectors is encoded and modeled using multiple fully connected layers and pooling layers. The output features of the fully connected layers and the pooling layers are concatenated and modeled a set number of times. Then, the output features are aggregated by max pooling to obtain the high-dimensional features.

[0015] Optionally, step 3 includes:

[0016] The high-dimensional feature P is linearly mapped to Pquery, Pkey, and Pvalue through a fully connected layer, while the output dimension remains unchanged.

[0017] Then, the interaction feature out-P is output through the self-attention mechanism, which is softmax(Pquery*inv(Pkey))*Pvalue. The high-dimensional interaction feature out-P is [batch, m, Fn*8].

[0018] Optionally, step 4 includes:

[0019] By modeling with multiple fully connected layers, the high-dimensional interactive features out-P of the elements are transformed into one-dimensional features [batch, m, 1].

[0020] Optionally, in step 4, the one-dimensional feature is used as the validity probability of the lane line, and the lane line is determined to be valid when the validity probability is greater than a set threshold.

[0021] According to a second aspect of the present invention, a lane line validity prediction system based on environment coding is provided, comprising: a basic vector construction module, a high-dimensional feature acquisition module, a high-dimensional interactive feature acquisition module, and a validity determination module;

[0022] The basic vector construction module is used to extract various elements and their basic features within the environment of the area to be tested, and to represent each element with a basic vector, and to represent each basic feature of the element with each dimension of the basic vector.

[0023] The high-dimensional feature acquisition module is used to encode each element based on the basic vector features to obtain the high-dimensional features of each element.

[0024] The high-dimensional interaction feature acquisition module is used to perform global-level interaction modeling on the high-dimensional features of the elements through a self-attention mechanism to obtain high-dimensional interaction features.

[0025] The validity determination module is used to convert the high-dimensional interactive features into one-dimensional features, and then make a validity determination on the elements with the attribute of lane lines in the environment to be tested based on their one-dimensional features.

[0026] According to a third aspect of the present invention, an electronic device is provided, including a memory and a processor, the processor being configured to implement a lane line validity prediction method based on environment coding when executing a computer management program stored in the memory.

[0027] According to a fourth aspect of the present invention, a computer-readable storage medium is provided having a computer management class program stored thereon, which, when executed by a processor, implements the steps of a lane line validity prediction method based on environment coding.

[0028] This invention provides a lane line validity prediction method, system, electronic device, and storage medium based on environment coding. Within a defined area, it extracts basic attributes of elements, such as start and end points, element type, solid / virtual nature, color, and length. All line elements related to lane line validity are represented as vector sets. Each element is encoded based on these basic vector features to obtain high-dimensional feature information. Then, a high-order model is performed on all elements globally, primarily through a self-attention mechanism to train the correlation between high-dimensional features of each element, extracting global element interaction features. Finally, a one-dimensional feature is output as the lane line validity probability through multiple fully connected layers. This scheme includes feature encoding of elements themselves and interaction encoding between elements, comprehensively considering the correlation between all elements within the area, resulting in a more comprehensive determination of lane line validity. With large datasets, the prediction accuracy of lane line validity can be continuously improved through iterative training of the environment coding model, fully utilizing the richness of crowdsourced data. The deep neural network modeling method avoids the preset parameter thresholds of human experience, making the program more robust in unknown road scenarios. Attached Figure Description

[0029] Figure 1 A flowchart of a lane line validity prediction method based on environmental coding provided by the present invention;

[0030] Figure 2 A structural block diagram of a lane line validity prediction system based on environmental coding provided by the present invention;

[0031] Figure 3 A schematic diagram of the hardware structure of a possible electronic device provided by the present invention;

[0032] Figure 4 This is a schematic diagram of the hardware structure of a possible computer-readable storage medium provided by the present invention. Detailed Implementation

[0033] The principles and features of the present invention are described below with reference to the accompanying drawings. The examples given are only for explaining the present invention and are not intended to limit the scope of the present invention.

[0034] Figure 1 A flowchart of a lane line validity prediction method based on environment coding provided by the present invention is shown below. Figure 1 As shown, the effectiveness prediction method includes: Step 1, extracting each element and its basic features in the environment of the area to be tested, representing each element with each basic vector, and representing each basic feature of the element with each dimension of the basic vector.

[0035] In practice, the types of elements include: guardrails, curbs, pedestrian crossings, directional arrows, and lane lines themselves.

[0036] Step 2: Encode each element based on the basic vector features to obtain the high-dimensional features of each element.

[0037] Step 3: Use a self-attention mechanism to perform global-level interaction modeling of the high-dimensional features of the elements to obtain high-dimensional interaction features.

[0038] Step 4: After converting the high-dimensional interactive features into one-dimensional features, the validity of the elements with the attribute of lane lines in the environment to be tested is judged based on their one-dimensional features.

[0039] To address the aforementioned issues, this invention provides a lane line validity prediction method based on environmental coding. Within a defined area, basic attributes of elements are extracted, such as start and end points, element type, solid / virtual representation, color, and length. All line elements related to lane line validity are represented as vector sets. Each element is encoded based on these basic vector features to obtain high-dimensional feature information. Then, high-order modeling is performed on all elements globally, primarily through a self-attention mechanism to train the correlation between high-dimensional features of each element, extracting global element interaction features. Finally, a one-dimensional feature is output through multiple fully connected layers as the lane line validity probability, resulting in a more comprehensive determination of lane line validity.

[0040] Example 1

[0041] Embodiment 1 provided by this invention is an embodiment of a lane line effectiveness prediction method based on environment coding provided by this invention, combined with... Figure 1 It can be seen that embodiments of this effectiveness prediction method include:

[0042] Step 1: Extract the various elements and their basic features within the environment to be tested. Represent each element with a basic vector and each basic feature of the element with a dimension of the basic vector.

[0043] In one possible embodiment, the process of constructing the basic vector based on the various elements in step 1 includes:

[0044] Each element is represented by a polyline segment, and the basic vector is constructed by taking two adjacent points of each polyline segment.

[0045] The dimensions of the basic vector include: the first and last points of the polyline, the feature ID to which the polyline belongs, the feature type to which the polyline belongs, color, whether it is solid or dashed, the length of the polyline, and the road level of the environment to be measured.

[0046] In practice, the dimensions of the base vectors can be trimmed and expanded as needed. The feature format is as follows: a local environment contains m line features, each line feature is composed of N1, ..., Nm base vectors, and the base vectors contain Fn dimensions such as [Xs, Ys, Zs, Xe, Ye, Ze, type, solid, color, base_id].

[0047] Step 2: Encode each element based on the basic vector features to obtain the high-dimensional features of each element.

[0048] In one possible embodiment, step 2 includes:

[0049] Each basic vector is encoded separately, and modeled using multiple fully connected layers and pooling layers. The output features of the fully connected layers and the pooling layers are concatenated. That is, the feature dimension of the basic vector is doubled with each modeling iteration. After a set number of modeling iterations, the output features are aggregated using max pooling to obtain high-dimensional features. This process is then used to encode all elements, resulting in high-dimensional features for all elements.

[0050] In the embodiments provided by this invention, a 3-layer modeling is selected, that is, the feature dimension is expanded to 8 times the initial dimension. The feature dimension P of the element is [batch, m, Fn*8].

[0051] Step 3: Use a self-attention mechanism to perform global-level interaction modeling of the high-dimensional features of the elements to obtain high-dimensional interaction features.

[0052] In one possible embodiment, step 3 includes:

[0053] By using a fully connected layer, the high-dimensional feature P is linearly mapped to Pquery, Pkey, and Pvalue, while keeping the output dimension unchanged.

[0054] Then, the interaction feature out-P is output through the self-attention mechanism, which is softmax(Pquery*inv(Pkey))*Pvalue. The high-dimensional interaction feature out-P is [batch, m, Fn*8].

[0055] Step 4: After converting the high-dimensional interactive features into one-dimensional features, the validity of the elements with the attribute of lane lines in the environment to be tested is judged based on their one-dimensional features.

[0056] In one possible embodiment, step 3 includes:

[0057] By modeling with multiple fully connected layers, the high-dimensional interactive features out-P of the elements are transformed into one-dimensional features [batch, m, 1].

[0058] In one possible implementation, a one-dimensional feature is used as the validity probability of the lane line, and the lane line is determined to be valid when the validity probability is greater than a set threshold.

[0059] In practice, 0.5 is usually used as the dividing line. A probability greater than 0.5 is considered a reliable lane line, while a probability less than 0.5 is considered a false lane line.

[0060] Example 2

[0061] Embodiment 2 provided by this invention is an embodiment of a lane line validity prediction system based on environment coding provided by this invention. Figure 2This is a structural diagram of a lane line effectiveness prediction system based on environment coding provided in an embodiment of the present invention, combined with... Figure 2 It can be seen that the implementation of this validity prediction system includes: a basic vector construction module, a high-dimensional feature acquisition module, a high-dimensional interactive feature acquisition module, and a validity determination module.

[0062] The basic vector construction module is used to extract various elements and their basic features within the environment to be tested. Each element is represented by a basic vector, and each dimension of the basic vector represents a basic feature of the element.

[0063] In practice, the types of elements include: guardrails, curbs, pedestrian crossings, directional arrows, and lane lines themselves.

[0064] The high-dimensional feature acquisition module is used to encode each element based on the basic vector features to obtain the high-dimensional features of each element.

[0065] The high-dimensional interaction feature acquisition module is used to perform global-level interaction modeling of the high-dimensional features of elements through a self-attention mechanism to obtain high-dimensional interaction features.

[0066] The validity determination module is used to convert high-dimensional interactive features into one-dimensional features, and then to determine the validity of lane lines in the environment under test based on their one-dimensional features.

[0067] It is understood that the lane line validity prediction system based on environment coding provided by the present invention corresponds to the lane line validity prediction method based on environment coding provided in the foregoing embodiments. The relevant technical features of the lane line validity prediction system based on environment coding can be referred to the relevant technical features of the lane line validity prediction method based on environment coding, and will not be repeated here.

[0068] Please see Figure 3 , Figure 3 This is a schematic diagram illustrating an embodiment of the electronic device provided in this invention. For example... Figure 3As shown, this embodiment of the invention provides an electronic device, including a memory 1310, a processor 1320, and a computer program 1311 stored in the memory 1310 and executable on the processor 1320. When the processor 1320 executes the computer program 1311, it performs the following steps: extracting each element and its basic features in the environment to be tested; representing each element with each basic vector; representing each basic feature of the element with each dimension of the basic vector; encoding each element based on the basic vector features to obtain the high-dimensional features of each element; performing global-level interactive modeling of the high-dimensional features of the elements through a self-attention mechanism to obtain high-dimensional interactive features; and after converting the high-dimensional interactive features into one-dimensional features, judging the validity of the lane line attribute in the environment to be tested based on its one-dimensional features.

[0069] Please see Figure 4 , Figure 4 This is a schematic diagram illustrating an embodiment of a computer-readable storage medium provided by the present invention. (See diagram below.) Figure 4 As shown, this embodiment provides a computer-readable storage medium 1400, on which a computer program 1411 is stored. When the computer program 1411 is executed by a processor, it performs the following steps: extracting each element and its basic features in the environment to be tested; representing each element with each basic vector; representing each basic feature of the element with each dimension of the basic vector; encoding each element based on the basic vector features to obtain the high-dimensional features of each element; performing global-level interactive modeling of the high-dimensional features of the elements through a self-attention mechanism to obtain high-dimensional interactive features; after converting the high-dimensional interactive features into one-dimensional features, performing validity judgment on the elements with the attribute of lane lines in the environment to be tested based on their one-dimensional features.

[0070] This invention provides a lane line validity prediction method, system, electronic device, and storage medium based on environment coding. Within a defined area, it extracts basic attributes of elements, such as start and end points, element type, solid / virtual nature, color, and length. All line elements related to lane line validity are represented as vector sets. Each element is encoded based on these basic vector features to obtain high-dimensional feature information. Then, a high-order model is performed on all elements globally, primarily by training the correlation between high-dimensional features of each element through a self-attention mechanism to extract global element interaction features. Finally, a one-dimensional feature is output as the lane line validity probability through multiple fully connected layers. This scheme includes feature encoding of elements themselves and interaction encoding between elements, comprehensively considering the correlation between all elements within the area, resulting in a more comprehensive determination of lane line validity. With large datasets, the prediction accuracy of lane line validity can be continuously improved through iterative training of the environment coding model, fully utilizing the richness of crowdsourced data. The deep neural network modeling method avoids the preset parameter thresholds of human experience, making the program more robust in unknown road scenarios.

[0071] It should be noted that the descriptions of each embodiment in the above embodiments have different focuses. For parts that are not described in detail in a certain embodiment, please refer to the relevant descriptions in other embodiments.

[0072] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0073] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded computer, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0074] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0075] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0076] Although preferred embodiments of the invention have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including both the preferred embodiments and all changes and modifications falling within the scope of the invention.

[0077] Obviously, those skilled in the art can make various modifications and variations to this invention without departing from its spirit and scope. Therefore, if these modifications and variations fall within the scope of the claims of this invention and their equivalents, this invention also intends to include these modifications and variations.

Claims

1. A lane line validity prediction method based on environmental coding, characterized in that, The effectiveness prediction method includes: Step 1: Extract the various elements and their basic features within the environment to be tested. Represent each element with a basic vector, and represent each element's basic features with each dimension of the basic vector. The dimensions of the basic vector include: the first and last points of the polyline, the element ID to which the polyline belongs, the element type to which the polyline belongs, color, solid / dark, polyline length, and the road level to which the environment to be tested belongs. Step 2: Encode each element based on the basic vector features to obtain the high-dimensional features of each element; Step 2 includes: encoding each of the basic vectors respectively, modeling with multiple fully connected layers and pooling layers, concatenating the output features of the fully connected layers and the output features of the pooling layers, performing a set number of modeling operations, and then performing max pooling aggregation on the output features to obtain the high-dimensional features. Step 3: Perform global-level interaction modeling on the high-dimensional features of the elements through a self-attention mechanism to obtain high-dimensional interaction features; Step 3 includes: The high-dimensional feature P is linearly mapped to Pquery, Pkey, and Pvalue through a fully connected layer, while the output dimension remains unchanged. Then, the interaction feature out-P is output through the self-attention mechanism, which is out-P = softmax(Pquery×inv(Pkey))×Pvalue. The high-dimensional interaction feature out-P is [batch,m,Fn×8]. Step 4: After converting the high-dimensional interactive features into one-dimensional features, the validity of the lane line attributes within the test area environment is judged based on their one-dimensional features.

2. The effectiveness prediction method according to claim 1, characterized in that, The process of constructing the basic vector based on each element in step 1 includes: Each element is represented by a polyline segment, and the basic vector is constructed by taking two adjacent points of each polyline segment.

3. The effectiveness prediction method according to claim 1, characterized in that, Step 4 includes: By modeling with multiple fully connected layers, the high-dimensional interactive features out-P of the elements are transformed into one-dimensional features [batch, m, 1].

4. The effectiveness prediction method according to claim 1, characterized in that, In step 4, the one-dimensional feature is used as the validity probability of the lane line, and the lane line is determined to be valid when the validity probability is greater than a set threshold.

5. A lane line validity prediction system based on environmental coding that applies the validity prediction method as described in claims 1-4, characterized in that, The validity prediction system includes: a basic vector construction module, a high-dimensional feature acquisition module, a high-dimensional interactive feature acquisition module, and a validity determination module; The basic vector construction module is used to extract various elements and their basic features within the environment of the area to be tested, and to represent each element with a basic vector, and to represent each basic feature of the element with each dimension of the basic vector. The high-dimensional feature acquisition module is used to encode each element based on the basic vector features to obtain the high-dimensional features of each element. The high-dimensional interaction feature acquisition module is used to perform global-level interaction modeling on the high-dimensional features of the elements through a self-attention mechanism to obtain high-dimensional interaction features. The validity determination module is used to convert the high-dimensional interactive features into one-dimensional features, and then make a validity determination on the elements with the attribute of lane lines in the environment to be tested based on their one-dimensional features.

6. An electronic device, characterized in that, The system includes a memory and a processor, wherein the processor executes a computer program stored in the memory to implement the steps of the environment-coded lane line validity prediction method as described in any one of claims 1-4.

7. A computer-readable storage medium, characterized in that, It stores a computer program, which, when executed by a processor, implements the steps of the lane line validity prediction method based on environment coding as described in any one of claims 1-4.

Images