Road topology identification method and device based on hierarchical fusion of map priori and electronic equipment

Abstract

The embodiment of the invention provides a road topology identification method and device based on hierarchical fusion of map priori and electronic equipment. The method comprises the following steps: respectively carrying out feature processing on road data in a first representation format and a second representation format to obtain a first top view feature and a second top view feature; performing feature fusion on the first top view feature and the second top view feature to obtain a third top view feature, the first top view feature and the second top view feature being top view features; performing context fusion processing on query features obtained by performing feature extraction on vectorized map prior data and third top view features to obtain lane line information; analyzing the lane line information to obtain a first topological relation between the lane lines, and analyzing the lane line information and the traffic element information to obtain a second topological relation between the lane lines and the traffic elements; and summarizing the first topological relation and the second topological relation to obtain a road topology recognition result.

Description

Technical Field

[0001] The present application relates to the field of intelligent driving technology, and in particular to a road topology recognition method and device and electronic device for layered fusion map priors. Background Art

[0002] With the development of intelligent driving technology, in complex driving scenarios, vehicles need to perceive lanes and traffic elements, and infer road topology relationships (i.e., lane connectivity and the corresponding relationship with traffic elements) to provide comprehensive information for downstream path planning and motion control. Among them, lane line topology recognition in road topology relationships is a basic and indispensable function for vehicles to achieve intelligent driving. Its detection accuracy affects the performance of downstream tasks such as positioning, prediction, and planning, and is of great significance for intelligent driving scenarios.

[0003] In the related technologies, in actual driving scenarios, there are many difficulties in road topology recognition because lane lines are invisible and easily blocked, traffic elements are sparse and unevenly distributed, and topological associations are complex and diverse. Summary of the invention

[0004] In view of this, the embodiments of the present application provide a road topology recognition method and device and electronic device for layered fusion map priors to solve or alleviate the many difficulties in road topology recognition in related technologies.

[0005] According to a first aspect of an embodiment of the present application, a method for road topology recognition based on a hierarchical fusion map prior is provided, the method comprising: performing feature processing on road data in a first representation format and a second representation format respectively to obtain a first top-view feature and a second top-view feature; performing feature fusion on the first top-view feature and the second top-view feature to obtain a third top-view feature, wherein the top-view feature is obtained by the first top-view feature and the second top-view feature; performing context fusion processing on query features obtained by feature extraction of vectorized map prior data and the third top-view feature to obtain lane line information; parsing the lane line information to obtain a first topological relationship between each lane line, and parsing the lane line information and traffic element information to obtain a second topological relationship between each lane line and the traffic element; summarizing the first topological relationship and the second topological relationship to obtain a road topology recognition result.

[0006] In some embodiments of the present application, feature processing is performed on road data in a first representation format and a second representation format, respectively, to obtain a first overhead view feature and a second overhead view feature, including: acquiring real-time road data and rasterized map prior data, wherein the real-time road data includes at least lane line image data, lane line GPS data, and lane line trajectory data; performing feature extraction from the real-time road data through a feature extraction network to obtain a first feature map corresponding to the real-time road data, and converting the first feature map into a first overhead view feature through a feature conversion network; performing feature extraction on the rasterized map prior data to obtain a second overhead view feature.

[0007] In some embodiments of the present application, feature extraction is performed on the rasterized map prior data to obtain a second top view feature, including: if the rasterized road map prior data is a two-dimensional projection image, the two-dimensional projection image is directly input into the rasterized map coding network for feature extraction to obtain the second top view feature; if the rasterized road map prior data is a three-dimensional point cloud image, the three-dimensional point cloud image is converted into a three-dimensional image, and the three-dimensional image is input into the rasterized map coding network for feature extraction to obtain the second top view feature, or the three-dimensional point cloud image is directly input into a convolutional coding network for feature extraction to obtain the second top view feature.

[0008] In some embodiments of the present application, the road topology recognition method of hierarchical fusion map prior also includes: inputting vectorized map prior data into a vectorized map encoding network for feature extraction to obtain a query feature matrix; merging a pre-set feature vector with the query feature matrix to obtain a query feature.

[0009] In some embodiments of the present application, query features and third overhead view features obtained by feature extraction of vectorized map prior data are context-fused to obtain lane line information, including: inputting query features and third overhead view features obtained by feature extraction of vectorized map prior data into a vectorized map decoding network for context-fusion processing to obtain vehicle key point coordinates and lane line types; associating vehicle key point coordinates with corresponding lane line types to obtain lane line information.

[0010] In some embodiments of the present application, lane line information is parsed to obtain a first topological relationship between lane lines, including: inputting the lane line information into a first topological decoding network; and parsing the connection relationship between multiple lane lines according to the first topological decoding network to obtain a first topological relationship between multiple lane lines.

[0011] In some embodiments of the present application, lane line information and traffic element information are parsed to obtain a second topological relationship between each lane line and the traffic element, including: inputting the lane line information and the traffic element information into a second topological decoding network; determining the traffic element corresponding to each lane line according to the second topological decoding network; and associating each lane line with the corresponding traffic element to obtain a second topological relationship between the lane line and the traffic element.

[0012] According to a second aspect of an embodiment of the present application, a road topology recognition device based on a hierarchical fusion map prior is provided, the device comprising: a feature extraction module, used to perform feature processing on road data in a first representation format and a second representation format respectively, to obtain a first top view feature and a second top view feature; a first fusion module, used to perform feature fusion on the first top view feature and the second top view feature, to obtain a third top view feature, wherein the top view feature is obtained by the first top view feature and the second top view feature; a second fusion module, used to perform context fusion processing on query features and the third top view feature obtained by performing feature extraction on vectorized map prior data, to obtain lane line information; a parsing module, used to parse the lane line information, to obtain a first topological relationship between each lane line, and to parse the lane line information and traffic element information, to obtain a second topological relationship between each lane line and the traffic element; a summarizing module, used to summarize the first topological relationship and the second topological relationship, to obtain a road topology recognition result.

[0013] According to the third aspect of the embodiments of the present application, there is provided an electronic device comprising: a processor, a memory, a communication interface and a communication bus, wherein the processor, the memory and the communication interface communicate with each other via the communication bus; the memory is used to store at least one executable instruction, and the executable instruction enables the processor to perform operations corresponding to the method for identifying road topology of a priori layered fusion maps as in any of the above embodiments.

[0014] According to the layered fusion map prior road topology recognition method and its device and electronic device provided in the embodiment of the present application, a first top view feature and a second top view feature are obtained by feature processing from different road data, and the two are fused to obtain a third top view feature. Further, the third top view feature is fused with the query feature to obtain lane line information, and then the lane line information is parsed to obtain a first topological relationship between each lane line, and the lane line information and traffic element information are parsed to obtain a second topological relationship between each lane line and the traffic element, so that the first topological relationship and the second topological relationship are summarized to obtain a road topology recognition result, thereby effectively improving the road topology recognition performance. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in the embodiments of the present application. For ordinary technicians in this field, other drawings can also be obtained based on these drawings.

[0016] Figure 1 A schematic diagram of a process of a road topology recognition method using hierarchical fusion map priors according to an embodiment of the present application;

[0017] Figure 2 A schematic diagram of the structure of a road topology recognition system of a layered fusion map prior applied to a road topology recognition method of a layered fusion map prior according to an embodiment of the present application;

[0018] Figure 3 It is a structural block diagram of a road topology recognition device for hierarchical fusion map prior according to an embodiment of the present application;

[0019] Figure 4 The figure is a structural block diagram of an electronic device according to an embodiment of the present application. DETAILED DESCRIPTION

[0020] In order to enable those skilled in the art to better understand the technical solutions in the embodiments of the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments in the embodiments of the present application should fall within the scope of protection of the embodiments of the present application.

[0021] It is understandable that there are many difficulties in lane line topology recognition on the road, such as lane lines are different from general visible lane line detection tasks, lane line detection lacks clear markings, visible lane lines can only provide limited visual clues for lane line detection, and at intersections with complex topological structures, the visual clues are sparser, and both visible lane lines and invisible lane lines are easily blocked by traffic participants, which causes lane lines to be invisible and easily blocked; road topology estimation requires reasoning about the relationship between traffic lights and traffic signs, detecting traffic participants (such as vehicles, bicycles, and pedestrians), and the small number of traffic elements causes traffic elements to be sparse and unevenly distributed; the number and position of traffic elements at each traffic intersection, the difference in the number of lane lines, and the many scenes involving branches and merges that lead to highly complex topological relationships, which cause complex and diverse topological associations. Therefore, there are many difficulties in road topology recognition, resulting in low accuracy of predicted topological relationships.

[0022] Based on this, the present application provides a road topology recognition method and device and electronic device with hierarchical fusion map priors, which obtains a first top-view feature and a second top-view feature by feature processing from different road data, and fuses the two to obtain a third top-view feature. Further, the third top-view feature is fused with the query feature to obtain lane line information, and then the lane line information is parsed to obtain a first topological relationship between each lane line, and the lane line information and traffic element information are parsed to obtain a second topological relationship between each lane line and the traffic element, thereby summarizing the first topological relationship and the second topological relationship to obtain a road topology recognition result, thereby effectively improving the road topology recognition performance.

[0023] The specific implementation of the embodiments of the present application is further explained below in combination with the embodiments of the present application and the accompanying drawings.

[0024] Figure 1 The figure is a flowchart of a method for identifying road topology by layered fusion of map priors according to an embodiment of the present application.

[0025] like Figure 1 As shown, the road topology recognition method of the hierarchical fusion map prior includes:

[0026] S11: Perform feature processing on the road data in the first representation format and the road data in the second representation format respectively to obtain a first top view feature and a second top view feature.

[0027] Specifically, the road data in the first representation format refers to real-time road data, which includes at least lane line image data, lane line GPS data and lane line trajectory data. The road data in the second representation number refers to rasterized map prior data (i.e., road surface rasterized map prior, which can be a lidar point cloud map, IPM projection map, etc.). It can be understood that the road surface rasterized map prior is a preset assumption or model for the road network in map processing and analysis. It abstracts the actual road system into a grid structure. Each grid cell represents a fixed section of road or area, and each cell is assigned specific attributes, such as length, direction, traffic level, etc. The road surface rasterized map prior can help quickly locate the vehicle position, simplify route search and calculation, and predict traffic flow distribution, etc.

[0028] In some embodiments, feature processing is performed on the road data in the first representation format and the road data in the second representation format to obtain the first top view feature and the second top view feature, including:

[0029] Acquire real-time road data and rasterized map prior data, wherein the real-time road data at least includes lane line image data, lane line GPS data, and lane line trajectory data;

[0030] Extracting features from the real-time road data through a feature extraction network to obtain a first feature map corresponding to the real-time road data, and converting the first feature map into a first top view feature through a feature conversion network;

[0031] Specifically, the feature extraction network can be a backbone network in deep learning. The backbone network is used to extract high-level feature identifiers of input data. It is usually composed of multiple convolutional layers and pooling layers. It can perform layer-by-layer feature extraction and dimensionality reduction operations on input data (such as lane line image data, lane line GPS data, lane line trajectory data and other real-time road data), and finally obtain a more abstract and meaningful feature representation (such as the first feature map). This feature representation can be used for subsequent classification, target detection, image segmentation and other tasks. Common backbone networks are ResNet, RegNet, EfficientNet, PVT, Swin, etc.

[0032] Exemplarily, the backbone network of the present application may adopt a convolutional network, such as ResNet, RegNet, EfficientNet, etc., or a Transformer backbone network, such as PVT, Swin, etc. In addition, in order to further improve the performance, a feature pyramid module or a bidirectional feature pyramid module may be added to the backbone network to fuse image features of different scales.

[0033] Feature conversion networks usually refer to components used in machine learning and deep learning models, especially a technology in convolutional neural networks (CNNs). The main function of feature conversion networks is to help the model better understand and represent raw data by performing high-level feature extraction and transformation on input data (such as the first feature map). They convert images, texts, or other types of data into a set of more meaningful feature vectors (such as the first top view feature) through a series of convolutional layers, pooling layers, batch normalization, and other operations. This feature vector can capture the key information of the input data, making subsequent classification and recognition tasks more accurate.

[0034] Exemplarily, the feature conversion network of the present application can select deep learning networks such as IPM (inverse perspective mapping), LSS (Lift-Splat-Shoot, a bottom-up BEV (Bird's-Eye-View) feature modeling idea and method), BevFormer (a new type of autonomous driving technology, mainly used to learn and obtain bird's-eye view (BEV) representation from multi-camera images) and GKT (geometry-guided kernel transformer, a technology for obtaining BEV representation from multi-view images).

[0035] Among them, IPM is a technology used to eliminate the perspective effect in images, so that the deformation originally caused by perspective can be corrected, and then the scene in the image can be described and understood more accurately. The end scenarios can be lane line detection in driving, common obstacle detection in parking, etc.; the core of LSS is to extract features of the collected surround image by explicitly estimating the depth information of the image, and realize the conversion of image features to BEV features based on the estimated discrete depth information, thereby completing the pure visual perception task in autonomous driving; BEVFormer combines the spatial and temporal information of multi-camera images, uses deformation attention and temporal self-attention mechanisms, and obtains bird's-eye view features for autonomous driving; GKT uses geometric priors to guide the Transformer model to focus on important visual areas and effectively obtain BEV features. This method is particularly effective in processing the conversion from 2D images to 3D space, which helps to improve the environmental perception ability of autonomous driving systems.

[0036] Feature extraction is performed on the rasterized map prior data to obtain the second top view feature.

[0037] Among them, the rasterized map prior data is unobstructed road data collected in multiple trips, and the lane lines and ground marking information in the data are complete.

[0038] In some embodiments, performing feature extraction on the rasterized map priori data to obtain the second top view feature includes:

[0039] If the rasterized road map prior data is a two-dimensional projection map, the two-dimensional projection map is directly input into the rasterized map encoding network for feature extraction to obtain the second top view feature;

[0040] Exemplarily, when the prior data of the rasterized road map is a two-dimensional projection map (i.e., a road IPM (Intrusion Protection Matrix) projection map), the same convolutional network as the backbone network (such as ResNet, RegNet, EfficientNet, etc.) can be used as the rasterized map encoding network to perform feature extraction on the input two-dimensional projection map to obtain the second top view feature; the same Transformer network as the backbone network (such as PVT, Swin, etc.) can also be used as the rasterized map encoding network to perform feature extraction on the input two-dimensional projection map to obtain the second top view feature.

[0041] If the rasterized road map prior data is a three-dimensional point cloud map, the three-dimensional point cloud map is converted into a three-dimensional image, and the three-dimensional image is input into a rasterized map coding network for feature extraction to obtain a second top view feature. Alternatively, the three-dimensional point cloud map is directly input into a convolutional coding network for feature extraction to obtain a second top view feature.

[0042] For example, when the prior data of the rasterized road map is a three-dimensional point cloud (i.e., a laser radar point cloud), the following method can be used to extract features to obtain the second top view feature. For example, the laser radar point cloud is first converted into a three-dimensional image, and then the converted three-dimensional image is input into the same convolutional network as the backbone network (such as ResNet, RegNet, EfficientNet, etc.) for feature extraction to obtain the second top view feature, or the converted three-dimensional image is input into the same Transformer network as the backbone network (such as PVT, Swin, etc.) for feature extraction to obtain the second top view feature.

[0043] When the prior data of the rasterized road map is a three-dimensional point cloud (i.e., a laser radar point cloud), the following method can also be used to extract features to obtain the second top view features. For example, the laser radar point cloud is input into a convolutional coding network (such as a 3D sparse convolution) for feature extraction to obtain the second top view features.

[0044] In this way, by extracting the second top-view feature from the rasterized map prior data, the problem of difficulty in road topology recognition caused by occlusion by other traffic participants can be solved.

[0045] S12: Fusing the first top view feature and the second top view feature to obtain a third top view feature.

[0046] Specifically, the first top view feature and the second top view feature can be fused by weighting, maximum pooling, convolution, cross attention, etc. to obtain the third top view feature.

[0047] Optionally, the feature-level fusion network can use weighted, maximum pooling (a common pooling operation, in which the maximum pooling selects the maximum value in a specific local window (e.g., 2x2) as the output value of the window, with the purpose of retaining the most significant feature information while reducing the resolution of the feature map, emphasizing highlighted features (with large activation values), and is suitable for retaining significant features such as edges and contours), convolution, and cross attention (calculating attention on two different sequences to process the semantic relationship between the two sequences) to achieve feature-level fusion of the two top-view features.

[0048] S13: Perform context fusion processing on query features obtained by performing feature extraction on the vectorized map priori data and the third top view features to obtain lane line information.

[0049] Among them, the vectorized map prior data (i.e., vectorized map prior) can be a non-public or publicly downloaded road navigation map, which contains a rough topological relationship of lane lines. Context fusion processing can be achieved through the combination of multi-text splicing and attention mechanism, the multi-head self-attention mechanism in the Transformer model, model pruning, quantization and other technologies.

[0050] In some embodiments, the road topology recognition method of hierarchical fusion map prior also includes: inputting vectorized map prior data into a vectorized map encoding network for feature extraction to obtain a query feature matrix; merging a pre-set feature vector with the query feature matrix to obtain a query feature.

[0051] Specifically, the vectorized map encoding network can adopt a Transformer-type network. After the vectorized map prior data is input into the vectorized map encoding network, the vectorized map prior data is first passed through an embedding layer and then through a self-attention layer to obtain the encoding features of the vectorized map prior data, so as to use the encoding features as a query feature matrix.

[0052] Furthermore, in order to improve the accuracy of the road topology recognition method of the hierarchical fusion map prior, after obtaining the query feature matrix, the pre-set feature vector (i.e., the learning vector, a matrix composed of 0 and 1) is merged with the query feature matrix to obtain the query feature.

[0053] In this way, by extracting query features from vectorized map prior data, the accuracy of recognition results for more complex scenes in road topology (ie, lane centerline topology) can be enhanced.

[0054] S14: parsing the lane line information to obtain a first topological relationship between each lane line, and parsing the lane line information and the traffic element information to obtain a second topological relationship between each lane line and the traffic element;

[0055] S15: Summarize the first topological relationship and the second topological relationship to obtain a road topology recognition result.

[0056] Among them, traffic element information (i.e., traffic element categories) includes traffic lights, road signs indicating lane directions, and other relevant traffic sign information indicating lane directions. Specific categories of traffic elements include red light, green light, yellow light, traffic light of unknown color, straight ahead, left turn, right turn, straight ahead and left turn, straight ahead and right turn, U-turn, no U-turn, and other categories.

[0057] The traffic element information is information decoded from the first feature map. Specifically, the real-time road data (such as lane line image data, lane line GPS data, lane line trajectory data and other real-time road data) is subjected to layer-by-layer feature extraction and dimensionality reduction operations through the backbone network. After obtaining the first feature map, the first feature map is input into the traffic element decoding network (such as the Yolo series (You Only Look Once) is an efficient deep learning model), Deformable-DETR (an improved version of the target detection model, based on the DETR (Detection Transformer) architecture), etc.) for decoding to obtain the traffic element information. Among them, the Yolo series, as an efficient deep learning model, is mainly used for real-time target detection and tracking, and Deformable-DETR is an efficient and fast-converging target detection model, suitable for scenarios requiring high precision and small object detection.

[0058] In summary, the present application obtains the first top view feature and the second top view feature by feature processing from different road data, and fuses the two to obtain the third top view feature. Further, the third top view feature is fused with the query feature to obtain lane line information, and then the lane line information is parsed to obtain the first topological relationship between each lane line, and the lane line information and traffic element information are parsed to obtain the second topological relationship between each lane line and the traffic element, so as to summarize the first topological relationship and the second topological relationship to obtain the road topology recognition result, thereby effectively improving the road topology recognition performance.

[0059] In some embodiments, query features obtained by extracting features from the vectorized map priori data and third top view features are context-fused to obtain lane line information, including:

[0060] The query features and the third top view features obtained by feature extraction of the vectorized map prior data are input into the vectorized map decoding network for context fusion processing to obtain the vehicle key point coordinates and lane line type; the vehicle key point coordinates and the corresponding lane line type are associated to obtain the lane line information.

[0061] Specifically, the query features and the third overhead view features obtained by feature extraction of the vectorized map prior data are input into the instance-level fusion network (i.e., the vectorized map decoding network) for cross-attention calculation (i.e., context fusion processing) to obtain the three-dimensional coordinates in the vehicle coordinate system (i.e., the vehicle key point coordinates), and the lane line instance type (i.e., the lane line type). The lane line instance type includes but is not limited to non-intersection lane lines and intersection connecting lines, thereby associating the vehicle key point coordinates with the corresponding lane line type to obtain accurate lane line information.

[0062] In some embodiments, the lane line information is parsed to obtain a first topological relationship between lane lines, including:

[0063] The lane line information is input into a first topology decoding network; and according to the first topology decoding network, the connection relationship between the plurality of lane lines is parsed to obtain a first topology relationship between the plurality of lane lines.

[0064] Specifically, the first topological relationship between lane lines is generally expressed in the form of a matrix. If there is a connection relationship between two lane line instances (such as there is a connection relationship between the left-turn lane line and the straight lane line at the same intersection), the corresponding position element is 1. If there is no connection relationship between the two lane line instances (such as there is no connection relationship between the straight lane line 1 and the straight lane line 2 at different intersections), the corresponding position element is 0. By analyzing the connection relationship between multiple lane lines, a matrix composed of elements 0 and 1 is obtained, which indicates the first topological relationship between the multiple lane lines.

[0065] In some embodiments, the lane line information and the traffic element information are parsed to obtain a second topological relationship between each lane line and the traffic element, including:

[0066] The lane line information and the traffic element information are input into the second topological decoding network; the traffic element corresponding to each lane line is determined according to the second topological decoding network; each lane line is associated with the corresponding traffic element to obtain a second topological relationship between the lane line and the traffic element.

[0067] Specifically, the second topological relationship between lane lines and traffic elements is generally expressed in the form of a matrix. According to the second topological decoding network, the traffic element corresponding to each lane line is determined. If there is a corresponding relationship between the lane line instance and the traffic element (such as there is a corresponding relationship between the straight lane line and the straight traffic light at an intersection), the corresponding position element is 1, and the straight lane line is associated with the corresponding traffic light to obtain the second topological relationship between the straight lane line and the traffic light.

[0068] If there is no corresponding relationship between the lane line instance and the traffic element (such as there is no corresponding relationship between the straight lane line at intersection A and the straight traffic light at intersection B, A and B are different intersections), the corresponding position element is 0, and the straight lane line at intersection A is associated with the straight traffic light at intersection B to obtain the second topological relationship between the straight lane line and the traffic light, thereby obtaining a matrix composed of elements 0 and 1, which indicates the second topological relationship between the lane line and the traffic element.

[0069] For example, Figure 2The present invention is a schematic structural diagram of a road topology recognition system using a layered fusion map prior and a road topology recognition method using a layered fusion map prior according to an embodiment of the present application.

[0070] like Figure 2 As shown, the hierarchical fusion map prior road topology recognition system 20 includes:

[0071] Data collection and preprocessing module 200: This module is used to collect real-time road data, rasterized map prior data and vectorized map prior data, analyze and preprocess the above data, and store the above data in the data storage module 209;

[0072] Backbone network and conversion module 201: This module is used to extract features from real-time road data, a first feature map, and convert the first feature map into a first top view feature through a feature conversion network;

[0073] Rasterized map encoding module 202: This module is used to extract features from the rasterized map priori data to obtain second top view features;

[0074] Vector map encoding module 203: This module is used to extract the row features from the vector map prior data to obtain the query features;

[0075] Feature level fusion module 204: This module is used to fuse the first top view feature and the second top view feature into a third top view feature;

[0076] Instance-level fusion module 205: This module is used to perform cross-attention calculation on the query feature and the third top view feature to obtain lane line information;

[0077] Traffic element decoding module 204: This module is used to decode traffic element information from the first feature map;

[0078] Lane line topology relationship decoding module 207: This module is used to parse the lane line information to obtain the first topology relationship between each lane line;

[0079] A lane line-traffic element topological relationship decoding module 208, which is used to parse the lane line information and the traffic element information to obtain a second topological relationship between each lane line and the traffic element;

[0080] Data storage module 209: stores all data and topological recognition results, including lane line information, traffic element information, the first topological relationship between lane lines, the second topological relationship between lane lines and traffic elements, etc.

[0081] It should be noted that the method for performing data processing in each module of the road topology recognition system refers to the description of the above embodiment.

[0082] In summary, the present application obtains the first top view feature and the second top view feature by feature processing from different road data, and fuses the two to obtain the third top view feature. Further, the third top view feature is fused with the query feature to obtain lane line information, and then the lane line information is parsed to obtain the first topological relationship between each lane line, and the lane line information and traffic element information are parsed to obtain the second topological relationship between each lane line and the traffic element, so as to summarize the first topological relationship and the second topological relationship to obtain the road topology recognition result, thereby effectively improving the road topology recognition performance.

[0083] The hierarchical fusion map priori road topology recognition method of this embodiment can be executed by any appropriate electronic device with data processing capabilities, including but not limited to: servers, mobile terminals (such as mobile phones, PADs, etc.) and PCs, etc.

[0084] Figure 3 It is a structural block diagram of a road topology recognition device based on hierarchical fusion map prior according to an embodiment of the present application.

[0085] like Figure 3 As shown, the hierarchical fusion map prior road topology recognition device 30 can perform Figure 1 A road topology recognition method based on a hierarchical fusion map prior, a road topology recognition device 30 based on a hierarchical fusion map prior includes: a feature extraction module 301, used to perform feature processing on road data in a first representation format and a second representation format respectively, to obtain a first top view feature and a second top view feature; a first fusion module 302, used to perform feature fusion on the first top view feature and the second top view feature, to obtain a third top view feature, wherein the top view feature is obtained by the first top view feature and the second top view feature; a second fusion module 303, used to perform context fusion processing on the query feature and the third top view feature obtained by performing feature extraction on the vectorized map prior data, to obtain lane line information; a parsing module 304, used to parse the lane line information, to obtain a first topological relationship between each lane line, and to parse the lane line information and the traffic element information, to obtain a second topological relationship between each lane line and the traffic element; a summarizing module 305, used to summarize the first topological relationship and the second topological relationship, to obtain a road topology recognition result.

[0086] In some embodiments of the present application, the feature extraction module 301 is specifically used to: obtain real-time road data and rasterized map prior data, wherein the real-time road data includes at least lane line image data, lane line GPS data and lane line trajectory data; perform feature extraction from the real-time road data through a feature extraction network to obtain a first feature map corresponding to the real-time road data, and convert the first feature map into a first overhead view feature through a feature conversion network; perform feature extraction on the rasterized map prior data to obtain a second overhead view feature.

[0087] In some embodiments of the present application, the feature extraction module 301 is specifically used for: if the rasterized road map prior data is a two-dimensional projection map, the two-dimensional projection map is directly input into the rasterized map coding network for feature extraction to obtain a second top view feature; if the rasterized road map prior data is a three-dimensional point cloud map, the three-dimensional point cloud map is converted into a three-dimensional image, and the three-dimensional image is input into the rasterized map coding network for feature extraction to obtain a second top view feature, or the three-dimensional point cloud map is directly input into the convolutional coding network for feature extraction to obtain a second top view feature.

[0088] In some embodiments of the present application, the feature extraction module 301 is also used to: input the vectorized map prior data into the vectorized map encoding network for feature extraction to obtain a query feature matrix; merge the pre-set feature vector with the query feature matrix to obtain the query feature.

[0089] In some embodiments of the present application, the second fusion module 303 is specifically used to: input the query features and the third overhead view features obtained by feature extraction of the vectorized map prior data into the vectorized map decoding network for context fusion processing to obtain the vehicle key point coordinates and the lane line type; associate the vehicle key point coordinates with the corresponding lane line type to obtain lane line information.

[0090] In some embodiments of the present application, the parsing module 304 is specifically used to: input lane line information into a first topology decoding network; and parse the connection relationship between multiple lane lines according to the first topology decoding network to obtain a first topological relationship between the multiple lane lines.

[0091] In some embodiments of the present application, the parsing module 304 is specifically used to: input lane line information and traffic element information into a second topological decoding network; determine the traffic element corresponding to each lane line based on the second topological decoding network; associate each lane line with the corresponding traffic element to obtain a second topological relationship between the lane line and the traffic element.

[0092] In addition, the technical solutions of the above-mentioned embodiments of the present application can be combined with each other and are not limited here.

[0093] According to the hierarchical fusion map prior road topology recognition device of the embodiment of the present application, a first top view feature and a second top view feature are obtained by feature processing from different road data, and the two are fused to obtain a third top view feature. Further, the third top view feature is fused with the query feature to obtain lane line information, and then the lane line information is parsed to obtain a first topological relationship between each lane line, and the lane line information and traffic element information are parsed to obtain a second topological relationship between each lane line and the traffic element, so that the first topological relationship and the second topological relationship are summarized to obtain a road topology recognition result, thereby effectively improving the road topology recognition performance.

[0094] The hierarchical fusion map prior road topology recognition device 30 of this embodiment is used to implement the corresponding hierarchical fusion map prior road topology recognition method in the aforementioned multiple method embodiments, and has the beneficial effects of the corresponding method embodiments, which will not be repeated here. In addition, the functional implementation of each module in the hierarchical fusion map prior road topology recognition device 30 of this embodiment can refer to the description of the corresponding parts in the aforementioned method embodiments, which will not be repeated here.

[0095] Figure 4 This is a structural block diagram of an electronic device according to an embodiment of the present application. The specific embodiment of the present application does not limit the specific implementation of the electronic device.

[0096] like Figure 4 As shown, the electronic device 40 may include: a processor 401, a memory 402, a communication interface 403 and a communication bus 404. The processor 401, the memory 402 and the communication interface 403 communicate with each other through the communication bus 404; the memory 402 is used to store at least one executable instruction, and the executable instruction enables the processor 401 to perform the operation corresponding to the road topology recognition method of any of the above embodiments.

[0097] The processor 401 may be a central processing unit (CPU), or an application-specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of the present application. The one or more processors included in the smart device may be processors of the same type, such as one or more CPUs; or processors of different types, such as one or more CPUs and one or more ASICs.

[0098] The memory 402 is used to store programs, and may include a high-speed RAM memory, and may also include a non-volatile memory (non-volatile memory), such as at least one disk memory.

[0099] According to the electronic device of the embodiment of the present application, a first top view feature and a second top view feature are obtained by feature processing from different road data, and the two are fused to obtain a third top view feature. Further, the third top view feature is fused with the query feature to obtain lane line information, and then the lane line information is parsed to obtain a first topological relationship between each lane line, and the lane line information and traffic element information are parsed to obtain a second topological relationship between each lane line and the traffic element, thereby aggregating the first topological relationship and the second topological relationship to obtain a road topology recognition result, thereby effectively improving the road topology recognition performance.

[0100] It should be pointed out that, according to the needs of implementation, the various components / steps described in the embodiments of the present application can be split into more components / steps, or two or more components / steps or partial operations of components / steps can be combined into new components / steps to achieve the purpose of the embodiments of the present application.

[0101] The above-mentioned method according to the embodiment of the present application can be implemented in hardware, firmware, or implemented as software or computer code that can be stored in a recording medium (such as a CD ROM, RAM, floppy disk, hard disk or magneto-optical disk), or implemented as a computer code originally stored in a remote recording medium or a non-temporary machine-readable medium downloaded through a network and to be stored in a local recording medium, so that the method described herein can be stored in such software processing on a recording medium using a general-purpose computer, a dedicated processor or programmable or dedicated hardware (such as an ASIC or FPGA). It can be understood that a computer, a processor, a microprocessor controller or programmable hardware includes a storage component (e.g., RAM, ROM, flash memory, etc.) that can store or receive software or computer code, and when the software or computer code is accessed and executed by a computer, a processor or hardware, the road topology recognition method for a priori hierarchical fusion maps described herein is implemented. In addition, when a general-purpose computer accesses code for implementing the road topology recognition method for a priori hierarchical fusion maps shown herein, the execution of the code converts the general-purpose computer into a dedicated computer for executing the road topology recognition method for a priori hierarchical fusion maps shown herein.

[0102] Those of ordinary skill in the art will appreciate that the units and method steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the embodiments of the present application.

[0103] The above implementation methods are only used to illustrate the embodiments of the present application, and are not intended to limit the embodiments of the present application. Ordinary technicians in the relevant technical field can make various changes and modifications without departing from the spirit and scope of the embodiments of the present application. Therefore, all equivalent technical solutions also belong to the scope of the embodiments of the present application, and the scope of patent protection of the embodiments of the present application should be defined by the claims. The systems, devices or modules described in the above embodiments can be implemented by computer chips or entities, or by products with certain functions.

Claims

1. A road topology recognition method based on hierarchical fusion map priors, characterized in that: The method comprises: Perform feature processing on the road data in the first representation format and the road data in the second representation format respectively to obtain a first top view feature and a second top view feature; Performing feature fusion on the first top view feature and the second top view feature to obtain a third top view feature, wherein the top view feature is obtained by combining the first top view feature and the second top view feature; Performing context fusion processing on query features obtained by extracting features from the vectorized map priori data and the third top view features to obtain lane line information; Parsing the lane line information to obtain a first topological relationship between lane lines, and parsing the lane line information and traffic element information to obtain a second topological relationship between lane lines and traffic elements; The first topological relationship and the second topological relationship are summarized to obtain a road topology recognition result.

2. The road topology recognition method based on hierarchical fusion map prior according to claim 1 is characterized in that: Feature processing is performed on the road data in the first representation format and the road data in the second representation format to obtain a first top view feature and a second top view feature, including: Acquire real-time road data and rasterized map prior data, wherein the real-time road data at least includes lane line image data, lane line GPS data, and lane line trajectory data; Extracting features from the real-time road data through a feature extraction network to obtain a first feature map corresponding to the real-time road data, and converting the first feature map into a first top view feature through a feature conversion network; Feature extraction is performed on the rasterized map priori data to obtain second top view features.

3. The road topology recognition method based on hierarchical fusion map prior according to claim 2 is characterized in that: Extracting features from the rasterized map priori data to obtain second top view features includes: If the rasterized road map priori data is a two-dimensional projection map, the two-dimensional projection map is directly input into the rasterized map encoding network for feature extraction to obtain the second top view feature; If the rasterized road map prior data is a three-dimensional point cloud map, after converting the three-dimensional point cloud map into a three-dimensional image, the three-dimensional image is input into the rasterized map coding network for feature extraction to obtain the second top view feature, or the three-dimensional point cloud map is directly input into a convolutional coding network for feature extraction to obtain the second top view feature.

4. The road topology recognition method based on hierarchical fusion map prior according to claim 1 is characterized in that: The method further comprises: Inputting the vectorized map prior data into a vectorized map encoding network for feature extraction to obtain a query feature matrix; The preset feature vector is combined with the query feature matrix to obtain the query feature.

5. The road topology recognition method based on hierarchical fusion map prior according to claim 1 is characterized in that: The query feature obtained by extracting the priori data of the vectorized map and the third top view feature are subjected to context fusion processing to obtain lane line information, including: Inputting the query features obtained by extracting the priori data of the vectorized map and the third top view features into the vectorized map decoding network for context fusion processing to obtain the coordinates of the key points of the vehicle and the type of the lane line; The vehicle key point coordinates are associated with the corresponding lane line type to obtain the lane line information.

6. The road topology recognition method based on hierarchical fusion map prior according to claim 1, characterized in that: The lane line information is parsed to obtain a first topological relationship between lane lines, including: Inputting the lane line information into a first topology decoding network; According to the first topology decoding network, the connection relationship between the multiple lane lines is analyzed to obtain a first topological relationship between the multiple lane lines.

7. The road topology recognition method based on hierarchical fusion map prior according to claim 1 is characterized in that: The lane line information and the traffic element information are parsed to obtain a second topological relationship between each lane line and the traffic element, including: Inputting the lane line information and the traffic element information into a second topology decoding network; Determine the traffic element corresponding to each lane line according to the second topology decoding network; Each lane line is associated with a corresponding traffic element to obtain a second topological relationship between the lane line and the traffic element.

8. A road topology recognition device based on hierarchical fusion map priors, characterized in that: The device comprises: A feature extraction module, used to perform feature processing on the road data in the first representation format and the road data in the second representation format to obtain a first top view feature and a second top view feature; A first fusion module is used to fuse the first top view feature and the second top view feature to obtain a third top view feature, wherein the top view feature is obtained by the first top view feature and the second top view feature; A second fusion module is used to perform context fusion processing on the query feature obtained by extracting the prior data of the vectorized map and the third top view feature to obtain lane line information; A parsing module, used to parse the lane line information to obtain a first topological relationship between each lane line, and parse the lane line information and traffic element information to obtain a second topological relationship between each lane line and the traffic element; The summarizing module is used to summarize the first topological relationship and the second topological relationship to obtain a road topology recognition result.

9. An electronic device, comprising: A processor, a memory, a communication interface and a communication bus, wherein the processor, the memory and the communication interface communicate with each other via the communication bus; The memory is used to store at least one executable instruction, and the executable instruction enables the processor to perform operations corresponding to the layered fusion map a priori road topology recognition method according to any one of claims 1-7.

Images