Vector map generation method and device, equipment and storage medium

Abstract

The invention provides a vector map generation method and device, equipment and a storage medium, and the method comprises the steps: obtaining environment data collected by a sensor on a vehicle at a current position, carrying out the encoding of the environment data through a pre-trained encoder of a mapping model, and obtaining an online map encoding feature; obtaining vector data of at least one target priori map, and encoding the vector data through a pre-trained vector encoder to obtain priori map vector features; wherein a geographic area corresponding to the target prior map is overlapped with a sensor sensing area; and decoding the online map coding features and the prior map vector features through a decoder of the mapping model to obtain a local vector map of the sensor sensing area. Local vector map reconstruction based on a prior map is realized, and the integrity and accuracy of map reconstruction are improved.

Description

Technical Field

[0001] The present application relates to the field of high-precision map technology, and in particular to a method, device, equipment and storage medium for generating a vector map. Background Art

[0002] High-definition maps (HD maps) can provide key positioning information of vehicles and detailed descriptions of the surrounding environment, which are crucial for autonomous driving decisions. However, the data acquisition and maintenance costs of HD maps are high, resulting in the problem of untimely updates of HD maps.

[0003] In order to reduce the dependence of autonomous driving on high-precision maps, local high-precision maps can be constructed in real time using data collected by on-board sensors during driving, saving the cost of map construction. However, the data collected by on-board sensors is sensitive to occlusion, and in adverse weather conditions or scenes such as dim lights at night, local high-precision maps will be incomplete or inaccurate, affecting the decision-making of autonomous driving.

[0004] Therefore, there is an urgent need to provide a solution for constructing local high-precision maps with high accuracy. Summary of the invention

[0005] The present application provides a method, device, equipment and storage medium for generating a vector map, which realizes the correction and supplementation of a map constructed by using data collected by vehicle-mounted sensors using supplementary information provided by a priori maps, thereby improving the accuracy and completeness of map construction.

[0006] In a first aspect, the present application provides a method for generating a vector map, comprising:

[0007] Obtaining environmental data collected by sensors on the vehicle at the current location, and encoding the environmental data through an encoder of a pre-trained mapping model to obtain online map encoding features;

[0008] Obtaining vector data of at least one target prior map, and encoding the vector data through a pre-trained vector encoder to obtain a prior map vector feature; wherein the geographical area corresponding to the target prior map overlaps with the sensor perception area, and the sensor perception area is a geographical area determined based on the field of view of the sensor and the current position;

[0009] The online map encoding features and the priori map vector features are decoded by the decoder of the mapping model to obtain a local vector map of the sensor sensing area.

[0010] In a second aspect, the present application provides a device for generating a vector map, comprising:

[0011] An online feature acquisition module is used to obtain environmental data collected by sensors on the vehicle at the current position, and encode the environmental data through an encoder of a pre-trained mapping model to obtain online map encoding features;

[0012] A priori vector acquisition module, used to obtain vector data of at least one target priori map, and encode the vector data through a pre-trained vector encoder to obtain a priori map vector feature; wherein the geographical area corresponding to the target priori map overlaps with the sensor perception area, and the sensor perception area is a geographical area determined based on the field of view of the sensor and the current position;

[0013] The vector map construction module is used to decode the online map encoding features and the prior map vector features through the decoder of the mapping model to obtain a local vector map of the sensor perception area.

[0014] In a third aspect, the present application provides an electronic device comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the electronic device performs the method provided in the first aspect of the present application.

[0015] In a fourth aspect, the present application provides a computer-readable storage medium, in which computer-executable instructions are stored. When a processor executes the computer-executable instructions, the method provided in the first aspect of the present application is implemented.

[0016] In a fifth aspect, the present application provides a program product, including a computer program, which, when executed by a processor, implements the method provided in the first aspect of the present application.

[0017] The vector map generation method, device, equipment and storage medium provided by the present application introduce a priori map on the basis of constructing a vector map using environmental data collected by vehicle-mounted sensors and a mapping model of a pre-trained encoder-decoder structure. The vector data of one or more priori maps are encoded by a pre-trained vector encoder to obtain the priori map vector features, so that the decoder of the mapping model predicts the vector map based on the output of the encoder of the mapping model and the priori map vector features. By introducing the priori map vector features, correction and supplementation of online map construction based on the priori map are realized, and the accuracy and completeness of map construction are improved; at the same time, a unified vector encoder is provided, which can realize the unified encoding of vector data of multiple priori maps, support the reference of multiple types of prior map data when building maps, further improve the accuracy of map construction, and has a wide range of applications and high flexibility. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0019] Figure 1 A schematic diagram of a local high-precision map reconstruction process provided in an embodiment of the present application;

[0020] Figure 2 A schematic diagram of a process for generating a vector map provided in an embodiment of the present application;

[0021] Figure 3 A schematic diagram of the process of constructing a vector map framework provided in an embodiment of the present application;

[0022] Figure 4 For this application Figure 2 The flowchart of step S202 provided in the illustrated embodiment;

[0023] Figure 5 A schematic diagram of the structure of a vector encoder provided in an embodiment of the present application;

[0024] Figure 6 A schematic diagram of the structure of another vector map construction framework provided in an embodiment of the present application;

[0025] Figure 7 A schematic diagram of a vector encoder pre-training process provided in an embodiment of the present application;

[0026] Figure 8 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application.

[0027] The above drawings have shown clear embodiments of the present application, which will be described in more detail later. These drawings and text descriptions are not intended to limit the scope of the present application in any way, but to illustrate the concept of the present application to those skilled in the art by referring to specific embodiments. DETAILED DESCRIPTION

[0028] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The implementations described in the following exemplary embodiments do not represent all implementations consistent with the present application. Instead, they are merely examples of devices and methods consistent with some aspects of the present application as detailed in the appended claims.

[0029] It should be noted that the user information (including but not limited to user device information, user attribute information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant laws, regulations and standards, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0030] First, some of the terms involved in this application are explained:

[0031] Standard Definition Map (SD Map): A map with moderate accuracy and detail, usually at the meter level, consisting of coarse-grained road topology information, mainly used for basic navigation functions and positioning needs. This type of map can provide basic roads, landmarks and geographic information, suitable for users to conduct daily map navigation and query.

[0032] High Definition Map (HD Map): It is a high-precision, information-rich map with an accuracy of up to centimeters. It can express map elements such as roads, lanes, traffic signs, and ground markings. It is used to provide vehicles with more accurate positioning, route planning, and environmental perception. It is widely used in autonomous driving, lane-level navigation, and other fields.

[0033] High-precision map: refers to a map with higher accuracy than standard-precision maps, including high-precision maps and maps with accuracy between standard-precision maps and high-precision maps, such as maps with sub-meter accuracy.

[0034] It should be noted that there are high-precision maps such as high-quality maps, lane-level maps, and light maps in the industry, all of which fall into the category of high-precision maps mentioned in this application. This application does not limit the naming method of such maps.

[0035] Local map reconstruction: also known as online map reconstruction, refers to the timely acquisition of information about the vehicle's surroundings through on-board sensor data, and using it to build and update maps around the vehicle for vehicle positioning and navigation.

[0036] High-precision maps are widely used in lane-level navigation, assisted driving, autonomous driving and other fields due to their centimeter-level accuracy. However, due to the high requirements for data accuracy, the data collection cost of high-precision maps is expensive, resulting in the untimely update of high-precision maps and their inability to cope with changing road conditions.

[0037] In the process of autonomous driving, timely and accurate perception of the surrounding environment is crucial. In order to overcome the problem of untimely updates of high-precision maps, a method of local map reconstruction has been proposed in recent studies. By using data collected by sensors on the vehicle during autonomous driving, such as images, lidar point clouds, etc., local high-precision maps can be constructed, saving a lot of costs.

[0038] For example, Figure 1 A schematic diagram of a local high-precision map reconstruction process provided in an embodiment of the present application is shown in FIG. Figure 1 As shown in Figure 1, the model that local high-precision map reconstruction relies on includes two main modules: encoder and decoder. During the driving process of the vehicle, the vehicle's surrounding environment data, such as images, lidar point clouds, etc., are collected through on-board sensors. Figure 1 Take the panoramic multi-view RGB images img1 to img6 collected by the surround-view camera on the vehicle as an example; the data collected by the on-board sensor is preprocessed and input into the encoder for encoding and then passed through the decoder to obtain the prediction result, that is, the local high-precision map.

[0039] Reconstructing local high-precision maps through data collected by sensors reduces the reliance on high-precision maps and reduces costs. However, the data collected by sensors is sensitive to occlusion and easily affected by environmental visibility. For example, when the traffic volume is heavy, map elements such as intersections and lane lines are blocked by vehicles, resulting in the inability to extract the blocked map elements; or on rainy days, some map elements cannot be perceived due to ground reflection, resulting in incomplete reconstruction of local maps, affecting the progress of downstream tasks of autonomous driving.

[0040] In order to improve the completeness of local map reconstruction, the present application provides a method for generating a vector map, which realizes the use of additional information provided by the prior map to supplement the perception data and improve the completeness of local vector map reconstruction. The local vector map is a map with high precision that can display lane-level road vectors. Specifically, on the basis of using the data collected by the vehicle-mounted sensor to construct the map through a mapping model with an encoder-decoder structure, the vector data of the prior map is uniformly encoded through an additional vector encoder to obtain the prior map vector features. The decoding results of the prior map vector features and the feature representation of the sensor data, namely the online map encoding features, are used to predict the map elements of the local map vectorization to obtain the local vector map. The introduction of the prior map improves the completeness of local map reconstruction.

[0041] Figure 2 A flow chart of a method for generating a vector map provided in an embodiment of the present application, which method can be executed by an electronic device with corresponding data processing capabilities, such as a vehicle-mounted terminal, an automatic driving system, etc.

[0042] like Figure 2 As shown, the method for generating the vector map includes the following steps:

[0043] Step S201, obtaining environmental data collected by a sensor on the vehicle at a current position, and encoding the environmental data through an encoder of a pre-trained mapping model to obtain online map encoding features.

[0044] The vehicle may be a vehicle in motion, such as a vehicle in an automatic driving state. The sensors on the vehicle may also be referred to as on-board sensors, including but not limited to cameras, laser radars, etc. Correspondingly, the environmental data includes images, point cloud data, etc.

[0045] When the vehicle is at its current position, the sensors on the vehicle collect environmental data of the surrounding environment within its field of view. Taking the surround view camera as an example, during the process of automatic driving of the vehicle, the surround view camera can be used to periodically collect panoramic images of the vehicle's surrounding environment.

[0046] The pre-trained mapping model can be any model with an encoder-decoder structure for local map reconstruction, such as the HDMapNet (High-Definition Map Network) model, the VectorMapNet (Vector Map Network) model, the MapTR (Map Transformer) model, etc.

[0047] For example, the encoder of the mapping model may be a Bird's Eye View (BEV) encoder, which is used to extract features in the BEV space to obtain online map encoding features, namely, BEV features f bev .

[0048] Taking the environmental data collected by the sensor including the panoramic image and point cloud data of the surrounding environment as an example, the panoramic image and point cloud data of the surrounding environment can be mapped to the BEV space through the BEV encoder of the pre-trained mapping model, and feature extraction is performed to obtain the BEV feature f bev .

[0049] For panoramic images in environmental data, the panoramic image and the parameters of the camera or camera that captures each image in the panoramic image can be input into the BEV encoder. The BEV encoder extracts the depth information of the image based on the parameters of the camera or camera and the captured image, and uses the depth information to convert the perspective of the image to obtain the BEV features of the image.

[0050] For the point cloud data in the environmental data, the BEV encoder can first downsample the point cloud data, and then convert the perspective, that is, convert the point cloud data in the three-dimensional space into a bird's-eye view on a two-dimensional plane, and perform feature extraction to obtain the BEV features of the point cloud data.

[0051] Step S202, obtaining vector data of at least one target prior map, and encoding the vector data by a pre-trained vector encoder to obtain a priori map vector features.

[0052] Among them, the prior map is a pre-constructed map, the target prior map is a geographical area that overlaps with the sensor perception area, and the sensor perception area is a geographical area determined based on the field of view of the sensor and the current position of the vehicle.

[0053] The vector data of the prior map is used to represent the vectorized map elements, which can include point vectors and segment vectors. The vector data of the point vector can be represented by the position and direction of the point, and the segment vector is the vector corresponding to the line segment, including multiple point vectors. For example, a lane line can be represented by the vector data of one or more segment vectors.

[0054] In some embodiments, the geographic area corresponding to the target a priori map includes the current location.

[0055] Prior maps can be divided into standard maps, high-precision maps and local vector maps. Figure 3 The local vector map is a map of the area around the vehicle obtained by using the local map reconstruction technology.

[0056] Optionally, the type of the at least one target prior map includes multiple types of standard maps, high-precision maps and local vector maps.

[0057] Exemplarily, at least one target prior map may include at least one standard precision prior map, at least one high precision prior map and at least one historical local vector map. The standard precision prior map is a target prior map of the standard precision map type, the high precision prior map is a target prior map of the standard precision map type, and the historical local vector map is a target prior map of the local vector map type.

[0058] When at least one target prior map is a plurality of target prior maps, it may be a plurality of target prior maps of the same type, or a plurality of target prior maps of different types. A prior map is a pre-generated map that describes the distribution of map elements in a certain geographical area, such as a high-precision map or a standard map of a certain city, or a local vector map generated in historical time, i.e., a historical local vector map; a target prior map may be a portion selected from a pre-generated high-precision map or a standard map, such as a portion with a sensor perception area as a geographical area, or may be a historical local vector map, specifically a historical local vector map where the corresponding geographical area overlaps with the sensor perception area.

[0059] Vector data of multiple prior maps can be pre-stored, and vector data of different prior maps can be stored in different areas. At the same time, as local vector maps are continuously generated, vector data of newly generated local vector maps are added in the corresponding storage area.

[0060] By introducing various types of prior maps, more sufficient additional information and reference information can be provided for online map construction, further improving the completeness and accuracy of local map reconstruction.

[0061] The sensor perception area can be determined based on the current position of the vehicle and the field of view of the sensor; then, part of the sensor perception area is read from the vector data of each pre-stored standard precision map and high-precision map to obtain a target prior map of the type of standard precision map and a target prior map of the type of high-precision map. For the pre-stored historical local vector map, that is, the local vector map generated in the historical time, the historical vector map in which the corresponding geographical area overlaps with the sensor perception area can be used as the target prior map of the type of local vector map.

[0062] The sources of vector data of different prior maps may be different, resulting in inconsistent dimensions of vector data of the target prior map. The vector data of multiple target prior maps may be pre-selected for data padding, so that the dimensions of the vector data input to the vector encoder are consistent. The vector encoder may include several layers of RNN (Recurrent Neural Network) or its variants, such as LSTM (Long Short-Term Memory), GRU (Gate Recurrent Unit), etc.

[0063] In order to reduce the workload before vector data is input into the vector encoder, solve the problem of sparse and discrete vector data, and inconsistent vector data dimensions of prior maps from different sources, and realize unified encoding of different target prior maps, the vector encoder may include an embedding layer and an encoding layer. The introduction of the embedding layer can solve the problem of inconsistent and discrete vector data dimensions of different target prior maps. The vector data of the target prior map is mapped to a continuous feature space through the embedding layer, and then represented by the encoding layer to obtain the vector features of the prior map.

[0064] Step S203: decoding the online map encoding features and the priori map vector features through the decoder of the mapping model to obtain a local vector map of the sensor sensing area.

[0065] The local vector map of the sensor sensing area is a local vector map with higher accuracy than the standard map, which can provide information such as lane lines, traffic signs, and ground markings. The construction method of local vector maps is different from that of traditional high-precision maps. The construction cost of traditional high-precision maps is expensive and is an offline process; the construction cost of local vector maps is lower and is an online process.

[0066] The prior map vector features can be integrated with the online map encoding features or query vectors, and then in the decoder, the integrated features are used to predict the map elements in the local vector map, thereby obtaining a local vector map of the vehicle's surrounding environment, that is, a local vector map of the sensor perception area.

[0067] The mapping model and the vector encoder constitute a vector map construction framework. For example, Figure 3 A schematic diagram of the process of constructing a vector map framework provided in the embodiment of the present application is shown in FIG. Figure 3As shown, the number of target prior maps of the type of standard precision map and high precision map is 1, and the number of target prior maps of the type of local vector map is multiple. For each target prior map, the target prior vector data Mprior is obtained. The target prior vector data Mprior can be composed of multiple vectors, each vector corresponds to a segment vector Vector, and each segment vector is composed of vector data of multiple point vectors. The vector data of the point vector includes position (x, y), direction (vx, vy) and type (vy). Figure 3 The vector data of the segment vector Vector1 in is The superscript indicates the serial number of the point vector to which it belongs, and the subscript indicates the serial number of the segment vector to which it belongs. The a priori vector data of each target is spliced ​​together and input into the vector encoder to obtain the a priori map vector features. For environmental data collected online by vehicle sensors, such as panoramic images, the PV (perspective view) features are extracted through the backbone network Backbone of the BEV encoder of the mapping model, and then the PV features are converted to the BEV space through the PV2BEV module to obtain the BEV features. The BEV features are an example of online map encoding features; then, through the decoder of the mapping model, based on the BEV features and the a priori map vector features, the prediction results are output, that is, the vectorized map elements in the local vector map are predicted, thereby realizing the construction of the local vector map.

[0068] The method for generating a local vector map provided by the present application introduces a priori maps on the basis of constructing vector maps using environmental data collected by vehicle-mounted sensors and a mapping model of a pre-trained encoder-decoder structure. The vector data of one or more priori maps are encoded by a pre-trained vector encoder to obtain the priori map vector features, so that the decoder of the mapping model predicts the vector map based on the output of the encoder of the mapping model and the priori map vector features. By introducing the priori map vector features, correction and supplementation of online map construction based on the priori map is realized, and the accuracy and completeness of map construction are improved; at the same time, a unified vector encoder is provided, which can realize the unified encoding of vector data of multiple priori maps, support the reference of multiple types of prior map data when building maps, further improve the accuracy of map construction, and has a wide range of applications and high flexibility.

[0069] In order to solve the problems of discrete, sparse and inconsistent length of vector data, embedding layer and attention mechanism can be introduced when designing vector encoder. The embedding layer can map discrete vector data of different lengths to continuous feature space, and the attention mechanism can promote the interaction between vector data features and better explore the connection between different channel features, so as to better understand and represent the characteristics of vector data and provide strong support for the construction of vector maps.

[0070] Optionally, the vector encoder may include an embedding layer and an attention encoding layer, wherein the embedding layer is used to embed and concatenate various data in the input prior map vector data to obtain embedded vector features; the attention encoding layer is used to encode the embedded vector features using the attention mechanism to obtain the prior map vector features. Accordingly, encoding the vector data based on the pre-trained vector encoder to obtain the prior map vector features includes:

[0071] Based on the embedding layer, each item of data in the vector data is embedded and concatenated to obtain an embedded vector feature; based on the attention encoding layer, the embedded vector feature is encoded using an attention mechanism to obtain the prior map vector feature.

[0072] The embedding layer can embed and concatenate the data of each vector in the vector data of the prior map through sinusoidal embedding to obtain the embedded vector features. When concatenating, in order to facilitate the attention encoding layer to better identify the features of different vectors, a special beginning marker can also be added to the vector.

[0073] Since the amount of information in vector data itself is limited, learning the feature representation of vector data is challenging. It is necessary to introduce an attention mechanism to enhance the perception of vectors to other vectors through feature interactions between different vectors. The attention encoding layer can be a self-attention mechanism-based encoding layer, and attention mask technology can also be introduced to further improve the perception of vector encoders to vector data.

[0074] Optional, Figure 4 For this application Figure 2 The illustrated embodiment provides a flow chart of step S202. In this embodiment, the attention encoding layer includes an intra-vector attention encoder and an inter-vector attention encoder.

[0075] This embodiment is described by taking multiple target prior maps as an example. Figure 4 As shown, step S202 may specifically include the following steps:

[0076] Step S401, obtaining vector data of multiple target prior maps.

[0077] For example, the multiple target prior maps may include a standard map, a high-precision map, and a local vector map. Figure 3 The local vector map type of prior map is a local vector map generated by reconstructing local maps in historical time, which can also be called a historical local vector map.

[0078] Step S402, concatenating the vector data of the plurality of target prior maps and inputting the concatenated vector data into the embedding layer of the vector encoder.

[0079] The vector data of multiple target prior maps can be concatenated in a certain order to obtain the input of the vector encoder, that is, the input of the embedding layer.

[0080] The vectors described in the vector data can be divided into point vectors and segment vectors. The segment vector includes multiple point vectors. The point vector can be regarded as a word, and the segment vector can be regarded as a sentence. The embedding layer is used to embed the vector data and convert the discrete vector data into a continuous vector space, so as to facilitate the model to learn and understand the vector data.

[0081] Step S403: Based on the embedding layer, the absolute position, relative position, direction and type of the point vector in the vector data are respectively embedded and concatenated to obtain point-level embedding features.

[0082] Among them, the point-level embedding feature is used to describe the embedding result of the point vector. The point vector is a vectorized point element. In the vector data, the data of the point vector includes the position, direction and type of the point vector. The position of the point vector can be divided into absolute position and relative position. The relative position is used to describe the position of the point vector in the segment vector, such as the number, and the absolute position can be the coordinates of the point vector. The type of the point vector can be represented by the type of the segment vector where the point vector is located, such as lane lines, zebra crossings, safety islands, traffic signs, etc.

[0083] For point vectors in vector data, the embedding layer can be used to embed the absolute position, relative position, direction and type of the point vector respectively, and the embedding results of the absolute position, relative position, direction and type of the point vector can be spliced ​​to obtain the point-level embedding features of the point vector.

[0084] The embedding layer may include multiple embedding modules, and different embedding modules are used to embed different items of point vector data to obtain corresponding embedding results, such as absolute position embedding, relative position embedding, direction embedding and type embedding, and then concatenate the absolute position embedding, relative position embedding, direction embedding and type embedding of the same point vector to obtain the point-level embedding features of the point vector.

[0085] Step S404, concatenating the point-level embedding features of the plurality of point vectors included in the segment vector in the vector data, and adding a preset beginning mark to obtain the segment-level embedding features of the segment vector.

[0086] The preset beginning mark is used to distinguish the embedding results of the point vector and the segment vector.

[0087] In some embodiments, the segment vector may also be referred to as an instance, and the segment-level embedding feature may be referred to as an instance-level embedding feature.

[0088] In order to obtain the embedding result of the segment vector in the vector data, that is, the segment-level embedding feature, the point-level embedding features of each point vector contained in the segment vector can be spliced, and a special mark, that is, a preset beginning mark such as "[VEC]", is added to the beginning of the vector obtained after splicing to obtain the segment-level embedding feature of the segment vector.

[0089] Exemplarily, for a segment vector including three point vectors, the point-level embedding features of the three point vectors are E1, E2 and E3 respectively, then the segment-level embedding features of the segment vector can be expressed as: {[VEC]E1 E2 E3}.

[0090] Step S405 , concatenating the segment-level embedded features of each segment vector in the vector data to obtain an embedded vector feature.

[0091] Step S406, for the segment-level embedded features in the embedded vector features, based on the intra-vector attention encoder, use the attention mechanism to determine the intra-vector attention score between the point-level embedded features within the segment-level embedded features, and based on the intra-vector attention score and the intra-vector attention mask, obtain the intra-vector encoding feature of the segment-level embedded features.

[0092] Step S407, based on the inter-vector attention encoder, using the attention mechanism, determines the inter-vector attention scores between the intra-vector encoding features of different segment-level embedded features, and obtains the prior map vector features based on the inter-vector attention scores and the inter-vector attention masks.

[0093] In order to further enhance the perception ability of point vectors to other point vectors, an attention encoding layer with a dual encoding mechanism is designed, namely, the intra-vector attention encoder and the inter-vector attention encoder, to realize the feature interaction within the segment vector and between vectors. The attention mask mechanism is used to promote the feature interaction between different segment vectors within the segment vector, so that vector data can be deeply understood and represented, providing strong support for the construction of local vector maps.

[0094] Attention mask, also known as mask tensor, is a technique used to control the encoder to ignore invalid features when processing the input sequence. The attention mask is a matrix composed of 0 and 1, with the same length as the input sequence. 0 means that the feature at the corresponding position is invalid, and 1 means that the feature at the corresponding position is valid. The feature at the position corresponding to 0 includes a portion filled to ensure length consistency. By using the attention mask, the model can better capture the valid information in the input sequence.

[0095] Both the intra-vector attention mask and the inter-vector attention mask are attention masks. The intra-vector attention mask is the attention mask used when features within the segment vector interact, while the inter-vector attention mask is the attention mask used when features between different segment vectors interact.

[0096] For the segment-level embedding features of the same segment vector, the intra-vector attention encoder obtains the query vector based on the point-level embedding features of one of the point vectors in the segment vector through the attention mechanism, and obtains the key vector and value vector based on the point-level embedding features of other point vectors in the segment vector; based on the similarity between the query vector and the key vector, the attention score of the point vector relative to other point vectors is obtained. In order to distinguish, the attention score here is recorded as the intra-vector attention score; at the same time, the intra-vector attention mask is applied to calculate the attention weight, which is converted through the output layer of the intra-vector attention encoder to obtain the intra-vector encoding feature. When calculating the attention weight, the attention weight at the position corresponding to 0 in the attention mask is set to a very small value, so as to ignore the influence of this position and focus on the information of the effective position.

[0097] The inter-vector attention encoder is used to interact features between segment vectors, that is, the interaction between the intra-vector encoded features of different segment vectors. The query vector is obtained based on the intra-vector encoded features of one of the segment vectors, and the key vector and value vector are obtained based on the intra-vector encoded features of other segment vectors. Based on the similarity between the query vector and the key vector, the attention score of the segment vector relative to other segment vectors is obtained. In order to distinguish, the attention score here is recorded as the inter-vector attention score. At the same time, the inter-vector attention mask is applied to calculate the attention weight, which is converted through the output layer of the inter-vector attention encoder to obtain the prior map vector feature.

[0098] Exemplarily, the output layer of the intra-vector attention encoder and the inter-vector attention encoder is a fully connected neural network (FFN).

[0099] Figure 5 A schematic diagram of the structure of a vector encoder provided in an embodiment of the present application is shown in FIG. Figure 5As shown, the vector encoder includes an embedding layer, an intra-vector attention encoder, and an inter-vector attention encoder.

[0100] The vector data Mprior of the prior map includes data of multiple segment vectors such as Vector1, Vector2, etc. The data of each segment vector is composed of data of multiple point vectors such as Point1, Point2, Point3 contained in the segment vector. The data of the point vector includes relative position (such as Point1), absolute position (such as (x1, y1)), direction (such as (vx1, vy1)) and type (such as c1). Through the embedding corresponding to the relative position, absolute position, direction and type in the embedding layer, namely relative position embedding, absolute position embedding, direction embedding and type embedding, the data of the corresponding items of the point vector are mapped into a vector of a fixed size in the continuous feature space, and then the embedding of each item of the point vector data is concatenated in units of point vectors to obtain point-level embedding features such as E1, E2, E3, etc.; the segment-level embedding features of each point vector contained in the segment vector are concatenated in units of segment vectors, and a special beginning mark such as "[VEC]" is added through segment-level embedding to obtain segment-level embedding features, such as {[VEC]1E1 E2 E3}, where the subscript in [VEC]1 is used to indicate the serial number of the corresponding segment vector; the segment-level embedding features of each segment vector are concatenated in sequence according to the order of the segment vector in the vector data to obtain the output of the embedding layer, namely the embedded vector feature.

[0101] Continue to see Figure 5 The attention encoding layer may include M layers of intra-vector attention encoders and N layers of inter-vector attention encoders. Both the intra-vector attention encoder and the inter-vector attention encoder include corresponding attention layers and a fully connected network FFN. The attention layer of the intra-vector attention encoder is the intra-vector attention layer, and the attention layer of the inter-vector attention encoder is the inter-vector attention layer.

[0102] The output of the embedding layer passes through the intra-vector attention layer, and uses the self-attention mechanism and the intra-vector attention mask to calculate the attention weight. The attention weight passes through the fully connected network to obtain the output of the intra-vector attention encoder, that is, the intra-vector encoded feature.

[0103] The output of the intra-vector attention encoder passes through the inter-vector attention layer, and the attention weight is calculated using the self-attention mechanism and the inter-vector attention mask. The attention weight passes through the fully connected network to obtain the output of the inter-vector attention encoder, which is the prior map vector feature f prior , such as the corresponding features in the prior map vector features obtained after the embedded vector feature {[VEC]1E1 E2 E3} passes through the attention encoding layer:

[0104] By describing point vectors for multi-dimensional data, including position, direction and type, the vector encoder's ability to understand and learn vector data is significantly improved. The introduction of relative position ensures the order of point vectors during embedding. The introduction of type enables the vector encoder to learn the relationship between point vectors of the same and different types. The embedding layer realizes the mapping of vector data of different lengths to a fixed size, which facilitates subsequent feature representation. By introducing a dual attention encoding mechanism, the perception of point vectors to other point vectors and segment vectors to other segment vectors is enhanced, which can more deeply understand and represent vector data, provide strong support for the construction of local vector maps, and improve the accuracy and completeness of the construction of local vector maps.

[0105] After the prior vector features of the prior map are obtained through the vector encoder, and the online map coding features are obtained by encoding the environmental data collected online by the vehicle-mounted sensors through the encoder of the mapping model, it is necessary to integrate the prior vector features into the mapping model in different ways based on the different architectures of the mapping model, so as to reconstruct the local map based on the additional information provided by the prior map.

[0106] Mapping models can be divided into two types according to whether they are based on learnable query methods. One type is a mapping model that is not based on a learnable query method, such as the HDMapNet model, and the other type is a mapping model that is based on a learnable query method, such as the MapTR model series.

[0107] The learnable query method is a method that uses a learnable query vector. The query vector is obtained by transforming the input data through the query matrix, and the calculation method is fixed; the learnable query vector is dynamically generated and is a query vector obtained through learning. It can better capture the characteristics of the input and the relationship between the input and the output.

[0108] In the decoder, the learnable query vector is specifically used to learn the statistics of the input positions.

[0109] The vector map construction framework may further include a feature fusion layer and a query update layer. The feature fusion layer is configured to be connected to a vector encoder and a mapping model that is not based on a learnable query method, and is used to fuse online map encoding features and prior map vectors to obtain fused vector features, and input the fused vector features into a decoder of the mapping model; the query update layer is configured to be connected to a vector encoder and a mapping model that is not based on a learnable query method, and is used to update the learnable query method in the mapping model decoder based on the prior map vector.

[0110] For mapping models that are not based on learnable query methods, when deploying the vector map construction framework, either the feature fusion layer or the query update layer is deployed. Specifically, for mapping models that are not based on learnable query methods, the feature fusion layer is connected to the mapping model; for mapping models based on learnable query methods, the query update layer is connected to the mapping model.

[0111] When the mapping model is updated from a mapping model not based on a learnable query method to a mapping model based on a learnable query method, the connected feature fusion layer can be disconnected, and the query update layer can be connected to the updated mapping model, so as to realize the adjustment of the vector map construction framework. The operation is simple, and it can be applied to various types of mapping models, with high flexibility and a wide range of applications.

[0112] Optionally, obtaining, by a decoder of the mapping model, a local vector map of the sensor sensing area based on the online map encoding features and the prior map vector features includes:

[0113] The online map coding features and the prior map vector features are subjected to feature fusion to obtain fused vector features; based on a decoder of the mapping model, the fused vector features are decoded to obtain a local vector map of the sensor sensing area.

[0114] For mapping models that are not based on learnable query methods, the online map encoding features and the prior map vector features can be fused first, and then the prediction results can be output through the decoder of the mapping model.

[0115] The operations involved in feature fusion include but are not limited to: matrix conversion, reshaping, upsampling, etc. The alignment of online map encoding features and prior map vector features is achieved through feature fusion.

[0116] Taking the BEV encoder as an example, the encoder of the mapping model can be mapped to the BEV space through related operations of feature fusion to obtain the BEV features of the prior map, thereby realizing the alignment of the online map encoding features and the prior map vector features.

[0117] Optionally, performing feature fusion on the online map coding feature and the priori map vector feature to obtain a fused vector feature includes:

[0118] After reshaping the prior map vector features, a deconvolution operation is performed to obtain a prior map vector feature consistent with the dimension of the online map encoding feature; the prior map vector features consistent with the dimension of the online map encoding feature are spliced ​​and aligned through a convolution operation to obtain the fused vector feature.

[0119] Reshape is an operation that changes the shape or structure of data. It can reshape a one-dimensional array into a multidimensional array or reshape a multidimensional array into a one-dimensional array.

[0120] The deconvolution operation is used to upsample the data to obtain features that are consistent with the output dimensions of the encoder of the mapping model. The convolution operation is used to align the online map encoding features with the prior map vector features.

[0121] Taking the encoder of the mapping model as the BEV encoder as an example, the above alignment operation can be expressed as:

[0122]

[0123] Among them, f bev represents the online map encoding feature; f prior Represents the prior map vector features; Represents fused vector features; reshape() is a reshaping function, Conv() is a convolution function, Deconv() is a deconvolution function, and concat() is a concatenation (or connection) function.

[0124] After obtaining the fused vector features, the fused vector features are decoded by a decoder to obtain a prediction result.

[0125] Optionally, obtaining, by a decoder of the mapping model, a local vector map of the sensor sensing area based on the online map encoding features and the prior map vector features includes:

[0126] A feature fusion operation is performed on the features of the point vector and the segment vector at the corresponding position in the learnable query vector in the decoder and the prior map vector features, respectively, to obtain an updated learnable query vector; and a local vector map of the sensor perception area is obtained through the decoder using an attention mechanism based on the online map encoding features and the updated learnable query vector.

[0127] The feature fusion operation performed on the learnable query vector and the features of the vector at the corresponding position in the prior map vector features can be a replace operation, an addition operation, or a concatenation operation. The learnable query vector includes two parts, namely, a point-level query vector and a segment-level query vector. The point-level query vector is the query vector of the point vector, and the segment-level query vector is the query vector of the segment vector.

[0128] For the mapping model based on learnable queries, a learnable query vector is stored in its decoder. Before performing operations based on the attention mechanism, the learnable query vector stored in the decoder needs to be updated based on the prior map vector features.

[0129] Taking the feature fusion operation as a replacement operation as an example, for the features of point vectors and segment vectors, the features of the point vector and segment vector at a certain position in the learnable query vector can be replaced by the features of the point vector and segment vector at that position in the prior map vector features, respectively.

[0130] Taking the feature fusion operation as an addition operation as an example, for the features of point vectors and segment vectors, the features of the point vector and segment vector at a certain position in the learnable query vector can be added to the features of the point vector and segment vector at that position in the prior map vector features to achieve the update of the learnable query vector.

[0131] Taking the feature fusion operation as a splicing operation as an example, for the features of point vectors and segment vectors, the features of the point vector and segment vector at a certain position in the learnable query vector can be spliced ​​with the features of the point vector and segment vector at that position in the prior map vector features to achieve the update of the learnable query vector.

[0132] After updating the learnable query vector using the prior map vector features, the decoder of the mapping model uses the self-attention mechanism through each attention layer to predict the output sequence of the online map encoding features in parallel based on the updated learnable query vector, and obtains the coordinates and category labels of the prediction box through the feedforward network to realize the prediction of vectorized map elements and obtain a local vector map.

[0133] For example, Figure 6 A structural diagram of another vector map construction framework provided in an embodiment of the present application is shown in FIG. Figure 6 As shown, the vector map construction framework includes: online mapping model, vector encoder, feature fusion layer and query update layer.

[0134] The online mapping model includes a BEV encoder and a map decoder. The BEV encoder is used to encode the environmental data collected by the vehicle-mounted sensors to obtain BEV features; the vector encoder is used to encode the prior map, specifically to encode multiple target prior maps to obtain prior map vector features.

[0135] When the online mapping model is a method without a learnable query, that is, an online mapping model that is not based on a learnable query, the feature fusion layer is used to reshape and deconvulate the prior map vector features, and the prior map vector features are mapped to the BEV space. Then, the prior map vector features mapped to the BEV space and the BEV features output by the BEV encoder are concatenated and convolved to achieve alignment of the prior map vector features and the online map encoding features, namely, the BEV features, to obtain fused vector features; the fused vector features are input into the map decoder to obtain the prediction result of the local vector map.

[0136] When the online mapping model is based on a learnable query method, the prior map vector features and the learnable query vector stored in the map decoder can be integrated through any of the operations of addition, replacement, and concatenation to achieve the update of the learnable query vector. The features in the learnable query vector and the prior map vector features can be divided into two categories, namely, point-level features and segment-level features. The point-level features at the same position can be concatenated, and the segment-level features at the same position can be concatenated to obtain the updated learnable query vector. Then the map decoder uses the updated learnable query vector and the BEV features output by the BEV encoder to output the prediction result of the local vector map.

[0137] For different types of mapping models, different methods are used to integrate the prior map feature vectors into the mapping models, so that different types of mapping models can use the additional information provided by the prior map to construct local vector maps. It has a wide range of applications, and when the mapping model is updated, the framework can be updated by switching the connected layers. The operation is simple and there is no need to adjust the internal structure of the mapping model.

[0138] In each step of the method provided in the foregoing embodiments of the present application, under the premise of being logical, some serial steps may also be executed in parallel to improve efficiency, such as step S201 and step S202; some parallel steps may also be executed serially, which is not limited in the present application.

[0139] The vector encoder provided in the aforementioned embodiment of the present application may be pre-trained before training. During pre-training, in order to simulate errors in the prior map, random noise or masks may be added to the prior map samples to obtain a large number of samples for pre-training, thereby improving the vector encoder's ability to learn vector data.

[0140] Optionally, the pre-training samples of the vector encoder during pre-training include random noise samples and mask samples; the random noise samples are obtained by adding random noise to a first percentage of point vectors in the prior map samples, and adding random noise to each point vector included in a second percentage of segment vectors in the prior map samples; the mask samples are obtained by adding masks to a third percentage of point vectors in the prior map samples, and adding masks to a fourth percentage of segment vectors in the prior map samples.

[0141] In some embodiments, the second percentage is higher than the first percentage, and the fourth percentage is higher than the third percentage.

[0142] For example, the first percentage and the third percentage may be 5%, and the second percentage and the fourth percentage may be 10%.

[0143] For the acquired prior map sample, 5% of the vector points are randomly selected from the prior map sample to add random noise, and the random noise can be added only to the coordinates of the vector points. Also, 10% of the segment vectors are randomly selected from the prior map sample, and random noise is added to all the point vectors contained in the selected segment vectors to obtain a random noise sample.

[0144] For the prior map sample taken, 5% of the vector points are randomly selected from the prior map sample to add a mask. The coordinates of the vector points can be masked by the mask, such as modifying their coordinates to -1, and 10% of the segment vectors are randomly selected from the prior map sample, and masks are added to all point vectors contained in the selected segment vectors to obtain a mask sample.

[0145] During the pre-training process, an MLP (Multilayer Perceptron) layer can be connected after the vector encoder. The MLP layer is used to simulate the function of the decoder of the mapping model to predict the coordinates of each vector point in the input pre-training sample. The distance between the predicted coordinates of the vector point and its true coordinates is used as supervision to adjust the parameters of the vector encoder.

[0146] For example, Figure 7 A schematic diagram of the vector encoder pre-training process provided in the embodiment of the present application, such as Figure 7As shown in the figure, after obtaining the original prior map sample, the noise generator and the mask generator are used to add random noise and mask to the prior map sample to obtain the corresponding pre-training sample, and the predicted or reconstructed local vector map is obtained through the vector encoder and the MLP layer. The distance between the coordinates of each vector point in the reconstructed local vector map and the coordinates of each vector point in the original prior map sample, that is, the prior map sample without adding random noise and mask is used as supervision, the loss value is calculated through the loss function, and the parameters of the vector encoder are adjusted based on the loss value until the pre-training end condition is met.

[0147] By way of example, the random noise may be Gaussian noise.

[0148] In addition to adding random noise and masks, other sample enhancement methods can be used to increase the number of pre-training samples, such as performing geometric transformations on segment vectors, such as translation and rotation.

[0149] By adding random noise and masks to the prior map samples, the pre-training samples are enhanced, the diversity and quantity of the dataset are increased, and the encoding ability and robustness of the vector encoder are improved.

[0150] For the mapping model, the environmental data collected by the on-board sensors during the historical vehicle autonomous driving process can be collected and stored as pre-training samples for the mapping model. The mapping model is pre-trained, and the prediction results output by the decoder of the mapping model are compared with the true values ​​of the local vector map corresponding to the pre-training samples. The loss value is calculated through the loss function, and the back propagation of the loss value is used to adjust the parameters of the mapping model to achieve pre-training of the mapping model.

[0151] After the pre-training is completed, the pre-trained vector encoder can be connected to the pre-trained mapping model to obtain a training framework, and then the vector encoder connected to the mapping model, that is, the training framework, is trained based on the training samples in the training set. The loss value is calculated based on the local vector map predicted by the decoder of the mapping model and the true value (GT, Ground Truth) of the local vector map corresponding to the training sample and the designed loss function. The back propagation of the loss value is used to adjust the parameters of the vector encoder and the mapping model until the training end conditions are met, thereby obtaining the pre-trained mapping model and the pre-trained vector encoder.

[0152] Corresponding to the aforementioned method embodiment, the present application embodiment further provides a vector map generation device, the vector map generation device comprising:

[0153] An online feature acquisition module is used to acquire environmental data collected by a sensor on a vehicle at a current position, and encode the environmental data through an encoder of a pre-trained mapping model to obtain online map coding features; a priori vector acquisition module is used to acquire vector data of at least one target priori map, and encode the vector data through a pre-trained vector encoder to obtain prior map vector features; wherein the geographical area corresponding to the target priori map overlaps with the sensor perception area, and the sensor perception area is a geographical area determined based on the field of view of the sensor and the current position; a vector map construction module is used to decode the online map coding features and the priori map vector features through a decoder of the mapping model to obtain a local vector map of the sensor perception area.

[0154] Optionally, the vector encoder includes an embedding layer and an attention encoding layer, and the prior vector acquisition module includes:

[0155] A vector embedding unit is used to embed and concatenate various data in the vector data based on the embedding layer to obtain embedded vector features; a vector encoding unit is used to encode the embedded vector features based on the attention encoding layer using an attention mechanism to obtain the prior map vector features.

[0156] Optionally, the vector embedding unit is specifically used for:

[0157] Based on the embedding layer, the absolute position, relative position, direction and type of the point vector in the vector data are embedded and spliced ​​respectively to obtain point-level embedding features; the point-level embedding features of multiple point vectors included in the segment vector in the vector data are spliced, and a preset beginning mark is added to obtain the segment-level embedding features of the segment vector; the segment-level embedding features of each segment vector in the vector data are spliced ​​to obtain the embedded vector features.

[0158] Optionally, the attention encoding layer includes an intra-vector attention encoder and an inter-vector attention encoder, and the vector encoding unit is specifically used to:

[0159] For the segment-level embedded features in the embedded vector features, based on the intra-vector attention encoder, an attention mechanism is used to determine the intra-vector attention scores between the point-level embedded features within the segment-level embedded features, and based on the intra-vector attention scores and the intra-vector attention masks, the intra-vector encoded features of the segment-level embedded features are obtained; based on the inter-vector attention encoder, an attention mechanism is used to determine the inter-vector attention scores between the intra-vector encoded features of different segment-level embedded features, and based on the inter-vector attention scores and the inter-vector attention masks, the prior map vector features are obtained.

[0160] Optionally, the vector map construction module includes:

[0161] A feature fusion unit is used to fuse the online map coding features and the prior map vector features to obtain fused vector features; a vector map construction unit is used to decode the fused vector features based on the decoder of the mapping model to obtain a local vector map of the sensor perception area.

[0162] Optionally, the feature fusion unit is specifically used for:

[0163] After reshaping the prior map vector features, a deconvolution operation is performed to obtain a prior map vector feature consistent with the dimension of the online map encoding feature; the prior map vector features consistent with the dimension of the online map encoding feature are spliced ​​and aligned through a convolution operation to obtain the fused vector feature.

[0164] Optionally, the vector map construction module is specifically used for:

[0165] A feature fusion operation is performed on the features of the point vector and the segment vector at the corresponding position in the learnable query vector in the decoder and the prior map vector features, respectively, to obtain an updated learnable query vector; wherein the feature fusion operation is a replacement operation, an addition operation or a concatenation operation; through the decoder, an attention mechanism is used to obtain a local vector map of the sensor perception area based on the online map encoding features and the updated learnable query vector.

[0166] Optionally, the local vector map generation device further includes a pre-training module for:

[0167] After the MLP layer is connected after the vector encoder, the vector encoder is pre-trained based on multiple pre-training samples. The loss function composed of the distance between the predicted position of each vector point in the predicted vector map output by the MLP layer and the actual position of the vector point at the same position in the prior map sample corresponding to the pre-training sample is used to adjust the parameters of the vector encoder until the pre-training end conditions are met.

[0168] Among them, the multiple pre-training samples include random noise samples and mask samples; the random noise samples are obtained by adding random noise to the first percentage of point vectors in the prior map samples, and adding random noise to each point vector included in the second percentage of segment vectors in the prior map samples; the mask samples are obtained by adding masks to the third percentage of point vectors in the prior map samples, and adding masks to the fourth percentage of segment vectors in the prior map samples.

[0169] Optionally, the type of the at least one target prior map includes multiple types of standard maps, high-precision maps and local vector maps.

[0170] The device for generating a local vector map provided in an embodiment of the present application can be used to execute the technical solution of the method for generating a local vector map provided in any of the above embodiments of the present application. The implementation principle and technical effect are similar and will not be described in detail in this embodiment.

[0171] Figure 8 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application. Figure 8 As shown, the electronic device of this embodiment may include: at least one processor 801; and a memory 802 communicatively connected to the at least one processor; wherein the memory 802 stores instructions executable by the at least one processor 801, and the instructions are executed by the at least one processor 801 so that the electronic device executes the method described in any of the above embodiments.

[0172] Optionally, the memory 802 may be independent or integrated with the processor 801 .

[0173] The implementation principle and technical effects of the electronic device provided in this embodiment can be found in the aforementioned embodiments and will not be described in detail here.

[0174] An embodiment of the present application further provides a computer-readable storage medium, wherein the computer-readable storage medium stores computer-executable instructions. When a processor executes the computer-executable instructions, the method described in any of the above embodiments is implemented.

[0175] An embodiment of the present application further provides a computer program product, including a computer program, which implements the method described in any of the above embodiments when executed by a processor.

[0176] In the several embodiments provided in this application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic, for example, the division of the modules is only a logical function division, and there may be other division methods in actual implementation, such as multiple modules can be combined or integrated into another system, or some features can be ignored or not executed.

[0177] The above-mentioned integrated module implemented in the form of a software function module can be stored in a computer-readable storage medium. The above-mentioned software function module is stored in a storage medium, including a number of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) or a processor to perform some steps of the method described in each embodiment of the present application.

[0178] It should be understood that the above-mentioned processor can be a processing unit (Central Processing Unit, CPU), or other general-purpose processors, digital signal processors (Digital Signal Processor, DSP), application-specific integrated circuits (Application Specific Integrated Circuit, ASIC), etc. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor, etc. The steps of the method disclosed in the application can be directly embodied as being executed by a hardware processor, or can be executed by a combination of hardware and software modules in the processor. The memory may include RAM (Random Access Memory), and may also include NVM (Non-Volatile Memory), such as at least one disk storage, and can also be a USB flash drive, a mobile hard disk, a read-only memory, a disk or an optical disk, etc.

[0179] The above storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read only memory (EEPROM), erasable programmable read only memory (EPROM), programmable read only memory (PROM), read only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk. The storage medium can be any available medium that can be accessed by a general or special-purpose computer.

[0180] An exemplary storage medium is coupled to a processor so that the processor can read information from the storage medium and write information to the storage medium. Of course, the storage medium can also be a component of the processor. The processor and the storage medium can be located in an application specific integrated circuit (ASIC). Of course, the processor and the storage medium can also exist as discrete components in an electronic device or a main control device.

[0181] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the existence of other identical elements in the process, method, article or device including the element.

[0182] The serial numbers of the above-mentioned embodiments of the present application are for description only and do not represent the advantages or disadvantages of the embodiments.

[0183] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), and includes a number of instructions for a terminal device (which can be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in each embodiment of the present application.

[0184] The above are only preferred embodiments of the present application, and are not intended to limit the patent scope of the present application. Any equivalent structure or equivalent process transformation made using the contents of the present application specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present application.

Claims

1. A method for generating a vector map, characterized in that: include: Obtaining environmental data collected by sensors on the vehicle at the current location, and encoding the environmental data through an encoder of a pre-trained mapping model to obtain online map encoding features; Obtaining vector data of at least one target prior map, and encoding the vector data through a pre-trained vector encoder to obtain a prior map vector feature; wherein the geographical area corresponding to the target prior map overlaps with the sensor perception area, and the sensor perception area is a geographical area determined based on the field of view of the sensor and the current position; The online map encoding features and the priori map vector features are decoded by the decoder of the mapping model to obtain a local vector map of the sensor sensing area.

2. The method according to claim 1, characterized in that The vector encoder includes an embedding layer and an attention encoding layer, and encodes the vector data based on a pre-trained vector encoder to obtain a priori map vector features, including: Embed and concatenate each item of data in the vector data based on the embedding layer to obtain an embedded vector feature; Based on the attention encoding layer, the embedded vector features are encoded using an attention mechanism to obtain the prior map vector features.

3. The method according to claim 2, characterized in that Embedding and concatenating each item of data in the vector data based on the embedding layer to obtain embedded vector features, including: Based on the embedding layer, the absolute position, relative position, direction and type of the point vector in the vector data are respectively embedded and concatenated to obtain point-level embedding features; splicing point-level embedding features of a plurality of point vectors included in a segment vector in the vector data, and adding a preset beginning mark to obtain a segment-level embedding feature of the segment vector; The segment-level embedded features of each segment vector in the vector data are concatenated to obtain the embedded vector feature.

4. The method according to claim 3, characterized in that The attention encoding layer includes an intra-vector attention encoder and an inter-vector attention encoder; Based on the attention encoding layer, the embedded vector features are encoded using the attention mechanism to obtain the prior map vector features, including: For the segment-level embedded features in the embedded vector features, based on the intra-vector attention encoder, using an attention mechanism, determine the intra-vector attention scores between the point-level embedded features in the segment-level embedded features, and obtain the intra-vector encoding features of the segment-level embedded features based on the intra-vector attention scores and the intra-vector attention masks; Based on the inter-vector attention encoder, an attention mechanism is used to determine the inter-vector attention scores between the intra-vector encoded features of different segment-level embedded features, and based on the inter-vector attention scores and the inter-vector attention masks, the prior map vector features are obtained.

5. The method according to any one of claims 1 to 4, characterized in that: The online map encoding feature and the prior map vector feature are decoded by the decoder of the mapping model to obtain a local vector map of the sensor sensing area, including: Performing feature fusion on the online map coding feature and the priori map vector feature to obtain a fused vector feature; Based on the decoder of the mapping model, the fused vector features are decoded to obtain a local vector map of the sensor sensing area.

6. The method according to claim 5, characterized in that The online map coding feature and the prior map vector feature are subjected to feature fusion to obtain a fused vector feature, including: After reshaping the prior map vector features, a deconvolution operation is performed to obtain a prior map vector feature consistent with the dimension of the online map encoding feature; The prior map vector features having the same dimension as the online map encoding features are concatenated and aligned through a convolution operation to obtain the fused vector features.

7. The method according to any one of claims 1 to 4, characterized in that: The online map encoding feature and the prior map vector feature are decoded by the decoder of the mapping model to obtain a local vector map of the sensor sensing area, including: Performing feature fusion operations on the learnable query vector in the decoder and the features of the point vector and the segment vector at the corresponding position in the prior map vector feature, respectively, to obtain an updated learnable query vector; wherein the feature fusion operation is a replacement operation, an addition operation or a concatenation operation; The decoder uses an attention mechanism to obtain a local vector map of the sensor perception area based on the online map encoding features and the updated learnable query vector.

8. The method according to any one of claims 1 to 4, characterized in that: The type of the at least one target prior map includes multiple types of standard maps, high-precision maps and local vector maps.

9. A device for generating a vector map, characterized in that: include: An online feature acquisition module is used to obtain environmental data collected by sensors on the vehicle at the current position, and encode the environmental data through an encoder of a pre-trained mapping model to obtain online map encoding features; A priori vector acquisition module, used to obtain vector data of at least one target priori map, and encode the vector data through a pre-trained vector encoder to obtain a priori map vector feature; wherein the geographical area corresponding to the target priori map overlaps with the sensor perception area, and the sensor perception area is a geographical area determined based on the field of view of the sensor and the current position; The vector map construction module is used to decode the online map encoding features and the prior map vector features through the decoder of the mapping model to obtain a local vector map of the sensor perception area.

10. An electronic device, characterized in that: include: at least one processor; as well as a memory communicatively coupled to the at least one processor; The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the electronic device to perform the method according to any one of claims 1 to 8.

11. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-executable instructions, and when the processor executes the computer-executable instructions, the method according to any one of claims 1 to 8 is implemented.

Images