Vector map construction method and system based on prior map and multi-stage diffusion reasoning

Abstract

The application provides a vector map construction method and system based on prior map and multi-stage diffusion reasoning. A unified vector encoder is used to encode multiple prior maps to generate prior features. Real-time data obtained is converted into bird's-eye view features, and multi-stage diffusion reasoning is performed to map the bird's-eye view features of the degradation domain to diffusion features of the normal domain. The prior features and the diffusion features are fused to obtain fused features. The fused features are input into a constructed map decoder to generate an online vector map. In the scheme, multiple prior maps are introduced, and a unified vector encoder is used to efficiently encode different prior maps, thereby enhancing the robustness and accuracy of online map construction. Through multi-stage diffusion reasoning technology, image details are gradually optimized, which not only uniformly processes multiple image degradation problems, but also provides high-quality image restoration and enhancement effect in complex environments, thereby providing strong support for an automatic driving perception system.

Description

Technical Field

[0001] This application relates to the field of autonomous driving technology, and more specifically, to a vector map construction method and system based on prior maps and multi-stage diffusion reasoning. Background Technology

[0002] The field of autonomous driving perception relies heavily on high-precision maps and real-time environmental perception technologies to ensure vehicles can navigate safely and accurately in complex environments. However, existing online map building technologies have significant limitations when dealing with complex environments (such as camera occlusion and inclement weather). In particular, in the construction and updating of high-precision maps, traditional methods often rely on a single perception data source (such as onboard cameras or LiDAR), which is easily affected by environmental interference, resulting in incomplete map data or large errors.

[0003] Furthermore, existing image restoration and enhancement methods are usually designed for single tasks (such as denoising, deblurring, or low-light enhancement), making it difficult to uniformly address multiple degradation problems and limiting their application in complex scenes. Summary of the Invention

[0004] The purpose of this application is to provide a vector map construction method and system based on prior maps and multi-stage diffusion inference, so as to improve image restoration and enhancement effects and improve the robustness and accuracy of online map construction.

[0005] In a first aspect, the present invention provides a vector map construction method based on prior maps and multi-stage diffusion reasoning, the method comprising:

[0006] Obtain multiple pre-stored prior maps, and use a unified vector encoder to encode each prior map to generate prior features;

[0007] The system obtains real-time data detected by vehicle-mounted sensors, converts the real-time data into bird's-eye view features, and performs multi-stage diffusion inference on the bird's-eye view features to map the bird's-eye view features in the degradation domain to the diffusion features in the normal domain.

[0008] The prior features and the diffusion features are fused to obtain the fused features;

[0009] The fused features are input into the constructed map decoder to generate an online vector map.

[0010] In an optional implementation, the prior map includes multiple data vectors, each of which consists of multiple points;

[0011] The step of encoding each of the prior maps using a unified vector encoder to generate prior features includes:

[0012] The data vectors and points in the prior maps are processed to obtain various types of feature information.

[0013] Multiple types of feature information are input into the unified vector encoder, and the intra-vector attention encoder in the unified vector encoder is used to process the interaction between points inside the data vector.

[0014] The inter-vector attention encoder in the unified vector encoder is used to process the interaction between different data vectors to generate prior features.

[0015] In an optional implementation, the step of processing the data vectors and points in each of the prior maps to obtain various types of feature information includes:

[0016] The location and orientation information of each point in each data vector in the prior map are encoded to obtain the local features of each point;

[0017] Instance-level labels are set for each of the data vectors, and the instance-level labels are encoded using a multilayer perceptron to generate instance-level embedding features;

[0018] Based on the element type of each data vector, a corresponding type code is assigned, and the type code is encoded using an embedding matrix to generate map element features;

[0019] Each point is assigned a location index, and the location index is encoded using a location encoding function to generate location information features.

[0020] In an optional implementation, the step of performing multi-stage diffusion inference on the bird's-eye view features to map the bird's-eye view features of the degraded domain to the diffusion features of the normal domain includes:

[0021] The bird's-eye view features of the degradation domain are subjected to forward diffusion processing to add noise data, converting them into a noisy image;

[0022] Perform reverse reasoning on the noisy image to recover the original data, generate a preliminary image in the normal domain, and learn the degradation parameters;

[0023] Based on the preliminary image and the degradation parameters, the reverse reasoning operation is continued to generate a normal domain image of the normal domain;

[0024] The normal domain image is calibrated and optimized to generate diffusion features of the normal domain.

[0025] In an optional implementation, the step of continuing to perform reverse reasoning based on the preliminary image and the degradation parameters to generate a normal domain image of the normal domain includes:

[0026] Based on the degradation parameters, a reverse reasoning operation is performed on the preliminary image to gradually remove noise from the preliminary image, wherein the preliminary image has text prompts;

[0027] The initial image is encoded based on the constructed image encoder, and the text prompt is encoded based on the constructed text encoder;

[0028] The semantic similarity between the encoded preliminary image and the text prompt is calculated based on the constructed loss function, and the preliminary image is optimized by iterating the loss function to generate a normal domain image of the normal domain.

[0029] In an optional implementation, the step of calibrating and optimizing the normal domain image to generate diffusion features in the normal domain includes:

[0030] The information of the normal domain image is decomposed into low-frequency information and high-frequency information;

[0031] The low-frequency information is calibrated, and diffusion inference is performed on the calibrated low-frequency information based on the learned degradation prior knowledge to refine the low-frequency information.

[0032] The high-frequency information is optimized using a feature gain model to remove redundant features.

[0033] The processed low-frequency and high-frequency information is merged to obtain the diffusion characteristics of the normal domain.

[0034] In an optional implementation, the step of optimizing the high-frequency information using a feature gain model to remove redundant features from the high-frequency information includes:

[0035] The high-frequency information is used to extract features using the convolutional layer in the feature gain model to obtain shallow features;

[0036] The shallow features are extracted using residual layers and activation functions;

[0037] The shallow features and the refined features are fused together;

[0038] Convolutional layers and activation functions are used to reconstruct noise mappings on the fused features in order to obtain high-frequency information after removing redundant features.

[0039] In an optional implementation, the prior features include instance-level prior features and point-level prior features;

[0040] The step of fusing the prior features and the diffusion features to obtain the fused features includes:

[0041] The diffusion features are converted into instance-level diffusion features and point-level diffusion features;

[0042] Addition, substitution, and connection operations are performed on the instance-level prior features and instance-level diffusion features, as well as the point-level prior features and point-level diffusion features, respectively, to obtain the fused features.

[0043] In an optional implementation, the step of generating an online vector map by inputting the fused features into the constructed map decoder includes:

[0044] The fused features are input into the constructed map decoder to generate vectorized map elements;

[0045] An online vector map is generated based on the point set, direction, and type contained in each of the map elements.

[0046] Secondly, the present invention provides a vector map construction system based on prior maps and multi-stage diffusion reasoning, the system comprising:

[0047] The encoding module is used to obtain multiple pre-stored prior maps, and to encode each of the prior maps using a unified vector encoder to generate prior features;

[0048] The inference module is used to obtain real-time data detected by vehicle-mounted sensors, convert the real-time data into bird's-eye view features, and perform multi-stage diffusion inference on the bird's-eye view features to map the bird's-eye view features in the degradation domain to the diffusion features in the normal domain.

[0049] A fusion module is used to fuse the prior features and the diffusion features to obtain fused features;

[0050] The generation module is used to input the fused features into the constructed map decoder to generate an online vector map.

[0051] The vector map construction method and system based on prior maps and multi-stage diffusion inference provided in this application significantly enhance the robustness and accuracy of online map construction by introducing multiple prior maps and using a unified vector encoder to efficiently encode different prior map data. In image processing, the multi-stage diffusion inference technique progressively optimizes image details, enabling unified handling of various image degradation problems and providing high-quality image restoration and enhancement effects in complex environments, thus providing strong support for autonomous driving perception systems. Attached Figure Description

[0052] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings used in the embodiments of this application will be briefly introduced below. It should be understood that the following drawings only show some embodiments of this application and should not be regarded as a limitation of the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0053] Figure 1 A flowchart illustrating the vector map construction method based on prior maps and multi-stage diffusion reasoning provided in this application embodiment;

[0054] Figure 2 for Figure 1 A flowchart of the sub-steps included in S11;

[0055] Figure 3 for Figure 1 A flowchart of the sub-steps included in S12;

[0056] Figure 4 for Figure 1 A flowchart of the sub-steps included in S13;

[0057] Figure 5 for Figure 1 A flowchart of the sub-steps included in S14;

[0058] Figure 6 A functional block diagram of a vector map construction system based on prior maps and multi-stage diffusion reasoning provided in the embodiments of this application;

[0059] Figure 7 A structural block diagram of an electronic device provided in an embodiment of this application. Detailed Implementation

[0060] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0061] To facilitate understanding of the embodiments of this application, the following will provide further explanation and description with reference to the accompanying drawings and specific embodiments. These embodiments do not constitute a limitation on the embodiments of this application.

[0062] Please see Figure 1This document presents a flowchart of a vector map construction method based on prior maps and multi-stage diffusion reasoning, provided by an embodiment of the present invention. This vector map construction method can be executed by a vector map construction system, which can be implemented by software and / or hardware and can be configured in an electronic device, such as a computer device, server, or, for example, a server in a back-end control platform. The detailed steps of this vector map construction method are described below.

[0063] S11: Obtain multiple pre-stored prior maps, and use a unified vector encoder to encode each prior map to generate prior features.

[0064] S12, obtain real-time data detected by vehicle-mounted sensors, convert the real-time data into bird's-eye view features, and perform multi-stage diffusion inference on the bird's-eye view features to map the bird's-eye view features of the degradation domain to the diffusion features of the normal domain.

[0065] S13, perform fusion processing on the prior features and the diffusion features to obtain fused features.

[0066] S14, the fused features are input into the constructed map decoder to generate an online vector map.

[0067] The vector map construction method provided in this embodiment is applied in autonomous driving scenarios and can be used to generate online vector maps for autonomous driving control guidance.

[0068] The pre-stored prior maps can be obtained from open prior knowledge bases. Prior maps may include standard definition maps, existing but outdated high-precision maps, and historical prediction maps. Standard definition maps provide road framework information, while high-precision maps offer high accuracy but have a lower update frequency. Historical prediction maps reflect the latest observation data when vehicles repeatedly pass through the same location.

[0069] In this embodiment, by combining the three types of prior maps mentioned above and through the mutual supplementation between different prior maps, key road information such as road framework, lane lines, and intersections can be provided more completely.

[0070] A unified vector encoder is used to encode each prior map, including multi-feature representation of vector data (lane lines, intersections, etc.) in the prior maps, processing of internal information of vector data, and processing of correlations between vector data. Finally, prior features are obtained based on the prior maps. These prior features represent static, existing map information.

[0071] In addition, the vehicle is equipped with multiple onboard sensors, including cameras and LiDAR. During vehicle operation, these sensors detect real-time data, including multi-view image data and LiDAR point cloud data. To reflect the global state of the vehicle's environment, the real-time data can be converted into bird's-eye view features.

[0072] Real-time data detected by vehicle sensors may be obtained under conditions such as camera obstruction or inclement weather. This type of real-time data is of poor quality and can be referred to as degraded-domain real-time data. It is necessary to transform degraded-domain real-time data into the normal domain. Therefore, the normal domain and the degraded domain are two relative concepts; the degraded domain represents data of poor quality, while the normal domain represents data of higher quality.

[0073] In this embodiment, multi-stage diffusion inference is performed on the bird's-eye view features of the degradation domain. Through processing such as denoising, deblurring, and low-light enhancement, the image details are gradually optimized, and finally the diffusion features of the normal domain are obtained.

[0074] Based on this, the diffusion features are fused with the prior features obtained from the prior map to obtain fused features. Finally, after decoding by the map decoder, an online vector map is generated.

[0075] This solution significantly enhances the robustness and stability of online map construction by introducing multiple prior maps, especially in complex environments (such as occlusion and severe weather), effectively compensating for the shortcomings of single-sensory data and improving the accuracy and completeness of the map. Furthermore, by using a unified vector encoder to uniformly encode each prior map, it can efficiently process different prior map data, improve the model's generalization ability, and enable it to adapt to various map types and environmental changes.

[0076] Furthermore, through a multi-stage diffusion inference process, image details can be gradually optimized to ensure high-quality mapping from the degradation domain to the normal domain. This enables unified handling of various image degradation problems, including denoising, deblurring, and low-light enhancement, thereby improving the effects of image restoration and enhancement.

[0077] This solution supports real-time map updates and rapid map construction, enabling it to quickly adapt to changes in road structure and significantly improve the stability of autonomous driving perception systems in complex environments. Especially under conditions such as severe weather and obstructions, it ensures that vehicles can accurately perceive their surroundings and achieve safe navigation.

[0078] The specific implementation of each of the above steps is described below. In this embodiment, each prior map includes multiple data vectors, and each data vector consists of multiple points, such as lane lines and intersections. Please refer to [link to relevant documentation]. Figure 2The steps described above, which use a unified vector encoder to encode each prior map and generate prior features, can be implemented in the following way:

[0079] S111, the data vectors and points in the prior maps are processed to obtain various types of feature information.

[0080] S112, input various types of feature information into the unified vector encoder, and use the vector intra-vector attention encoder in the unified vector encoder to process the interaction between points inside the data vector.

[0081] S113, the inter-vector attention encoder in the unified vector encoder is used to process the interaction between different data vectors to generate prior features.

[0082] In this embodiment, when processing each prior map, the vector data in the prior map can be processed through a hybrid prior embedding and dual encoding mechanism, encoding it into a fixed-length feature representation. Specifically, the prior map is first processed using hybrid prior embedding; in this embodiment, the data vectors in the prior map and the points within the data vectors are processed. The obtained feature information includes various types, such as local features, instance-level embedding features, map element features, and location information features. Specifically, these can be obtained in the following ways:

[0083] The local features of each point in each data vector in the prior map are obtained by encoding the location and orientation information of each point. Instance-level labels are set for each data vector and the instance-level labels are encoded using a multilayer perceptron to generate instance-level embedded features. Corresponding type codes are assigned based on the element type of each data vector and the type codes are encoded using an embedding matrix to generate map element features. Location indices are assigned to each point and the location encoding function is used to encode the location indices to generate location information features.

[0084] In this embodiment, point-level embedding is performed on each point in each data vector, and the position information (x, y) and orientation information (v) of the point are encoded by a multilayer perceptron. x ,v y ), to capture the local features E of each point. point (p i This can be characterized as follows:

[0085]

[0086] Where, p i Point p i x i and y i Point p i Location information, and Point p i The direction information is represented by MLP, which indicates the processing of the multilayer perceptron, and Concat represents the concatenation function.

[0087] Since each data vector consists of multiple ordered points, points within the same data vector are grouped into the same vector instance. Each vector instance is assigned an instance-level label [VEC], which can be, for example, a number. Instance-level labels can uniformly represent the features of the entire vector instance, capturing the relationships between points within the instance vector. A multilayer perceptron is used to encode the instance-level labels, generating instance-level embedding features E. instance The characteristics are as follows:

[0088] E instance =MLP([VEC])

[0089] Furthermore, this embodiment introduces type embedding, meaning that data vectors have different element types, such as intersections and road lines. A corresponding type code t is assigned to each data vector of different element types. The form of the type code is not limited, as long as it distinguishes different element types; for example, it can be a label or a numerical value. A learnable embedding matrix is ​​then used to encode the type codes, generating map element features E to distinguish different types of map elements. type :

[0090] Et ype =Embedding(t)

[0091] Furthermore, in this embodiment, a location index pos is assigned to each point based on the location information of each point. i This position index can be indexed according to the position of the points in a certain order, thereby preserving the order information of the points. Then, the position index is encoded using the sine and cosine position encoding functions PositionalEncoding to obtain the position information feature E. pos (p i ):

[0092] E pos (p i =PositionalEncoding(pos i )

[0093] Based on the above, multiple types of feature information are concatenated together to form a hybrid prior embedding E. mixed :

[0094] E mixed =Concat(E point E instance Etype E pos )

[0095] Hybrid prior embeddings serve as input to the unified vector encoder, providing the encoder with multi-level feature representations.

[0096] The interaction processing between points and between data vectors is performed separately within the unified vector encoder.

[0097] Specifically, the unified vector encoder includes M layers of intra-vector attention encoders, which handle point-to-point interactions within the data vector. The mixed prior embedding E for the input... mixed First, it is divided into embedding representations E of each data vector. vector For each data vector, the interactions between points are calculated using the formula shown below:

[0098]

[0099] Where Q1, K1, and V1 are obtained by linear transformation from E vector The resulting query, key, and value matrix, d k It is the dimension of the key. Through a multi-layer self-attention mechanism, the ability of each point to perceive other points within the same data vector is enhanced, capturing local features within the data vector.

[0100] Furthermore, the unified vector encoder also includes an N-layer inter-vector attention encoder, which handles the interactions between different data vectors and captures global contextual information between data vectors. First, the embedding representation E of all data vectors is... vector They are then concatenated together to form the global context representation E. global Then, calculate the interaction between different data vectors using the following formula:

[0101]

[0102] Where Q2, K2, and V2 are obtained by linear transformation from E global The resulting query, key, and value matrix is ​​obtained. A multi-layered self-attention mechanism is used to capture global contextual information between data vectors.

[0103] During the dual encoding process, an attention masking mechanism is used to control attention interactions within and between data vectors, ensuring that points within a data vector only interact with other points within the same data vector, while interactions between data vectors span different vector instances.

[0104] After the above two stages, the unified vector encoder can output the prior features f. prior .

[0105] Based on this, after obtaining the real-time data detected by the vehicle-mounted sensors, the real-time data is first converted into bird's-eye view features. For example, features of the real-time data can be extracted by a perspective view feature extractor, and the extracted features are converted into bird's-eye view features.

[0106] Based on this, through multi-stage diffusion reasoning, preliminary normal domain map features are gradually generated from the degenerate domain, and then further optimized to generate refined normal domain map features.

[0107] Please see Figure 3 The step of performing multi-stage diffusion inference on bird's-eye view features to map the bird's-eye view features of the degraded domain to the diffusion features of the normal domain can be achieved in the following way:

[0108] S121, perform forward diffusion processing on the bird's-eye view features of the degradation domain to add noise data, converting it into a noisy image.

[0109] S122, Perform reverse reasoning on the noisy image to recover the original data, generate a preliminary image in the normal domain, and learn the degradation parameters.

[0110] S123, based on the preliminary image and the degradation parameters, continue to perform reverse reasoning operation to generate a normal domain image of the normal domain.

[0111] S124, perform calibration and optimization processing on the normal domain image to generate diffusion features of the normal domain.

[0112] First, in the first stage of multi-stage diffusion inference, the bird's-eye view features of the degradation domain are gradually increased with noise through a forward diffusion process, which can be represented as follows:

[0113]

[0114] Where x0 represents the bird's-eye view features of the degradation domain of the original input, x t It is corrupted, noisy data at time step t, β t It is a predefined noise variance, where ∈ represents Gaussian noise, following a standard normal distribution N(0,I), and α t =1-β t , Through a forward diffusion process, the bird's-eye view features LQ of the degradation domain are gradually transformed into a noisy image x. T .

[0115] Next, through a reverse reasoning process, the original data is recovered from the Gaussian noise, generating a preliminary image in the coarse normal domain. The reverse reasoning process can be represented as follows:

[0116]

[0117] Where, ∈ θ It is a noise predictor for the diffusion model, and noise is gradually removed by optimizing the noise predictor. μ θ It is the mean of the Gaussian distribution in backward inference, determined by the noise predictor ∈ θ Calculations show that This represents the variance of the Gaussian distribution in backward reasoning, usually a fixed value or equal to β. t related. The predicted data in the backward inference process is the data obtained by the model through the noise predictor at time step t. θ From noise data x t The estimated denoised data.

[0118] During this stage, the model learns degradation parameters and stores some prior knowledge of degradation.

[0119] Building upon this, in the second stage of multi-stage diffusion inference, the information obtained in the first stage is processed. Specifically, the step of continuing the reverse inference operation based on the preliminary image and degradation parameters to generate the normal domain image of the normal domain can be achieved in the following way:

[0120] The process continues to perform reverse reasoning on the preliminary image based on degradation parameters to gradually remove noise from the preliminary image, which contains text prompts. The preliminary image is encoded based on a constructed image encoder, and the text prompts are encoded based on a constructed text encoder. The semantic similarity between the encoded preliminary image and the text prompts is calculated based on a constructed loss function, and the preliminary image is optimized by iterating over the loss function to generate a normal domain image.

[0121] In the second stage of multi-stage diffusion inference, the initial image generated in the first stage is input. Using the degradation parameters learned in the first stage, we continue to infer backwards through the diffusion model to generate a more refined normal domain image. The reverse reasoning process can be represented as follows:

[0122]

[0123] Where y is the input condition (i.e. ), ∈ θ (x t (t,y) is a noise predictor used to predict noise and gradually remove it.

[0124] At this stage, image-related text prompts also need to be considered. Text prompts are text used to represent image representation information. These text prompts can include positive and negative examples. Positive examples are those where the image representation information matches the text prompt's representation information, while negative examples are those where the image representation information does not match the text prompt's representation information.

[0125] In this stage, a Contrastive Language-Image Pre-training (CLIP) model is used to encode the text prompts, calculating the semantic similarity between the initial image and the text prompts to ensure that the generated image is closer to a normal image. Specifically, a constructed loss function is used to measure the semantic similarity between the initial image and the text prompts; the loss function L... clip as follows:

[0126]

[0127] Among them, T p This is a positive example text suggestion, T n It is a negative example text suggestion, φ text and φ image These are a text encoder and an image encoder.

[0128] A loss function is used to measure the semantic similarity between the generated preliminary image and the text prompt. The generation of the preliminary image is guided by the loss function. Under certain conditions, such as the convergence of the loss function, a normal domain image that meets the requirements can be obtained.

[0129] In the third stage of multi-stage diffusion inference, the normal domain image generated in the second stage is calibrated and optimized to generate diffusion features of the normal domain. Specifically, this can be achieved in the following way:

[0130] The information of the normal domain image is decomposed into low-frequency and high-frequency information; the low-frequency information is calibrated, and diffusion inference is performed on the calibrated low-frequency information based on the learned degradation prior knowledge to refine the low-frequency information; a feature gain model is used to optimize the high-frequency information to remove redundant features in the high-frequency information; the processed low-frequency and high-frequency information are merged to obtain the diffusion features of the normal domain.

[0131] In this embodiment, the obtained normal domain image Discrete wavelet transform is performed to decompose the image into low-frequency information L (containing the main structural information of the image content) and high-frequency information H (containing high-frequency sub-bands in the vertical, horizontal, and diagonal directions), as represented below:

[0132]

[0133] Fine calibration of low-frequency information is performed in the wavelet low-frequency domain to remove redundant information and generate calibrated low-frequency information. In this embodiment, the low-frequency information can be further refined by performing 10 steps of diffusion reasoning based on the degenerate prior knowledge learned in the previous stages.

[0134] For high-frequency information, redundant features are further removed using a feature gain model to optimize the high-frequency information. Specifically, this step can be achieved in the following way:

[0135] The high-frequency information is used to extract features using convolutional layers in the feature gain model to obtain shallow features; the shallow features are then refined using residual layers and activation functions; the shallow features and refined features are fused; and the fused features are reconstructed using convolutional layers and activation functions to obtain high-frequency information after removing redundant features.

[0136] In this embodiment, the feature gain model includes convolutional layers, residual layers, and activation functions. First, high-frequency features are extracted using convolutional layers as shallow features. Multiple residual layers (e.g., four layers) and activation functions (e.g., ReLU activation function) are used to extract further features from the shallow features, serving as refined features. Then, residual operations in the residual layers are used to fuse the shallow and refined features, enhancing the feature retention capacity. Finally, the noise mapping is reconstructed using convolutional layers and activation functions. After removing redundant features such as noise, clean high-frequency information is generated.

[0137] Based on the above, the calibrated low-frequency information and optimized high-frequency information Merge the samples to generate high-quality normal domain diffusion features HQ:

[0138]

[0139] After obtaining the prior features and diffusion features through the above processing, please refer to... Figure 4 The step of fusing prior features and diffusion features to obtain fused features can be achieved in the following way:

[0140] S131, the diffusion features are converted into instance-level diffusion features and point-level diffusion features.

[0141] S132, perform addition, replacement and connection operations on the instance-level prior features and instance-level diffusion features, as well as the point-level prior features and point-level diffusion features, respectively, to obtain fused features.

[0142] As mentioned above, prior features include instance-level prior features and point-level prior features. To align the diffusion features with the prior features, the diffusion features first need to be converted into instance-level diffusion features and point-level diffusion features. The conversion method is similar to that of the prior features and will not be elaborated here. In this way, when facing complex processing scenarios, instance-level or point-level features can perform more fine-grained interactive processing.

[0143] Based on this, for the prior feature f prior (including instance-level prior features f) ins and point-level prior features f pt ) and diffusion feature Q (including instance-level diffusion feature q) ins and point-level diffusion characteristics q pt The fusion process involves addition, substitution, and join operations to obtain fused features, which can be characterized as follows:

[0144]

[0145] Addition operation:

[0146] Replacement operation:

[0147] Connection operation:

[0148] in, It is the fusion feature after fusion.

[0149] Experimental results show that the join operation is the most effective ensemble method, which can significantly improve the performance of the model.

[0150] After obtaining the fusion features using the methods described above, please refer to... Figure 5 The fused features are input into the constructed map decoder to generate an online vector map. This step can be achieved in the following way:

[0151] S141, The fused features are input into the constructed map decoder to generate vectorized map elements.

[0152] S142, Based on the point set, direction and type contained in each of the map elements, generate an online vector map.

[0153] In this embodiment, the map decoder can be a Transformer decoder, which inputs the fused features to directly generate vectorized map elements, such as lane lines, intersections, and lane segments. Each map element consists of a corresponding set of points, and each map element has direction information and its type information. By combining the point set, direction, and type of each map element, an online vector map can be generated.

[0154] In summary, the vector map construction scheme based on prior maps and multi-stage diffusion inference provided in this embodiment significantly enhances the robustness and accuracy of online map construction by introducing multiple prior maps and utilizing a unified vector encoder to efficiently encode different prior map data. In image processing, the multi-stage diffusion inference process progressively optimizes image details, not only uniformly handling various image degradation problems (such as denoising, deblurring, and low-light enhancement), but also providing high-quality image restoration and enhancement effects in complex environments, offering strong support for autonomous driving perception systems. This can help autonomous vehicles achieve safer and more reliable navigation and perception in complex environments, especially under conditions such as severe weather and camera obstruction, significantly improving the system's robustness and accuracy. It also provides a sound theoretical basis for navigation in adverse weather conditions.

[0155] Based on the same inventive concept, please refer to Figure 6 This invention also provides a functional module diagram of a vector map construction system based on prior maps and multi-stage diffusion reasoning. This embodiment divides the vector map construction system into functional modules according to the above method embodiments. For example, each function can be divided into its own functional modules, or two or more functions can be integrated into one processing module. The integrated modules can be implemented in hardware or as software functional modules. It should be noted that the module division in this embodiment is illustrative and only represents one logical functional division; other division methods may be used in actual implementation.

[0156] For example, when dividing functional modules according to their respective functions, Figure 6 The vector map construction system shown is only a schematic diagram of a device. The vector map construction system may include an encoding module, an inference module, a fusion module, and a generation module. The functions of each module of the vector map construction system will be described in detail below.

[0157] The encoding module is used to obtain multiple pre-stored prior maps, and to encode each of the prior maps using a unified vector encoder to generate prior features;

[0158] The inference module is used to obtain real-time data detected by vehicle-mounted sensors, convert the real-time data into bird's-eye view features, and perform multi-stage diffusion inference on the bird's-eye view features to map the bird's-eye view features in the degradation domain to the diffusion features in the normal domain.

[0159] A fusion module is used to fuse the prior features and the diffusion features to obtain fused features;

[0160] The generation module is used to input the fused features into the constructed map decoder to generate an online vector map.

[0161] The vector map construction system provided in this embodiment can be used to execute the vector map construction method under any of the above embodiments. For details not covered in this embodiment, please refer to the corresponding descriptions in the above embodiments. This embodiment will not elaborate further here.

[0162] Please see Figure 7 This is a structural block diagram of an electronic device provided in an embodiment of the present invention. The electronic device can be a computer device, server, or similar component in an autonomous driving control platform. The electronic device includes a memory, a processor, and a communication module. The memory, processor, and communication module are electrically connected directly or indirectly to each other to achieve data transmission or interaction. For example, these components can be electrically connected to each other through one or more communication buses or signal lines.

[0163] The memory is used to store computer programs or data. Memory can be, but is not limited to, Random Access Memory (RAM), Read Only Memory (ROM), Programmable Read-Only Memory (PROM), Erasable Programmable Read-Only Memory (EPROM), Electrically Erasable Programmable Read-Only Memory (EEPROM), etc.

[0164] The processor is used to read / write data or programs stored in the memory and execute the vector map construction method based on prior maps and multi-stage diffusion reasoning provided in any embodiment of the present invention.

[0165] The communication module is used to establish communication connections between electronic devices and other communication terminals via a network, and to send and receive data via the network.

[0166] It should be understood that, Figure 7 The structure shown is only a schematic diagram of an electronic device; the electronic device may also include components that are larger than those shown. Figure 7 The more or fewer components shown, or having the same Figure 7 The different configurations shown.

[0167] Furthermore, embodiments of the present invention also provide a computer-readable storage medium storing machine-executable instructions, which, when executed, implement the vector map construction method based on prior maps and multi-stage diffusion reasoning provided in the above embodiments.

[0168] Specifically, the computer-readable storage medium can be a general-purpose storage medium, such as a removable disk or hard disk. When the computer program on the computer-readable storage medium is run, it can execute the aforementioned vector map construction method based on prior maps and multi-stage diffusion inference. The processes involved in the execution of the executable instructions on the computer-readable storage medium can be referred to the relevant descriptions in the above method embodiments, and will not be detailed here.

[0169] In the embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. The apparatus embodiments described above are merely illustrative. For example, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. Furthermore, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Additionally, the displayed or discussed mutual couplings, direct couplings, or communication connections may be through some communication interfaces; indirect couplings or communication connections between devices or units may be electrical, mechanical, or other forms.

[0170] Furthermore, the units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0171] Furthermore, the functional modules in the various embodiments of this application can be integrated together to form an independent part, or each module can exist independently, or two or more modules can be integrated to form an independent part.

[0172] It should be noted that if the function is implemented as a software module and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0173] In this document, relational terms such as first and second are used only to distinguish one entity or operation from another entity or operation, without necessarily requiring or implying any such actual relationship or order between these entities or operations.

[0174] The above description is merely an embodiment of this application and is not intended to limit the scope of protection of this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of protection of this application.

Claims

1. A vector map construction method based on prior maps and multi-stage diffusion reasoning, characterized in that, The method includes: Multiple pre-stored prior maps are obtained, and various types of feature information obtained from the prior maps are input into a unified vector encoder. The intra-vector attention encoder in the unified vector encoder processes the interaction between points within the data vectors included in the prior maps, and the inter-vector attention encoder in the unified vector encoder processes the interaction between different data vectors to generate prior features. The system obtains real-time data detected by vehicle-mounted sensors, converts the real-time data into bird's-eye view features, and performs multi-stage diffusion inference on the bird's-eye view features to map the bird's-eye view features in the degradation domain to the diffusion features in the normal domain. The prior features and the diffusion features are fused to obtain the fused features; The fused features are input into the constructed map decoder to generate an online vector map; The step of performing multi-stage diffusion inference on the bird's-eye view features to map the bird's-eye view features of the degraded domain to the diffusion features of the normal domain includes: The bird's-eye view features of the degradation domain are subjected to forward diffusion to add noise data, which is then converted into a noisy image. The noisy image is then subjected to backward inference to recover the original data, generate a preliminary image of the normal domain, and learn the degradation parameters. Based on the degradation parameters, a reverse reasoning operation is performed on the preliminary image to gradually remove noise from the preliminary image, wherein the preliminary image has a text prompt; the preliminary image is encoded based on a constructed image encoder, and the text prompt is encoded based on a constructed text encoder; the semantic similarity between the encoded preliminary image and the text prompt is calculated based on a constructed loss function, and the preliminary image is optimized by iterating over the loss function to generate a normal domain image of the normal domain; The information of the normal domain image is decomposed into low-frequency information and high-frequency information; the low-frequency information is calibrated, and diffusion inference is performed on the calibrated low-frequency information based on the learned degradation prior knowledge to refine the low-frequency information; the high-frequency information is optimized using a feature gain model to remove redundant features in the high-frequency information; the processed low-frequency information and high-frequency information are merged to obtain the diffusion features of the normal domain.

2. The vector map construction method based on prior maps and multi-stage diffusion reasoning according to claim 1, characterized in that, The prior map includes multiple data vectors, and each data vector is composed of multiple points; The feature information is obtained through the following methods: The data vectors and points in the prior maps are processed to obtain various types of feature information.

3. The vector map construction method based on prior maps and multi-stage diffusion reasoning according to claim 2, characterized in that, The step of processing the data vectors and points in each of the prior maps to obtain various types of feature information includes: The location and orientation information of each point in each data vector in the prior map are encoded to obtain the local features of each point; Instance-level labels are set for each of the data vectors, and the instance-level labels are encoded using a multilayer perceptron to generate instance-level embedding features; Based on the element type of each data vector, a corresponding type code is assigned, and the type code is encoded using an embedding matrix to generate map element features; Each point is assigned a location index, and the location index is encoded using a location encoding function to generate location information features.

4. The vector map construction method based on prior maps and multi-stage diffusion reasoning according to claim 1, characterized in that, The step of optimizing the high-frequency information using a feature gain model to remove redundant features from the high-frequency information includes: The high-frequency information is used to extract features using the convolutional layer in the feature gain model to obtain shallow features; The shallow features are extracted using residual layers and activation functions; The shallow features and the refined features are fused together; Convolutional layers and activation functions are used to reconstruct noise mappings on the fused features in order to obtain high-frequency information after removing redundant features.

5. The vector map construction method based on prior maps and multi-stage diffusion reasoning according to claim 1, characterized in that, The prior features include instance-level prior features and point-level prior features; The step of fusing the prior features and the diffusion features to obtain the fused features includes: The diffusion features are converted into instance-level diffusion features and point-level diffusion features; Addition, substitution, and connection operations are performed on the instance-level prior features and instance-level diffusion features, as well as the point-level prior features and point-level diffusion features, respectively, to obtain the fused features.

6. The vector map construction method based on prior maps and multi-stage diffusion reasoning according to claim 1, characterized in that, The step of inputting the fused features into the constructed map decoder to generate an online vector map includes: The fused features are input into the constructed map decoder to generate vectorized map elements; An online vector map is generated based on the point set, direction, and type contained in each of the map elements.

7. A vector map construction system based on prior maps and multi-stage diffusion reasoning, characterized in that, The system is used to implement the vector map construction method based on prior maps and multi-stage diffusion reasoning as described in any one of claims 1-6, the system comprising: The encoding module is used to obtain multiple pre-stored prior maps, and to encode each of the prior maps using a unified vector encoder to generate prior features; The inference module is used to obtain real-time data detected by vehicle-mounted sensors, convert the real-time data into bird's-eye view features, and perform multi-stage diffusion inference on the bird's-eye view features to map the bird's-eye view features in the degradation domain to the diffusion features in the normal domain. A fusion module is used to fuse the prior features and the diffusion features to obtain fused features; The generation module is used to input the fused features into the constructed map decoder to generate an online vector map.

Images