A method for constructing semantic vector maps that support beyond line-of-sight and dynamic updates
By using a panoramic image input network and multi-frame information fusion technology, combined with high-precision map prior data and diffusion denoising technology, the problem of insufficient real-time performance and accuracy of map construction in dynamic environments is solved, and efficient dynamic updating and environmental perception of beyond-line-of-sight maps are achieved.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- BEIHANG UNIV
- Filing Date
- 2025-06-18
- Publication Date
- 2026-07-17
AI Technical Summary
Existing map building methods cannot meet the requirements of real-time performance and high accuracy in dynamic environments. In particular, map accuracy decreases in complex scenarios. Furthermore, traditional methods face challenges in terms of computational resource consumption and real-time performance, making it difficult to effectively integrate real-time and offline data, resulting in untimely and incomplete map updates.
A toroidal image input network is adopted, and BEV features are extracted by combining a feature generation module and a mapping function. Multi-frame information fusion is achieved through temporal fusion and convolutional neural network. The prior data of high-precision map and navigation map are weighted and fused using a cross-attention mechanism. Combined with query propagation mechanism and diffusion denoising technology, a semantic vector map beyond line of sight is generated.
It effectively expands the map building range to beyond line of sight, improves map accuracy and integrity, enhances environmental perception, ensures the consistency of map elements in the time dimension, reduces noise interference, and generates more accurate semantic vector maps.
Smart Images
Figure CN120655776B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to a semantic vector map construction method, and more particularly to a semantic vector map construction method that supports beyond-line-of-sight and dynamic updates, belonging to the technical field of semantic vector map construction methods. Background Technology
[0002] With the continuous development of autonomous driving technology, high-precision maps are becoming increasingly important in autonomous driving systems. High-precision maps provide support for vehicle perception, localization, and path planning, and can provide detailed road geometry and semantic information. However, in complex dynamic scenarios, such as multi-target interaction, dynamic occlusion, and frequent environmental changes, traditional high-precision map construction and updating methods cannot meet the requirements of real-time performance and high accuracy.
[0003] Existing map building methods can provide relatively accurate geometric and semantic representations in static environments, but in dynamic environments, due to changes in viewpoint, dynamic targets, and occlusion issues, the feature fusion process is often insufficient, leading to a decrease in map accuracy. Especially in scenarios with long time spans, how to make up for the missing information in a single frame image and capture information about long distances and dynamic changes has become a challenge for map updates.
[0004] Furthermore, traditional methods are insufficient in terms of the speed and scope of temporal information updates when dealing with static areas, resulting in untimely map updates. To address these issues, researchers have begun to introduce high-precision maps and offline prior data for navigation maps. These data provide stable and comprehensive road information, which plays an important role in improving the accuracy and completeness of maps, especially in areas without interference from dynamic objects. However, how to effectively integrate real-time data with offline data to ensure spatiotemporal consistency remains a technical challenge.
[0005] Meanwhile, existing map update technologies face challenges in terms of computational resource consumption and real-time performance. When processing large-scale data, how to efficiently achieve map updates while ensuring real-time performance and avoiding computational delays is one of the current technical bottlenecks. To address these issues, a semantic vector map construction method that supports beyond-line-of-sight and dynamic updates is designed. Summary of the Invention
[0006] The main objective of this invention is to provide a method for constructing semantic vector maps that supports beyond-line-of-sight and dynamic updates.
[0007] The objective of this invention can be achieved by adopting the following technical solution:
[0008] A method for constructing semantic vector maps that supports beyond-line-of-sight and dynamic updates includes the following steps:
[0009] Step S1: Input the loop image into the network;
[0010] Step S2: Based on the above temporal fusion characteristics.
[0011] Step S3: Implement dynamic map updates through a query propagation mechanism to ensure the consistency of map elements over time.
[0012] Preferably, in step one, a feature generation module and a mapping function are used to extract and enhance BEV features layer by layer. Combined with temporal fusion and convolutional neural networks, multi-frame information fusion is achieved to improve the spatial and temporal integrity of environmental perception.
[0013] In step two, the prior data of high-precision map and navigation map are input into the network, and weighted fusion is performed using a cross-attention mechanism to generate a beyond-line-of-sight (BEV) feature map. Subsequently, the fused features are mapped into a semantic vector map using a decoder, and a deep neural network is used to transform the features into semantic elements of actual roads, obstacles and lane lines, thereby realizing the construction of the beyond-line-of-sight map.
[0014] Preferably, in step three, the latent variables of historical memory are combined with the features of the current frame to improve the accuracy of map updates. At the same time, diffusion denoising technology is used to remove noise. Finally, the high-resolution feature map is refined and fused to dynamically generate an accurate semantic vector map that adapts to environmental changes in real time.
[0015] Preferably, step one includes using a surround view image, which is a 360-degree panoramic view generated by stitching together images captured by multiple cameras around the vehicle.
[0016] The feature of each frame of the image is set as I. Based on this, the bird's-eye view features can be represented in the following way:
[0017]
[0018] Among them, I t This represents the panoramic image of the current frame;
[0019] n refers to the number of history frames;
[0020] f BEV This is a function used to map panoramic images to a bird's-eye view, synthesizing images from multiple perspectives;
[0021] By fusing the BEV features of the current frame with those of several previous frames, richer spatial and temporal information is obtained. This temporal fusion module performs a fusion operation on the BEV features of the current frame with those of the previous n frames, ultimately obtaining the temporally fused features.
[0022]
[0023] Among them, f temporal This refers to the temporal fusion function, which is achieved through convolutional layers and recurrent neural networks (RNNs).
[0024] Time-series fusion integrates historical and current information to compensate for missing information in distant areas and expand the scope of map construction.
[0025] Preferably, step two further includes introducing offline prior data, that is, using navigation maps and high-precision maps as prior information to further enrich the map construction process;
[0026] High-precision maps typically include more detailed road information than traditional maps;
[0027] After introducing offline prior data, a cross-attention mechanism is adopted;
[0028] Assuming the features after temporal fusion are The prior data for offline maps is The fusion process of the cross-attention mechanism can then be represented as:
[0029]
[0030] Among them, f prior This represents the cross-attention mechanism function, which gradually completes the weighted fusion of features by calculating the correlation between temporal features and prior offline map data.
[0031] The cross-attention mechanism enables the system to selectively integrate data from offline maps with temporal fusion information.
[0032] Preferably, by introducing temporal fusion technology and offline prior data, the autonomous driving system can generate a BEV feature map beyond line of sight.
[0033] The feature map contains environmental information of the current frame, as well as spatial and temporal information from historical frames, and prior data from high-precision maps and navigation maps;
[0034] During the construction of the beyond-line-of-sight map, the system will decode the fused BEV features through a deep neural network to generate a semantic vector map containing rich semantic information. The decoding process uses a fully convolutional network to map image features into semantic elements of actual roads, obstacles, lane lines, and traffic signs.
[0035] The goal of the decoding process is to generate a semantic vector map beyond the line of sight, thereby providing more accurate environmental perception for autonomous driving systems, especially in scenarios with extended line of sight.
[0036] Specifically, the decoding process can be represented by the following formula:
[0037]
[0038] Here, g() represents the decoding function, which maps the fused features into a vector map with semantic annotations.
[0039] Preferably, step S3 includes assuming that a map element has been successfully identified and tracked in the previous frame, and using a query propagation mechanism, the location information of this element is transmitted to the current frame. We rely on calculating the spatial correlation between the current frame and the previous frame to obtain the updated position of the corresponding element in the current frame. This process can be represented by the following formula:
[0040] E t =f query (E t-1 ,I t );
[0041] Among them, E t It is a map element in the current frame, E t-1 It is a map element from the previous frame, I t It is the image of the current frame, f query It is a function for query propagation mechanism;
[0042] The concept of latent variables in historical memory was introduced;
[0043] Historical memory latent variables are latent feature information accumulated in historical frames. They can help the model better understand the map elements in the current frame, integrate historical information with the map elements in the current frame, make up for the key information that may be missing in the current frame, and improve the accuracy of map updates.
[0044] Specifically, this involves using historical memory latent variables to fuse historical information with the characteristics of current map elements. The latent variable for the historical frame is set as H. t-1 The map element in the current frame is E. t The process of integrating historical information with current map elements can then be represented as:
[0045] E fusion =f fusion (E t H t-1 );
[0046] Where f fusion As a fusion function, it combines the map elements of the current frame with historical latent variables to obtain the fused map element features E. fusion ;
[0047] This fusion approach allows for a deeper understanding of map elements, improving the accuracy of map updates in dynamically changing environments.
[0048] In the map update process, the first step in the diffusion denoising process is to initialize the semantic vector map with noise, and then remove the noise step by step through the diffusion process.
[0049] Specifically, the diffusion process can be defined by the following formula:
[0050]
[0051] Where, ε noisy This is a preliminary noise map;
[0052] τ is the time step of the diffusion process;
[0053] It is a diffusion operation that removes noise through a step-by-step iterative process, ultimately obtaining a denoised semantic map E. denosied ;
[0054] After noise removal, the feature information at different levels is refined and fused to improve the spatial resolution and detail of the map. Refinement is achieved by increasing the resolution of the feature map, making the details in the map clearer. The fusion process combines feature information at multiple levels to improve the accuracy of the map and more effectively capture the details of roads and obstacles in complex environments.
[0055] Specifically, the refinement and fusion process can be represented by the following formula:
[0056] E refined =f refine (E denosied ,F high-res );
[0057] Among them, E denosied This refers to the map obtained after noise reduction processing;
[0058] F high-res It is a feature map with high resolution characteristics;
[0059] f refine This is for refining the function.
[0060] Beneficial technical effects of the present invention:
[0061] This invention provides a semantic vector map construction method that supports beyond-line-of-sight and dynamic updates. It integrates the bird's-eye view features of the current frame and historical frames through temporal fusion technology, and uses convolutional layers and recurrent neural networks (RNNs) to model temporal dimension information, effectively making up for the lack of long-distance information in single-frame images, and extending the map construction range from the near-range area of traditional methods to the beyond-line-of-sight range.
[0062] By introducing prior data from high-precision maps and navigation maps, and using a cross-attention mechanism to weightedly fuse temporal features and offline data, the map integrity of static areas is further enhanced. For example, within a 120m × 60m mapping area, the spatial consistency of map features is improved by 12.27% after incorporating prior data. Attached Figure Description
[0063] Figure 1 This is a flowchart of a semantic vector map construction method supporting beyond-line-of-sight and dynamic updates, provided by an embodiment of the present invention;
[0064] Figure 2 This is a schematic diagram illustrating the model effect of a semantic vector map construction method supporting beyond-line-of-sight and dynamic updates provided by an embodiment of the present invention. Detailed Implementation
[0065] To enable those skilled in the art to understand the technical solution of the present invention more clearly, the present invention will be further described in detail below with reference to the embodiments and accompanying drawings, but the embodiments of the present invention are not limited thereto.
[0066] The method includes:
[0067] Step S1: Input the panoramic image into the network, use the feature generation module and mapping function to extract and enhance BEV features layer by layer, combine temporal fusion and convolutional neural network (RNN) to achieve multi-frame information fusion and improve the spatial and temporal integrity of environmental perception.
[0068] Step S2: Based on the aforementioned temporal fusion features, the prior data from the high-precision map and navigation map are input into the network. A weighted fusion is performed using a cross-attention mechanism to generate a beyond-line-of-sight (BEV) feature map. Subsequently, a decoder maps the fused features into a semantic vector map. A deep neural network is then used to transform the features into semantic elements representing actual roads, obstacles, and lane lines, thus achieving beyond-line-of-sight map construction.
[0069] Step S3: Dynamic map updates are achieved through a query propagation mechanism to ensure consistency of map elements over time. The accuracy of map updates is improved by combining historical memory latent variables with current frame features. Simultaneously, noise is removed using diffusion denoising techniques, and finally, high-resolution feature maps are refined and fused to dynamically generate accurate semantic vector maps that adapt to environmental changes in real time.
[0070] Step S1 includes,
[0071] In traditional autonomous driving systems, maps are typically built using front-view or surround-view images. However, single-view images often have limitations in capturing information about distant roads. In complex urban environments, surround-view images are used to generate bird's-eye view features.
[0072] A surround view image is a 360-degree panoramic view generated by stitching together images taken by multiple cameras around the vehicle. BEV features are a projection method viewed from above, which can effectively present the topology of the road and lane information. By fusing images from multiple perspectives, high-quality BEV features can be obtained.
[0073] The feature of each frame of the image is set as I. Based on this, the bird's-eye view features can be represented in the following way:
[0074]
[0075] Here, I t This represents the panoramic image of the current frame, while n refers to the number of historical frames, and f BEV This is a function used to map the panoramic image to a bird's-eye view. By synthesizing images from multiple perspectives, a more complete environmental information can be constructed from the BEV's perspective.
[0076] The introduction of temporal fusion technology aims to compensate for the lack of information in single-frame images and the loss of road information over long periods of time. This technology uses time information modeling to allow historical frame images to supplement long-distance information that is difficult to capture in the current frame, effectively expanding the line-of-sight range of map construction.
[0077] The core idea behind the temporal fusion module is to obtain richer spatial and temporal information by fusing the BEV features of the current frame with those of the previous n frames. This temporal fusion module performs the fusion operation on the BEV features of the current frame with those of the previous n frames, ultimately obtaining the temporally fused features.
[0078]
[0079] Among them, f temporal This refers to the temporal fusion function, which relies on convolutional layers and recurrent neural networks (RNNs). Temporal fusion integrates historical and current information, compensating for missing information in distant areas and expanding the scope of map construction.
[0080] Step S2 includes,
[0081] While temporal fusion technology can compensate for the deficiencies of the current frame and extend the line-of-sight range by using historical frame information, in some scenarios, especially in static environments, the update speed and range of temporal information may not be sufficient to meet practical needs. For example, in areas without dynamic object interference, the vehicle's trajectory changes little, and the effect of temporal fusion may lag, or even lose key information.
[0082] To overcome the limitations of temporal fusion technology, offline prior data can be introduced, that is, navigation maps and high-precision maps can be used as prior information to further enrich the map building process. This data can provide more stable and comprehensive road information, making up for the deficiencies of temporal information in static areas.
[0083] High-precision maps typically include more detailed road information than traditional maps, such as lane widths, lane markings, intersection shapes, terrain information, traffic signs, and traffic lights. High-precision maps are updated less frequently, but the information is very stable and accurate, making them ideal for navigation and positioning in static environments. Navigation maps, on the other hand, usually include routes, traffic information, navigation instructions, and dynamic information on route changes. Navigation maps are updated more frequently and are less expensive.
[0084] After introducing offline prior data, a key issue becomes how to effectively combine it with temporal fusion information. To achieve this goal, a cross-attention mechanism can be used. This mechanism can calculate the correlation between temporal features and offline map prior data, and selectively retain valuable information through weighted fusion.
[0085] The core idea of the cross-attention mechanism is to determine which information needs to be retained based on the similarity and correlation between temporal features and prior data, and then incorporate this information into the system's current view. The introduction of this mechanism enables autonomous driving systems to better integrate real-time perception information with static information from high-precision maps and navigation maps, providing stronger support for map updates and environmental perception.
[0086] Assuming the features after temporal fusion are The prior data for offline maps is The fusion process of the cross-attention mechanism can then be represented as:
[0087]
[0088] Among them, f prior This represents the cross-attention mechanism function, which progressively completes the weighted fusion of features by calculating the correlation between temporal features and prior offline map data. The cross-attention mechanism enables the system to selectively integrate data from offline maps with temporal fusion information, thereby improving the system's environmental awareness.
[0089] Through this process, the system can not only effectively compensate for the deficiencies in temporal fusion, but also further enhance the accuracy and completeness of map construction by leveraging static data provided by high-precision maps and navigation maps. Especially in static environments, the introduction of offline prior data greatly enriches the system's understanding of the surrounding environment, avoiding the errors and lags that may result from relying solely on temporal information.
[0090] By introducing temporal fusion technology and offline prior data, the autonomous driving system can generate a BEV feature map that extends beyond line of sight. This feature map not only includes environmental information from the current frame but also incorporates spatial and temporal information from historical frames, as well as prior data from high-precision maps and navigation maps, thus possessing a wider line-of-sight range.
[0091] During the beyond-line-of-sight map construction process, the system decodes the fused BEV features using a deep neural network (DNN) to generate a semantic vector map containing rich semantic information. The decoding process uses a fully convolutional network (FCN) to map image features into semantic elements of actual roads, obstacles, lane lines, and traffic signs.
[0092] The goal of the decoding process is to generate a semantic vector map beyond the line of sight, thereby providing more accurate environmental perception for autonomous driving systems, especially in scenarios with extended line of sight.
[0093] Specifically, the decoding process can be represented by the following formula:
[0094]
[0095] Here, g() represents the decoding function, which maps the fused features into a vector map with semantic annotations.
[0096] The construction of beyond-line-of-sight maps offers significant advantages in autonomous driving. By comprehensively utilizing temporal fusion techniques and offline prior data, the system can achieve a wider line-of-sight range, enhancing its perception of the surrounding environment. Particularly when the vehicle enters a static area or moves away from dynamic objects, offline map data can effectively compensate for the lack of temporal information, ensuring the accuracy and stability of the map.
[0097] 4. The method according to claim 1, wherein step S3 comprises:
[0098] As road environments continue to change, autonomous driving systems need the ability to update semantic vector maps in real time to adapt to new traffic conditions. However, traditional map update methods often face problems such as poor time consistency and significant noise interference. To improve map update capabilities, a map update technique based on map element tracking and diffusion denoising is proposed, which can solve the aforementioned problems.
[0099] Traditional map update methods generally neglect the issue of temporal consistency. This can lead to inaccurate changes in the positions of map elements at different points in time, and even duplicate or missing map elements. To improve temporal consistency, a query propagation mechanism has been proposed. Its function is to explicitly associate tracked road elements from the previous frame with the current frame. This mechanism ensures that map elements remain consistent across time, improving the accuracy of the map update process.
[0100] To elaborate, assuming that a map element (such as lane lines or traffic signs) has been successfully identified and tracked in the previous frame, the location information of this element will be transmitted to the current frame through a query propagation mechanism. We rely on calculating the spatial correlation between the current frame and the previous frame to obtain the updated position of the corresponding element in the current frame. This process can be represented by the following formula:
[0101] E t =f query (E t-1 ,I t );
[0102] Among them, E t It is a map element in the current frame, E t-1 It is a map element from the previous frame, I t It is the image of the current frame, f query It is a function for query propagation mechanism;
[0103] To further improve the accuracy of map updates, we introduce the concept of historical memory latent variables. Historical memory latent variables are latent feature information accumulated in historical frames, which can help the model better understand the map elements in the current frame. By fusing historical information with the map elements in the current frame, we can make up for key information that may be missing in the current frame and improve the accuracy of map updates.
[0104] To elaborate, by using historical memory latent variables to fuse historical information with the characteristics of current map elements, the latent variable of the historical frame is set as H. t-1 The map element in the current frame is E. t The process of integrating historical information with current map elements can then be represented as:
[0105] E fusion =ffusion (E t H t-1 );
[0106] Where f fusion As a fusion function, it combines the map elements of the current frame with historical latent variables to obtain the fused map element features E. fusion This fusion approach allows for a deeper understanding of map elements and improves the accuracy of map updates in dynamically changing environments.
[0107] In practical applications, sensor noise and environmental interference factors can cause a lot of noise in the generated semantic vector map. This noise can affect the map accuracy, making the map inaccurate and affecting the decision-making of the autonomous driving system. To remove noise and improve the map update accuracy, diffusion denoising technology has been introduced.
[0108] Diffusion denoising, as an image processing method, is based on the diffusion process. This method uses the diffusion of simulated information in space to gradually remove noise. In the map update process, noise initialization operation is first performed on the semantic vector map, and then noise is removed step by step through the diffusion process, ultimately resulting in a clearer and more accurate map.
[0109] Specifically, the diffusion process can be defined by the following formula:
[0110]
[0111] Where, ε noisy This is a preliminary noise plot, where τ is the time step of the diffusion process. It is a diffusion operation that removes noise through a step-by-step iterative process, ultimately obtaining a denoised semantic map E. denosied This diffusion denoising technique improves the accuracy of map updates and eliminates unwanted noise interference.
[0112] After noise removal, the feature information at different levels is refined and fused to improve the spatial resolution and detail of the map. Refinement is achieved by increasing the resolution of the feature map, making the details in the map clearer. The fusion process combines feature information at multiple levels to improve the accuracy of the map. In complex environments, it can more effectively capture detailed information about roads and obstacles.
[0113] Specifically, the refinement and fusion process can be represented by the following formula:
[0114] E refined =f refine (Edenosied ,F high-res );
[0115] Here, E denosied This refers to the map obtained after noise reduction processing, F high-res Feature maps with high resolution characteristics, f refine To refine the function, it leverages the fusion of high-resolution feature information, thereby enhancing the map's detail and spatial resolution. This allows for the generation of detailed semantic vector maps, improving map accuracy and usability.
[0116] Example: The table comparing the IOU (Intersection over Union, a measure of the overlap between the predicted bounding box and the ground truth bounding box) and AP (Prediction Accuracy) values of the map construction system obtained by this patent with those of other existing examples shows that, under different mapping ranges, the IOU and AP values of this patent are higher than those of existing examples, verifying the effectiveness of the patent solution.
[0117] The formula for calculating the IOU index is as follows:
[0118]
[0119] Where A represents the prediction region.
[0120] B: The actual labeled area.
[0121] |A∩B|: The number of pixels in the intersection region of the predicted region and the ground truth labeled region; |A∪B|: The number of pixels in the union region of the predicted region and the ground truth labeled region.
[0122] The formula for calculating AP value is as follows:
[0123]
[0124] r i It is the recall value of different map elements, usually taking a series of predetermined recall values, such as [0, 0.1, 0.2, ..., 1.0].
[0125] Precision(r i ) is when the recall rate is r i Precision of time.
[0126] N is the number of map elements.
[0127] Based on the above indicators, experiments were conducted to compare the baseline method and the method of this patent, and the final experimental data are as follows:
[0128]
[0129]
[0130] The above description is merely a further embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any substitutions or modifications made by those skilled in the art within the scope disclosed in the present invention, based on the technical solution and concept of the present invention, shall fall within the scope of protection of the present invention.
Claims
1. A method for constructing semantic vector maps that supports beyond-line-of-sight and dynamic updates, characterized in that: Includes the following steps: Step S1: Input the loop image into the network; Step S2: Based on temporal fusion features; Step S3: Implement dynamic map updates through a query propagation mechanism to ensure the consistency of map elements over time; Step S3 involves assuming that a map element has been successfully identified and tracked in the previous frame. Using a query propagation mechanism, the location information of this element is transmitted to the current frame. We calculate the spatial correlation between the current and previous frames to determine the updated position of the corresponding element in the current frame. This process can be represented by the following formula: ； in, It is a map element in the current frame. It's a map element from the previous frame. It is the image of the current frame. It is a function for query propagation mechanism; The concept of latent variables in historical memory was introduced; Historical memory latent variables are latent feature information accumulated in historical frames. They can help the model better understand the map elements in the current frame, integrate historical information with the map elements in the current frame, make up for the key information that may be missing in the current frame, and improve the accuracy of map updates. Specifically, this involves using historical memory latent variables to fuse historical information with the characteristics of current map elements, and setting the latent variables of historical frames as follows: The map elements in the current frame are The process of integrating historical information with current map elements can then be represented as: ； in As a fusion function, it combines the map elements of the current frame with historical latent variables to derive the fused map element features. ; This fusion approach allows for a deeper understanding of map elements, improving the accuracy of map updates in dynamically changing environments. In the map update process, the first step in the diffusion denoising process is to initialize the semantic vector map with noise, and then remove the noise step by step through the diffusion process. Specifically, the diffusion process can be defined by the following formula: ； in, This is a preliminary noise map; It is the time step of the diffusion process; It is a diffusion operation that removes noise through a step-by-step iterative process, ultimately obtaining a denoised semantic map. ; After noise removal, the feature information at different levels is refined and fused to improve the spatial resolution and detail of the map. Refinement is achieved by increasing the resolution of the feature map, making the details in the map clearer. The fusion process combines feature information at multiple levels to improve the accuracy of the map and more effectively capture the details of roads and obstacles in complex environments. Specifically, the refinement and fusion process can be represented by the following formula: ； in, This refers to the map obtained after noise reduction processing; It is a feature map with high resolution characteristics; This is for refining the function.
2. The semantic vector map construction method supporting beyond-line-of-sight and dynamic updates according to claim 1, characterized in that: In step S1, a feature generation module and a mapping function are used to extract and enhance BEV features layer by layer. Combined with temporal fusion and convolutional neural network, multi-frame information fusion is achieved to improve the spatial and temporal integrity of environmental perception. In step S2, the prior data of high-precision map and navigation map are input into the network, and weighted fusion is performed using a cross-attention mechanism to generate a beyond-line-of-sight (BEV) feature map. Subsequently, the fused features are mapped into a semantic vector map using a decoder, and the features are transformed into semantic elements of actual roads, obstacles and lane lines using a deep neural network to realize the construction of the beyond-line-of-sight map.
3. The semantic vector map construction method supporting beyond-line-of-sight and dynamic updates according to claim 1, characterized in that: In step S3, the latent variables of historical memory are combined with the features of the current frame to improve the accuracy of map updates. At the same time, diffusion denoising technology is used to remove noise. Finally, the high-resolution feature map is refined and fused to dynamically generate an accurate semantic vector map that adapts to environmental changes in real time.
4. The semantic vector map construction method supporting beyond line-of-sight and dynamic updates according to claim 1, characterized in that: Step S1 includes using a surround view image, which is a 360-degree panoramic view generated by stitching together images captured by multiple cameras around the vehicle. The feature of each frame of the image is set as I. Based on this, the bird's-eye view features can be represented in the following way: ； in, This represents the panoramic image of the current frame; n refers to the number of history frames; This is a function used to map panoramic images to a bird's-eye view, synthesizing images from multiple perspectives; By fusing the BEV features of the current frame with those of several previous frames, richer spatial and temporal information is obtained. This temporal fusion module performs a fusion operation on the BEV features of the current frame with those of the previous n frames, ultimately obtaining the temporally fused features. : ； in, This refers to the temporal fusion function, which is achieved through convolutional layers and recurrent neural networks (RNNs). Time-series fusion integrates historical and current information to compensate for missing information in distant areas and expand the scope of map construction.
5. The semantic vector map construction method supporting beyond-line-of-sight and dynamic updates according to claim 1, characterized in that: Step S2 also includes introducing offline prior data, that is, using navigation maps and high-precision maps as prior information to further enrich the map construction process; High-precision maps include more detailed road information than traditional maps; After introducing offline prior data, a cross-attention mechanism is adopted; Assuming the features after temporal fusion are The prior data for offline maps is The fusion process of the cross-attention mechanism can then be represented as: ； in, This represents the cross-attention mechanism function, which gradually completes the weighted fusion of features by calculating the correlation between temporal features and prior offline map data. The cross-attention mechanism enables the system to selectively integrate data from offline maps with temporal fusion information.
6. The semantic vector map construction method supporting beyond-line-of-sight and dynamic updates according to claim 5, characterized in that: By introducing temporal fusion technology and offline prior data, the autonomous driving system can generate a BEV feature map beyond line of sight. The feature map contains environmental information of the current frame, as well as spatial and temporal information from historical frames, and prior data from high-precision maps and navigation maps; During the construction of the beyond-line-of-sight map, the system will decode the fused BEV features through a deep neural network to generate a semantic vector map containing rich semantic information. The decoding process uses a fully convolutional network to map image features into semantic elements of actual roads, obstacles, lane lines, and traffic signs. The goal of the decoding process is to generate a semantic vector map beyond the line of sight, thereby providing more accurate environmental perception for autonomous driving systems in scenarios with extended line of sight. Specifically, the decoding process can be represented by the following formula: ； in, Represents the decoding function. Representing prior map information, the fused features are mapped to a vector map with semantic annotations. .