A method and system for constructing a BEV semantic map
By combining two-stage training and channel attention fusion modules, and using datasets generated under normal weather conditions to create a storage pool, the problem of low accuracy of BEV semantic maps under complex weather conditions is solved, and the scene understanding ability of autonomous driving systems under conditions such as fog and snow is improved.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- SHANDONG UNIV
- Filing Date
- 2023-09-12
- Publication Date
- 2026-07-31
AI Technical Summary
Existing BEV semantic maps have low accuracy under complex weather conditions (such as fog and snow), making it difficult to effectively understand traffic scenarios and resulting in poor performance of autonomous driving systems under these conditions.
A joint two-stage training method is adopted. A storage pool is generated using the dataset under normal weather conditions to save the features of normal scenes. These features are then called for feature enhancement under complex weather conditions. Combined with the channel attention fusion module, a BEV semantic map is generated.
It improves the accuracy of BEV semantic maps under complex weather conditions and enhances the scene understanding ability of autonomous driving systems under complex weather conditions.
Smart Images
Figure CN117372825B_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the field of map construction technology, and in particular relates to a BEV semantic map construction method and system. Background Technology
[0002] The statements in this section are merely background information related to the present invention and do not necessarily constitute prior art.
[0003] Scene understanding, localization and navigation, path planning, and control execution are the four basic modules of an autonomous driving system. Among them, scene understanding is the core module and a prerequisite for autonomous driving. Scene understanding obtains scene perception data through multiple sensors, capturing the road layout in the static world and traffic participants in the dynamic world, thereby providing a rich and detailed representation of the surrounding environment. In intelligent driving systems, modules such as prediction, decision-making, and planning are all carried out in 3D space. The information used by the perception module is only the projection of the real physical world onto the image. The information obtained from the image needs to undergo complex post-processing before it can be used. However, directly converting visual information into the BEV (Bridge's Eye View) space can easily connect the perception module and the prediction and planning module.
[0004] Semantic maps built on BEVs can acquire road layouts in traffic scenarios, enabling static scene understanding; BEV semantic maps Figure 1 Generally, based on information from multiple cameras, local semantic maps are generated online from static objects such as lane lines, curbs, and sidewalks within the BEV space for use in downstream planning and control tasks. However, current research on BEV maps mainly focuses on traffic scenarios with good weather conditions, such as sunny or cloudy days, without considering traffic scenarios with adverse weather conditions such as fog or snow. Such adverse weather conditions pose challenges to autonomous driving, and data collection under these conditions is also difficult. As a result, the accuracy of BEV semantic maps constructed by existing methods is low under complex weather conditions, which does not allow autonomous driving systems to accurately understand the scene. Summary of the Invention
[0005] To overcome the shortcomings of the existing technology, this invention provides a method and system for constructing BEV semantic maps. It defines weather conditions that affect driving safety, such as fog, rain, and snow, as complex weather, and considers sunny and cloudy days as ordinary weather. Rain, fog, and snow effects are added to datasets under ordinary weather conditions to obtain complex weather data. By calling traffic scene information under ordinary weather conditions, it promotes the generation of BEV semantic maps for traffic scenes under complex weather conditions, effectively improving the accuracy of BEV semantic maps for traffic scenes under complex weather conditions and alleviating the current problem of missing BEV semantic maps for traffic scenes under complex weather conditions.
[0006] To achieve the above objectives, one or more embodiments of the present invention provide the following technical solutions:
[0007] The first aspect of this invention provides a method for constructing a BEV semantic map.
[0008] A method for constructing a BEV semantic map, comprising:
[0009] Obtain both ordinary weather scene images and complex weather scene images of the traffic scenario to be constructed;
[0010] Based on ordinary weather scene maps and complex weather scene maps, the constructed map building model is jointly trained in two stages to output a BEV semantic map.
[0011] The map construction model employs a joint two-stage training process. The first stage involves saving the ordinary scene features of the ordinary weather scene map to a storage pool. The second stage involves retrieving the stored ordinary scene features from the storage pool to enhance the complex scene features of the complex weather scene map. Based on the enhanced features, a BEV semantic map is generated.
[0012] Furthermore, the map construction model includes a feature extraction module, a feature enhancement module, a feature fusion module, a feature transformation module, and a BEV decoder connected in sequence. The specific processing steps are as follows:
[0013] Feature extraction is performed on complex weather scene images to obtain complex scene features;
[0014] By utilizing a storage pool built based on ordinary weather scene images, feature enhancement is performed on complex scene features to obtain memory features;
[0015] The memory features are fused with the features of the complex scene to obtain the fused features;
[0016] The fused features are then transformed and BEV decoded to generate a BEV semantic map.
[0017] Furthermore, the specific construction process of the storage pool is as follows:
[0018] Extracting common scene features from ordinary weather scene images;
[0019] Perform channel feature decomposition on features of ordinary scenes;
[0020] The decomposed features are stored in a storage pool using the storage parameter M. The storage parameter M is updated through backpropagation in the first stage and stops updating in the second stage.
[0021] Furthermore, the feature enhancement for complex scene features specifically includes:
[0022] (1) Perform normalization operations on the complex scene features and the storage parameters M of the storage pool respectively, multiply the normalized results of the two and input them into the softmax function to output the weight α.
[0023] (2) Use the weight α in (1) to weight the storage pool parameters, integrate the weighted local features, and obtain the memory features.
[0024] Furthermore, the fusion of memory features with complex scene features specifically involves:
[0025] First, the memory features and complex scene features are concatenated. Then, the concatenated result is input into a channel attention module consisting of an average pooling layer, a linear layer, and an activation function layer to obtain the weights of each channel. Next, the memory features are weighted using the weights of each channel. Finally, the weighted features and complex scene features are concatenated again to obtain the result of the fusion of the two features.
[0026] Furthermore, the feature transformation converts the fused features into BEV features, specifically as follows:
[0027] Using the fused features as input, a multilayer perceptron is used to convert the image view features in the pixel coordinate system into features in the camera coordinate system.
[0028] Features in the camera coordinate system are transformed into features in the BEV space using camera parameters.
[0029] Furthermore, the BEV decoding, using BEV features as input, generates a BEV semantic map for traffic scenarios under complex weather conditions, specifically as follows:
[0030] The input BEV features are processed through a convolutional layer, a batch normalization layer, and a ReLU activation function, reducing their size to half of the original.
[0031] By using ResNet18 for downsampling, the size of the feature map is reduced to 1 / 4 and 1 / 8 of the original feature map; then, the upsampling module is used to restore the feature map to the original size.
[0032] A second aspect of the present invention provides a BEV semantic map construction system.
[0033] A BEV semantic map construction system includes an acquisition module and a generation module:
[0034] The acquisition module is configured to acquire ordinary weather scene images and complex weather scene images of the traffic scenario to be constructed.
[0035] The generation module is configured to perform joint two-stage training on the constructed map building model based on ordinary weather scene maps and complex weather scene maps, and output a BEV semantic map.
[0036] The map construction model employs a joint two-stage training process. The first stage involves saving the ordinary scene features of the ordinary weather scene map to a storage pool. The second stage involves retrieving the stored ordinary scene features from the storage pool to enhance the complex scene features of the complex weather scene map. Based on the enhanced features, a BEV semantic map is generated.
[0037] A third aspect of the present invention provides a computer-readable storage medium having a program stored thereon, which, when executed by a processor, implements the steps of a BEV semantic map construction method as described in the first aspect of the present invention.
[0038] A fourth aspect of the present invention provides an electronic device including a memory, a processor, and a program stored in the memory and executable on the processor, wherein the processor executes the program to implement the steps of a BEV semantic map construction method as described in the first aspect of the present invention.
[0039] The above one or more technical solutions have the following beneficial effects:
[0040] This invention proposes a method for constructing BEV semantic maps for traffic scenarios under complex weather conditions. It designs a scheme to build a scene information storage pool by jointly training traffic scenarios in normal weather and complex weather. This realizes that the features of normal scenes can guide the generation of BEV maps for traffic scenarios under complex weather conditions, and makes up for the lack of BEV maps under complex weather conditions such as fog and snow.
[0041] This invention designs a fusion module based on channel attention, which fuses complex scene features with memory features output from the storage pool, which is beneficial for selecting important features.
[0042] Advantages of additional aspects of the invention will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention. Attached Figure Description
[0043] The accompanying drawings, which form part of this invention, are used to provide a further understanding of the invention. The illustrative embodiments of the invention and their descriptions are used to explain the invention and do not constitute an improper limitation of the invention.
[0044] Figure 1 This is a flowchart of the method in the first embodiment.
[0045] Figure 2 A model structure diagram is constructed for the map in the first embodiment.
[0046] Figure 3 This is a detailed method flowchart for the first embodiment.
[0047] Figure 4 The first embodiment provides image examples of traffic scenes under normal weather conditions and traffic scenes under complex weather conditions.
[0048] Figure 5 This is an example of a BEV semantic map in the first implementation. Detailed Implementation
[0049] It should be noted that the following detailed descriptions are illustrative and intended to provide further explanation of this application. Unless otherwise specified, all technical and scientific terms used in this invention have the same meaning as commonly understood by one of ordinary skill in the art to which this application pertains.
[0050] It should be noted that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the exemplary embodiments according to this application. As used herein, the singular form is intended to include the plural form as well, unless the context clearly indicates otherwise. Furthermore, it should be understood that when the terms "comprising" and / or "including" are used in this specification, they indicate the presence of features, steps, operations, devices, components, and / or combinations thereof.
[0051] Scene understanding is the foundation of autonomous driving, and Bird's Eye View (BEV) semantic maps, which can display road layouts, are one of the methods to achieve this foundation. Most current research focuses on traffic scenarios with good weather conditions, but some weather changes, such as fog and snow, bring problems to autonomous driving, and data collection is also more difficult in these weather conditions.
[0052] To alleviate this problem, this invention defines weather conditions that affect driving safety, such as fog, rain, and snow, as complex weather, and considers sunny and cloudy days as ordinary weather. It adds rain, fog, and snow effects to datasets under ordinary weather conditions to obtain complex weather data. Then, it proposes a method for constructing BEV semantic maps for traffic scenarios under complex weather conditions, which promotes the generation of BEV semantic maps for traffic scenarios under complex weather conditions by calling traffic scenario information under ordinary weather conditions.
[0053] First, a joint two-stage training method based on normal and complex weather traffic scenarios is proposed, and a storage pool that can save scene information under normal weather conditions is constructed. In the first stage of joint training, the loop view of the traffic scene under normal weather conditions is used as input, and the normal scene features are saved in the storage pool. In the second stage of joint training, the loop view of the traffic scene under complex weather conditions is used as input, and the normal scene features retained in the first stage are called from the storage pool using the complex scene features, thus realizing the guidance of normal scene features for the generation of BEV semantic maps in complex scenes.
[0054] Then, a feature fusion module based on channel attention was constructed. This module achieves the purpose of refining features by fusing memory features output from the storage pool and complex scene features. The fused features are converted into BEV features by a conversion module, and a BEV semantic map of traffic scene under complex weather conditions is output.
[0055] Example 1
[0056] In one or more embodiments, a BEV semantic map construction method is disclosed, such as Figure 1 As shown, it includes:
[0057] Step S1: Obtain the normal weather scene map and the complex weather scene map of the traffic scene to be constructed;
[0058] Step S2: Based on ordinary weather scene maps and complex weather scene maps, perform joint two-stage training on the constructed map building model to output a BEV semantic map;
[0059] The map construction model employs a joint two-stage training process. The first stage involves saving the ordinary scene features of the ordinary weather scene map to a storage pool. The second stage involves retrieving the stored ordinary scene features from the storage pool to enhance the complex scene features of the complex weather scene map. Based on the enhanced features, a BEV semantic map is generated.
[0060] The implementation process of a BEV semantic map construction method in this embodiment will be described in detail below.
[0061] Map building models, such as Figure 2 As shown, it includes a feature extraction module, a feature enhancement module, a feature fusion module, a feature transformation module, and a BEV decoder connected in sequence:
[0062] The feature extraction and feature enhancement modules construct a scene information storage pool through joint training on traffic scenarios in both normal and complex weather conditions. Weather conditions affecting driving safety, such as fog and snow, are defined as complex weather, while clear skies and cloudy skies are considered normal weather. Based on this, a method is proposed that utilizes joint two-stage training to store traffic scene information under normal weather conditions, thereby driving the generation of BEV maps for traffic scenarios in complex weather conditions. This achieves guidance from normal scene features for complex weather scenarios. In the first stage of training, normal scene features are saved to the storage pool. In the second stage, complex scene features are used to retrieve the information retained in the previous stage.
[0063] The feature fusion module based on channel attention fuses the memory features output from the storage pool with the features of complex scenes, thereby enabling the selection of important features.
[0064] Feature conversion module: The fused image view features are converted into BEV features through the conversion module.
[0065] BEV Decoder: Using BEV features as input, it generates a BEV semantic map for traffic scenarios under complex weather conditions.
[0066] In the various modules of the map building model described above, such as Figure 3 As shown, the detailed steps are as follows:
[0067] Step 1: Extract traffic scene features under normal weather conditions
[0068] A joint two-stage training method was adopted. The first stage takes ordinary weather images as input and stores traffic scene information (i.e., ordinary scene features) under ordinary weather conditions in a storage pool. The second stage takes complex weather images as input and matches them with the ordinary scene features associated with them.
[0069] In the first phase of joint training, EfficientNet-B0 is used as the image encoder E. S Feature extraction is performed, with input consisting of N images of ordinary weather scenes surrounding the vehicle. The output is the image features of this scene. Where N, H / H pv W / W pv These represent the number of cameras, image / feature map height, and image / feature map width, respectively.
[0070] Step 2: Construct a traffic scene information storage pool under normal weather conditions.
[0071] To improve the accuracy of semantic maps under complex weather conditions, a traffic scene information storage pool for normal weather conditions is constructed. This pool is responsible for saving the features of normal scenes from the first stage and providing support for generating BEV semantic maps for traffic scenes under complex weather conditions in the second stage. The construction process of the storage pool includes:
[0072] (a) Normalize the local features f and the storage parameters M of the storage pool respectively, and multiply the normalized results together before inputting them into the softmax function. Here, f is the input feature of the storage pool, which represents the local features after the image features under normal weather conditions are decomposed in the first stage, and the complex local features after the image features under complex weather conditions are decomposed in the second stage; the storage parameters M are updated through backpropagation in the first stage and stop updating in the second stage.
[0073] (b) Use the output α from (a) to weight the storage pool parameters, and then integrate the weighted local features as the output of the storage pool.
[0074] Therefore, a storage pool can be represented as:
[0075] α=softmax([Norm(f)·Norm(M)]) (1)
[0076] F=Ψ([α·M]) (2)
[0077] Where [·] represents the product of two matrices; Ψ represents the integration of the decomposed features; and F is the output feature of the storage pool.
[0078] In the first stage, the traffic scene features F obtained in step 1 under normal weather conditions are... pv_ Decomposed into ordinary local features Input is fed into the storage pool to participate in the first phase of training in order to retain traffic scene information under normal weather conditions.
[0079] Step 3: Retrieve relevant information from the storage pool based on traffic scene characteristics under complex weather conditions.
[0080] Similar to step 1, in the second stage of joint training, this embodiment also uses EfficientNet-B0 as the image encoder E for this stage. C Feature extraction is performed using traffic scene images under complex weather conditions as input. Output the corresponding complex image features E C and the first-stage image encoder E S Parameters are not shared.
[0081] F pv_ Decomposed into complex local features The complex local features at this point serve as input to the second stage of the storage pool, and then the corresponding operations are performed according to formulas (1) and (2). Since the storage pool parameters are not updated in the second stage, the complex local features can call the ordinary scene features saved in the first stage by the storage pool. Therefore, the output of the storage pool in this stage is the memory feature that retains traffic scene information under ordinary weather conditions.
[0082] Step 4: Merge the output features of the storage pool with the scene features
[0083] As can be seen from step 3, the output of the storage pool in the second stage is the memory feature F. m The image features under complex weather conditions are F pv_ To better select features, this embodiment constructs a feature fusion module based on channel attention; firstly, for F... m and F pv_ A concatenation operation is performed, and the concatenated result is then input into a channel attention module consisting of six layers, including average pooling layers, linear layers, and activation function layers. This module outputs the weights of each channel, and the output of the attention module is used to adjust the F... mWeighting, and finally the weighted features are compared with F. pv_ Perform the splicing operation again to obtain the result after fusing the two features; the process can be represented as:
[0084]
[0085] Where (,) represents the concatenate operation. F represents the channel attention module. pv_ These are the characteristics after fusion.
[0086] Step 5: Convert the fused image features into BEV features.
[0087] To obtain a semantic map from the BEV perspective, it is necessary to fuse the image features. Converting to BEV features; This embodiment utilizes a combination of MLP and camera parameters to achieve feature transformation, and the process can be represented as follows:
[0088]
[0089] F bev =R -1 (F cam -t) (5)
[0090] in, F represents a multilayer perceptron. cam F bev R and t represent the features in camera coordinates and BEV space, respectively; R and t represent the rotation and translation matrices, respectively; Equation (4) uses the multilayer perceptron to fuse the image features F. pv_ Feature F converted to camera coordinate system cam Then, according to formula (5), the features in the camera coordinate system are converted into features in the bird's-eye view using the camera parameters, namely BEV features; the BEV features are then used by the BEV decoder to generate a BEV semantic map.
[0091] Joint training of traffic scenarios in normal and complex weather conditions
[0092] Figure 4 These are image examples of traffic scenes under normal and complex weather conditions, based on, for example... Figure 4 The image data shown is used to jointly train the map building model for traffic scenarios in both normal and complex weather conditions.
[0093] The joint training proposed in this embodiment includes two stages. The first stage stores traffic scene information under normal weather conditions in a storage pool. The second stage matches traffic scenes under complex weather conditions with their related normal scene features.
[0094] In the first stage of joint training, the image encoder uses EfficientNet-B0 as the backbone network, taking six traffic scene images under normal weather conditions as input, with the image size set to 128×352, and outputting the corresponding ordinary image features. Next, the ordinary image features are decomposed into ordinary local features and input into a constructed storage pool. Then, the output of the storage pool executes steps 4 and 5, using a binary classification cross-entropy loss function to train the network to preserve the information from the first stage.
[0095] The second stage of joint training is similar to the first stage, using EfficientNet-B0 as the image encoder. Its input consists of six traffic scene images under complex weather conditions, and its output is the image features of the complex scene. These features are decomposed into complex local features and input into a storage pool. After the first stage, the output features of the storage pool in this stage contain relevant information about the ordinary scene corresponding to the complex scene. Next, the output features of the storage pool are trained using steps 4 and 5, employing a binary classification cross-entropy loss function. The difference is that the parameters of the storage pool are not updated in this stage.
[0096] This embodiment verifies the effectiveness of the method through experiments:
[0097] Experimental conditions
[0098] The hardware requirements for the verification experiment were as follows: Operating system: Ubuntu 20.04LTS; CPU: Intel Xeon Platinum 8373CR; GPU: NVIDIA GeForce RTX 4090.
[0099] The software environment used in the experiment was: Python 3.8, torch = 1.10.1, torchvision = 0.11.3.
[0100] Since there is currently a lack of datasets for generating semantic maps of BEVs under complex weather conditions, this embodiment enhances the NuScenes dataset to obtain the NuScenes-Weather dataset, which includes scenes under four weather conditions: fog, light rain, light snow, and heavy snow. The data collected by NuScenes includes 1000 driving scenarios collected in Boston and Singapore, with each scenario lasting approximately 20 seconds; the dataset samples keyframes at a frequency of 2Hz, with 28130 keyframes in the training set and 6019 keyframes in the validation set.
[0101] Experimental results
[0102] This embodiment conducted experiments on the NuScenes-Weather dataset; Table 1 compares the method designed in this embodiment with previous algorithms, where the evaluation metric is the Intersection over Union (IoU) (%), with a higher value indicating better performance; Table 2 compares the joint training and feature fusion scheme proposed in this embodiment using different transformation methods, demonstrating the effectiveness of the proposed scheme; Table 3 compares the results with different storage parameters; Table 4 shows the results when setting two different backbone networks.
[0103] Table 1 Comparison of different algorithms on the NuScenes-Weather dataset.
[0104]
[0105] Table 2 Comparison of results using the proposed scheme in this embodiment for different conversion methods
[0106]
[0107] Where Ours-L represents the result of using the proposed solution in this embodiment on LSS; Ours represents the result of using the proposed solution in this embodiment on HDMapNet, and the generated BEV semantic map is as follows: Figure 5 As shown.
[0108] Table 3 Comparison of results when setting different storage parameters
[0109]
[0110] Table 4 Comparison of results after setting two different backbone networks
[0111] Ours1 efficientnet-b0 36.7 18.1 36.8 Ours2 efficientnet-b4 37.6 18.5 37.3
[0112] Therefore, the BEV semantic map construction method in this embodiment defines weather conditions that affect driving safety, such as fog, rain, and snow, as complex weather, and considers sunny and cloudy days as ordinary weather. It adds rain, fog, and snow effects to the dataset under ordinary weather conditions to obtain complex weather data. By calling traffic scene information under ordinary weather conditions, it promotes the generation of BEV semantic maps for traffic scenes under complex weather conditions, effectively improves the accuracy of BEV semantic maps for traffic scenes under complex weather conditions, and alleviates the current problem of missing BEV semantic maps for traffic scenes under complex weather conditions.
[0113] Example 2
[0114] In one or more embodiments, a BEV semantic map construction system is disclosed, including an acquisition module and a generation module:
[0115] The acquisition module is configured to acquire ordinary weather scene images and complex weather scene images of the traffic scenario to be constructed.
[0116] The generation module is configured to perform joint two-stage training on the constructed map building model based on ordinary weather scene maps and complex weather scene maps, and output a BEV semantic map.
[0117] The map construction model employs a joint two-stage training process. The first stage involves saving the ordinary scene features of the ordinary weather scene map to a storage pool. The second stage involves retrieving the stored ordinary scene features from the storage pool to enhance the complex scene features of the complex weather scene map. Based on the enhanced features, a BEV semantic map is generated.
[0118] Example 3
[0119] The purpose of this embodiment is to provide a computer-readable storage medium.
[0120] A computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of a BEV semantic map construction method as described in Embodiment 1 of this disclosure.
[0121] Example 4
[0122] The purpose of this embodiment is to provide an electronic device.
[0123] An electronic device includes a memory, a processor, and a program stored in the memory and executable on the processor, wherein the processor executes the program to implement the steps in a BEV semantic map construction method as described in Embodiment 1 of this disclosure.
[0124] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.
Claims
1. A method for constructing a BEV semantic map, characterized in that, include: Obtain both ordinary weather scene images and complex weather scene images of the traffic scenario to be constructed; Based on ordinary weather scene maps and complex weather scene maps, the constructed map building model is jointly trained in two stages to output a BEV semantic map. The map building model adopts a joint two-stage training. The first stage is to save the ordinary scene features of the ordinary weather scene map to the storage pool. The second stage is to call the stored ordinary scene features from the storage pool to enhance the complex scene features of the complex weather scene map. Based on the enhanced features, a BEV semantic map is generated. The map construction model includes a feature extraction module, a feature enhancement module, a feature fusion module, a feature transformation module, and a BEV decoder connected in sequence. The specific processing steps are as follows: Feature extraction is performed on complex weather scene images to obtain complex scene features; By utilizing a storage pool built based on ordinary weather scene images, feature enhancement is performed on complex scene features to obtain memory features; The memory features are fused with the features of the complex scene to obtain the fused features; The fused features are then transformed and BEV decoded to generate a BEV semantic map. The specific construction process of the storage pool is as follows: Extracting common scene features from ordinary weather scene images; Perform channel feature decomposition on features of ordinary scenes; Using the storage parameter M, the decomposed features are stored in the storage pool. The storage parameter M is updated through backpropagation in the first stage and stops updating in the second stage. The feature enhancement for complex scenes specifically includes: Storage parameters for complex scene features and storage pools Perform normalization operations separately, multiply the normalized results of the two operations, and then input the result into the database. In the function, the output weight α is used; By weighting the storage pool parameters with weight α, the weighted local features are integrated to obtain the memory features.
2. The BEV semantic map construction method as described in claim 1, characterized in that, The fusion of memory features with complex scene features specifically involves: First, the memory features and complex scene features are concatenated. Then, the concatenated result is input into a channel attention module consisting of an average pooling layer, a linear layer, and an activation function layer to obtain the weights of each channel. Next, the memory features are weighted using the weights of each channel. Finally, the weighted features and complex scene features are concatenated again to obtain the result of the fusion of the two features.
3. The BEV semantic map construction method as described in claim 1, characterized in that, The feature transformation converts the fused features into BEV features, specifically as follows: Using the fused features as input, a multilayer perceptron is used to convert the image view features in the pixel coordinate system into features in the camera coordinate system. Features in the camera coordinate system are transformed into features in the BEV space using camera parameters.
4. The BEV semantic map construction method as described in claim 1, characterized in that, The BEV decoding, using BEV features as input, generates a BEV semantic map for traffic scenarios under complex weather conditions, specifically as follows: The input BEV features are processed through a convolutional layer, a batch normalization layer, and a ReLU activation function, reducing their size to half of the original. By using ResNet18 for downsampling, the size of the feature map is reduced to 1 / 4 and 1 / 8 of the original feature map; then, the upsampling module is used to restore the feature map to the original size.
5. A BEV semantic map construction system, characterized in that, The BEV semantic map construction method according to any one of claims 1-4 includes an acquisition module and a generation module: The acquisition module is configured to acquire ordinary weather scene images and complex weather scene images of the traffic scenario to be constructed. The generation module is configured to perform joint two-stage training on the constructed map building model based on ordinary weather scene maps and complex weather scene maps, and output a BEV semantic map. The map construction model employs a joint two-stage training process. The first stage involves saving the ordinary scene features of the ordinary weather scene map to a storage pool. The second stage involves retrieving the stored ordinary scene features from the storage pool to enhance the complex scene features of the complex weather scene map. Based on the enhanced features, a BEV semantic map is generated.
6. An electronic device, characterized in that it comprises: Memory is used to store computer-readable instructions in a non-transitory manner. as well as Processor, for executing the computer-readable instructions, When the computer-readable instructions are executed by the processor, they perform the method described in any one of claims 1-4.
7. A storage medium, characterized in that, The computer-readable instructions are stored non-temporarily, wherein when executed by a computer, the instructions of the method according to any one of claims 1-4 are executed.