Map updating method and device, electronic equipment and computer readable storage medium

Abstract

The application provides a map updating method and device, electronic equipment, computer readable storage medium and computer program product. The method comprises: performing prediction processing on the global map based on historical change data and current environment data of the global map through a first convolutional neural network model to obtain a first prediction result of a map element in the global map; performing first fusion processing on the global map and a local map at multiple collection times to obtain fusion results at the multiple collection times, and performing prediction processing on the fusion results at the multiple collection times through a recurrent neural network model to obtain a second prediction result of the map element in the global map; wherein the local map is determined according to vehicle perception data of a vehicle traveling in the global map; and performing map updating processing on the global map according to the first prediction result and the second prediction result of the map element in the global map. The application can significantly improve the timeliness and accuracy of map updating.

Description

Technical Field

[0001] This application relates to computer technology, and more particularly to a map updating method, apparatus, electronic device, computer-readable storage medium, and computer program product. Background Technology

[0002] With the rapid development of autonomous driving technology, the requirements for the accuracy and real-time performance of map information are increasing. Traditional map update solutions mostly adopt reactive update mechanisms, which rely on data collection and updates based on actual changes, such as through on-site surveys, user feedback, or officially released change information to achieve map updates.

[0003] However, when environmental changes occur in a region, such as road construction, road closures, or accidents, traditional map update schemes often fail to update the map in a timely manner. This not only results in a certain lag in map information but may also affect the accuracy and reliability of related applications such as navigation and location services that rely on map information. Summary of the Invention

[0004] This application provides a map updating method, apparatus, electronic device, computer-readable storage medium, and computer program product, which significantly improves the timeliness and accuracy of map updates by introducing a predictive update mechanism.

[0005] The technical solution of this application is implemented as follows:

[0006] This application provides a map updating method, including:

[0007] The first prediction result of the map elements in the global map is obtained by performing prediction processing based on the historical change data of the global map and the current environmental data through the first convolutional neural network model.

[0008] The global map and the local map are fused together at multiple acquisition times to obtain a fusion result for multiple acquisition times. A recurrent neural network model is then used to perform prediction processing based on the fusion result for multiple acquisition times to obtain a second prediction result for map elements in the global map. The local map is determined based on vehicle perception data of vehicles traveling on the global map.

[0009] Based on the first and second prediction results of the map elements in the global map, the global map is updated.

[0010] This application provides a map updating device, comprising:

[0011] The first prediction module is used to perform prediction processing based on historical change data of the global map and current environmental data through a first convolutional neural network model to obtain the first prediction result of map elements in the global map.

[0012] The second prediction module is used to perform a first fusion process on portions of the global map and the local map at multiple acquisition times to obtain a fusion result of multiple acquisition times, and then perform prediction processing based on the fusion result of multiple acquisition times using a recurrent neural network model to obtain a second prediction result of map elements in the global map; wherein, the local map is determined based on vehicle perception data of vehicles traveling on the global map.

[0013] The predictive update module is used to perform map update processing on the global map based on the first prediction result and the second prediction result of the map elements in the global map.

[0014] This application provides an electronic device, including:

[0015] Memory, used to store executable instructions;

[0016] The processor is configured to implement the map update method provided in this application when executing executable instructions stored in the memory.

[0017] This application provides a computer-readable storage medium storing executable instructions for inducing a processor to execute and implement the map update method provided in this application.

[0018] This application provides a computer program product including executable instructions for implementing the map update method provided in this application when executed by a processor.

[0019] This application has the following beneficial effects:

[0020] This application uses a first convolutional neural network model to perform predictive processing based on historical change data of the global map and current environmental data to obtain a first predicted result for map elements in the global map. Then, it performs a first fusion process on portions of the global map and local maps at multiple acquisition times to obtain a fusion result for multiple acquisition times. A recurrent neural network model then performs predictive processing based on this fusion result to obtain a second predicted result for map elements in the global map. The local map is determined based on vehicle perception data of vehicles traveling on the global map. Finally, based on the first and second predicted results for map elements in the global map, the global map is updated. This application effectively combines the update prediction results output by the two models and performs map updates in advance, improving the accuracy of map updates and thus contributing to the foresight and adaptability of autonomous driving systems. Attached Figure Description

[0021] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0022] Figure 1 This is a schematic diagram of the first architecture of the map update system provided in the embodiments of this application;

[0023] Figure 2 This is a schematic diagram of the structure of a server provided in an embodiment of this application;

[0024] Figure 3A This is a first flowchart illustrating the map update method provided in this application embodiment;

[0025] Figure 3B This is a schematic diagram of the second process of the map update method provided in the embodiments of this application;

[0026] Figure 3C This is a schematic diagram of the third process of the map update method provided in the embodiments of this application;

[0027] Figure 4 This is a schematic diagram of the second architecture of the map update system provided in the embodiments of this application;

[0028] Figure 5 This is a schematic diagram of the fourth process of the map update method provided in the embodiments of this application;

[0029] Figure 6 This is a schematic diagram of the fifth process of the map update method provided in the embodiments of this application. Detailed Implementation

[0030] To make the objectives, technical solutions, and advantages of this application clearer, the application will be further described in detail below with reference to the accompanying drawings. The described embodiments should not be regarded as limitations on this application. All other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0031] In the following description, references to "some embodiments" describe a subset of all possible embodiments; however, it is understood that "some embodiments" may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict. In the following description, the term "a plurality of" means at least two.

[0032] In the following description, the terms "first, second, third" are used merely to distinguish similar objects and do not represent a specific ordering of objects. It is understood that "first, second, third" may be interchanged in a specific order or sequence where permitted, so that the embodiments of this application described herein can be implemented in an order other than that illustrated or described herein.

[0033] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein is for the purpose of describing embodiments of this application only and is not intended to limit this application.

[0034] This application provides a map updating method, apparatus, electronic device, computer-readable storage medium, and computer program product. By introducing a predictive update mechanism, the timeliness and accuracy of map updates are significantly improved. The following describes exemplary applications of the electronic device provided in this application. The electronic device provided in this application can be implemented as various types of terminal devices, such as in-vehicle devices, or as a server.

[0035] See Figure 1 , Figure 1 This is a schematic diagram of the architecture of the map update system 100 provided in this application embodiment. The vehicle-mounted device 400 is connected to the server 200 through the network 300, wherein the network 300 can be a wide area network or a local area network, or a combination of the two.

[0036] In some embodiments, the map update method provided in this application can be implemented by a server. For example, the server 200 performs prediction processing based on historical change data of the global map and current environmental data using a first convolutional neural network model to obtain a first prediction result of map elements in the global map; the server 200 performs a first fusion processing on portions of the global map and the local map at multiple acquisition times to obtain a fusion result of multiple acquisition times, and performs prediction processing based on the fusion result of multiple acquisition times using a recurrent neural network model to obtain a second prediction result of map elements in the global map; wherein, the local map is determined based on vehicle perception data of vehicles traveling on the global map; the server 200 performs map update processing on the global map based on the first prediction result and the second prediction result of map elements in the global map.

[0037] In some embodiments, the map update method provided in this application can be implemented collaboratively by a terminal device and a server. For example, the vehicle-mounted device 400 can collect multiple vehicle perception data through vehicle-mounted perception sensors, generate a local map based on the perception data of each vehicle trip, and send the local maps corresponding to the multiple vehicle perception data trips to the server 200. The server 200 performs a second fusion processing on the local maps corresponding to the multiple vehicle perception data trips to obtain a global map. It is worth noting that the above process is exemplified by the fact that the perception data of multiple vehicle trips all come from the vehicle-mounted device 400. In some embodiments, the perception data of multiple vehicle trips may also come from different vehicle-mounted devices. For another example, after the server 200 performs map update processing on the global map, it can send the updated global map to the vehicle-mounted device 400 so that the vehicle-mounted device 400 can perform autonomous driving based on the updated global map, thereby improving the foresight and adaptability of autonomous driving.

[0038] In some embodiments, the terminal device or server can implement the map update method provided in this application by running a computer program. For example, the computer program can be a native program or software module in an operating system; it can be a native application (APP), i.e., a program that needs to be installed in the operating system to run; it can also be a mini-program, i.e., a program that only needs to be downloaded to a browser environment to run; or it can be a mini-program that can be embedded in any APP, and the mini-program can be controlled by the user to run or close. In short, the above-mentioned computer program can be any form of application, module, or plugin.

[0039] Taking the electronic device provided in this application as a server as an example, see [link to relevant documentation]. Figure 2 , Figure 2 This is a schematic diagram of the structure of the server 200 provided in the embodiments of this application. Figure 2 The server 200 shown includes at least one processor 210, memory 250, and at least one network interface 220. The various components of server 200 are coupled together via a bus system 240. It is understood that the bus system 240 is used to implement communication between these components. In addition to a data bus, the bus system 240 also includes a power bus, a control bus, and a status signal bus. However, for clarity, ... Figure 2 The general labeled all buses as Bus System 240.

[0040] The processor 210 can be an integrated circuit chip with signal processing capabilities, such as a general-purpose processor, a digital signal processor (DSP), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or any conventional processor, etc.

[0041] The memory 250 may be removable, non-removable, or a combination thereof. Exemplary hardware devices include solid-state storage, hard disk drives, optical disk drives, etc. The memory 250 may optionally include one or more storage devices physically located away from the processor 210.

[0042] The memory 250 may include volatile memory or non-volatile memory, or both. The non-volatile memory may be read-only memory (ROM), and the volatile memory may be random access memory (RAM). The memory 250 described in this application embodiment is intended to include any suitable type of memory.

[0043] In some embodiments, memory 250 is capable of storing data to support various operations, examples of which include programs, modules, and data structures or subsets or supersets thereof, as illustrated below.

[0044] Operating system 251 includes system programs for handling various basic system services and performing hardware-related tasks, such as the framework layer, core library layer, driver layer, etc., for implementing various basic business functions and handling hardware-based tasks;

[0045] The network communication module 252 is used to reach other computing devices via one or more (wired or wireless) network interfaces 220, exemplary network interfaces 220 including: Bluetooth, WiFi, and Universal Serial Bus (USB), etc.

[0046] In some embodiments, the map updating device provided in this application can be implemented in software. Figure 2 A map updating device 255 stored in memory 250 is shown. This device can be software in the form of programs and plugins, and includes the following software modules: a first prediction module 2551, a second prediction module 2552, and a predictive update module 2553. These modules are logically linked and can therefore be arbitrarily combined or further divided according to their implemented functions. The functions of each module will be described below.

[0047] The map update method provided in this application will be described in conjunction with exemplary applications and implementations of the electronic devices provided in the embodiments of this application.

[0048] See Figure 3A , Figure 3A This is a flowchart illustrating a map update method provided in an embodiment of this application. The map update method provided in this application can be implemented by an electronic device, such as a terminal device and / or a server, and will be combined with... Figure 3A The steps shown are explained.

[0049] In step 101, the first prediction result of the map elements in the global map is obtained by using the first convolutional neural network model to perform prediction processing based on the historical change data of the global map and the current environmental data.

[0050] Here, we obtain historical change data and current environmental data for the global map. Historical change data represents the historical changes in map elements, including: 1) Historical global map: past map versions, recording changes to various map elements (such as ground elements like roads, and aerial elements like traffic signs); 2) Map update log: detailed records of each map update, such as retained, deleted, or updated map elements. Current environmental data represents the current environment (such as weather, road conditions, and time), including: 1) Weather conditions: such as sunny, rainy, or snowy weather, which affects road conditions and visibility; 2) Traffic flow: current traffic flow and congestion, which can be obtained through vehicle-mounted sensors or traffic monitoring systems; 3) Special events: such as construction, accidents, or other temporary events, which may lead to road closures or detours; 4) Time factors: different times of day or different days of the week.

[0051] Then, the first convolutional neural network model performs prediction processing based on historical change data of the global map and current environmental data to obtain the first prediction result of map elements in the global map. This first prediction result indicates whether the map element should be retained, deleted, or updated. Prior to this, the first convolutional neural network model can be trained to improve the accuracy of the prediction processing.

[0052] It is worth noting that the map elements involved in the embodiments of this application refer to all points in the global map; the global map refers to a map described by a global coordinate system (geocentric coordinate system), while the local map mentioned below refers to a map described by a vehicle coordinate system.

[0053] It's worth noting that Convolutional Neural Networks (CNNs) are a type of deep feedforward neural network that incorporates convolutional computations. They automatically extract features from data and make predictions using components such as convolutional layers, pooling layers, and fully-connected layers. CNN models are well-suited for processing data with a grid-like structure.

[0054] In some embodiments, the first convolutional neural network model may also output a confidence score for the first prediction result, which reflects the credibility of the first prediction result and facilitates more accurate and effective decision-making in the subsequent map update process of step 103.

[0055] In step 102, the global map and the local map are subjected to a first fusion process at multiple acquisition times to obtain a fusion result at multiple acquisition times. Then, a recurrent neural network model is used to perform prediction processing based on the fusion result at multiple acquisition times to obtain a second prediction result of map elements in the global map. The local map is determined based on vehicle perception data of vehicles traveling on the global map.

[0056] For example, vehicle perception data is collected by the onboard perception sensors of a vehicle traveling on a global map, and a local map described by the vehicle coordinate system is determined based on the vehicle perception data. The global map and the local map are then subjected to a first fusion process (such as stitching) at multiple acquisition times (or time steps) to obtain a fusion result for multiple acquisition times. For example, the global map and the local map are subjected to a first fusion process at the first acquisition time to obtain a fusion result for the first acquisition time. In this way, prior information in the global map is taken into account, and missing information in the local map (such as lane lines being obscured or blurred by other vehicles) is supplemented.

[0057] Then, a recurrent neural network model is used to perform prediction processing based on the fusion results of multiple data collection times to obtain a second prediction result for map elements in the global map. In this way, time series analysis is used to simulate the changes in map data over time. The second prediction result is also used to indicate the retention, deletion, or updating of map elements. Prior to this, the recurrent neural network model can be trained to improve the prediction performance.

[0058] It's worth noting that a Recurrent Neural Network (RNN) is a neural network model used to process sequential data. Its key characteristic is the presence of recurrent connections within the network structure, allowing information to be passed cyclically through the network. Structurally, a recurrent neural network model mainly consists of an input layer, hidden layers, and an output layer. Unlike traditional feedforward neural network models, the neurons in the hidden layers of a recurrent neural network are connected cyclically.

[0059] In some embodiments, the recurrent neural network model can be a Long Short-Term Memory (LSTM) network model.

[0060] It is worth noting that vehicle perception data can be divided according to the collection time, and therefore, local maps generated based on vehicle perception data can also be divided according to the collection time.

[0061] In some embodiments, if there are local maps corresponding to multiple vehicle perception data, step 102 can be executed multiple times to obtain a more accurate second prediction result by combining the global map and the local maps corresponding to multiple vehicle perception data.

[0062] In step 103, the global map is updated based on the first and second prediction results of the map features in the global map.

[0063] Here, based on the first and second prediction results of map elements in the global map, map update processing is performed on the global map. That is, the decision to retain, delete, or update map elements in the global map is made by combining the first and second prediction results of the map elements in the global map. The above map update processing can be implemented through preset decision rules. For example, for a certain map element in the global map, when the first and second prediction results of the map element are consistent, the map element is processed according to the consistent result (e.g., if both the first and second prediction results of the map element are for deletion, then the map element is deleted in the global map); when the first and second prediction results of the map element are inconsistent, the map element is kept unchanged in the global map, that is, the map element is retained.

[0064] In some embodiments, the above-mentioned map update processing of the global map based on the first and second prediction results of map elements in the global map can be achieved in the following manner: a second convolutional neural network model performs prediction processing based on the first and second prediction results of map elements in the global map to obtain a third update prediction result of map elements in the global map; based on the third update prediction result of map elements in the global map, it is determined whether to retain, delete, or update the map elements in the global map.

[0065] Here, the first and second prediction results of map features in the global map can be input into the second convolutional neural network model, allowing the model to perform prediction processing and obtain the third update prediction result for the map features in the global map. Then, based on the third update prediction result, it is determined whether to retain, delete, or update the map features in the global map, resulting in an updated global map. For example, if the third update prediction result for a map feature is to retain it, a decision to retain the map feature is made in the global map, and so on. Prior to this, the second convolutional neural network model can be trained to improve the prediction processing efficiency.

[0066] The above method does not rely on fixed decision rules, but achieves automatic decision-making through a second convolutional neural network model, enabling more accurate map update processing.

[0067] In some embodiments, autonomous driving can be performed based on the updated global map. Taking steps 101 to 103 as an example implemented on a server, the server can send the updated global map to multiple in-vehicle devices, enabling each in-vehicle device to perform autonomous driving based on the updated global map. Since steps 101 to 103 can accurately predict future changes in map features, the foresight and adaptability of autonomous driving can be improved.

[0068] like Figure 3AAs shown, this embodiment of the application uses a first convolutional neural network model to perform prediction processing based on historical change data of the global map and current environmental data to obtain a first prediction result of map elements in the global map. The global map and local maps at multiple acquisition times are then fused to obtain a fusion result across multiple acquisition times. A recurrent neural network model is then used to perform prediction processing based on this fusion result to obtain a second prediction result of map elements in the global map. The local map is determined based on vehicle perception data of vehicles traveling on the global map. Based on the first and second prediction results of map elements in the global map, the global map is updated. This embodiment of the application effectively combines the update prediction results output by the two models and performs map updates in advance, improving the accuracy of map updates and thus contributing to the improvement of the foresight and adaptability of the autonomous driving system.

[0069] In some embodiments, see Figure 3B , Figure 3B This is a flowchart illustrating a map update method provided in an embodiment of this application, based on... Figure 3A Before step 101, steps 201 to 203 can also be performed.

[0070] In step 201, feature encoding processing is performed on the perception data of multiple vehicles to obtain the bird's-eye view BEV features corresponding to the perception data of multiple vehicles.

[0071] Here, multiple vehicle perception data can be collected. For the i-th vehicle perception data, feature encoding processing is performed on the i-th vehicle perception data to obtain the corresponding Bird's Eye View (BEV) feature.

[0072] It is worth noting that the perception data from multiple vehicles can come from the same vehicle or from different vehicles; the perception data from multiple vehicles can be collected on the same driving route or on different driving routes, and there is no limitation on this.

[0073] In some embodiments, the above-mentioned feature encoding processing of multiple vehicle perception data can be implemented to obtain the bird's-eye view BEV features corresponding to each of the multiple vehicle perception data: For any vehicle perception data, the following processing is performed: When any vehicle perception data is an image captured by an onboard camera from a specific viewpoint, the image is processed by an image feature extraction model to obtain a feature map of the specific viewpoint, and the feature map of the specific viewpoint is processed by viewpoint transformation to obtain the image BEV features, and the image BEV features are determined as the BEV features for the second fusion processing; When any vehicle perception data is a point cloud captured by onboard radar, the point cloud is processed by... The feature extraction model extracts features from the point cloud to obtain point cloud BEV features, which are then used as the BEV features for the second fusion process. When any vehicle perception data includes an image captured by an onboard camera from a specific viewpoint and a point cloud captured by onboard radar, the image feature extraction model extracts features from the image to obtain a feature map from the specific viewpoint. The feature map from the specific viewpoint is then subjected to viewpoint transformation processing to obtain image BEV features. The point cloud feature extraction model extracts features from the point cloud to obtain point cloud BEV features. Finally, the image BEV features and the point cloud BEV features are subjected to a third fusion process to obtain the BEV features used for the second fusion process.

[0074] To facilitate understanding, we will use the perception data of the i-th vehicle as an example. Depending on the type of onboard perception sensor, feature encoding processing can include the following three cases:

[0075] 1) The perception data for the i-th vehicle is an image captured by an onboard camera from a specific perspective (such as a forward-looking view or a surround-view view). At this point, an image feature extraction model is used to extract features from the image to obtain a feature map of the specific perspective. This feature map is then subjected to perspective transformation processing to obtain the image's BEV features, which are then used as the BEV features for the second fusion process. The type of image feature extraction model is not limited; it can be a 2D backbone network, such as ResNet, or a combination of a 2D backbone network and a neck network, such as ResNet+FPN. The perspective transformation processing is used to transform the perspective from a perspective view (PV) to a bird's-eye view. This can employ geometry-based methods, such as Inverse Perspective Mapping (IPM), depth-based methods, such as LSS (Lift, Splat, Shoot), or Transformer-based methods, such as Deformable Attention.

[0076] 2) The perception data for the i-th vehicle is a point cloud collected by the vehicle-mounted radar. At this point, the point cloud feature extraction model is used to extract features from the point cloud to obtain the BEV features, which are then used as the BEV features for the second fusion process. The type of point cloud feature extraction model is not limited; for example, it can be a 3D backbone network, such as PointPillars or SECOND.

[0077] 3) The vehicle perception data for the i-th trip includes images acquired by an onboard camera from a specific viewpoint and point clouds acquired by onboard radar. At this point, on one hand, an image feature extraction model is used to extract features from the images to obtain feature maps from the specific viewpoint; then, viewpoint transformation processing is performed on these feature maps to obtain image BEV features. On the other hand, a point cloud feature extraction model is used to extract features from the point cloud to obtain point cloud BEV features. Finally, a third fusion process (such as stitching) is performed on the image BEV features and the point cloud BEV features to obtain the BEV features used for the second fusion process.

[0078] The above method implements feature encoding processing in a targeted manner according to the type of vehicle-mounted perception sensor, which can extract accurate and effective BEV features from vehicle perception data.

[0079] In step 202, the BEV features corresponding to the multiple vehicle perception data are subjected to feature decoding processing to obtain the local maps corresponding to the multiple vehicle perception data.

[0080] Here, the BEV features corresponding to the i-th vehicle perception data are processed by feature decoding to obtain the local map corresponding to the i-th vehicle perception data. The local map is represented by a vectorized road structure. It is worth noting that the feature decoding process can be implemented using a feature decoder. During feature decoding, the feature decoder uses the BEV features to perform 3D object detection and road structure classification and localization tasks. That is, the feature decoder generates output embeddings through interaction with the BEV features. These output embeddings contain the position and shape information of the target object in the BEV space. The above feature decoding process involves self-attention and cross-attention mechanisms to ensure that different BEV features can be effectively fused, thereby obtaining an accurate vectorized road structure.

[0081] In step 203, the local maps corresponding to the perception data of multiple vehicles are subjected to a second fusion process to obtain a global map.

[0082] The local map obtained in step 202 is described in the vehicle coordinate system. Therefore, a second fusion process is performed on the local maps corresponding to multiple vehicle perception data to obtain a global map described in the global coordinate system. It is worth noting that the above-mentioned second fusion process is implemented on the portions of the local maps corresponding to multiple vehicle perception data within the same local area, that is, fusion is performed separately for each local area.

[0083] In some embodiments, the second fusion processing of the local maps corresponding to multiple vehicle perception data to obtain a global map can be achieved in the following manner: determining the data quality index and historical accuracy index corresponding to each vehicle perception data; determining the weight of the local map corresponding to the vehicle perception data based on the data quality index and historical accuracy index corresponding to each vehicle perception data; and performing a weighted summation of the local maps corresponding to multiple vehicle perception data based on their respective weights to obtain a global map.

[0084] Here, for the perception data of the i-th vehicle trip, the data quality index and historical accuracy index corresponding to the i-th vehicle trip can be determined. The data quality index represents the data quality of the i-th vehicle trip itself, and the historical accuracy index represents the accuracy of previous vehicle perception data (both collected from the same onboard sensor in the same vehicle). Then, based on the data quality index and historical accuracy index of the i-th vehicle trip, the weight of the local map corresponding to the i-th vehicle trip is determined. The weight of the local map corresponding to the i-th vehicle trip is positively correlated with the data quality index and positively correlated with the historical accuracy index. For example, the weight of the local map corresponding to the i-th vehicle trip can be obtained by weighted summation of the data quality index and historical accuracy index (the weights can be preset). Then, based on the weights of the local maps corresponding to the multiple vehicle perception data, a weighted summation is performed on the local maps corresponding to the multiple vehicle perception data to obtain the global map.

[0085] The above method takes into account both real-time data quality and historical accuracy, assigns weights to local maps and then performs weighted summation, which can improve the accuracy of the final global map.

[0086] In some embodiments, the determination of the data quality index and historical accuracy index corresponding to each vehicle perception data trip can be achieved in the following manner: For any vehicle perception data trip, the following processing is performed: Based on the signal-to-noise ratio and measurement error of the on-board perception sensor, the data quality index corresponding to any vehicle perception data trip is determined; wherein, the on-board perception sensor is used to collect the vehicle perception data trip; the accuracy index of the vehicle perception data historically collected by the on-board perception sensor is determined as the historical accuracy index corresponding to any vehicle perception data trip.

[0087] Taking the perception data of the i-th vehicle trip as an example, on the one hand, based on the signal-to-noise ratio and measurement error of the onboard perception sensor used to collect the perception data of the i-th vehicle trip, the data quality index corresponding to the perception data of the i-th vehicle trip is determined. The data quality index corresponding to the perception data of the i-th vehicle trip is positively correlated with the signal-to-noise ratio of the onboard perception sensor, and negatively correlated with the measurement error of the onboard perception sensor. The measurement error of the onboard perception sensor refers to the error (such as standard deviation) between the measured value and the true value (which can be obtained through high-precision maps or surveying). On the other hand, the accuracy index of the historical vehicle perception data collected by the onboard perception sensor used to collect the perception data of the i-th vehicle trip is determined as the historical accuracy index corresponding to the perception data of the i-th vehicle trip. The accuracy index can be reflected as the ratio between the number of correct predictions and the total number of predictions within a certain time period. Here, prediction refers to predicting changes in map elements, and correct prediction means that the predicted changes are consistent with the actual changes.

[0088] In the above approach, data quality can be comprehensively evaluated by using signal-to-noise ratio and measurement error to obtain accurate and reliable data quality indicators; by determining the accuracy indicators of historically collected vehicle perception data as historical accuracy indicators, the performance of the i-th vehicle perception data in the past can be accurately evaluated.

[0089] like Figure 3B As shown, this embodiment of the application performs feature encoding processing on multiple vehicle perception data to obtain bird's-eye view BEV features corresponding to each of the multiple vehicle perception data; performs feature decoding processing on the BEV features corresponding to each of the multiple vehicle perception data to obtain local maps corresponding to each of the multiple vehicle perception data; and performs a second fusion processing on the local maps corresponding to each of the multiple vehicle perception data to obtain a global map. In this way, on the one hand, it can fully utilize regional global information, such as geometric smoothness constraints, semantic relevance, and global accuracy consistency; on the other hand, by collecting multiple vehicle perception data, it can alleviate unavoidable challenges such as accuracy deviation and dynamic occlusion.

[0090] In some embodiments, see Figure 3C , Figure 3C This is a flowchart illustrating a map update method provided in an embodiment of this application, based on... Figure 3B After step 203, steps 301 to 302 can also be executed.

[0091] In step 301, update detection processing is performed on the global map and the historical global map to obtain the update detection results of map elements in the global map.

[0092] After performing a second fusion process on the local maps corresponding to multiple vehicle perception data to obtain a global map, update detection processing can be performed on the global map and the historical global map to obtain the update detection results of map elements in the global map. The historical global map refers to the historical version of the global map.

[0093] In some embodiments, the update detection process is used to detect at least one of geometric changes, attribute changes, and topological relationship changes. These will be described separately below.

[0094] 1) Geometric Changes. Geometric changes refer to situations where map features in the global map have been added, deleted, or changed in position or shape relative to historical global maps. For example, the matching degree between a map feature in the global map and historical global maps can be calculated, and the existence probability of the map feature can be estimated based on the matching degree. When the existence probability of the map feature is greater than a preset existence probability threshold, the map feature is determined to be a newly added map feature; when the existence probability of the map feature is less than or equal to the existence probability threshold, the map feature is determined to be a deleted map feature; when the matching degree between the map feature and historical global maps is less than or equal to a preset matching degree threshold, the map feature is determined to be a changed map feature in position or shape.

[0095] 2) Attribute Changes. Attribute changes refer to situations where map features in the global map exhibit changes in their attributes compared to map features in the same location in historical global maps. Attributes such as the number of lanes and road names are not limited to this.

[0096] 3) Changes in topological relationships. Changes in topological relationships refer to changes in the topological relationships between map features in the global map compared to the topological relationships between map features in the historical global map. These topological relationships include connections, intersections, and containment.

[0097] In step 302, the global map is updated based on the update detection results of map features in the global map.

[0098] Here, the global map is updated based on the update detection results of map elements in the global map, that is, the retention, deletion or update of map elements is determined based on the update detection results of map elements in the global map.

[0099] like Figure 3C As shown, this embodiment of the application performs update detection processing on the global map and the historical global map to obtain the update detection results of map elements in the global map; based on the update detection results of map elements in the global map, the global map is updated. Thus, by comparing the global map with the historical global map to identify changed areas, accurate map update processing is achieved.

[0100] The following describes an exemplary application of the embodiments of this application in a real-world scenario. This application proposes an Adaptive Real-time Map Update Optimization Algorithm (ARMUOA) to address the limitations of traditional map update methods in terms of efficiency, accuracy, real-time performance, and resource optimization.

[0101] As an example, embodiments of this application provide, for instance, the following: Figure 4 The diagram shows an architecture of a map update system, and as shown below. Figure 5 The diagram shown is a flowchart of the map update method, combined with Figure 4 as well as Figure 5 The embodiments of this application can be implemented through the following steps.

[0102] Step S1, Data Preprocessing: Collect perception data from multiple vehicle trips, and clean and standardize the input data through data preprocessing.

[0103] Step S2, Adaptive weight allocation: Assign weights based on data quality and historical accuracy, and then extract and fuse the weighted data for features.

[0104] Step S3, Change Detection: Identify changed areas in the global map to make update decisions.

[0105] Step S4, Predictive Update: Predict and update potentially changing areas in advance on the global map.

[0106] Step S5, Resource Scheduling: Optimize the allocation of computing resources to ensure real-time performance; adjust algorithm parameters according to requirements through a user-defined interface.

[0107] Step S6, Feedback Learning: Collect user feedback and performance data to perform self-optimization.

[0108] The above steps will be explained in detail below.

[0109] 1) Step S1.

[0110] Step S1 is achieved through collaboration between the vehicle and the cloud.

[0111] For the vehicle, vehicle perception data is collected through onboard sensors, and vectorized road structure (referring to a local map) is extracted from it. Depending on the installation configuration of the onboard perception sensors and the data modality, the operations performed by the vehicle can be achieved in one of the following ways.

[0112] The first approach is suitable for using only the vehicle-mounted camera as the vehicle's perception sensor. In this case, the feature encoder includes an image feature extraction model and a viewpoint transformation module. The specific steps include:

[0113] Step S11a involves extracting features from the images captured by the vehicle-mounted camera using an image feature extraction model to obtain feature maps for the corresponding viewpoints. The vehicle-mounted camera can be a front-view camera, in which case the extracted feature map is the front-view view; or it can be a surround-view camera, in which case the extracted feature map is the surround-view view (multi-view) feature map. The image feature extraction model can be a 2D backbone network, such as ResNet, or a combination of a 2D backbone network and a neck network, such as ResNet+FPN.

[0114] Step S12a involves performing a perspective transformation on the feature map using the perspective transformation module to obtain BEV features. This module transforms the feature map from a perspective view (PV) to a bird's-eye view. Specifically, it can employ geometric methods such as Inverse Perspective Mapping (IPM), depth-based methods such as LSS (Lift, Splat, Shoot), or Transformer-based methods such as Deformable Attention.

[0115] The second method is suitable for using only vehicle-mounted LiDAR as the vehicle's perception sensor. In this case, the feature encoder includes a point cloud feature extraction model, and the specific steps include:

[0116] Step S11b involves extracting features from the point cloud data collected by the vehicle-mounted LiDAR using a point cloud feature extraction model to obtain BEV features. The point cloud feature extraction model can be a 3D backbone network, such as PointPillars or SECOND.

[0117] The third approach is suitable for using both vehicle-mounted cameras and vehicle-mounted LiDAR as vehicle-mounted perception sensors. In this approach, the feature encoder includes an image feature extraction model, a viewpoint transformation module, a point cloud feature extraction model, and a multimodal fusion module. The specific steps include:

[0118] Step S11c: Obtain image BEV features through steps S11a and S12a of the first method; obtain point cloud BEV features through step S11b of the second method.

[0119] In step S12c, the image BEV features and point cloud BEV features are fused using a multimodal fusion module to obtain the BEV features. The multimodal fusion module can fuse (e.g., stitch) the BEV features corresponding to two different modalities, namely image BEV features and point cloud BEV features.

[0120] After obtaining BEV features through the first, second, or third method described above, the BEV features are converted into vectorized road structures by a feature decoder. For example, the feature decoder uses these BEV features to perform 3D object detection and road structure classification and localization tasks. That is, the feature decoder generates output embeddings through interaction with the BEV features. These output embeddings contain the position and shape information of the target object in the BEV space. This process involves self-attention and cross-attention mechanisms to ensure that features from different perspectives can be effectively fused, thereby obtaining an accurate vectorized road structure.

[0121] It is worth noting that, Figure 4 The image also shows an onboard positioning sensor, which is used to collect vehicle position and movement information to achieve high-precision vehicle positioning and help update maps in real time to reflect road changes.

[0122] For the cloud, it can obtain the local map uploaded by the vehicle and perform the following step S13.

[0123] Step S13: To obtain a city-scale lane-level vector map (i.e., a global map), the cloud uses a sliding window method to process the entire city area in a zigzag scanning sequence. For each local area, a local map has already been extracted at the vehicle end; therefore, the input data, after data preprocessing, cleaning, and standardization at the cloud level, is data from a rectangular area outside or in front of the vehicle, within the global coordinate system. Compared to most in-vehicle methods that use multi-view images as input to operate on BEV features, the main advantages of this embodiment are twofold: firstly, it can fully utilize global regional information, such as geometric smoothness constraints, semantic relevance, and global accuracy consistency; secondly, multiple data collections can alleviate unavoidable challenges such as accuracy deviations and dynamic occlusion.

[0124] 2) Step S2.

[0125] Weights are assigned based on the data quality and historical accuracy of the perception data for each vehicle trip. An adaptive weight allocator then performs feature extraction and fusion on the weighted data. Step S2 can be implemented through the following steps S21 and S22.

[0126] Step S21: Calculate the data quality index Q of the perception data for each vehicle trip. i (t) and historical accuracy index H i .

[0127] Data quality indicator Q i (t): The data quality index of the i-th data source (referring to the perception data of the i-th vehicle) at time t is quantified based on factors such as signal-to-noise ratio and measurement error. The formula is as follows:

[0128] Q i (t)=f qual ity (SNR i (t),error i (t))

[0129] Specifically, SNR is an abbreviation for Signal-to-Noise Ratio. Among them, P signal,i (t) is the signal power of the vehicle-mounted sensing sensor corresponding to the i-th data source at time t, P noise,i (t) is the noise power of the vehicle-mounted sensing sensor corresponding to the i-th data source at time t; the measurement error is the standard deviation between the measured value and the true value (the true value can be obtained through high-precision maps or surveying). i (t) = std(Measurements) i —GT), where std represents the standard deviation, Measurements i These are measured values, GT is the true value; f qual ity It is a weighted summation function for calculating data quality indicators, where the weight of each factor is adjusted according to the importance of the actual application scenario.

[0130] Historical accuracy index H i Historical accuracy metrics are assigned based on the past accuracy performance of the i-th data source. The formula is as follows:

[0131]

[0132] Specifically, the number of correct predictions (CorrectPredictions) i This represents the number of times that the predictions of the i-th data source match the actual changes over a period of time. It reflects the accuracy of the i-th data source in predicting changes historically. The number of correct predictions can be determined by comparing historical prediction data with actual change data. Each time the data source successfully predicts a change (e.g., correctly identifying changes in lane lines, traffic signs, or other map features), it is recorded as a correct prediction. TotalPredictions represents the total number of predictions. i This represents the total number of predictions made by the i-th data source within the same time period, regardless of whether these predictions were ultimately proven correct. Historical accuracy metric H i The higher the value, the better the historical performance of the i-th data source, and the higher the reliability of its prediction.

[0133] Step S22: Dynamically adjust the weights of each data source. When the data quality index and historical accuracy index of the i-th data source are inconsistent, the system will adjust the weights according to the degree of deviation between the two to optimize the map building process.

[0134] Set adaptive weight W i (t), combining data quality metrics and historical accuracy metrics, calculates the adaptive weight of time t. The formula is as follows:

[0135] W i (t)=ω1·Q i (t)+ω2·H i

[0136] Specifically, ω1 and ω2 are used to balance the data quality metric Q. i (t) and historical accuracy index H i For the final weight W i The weights (t) influence the algorithm's performance and can be adjusted based on the algorithm's capabilities and the specific application scenario to ensure the algorithm provides high-quality map data under various conditions. In practical applications, adjusting these weights requires consideration of multiple factors, including the type of data source, environmental conditions, and the algorithm's performance objectives. For example, if a data source performs well in a particular environment, its weight may need to be increased; conversely, if a data source performs poorly, its weight may need to be decreased.

[0137] Adaptive weights W i The calculation of (t) takes into account the data quality index Q of the i-th data source at time t. i Q i (t), and also considers the past performance of the i-th data source, i.e., the historical accuracy metric H. iIn this way, the algorithm can flexibly respond to changes in data quality, thereby improving the accuracy and efficiency of map updates.

[0138] To ensure that the sum of all weights is 1, normalization is required to guarantee the rationality of data fusion. The formula is as follows:

[0139]

[0140] The normalized weights are used to fuse the information from each data source (referring to the local maps corresponding to the perception data of multiple vehicles).

[0141]

[0142] Among them, X i (t) is the data output (i.e., local map) of the i-th data source at time t, and n is the total number of data sources.

[0143] The adaptive weight allocation mechanism used in step S2 allows the system to dynamically adjust the contribution of the data source in the map update process based on the real-time performance and historical accuracy of the data source, thereby improving the accuracy and reliability of the overall map data.

[0144] 3) Step S3.

[0145] Step S3 identifies the changed areas in the global map through change detection (corresponding to update detection above) and makes an update decision for the global map. Step S3 can be implemented through the following steps S31 and S33.

[0146] Step S31: Identify the changed regions in the global map. For each map feature e in the global map, evaluate its matching degree MC(e,M) with the historical global map M. This can be achieved through various methods, such as calculating the Hausdorff distance as the matching degree. Then, estimate the existence probability P of map feature e based on the matching degree MC(e,M). exist (e), the formula is P exist (e) = σ(W·F(e) + b), where σ is the sigmoid function, W and b are model parameters, and F(e) is MC(e,M).

[0147] The change detection scenarios considered in this application's embodiments include: ① addition of map features; ② deletion of map features; ③ changes in the position or shape of map features; ④ changes in map feature attributes; ⑤ changes in the topological relationships between map features. Wherein:

[0148] ① New map element: refers to P exist (e) Greater than the preset existence probability threshold P t .

[0149] ②Map element deletion: refers to P exist (e) Less than or equal to the probability threshold P t .

[0150] ③ Changes in the location or shape of map features: This refers to a matching degree MC(e,M) that is less than or equal to the matching degree threshold T. position .

[0151] ④ Map feature attribute changes: refers to the difference between attribute vectors A(e) and A(M) exceeding the attribute difference threshold T. attribute .

[0152] ⑤ Changes in topological relationships between map features: This refers to changes in the topological relationships (such as connection, intersection, containment, etc.) between map features. A topological relationship matrix R is used to represent the topological relationships between map features. If R... new≠ R old This proves that the topological relationships between map features have changed, where R new R represents the topological relationships between map features in the global map (the currently generated global map). old It represents the topological relationships between map elements in the historical global map.

[0153] Step S32, generate the change mapping C map The change map marks all detected areas of change, providing a basis for update decisions. It is a two-dimensional array or matrix where each element corresponds to a point on the global map. If a change is detected at that point, it is marked as 1 in the change map; otherwise, it is marked as 0. The change map is used to identify and locate areas that need updating. The formula is as follows:

[0154]

[0155] Where (i,j) represents the location on the map, P t It is a preset probability threshold; changedetected in the formula means that changes in the location or shape of map features, changes in map feature attributes, or changes in the topological relationship between map features are detected.

[0156] Step S33, Update Decision. For each map element e in the global map, based on the change mapping and existence probability, decide whether to retain, delete, or update the map element. The formula is as follows:

[0157]

[0158] 4) Step S4.

[0159] Step S4 involves predictive updates to anticipate and update potentially changing areas in the global map. Step S4 can be implemented through steps S41 and S43.

[0160] Step S41: Use a recurrent neural network model to perform time series prediction, thereby simulating the changes of the map over time.

[0161] Here, prior information from the global map needs to be taken into consideration to supplement missing information in the local map corresponding to the current single-trip vehicle perception data (such as lane lines being obscured or blurred by other vehicles), thereby obtaining map features with significantly improved representation capabilities. In the embodiments of this application, the recurrent neural network model can be an LSTM model.

[0162] Let X t Let X be the local map portion at acquisition time t (which can be represented by eigenvectors), and G be the prior information matrix of the global map. t The input H is fused with G and used as the input to the LSTM model. t =Concat(X) t ,G).

[0163] Let h t＝0 Let h be the initial hidden state of the LSTM model at acquisition time t=0. Wherein, the hidden state h... t It is a compressed representation of sequence information by the LSTM model, which is continuously updated as the acquisition time progresses, providing necessary contextual information for time series prediction tasks.

[0164] The LSTM unit updates its hidden state h at each acquisition time. t and cell state c t This allows the LSTM model to better handle long-term dependencies in time series data. The specific formula is as follows:

[0165] i t =σ(W i ·H t +b i )

[0166] f t =σ(W f ·H t +b f )

[0167] o t =σ(W o ·H t +b o )

[0168]

[0169]

[0170] h t =o t *tanh(c t )

[0171] Where σ is the singmoid activation function, W is the weight parameter, b is the bias parameter, and i t f t o t These are the input gate, forget gate, and output gate, i t Determines whether the corresponding map feature e is newly stored in the cell state, f t Determine whether the corresponding map element 'e' should be deleted from the cell state, o t It determines how the hidden state reflects the current cell state.

[0172] Use the updated hidden state h t Map features for predicting the next data collection time

[0173] In this way, the LSTM model can fully integrate multiple incomplete local maps with prior information from the global map to update the representation of real-time map features.

[0174] Step S42, Trend Prediction. Based on historical change data and current environmental data, predict the trend of change for each point x in the global map. Here, each point x is equivalent to a map feature.

[0175] Specifically, historical change data includes: ① Historical global map: past map versions, recording changes to various map elements (such as ground elements like roads, aerial elements like traffic signs); ② Map update log: detailed records of specific changes in each map update, such as map elements that were retained, deleted, or updated.

[0176] Current environmental data includes: ① Weather conditions: such as sunny, rainy, snowy, etc., which affect road conditions and visibility; ② Traffic flow: current traffic flow and congestion, which can be obtained through vehicle-mounted sensing sensors or traffic monitoring systems; ③ Special events: such as construction, accidents or other temporary events, which may lead to road closures or detours; ④ Time factors: different times of day or different days of week.

[0177] Next, we will process the historical change data and current environmental data as described above:

[0178] ① Data preprocessing: Cleaning, formatting, and synchronizing historical change data and current environmental data to ensure data consistency and availability.

[0179] ② Feature extraction: Extract relevant features from historical change data and current environmental data, such as change frequency, change type, weather patterns, etc.

[0180] ③ Feature selection: Select the features that have the most impact on the prediction of the trend of change. These features will be used as inputs to the prediction model.

[0181] The functions of the first convolutional neural network model include:

[0182] P(x) = f predict (F historical F environmental )

[0183] Among them, F historical F is the feature vector of historical change data. environmental It is the feature vector of the current environmental data.

[0184] An input feature vector can be constructed, which includes feature vectors from historical change data and feature vectors from current environmental data. The input feature vector is represented as x = [F historical F environmental ].

[0185] The above function can be reformulated as:

[0186] P(x)=σ L (W L ·σ L―1 (W L―1 ·…σ1(W1·x+b1)+b2)+…+b L )

[0187] Where, σ i W is the activation function of the i-th layer. i b is the weight parameter of the i-th layer. i These are the bias parameters of the i-th layer. Activation functions are used to introduce nonlinearity; for example, the ReLU function can be used, σ... z =max(0,z).

[0188] Define a loss function to train the first convolutional neural network model so that its predictions closely approximate actual changes.

[0189]

[0190] Where N is the number of training samples, y i It is the actual change label of the i-th sample.

[0191] The gradient descent algorithm is used to update the weight parameters of the first convolutional neural network model according to the loss function L:

[0192]

[0193] Where η is the learning rate. This represents the gradient.

[0194] Furthermore, a confidence level is assigned to the predicted result P(x) (i.e., the trend of change), representing the reliability of the prediction. The formula is as follows:

[0195] Conf(x)=σ(W·P(x)+b)

[0196] Where σ is the sigmoid function, and W and b refer to the weight parameter and bias parameter, respectively.

[0197] Step S43: The second convolutional neural network model combines the outputs of step S41 and step S42 to perform map update processing. The formula is as follows:

[0198] M updated =g(X) fused Change θ (e,M))

[0199] Among them, M updated Represents the updated global map, X fused This indicates the output of step S41, Change. θ (e,M) represents the output of step S42, θ represents the confidence threshold used to constrain the confidence, and g represents the model parameters of the second convolutional neural network model.

[0200] Step S4 enables intelligent map update decisions through predictive updates.

[0201] In some embodiments, to ensure the accuracy of predictive updates under different road conditions, a series of measures can be taken to enhance the generalization ability and adaptability of the algorithm. For example, to ensure the diversity of training data, the collection of training data can cover various possible environmental changes and geographical features, including vehicle perception data under different road conditions such as urban areas, suburbs, mountains, and different climate zones.

[0202] 5) Step S5.

[0203] Through the resource scheduler, the ARMUOA algorithm can improve resource utilization efficiency while ensuring real-time performance and accuracy, meeting the needs of different users and scenarios. The resource scheduler is a key component in the ARMUOA algorithm responsible for managing and allocating computing resources, ensuring that the algorithm can achieve real-time performance under limited resource conditions. Step S5 can be implemented through the following steps S51 and S52.

[0204] Step S51: Allocate resources based on the current system load L and task priority P. Here, "task" broadly refers to various data acquisition and computation tasks involved in the embodiments of this application. The resource scheduler evaluates the current system load L, including CPU utilization, memory usage, I / O operations, etc., and determines the priority P of each task. The priority can be determined based on the urgency, importance, or expected impact of the task. Then, based on the system load and task priority, computing resources are allocated using a round-robin method. The formula is as follows:

[0205] Resources=ResourceScheduler(L,P)

[0206] Step S52, parameter adjustment feedback. Users can adjust algorithm parameters, such as the confidence threshold θ used to constrain confidence. The parameters adjusted by the user through a custom interface are fed back to the resource scheduler, which adjusts its resource allocation strategy based on the new parameters. The resource scheduler dynamically adjusts resource allocation based on real-time monitoring data to cope with changes in system load.

[0207] 6) Step S6.

[0208] By collecting user feedback and performance data through a feedback learning mechanism, the system can perform self-optimization. Step S6 can be implemented through step S61.

[0209] Step S61: Adjust the algorithm parameters based on the performance metric PI and user feedback UF to form a continuous optimization loop. The formula is as follows:

[0210] Params new =AdaptationMode l(Params old ,PI,UF)

[0211] In the formula above, Params refers to the algorithm parameters involved in the ARMUOA algorithm.

[0212] In this way, the system can flexibly handle inconsistencies in perception data from multiple vehicles at different times, and ensure the accuracy and real-time nature of map updates.

[0213] The embodiments of this application can be applied to autonomous vehicles, wherein the autonomous vehicles are equipped with a variety of onboard sensors, including but not limited to:

[0214] ① High-resolution camera, used to capture visual information about the road.

[0215] ②LiDAR (Light Detection and Ranging) is used to generate 3D point clouds of the surrounding environment.

[0216] ③ GPS and inertial measurement unit (IMU) are used for vehicle positioning and attitude determination.

[0217] Based on this, the embodiments of this application also provide, for example... Figure 6 The diagram shown is a flowchart of the map update method, which will combine... Figure 6 The explanation is presented step by step.

[0218] Step P1, Data Acquisition and Synchronization. The vehicle collects data in real time through onboard sensors and synchronizes it using timestamps; the cloud performs noise reduction, correction, and normalization processing on the vehicle data.

[0219] Step P2, Adaptive Weight Allocation. The cloud calculates weights based on the data quality indicators and historical accuracy indicators of the vehicle-side data, and uses a fusion algorithm to combine the weighted data to obtain a global map.

[0220] Step P3, Change Detection and Trend Prediction. The cloud uses a change detection algorithm to compare the currently generated global map with historical global maps to detect changed areas; the cloud applies a recurrent neural network model (i.e., Figure 6 The time series forecasting model shown predicts future changes and updates the global map in advance.

[0221] Step P4: Map data synchronization and distribution. The updated global map will be synchronized from the cloud to the vehicle's internal system and uploaded to a shared server for other vehicles to share.

[0222] Step P5: Feedback, Learning, and Optimization. The cloud-based system determines system load and task priorities based on different city environments and vehicle types, dynamically allocating computing resources. The cloud also collects user feedback and performance monitoring data to adjust and optimize algorithm parameters.

[0223] Figure 6 The application of the ARMUOA algorithm in autonomous vehicles was demonstrated. Through adaptive weight allocation and intelligent resource scheduling, it significantly improved the real-time performance and accuracy of map updates, and enhanced the environmental adaptability and safety of autonomous vehicles.

[0224] The embodiments of this application can achieve at least the following technical effects:

[0225] 1) By fusing global crowdsourced data, more comprehensive map updates can be achieved, enabling the handling of a wider perception range, not just the area around vehicles. During the data fusion process, on the one hand, regional global information, such as geometric smoothness constraints, semantic relevance, and global accuracy consistency, can be fully utilized; on the other hand, multiple data collections can alleviate unavoidable challenges such as accuracy deviations and dynamic occlusion.

[0226] 2) It can dynamically adjust the weights of perception data from multiple vehicles at different times based on data quality indicators and historical accuracy indicators. This enables the algorithm to more intelligently integrate multi-source and multi-trip data and improve the accuracy of map updates.

[0227] 3) Predictive updates can be used to predict potential changes in the road environment in advance based on historical data and trend analysis, and to update the map in advance, thereby improving the foresight and adaptability of the autonomous driving system.

[0228] 4) Dynamically allocate computing resources based on the current computing load and task priority to ensure the real-time performance of the algorithm while reducing computing costs. Improve user experience through parameter adjustment feedback, making the map update service more personalized and accurate.

[0229] 5) Possesses the ability to learn and optimize itself, and can continuously adjust and update strategies based on user feedback and performance indicators to achieve continuous performance improvement.

[0230] 6) Applicable to autonomous vehicles, ensuring that autonomous vehicles have stronger adaptability and safety in complex and ever-changing environments, providing an efficient, reliable and cost-effective map update solution for autonomous vehicles.

[0231] The following continues to describe the exemplary structure of the map updating device 255 provided in the embodiments of this application as a software module. In some embodiments, such as Figure 2 As shown, the software modules stored in the map updating device 255 in the memory 250 may include: a first prediction module 2551, used to perform prediction processing based on historical change data of the global map and current environmental data using a first convolutional neural network model to obtain a first prediction result of map elements in the global map; a second prediction module 2552, used to perform a first fusion processing on portions of the global map and the local map at multiple acquisition times to obtain a fusion result of multiple acquisition times, and perform prediction processing based on the fusion result of multiple acquisition times using a recurrent neural network model to obtain a second prediction result of map elements in the global map; wherein, the local map is determined based on vehicle perception data of vehicles traveling on the global map; and a predictive update module 2553, used to perform map update processing on the global map based on the first prediction result and the second prediction result of map elements in the global map.

[0232] In some embodiments, the predictive update module 2553 is further configured to: perform prediction processing based on the first prediction result and the second prediction result of map elements in the global map using the second convolutional neural network model to obtain the third update prediction result of map elements in the global map; and determine whether to retain, delete or update map elements in the global map based on the third update prediction result of map elements in the global map.

[0233] In some embodiments, the map updating device 255 further includes a feature encoding module for performing feature encoding processing on multiple vehicle perception data to obtain bird's-eye view BEV features corresponding to each of the multiple vehicle perception data; the map updating device 255 further includes a feature decoding module for performing feature decoding processing on the BEV features corresponding to each of the multiple vehicle perception data to obtain local maps corresponding to each of the multiple vehicle perception data; the map updating device 255 further includes a multi-source fusion module for performing a second fusion processing on the local maps corresponding to each of the multiple vehicle perception data to obtain a global map.

[0234] In some embodiments, the feature encoding module is further configured to perform the following processing for any vehicle perception data: when the vehicle perception data is an image acquired by an onboard camera from a specific viewpoint, the image is processed by an image feature extraction model to obtain a feature map of the specific viewpoint, the feature map of the specific viewpoint is processed by viewpoint transformation to obtain image BEV features, and the image BEV features are determined as BEV features for the second fusion processing; when the vehicle perception data is a point cloud acquired by an onboard radar, the point cloud is processed by a point cloud feature extraction model to obtain point cloud BEV features. The features are determined by identifying point cloud BEV features as BEV features for the second fusion process. When any vehicle perception data includes an image acquired by an onboard camera from a specific viewpoint and a point cloud acquired by onboard radar, the image feature extraction model is used to extract features from the image to obtain a feature map from a specific viewpoint. The feature map from the specific viewpoint is then subjected to viewpoint transformation processing to obtain image BEV features. The point cloud feature extraction model is then used to extract features from the point cloud to obtain point cloud BEV features. Finally, the image BEV features and the point cloud BEV features are subjected to a third fusion process to obtain BEV features for the second fusion process.

[0235] In some embodiments, the multi-source fusion module is further configured to: determine the data quality index and historical accuracy index corresponding to each vehicle perception data trip; determine the weight of the local map corresponding to the vehicle perception data trip based on the data quality index and historical accuracy index corresponding to each vehicle perception data trip; and perform weighted summation processing on the local maps corresponding to the multiple vehicle perception data trips based on the weights of the local maps corresponding to the multiple vehicle perception data trips to obtain a global map.

[0236] In some embodiments, the multi-source fusion module is further configured to perform the following processing on any vehicle perception data: determine the data quality index corresponding to any vehicle perception data based on the signal-to-noise ratio and measurement error of the vehicle-mounted perception sensor; wherein the vehicle-mounted perception sensor is used to collect any vehicle perception data; and determine the accuracy index of the vehicle perception data historically collected by the vehicle-mounted perception sensor as the historical accuracy index corresponding to any vehicle perception data.

[0237] In some embodiments, the map updating device 255 further includes a change detection module for performing update detection processing on the global map and the historical global map to obtain update detection results of map elements in the global map; the map updating device 255 further includes a real-time update module for performing map updating processing on the global map based on the update detection results of map elements in the global map.

[0238] This application provides a computer program product or computer program, which includes executable instructions stored in a computer-readable storage medium. The processor of an electronic device reads the executable instructions from the computer-readable storage medium and executes the executable instructions, causing the electronic device to implement the map update method described above in this application.

[0239] This application provides a computer-readable storage medium storing executable instructions, wherein the executable instructions are stored and, when executed by a processor, will cause the processor to implement the map update method provided in this application.

[0240] In some embodiments, the computer-readable storage medium may be a memory such as FRAM, ROM, PROM, EPROM, EEPROM, flash memory, magnetic surface memory, optical disk, or CD-ROM; or it may be a variety of devices including one or any combination of the above-mentioned memories.

[0241] In some embodiments, executable instructions may take the form of a program, software, software module, script, or code, written in any form of programming language (including compiled or interpreted languages, or declarative or procedural languages), and may be deployed in any form, including as a standalone program or as a module, component, subroutine, or other unit suitable for use in a computing environment.

[0242] As an example, executable instructions may, but do not necessarily, correspond to files in a file system. They may be stored as part of a file that holds other programs or data, for example, in one or more scripts in a Hyper Text Markup Language (HTML) document, in a single file dedicated to the program in question, or in multiple collaborating files (e.g., a file that stores one or more modules, subroutines, or code sections).

[0243] As an example, executable instructions can be deployed to execute on a single electronic device, or on multiple electronic devices located at one location, or on multiple electronic devices distributed across multiple locations and interconnected via a communication network.

[0244] The above are merely embodiments of this application and are not intended to limit the scope of protection of this application. Any modifications, equivalent substitutions, and improvements made within the spirit and scope of this application are included within the scope of protection of this application.

Claims

1. A map updating method, characterized in that, include: The first prediction result of the map elements in the global map is obtained by performing prediction processing based on the historical change data of the global map and the current environmental data through the first convolutional neural network model. The global map and the local map are fused together at multiple acquisition times to obtain a fusion result for multiple acquisition times. A recurrent neural network model is then used to perform prediction processing based on the fusion result for multiple acquisition times to obtain a second prediction result for map elements in the global map. The local map is determined based on vehicle perception data of vehicles traveling on the global map. Based on the first and second prediction results of map elements in the global map, map update processing is performed on the global map. This process includes: using a second convolutional neural network model to perform prediction processing based on the first and second prediction results of map elements in the global map, obtaining a third update prediction result for the map elements in the global map; and determining whether to retain, delete, or update the map elements in the global map based on the third update prediction result.

2. The method according to claim 1, characterized in that, Before performing prediction processing using the first convolutional neural network model based on historical change data of the global map and current environmental data, the method further includes: The multiple vehicle perception data are processed by feature encoding to obtain the bird's-eye view BEV features corresponding to the multiple vehicle perception data. The BEV features corresponding to the multiple vehicle perception data are subjected to feature decoding processing to obtain the local maps corresponding to the multiple vehicle perception data. The local maps corresponding to the perception data of the multiple vehicles are subjected to a second fusion process to obtain the global map.

3. The method according to claim 2, characterized in that, The step of performing feature encoding processing on the perception data of multiple vehicles to obtain the bird's-eye view BEV features corresponding to the perception data of the multiple vehicles includes: For any given vehicle perception data, perform the following processing: When the perception data of any vehicle is an image captured by an onboard camera from a specific perspective, the image is processed by an image feature extraction model to obtain a feature map of the specific perspective. The feature map of the specific perspective is then processed by perspective transformation to obtain image BEV features. The image BEV features are then determined as BEV features for the second fusion process. The specific perspective is either a forward-looking perspective or a surround-view perspective. When the perception data of any vehicle is a point cloud collected by vehicle radar, the point cloud is processed by the point cloud feature extraction model to obtain point cloud BEV features, and the point cloud BEV features are determined as BEV features for the second fusion processing. When any vehicle perception data includes an image acquired by an onboard camera from a specific viewpoint and a point cloud acquired by onboard radar, the image is processed by an image feature extraction model to obtain a feature map of the specific viewpoint. The feature map of the specific viewpoint is then processed by a viewpoint transformation to obtain image BEV features. The point cloud is then processed by a point cloud feature extraction model to obtain point cloud BEV features. Finally, the image BEV features and the point cloud BEV features are processed by a third fusion process to obtain BEV features for the second fusion process.

4. The method according to claim 2, characterized in that, The second fusion process, which involves performing a second fusion process on the local maps corresponding to the perception data of the multiple vehicles to obtain the global map, includes: Determine the data quality indicators and historical accuracy indicators corresponding to the perception data of each vehicle trip. The weight of the local map corresponding to the perception data of each vehicle is determined based on the data quality index and historical accuracy index corresponding to the perception data of each vehicle. The global map is obtained by weighting and summing the local maps corresponding to the multiple vehicle perception data based on their respective weights.

5. The method according to claim 4, characterized in that, The determination of the data quality indicators and historical accuracy indicators corresponding to the perception data of each vehicle trip includes: For any given vehicle perception data, perform the following processing: Based on the signal-to-noise ratio and measurement error of the vehicle-mounted sensing sensor, the data quality index corresponding to the sensing data of any given vehicle trip is determined; wherein, the vehicle-mounted sensing sensor is used to collect the sensing data of any given vehicle trip. The accuracy index of the vehicle perception data historically collected by the vehicle-mounted perception sensor is determined, and used as the historical accuracy index for any one trip of vehicle perception data.

6. The method according to claim 2, characterized in that, After performing a second fusion process on the local maps corresponding to the perception data of the multiple vehicles to obtain the global map, the method further includes: The global map and the historical global map are subjected to update detection processing to obtain the update detection results of map elements in the global map; The global map is updated based on the update detection results of map features in the global map.

7. A map updating device, characterized in that, include: The first prediction module is used to perform prediction processing based on historical change data of the global map and current environmental data through a first convolutional neural network model to obtain the first prediction result of map elements in the global map. The second prediction module is used to perform a first fusion process on portions of the global map and the local map at multiple acquisition times to obtain a fusion result of multiple acquisition times, and then perform prediction processing based on the fusion result of multiple acquisition times using a recurrent neural network model to obtain a second prediction result of map elements in the global map; wherein, the local map is determined based on vehicle perception data of vehicles traveling on the global map. The predictive update module is used to perform map update processing on the global map based on the first and second prediction results of map elements in the global map. When performing map update processing on the global map based on the first and second prediction results of map elements in the global map, the predictive update module is used to: perform prediction processing based on the first and second prediction results of map elements in the global map using a second convolutional neural network model to obtain a third update prediction result of map elements in the global map; and determine whether to retain, delete, or update map elements in the global map based on the third update prediction result of map elements in the global map.

8. An electronic device, characterized in that, include: Memory, used to store executable instructions; A processor, when executing executable instructions stored in the memory, implements the method according to any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that, It stores executable instructions for implementing the method of any one of claims 1 to 6 when executed by a processor.

Images