Map updating method and device, electronic equipment and computer readable storage medium

Abstract

The invention provides a map updating method and device, electronic equipment, a computer readable storage medium and a computer program product. The method comprises the steps of performing prediction processing based on historical change data of a global map and current environment data through a first convolutional neural network model to obtain a first prediction result of map elements in the global map; performing first fusion processing on the global map and the part of the local map at the plurality of collection times to obtain a fusion result of the plurality of collection times, and performing prediction processing based on the fusion result of the plurality of collection times through a recurrent neural network model to obtain a second prediction result of map elements in the global map; wherein the local map is determined according to the vehicle perception data of the vehicle driving on the global map; and performing map updating processing on the global map according to the first prediction result and the second prediction result of the map elements in the global map. According to the method, the timeliness and accuracy of map updating can be remarkably improved.

Description

Technical Field

[0001] This application relates to computer technology, and in particular, to a map update method, apparatus, electronic device, computer-readable storage medium, and computer program product. Background Art

[0002] With the rapid development of autonomous driving technology, the requirements for the accuracy and timeliness of map information are increasing day by day. Most traditional map update schemes adopt a reactive update mechanism, that is, they rely on data collection and update after actual changes occur, such as achieving map update through on-site surveys, user feedback, or officially released change information.

[0003] However, when environmental changes such as road construction, road closures, and accidents occur in a certain area, traditional map update schemes often fail to update the map in a timely manner, which not only leads to a certain lag in map information but also may affect the accuracy and reliability of related applications such as navigation and location services that rely on map information. Summary of the Invention

[0004] This application provides a map update method, apparatus, electronic device, computer-readable storage medium, and computer program product, which significantly improve the timeliness and accuracy of map update by introducing a predictive update mechanism.

[0005] The technical solution of this application is implemented as follows:

[0006] This application provides a map update method, including:

[0007] Performing prediction processing on historical change data of a global map and current environmental data through a first convolutional neural network model to obtain a first prediction result of map elements in the global map;

[0008] Performing first fusion processing on the global map and parts of a local map at multiple acquisition times to obtain fusion results at multiple acquisition times, and performing prediction processing on the fusion results at multiple acquisition times through a recurrent neural network model to obtain a second prediction result of map elements in the global map; wherein, the local map is determined according to vehicle perception data of a vehicle traveling on the global map;

[0009] Performing map update processing on the global map according to the first prediction result and the second prediction result of map elements in the global map.

[0010] This application provides a map update apparatus, including:

[0011] The first prediction module is used to perform prediction processing based on the historical change data of the global map and the current environmental data through the first convolutional neural network model, and obtain the first prediction result of the map elements in the global map;

[0012] The second prediction module is used to perform first fusion processing on the global map and parts of the local map at multiple acquisition times to obtain fusion results at multiple acquisition times, and perform prediction processing based on the fusion results at multiple acquisition times through a recurrent neural network model to obtain the second prediction result of the map elements in the global map; wherein, the local map is determined according to the vehicle perception data of the vehicle driving on the global map;

[0013] The predictive update module is used to perform map update processing on the global map according to the first prediction result and the second prediction result of the map elements in the global map.

[0014] This application provides an electronic device, including:

[0015] A memory for storing executable instructions;

[0016] A processor, when executing the executable instructions stored in the memory, implements the map update method provided by this application.

[0017] This application provides a computer-readable storage medium storing executable instructions for causing a processor to implement the map update method provided by this application when executed.

[0018] This application provides a computer program product, which includes executable instructions for causing a processor to implement the map update method provided by this application when executed.

[0019] This application has the following beneficial effects:

[0020] This application performs prediction processing based on the historical change data of the global map and the current environmental data through the first convolutional neural network model to obtain the first prediction result of the map elements in the global map; performs first fusion processing on the global map and parts of the local map at multiple acquisition times to obtain fusion results at multiple acquisition times, and performs prediction processing based on the fusion results at multiple acquisition times through a recurrent neural network model to obtain the second prediction result of the map elements in the global map; wherein, the local map is determined according to the vehicle perception data of the vehicle driving on the global map; performs map update processing on the global map according to the first prediction result and the second prediction result of the map elements in the global map. This application can effectively combine the update prediction results respectively output by the two models and perform map update in advance, which can improve the accuracy of map update, thereby helping to improve the forward-looking and adaptability of the autonomous driving system. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following will briefly introduce the drawings required for the description of the embodiments. Obviously, the drawings in the following description are only some embodiments of the present application. For those skilled in the art, without creative efforts, other drawings can be obtained based on these drawings.

[0022] Figure 1 It is the first schematic structural diagram of the map update system provided by the embodiments of the present application;

[0023] Figure 2 It is a schematic structural diagram of a server provided by the embodiments of the present application;

[0024] Figure 3A It is the first schematic flowchart of the map update method provided by the embodiments of the present application;

[0025] Figure 3B It is the second schematic flowchart of the map update method provided by the embodiments of the present application;

[0026] Figure 3C It is the third schematic flowchart of the map update method provided by the embodiments of the present application;

[0027] Figure 4 It is the second schematic structural diagram of the map update system provided by the embodiments of the present application;

[0028] Figure 5 It is the fourth schematic flowchart of the map update method provided by the embodiments of the present application;

[0029] Figure 6 It is the fifth schematic flowchart of the map update method provided by the embodiments of the present application. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0030] In order to make the purpose, technical solutions and advantages of the present application clearer, the following will further describe the present application in detail with reference to the drawings. The described embodiments should not be regarded as limitations on the present application. All other embodiments obtained by those of ordinary skill in the art without creative efforts fall within the scope of protection of the present application.

[0031] In the following description, "some embodiments" are involved, which describe a subset of all possible embodiments. However, it can be understood that "some embodiments" can be the same subset or different subsets of all possible embodiments, and can be combined with each other without conflict. In the following description, the term "a plurality" refers to at least two.

[0032] In the following description, the terms "first", "second", and "third" are only used to distinguish similar objects and do not represent a specific order for the objects. Understandably, "first", "second", and "third" can be interchanged in a specific order or sequence when permitted, so that the embodiments of the present application described herein can be implemented in an order other than that illustrated or described herein.

[0033] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by those of ordinary skill in the technical field to which this application belongs. The terms used herein are only for the purpose of describing the embodiments of this application and are not intended to limit this application.

[0034] Embodiments of the present application provide a map update method, apparatus, electronic device, computer-readable storage medium, and computer program product. By introducing a predictive update mechanism, the timeliness and accuracy of map updates are significantly improved. The following describes an exemplary application of the electronic device provided by the embodiments of the present application. The electronic device provided by the embodiments of the present application can be implemented as various types of terminal devices, such as vehicle-mounted devices, or can also be implemented as a server.

[0035] See Figure 1 , Figure 1 FIG. 13 is a schematic diagram of the architecture of the map update system 100 provided by the embodiments of the present application. The vehicle-mounted device 400 is connected to the server 200 through the network 300. Among them, the network 300 can be a wide area network, a local area network, or a combination of the two.

[0036] In some embodiments, the map update method provided by the embodiments of the present application can be implemented by the server. For example, the server 200 performs prediction processing on the historical change data of the global map and the current environmental data based on the first convolutional neural network model to obtain the first prediction result of the map elements in the global map; the server 200 performs the first fusion processing on the global map and the parts of the local map at multiple acquisition times to obtain the fusion results at multiple acquisition times, and performs prediction processing on the fusion results at multiple acquisition times based on the recurrent neural network model to obtain the second prediction result of the map elements in the global map; wherein, the local map is determined according to the vehicle perception data of the vehicle traveling on the global map; the server 200 performs map update processing on the global map according to the first prediction result and the second prediction result of the map elements in the global map.

[0037] In some embodiments, the map update method provided by the embodiments of the present application can be implemented in cooperation with a terminal device and a server. For example, the vehicle-mounted device 400 can collect multi-trip vehicle perception data through vehicle-mounted perception sensors, generate local maps according to each trip of vehicle perception data, and send the local maps corresponding to each trip of vehicle perception data to the server 200; the server 200 performs a second fusion process on the local maps corresponding to each trip of vehicle perception data to obtain a global map. It should be noted that the above process is exemplified by the multi-trip vehicle perception data all originating from the vehicle-mounted device 400. In some embodiments, the multi-trip vehicle perception data can also originate from different vehicle-mounted devices respectively. For another example, after the server 200 performs map update processing on the global map, it can send the updated global map to the vehicle-mounted device 400, so that the vehicle-mounted device 400 can perform autonomous driving according to the updated global map, thereby improving the forward-looking and adaptability of autonomous driving.

[0038] In some embodiments, the terminal device or the server can implement the map update method provided by the embodiments of the present application by running a computer program. For example, the computer program can be a native program or software module in the operating system; it can be a local (Native) application (APP, Application), that is, a program that needs to be installed in the operating system to run; it can also be a small program, that is, a program that only needs to be downloaded to the browser environment to run; it can also be a small program that can be embedded into any APP, and the small program can be controlled by the user to run or close. In short, the above computer program can be any form of application program, module or plug-in.

[0039] Taking the electronic device provided by the embodiments of the present application as a server as an example, see Figure 2 , Figure 2 is a schematic structural diagram of the server 200 provided by the embodiments of the present application. Figure 2 The server 200 shown includes: at least one processor 210, a memory 250, and at least one network interface 220. Each component in the server 200 is coupled together through a bus system 240. It can be understood that the bus system 240 is used to realize the connection and communication between these components. In addition to the data bus, the bus system 240 also includes a power bus, a control bus, and a status signal bus. However, for the sake of clear illustration, in Figure 2 all kinds of buses are labeled as the bus system 240.

[0040] The processor 210 may be an integrated circuit chip with signal processing capabilities, such as a general-purpose processor, a digital signal processor (DSP), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. Among them, the general-purpose processor may be a microprocessor or any conventional processor, etc.

[0041] The memory 250 may be removable, non-removable, or a combination thereof. Exemplary hardware devices include solid-state memory, hard disk drives, optical disc drives, etc. The memory 250 optionally includes one or more storage devices that are physically located far from the processor 210.

[0042] The memory 250 includes volatile memory or non-volatile memory, and may also include both volatile and non-volatile memory. The non-volatile memory may be a read-only memory (ROM), and the volatile memory may be a random access memory (RAM). The memory 250 described in the embodiments of the present application is intended to include any suitable type of memory.

[0043] In some embodiments, the memory 250 is capable of storing data to support various operations. Examples of such data include programs, modules, and data structures, or subsets or supersets thereof, which are illustrated below.

[0044] The operating system 251 includes system programs for processing various basic system services and performing hardware-related tasks, such as the framework layer, the core library layer, the driver layer, etc., for implementing various basic services and processing hardware-based tasks;

[0045] The network communication module 252 is used to reach other computing devices via one or more (wired or wireless) network interfaces 220. Exemplary network interfaces 220 include: Bluetooth, Wi-Fi (Wireless Fidelity), and Universal Serial Bus (USB), etc.;

[0046] In some embodiments, the map update device provided by the embodiments of the present application may be implemented in software. Figure 2 The map update device 255 stored in the memory 250 is shown, which may be software in the form of programs and plugins, etc., and includes the following software modules: a first prediction module 2551, a second prediction module 2552, and a predictive update module 2553. These modules are logical, and thus can be combined or further split arbitrarily according to the functions implemented. The functions of each module will be described below.

[0047] The map updating method provided in the embodiment of the present application will be described in combination with the exemplary application and implementation of the electronic device provided in the embodiment of the present application.

[0048] See also Figure 3A , Figure 3A is a flowchart of a map updating method provided in an embodiment of the present application. The map updating method provided in an embodiment of the present application can be implemented by an electronic device, such as a terminal device and / or a server. Figure 3A The steps shown are explained.

[0049] In step 101, a first convolutional neural network model is used to perform prediction processing based on historical change data of a global map and current environmental data to obtain a first prediction result of a map element in the global map.

[0050] Here, the historical change data of the global map and the current environment data are obtained. Among them, the historical change data is used to represent the historical changes of map elements, for example, the historical change data includes: 1) Historical global map: past map versions, recording the changes of various map elements (such as ground elements such as roads, aerial elements such as traffic signs); 2) Map update log: detailed records of specific changes in each map update, such as map elements retained, deleted or updated. The current environment data is used to represent the current environment (such as weather, roads, time and other factors). For example, the current environment data includes: 1) Weather conditions: such as sunny, rainy, snowy, etc., weather conditions will affect road conditions and visibility; 2) Traffic flow: current traffic flow and congestion, this information can be obtained through vehicle-mounted perception sensors or traffic monitoring systems; 3) Special events: such as construction, accidents or other temporary events, which may cause road closures or diversions; 4) Time factors: different times of the day or different days of the week.

[0051] Then, the first convolutional neural network model is used to perform prediction processing based on the historical change data of the global map and the current environmental data to obtain a first prediction result of the map element in the global map, wherein the first prediction result is used to indicate the retention, deletion or update of the map element. Prior to this, the first convolutional neural network model can be trained to improve the accuracy of the prediction processing.

[0052] It is worth noting that the map elements involved in the embodiments of the present application generally refer to various points in the global map; the global map refers to the map described by the global coordinate system (geodetic coordinate system), and the local map mentioned below refers to the map described by the vehicle coordinate system.

[0053] It should be noted that convolutional neural networks (CNNs) are a type of feedforward neural network with a deep structure that includes convolutional computations. They automatically extract features from data and make predictions through components such as convolutional layers, pooling layers, and fully-connected layers. Convolutional neural network models are suitable for processing data with a grid structure, and

[0054] In some embodiments, the first convolutional neural network model can also output a confidence level for the first prediction result. The confidence level is used to reflect the credibility of the first prediction result, facilitating more accurate and effective decision-making in the subsequent map update process of step 103.

[0055] In step 102, the global map is first fused with parts of the local map at multiple acquisition times to obtain fusion results at multiple acquisition times, and a prediction process is performed on the fusion results at multiple acquisition times through a recurrent neural network model to obtain a second prediction result of the map elements in the global map; wherein, the local map is determined based on the vehicle perception data of the vehicle traveling on the global map.

[0056] For example, vehicle perception data is collected through in-vehicle perception sensors of the vehicle traveling on the global map, and a local map described in the vehicle coordinate system is determined based on the vehicle perception data. The global map is first fused with parts of the local map at multiple acquisition times (or time steps) (such as stitching processing) to obtain fusion results at multiple acquisition times. For example, the global map is first fused with parts of the local map at the first acquisition time to obtain a fusion result at the first acquisition time. In this way, the prior information in the global map is taken into consideration, and the missing information in the local map (such as lane lines being blocked or blurred by other vehicles) is supplemented.

[0057] Then, a prediction process is performed on the fusion results at multiple acquisition times through a recurrent neural network model to obtain a second prediction result of the map elements in the global map. In this way, the change of map data over time is simulated through time series analysis, where the second prediction result is also used to indicate the retention, deletion, or update of map elements. Before this, the recurrent neural network model can be trained to improve the prediction effect.

[0058] It should be noted that the Recurrent Neural Network (RNN) model is a neural network model for processing sequential data. Its characteristic is that there are recurrent connections in the network structure, enabling information to be circulated and transmitted in the network. Structurally, the recurrent neural network model mainly consists of an input layer, a hidden layer, and an output layer. Different from the traditional feedforward neural network model, there are recurrent connections between the neurons in the hidden layer of the recurrent neural network model.

[0059] In some embodiments, the recurrent neural network model may be a Long Short-Term Memory (LSTM) model.

[0060] It should be noted that vehicle perception data can be divided according to the acquisition time (collection time), so the local maps generated based on the vehicle perception data can also be divided according to the acquisition time.

[0061] In some embodiments, if there are local maps corresponding to multiple trips of vehicle perception data, the above step 102 can be executed multiple times, so as to comprehensively obtain a more accurate second prediction result by combining the global map and the local maps corresponding to multiple trips of vehicle perception data.

[0062] In step 103, according to the first prediction result and the second prediction result of the map elements in the global map, map update processing is performed on the global map.

[0063] Here, performing map update processing on the global map according to the first prediction result and the second prediction result of the map elements in the global map means making decisions on the retention, deletion, or update of the map elements in the global map by comprehensively considering the first prediction result and the second prediction result of the map elements in the global map. The above map update processing can be implemented through a preset decision rule. For example, for a certain map element in the global map, when the first prediction result and the second prediction result of the map element are the same, the map element is processed according to the same result (for example, if the first prediction result and the second prediction result of the map element are both deletion, then the map element is deleted in the global map); when the first prediction result and the second prediction result of the map element are different, the map element remains unchanged in the global map, that is, the map element is retained.

[0064] In some embodiments, the above-mentioned map update processing of the global map according to the first prediction result and the second prediction result of the map elements in the global map can be achieved in the following manner: The second convolutional neural network model performs prediction processing based on the first prediction result and the second prediction result of the map elements in the global map to obtain the third updated prediction result of the map elements in the global map; according to the third updated prediction result of the map elements in the global map, determine the retention, deletion or update of the map elements in the global map.

[0065] Here, the first prediction result and the second prediction result of the map elements in the global map can be input into the second convolutional neural network model, so that the second convolutional neural network model performs prediction processing to obtain the third updated prediction result of the map elements in the global map. Then, according to the third updated prediction result of the map elements in the global map, determine the retention, deletion or update of the map elements in the global map to obtain the updated global map. For example, if the third updated prediction result of the map elements is retention, then a decision to retain the map elements is made in the global map, and so on. Before this, the second convolutional neural network model can be trained to improve the effect of prediction processing.

[0066] The above method does not rely on fixed decision rules, but realizes automatic decision-making through the second convolutional neural network model, and can achieve more accurate map update processing.

[0067] In some embodiments, autonomous driving can be performed according to the updated global map. Taking the implementation of the above steps 101 to 103 in the server as an example, the server can send the updated global map to multiple vehicle-mounted devices, so that each vehicle-mounted device performs autonomous driving according to the updated global map. Since the future changes of the map elements can be accurately predicted through the above steps 101 to 103, the foresight and adaptability of autonomous driving can be improved.

[0068] Such as Figure 3AAs shown in the figure, in the embodiment of the present application, the first convolutional neural network model performs prediction processing based on the historical change data of the global map and the current environmental data to obtain the first prediction result of the map elements in the global map; the global map is respectively subjected to the first fusion processing with the parts of the local map at multiple acquisition times to obtain the fusion results at multiple acquisition times, and the recurrent neural network model performs prediction processing based on the fusion results at multiple acquisition times to obtain the second prediction result of the map elements in the global map; wherein, the local map is determined according to the vehicle perception data of the vehicle driving on the global map; according to the first prediction result and the second prediction result of the map elements in the global map, the global map is updated. The embodiment of the present application can effectively combine the updated prediction results respectively output by the two models and perform map update in advance, which can improve the accuracy of map update, thereby helping to improve the forward-looking and adaptability of the automatic driving system.

[0069] In some embodiments, referring to Figure 3B , Figure 3B is a schematic flowchart of a map update method provided by an embodiment of the present application. Based on Figure 3A , before step 101, steps 201 to 203 may also be executed.

[0070] In step 201, feature encoding processing is performed on multiple trips of vehicle perception data to obtain the bird's-eye view (BEV) features corresponding to the multiple trips of vehicle perception data respectively.

[0071] Here, multiple trips of vehicle perception data can be collected. For the i-th trip of vehicle perception data, feature encoding processing is performed on the i-th trip of vehicle perception data to obtain the bird's-eye view (BEV) feature corresponding to the i-th trip of vehicle perception data.

[0072] It should be noted that the multiple trips of vehicle perception data can be from the same vehicle or from different vehicles respectively; the multiple trips of vehicle perception data can be collected on the same driving path or on different driving paths respectively, and this is not limited.

[0073] In some embodiments, the above-mentioned feature encoding process for multi-trip vehicle perception data can be implemented in the following manner to obtain the bird's-eye view (BEV) features corresponding to each trip of vehicle perception data: For any trip of vehicle perception data, the following processing is performed: When any trip of vehicle perception data is an image collected by an in-vehicle camera from a specific perspective, the image is subjected to feature extraction processing by an image feature extraction model to obtain a feature map of the specific perspective, and the feature map of the specific perspective is subjected to perspective transformation processing to obtain the image BEV feature, and the image BEV feature is determined as the BEV feature for the second fusion processing; When any trip of vehicle perception data is a point cloud collected by an in-vehicle radar, the point cloud is subjected to feature extraction processing by a point cloud feature extraction model to obtain the point cloud BEV feature, and the point cloud BEV feature is determined as the BEV feature for the second fusion processing; When any trip of vehicle perception data includes an image collected by an in-vehicle camera from a specific perspective and a point cloud collected by an in-vehicle radar, the image is subjected to feature extraction processing by an image feature extraction model to obtain a feature map of the specific perspective, the feature map of the specific perspective is subjected to perspective transformation processing to obtain the image BEV feature, the point cloud is subjected to feature extraction processing by a point cloud feature extraction model to obtain the point cloud BEV feature, and the image BEV feature and the point cloud BEV feature are subjected to a third fusion processing to obtain the BEV feature for the second fusion processing.

[0074] For ease of understanding, the feature encoding process is described by taking the i-th trip of vehicle perception data as an example. According to different types of in-vehicle perception sensors, the feature encoding process may include the following three cases:

[0075] 1) The i-th trip of vehicle perception data is an image collected by an in-vehicle camera from a specific perspective (such as the front view perspective or the surround view perspective). At this time, the image is subjected to feature extraction processing by an image feature extraction model to obtain a feature map of the specific perspective, and the feature map of the specific perspective is subjected to perspective transformation processing to obtain the image BEV feature, and the image BEV feature is determined as the BEV feature for the second fusion processing. Among them, the type of the image feature extraction model is not limited. For example, it can be a 2D backbone network, such as ResNet, etc., or a combination of a 2D backbone network and a neck network, such as ResNet+FPN, etc.; The perspective transformation processing is used to transform from the perspective view (PV) to the bird's-eye view, and can adopt a geometry-based method, such as inverse perspective mapping (IPM), or a depth-based method, such as LSS (Lift, splat, shoot), or a Transformer-based method, such as deformable attention, etc.

[0076] 2) The i-th vehicle perception data is a point cloud collected by the vehicle-mounted radar. At this time, the point cloud is subjected to feature extraction processing by the point cloud feature extraction model to obtain the point cloud BEV feature, and the point cloud BEV feature is determined as the BEV feature for the second fusion processing. The type of the point cloud feature extraction model is not limited, for example, it can be a 3D backbone network, such as PointPillars, SECOND, etc.

[0077] 3) The i-th vehicle perception data includes images acquired by the vehicle-mounted camera at a specific viewing angle and point clouds acquired by the vehicle-mounted radar. At this point, on the one hand, the image is subjected to feature extraction processing by the image feature extraction model to obtain a feature map at a specific viewing angle, and the feature map at a specific viewing angle is subjected to perspective conversion processing to obtain image BEV features; on the other hand, the point cloud is subjected to feature extraction processing by the point cloud feature extraction model to obtain point cloud BEV features. Finally, the image BEV features and the point cloud BEV features are subjected to a third fusion processing (such as splicing processing) to obtain BEV features for the second fusion processing.

[0078] The above method implements feature encoding processing in a targeted manner according to the type of on-board perception sensor, and can extract accurate and effective BEV features from vehicle perception data.

[0079] In step 202, feature decoding processing is performed on the BEV features corresponding to the plurality of vehicle perception data to obtain the local maps corresponding to the plurality of vehicle perception data.

[0080] Here, the BEV features corresponding to the i-th vehicle perception data are subjected to feature decoding processing to obtain a local map corresponding to the i-th vehicle perception data, wherein the local map is represented by a vectorized road structure. It is worth noting that the feature decoding processing can be implemented by a feature decoder. During the feature decoding processing, the feature decoder uses the BEV features to perform 3D target detection and road structure classification and positioning tasks, that is, the feature decoder generates output embeddings through interaction with the BEV features. These output embeddings contain the position and shape information of the target object in the BEV space. The above-mentioned feature decoding processing involves a self-attention mechanism and a cross-attention mechanism to ensure that different BEV features can be effectively integrated to obtain an accurate vectorized road structure.

[0081] In step 203, a second fusion process is performed on the local maps corresponding to the plurality of vehicle perception data to obtain a global map.

[0082] The local map obtained through step 202 is described in the vehicle coordinate system. Therefore, a second fusion process is performed on the local maps corresponding to multiple trips of vehicle perception data to obtain a global map described in the global coordinate system. It should be noted that the above second fusion process is implemented for the parts of the local maps corresponding to multiple trips of vehicle perception data in the same local area, that is, the fusion is performed separately in each local area.

[0083] In some embodiments, the above-mentioned second fusion process for the local maps corresponding to multiple trips of vehicle perception data to obtain a global map can be implemented in the following way: Determine the data quality index and historical accuracy index corresponding to each trip of vehicle perception data; According to the data quality index and historical accuracy index corresponding to each trip of vehicle perception data, determine the weight of the local map corresponding to that trip of vehicle perception data; According to the weights of the local maps corresponding to multiple trips of vehicle perception data, perform a weighted summation process on the local maps corresponding to multiple trips of vehicle perception data to obtain a global map.

[0084] Here, for the i-th trip of vehicle perception data, the data quality index and historical accuracy index corresponding to the i-th trip of vehicle perception data can be determined. Among them, the data quality index corresponding to the i-th trip of vehicle perception data is used to represent the data quality of the i-th trip of vehicle perception data itself, and the historical accuracy index corresponding to the i-th trip of vehicle perception data is used to represent the accuracy of the vehicle perception data before the i-th trip of vehicle perception data (both are collected by the same on-vehicle perception sensor in the same vehicle). Then, according to the data quality index and historical accuracy index corresponding to the i-th trip of vehicle perception data, determine the weight of the local map corresponding to the i-th trip of vehicle perception data. Among them, the weight of the local map corresponding to the i-th trip of vehicle perception data is positively correlated with the data quality index corresponding to the i-th trip of vehicle perception data, and the weight of the local map corresponding to the i-th trip of vehicle perception data is positively correlated with the historical accuracy index corresponding to the i-th trip of vehicle perception data. For example, a weighted summation process (the weights can be preset in advance) can be performed on the data quality index and historical accuracy index corresponding to the i-th trip of vehicle perception data to obtain the weight of the local map corresponding to the i-th trip of vehicle perception data. Then, according to the weights of the local maps corresponding to multiple trips of vehicle perception data, perform a weighted summation process on the local maps corresponding to multiple trips of vehicle perception data to obtain a global map.

[0085] The above method comprehensively considers the real-time data quality and historical accuracy, assigns weights to the local maps and then performs a weighted summation process, which can improve the accuracy of the finally obtained global map.

[0086] In some embodiments, the above-mentioned determination of the data quality index and the historical accuracy index corresponding to each trip of vehicle perception data can be achieved in the following manner: For any trip of vehicle perception data, perform the following processing: Determine the data quality index corresponding to any trip of vehicle perception data according to the signal-to-noise ratio and measurement error of the in-vehicle perception sensor; wherein, the in-vehicle perception sensor is used to collect any trip of vehicle perception data; Determine the accuracy index of the vehicle perception data historically collected by the in-vehicle perception sensor as the historical accuracy index corresponding to any trip of vehicle perception data.

[0087] Taking the i-th trip of vehicle perception data as an example, on the one hand, determine the data quality index corresponding to the i-th trip of vehicle perception data according to the signal-to-noise ratio and measurement error of the in-vehicle perception sensor used to collect the i-th trip of vehicle perception data. Among them, the data quality index corresponding to the i-th trip of vehicle perception data is positively correlated with the signal-to-noise ratio of the in-vehicle perception sensor, and the data quality index corresponding to the i-th trip of vehicle perception data is negatively correlated with the measurement error of the in-vehicle perception sensor. The measurement error of the in-vehicle perception sensor refers to the error (such as standard deviation) between the measured value and the true value (the true value can be obtained through high-precision maps or surveying and mapping); on the other hand, determine the accuracy index of the vehicle perception data historically collected by the in-vehicle perception sensor used to collect the i-th trip of vehicle perception data as the historical accuracy index corresponding to the i-th trip of vehicle perception data. Among them, the accuracy index can be reflected as the ratio of the number of correct predictions to the total number of predictions within a certain time period. Here, the prediction refers to predicting the changes of map elements, and the correct prediction means that the predicted changes are consistent with the actual changes.

[0088] In the above manner, by comprehensively evaluating the data quality through the signal-to-noise ratio and measurement error, an accurate and reliable data quality index can be obtained; determining the accuracy index of the historically collected vehicle perception data as the historical accuracy index can accurately evaluate the past performance of the i-th trip of vehicle perception data.

[0089] As Figure 3B shown, the embodiments of the present application perform feature encoding processing on multiple trips of vehicle perception data to obtain the bird's-eye view BEV features corresponding to multiple trips of vehicle perception data respectively; perform feature decoding processing on the BEV features corresponding to multiple trips of vehicle perception data respectively to obtain the local maps corresponding to multiple trips of vehicle perception data respectively; perform a second fusion process on the local maps corresponding to multiple trips of vehicle perception data respectively to obtain a global map. In this way, on the one hand, it can make full use of regional global information, such as geometric smoothness constraints, semantic relevance, and global accuracy consistency; on the other hand, by collecting multiple trips of vehicle perception data, inevitable challenges such as accuracy deviation and dynamic occlusion can be alleviated.

[0090] In some embodiments, refer to Figure 3C ,Figure 3C is a schematic flowchart of a map update method provided by an embodiment of the present application. Based on Figure 3B , after step 203, steps 301 to 302 can also be executed.

[0091] In step 301, update detection processing is performed on the global map and the historical global map to obtain the update detection result of the map elements in the global map.

[0092] After performing second fusion processing on the local maps corresponding to multiple trips of vehicle perception data to obtain the global map, update detection processing can be performed on the global map and the historical global map to obtain the update detection result of the map elements in the global map, where the historical global map refers to the global map of the historical version.

[0093] In some embodiments, the update detection processing is used to detect at least one of geometric changes, attribute changes, and topological relationship changes. The following will be described separately.

[0094] 1) Geometric changes. Geometric changes refer to situations where there are new map elements, deleted map elements, or changes in the position or shape of map elements in the global map relative to the historical global map. For example, the matching degree between a certain map element in the global map and the historical global map can be calculated, and the existence probability of the map element can be estimated based on the matching degree. When the existence probability of the map element is greater than the preset existence probability threshold, it is determined that the map element meets the condition of new map element addition; when the existence probability of the map element is less than or equal to the existence probability threshold, it is determined that the map element meets the condition of map element deletion; when the matching degree between the map element and the historical global map is less than or equal to the preset matching degree threshold, it is determined that the map element meets the condition of map element position or shape change.

[0095] 2) Attribute changes. Attribute changes refer to situations where there are attribute changes in the map elements in the global map relative to the map elements in the same position in the historical global map. Attributes such as the number of lanes, road names, etc. are not limited herein.

[0096] 3) Topological relationship changes. Topological relationship changes refer to situations where the topological relationship between map elements in the global map has changed relative to the topological relationship between map elements in the historical global map, where the topological relationship includes connection, intersection, inclusion, etc.

[0097] In step 302, map update processing is performed on the global map according to the update detection result of the map elements in the global map.

[0098] Here, the global map is updated according to the update detection results of map elements in the global map, that is, the retention, deletion, or update of map elements is determined according to the update detection results of map elements in the global map.

[0099] As Figure 3C shown, in the embodiment of the present application, update detection processing is performed on the global map and the historical global map to obtain the update detection results of map elements in the global map; the global map is updated according to the update detection results of map elements in the global map. In this way, the changed area is identified by comparing the global map with the historical global map, so as to achieve accurate map update processing.

[0100] Next, an exemplary application of the embodiment of the present application in an actual application scenario will be described. The embodiment of the present application proposes an Adaptive Real-time Map Update Optimization Algorithm (ARMUOA), aiming to solve the limitations of traditional map update methods in terms of efficiency, accuracy, real-time performance, and resource optimization.

[0101] As an example, the embodiment of the present application provides an architecture schematic diagram of a map update system as Figure 4 shown, and a flow schematic diagram of a map update method as Figure 5 shown. Combining Figure 4 and Figure 5 , the embodiment of the present application can be implemented through the following steps.

[0102] Step S1, data preprocessing: Collect multi-trip vehicle perception data, and clean and standardize the input data through data preprocessing.

[0103] Step S2, adaptive weight assignment: Assign weights according to data quality and historical accuracy, and perform feature extraction and fusion on the weighted data.

[0104] Step S3, change detection: Identify the changed area in the global map to make an update decision.

[0105] Step S4, predictive update: Predict and update in advance the areas that may change in the global map.

[0106] Step S5, resource scheduling: Optimize the calculation resource allocation to ensure real-time performance; through the user-defined interface, adjust the algorithm parameters according to the requirements.

[0107] Step S6, feedback learning: Collect user feedback and performance data for self-optimization.

[0108] Next, the above steps will be elaborated in detail.

[0109] 1) Step S1.

[0110] Step S1 is realized through the cooperation of the vehicle end and the cloud end.

[0111] For the vehicle end, vehicle perception data is collected through in-vehicle perception sensors, and the vectorized road structure (here refers to the local map) is extracted from it. According to the differences in the installation configuration and data modality of the in-vehicle perception sensors, the operations performed by the vehicle end can be realized in one of the following ways.

[0112] The first way is applicable when only in-vehicle cameras are used as in-vehicle perception sensors. At this time, the feature encoder includes an image feature extraction model and a perspective transformation module. The specific steps are as follows:

[0113] Step S11a: The images collected by the in-vehicle camera are subjected to feature extraction processing through the image feature extraction model to obtain the feature map corresponding to the perspective. Among them, the in-vehicle camera can be a front-view camera, and in this case, the front-view perspective feature map is extracted; the in-vehicle camera can be a surround-view camera, and in this case, the surround-view (multi-perspective) feature map is extracted. Among them, the image feature extraction model can be a 2D backbone network, such as ResNet, etc., or a combination of a 2D backbone network and a neck network, such as ResNet+FPN, etc.

[0114] Step S12a: The feature map is subjected to perspective transformation processing through the perspective transformation module to obtain the BEV feature. Among them, the perspective transformation module can transform the feature map from the perspective view (PV) to the bird's-eye view. Specifically, a geometric-based method can be used, such as inverse perspective mapping (IPM), or a depth-based method can be used, such as LSS (Lift, splat, shoot), or a Transformer-based method can be used, such as deformable attention, etc.

[0115] The second way is applicable when only in-vehicle lidar is used as in-vehicle perception sensors. At this time, the feature encoder includes a point cloud feature extraction model. The specific steps are as follows:

[0116] Step S11b: The point cloud collected by the in-vehicle lidar is subjected to feature extraction through the point cloud feature extraction model to obtain the BEV feature. Among them, the point cloud feature extraction model can be a 3D backbone network, such as PointPillars, SECOND, etc.

[0117] The third method is applicable to the simultaneous use of an in-vehicle camera and an in-vehicle lidar as in-vehicle perception sensors. At this time, the feature encoder includes an image feature extraction model, a perspective transformation module, a point cloud feature extraction model, and a multimodal fusion module. The specific steps are as follows:

[0118] Step S11c: Obtain the image BEV feature through step S11a and step S12a of the first method; obtain the point cloud BEV feature through step S11b of the second method.

[0119] Step S12c: Fuse the image BEV feature and the point cloud BEV feature through the multimodal fusion module to obtain the BEV feature. The multimodal fusion module can fuse the BEV features corresponding to two different modalities of images and point clouds (i.e., the image BEV feature and the point cloud BEV feature) (such as splicing, etc.).

[0120] After obtaining the BEV feature through the above first method, second method, or third method, the BEV feature is converted into a vectorized road structure through a feature decoder. For example, the feature decoder uses these BEV features for 3D object detection and road structure classification and localization tasks. That is, the feature decoder generates output embeddings through interaction with the BEV features. These output embeddings contain the position and shape information of the target object in the BEV space. This process involves self-attention mechanisms and cross-attention mechanisms to ensure effective fusion of features from different perspectives, thereby obtaining an accurate vectorized road structure.

[0121] It should be noted that Figure 4 An in-vehicle positioning sensor is also shown. The in-vehicle positioning sensor is used to collect vehicle position and movement information to achieve high-precision vehicle positioning; it helps to update the map in real time to reflect road changes.

[0122] For the cloud, the cloud can obtain the local map uploaded by the vehicle end and execute the following step S13.

[0123] Step S13: In order to obtain a lane-level vector map (i.e., a global map) at the city scale, the cloud uses a sliding window method to process the entire city area according to a zigzag scan sequence. Among them, for each local area, the local map has been extracted at the vehicle end. Therefore, at the cloud, the input data is cleaned and standardized through data preprocessing to be data in the global coordinate system and in a rectangular area that is not in front of or around the ego vehicle. Compared with most in-vehicle methods that use multi-view images as input and operate on BEV features, the main advantages of the embodiments of the present application are in two aspects: on the one hand, it can make full use of regional global information, such as geometric smoothness constraints, semantic relevance, and global accuracy consistency; on the other hand, multiple data collections can alleviate inevitable challenges such as accuracy deviation and dynamic occlusion.

[0124] 2) Step S2.

[0125] Assign weights according to the data quality and historical accuracy of each trip of vehicle perception data, and perform feature extraction and fusion on the weighted data through an adaptive weight allocator. Step S2 can be implemented through the following Step S21 and Step S22.

[0126] Step S21, calculate the data quality index Q i (t) and the historical accuracy index H i .

[0127] Data quality index Q i (t): The data quality index of the i-th data source (referring to the vehicle perception data of the i-th trip) at time t is quantified based on factors such as signal-to-noise ratio and measurement error. The formula is as follows:

[0128] Q i (t) = f qual ity (SNR i (t), error i (t))

[0129] Specifically, SNR is the abbreviation of Signal-to-Noise Ratio, where P signal,i (t) is the signal power of the vehicle-mounted perception sensor corresponding to the i-th data source at time t, and P noise,i (t) is the noise power of the vehicle-mounted perception sensor corresponding to the i-th data source at time t; the measurement error (error) is the standard deviation between the measured value and the true value (the true value can be obtained through high-precision maps or surveying and mapping), error i (t) = std(Measurements i ―GT), where std represents the standard deviation, Measurements i is the measured value, and GT is the true value; f qual ity is a weighted summation function for calculating the data quality index, and the weight of each factor is adjusted according to the importance of the actual application scenario.

[0130] Historical accuracy index H i : Assign the historical accuracy index based on the past accuracy performance of the i-th data source. The formula is as follows:

[0131]

[0132] Specifically, the number of correct predictions, CorrectPredictions i represents the number of times the prediction of the i-th data source has matched the actual changes over a past period of time. It reflects the accuracy of the i-th data source in predicting changes in history. The number of correct predictions can be determined by comparing historical prediction data and actual change data. Each time the data source successfully predicts a change (e.g., correctly identifies a change in lane lines, traffic signs, or other map elements), it is recorded as one correct prediction; the total number of predictions, TotalPredictions i represents the total number of predictions made by the i-th data source during the same period of time, regardless of whether these predictions are ultimately confirmed as correct. The historical accuracy metric H i The higher it is, the better the historical performance of the i-th data source and the higher the credibility of its predictions.

[0133] Step S22, dynamically adjust the weights of each data source. When the data quality metric and the historical accuracy metric of the i-th data source are inconsistent, the system adjusts the weights according to the degree of deviation between the two to optimize the map construction process.

[0134] Set the adaptive weight W i (t), combining the data quality metric and the historical accuracy metric, calculate the adaptive weight at time t. The formula is as follows:

[0135] W i (t) = ω 1 ·Q i (t) + ω 2 ·H i

[0136] Specifically, ω 1 and ω 2 are weights used to balance the influence of the data quality metric Q i (t) and the historical accuracy metric H i on the final weight W i (t). They can be adjusted according to the performance of the algorithm and the requirements of specific application scenarios to ensure that the algorithm can provide high-quality map data under different conditions. In practical applications, the adjustment of weight parameters needs to consider various factors, including the type of data source, environmental conditions, and the performance objectives of the algorithm. For example, if a data source performs well in a specific environment, its weight may need to be increased; conversely, if a data source performs poorly, its weight may need to be decreased.

[0137] The calculation of the adaptive weight W i (t) takes into account both the data quality metric Q i Q i(t), and also takes into account the past performance of the i-th data source, i.e., the historical accuracy metric H i . In this way, the algorithm can flexibly respond to changes in data quality, thereby improving the accuracy and efficiency of map updates.

[0138] To ensure that the sum of all weights is 1, normalization is required to ensure the rationality of data fusion. The formula is as follows:

[0139]

[0140] Use the normalized weights to perform data fusion on the information of each data source (referring to the local maps corresponding to the multi-trip vehicle perception data respectively).

[0141]

[0142] Among them, X i (t) is the data output (i.e., the local map) of the i-th data source at time t, and n is the total number of data sources.

[0143] The adaptive weight allocation mechanism adopted in step S2 allows the system to dynamically adjust its contribution degree in the map update process according to the real-time performance and historical accuracy of the data source, thereby improving the accuracy and reliability of the overall map data.

[0144] 3) Step S3.

[0145] Step S3 identifies the changed areas in the global map through change detection (corresponding to the update detection above) and makes an update decision on the global map. Step S3 can be implemented through the following step S31 and step S33.

[0146] Step S31, identify the changed areas in the global map. For each map element e in the global map, evaluate its matching degree MC(e, M) with the historical global map M, which can be achieved through various methods. For example, the hausdorff distance can be calculated as the matching degree. Then, estimate the existence probability P exist (e) of the map element e according to the matching degree MC(e, M). The formula is P exist (e) = σ(W·F(e) + b), where σ is the sigmoid function, W and b are model parameters, and F(e) is MC(e, M).

[0147] The change detection situations considered in the embodiments of this application include: ① Map element addition; ② Map element deletion; ③ Map element position or shape change; ④ Map element attribute change; ⑤ Change in the topological relationship between map elements. Among them:

[0148] ① Map element addition: It means P exist(e) is greater than the preset existence probability threshold P t .

[0149] ② Map element deletion: refers to the case where P exist (e) is less than or equal to the existence probability threshold P t .

[0150] ③ Map element position or shape change: refers to the case where the matching degree MC(e,M) is less than or equal to the matching degree threshold T position .

[0151] ④ Map element attribute change: refers to the case where the difference between the attribute vectors A(e) and A(M) is greater than the attribute difference threshold T attribute .

[0152] ⑤ Map element topological relationship change: refers to the change in the topological relationship (such as connection, intersection, inclusion, etc.) between map elements. The topological relationship between map elements is represented by a topological relationship matrix R. If R new≠ R old , it proves that the topological relationship between map elements has changed, where R new represents the topological relationship between map elements in the global map (the currently generated global map), and R old represents the topological relationship between map elements in the historical global map.

[0153] Step S32, generate a change map C map . Mark all detected change regions to provide a basis for update decisions. The change map is a two-dimensional array or matrix, and each element in it corresponds to a point in the global map. If a change is detected at this point, it is marked as 1 in the change map, otherwise it is marked as 0. The change map is used to identify and locate the regions that need to be updated. The formula is as follows:

[0154]

[0155] where (i,j) represents the position on the map, and P t is the preset existence probability threshold; changedetected in the formula indicates that a change in the position or shape of a map element, a change in the attributes of a map element, or a change in the topological relationship between map elements is detected.

[0156] Step S33, update decision. For each map element e in the global map, decide whether to retain, delete, or update the map element based on the change map and the existence probability. The formula is as follows:

[0157]

[0158] 4) Step S4.

[0159] Step S4 predicts and updates in advance the areas that may change in the global map through predictive update. Step S4 can be implemented through the following Step S41 and Step S43.

[0160] In Step S41, a recurrent neural network model is used for time series prediction to simulate the change of the map over time.

[0161] Here, the prior information in the global map needs to be taken into consideration to supplement the missing information (such as the lane lines being blocked or blurred by other vehicles) in the local map corresponding to the current single-trip vehicle perception data, so as to obtain map features with greatly improved representation ability. In the embodiments of the present application, the recurrent neural network model can be an LSTM model.

[0162] Let X t be a part of the local map at the acquisition time t (which can be represented by a feature vector), and G be the prior information matrix of the global map. And X t and G are fused as the input of the LSTM model, that is, H t = Concat(X t , G).

[0163] Let h t＝0 be the initial hidden state of the LSTM model at the acquisition time t = 0. Among them, the hidden state h t is the compressed representation of the sequence information by the LSTM model, which is continuously updated as the acquisition time progresses, providing the necessary context information for the time series prediction task.

[0164] The LSTM unit updates its hidden state h t and cell state c t at each acquisition time, so that the LSTM model can better handle the long-term dependence problem in time series data. The specific formula is as follows:

[0165] i t = σ(W i ·H t + b i )

[0166] f t = σ(W f ·H t + b f )

[0167] o t = σ(W o ·H t + b o )

[0168]

[0169]

[0170] h t = o t *tanh(c t )

[0171] where σ is the sigmoid activation function, W is the weight parameter, b is the bias parameter, and i t , f t , o t are the input gate, forget gate, and output gate respectively, and i t determines whether the corresponding map element e is newly stored in the cell state, f t determines whether the corresponding map element e should be deleted from the cell state, and o t determines how the hidden state reflects the current cell state.

[0172] Use the updated hidden state h t to predict the map features at the next acquisition time

[0173] In this way, the LSTM model can fully integrate multiple incomplete local maps with the prior information of the global map to update the representation ability of real-time map features.

[0174] Step S42, trend prediction. Based on historical change data and current environmental data, predict the change trend of each point x in the global map, where each point x is equivalent to a map element.

[0175] Specifically, the historical change data includes: ① Historical global map: past map versions, recording the changes of various map elements (such as ground elements like roads and aerial elements like traffic signs); ② Map update log: detailed records of the specific changes in each map update, such as retained, deleted, or updated map elements.

[0176] The current environmental data includes: ① Weather conditions: such as sunny, rainy, snowy, etc., which will affect road conditions and visibility; ② Traffic flow: the current traffic flow and congestion situation, and this information can be obtained through in-vehicle sensing sensors or traffic monitoring systems; ③ Special events: such as construction, accidents, or other temporary events, which may cause road closures or diversions; ④ Time factor: different times of the day or different days of the week.

[0177] Next, perform data processing on the above historical change data and current environmental data:

[0178] ① Data preprocessing: Clean, format, and synchronize the historical change data and current environmental data to ensure data consistency and availability.

[0179] ② Feature extraction: Extract relevant features from historical change data and current environmental data, such as change frequency, change type, weather pattern, etc.

[0180] ③ Feature selection: Select the features that have the most influence on the change trend prediction, and these features will be used as the input of the prediction model.

[0181] The function of the first convolutional neural network model includes:

[0182] P(x) = f predict (F historical , F environmental )

[0183] where F historical is the feature vector of historical change data, and F environmental is the feature vector of current environmental data.

[0184] An input feature vector can be constructed, and the input feature vector includes the feature vector of historical change data and the feature vector of current environmental data. The input feature vector is represented as x = [F historical , F environmental .

[0185] The above function can be rewritten as:

[0186] P(x) = σ L (W L ·σ L―1 (W L―1 ·…σ 1 (W 1 ·x + b 1 ) + b 2 ) + … + b L )

[0187] where σ i is the activation function of the i-th layer, W i is the weight parameter of the i-th layer, and b i is the bias parameter of the i-th layer. The activation function is used to introduce non-linearity. For example, the ReLU function can be used, σ z = max(0, z).

[0188] Define a loss function to train the first convolutional neural network model to make its prediction close to the actual change.

[0189]

[0190] where N is the number of training samples, and y i is the actual change label of the i-th sample.

[0191] Update the weight parameters of the first convolutional neural network model according to the loss function L using the gradient descent algorithm:

[0192]

[0193] where η is the learning rate, represents the gradient.

[0194] Furthermore, assign a confidence level to the prediction result P(x) (i.e., the change trend) to indicate the reliability of the prediction. The formula is as follows:

[0195] Conf(x) = σ(W·P(x) + b)

[0196] where σ is the sigmoid function, and W and b refer to the weight parameter and the bias parameter respectively.

[0197] Step S43, the second convolutional neural network model performs map update processing by combining the output of step S41 and the output of step S42. The formula is as follows:

[0198] M updated = g(X fused , Change θ (e, M))

[0199] where M updated represents the updated global map, X fused represents the output of step S41, Change θ (e, M) represents the output of step S42, θ represents the confidence threshold for constraining the confidence level, and g represents the model parameters of the second convolutional neural network model.

[0200] Through step S4, intelligent map update decisions can be achieved through predictive updates.

[0201] In some embodiments, to ensure the accuracy of predictive updates in different working condition road environments, a series of measures can be taken to enhance the generalization ability and adaptability of the algorithm. For example, to ensure the diversity of training data, the collection of training data can cover various possible environmental changes and geographical features, including vehicle perception data in different working condition road environments such as cities, suburbs, mountains, and different climate regions.

[0202] 5) Step S5.

[0203] Through the resource scheduler, the ARMUOA algorithm can improve resource utilization efficiency while ensuring real-time performance and accuracy, meeting the requirements of different users and different scenarios. The resource scheduler is a key component in the ARMUOA algorithm responsible for managing and allocating computing resources, ensuring that the algorithm can achieve real-time performance under limited resource conditions. Step S5 can be implemented through the following steps S51 and S52.

[0204] Step S51, allocate resources according to the current system load L and task priority P. Here, tasks generally refer to various data collection and computing tasks involved in the embodiments of this application. The resource scheduler evaluates the current system load L, including CPU usage, memory occupancy, I / O operations, etc., and at the same time determines the priority P of each task. The priority can be determined based on the urgency, importance, or expected impact range of the task. Then, according to the system load and task priority, the computing resources are allocated in a round-robin manner. The formula is as follows:

[0205] Resources=ResourceScheduler(L,P)

[0206] Step S52, parameter adjustment feedback. Users can adjust algorithm parameters, such as the confidence threshold θ used to constrain confidence. The parameters adjusted by the user through the custom interface are fed back to the resource scheduler, and the scheduler adjusts the resource allocation strategy according to the new parameters. The resource scheduler dynamically adjusts resource allocation based on the real-time monitored data to cope with changes in system load.

[0207] 6) Step S6.

[0208] Collect user feedback and performance data through the feedback learning mechanism to enable the system to self-optimize. Step S6 can be implemented through step S61.

[0209] Step S61, adjust algorithm parameters according to the performance index PI and user feedback UF, thus forming a continuous optimization loop. The formula is as follows:

[0210] Params new =AdaptationMode l(Params old ,PI,UF)

[0211] Params in the above formula generally refers to the algorithm parameters involved in the ARMUOA algorithm.

[0212] In this way, the system can flexibly handle the inconsistencies of multi-trip vehicle perception data at different times and ensure the accuracy and real-time performance of map updates.

[0213] Embodiments of this application can be applied to autonomous vehicles, which are equipped with a variety of on-vehicle sensors, including but not limited to:

[0214] ① High-resolution cameras for capturing road visual information.

[0215] ② Light Detection and Ranging (LiDAR) for generating 3D point clouds of the surrounding environment.

[0216] ③ Global Positioning System (GPS) and Inertial Measurement Unit (IMU) for vehicle positioning and attitude determination.

[0217] Based on this, embodiments of this application also provide a schematic flowchart of a map update method as shown in Figure 6 which will be described in step form in combination with Figure 6 ...

[0218] Step P1: Data collection and synchronization. The vehicle end collects data in real time through on-vehicle sensors and synchronizes it through timestamps; the cloud end performs denoising, correction, and normalization processing on the vehicle end data.

[0219] Step P2: Adaptive weight assignment. The cloud end calculates weights based on the data quality index and historical accuracy index of the vehicle end data, and uses a fusion algorithm to combine the weighted data to obtain a global map.

[0220] Step P3: Change detection and change trend prediction. The cloud end uses a change detection algorithm to compare the currently generated global map with the historical global map to detect the changed area; the cloud end applies a recurrent neural network model (i.e., the time series prediction model shown in Figure 6 ...) to predict future changes and update the global map in advance.

[0221] Step P4: Map data synchronization and distribution. The cloud end synchronizes the updated global map to the vehicle internal system and uploads it to a shared server for sharing by other vehicles.

[0222] Step P5: Feedback learning and optimization. The cloud end determines the system load and task priority according to different urban environments and vehicle types, and dynamically allocates computing resources; the cloud end collects user feedback and performance monitoring data to adjust and optimize the algorithm parameters.

[0223] Figure 6 ... shows the application of the ARMUOA algorithm in autonomous vehicles. Through adaptive weight assignment and intelligent resource scheduling, it significantly improves the real-time performance and accuracy of map updates, and enhances the environmental adaptability and safety of autonomous vehicles.

[0224] Embodiments of this application can at least achieve the following technical effects:

[0225] 1) Through global crowdsourced data fusion, more comprehensive map updates can be achieved, enabling the processing of a wider perception range, not limited to the area around the vehicle. During the data fusion process, on the one hand, regional global information such as geometric smoothness constraints, semantic relevance, and global accuracy consistency can be fully utilized; on the other hand, multiple data collections can mitigate inevitable challenges such as accuracy deviation and dynamic occlusion.

[0226] 2) It can dynamically adjust the weights of multi-trip vehicle perception data from different times according to data quality indicators and historical accuracy indicators, which enables the algorithm to more intelligently fuse multi-source multi-trip data and improve the accuracy of map updates.

[0227] 3) By adopting predictive updates, it can predict potential changes in the road environment in advance based on historical data and trend analysis, and perform map updates in advance, improving the forward-looking and adaptability of the autonomous driving system.

[0228] 4) Dynamically allocate computing resources according to the current computing load and task priorities to ensure the real-time performance of the algorithm while reducing the computing cost. Improve the user experience through parameter adjustment feedback, making the map update service more personalized and accurate.

[0229] 5) It has the ability of self-learning and optimization, and can continuously adjust the update strategy according to user feedback and performance indicators to achieve continuous performance improvement.

[0230] 6) It is applicable to autonomous driving vehicles, ensuring that autonomous driving vehicles have stronger adaptability and safety in complex and changing environments, and providing an efficient, reliable and cost-effective map update solution for autonomous driving vehicles.

[0231] Next, the exemplary structure of the map update device 255 implemented as a software module provided by the embodiments of the present application will be further described. In some embodiments, as Figure 2 shown, the software modules in the map update device 255 stored in the memory 250 may include: a first prediction module 2551, configured to perform prediction processing through a first convolutional neural network model based on historical change data of the global map and current environmental data to obtain a first prediction result of map elements in the global map; a second prediction module 2552, configured to perform a first fusion process on the global map and parts of the local map at multiple acquisition times to obtain fusion results at multiple acquisition times, and perform prediction processing through a recurrent neural network model based on the fusion results at multiple acquisition times to obtain a second prediction result of map elements in the global map; wherein, the local map is determined according to vehicle perception data of a vehicle traveling on the global map; a predictive update module 2553, configured to perform map update processing on the global map according to the first prediction result and the second prediction result of map elements in the global map.

[0232] In some embodiments, the predictive update module 2553 is further configured to: perform a prediction process based on the first prediction result and the second prediction result of the map elements in the global map through the second convolutional neural network model to obtain a third updated prediction result of the map elements in the global map; determine the retention, deletion, or update of the map elements in the global map according to the third updated prediction result of the map elements in the global map.

[0233] In some embodiments, the map update device 255 further includes a feature encoding module, configured to perform feature encoding processing on multiple trips of vehicle perception data to obtain bird's-eye view (BEV) features corresponding to the multiple trips of vehicle perception data respectively; the map update device 255 further includes a feature decoding module, configured to perform feature decoding processing on the BEV features corresponding to the multiple trips of vehicle perception data respectively to obtain local maps corresponding to the multiple trips of vehicle perception data respectively; the map update device 255 further includes a multi-source fusion module, configured to perform a second fusion process on the local maps corresponding to the multiple trips of vehicle perception data respectively to obtain a global map.

[0234] In some embodiments, the feature encoding module is further configured to perform the following processing for any trip of vehicle perception data: when any trip of vehicle perception data is an image collected by an in-vehicle camera through a specific perspective, perform feature extraction processing on the image through an image feature extraction model to obtain a feature map of the specific perspective, perform perspective conversion processing on the feature map of the specific perspective to obtain an image BEV feature, and determine the image BEV feature as the BEV feature for the second fusion process; when any trip of vehicle perception data is a point cloud collected by an in-vehicle radar, perform feature extraction processing on the point cloud through a point cloud feature extraction model to obtain a point cloud BEV feature, and determine the point cloud BEV feature as the BEV feature for the second fusion process; when any trip of vehicle perception data includes an image collected by an in-vehicle camera through a specific perspective and a point cloud collected by an in-vehicle radar, perform feature extraction processing on the image through an image feature extraction model to obtain a feature map of the specific perspective, perform perspective conversion processing on the feature map of the specific perspective to obtain an image BEV feature, perform feature extraction processing on the point cloud through a point cloud feature extraction model to obtain a point cloud BEV feature, and perform a third fusion process on the image BEV feature and the point cloud BEV feature to obtain the BEV feature for the second fusion process.

[0235] In some embodiments, the multi-source fusion module is further configured to: determine the data quality index and the historical accuracy index corresponding to each trip of vehicle perception data; determine the weight of the local map corresponding to the trip of vehicle perception data according to the data quality index and the historical accuracy index corresponding to each trip of vehicle perception data; perform weighted summation processing on the local maps corresponding to the multiple trips of vehicle perception data respectively according to the weights of the local maps corresponding to the multiple trips of vehicle perception data to obtain a global map.

[0236] In some embodiments, the multi-source fusion module is further configured to perform the following processing on any trip of vehicle perception data: determine a data quality index corresponding to any trip of vehicle perception data according to the signal-to-noise ratio and measurement error of in-vehicle perception sensors, where the in-vehicle perception sensors are used to collect any trip of vehicle perception data; determine an accuracy index of the vehicle perception data historically collected by the in-vehicle perception sensors as the historical accuracy index corresponding to any trip of vehicle perception data.

[0237] In some embodiments, the map update device 255 further includes a change detection module, configured to perform an update detection process on the global map and the historical global map to obtain an update detection result of map elements in the global map; the map update device 255 further includes a real-time update module, configured to perform a map update process on the global map according to the update detection result of map elements in the global map.

[0238] The embodiments of the present application provide a computer program product or a computer program. The computer program product or the computer program includes executable instructions, and the executable instructions are stored in a computer-readable storage medium. The processor of the electronic device reads the executable instructions from the computer-readable storage medium, and the processor executes the executable instructions, so that the electronic device implements the map update method described above in the embodiments of the present application.

[0239] The embodiments of the present application provide a computer-readable storage medium storing executable instructions, where the executable instructions are stored. When the executable instructions are executed by a processor, the processor will be caused to implement the map update method provided by the embodiments of the present application.

[0240] In some embodiments, the computer-readable storage medium may be a memory such as FRAM, ROM, PROM, EPROM, EEPROM, flash memory, magnetic surface memory, optical disc, or CD-ROM; or may be various devices including one or any combination of the above memories.

[0241] In some embodiments, the executable instructions may be in the form of a program, software, software module, script, or code, and may be written in any form of programming language (including compiled or interpreted languages, or declarative or procedural languages), and may be deployed in any form, including being deployed as an independent program or being deployed as a module, component, subroutine, or other unit suitable for use in a computing environment.

[0242] As an example, the executable instructions may or may not correspond to files in a file system, and may be stored as part of a file that holds other programs or data. For example, they may be stored in one or more scripts in a Hyper Text Markup Language (HTML) document, stored in a single file dedicated to the program under discussion, or stored in multiple cooperating files (such as files that store one or more modules, subroutines, or portions of code).

[0243] As an example, the executable instructions may be deployed to execute on one electronic device, or on multiple electronic devices located at one location, or on multiple electronic devices distributed across multiple locations and interconnected by a communication network.

[0244] The above are only embodiments of the present application and are not intended to limit the protection scope of the present application. Any modifications, equivalent replacements, improvements, etc. made within the spirit and scope of the present application are included in the protection scope of the present application.

Claims

1. A map updating method, characterized in that: include: Performing prediction processing based on historical change data of the global map and current environmental data through a first convolutional neural network model to obtain a first prediction result of a map element in the global map; Performing a first fusion process on the global map and the local map at the parts of the multiple acquisition times to obtain the fusion results of the multiple acquisition times, and performing a prediction process based on the fusion results of the multiple acquisition times by a recurrent neural network model to obtain a second prediction result of the map elements in the global map; wherein the local map is determined according to the vehicle perception data of the vehicle traveling on the global map; According to the first prediction result and the second prediction result of the map element in the global map, map update processing is performed on the global map.

2. The method according to claim 1, characterized in that The performing map update processing on the global map according to the first prediction result and the second prediction result of the map element in the global map comprises: Performing prediction processing based on the first prediction result and the second prediction result of the map element in the global map by a second convolutional neural network model to obtain a third updated prediction result of the map element in the global map; The retention, deletion or update of the map element in the global map is determined according to the third update prediction result of the map element in the global map.

3. The method according to claim 1, characterized in that Before performing prediction processing based on the historical change data of the global map and the current environment data by the first convolutional neural network model, the method further includes: Performing feature coding processing on the multiple vehicle perception data to obtain the bird's-eye view BEV features corresponding to the multiple vehicle perception data respectively; Performing feature decoding processing on the BEV features respectively corresponding to the plurality of vehicle perception data to obtain the local maps respectively corresponding to the plurality of vehicle perception data; A second fusion process is performed on the local maps corresponding to the plurality of vehicle perception data to obtain the global map.

4. The method according to claim 3, characterized in that The feature encoding process is performed on the plurality of vehicle perception data to obtain the bird's-eye view BEV features corresponding to the plurality of vehicle perception data, including: For any vehicle perception data, perform the following processing: When the perception data of any one trip of vehicles is an image acquired by a vehicle-mounted camera at a specific viewing angle, a feature extraction process is performed on the image through an image feature extraction model to obtain a feature map of the specific viewing angle, a view conversion process is performed on the feature map of the specific viewing angle to obtain an image BEV feature, and the image BEV feature is determined as a BEV feature for the second fusion process; When the perception data of any one vehicle trip is a point cloud collected by a vehicle-mounted radar, a point cloud feature extraction model is used to perform feature extraction processing on the point cloud to obtain a point cloud BEV feature, and the point cloud BEV feature is determined as a BEV feature for the second fusion processing; When the perception data of any one trip of the vehicle includes an image captured by a vehicle-mounted camera at a specific viewing angle and a point cloud captured by a vehicle-mounted radar, feature extraction is performed on the image through an image feature extraction model to obtain a feature map of the specific viewing angle, perspective conversion is performed on the feature map of the specific viewing angle to obtain image BEV features, feature extraction is performed on the point cloud through a point cloud feature extraction model to obtain point cloud BEV features, and a third fusion process is performed on the image BEV features and the point cloud BEV features to obtain BEV features for a second fusion process.

5. The method according to claim 3, characterized in that: The performing a second fusion process on the local maps corresponding to the plurality of vehicle perception data to obtain the global map includes: Determine the data quality indicators and historical accuracy indicators corresponding to each vehicle perception data; Determine the weight of the local map corresponding to each trip of vehicle perception data based on the data quality index and historical accuracy index corresponding to the vehicle perception data; According to the weights of the local maps respectively corresponding to the multiple vehicle perception data, weighted sum processing is performed on the local maps respectively corresponding to the multiple vehicle perception data to obtain the global map.

6. The method according to claim 5, characterized in that The determination of the data quality index and the historical accuracy index corresponding to each vehicle perception data includes: For any vehicle perception data, perform the following processing: Determine the data quality index corresponding to the perception data of any one trip of vehicles according to the signal-to-noise ratio and the measurement error of the vehicle-mounted perception sensor; wherein the vehicle-mounted perception sensor is used to collect the perception data of any one trip of vehicles; Determine the accuracy index of the vehicle perception data historically collected by the on-board perception sensor to serve as the historical accuracy index corresponding to the arbitrary vehicle perception data.

7. The method according to claim 3, characterized in that After performing a second fusion process on the local maps corresponding to the plurality of vehicle perception data to obtain the global map, the method further includes: Performing update detection processing on the global map and the historical global map to obtain update detection results of map elements in the global map; Map update processing is performed on the global map according to update detection results of map elements in the global map.

8. A map updating device, characterized in that: include: A first prediction module, configured to perform prediction processing based on historical change data of a global map and current environmental data through a first convolutional neural network model to obtain a first prediction result of a map element in the global map; a second prediction module, configured to perform a first fusion process on the global map and the local map at parts of multiple acquisition times to obtain fusion results of multiple acquisition times, and perform a prediction process based on the fusion results of multiple acquisition times by a recurrent neural network model to obtain a second prediction result of the map element in the global map; wherein the local map is determined based on vehicle perception data of a vehicle traveling on the global map; The predictive update module is used to perform map update processing on the global map according to the first prediction result and the second prediction result of the map elements in the global map.

9. An electronic device, characterized in that: include: A memory for storing executable instructions; A processor, configured to implement the method according to any one of claims 1 to 7 when executing the executable instructions stored in the memory.

10. A computer-readable storage medium, characterized in that: Executable instructions are stored, and when executed by a processor, the method described in any one of claims 1 to 7 is implemented.

Images