Crowdsourced map data fusion method and apparatus, vehicle, and storage medium

Abstract

The application relates to a crowd-sourced map data fusion method and device, a vehicle and a storage medium. The method comprises the following steps: receiving a plurality of initial key frame feature point cloud data uploaded by a vehicle, pose data corresponding to each key frame and relative pose data between adjacent key frames in each data source; and when it is judged that there are key frame feature point cloud data pairs with a preset relationship in a plurality of data sources according to the plurality of initial key frame feature point cloud data of each data source, the pose data corresponding to each key frame and the relative pose data between adjacent key frames in each data source, the relationship between the key frame feature point cloud data pairs with the preset relationship is quantitatively represented based on a preset three-dimensional bag-of-words model. Therefore, the three-dimensional bag-of-words model is used for realizing fast search and matching of data, the problem that high-precision map updating cannot effectively ensure the instantaneity of map element updating is solved, and the updating speed and updating accuracy of high-precision map elements are improved.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular to a method, apparatus, vehicle, and storage medium for fusion of crowdsourced map data. Background Technology

[0002] In existing advanced autonomous driving systems, high-precision maps are one of the most fundamental and crucial functional modules, providing the system with "beyond-line-of-sight" perception capabilities. The accuracy of high-precision map data is a prerequisite for the stable operation of the autonomous driving system. In reality, road traffic elements are not static and may change due to construction or other reasons; therefore, timely updates to high-precision map data are extremely necessary.

[0003] In related technologies, the update cycle for high-precision map data in traditional solutions is generally more than two months.

[0004] However, traditional methods cannot effectively guarantee the timeliness of map element updates, which urgently needs to be addressed. Summary of the Invention

[0005] This application provides a method, apparatus, vehicle, and storage medium for fusion of crowdsourced map data to solve the problem that high-precision map updates cannot effectively guarantee the timeliness of map element updates, thereby improving the update speed and accuracy of high-precision map elements.

[0006] To achieve the above objectives, the first aspect of this application proposes a method for fusion of crowdsourced map data, comprising the following steps:

[0007] Receive multiple sets of data sources uploaded by the vehicle. Each set of data sources includes multiple initial keyframe feature point cloud data, pose data corresponding to each keyframe, and relative pose data between adjacent keyframes within each set of data sources.

[0008] Based on the multiple initial keyframe feature point cloud data of each group of data sources, the pose data corresponding to each keyframe, and the relative pose data between adjacent keyframes within each group of data sources, it is determined whether there are keyframe feature point cloud data pairs with a preset connection in the multiple groups of data sources.

[0009] If there are keyframe feature point cloud data pairs with a preset relationship among the multiple data sources, then the relationship between the keyframe feature point cloud data pairs with the preset relationship is quantified based on the preset three-dimensional bag-of-words model.

[0010] Based on the aforementioned technical means, by receiving multiple sets of data sources uploaded from different vehicles, a rich foundation is provided for the accurate construction of maps. The 3D bag-of-words model is an efficient scene description method that can transform complex point cloud data into concise, easy-to-compare and match description vectors, thereby greatly improving the efficiency and accuracy of data fusion.

[0011] According to one embodiment of this application, determining whether there are pre-defined keyframe feature point cloud data pairs in the multiple sets of data sources based on the multiple initial keyframe feature point cloud data of each set of data sources, the pose data corresponding to each keyframe, and the relative pose data between adjacent keyframes within each set of data sources includes:

[0012] Based on the multiple initial keyframe feature point cloud data of each group of data sources, the pose data corresponding to each keyframe, and the relative pose data between adjacent keyframes within each group of data sources, a preset clustering algorithm is used to divide the multiple initial keyframe feature point cloud data of each group of data sources into cluster centers to obtain an initial clustering result, wherein the initial clustering result includes multiple keyframe feature point cloud data groups.

[0013] Based on the preset clustering algorithm and preset data tree depth, each keyframe feature point cloud data group is clustered sequentially to obtain the final clustering result, and a preset data dictionary is obtained based on the final clustering result.

[0014] Based on the preset data dictionary, each initial keyframe feature point cloud data is mapped to at least one word in the preset data dictionary, and the at least one word is combined to obtain the word vector corresponding to each initial keyframe feature point cloud data.

[0015] Select any two word vectors from the multiple sets of data sources, calculate the similarity between the two word vectors using a preset similarity calculation strategy, and determine that there is a key frame feature point cloud data pair with a preset relationship in the multiple sets of data sources when the similarity meets a preset threshold.

[0016] Based on the aforementioned technical means, through precise clustering, word vector mapping, and efficient comparison, accurate judgment of key frame feature point cloud data pairs with preset connections in multiple data sources was achieved, providing strong support for subsequent data fusion.

[0017] According to one embodiment of this application, the step of quantifying the relationship between keyframe feature point cloud data pairs with a preset relationship based on the preset three-dimensional bag-of-words model includes:

[0018] Using a preset function, based on the key frame feature point cloud data pairs with preset relationships and the relative poses between the key frame feature point cloud data in the key frame feature point cloud data pairs with preset relationships, the optimal solution for the pose data corresponding to each key frame in each data source is obtained.

[0019] Based on the above technical means, the relationship between key frame feature point cloud data pairs with preset connections is quantified and the optimal solution of pose data corresponding to each key frame is obtained. This can provide accurate pose information and feature description for subsequent data fusion, which helps to improve the accuracy and efficiency of data fusion, and makes the fused map data more accurate and complete.

[0020] According to one embodiment of this application, the preset function is:

[0021]

[0022] Wherein, Φ is an abstract representation of the pose data corresponding to the keyframe, Ω is an abstract representation of the relative pose between the keyframe feature point cloud data that have a preset relationship, Ψ∈Φ is the set of keyframe feature point cloud data pairs that have a preset relationship, and Δ is the relative pose between the keyframe feature point cloud data.

[0023] According to one embodiment of this application, the multiple initial keyframe feature point cloud data, the pose data corresponding to each keyframe, and the relative pose data between adjacent keyframes within each group of data sources are obtained by the vehicle based on a preset LIO-SAM (Tightly-coupled Lidar Inertial Odometry via Smoothing and Mapping) algorithm, which performs feature extraction and local mapping processing on the collected lidar data, inertial measurement data, and GPS (Global Positioning System) / RTK (Real-Time Kinematic) measurement data.

[0024] Based on the above technical means, the initial keyframe feature point cloud data, pose data and relative pose data can be extracted from the collected data using the LIO-SAM algorithm. This enables high-precision pose estimation and mapping, while also supporting multi-sensor data fusion and real-time performance. It has a wide range of application scenarios and scalability.

[0025] According to the crowdsourced map data fusion method proposed in this application, by receiving multiple initial keyframe feature point cloud data uploaded by vehicles, pose data corresponding to each keyframe, and relative pose data between adjacent keyframes within each data source group, and based on the multiple initial keyframe feature point cloud data, pose data corresponding to each keyframe, and relative pose data between adjacent keyframes within each data source group, when determining that there are keyframe feature point cloud data pairs with a preset relationship in multiple data sources, the relationship between the keyframe feature point cloud data pairs with a preset relationship can be quantified based on a preset 3D bag-of-words model. Therefore, by achieving rapid data search and matching based on the 3D bag-of-words model, the problem of high-precision map updates failing to effectively guarantee the timeliness of map element updates is solved, improving the update speed and accuracy of high-precision map elements.

[0026] To achieve the above objectives, a second aspect of this application provides a crowdsourced map data fusion apparatus, comprising:

[0027] The receiving module is used to receive multiple sets of data sources uploaded by the vehicle. Each set of data sources includes multiple initial keyframe feature point cloud data, pose data corresponding to each keyframe, and relative pose data between adjacent keyframes within each set of data sources.

[0028] The judgment module is used to determine whether there are key frame feature point cloud data pairs with preset relationships in the multiple groups of data sources based on the multiple initial key frame feature point cloud data of each group of data sources, the pose data corresponding to each key frame, and the relative pose data between adjacent key frames in each group of data sources.

[0029] The quantization representation module is used to quantize the relationship between keyframe feature point cloud data pairs that have a preset relationship in the multiple sets of data sources, based on the preset three-dimensional bag-of-words model.

[0030] According to one embodiment of this application, the determining module is specifically used for:

[0031] Based on the multiple initial keyframe feature point cloud data of each group of data sources, the pose data corresponding to each keyframe, and the relative pose data between adjacent keyframes within each group of data sources, a preset clustering algorithm is used to divide the multiple initial keyframe feature point cloud data of each group of data sources into cluster centers to obtain an initial clustering result, wherein the initial clustering result includes multiple keyframe feature point cloud data groups.

[0032] Based on the preset clustering algorithm and preset data tree depth, each keyframe feature point cloud data group is clustered sequentially to obtain the final clustering result, and a preset data dictionary is obtained based on the final clustering result.

[0033] Based on the preset data dictionary, each initial keyframe feature point cloud data is mapped to at least one word in the preset data dictionary, and the at least one word is combined to obtain the word vector corresponding to each initial keyframe feature point cloud data.

[0034] Select any two word vectors from the multiple sets of data sources, calculate the similarity between the two word vectors using a preset similarity calculation strategy, and determine that there is a key frame feature point cloud data pair with a preset relationship in the multiple sets of data sources when the similarity meets a preset threshold.

[0035] According to one embodiment of this application, the quantification expression module is specifically used for:

[0036] Using a preset function, based on the key frame feature point cloud data pairs with preset relationships and the relative poses between the key frame feature point cloud data in the key frame feature point cloud data pairs with preset relationships, the optimal solution for the pose data corresponding to each key frame in each data source is obtained.

[0037] According to one embodiment of this application, the preset function is:

[0038]

[0039] Wherein, Φ is an abstract representation of the pose data corresponding to the keyframe, Ω is an abstract representation of the relative pose between the keyframe feature point cloud data that have a preset relationship, Ψ∈Φ is the set of keyframe feature point cloud data pairs that have a preset relationship, and Δ is the relative pose between the keyframe feature point cloud data.

[0040] According to one embodiment of this application, the multiple initial keyframe feature point cloud data, the pose data corresponding to each keyframe, and the relative pose data between adjacent keyframes in each group of data sources are obtained by the vehicle based on a preset LIO-SAM algorithm, which performs feature extraction and local mapping processing on the collected lidar data, inertial measurement data, and GPS / RTK measurement data.

[0041] According to the crowdsourced map data fusion device proposed in this application, by receiving multiple initial keyframe feature point cloud data uploaded by vehicles, pose data corresponding to each keyframe, and relative pose data between adjacent keyframes within each data source group, and based on the multiple initial keyframe feature point cloud data, pose data corresponding to each keyframe, and relative pose data between adjacent keyframes within each data source group, when determining that there are keyframe feature point cloud data pairs with a preset relationship in multiple data sources, the device can quantify the relationship between keyframe feature point cloud data pairs with a preset relationship based on a preset three-dimensional bag-of-words model. Therefore, by achieving rapid data search and matching based on the three-dimensional bag-of-words model, the device solves the problem that high-precision map updates cannot effectively guarantee the timeliness of map element updates, and improves the update speed and accuracy of high-precision map elements.

[0042] To achieve the above objectives, a third aspect of this application provides a vehicle comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the crowdsourced map data fusion method as described in the above embodiments.

[0043] To achieve the above objectives, a fourth aspect of this application provides a computer-readable storage medium having a computer program stored thereon, which is executed by a processor to implement the crowdsourced map data fusion method as described in the above embodiments.

[0044] Additional aspects and advantages of this application will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of this application. Attached Figure Description

[0045] The above and / or additional aspects and advantages of this application will become apparent and readily understood from the following description of the embodiments taken in conjunction with the accompanying drawings, wherein:

[0046] Figure 1 This is a flowchart of a crowdsourced map data fusion method provided according to an embodiment of this application;

[0047] Figure 2 This is a schematic diagram of crowdsourced data pose fusion according to an embodiment of this application;

[0048] Figure 3 This is a schematic diagram illustrating the construction of a data dictionary according to one embodiment of this application;

[0049] Figure 4 This is a data fusion flowchart according to one embodiment of the present application;

[0050] Figure 5 This is a block diagram of a crowdsourced map data fusion device provided according to an embodiment of this application;

[0051] Figure 6 This is a structural schematic diagram of a vehicle provided according to an embodiment of this application.

[0052] Among them, 10-crowdsourced map data fusion device, 100-receiving module, 200-judgment module, 300-quantization representation module; 601-memory, 602-processor, 603-communication interface. Detailed Implementation

[0053] The embodiments of this application are described in detail below. Examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and intended to explain this application, and should not be construed as limiting this application.

[0054] The following describes, with reference to the accompanying drawings, a method, apparatus, vehicle, and storage medium for fusion of crowdsourced map data according to embodiments of this application. First, the method for fusion of crowdsourced map data according to embodiments of this application will be described with reference to the accompanying drawings.

[0055] Figure 1 This is a flowchart of a crowdsourced map data fusion method according to an embodiment of this application.

[0056] Before introducing the crowdsourced map data fusion method proposed in the embodiments of this application, the relevant technical background will be introduced first.

[0057] In the crowdsourced mapping solution, data is collected in a distributed and parallel manner. This means that the data required for map updates is collected by several vehicles equipped with relevant sensors. These vehicles can be dedicated mapping vehicles for data collection or application vehicles equipped with relevant sensors (e.g., vehicles using high-precision maps to achieve advanced autonomous driving functions). This addresses the issue of slow high-precision map updates at the data source. In the above solution, the process of updating high-precision map elements using crowdsourced data is performed in the cloud. Therefore, efficient cloud data processing is one of the key issues that needs to be addressed in this embodiment.

[0058] Based on this, this application proposes a method for fusion of crowdsourced map data, wherein the pose fusion problem of crowdsourced data is as follows: Figure 2 As shown, each node represents an abstract representation of the measurement value and pose of a certain observation data. Different data sources are represented by different colors. The black connecting lines between nodes represent the connection relationship between the same group of data sources, and the red connecting lines between nodes represent the connection relationship between different groups of data sources. In this embodiment, the relative pose is used to represent the connection relationship.

[0059] For example, such as Figure 1 As shown, this crowdsourced map data fusion method includes the following steps:

[0060] In step S101, multiple sets of data sources uploaded by the vehicle are received. Each set of data sources includes multiple initial keyframe feature point cloud data, pose data corresponding to each keyframe, and relative pose data between adjacent keyframes within each set of data sources.

[0061] The initial keyframe feature point cloud data is obtained by scanning the surrounding environment with LiDAR. It contains a large number of points in three-dimensional space, which represent the shape and position information of objects in the environment. A keyframe is a representative or specific frame in a continuous data stream. Due to the large amount of LiDAR data, the changes between consecutive frames may be small. Therefore, usually only a portion of the keyframes are selected for subsequent processing. The pose data corresponding to each keyframe describes the position and attitude of the vehicle at each keyframe moment. It is calculated by the vehicle's positioning system and provides the vehicle with accurate position information on the map. The relative pose data between adjacent keyframes in each data source describes the relative movement of the vehicle between two adjacent keyframes. By comparing the point cloud data of adjacent keyframes, the displacement and rotation of the vehicle between these two moments can be estimated, thus obtaining the relative pose data.

[0062] Furthermore, in some embodiments, multiple initial keyframe feature point cloud data, pose data corresponding to each keyframe, and relative pose data between adjacent keyframes within each data source are obtained by the vehicle based on a preset LIO-SAM algorithm, which performs feature extraction and local mapping processing on the collected lidar data, inertial measurement data, and GPS / RTK measurement data.

[0063] The preset LIO-SAM algorithm is a tightly coupled lidar inertial odometry framework based on smoothing and mapping. By fusing data from lidar and inertial measurement units in a tightly coupled manner, it can achieve high-precision, real-time positioning and map building tasks.

[0064] In other words, each data source includes sequential temporal observation information (such as LiDAR data, images, etc.) and pose measurement information (such as GPS / RTK measurement data, IMU (Inertial Measurement Unit) data, wheel speedometers, etc.). LiDAR data can provide rich environmental feature information, inertial measurement data can provide high-frequency motion estimation over a short period of time, and GPS / RTK can provide absolute position reference within the signal coverage area. The LiDAR point cloud dataset can be represented as follows: (i is the index of the data source, i.e., the data collected from the i-th vehicle; 1, 2, ..., N are the number of data of this type, LiDAR point cloud data is point cloud data, M n (This refers to the number of 3D target points contained in the LiDAR cloud data at that point); GPS / RTK measurement data can be represented as... Where p represents the three-dimensional position information and R represents the attitude information; the data of the inertial measurement unit can be represented as Where acc is the measured value of acceleration, and gyr is the measured value of angular velocity; wheel speed measurement data can be expressed as... ws is the measured value of wheel speed.

[0065] In practical applications, to save on data upload costs and cloud storage and computing costs, this embodiment preprocesses the aforementioned data on the vehicle side. This is achieved using a preset LIO-SAM algorithm (a mainstream SLAM (Simultaneous Localization and Mapping) algorithm; this algorithm can be replaced as needed to achieve the same functionality, and no specific limitation is made here). Specifically, on the vehicle side, feature extraction and local mapping are performed on the aforementioned data based on the preset LIO-SAM algorithm to obtain... Figure 2 Using the same data format, this operation reduces the effective data volume from 40MB / sec to approximately 0.8MB / sec, thereby improving application capabilities. After performing feature extraction and local mapping on the collected LiDAR data, inertial measurement data, and GPS / RTK measurement data, multiple initial keyframe feature point cloud data can be obtained. Pose data for each keyframe Relative pose data between adjacent keyframes within each data source group Where M is the number of keyframes in a certain set of data sources.

[0066] Therefore, by comprehensively utilizing LiDAR data, inertial measurement data, and GPS / RTK data, high-precision positioning and map building can be achieved. Through multi-sensor fusion, the positioning accuracy and map building accuracy of the system can be significantly improved.

[0067] In step S102, based on the multiple initial keyframe feature point cloud data of each data source, the pose data corresponding to each keyframe, and the relative pose data between adjacent keyframes within each data source, it is determined whether there are keyframe feature point cloud data pairs with a preset relationship in the multiple data sources.

[0068] It is understandable that data fusion is the process of sharing information from multiple vehicle-side data. The core idea is that when different vehicles are driving and collecting data, they can preprocess the data they collect, extract keyframes, and calculate the poses of the keyframes. When there is a way to establish a connection between different groups of data sources, the fusion of crowdsourced data can be achieved, and the optimal solution for the poses of all data can be found to ensure the global consistency of crowdsourced data.

[0069] Among them, the preset connection is Figure 2 The connection relationships between different groups of data sources are shown.

[0070] Therefore, in this embodiment, based on the multiple initial keyframe feature point cloud data of each data source, the pose data corresponding to each keyframe, and the relative pose data between adjacent keyframes within each data source, it can be determined whether there are keyframe feature point cloud data pairs with a preset relationship in multiple data sources (between different data sources), thereby establishing a relationship between different data sources.

[0071] To facilitate understanding, the following details how to determine whether there are keyframe feature point cloud data pairs with a pre-defined relationship in multiple data sources.

[0072] As one possible implementation, in some embodiments, based on multiple initial keyframe feature point cloud data of each data source group, the pose data corresponding to each keyframe, and the relative pose data between adjacent keyframes within each data source group, it is determined whether there are keyframe feature point cloud data pairs with a pre-defined relationship in multiple data sources. This includes: based on multiple initial keyframe feature point cloud data of each data source group, the pose data corresponding to each keyframe, and the relative pose data between adjacent keyframes within each data source group, using a pre-defined clustering algorithm to divide the multiple initial keyframe feature point cloud data of each data source group into cluster centers to obtain initial clustering results, wherein the initial clustering results include multiple keyframe features... Point cloud data groups; based on a preset clustering algorithm and a preset data tree depth, each keyframe feature point cloud data group is clustered sequentially to obtain the final clustering result, and a preset data dictionary is obtained based on the final clustering result; based on the preset data dictionary, each initial keyframe feature point cloud data is mapped to at least one word in the preset data dictionary, and at least one word is combined to obtain the word vector corresponding to each initial keyframe feature point cloud data; any two word vectors from multiple data sources are selected, and the similarity between the two word vectors is calculated using a preset similarity calculation strategy. When the similarity meets a preset threshold, it is determined that there is a preset relationship between the keyframe feature point cloud data pairs in the multiple data sources.

[0073] The preset data tree depth can be pre-set by researchers in the field, obtained through a limited number of experiments, or obtained through a limited number of computer operations; no specific limitation is made here. Preferably, in this embodiment, the preset data tree depth is set to 3.

[0074] As can be understood, the similarity judgment problem involves comparing the elements of two frames of feature point cloud data to determine their degree of similarity. Solving this problem can be used to address... Figure 2 The question arises whether a red connecting line exists. Constructing a bag-of-words model, which converts keyframe feature point cloud data into word vectors composed of several words, and calculating the distance between word vectors generated from two frames of feature point cloud data, is an effective method for solving similarity judgment problems. This method is widely used in the field of two-dimensional images. Based on this, this application proposes a three-dimensional bag-of-words model to extend its application to three-dimensional point cloud data.

[0075] Specifically, the data dictionary is a crucial component of the bag-of-words model, used to convert raw data into words. Similar to the DBOW (Distributed Bag of Words) model widely used in computer vision, embodiments of this application can construct the data dictionary based on data structures (such as tree structures). Figure 3 As shown, a preset clustering algorithm (such as the Keans++ algorithm) can be used to divide the multiple initial keyframe feature point cloud data of each data source into cluster centers to obtain the initial clustering result, that is, multiple keyframe feature point cloud data groups. Based on the initial clustering result, the preset clustering algorithm and the preset data tree depth, further hierarchical clustering can be performed on each keyframe feature point cloud data group. The hierarchical clustering can be controlled according to the preset data tree depth to obtain the final clustering result. The preset data dictionary can be obtained based on the final clustering result.

[0076] For example, first, the center point of the data is calculated as the root node of the tree. In the first layer, a preset clustering algorithm is used to calculate K cluster centers. Figure 3 In the first layer, K=3), the original data (multiple initial keyframe feature point cloud data from each data source) is divided into K groups to obtain the initial clustering result (multiple keyframe feature point cloud data groups). Similarly, in each subsequent layer, a preset clustering algorithm can be used to cluster the keyframe feature point cloud data groups of the previous layer sequentially. Assuming the data tree depth is d, the last layer can contain a maximum of K groups. d There are K leaf nodes, each defined as a word, meaning the pre-defined data dictionary can hold a maximum of K words. d Each word represents a cluster center or cluster group, and can be an abstract representation of feature point cloud data.

[0077] By constructing a pre-defined data dictionary using the above method, given input data, the mapping from data to words can be achieved in just d steps. This greatly reduces computational complexity and enhances the algorithm's engineering application capabilities.

[0078] It should be noted that clustering keyframe feature point cloud data using a preset clustering algorithm (such as the Keans++ algorithm) is the core process for constructing the preset data dictionary. In practical applications, since point cloud data is a set of several three-dimensional points, it only contains the three-dimensional position information of the target point and cannot provide effective feature attribute information in the clustering process. Therefore, this application embodiment uses feature descriptors to expand the point cloud data elements (i.e., additional descriptors are generated from the keyframe feature point cloud data). This descriptor is implemented based on the LinK3D method. The descriptor corresponding to each feature point is a 256-dimensional vector, which can effectively describe the local attributes of the feature point. Correspondingly, the clustering process in the preset data dictionary construction process is also based on descriptors.

[0079] By constructing a pre-defined data dictionary, keyframe feature point cloud data can be converted into word vectors. Therefore, the similarity judgment between two frames of data can be achieved by calculating the distance between word vectors. The word vector conversion of keyframe feature point cloud data is the process of mapping all feature points contained in the keyframe feature point cloud data to words in the pre-defined data dictionary, and then combining all mapped words into a single word vector. That is, after obtaining the pre-defined data dictionary, each initial keyframe feature point cloud data can be mapped to the pre-defined data dictionary. Each initial keyframe feature point cloud data can be mapped to one or more words in the pre-defined data dictionary. Combining at least one word yields the word vector corresponding to each initial keyframe feature point cloud data. To ensure the feasibility of word vector distance calculation, the consistency of word vector dimensions needs to be guaranteed. In this embodiment, the dimension can be set to K. d This refers to the preset size of the data dictionary. The specific generation process is as follows: based on the given keyframe feature point cloud data KP... i Initialize word vectors Iterate through all keyframe feature point cloud data, find the word in the dictionary and determine its position, and then analyze the word vector Wvec. i The corresponding positions are assigned weights, and the weight setting method is shown in equation (1):

[0080]

[0081] Where, n j n represents the number of times a word appears in the keyframe feature point cloud data. p N represents the number of words in the keyframe feature point cloud data, and N represents the total number of words in the process of constructing the preset data dictionary.j This represents the number of times a word appears when a predefined data dictionary is constructed.

[0082] This method, Term Frequency–Inverse Document Frequency (TF-IDF), is a commonly used weighting technique for information retrieval. Keyframe feature point cloud data KP i and KP j Generate the corresponding word vectors Wvec i and Wvec j Their similarity can be calculated using equation (2):

[0083]

[0084] Select any two word vectors from multiple data sources, and use the preset similarity calculation strategy (i.e., Equation (2)) to calculate the similarity between the two word vectors. When the similarity meets the preset threshold, it indicates that there is a connection between the key frame feature point cloud data corresponding to the two word vectors. That is, it can be determined that there is a key frame feature point cloud data pair with a preset connection in multiple data sources. Traverse all word vector pairs in multiple data sources, repeatedly calculate the similarity to determine the connection, and finally output all key frame point cloud data pairs that are determined to have a preset connection.

[0085] In step S103, if there are key frame feature point cloud data pairs with a preset relationship in multiple data sources, then the relationship between the key frame feature point cloud data pairs with the preset relationship is quantified based on the preset three-dimensional bag-of-words model.

[0086] In other words, after completing the similarity judgment between keyframe feature point cloud data, quantifying the relationship between keyframe feature point cloud data pairs that have a pre-defined relationship is one of the key steps in the fusion process.

[0087] The following section details how to quantify the relationships between keyframe feature point cloud data pairs that have pre-defined connections.

[0088] As one possible implementation, in some embodiments, based on a preset three-dimensional bag-of-words model, the relationship between key frame feature point cloud data pairs with preset relationships is quantified, including: using a preset function, based on the relative poses between key frame feature point cloud data pairs with preset relationships and key frame feature point cloud data in key frame feature point cloud data pairs with preset relationships, to solve for the optimal solution of the pose data corresponding to each key frame in each data source.

[0089] In some embodiments, the preset function is:

[0090]

[0091] Where Φ is an abstract representation of the pose data corresponding to the keyframe, Ω is an abstract representation of the relative pose between keyframe feature point cloud data with a pre-defined relationship, Ψ∈Φ is a set of keyframe feature point cloud data pairs with a pre-defined relationship, and Δ is the relative pose between keyframe feature point cloud data determined by the GICP (Generalized Iterative Closest Point, a point cloud registration method) algorithm. The elements within the set of keyframe feature point cloud data pairs with a pre-defined relationship can be defined as... This indicates that the feature point cloud data of the m-th keyframe in the j-th data source has similarity to the feature point cloud data of the n-th keyframe in the k-th data source, and the relative pose calculated by the GICP algorithm is Δ. i ={p ri ,R ri}

[0092] Specifically, such as Figure 4 As shown, the embodiments of this application can quantify the relative poses between key frame feature point cloud data in key frame feature point cloud data pairs with preset relationships and the relative poses between key frame feature point cloud data in key frame feature point cloud data pairs with preset relationships through the GICP algorithm. The data fusion problem is defined as the problem of finding the optimal solution for the pose data corresponding to each key frame in multiple data sources, which can be calculated using the preset function shown in equation (3).

[0093] The function h(Φ) in equation (3) i ,Ω i The specific form of ) can be shown in equation (4):

[0094]

[0095] in, This is a generalized subtraction method used to calculate attitude error.

[0096] The function g(Ψ) in equation (3) i ,Δ i The specific form of ) can be shown in equation (5):

[0097]

[0098] Based on the specific mathematical form of the optimization problem defined by equation (3), its solution can be achieved through an iterative algorithm. In this embodiment, it is implemented based on the Levenberg-Marquardt (LM) algorithm (an estimation method for least squares estimation of regression parameters in nonlinear regression).

[0099] According to the crowdsourced map data fusion method proposed in this application, by receiving multiple initial keyframe feature point cloud data uploaded by vehicles, pose data corresponding to each keyframe, and relative pose data between adjacent keyframes within each data source group, and based on the multiple initial keyframe feature point cloud data, pose data corresponding to each keyframe, and relative pose data between adjacent keyframes within each data source group, when determining that there are keyframe feature point cloud data pairs with a preset relationship in multiple data sources, the relationship between the keyframe feature point cloud data pairs with a preset relationship can be quantified based on a preset 3D bag-of-words model. Therefore, by achieving rapid data search and matching based on the 3D bag-of-words model, the problem of high-precision map updates failing to effectively guarantee the timeliness of map element updates is solved, improving the update speed and accuracy of high-precision map elements.

[0100] Next, the crowdsourced map data fusion apparatus proposed according to the embodiments of this application is described with reference to the accompanying drawings.

[0101] Figure 5 This is a block diagram of a crowdsourced map data fusion apparatus according to an embodiment of this application.

[0102] like Figure 5 As shown, the crowdsourced map data fusion device 10 includes: a receiving module 100, a judging module 200, and a quantization representation module 300.

[0103] The receiving module 100 is used to receive multiple sets of data sources uploaded by the vehicle. Each set of data sources includes multiple initial keyframe feature point cloud data, pose data corresponding to each keyframe, and relative pose data between adjacent keyframes within each set of data sources.

[0104] The judgment module 200 is used to determine whether there are pre-connected key frame feature point cloud data pairs in multiple data sources based on the multiple initial key frame feature point cloud data of each data source, the pose data corresponding to each key frame, and the relative pose data between adjacent key frames in each data source.

[0105] The quantization representation module 300 is used to quantize the relationship between keyframe feature point cloud data pairs that have a preset relationship in multiple data sources, based on a preset three-dimensional bag-of-words model.

[0106] Furthermore, in some embodiments, the determination module 200 is specifically used for:

[0107] Based on multiple initial keyframe feature point cloud data of each data source, pose data corresponding to each keyframe, and relative pose data between adjacent keyframes within each data source, a preset clustering algorithm is used to divide the multiple initial keyframe feature point cloud data of each data source into cluster centers to obtain initial clustering results. The initial clustering results include multiple keyframe feature point cloud data groups.

[0108] Based on the preset clustering algorithm and preset data tree depth, each keyframe feature point cloud data group is clustered sequentially to obtain the final clustering result, and a preset data dictionary is obtained based on the final clustering result.

[0109] Based on a preset data dictionary, each initial keyframe feature point cloud data is mapped to at least one word in the preset data dictionary, and the at least one word is combined to obtain the word vector corresponding to each initial keyframe feature point cloud data.

[0110] Select any two word vectors from multiple data sources, calculate the similarity between the two word vectors using a preset similarity calculation strategy, and determine that the key frame feature point cloud data pairs in the multiple data sources have a preset relationship when the similarity meets the preset threshold.

[0111] Furthermore, in some embodiments, the quantification expression module 300 is specifically used for:

[0112] Using a preset function, based on the key frame feature point cloud data pairs with preset relationships and the relative poses between the key frame feature point cloud data in the key frame feature point cloud data pairs with preset relationships, the optimal solution for the pose data corresponding to each key frame in each data source is obtained.

[0113] According to one embodiment of this application, the preset function is:

[0114]

[0115] Where Φ is the abstract representation of the pose data corresponding to the keyframe, Ω is the abstract representation of the relative pose between keyframe feature point cloud data with a pre-defined relationship, Ψ∈Φ is the set of keyframe feature point cloud data pairs with a pre-defined relationship, and Δ is the relative pose between keyframe feature point cloud data.

[0116] Furthermore, in some embodiments, multiple initial keyframe feature point cloud data, pose data corresponding to each keyframe, and relative pose data between adjacent keyframes within each data source are obtained by the vehicle based on a preset LIO-SAM algorithm, which performs feature extraction and local mapping processing on the collected lidar data, inertial measurement data, and GPS / RTK measurement data.

[0117] It should be noted that the foregoing explanation of the embodiment of the crowdsourced map data fusion method also applies to the crowdsourced map data fusion device of this embodiment, and will not be repeated here.

[0118] According to the crowdsourced map data fusion device proposed in this application, by receiving multiple initial keyframe feature point cloud data uploaded by vehicles, pose data corresponding to each keyframe, and relative pose data between adjacent keyframes within each data source group, and based on the multiple initial keyframe feature point cloud data, pose data corresponding to each keyframe, and relative pose data between adjacent keyframes within each data source group, when determining that there are keyframe feature point cloud data pairs with a preset relationship in multiple data sources, the device can quantify the relationship between keyframe feature point cloud data pairs with a preset relationship based on a preset three-dimensional bag-of-words model. Therefore, by achieving rapid data search and matching based on the three-dimensional bag-of-words model, the device solves the problem that high-precision map updates cannot effectively guarantee the timeliness of map element updates, and improves the update speed and accuracy of high-precision map elements.

[0119] Figure 6 A schematic diagram of the structure of a vehicle provided in an embodiment of this application. The vehicle may include:

[0120] The memory 601, the processor 602, and the computer program stored on the memory 601 and capable of running on the processor 602.

[0121] When the processor 602 executes the program, it implements the crowdsourced map data fusion method provided in the above embodiments.

[0122] Furthermore, the vehicle also includes:

[0123] Communication interface 603 is used for communication between memory 601 and processor 602.

[0124] The memory 601 is used to store computer programs that can run on the processor 602.

[0125] The memory 601 may include high-speed RAM (Random Access Memory) memory, and may also include non-volatile memory, such as at least one disk storage.

[0126] If the memory 601, processor 602, and communication interface 603 are implemented independently, then the communication interface 603, memory 601, and processor 602 can be interconnected via a bus to complete communication between them. The bus can be an ISA (Industry Standard Architecture) bus, a PCI (Peripheral Component Interconnect) bus, or an EISA (Extended Industry Standard Architecture) bus, etc. The bus can be divided into address bus, data bus, control bus, etc. For ease of representation, Figure 6 The bus is represented by a single thick line, but this does not mean that there is only one bus or one type of bus.

[0127] Optionally, in a specific implementation, if the memory 601, processor 602, and communication interface 603 are integrated on a single chip, then the memory 601, processor 602, and communication interface 603 can communicate with each other through an internal interface.

[0128] The processor 602 may be a CPU (Central Processing Unit), an ASIC (Application Specific Integrated Circuit), or one or more integrated circuits configured to implement the embodiments of this application.

[0129] This application also provides a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the above-described crowdsourced map data fusion method.

[0130] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this application, "multiple" means at least two, such as two, three, etc., unless otherwise explicitly specified.

[0131] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of this application. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of different embodiments or examples.

[0132] Although embodiments of this application have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting this application. Those skilled in the art can make changes, modifications, substitutions and variations to the above embodiments within the scope of this application.

Claims

1. A method for fusing crowdsourced map data, characterized in that, Includes the following steps: Receive multiple sets of data sources uploaded by the vehicle. Each set of data sources includes multiple initial keyframe feature point cloud data, pose data corresponding to each keyframe, and relative pose data between adjacent keyframes within each set of data sources. Based on the multiple initial keyframe feature point cloud data of each group of data sources, the pose data corresponding to each keyframe, and the relative pose data between adjacent keyframes within each group of data sources, it is determined whether there are keyframe feature point cloud data pairs with a preset connection in the multiple groups of data sources. If there are key frame feature point cloud data pairs with a preset relationship in the multiple data sources, then the relationship between the key frame feature point cloud data pairs with a preset relationship is quantified based on the preset three-dimensional bag-of-words model. The step of determining whether there are pre-defined keyframe feature point cloud data pairs in the multiple data sources based on the multiple initial keyframe feature point cloud data, the pose data corresponding to each keyframe, and the relative pose data between adjacent keyframes within each data source group includes: using a pre-defined clustering algorithm to divide the multiple initial keyframe feature point cloud data of each data source group into cluster centers based on the multiple initial keyframe feature point cloud data, the pose data corresponding to each keyframe, and the relative pose data between adjacent keyframes within each data source group, to obtain an initial clustering result, wherein the initial clustering result includes multiple keyframe feature point cloud data groups; and based on the pre-defined clustering algorithm and a pre-defined data tree depth, respectively... Each keyframe feature point cloud data group is clustered sequentially to obtain a final clustering result, and a preset data dictionary is obtained based on the final clustering result. Based on the preset data dictionary, each initial keyframe feature point cloud data is mapped to at least one word in the preset data dictionary, and the at least one word is combined to obtain the word vector corresponding to each initial keyframe feature point cloud data. Any two word vectors from the multiple data sources are selected, and the similarity between the two word vectors is calculated using a preset similarity calculation strategy. When the similarity meets a preset threshold, it is determined that there is a keyframe feature point cloud data pair with a preset relationship in the multiple data sources, and the preset three-dimensional bag-of-words model is obtained based on the keyframe feature point cloud data pair with a preset relationship. The quantitative representation of the relationship between the key frame feature point cloud data pairs with a preset relationship based on the preset three-dimensional bag-of-words model includes: using a preset function, based on the key frame feature point cloud data pairs with a preset relationship and the relative pose between the key frame feature point cloud data in the key frame feature point cloud data pairs with a preset relationship, to solve for the optimal solution of the pose data corresponding to each key frame in each data source. The preset function is: in, This is an abstract representation of the pose data corresponding to the keyframe. This is an abstract representation of the relative poses between keyframe feature point cloud data that have a pre-defined relationship. The set of keyframe feature point cloud data pairs that have a preset relationship. The relative poses between the keyframe feature point cloud data. This refers to the pose constraint error between adjacent keyframes within the same set of data sources. This refers to the pose constraint error between keyframes that are related to different groups of data sources.

2. The method according to claim 1, characterized in that, The multiple initial keyframe feature point cloud data, the pose data corresponding to each keyframe, and the relative pose data between adjacent keyframes in each group of data sources are obtained by the vehicle based on the preset LIO-SAM algorithm, which performs feature extraction and local mapping processing on the collected lidar data, inertial measurement data, and GPS / RTK measurement data.

3. A crowdsourced map data fusion device, characterized in that, include: The receiving module is used to receive multiple sets of data sources uploaded by the vehicle. Each set of data sources includes multiple initial keyframe feature point cloud data, pose data corresponding to each keyframe, and relative pose data between adjacent keyframes within each set of data sources. The judgment module is used to determine whether there are key frame feature point cloud data pairs with preset relationships in the multiple groups of data sources based on the multiple initial key frame feature point cloud data of each group of data sources, the pose data corresponding to each key frame, and the relative pose data between adjacent key frames in each group of data sources. The quantization representation module is used to quantize the relationship between keyframe feature point cloud data pairs that have a preset relationship in the multiple sets of data sources, based on the preset three-dimensional bag-of-words model. Specifically, the judgment module is used to: based on multiple initial keyframe feature point cloud data from each data source group, the pose data corresponding to each keyframe, and the relative pose data between adjacent keyframes within each data source group, divide the multiple initial keyframe feature point cloud data from each data source group into cluster centers using a preset clustering algorithm to obtain an initial clustering result, wherein the initial clustering result includes multiple keyframe feature point cloud data groups; based on the preset clustering algorithm and a preset data tree depth, sequentially cluster each keyframe feature point cloud data group to obtain a final clustering result, and obtain a result based on the final clustering result. A preset data dictionary is used; based on the preset data dictionary, each initial keyframe feature point cloud data is mapped to at least one word in the preset data dictionary, and the at least one word is combined to obtain the word vector corresponding to each initial keyframe feature point cloud data; any two word vectors from the multiple sets of data sources are selected, and the similarity between the two word vectors is calculated using a preset similarity calculation strategy. When the similarity meets a preset threshold, it is determined that there is a keyframe feature point cloud data pair with a preset relationship in the multiple sets of data sources, and the preset three-dimensional bag-of-words model is obtained based on the keyframe feature point cloud data pair with a preset relationship. The quantization representation module is specifically used to: using a preset function, based on the key frame feature point cloud data pairs with preset relationships and the relative poses between the key frame feature point cloud data in the key frame feature point cloud data pairs with preset relationships, to solve for the optimal solution of the pose data corresponding to each key frame in each data source. The preset function is: in, This is an abstract representation of the pose data corresponding to the keyframe. This is an abstract representation of the relative poses between keyframe feature point cloud data that have a pre-defined relationship. The set of keyframe feature point cloud data pairs that have a preset relationship. The relative poses between the keyframe feature point cloud data. This refers to the pose constraint error between adjacent keyframes within the same set of data sources. This refers to the pose constraint error between keyframes that are related to different groups of data sources.

4. A vehicle, characterized in that, include: A memory, a processor, and a computer program stored in the memory and executable on the processor, the processor executing the program to implement the crowdsourced map data fusion method as described in any one of claims 1-2.

5. A computer-readable storage medium having a computer program stored thereon, characterized in that, The program is executed by the processor to implement the crowdsourced map data fusion method as described in any one of claims 1-2.

Images