Map data processing method and device

Abstract

The invention discloses a map data processing method and device, relates to the technical field of Internet of Vehicles, and can realize more accurate map construction by using data uploaded to a cloud end by a crowdsourcing vehicle. The method comprises the steps that at least one crowdsourcing sub-map obtained by segmenting crowdsourcing data and at least one historical sub-map obtained by segmenting historical map data are obtained, and the crowdsourcing data can be used for updating the historical map data. And then, according to the road network hierarchy corresponding to the at least one historical sub-map, determining the road network hierarchy corresponding to the at least one crowdsourcing sub-map, and according to the at least one historical sub-map, determining the pose of the at least one crowdsourcing sub-map, and the at least one crowdsourcing sub-map after determining the road network hierarchy and the pose can be used for optimizing crowdsourcing data. And finally, carrying out quality inspection on the optimized crowdsourcing data according to historical map data.

Description

Technical Field

[0001] The present application relates to the field of maps, and in particular to a method and device for processing map data. Background Art

[0002] At present, the positioning and navigation of vehicles all need to rely on map services, and map services are undergoing a transition from a centralized mapping model to a crowdsourcing mapping model. Among them, the centralized mapping model can refer to a model in which map suppliers use professional mapping vehicles to collect composition data and use a series of algorithms to achieve fusion mapping. In actual driving, road conditions often change, such as road closures, traffic sign changes, etc. In order to ensure the accuracy of the map, the map supplier needs to dispatch a mapping vehicle to collect the changed roads for a second time to achieve map updates.

[0003] However, due to the high cost and limited number of professional surveying and mapping vehicles, the timeliness of map updates is low, and frequent use of professional surveying and mapping vehicles to update maps will incur high costs. In order to solve the above problems, the crowdsourcing mapping model came into being. Among them, the crowdsourcing mapping model can refer to the use of existing acquisition equipment on low-cost and large-scale crowdsourcing vehicles (such as private cars used for family travel or commuting, buses that shuttle on fixed routes, and other non-dedicated surveying and mapping vehicles) to collect composition data, and upload the collected composition data to the cloud for map construction and update. However, due to the limitations of the sensor accuracy of the crowdsourcing vehicle and surveying and mapping regulations, the data uploaded to the cloud by the crowdsourcing vehicle may have various problems, such as: lack of elevation information, and one or more of global non-rigid body deviations and local rigid body deviations between the historical map data stored in the cloud, and then when the data with various problems is used to update the historical map data stored in the cloud, the accuracy of the updated map may be low. Summary of the invention

[0004] The embodiments of the present application provide a map data processing method and device, which can use the data uploaded to the cloud by crowdsourcing vehicles to achieve more accurate map construction.

[0005] In order to achieve the above objectives, this application adopts the following technical solutions:

[0006] In a first aspect, a map data processing method is provided, the method comprising: obtaining at least one crowdsourced sub-map and at least one historical sub-map, the at least one crowdsourced sub-map being obtained by segmenting crowdsourced data, the at least one historical sub-map being obtained by segmenting historical map data, the crowdsourced data being used to update the historical map data; determining a road network level corresponding to the at least one crowdsourced sub-map according to a road network level corresponding to the at least one historical sub-map; determining a position and posture of the at least one crowdsourced sub-map according to the at least one historical sub-map, and using the at least one crowdsourced sub-map after determining the road network level and the position and posture to optimize the crowdsourced data; and performing quality inspection on the optimized crowdsourced data according to the historical map data.

[0007] Based on the above technical solution, after obtaining the crowdsourcing sub-map obtained by segmenting the crowdsourcing data and the historical sub-map obtained by segmenting the historical map data, the road network level of the crowdsourcing sub-map can be determined according to the road network level corresponding to the historical sub-map, and then the relative height information of the crowdsourcing sub-map can be obtained, and the position and posture of the crowdsourcing sub-map can be determined according to the historical sub-map, so that the crowdsourcing sub-map and the historical sub-map can be aligned. In this way, when optimizing the crowdsourcing data according to the crowdsourcing sub-map with the aforementioned determined position and road network level, the optimized crowdsourcing data can be aligned with the historical map data, and the optimized crowdsourcing data can contain relative height information. Then, when the optimized crowdsourcing data is used to update the historical map data, not only can the road network binding be achieved, that is, the road network information (such as road identity) is bound to the road network in the crowdsourcing data to achieve road network update, but also the updated historical map can be made more accurate. In addition, due to various reasons such as changes in road conditions and registration errors, there may be inconsistencies between the old and new maps. Therefore, the optimized crowdsourced data is quality-checked based on historical map data, which can automatically detect the aforementioned problems. Compared with manual quality inspection methods, the quality inspection efficiency can be improved, thereby improving the efficiency of map construction.

[0008] In one possible design, determining the road network level corresponding to the at least one crowdsourced sub-map according to the road network level corresponding to the at least one historical sub-map includes: determining the road network level corresponding to the second sub-map according to the road network level corresponding to at least one first sub-map, the first sub-map being a historical sub-map that has a matching relationship with the second sub-map and whose corresponding road network level is a single layer, and the second sub-map being any one of the at least one crowdsourced sub-map; using the second sub-map with the road network level determined as a seed point, and determining the road network levels corresponding to some or all of the sub-maps in the at least one crowdsourced sub-map through regional growing.

[0009] Based on this design, the road network level corresponding to a crowdsourced submap is determined based on a historical submap with a single-layer road network level, and then the crowdsourced submap is used as a seed point to determine the road network level corresponding to some or all submaps in at least one crowdsourced submap through regional growth. There is no need to determine the road network level corresponding to the crowdsourced submaps with matching relationships based on the road network level corresponding to each historical submap, which can reduce the complexity of the algorithm, reduce the time spent in determining the road network level corresponding to the crowdsourced submap, and reduce the power consumption of the device.

[0010] In one possible design, determining a road network level corresponding to a second submap based on a road network level corresponding to at least one first submap includes: determining a road network level corresponding to the second submap based on a point cloud registration result between the at least one first submap and the second submap, and the road network level corresponding to the at least one first submap.

[0011] Based on this design, by performing point cloud registration between the historical sub-map and the crowd-sourced sub-map and combining the road network level corresponding to the historical sub-map, the road network level corresponding to the crowd-sourced sub-map can be determined, thereby realizing the road network level binding of the crowd-sourced sub-map.

[0012] In one possible design, after determining the road network levels corresponding to some sub-maps in the at least one crowdsourced sub-map through region growing, the method further includes: determining the road network level corresponding to a fourth sub-map according to the road network level corresponding to at least one third sub-map, the at least one third sub-map being a historical sub-map that has a matching relationship with the fourth sub-map and whose corresponding road network level is multi-layer, and the fourth sub-map being a crowdsourced sub-map whose road network level is not determined through the region growing.

[0013] Based on this design, for crowdsourced sub-maps whose corresponding road network levels cannot be determined through regional growing, that is, crowdsourced sub-maps that cannot be grown through regional growing, the road network level of the crowdsourced sub-map can be determined based on the road network level of historical sub-maps with multi-layer road network levels that have a matching relationship with the crowdsourced sub-map, thereby determining the road network level of all crowdsourced sub-maps obtained by dividing the crowdsourced data, and obtaining the complete road network level information corresponding to the crowdsourced data.

[0014] In one possible design, determining the road network level corresponding to the fourth submap based on the road network level corresponding to at least one third submap includes: determining the road network level corresponding to the fourth submap based on the point cloud alignment results of each road network layer in the fourth submap and each third submap, and the level corresponding to each road network layer in each third submap.

[0015] Based on this design, point cloud registration is performed on each layer of road network in the crowdsourced sub-map and the historical sub-map, and then the road network level of the crowdsourced sub-map is determined in combination with the level corresponding to each layer of the road network, so that a more accurate road network level of the crowdsourced sub-map can be obtained.

[0016] In one possible design, before determining the road network level corresponding to the fourth submap based on the road network level corresponding to at least one third submap, the method further includes: clustering the road centerlines contained in the first submap and the third submap respectively to obtain the number of roads contained in the first submap and the number of roads contained in the third submap, the road centerlines being used to divide the historical map data into the at least one historical submap; determining the road network level corresponding to the first submap based on the number of roads contained in the first submap and the level corresponding to each road contained in the first submap; determining the road network level corresponding to the third submap based on the number of roads contained in the third submap and the level corresponding to each road contained in the third submap.

[0017] Based on this design, the historical sub-map is divided according to the road centerlines. In this way, by clustering the road centerlines in the historical sub-map, the number of roads contained in the historical sub-map can be determined. Then, the road network level corresponding to the historical sub-map can be determined in combination with the road network level corresponding to each road. The determination of the road network level of the historical sub-map can be realized, so that the road network level of the crowdsourcing sub-map can be determined later according to the road network level corresponding to the historical sub-map.

[0018] In a possible design, determining the position and posture of the at least one crowdsourced sub-map based on the at least one historical sub-map includes: obtaining parameters, the parameters including at least one of a first registration result and a second registration result, the first registration result being a point cloud registration result between a fifth sub-map and a sixth sub-map, the fifth sub-map being a historical sub-map that has a matching relationship with the sixth sub-map, and the sixth sub-map being any one of the at least one crowdsourced sub-maps; the second registration result being an image registration result between a first bird's-eye view image and a second bird's-eye view image, the first bird's-eye view image being obtained based on the fifth sub-map, and the second bird's-eye view image being obtained based on the sixth sub-map; and determining the position and posture of the sixth sub-map based on the parameters.

[0019] Based on this design, the position and posture of the crowdsourced sub-map are determined according to the point cloud registration results obtained by performing point cloud registration between the crowdsourced sub-map and the historical sub-map with a matching relationship, as well as the image registration results obtained by performing image registration between the bird's-eye view image converted from the crowdsourced sub-map and the bird's-eye view image converted from the historical sub-map. This can achieve registration between crowdsourced data and historical map data and solve the problem of local rigid body deviation between crowdsourced data and historical map data.

[0020] In one possible design, the first registration result includes a first relative posture between the fifth submap and the sixth submap, and the second registration result includes a second relative posture between the fifth submap and the sixth submap; and determining the posture of the sixth submap according to the parameters includes: adjusting an initial posture of the sixth submap according to the first relative posture and the second relative posture to obtain the posture of the sixth submap.

[0021] Based on this design, the point cloud registration result includes the first relative pose between the crowdsourcing sub-map and the historical sub-map, and the image registration result includes the second relative pose between the crowdsourcing sub-map and the historical sub-map. The initial pose of the crowdsourcing sub-map is adjusted based on the first relative pose and the second relative pose to obtain the pose of the crowdsourcing sub-map. In other words, the pose of the crowdsourcing sub-map is adjusted by combining the point cloud registration result and the image registration result. Point cloud registration belongs to local registration, while image registration belongs to global registration. In this way, the pose obtained by image registration can optimize the pose obtained by image registration, reduce the error of point cloud registration, improve the accuracy of crowdsourcing data registration, and make the optimized pose of crowdsourcing data more accurate.

[0022] In a possible design, the parameters also include one or more of prior information and mileage information; wherein the prior information includes an initial position of the sixth submap; the mileage information includes a relative position between the sixth submap and a seventh submap, and the seventh submap is a crowdsourced submap adjacent to the sixth submap.

[0023] Based on this design, the parameters also include one or more of the initial posture of the crowdsourced sub-map and the relative posture between adjacent crowdsourced sub-maps. When the posture of the crowdsourced sub-map is optimized based on the parameters, a more accurate posture of the crowdsourced sub-map can be obtained, global optimization can be achieved, and the global non-rigid deviation between crowdsourcing data and historical map data can be better resolved.

[0024] In one possible design, the parameters include a second registration result; and the acquisition of the parameters includes: using the first bird's-eye view image as a template, performing template matching on the second bird's-eye view image and the template to determine the second relative posture.

[0025] Based on this design, the bird's-eye view image corresponding to the historical sub-map is used as a template to align the bird's-eye view image corresponding to the crowdsourcing sub-map, which can achieve global alignment and make the pose of the optimized crowdsourcing data more accurate.

[0026] In a possible design, the parameters include a first registration result; and the acquiring parameters includes: dividing the point cloud contained in the fifth submap into at least one element of points, lines, and surfaces according to the semantic information of the point cloud contained in the fifth submap; dividing the point cloud contained in the sixth submap into at least one element of points, lines, and surfaces according to the semantic information of the point cloud contained in the sixth submap; determining the residual between each element with the same semantic information in the fifth submap and the sixth submap; and determining the first relative pose according to the residual.

[0027] Based on this design, the result of point cloud registration is determined based on the semantic information of the historical sub-map and the crowd-sourced sub-map, which can avoid the interference of outliers and dynamic points and improve the robustness of point cloud registration. In addition, the first relative pose is determined based on the residuals between the elements of various semantic information, that is, the result of point cloud registration is determined. Compared with the method of determining the first relative pose based on the residuals between single semantic elements, the first relative pose can be determined more accurately, that is, the result of point cloud registration can be made more accurate, improve the robustness of point cloud registration, and further better solve the problem of local rigid body deviation between crowd-sourced data and historical map data.

[0028] In a possible design, determining the residuals between elements of the same semantic information in the fifth submap and the sixth submap includes: when the elements of the same semantic information are points, the residuals between the points are determined according to the distances between the points; or, when the elements of the same semantic information are lines, the residuals between the lines are determined according to the distances from points on one line to another line; or, when the elements of the same semantic information are surfaces, the residuals between the surfaces are determined according to the distances from points on one surface to another surface.

[0029] Based on this design, different methods are used to calculate the residuals for different elements of the same semantic information, and then the point cloud registration results are obtained according to the determined residuals. That is, the fusion mapping problem is converted into a multimodal least squares problem, which can better solve the local rigid body deviation problem between crowdsourcing data and historical map data.

[0030] In one possible design, the quality inspection of the optimized crowd-sourced data according to the historical map data includes: obtaining the optimized crowd-sourced data; projecting the point cloud in the optimized crowd-sourced data onto the target point cloud according to the semantic information of the point cloud in the optimized crowd-sourced data and the semantic information of the target point cloud in the historical map data, wherein the target point cloud matches the point cloud in the optimized crowd-sourced data; outputting a quality inspection label according to the projection result and preset inspection rules, wherein the quality inspection label includes: an index of a crowd-sourced sub-map that meets the preset inspection rules, and one or more corresponding preset inspection rules.

[0031] Based on this design, the point cloud in the optimized crowdsourcing data is projected onto the point cloud in the historical map data with a matching relationship based on semantic information, and the quality inspection label is output according to the projection result. It can realize the automatic detection of inconsistent areas between crowdsourcing data and historical map data. Compared with the manual detection method, it can improve the detection efficiency and reduce the cost. In addition, detection is performed based on the semantic information of the point cloud, so that only the elements of interest can be detected. Compared with the detection method based on geometric features, it will not be affected by outliers, dynamic objects or seasonally changing objects, which can improve the accuracy of the detection results.

[0032] In a possible design, the preset inspection rule includes one or more of the following: for an optimized crowd-sourced sub-map, determining, according to the projection result, that an average residual corresponding to the optimized crowd-sourced sub-map is greater than or equal to a first threshold; for an optimized crowd-sourced sub-map, determining, according to the projection result, that an area of ​​a point cloud whose residual in the optimized crowd-sourced sub-map is a large residual is greater than or equal to a second threshold, wherein the large residual means that the residual corresponding to the point cloud contained in the optimized crowd-sourced sub-map is greater than or equal to the first threshold; the sum of the first area and the second area is greater than or equal to a third threshold, wherein the first area is the sum of the areas of a first type of point cloud contained in an optimized crowd-sourced sub-map, and there is no historical point cloud in the neighborhood of the first type of point cloud, and the historical point cloud is a point cloud in a historical sub-map that has a matching relationship with the optimized crowd-sourced sub-map; the second area is the sum of the areas of a second type of point cloud contained in a historical sub-map that has a matching relationship with the optimized crowd-sourced sub-map, and there is no point cloud in the optimized crowd-sourced sub-map in the neighborhood of the second type of point cloud; wherein the optimized crowd-sourced sub-map is obtained by segmenting the optimized crowd-sourced data.

[0033] Based on this design, by setting these preset inspection rules, automatic detection of inconsistent areas between crowdsourced data and historical map data can be achieved.

[0034] In a second aspect, a map data processing device is provided, the map data processing device includes a module or unit corresponding to the above method, the module or unit can be implemented by hardware, software, or by hardware executing the corresponding software. In a possible design, the map data processing device includes a processing unit (or processing module); the processing unit is used to: obtain at least one crowdsourced sub-map and at least one historical sub-map, the at least one crowdsourced sub-map is obtained by segmenting crowdsourced data, the at least one historical sub-map is obtained by segmenting historical map data, and the crowdsourced data is used to update the historical map data; determine the road network level corresponding to the at least one crowdsourced sub-map according to the road network level corresponding to the at least one historical sub-map; determine the posture of the at least one crowdsourced sub-map according to the at least one historical sub-map, and the at least one crowdsourced sub-map after determining the road network level and posture is used to optimize the crowdsourced data; perform quality inspection on the optimized crowdsourced data according to the historical map data.

[0035] In one possible design, the processing unit is specifically used to: determine the road network level corresponding to the second submap according to the road network level corresponding to at least one first submap, the first submap is a historical submap that has a matching relationship with the second submap and the corresponding road network level is a single layer, and the second submap is any one of the at least one crowdsourced submap; use the second submap with the road network level determined as a seed point, and determine the road network levels corresponding to some or all of the submaps in the at least one crowdsourced submap through regional growing.

[0036] In one possible design, the processing unit is further used to determine the road network level corresponding to the second submap based on the point cloud alignment result between the at least one first submap and the second submap, and the road network level corresponding to the at least one first submap.

[0037] In one possible design, the processing unit is further used to determine the road network level corresponding to the fourth sub-map based on the road network level corresponding to at least one third sub-map, wherein the at least one third sub-map is a historical sub-map that has a matching relationship with the fourth sub-map and the corresponding road network level is a multi-layer, and the fourth sub-map is a crowdsourced sub-map whose road network level is not determined through the regional growth.

[0038] In one possible design, the processing unit is specifically used to determine the road network level corresponding to the fourth submap based on the point cloud alignment results of each road network layer in the fourth submap and each third submap, and the level corresponding to each road network layer in each third submap.

[0039] In one possible design, the processing unit is further used to: cluster the road centerlines contained in the first sub-map and the third sub-map respectively to obtain the number of roads contained in the first sub-map and the number of roads contained in the third sub-map, the road centerlines being used to divide the historical map data into the at least one historical sub-map; determine the road network level corresponding to the first sub-map according to the number of roads contained in the first sub-map and the level corresponding to each road contained in the first sub-map; determine the road network level corresponding to the third sub-map according to the number of roads contained in the third sub-map and the level corresponding to each road contained in the third sub-map.

[0040] In a possible design, the processing unit is specifically used to: obtain parameters, where the parameters include at least one of a first registration result and a second registration result, where the first registration result is a point cloud registration result between a fifth submap and a sixth submap, where the fifth submap is a historical submap that has a matching relationship with the sixth submap, and the sixth submap is any one of the at least one crowdsourced submap; the second registration result is an image registration result between a first bird's-eye view image and a second bird's-eye view image, where the first bird's-eye view image is obtained based on the fifth submap, and the second bird's-eye view image is obtained based on the sixth submap; and determine the posture of the sixth submap based on the parameters.

[0041] In one possible design, the first registration result includes a first relative posture between the fifth submap and the sixth submap, and the second registration result includes a second relative posture between the fifth submap and the sixth submap; the processing unit is specifically used to adjust the initial posture of the sixth submap according to the first relative posture and the second relative posture to obtain the posture of the sixth submap.

[0042] In a possible design, the parameters also include one or more of prior information and mileage information; wherein the prior information includes an initial position of the sixth submap; the mileage information includes a relative position between the sixth submap and a seventh submap, and the seventh submap is a crowdsourced submap adjacent to the sixth submap.

[0043] In one possible design, the parameters include a second registration result; and a processing unit is specifically used to use the first bird's-eye view image as a template, perform template matching on the second bird's-eye view image and the template, and determine the second relative posture.

[0044] In a possible design, the parameters include a first registration result; a processing unit is specifically used to: divide the point cloud contained in the fifth submap into at least one element of points, lines, and surfaces according to the semantic information of the point cloud contained in the fifth submap; divide the point cloud contained in the sixth submap into at least one element of points, lines, and surfaces according to the semantic information of the point cloud contained in the sixth submap; determine the residual between each element with the same semantic information in the fifth submap and the sixth submap; and determine the first relative pose according to the residual.

[0045] In one possible design, the elements of the same semantic information are points, and the processing unit is used to determine the residual between the points based on the distance between the points; or, the elements of the same semantic information are lines, and the processing unit is used to determine the residual between the lines based on the distance from a point on one line to another line; or, the elements of the same semantic information are surfaces, and the processing unit is used to determine the residual between the surfaces based on the distance from a point on one surface to another surface.

[0046] In one possible design, the processing unit is specifically used to: obtain the optimized crowd-sourcing data; project the point cloud in the optimized crowd-sourcing data onto the target point cloud according to the semantic information of the point cloud in the optimized crowd-sourcing data and the semantic information of the target point cloud in the historical map data, wherein the target point cloud matches the point cloud in the optimized crowd-sourcing data; output a quality inspection label according to the projection result and preset inspection rules, wherein the quality inspection label includes: an index of a crowd-sourcing sub-map that meets the preset inspection rules, and one or more corresponding preset inspection rules.

[0047] In a possible design, the preset inspection rule includes one or more of the following: for an optimized crowd-sourced sub-map, determining, according to the projection result, that an average residual corresponding to the optimized crowd-sourced sub-map is greater than or equal to a first threshold; for an optimized crowd-sourced sub-map, determining, according to the projection result, that an area of ​​a point cloud whose residual in the optimized crowd-sourced sub-map is a large residual is greater than or equal to a second threshold, wherein the large residual means that the residual corresponding to the point cloud contained in the optimized crowd-sourced sub-map is greater than or equal to the first threshold; the sum of the first area and the second area is greater than or equal to a third threshold, wherein the first area is the sum of the areas of a first type of point cloud contained in an optimized crowd-sourced sub-map, and there is no historical point cloud in the neighborhood of the first type of point cloud, and the historical point cloud is a point cloud in a historical sub-map that has a matching relationship with the optimized crowd-sourced sub-map; the second area is the sum of the areas of a second type of point cloud contained in a historical sub-map that has a matching relationship with the optimized crowd-sourced sub-map, and there is no point cloud in the optimized crowd-sourced sub-map in the neighborhood of the second type of point cloud; wherein the optimized crowd-sourced sub-map is obtained by segmenting the optimized crowd-sourced data.

[0048] In a third aspect, a map data processing device is provided, comprising a processor, the processor being coupled to a memory; the processor being configured to execute a computer program stored in the memory, so that the map data processing device performs the method as described in the first aspect and any one of the designs thereof. Optionally, the memory may be coupled to the processor or may be independent of the processor.

[0049] In a possible design, the map data processing device further includes a communication interface, which can be used for the map data processing device to communicate with other devices. Exemplarily, the communication interface can be a transceiver, an input / output interface, an interface circuit, an output circuit, an input circuit, a pin or a related circuit, etc.

[0050] In a fourth aspect, a computer-readable storage medium is provided, the computer-readable storage medium comprising a computer program or instructions, which, when executed on a map data processing device, enables the map data processing device to execute the method described in the first aspect and any one of the designs thereof.

[0051] In a fifth aspect, a computer program product is provided, comprising: a computer program or instructions, which, when executed on a computer, enables the computer to execute the method described in the first aspect and any one of the designs thereof.

[0052] In a sixth aspect, a chip system is provided, comprising at least one processor and at least one interface circuit, wherein the at least one interface circuit is used to perform transceiver functions and send instructions to at least one processor, and when the at least one processor executes the instructions, the at least one processor executes the method described in the first aspect and any one of the designs thereof.

[0053] In a seventh aspect, a communication system is provided, comprising a map data processing device and a map data acquisition device as described in the second aspect and any one of the designs thereof, wherein the map data acquisition device is used to collect crowdsourced data and send the crowdsourced data to the map data processing device.

[0054] It should be noted that the technical effects brought about by any design in the above-mentioned second to seventh aspects can refer to the technical effects brought about by the corresponding design in the first aspect, and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0055] Figure 1 A schematic diagram of a process of constructing a map using a centralized mapping mode provided in an embodiment of the present application;

[0056] Figure 2 A schematic diagram of the architecture of a communication system provided in an embodiment of the present application;

[0057] Figure 3 A schematic diagram of the structure of a communication device provided in an embodiment of the present application;

[0058] Figure 4 A flowchart of a map data processing method provided in an embodiment of the present application;

[0059] Figure 5 A schematic diagram of a process for preprocessing crowdsourcing provided in an embodiment of the present application;

[0060] Figure 6 A flowchart of another map data processing method provided in an embodiment of the present application;

[0061] Figure 7 A schematic diagram of a process for segmenting a crowd-source sub-map and a historical sub-map provided in an embodiment of the present application;

[0062] Figure 8 A schematic diagram of segmenting a crowd-source bun map provided in an embodiment of the present application;

[0063] Fig. 9 A schematic diagram of dividing a historical sub-map provided in an embodiment of the present application;

[0064] Fig.10 A schematic diagram of a process for determining a road network level corresponding to a crowdsourcing sub-map according to a road network level of a historical sub-map provided in an embodiment of the present application;

[0065] Fig.11 A schematic diagram of a historical sub-map for searching a crowdsourcing sub-map match provided in an embodiment of the present application;

[0066] Fig.12 A schematic diagram of a factor graph provided in an embodiment of the present application;

[0067] Fig.13 A schematic diagram of a process for quality inspection of optimized crowd-sourced data based on historical map data provided in an embodiment of the present application;

[0068] Fig.14 A schematic diagram of the structure of a map data processing device provided in an embodiment of the present application;

[0069] Fig.15 A schematic diagram of the structure of a chip system provided in an embodiment of the present application. DETAILED DESCRIPTION

[0070] In the description of this application, unless otherwise specified, " / " indicates that the objects associated with each other are in an "or" relationship, for example, A / B can represent A or B; "and / or" in this application is merely a description of the association relationship between associated objects, indicating that three relationships may exist, for example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone, where A and B can be singular or plural.

[0071] In the description of this application, unless otherwise specified, "plurality" means two or more than two. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single items or plural items. For example, at least one of a, b, or c can mean: a, b, c, a and b, a and c, b and c, a and b and c, where a, b, and c can be single or multiple.

[0072] In addition, in order to clearly describe the technical solutions of the embodiments of the present application, in the embodiments of the present application, words such as "first" and "second" are used to distinguish the same items or similar items with substantially the same functions and effects. Those skilled in the art can understand that words such as "first" and "second" do not limit the quantity and execution order, and words such as "first" and "second" do not necessarily limit the difference.

[0073] In the embodiments of the present application, words such as "exemplary" or "for example" are used to indicate examples, illustrations or descriptions. Any embodiment or design described as "exemplary" or "for example" in the embodiments of the present application should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of words such as "exemplary" or "for example" is intended to present related concepts in a concrete way for easy understanding.

[0074] The features, structures or characteristics in this application may be combined in one or more embodiments in any suitable manner. In various embodiments of this application, the size of the sequence number of each process does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0075] Some optional features in the embodiments of the present application may be implemented independently in some scenarios without relying on other features to solve corresponding technical problems and achieve corresponding effects. They may also be combined with other features in some scenarios as needed.

[0076] In this application, unless otherwise specified, the same or similar parts between the various embodiments can refer to each other. In each embodiment of this application, if there is no special description and logical conflict, the terms and / or descriptions between different embodiments are consistent and can be referenced to each other, and the technical features in different embodiments can be combined to form new embodiments according to their inherent logical relationships. The implementation methods of this application do not constitute a limitation on the scope of protection of this application.

[0077] In addition, the network architecture and business scenarios described in the embodiments of the present application are intended to more clearly illustrate the technical solutions of the embodiments of the present application, and do not constitute a limitation on the technical solutions provided in the embodiments of the present application. Ordinary technicians in this field can know that with the evolution of network architecture and the emergence of new business scenarios, the technical solutions provided in the embodiments of the present application are also applicable to similar technical problems.

[0078] To facilitate understanding, the technical terms and related concepts involved in the embodiments of the present application are first introduced below.

[0079] 1. Elevation

[0080] Elevation refers to the distance from a point along the plumb line to the absolute base surface, which is called absolute elevation, or elevation for short.

[0081] 2. Add bias

[0082] Biasing can refer to the operation of manually adding an offset to the measured point cloud or vector data.

[0083] 3. Simultaneous localization and mapping (SLAM)

[0084] SLAM can refer to positioning and map construction. Specifically, it can refer to the process in which a mapping data acquisition device (such as a crowdsourcing car) starts from an unknown location in an unknown environment, locates its own position and posture through the observed environmental information during the movement, and then incrementally constructs a map based on its own position.

[0085] 4. Rigid body transformation, non-rigid body transformation

[0086] Transformations that do not change the shape and size of an object can be called rigid transformations, such as translating or rotating an object, etc. Transformations that change the shape and / or size of an object can be called non-rigid transformations, such as stretching or compressing an object, etc.

[0087] 5. Regional Growth

[0088] Region growing can refer to the process of taking a sub-region as a seed point, starting from the seed point, and gradually merging other regions that meet the conditions with the seed point according to a pre-defined growth criterion to develop into a larger region.

[0089] 6. Point cloud registration

[0090] When a laser beam hits the surface of an object, the reflected laser will carry information such as direction and distance. If the laser beam is scanned along a certain trajectory, the reflected laser point information will be recorded while scanning. Since the scanning is extremely fine, a large number of laser points can be obtained, and each laser point contains three-dimensional coordinates, so a laser point cloud can be formed. These laser point clouds can be simply referred to as point clouds. In some embodiments, these point clouds can be obtained by laser scanning of a lidar. Of course, these point clouds can also be obtained by other types of sensors, and the present application is not limited to this.

[0091] Point cloud registration can refer to solving the problem of misalignment between different point clouds by solving the rotation and translation matrix (or rigid body transformation matrix, etc.) between different point clouds.

[0092] 7. Bird's Eye View (BEV) image

[0093] Bird's-eye view images refer to images of a certain area viewed from the air. In the fields of autonomous driving and robotics, data obtained by sensors (such as radars and cameras) can be converted into bird's-eye view images, which can simplify complex three-dimensional environments into two-dimensional images.

[0094] 8. Image Registration

[0095] Image registration can refer to solving the problem of misalignment between different images by solving the spatial transformation relationship between different images. In some implementations, image registration can be achieved by using a template matching method. Template matching can refer to the process of moving a template on the entire image, calculating the similarity between the template and the covered window on the image, and finally determining the best position of the template on the image. Of course, in other implementations, image registration can also be achieved by other algorithms.

[0096] At present, when constructing a map, a centralized mapping mode can be adopted. For example, Figure 1 A schematic diagram of the process of building a map using a centralized mapping model is shown. Figure 1 As shown, professional surveying and mapping vehicles can collect composition data. For example, the composition data may include but are not limited to point clouds and images (i.e., road data) that reflect road environment information, as well as data such as the global navigation satellite system (GNSS) and inertial navigation system (INS) that reflect the movement trajectory of the surveying and mapping vehicle. Subsequently, the professional surveying and mapping vehicle can upload the collected composition data to the cloud. The cloud undergoes a series of post-processing (such as but not limited to Figure 1 The map is constructed by performing various steps such as post-positioning solution, single-pass point cloud stitching, multi-pass fusion mapping, semantic feature extraction, and map vectorization as shown.

[0097] In actual situations, road conditions often change, such as road closures, traffic sign changes, etc. In order to ensure the accuracy of the map, it is necessary to update the map in a timely manner. However, considering the cost, quantity, and mapping costs of professional mapping vehicles, crowdsourcing mapping is usually used to update maps. However, the crowdsourcing mapping model also has great challenges. On the one hand, compared with the mapping-level sensors carried on professional mapping vehicles, crowdsourcing vehicles are usually equipped with automotive-grade or consumer-grade sensors with reduced accuracy, and the sensors have lower accuracy. On the other hand, due to the restrictions of mapping regulations, the data uploaded to the cloud by the crowdsourcing vehicle is not the original data collected, but the data pre-processed by the vehicle. Exemplarily, the preprocessing may include, but is not limited to, various processing such as de-elevation, biasing, semanticization, and vectorization.

[0098] Therefore, based on the above reasons, the data uploaded to the cloud by the crowdsourcing car may have various problems, such as but not limited to lack of elevation information, and one or more of global non-rigid body deviation and local rigid body deviation between the data and the historical map data stored in the cloud. When the data with various problems is used to update the historical map data stored in the cloud, the accuracy of the updated map may be low.

[0099] It is understood that in the embodiments of the present application, the global non-rigid deviation may refer to the deviation between the entire crowdsourced data and the historical map data. The local rigid deviation may refer to the deviation between the local data included in the crowdsourced data and the historical map data. It is understood that the crowdsourced data may refer to the composition data (or raw data) collected by the crowdsourced car, and the historical map data may refer to the data of the constructed map.

[0100] To solve the above problems, relevant solutions have been proposed. First, crowdsourcing vehicles are relied on to upload road network information (such as road identity) to the cloud, and road network level binding is implemented based on the road network information to solve the problem of lack of elevation information in crowdsourcing data. However, due to the low update frequency and poor preservation of road network information, there may be a risk of incorrect road network level binding, and the accuracy of the determined road network level is low.

[0101] Secondly, related solutions identify the geometric features of point clouds in crowdsourced data and combine them with the iterative closest point (ICP) method to perform point cloud registration to solve the problem of local rigid body deviation between crowdsourced data and historical map data. However, this solution is easily disturbed by outliers and dynamic points, resulting in low accuracy of point cloud registration results.

[0102] Then, in the related schemes, global optimization is performed only based on the results of point cloud registration to solve the global non-rigid deviation problem between crowdsourcing data and historical map data. This can easily cause the optimization process to fall into the wrong local optimal solution, resulting in low accuracy of the overall registration results between crowdsourcing data and historical map data.

[0103] Finally, in the relevant schemes, there are two ways to conduct map quality inspection. One is to rely on manual observation for quality inspection, which is costly and inefficient. The other is to conduct map quality inspection based on the difference of geometric features of point clouds in the map. This method is easily affected by outliers, dynamic mobile objects, and seasonally changing objects, and thus has a high probability of false detection.

[0104] Based on this, the embodiment of the present application provides a map data processing method, which does not rely on the road network information uploaded by the crowdsourcing vehicle. The road network level binding can be achieved through the crowdsourcing data itself, which can reduce the risk of road network level binding errors and improve the accuracy of the determined road network level. The semantic information of the point cloud in the crowdsourcing data is used for point cloud registration, which can avoid the interference of outliers and dynamic points and improve the robustness of point cloud registration. Global optimization is performed based on the results of point cloud registration and image registration. Point cloud registration belongs to the category of local registration, while image registration belongs to the category of global registration. In this way, the error of point cloud registration can be reduced, the probability of the optimization process falling into an erroneous local optimal solution can be reduced, and the accuracy of the overall registration results between crowdsourcing data and historical map data can be improved. The map quality inspection is performed by reprojecting based on the semantic information of the point cloud, which will not be affected by outliers, dynamic mobile objects, and seasonally changing objects, and the probability of false detection can be reduced. Compared with the manual map quality inspection method, it has low cost and high efficiency.

[0105] The technical solution provided by the embodiments of the present application can be applied to various scenarios where map construction is required, such as including but not limited to map construction in the field of intelligent driving, so as to use the constructed map for positioning, navigation, etc.; in the power industry, map construction is performed by using drones and the like to map power lines, so as to use the constructed map to inspect power lines; in the construction industry, map construction is performed by using robots and the like to map, so as to monitor the progress of construction; in the field of urban digital twins, outdoor road mapping is performed by using vehicles and the like, and indoor mapping is performed by using robots, handheld devices, etc., which are combined to realize the joint construction of indoor and outdoor maps; in forestry (such as forestry management, logging, etc.), forest mapping is performed by using drones, mechanical vehicles, etc. to map, so as to use the constructed map to realize tree quantity statistics, tree chest height diameter statistics, etc.; in the mining field, map construction is performed by mapping mines, so as to use the constructed map to calculate the volume of minerals, estimate the value, monitor the progress of mining, etc.; in the field of indoor navigation, such as mapping large shopping malls, so as to use the constructed map to navigate in the mall, etc.

[0106] For example, Figure 2 The schematic diagram of the architecture of a communication system to which a map data processing method provided in an embodiment of the present application is applied is shown. Figure 2 As shown, the communication system 200 includes a first device 201 and a second device 202 .

[0107] The first device 201 can collect the composition data and send the composition data to the second device 202 to construct the map. For example, the first device 201 can be a vehicle, an artificial intelligence (AI) device (such as but not limited to a robot, a mechanical dog, etc.), a handheld device (such as but not limited to a mobile phone, a tablet computer, a wearable device, etc.), a drone, or other devices. Figure 2 In the figure, the first device 201 is shown as a vehicle.

[0108] In some embodiments, the first device 201 is configured with a composition data acquisition device, such as various types of sensors including but not limited to lidar, cameras, positioning devices (such as but not limited to global positioning system (GPS), inertial measurement unit (IMU)), etc.), and the first device 201 can collect composition data through these sensors.

[0109] The second device 202 can receive the mapping data from the first device 201, and perform map construction and update based on the mapping data. The second device 202 can also provide map update and download services to the first device 201. Exemplarily, the second device 202 can include but is not limited to a server, a road side unit (RSU), etc. The server can be a single server, a server cluster consisting of multiple servers, or a cloud computing service center.

[0110] Optionally, the first device 201 and the second device 202 can communicate with each other via wired communication technology or wireless communication technology. Exemplarily, the wireless communication technology includes but is not limited to at least one of the following: fourth generation (4G) communication technology (e.g., long term evolution (LTE) technology), worldwide interoperability for microwave access (WiMAX) communication technology, fifth generation (5G) communication technology (e.g., new radio (NR) technology), and future mobile communication technology, such as sixth generation (6G) mobile communication technology.

[0111] Optionally, in the embodiment of the present application, the second device 202 may be implemented by different devices. For example, the second device 202 in the embodiment of the present application may be implemented by Figure 3 This is achieved by the communication equipment in. Figure 3 The hardware structure diagram of the communication device provided in the embodiment of the present application is shown in FIG. The communication device includes at least one processor 301 , a communication line 302 , a memory 303 and at least one communication interface 304 .

[0112] The processor 301 may be a general-purpose central processing unit (CPU), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits for controlling the execution of the program of the present application.

[0113] In some embodiments of the present application, the processor 301 may be used to construct and update maps based on crowdsourced data and historical map data.

[0114] The communication link 302 may include a pathway to transmit information between the above-mentioned components.

[0115] Communication interface 304 is used to communicate with other devices (such as but not limited to Figure 2 The first device 201, etc.) communicates.

[0116] The memory 303 may exist independently and be connected to the processor via the communication line 302. The memory 303 may also be integrated with the processor.

[0117] The memory 303 is used to store execution instructions for implementing the solution of the present application, and the execution is controlled by the processor 301. The processor 301 is used to execute the execution instructions stored in the memory 303, thereby implementing the method provided in the following embodiments of the present application.

[0118] In some embodiments of the present application, historical map data may be stored in the memory 303. Optionally, the memory 303 may also store updated map data obtained based on crowdsourcing data and historical map data.

[0119] For example, Figure 4 A flowchart of a map data processing method provided by an embodiment of the present application is shown. The method comprises the following steps:

[0120] S401: The first device obtains crowdsourcing data.

[0121] It can be understood that in the embodiment of the present application, crowdsourced data may refer to composition data collected by non-dedicated surveying and mapping equipment (such as but not limited to crowdsourced vehicles, etc.), and the crowdsourced data can be used for map construction, updating, etc.

[0122] Exemplarily, the crowdsourced data may include but is not limited to point clouds and images reflecting the road environment, navigation data reflecting the motion trajectory of the first device, etc. In some possible implementations, the first device may collect crowdsourced data by itself, such as collecting point clouds through lidar, collecting images through cameras, collecting navigation data through GNSS, IMU, etc. In other implementations, the first device may also directly obtain crowdsourced data from other devices (such as servers, roadside units, etc.).

[0123] In some embodiments, after the first device obtains the crowdsourced data, it may also preprocess the crowdsourced data before sending the crowdsourced data to the second device. Exemplarily, the preprocessing may include but is not limited to one or more of SLAM operations, semantic extraction operations, compliance processing operations, etc. Figure 5 , introduces SLAM operations, semantic extraction operations, compliance processing operations, etc.

[0124] like Figure 5 As shown, first, the first device can perform SLAM operations on one or more crowdsourced data such as the acquired point cloud, image, navigation data, etc., that is, input them into the SLAM module for fusion positioning and mapping, and output the constructed point cloud map through the SLAM module, as well as the motion trajectory of the first device contained in the point cloud map, etc. Optionally, the point cloud map output by the SALM module can be part or all of the area of ​​the historical map stored by the second device. It can be understood that the stored historical map can be a previously constructed map, and the crowdsourced data obtained by the first device can be used to update the historical map. Optionally, the historical map can be constructed using a centralized mapping mode or a crowdsourced mapping mode, and the embodiments of the present application do not impose specific restrictions on this.

[0125] Optionally, in order to achieve better positioning and mapping effects of the SLAM module, the first device may also obtain historical map data (or prior map data) from the second device or other devices, and input the historical map data and crowdsourcing data into the SLAM module together.

[0126] Then, the first device can perform semantic extraction operations on the point cloud map constructed by the SLAM module, and the images in the crowdsourcing data, and extract the semantics of the elements contained in the point cloud map constructed by the SLAM module. In some embodiments, the first device can extract the semantics of elements of preset semantic types, in other words, retain the data of elements of preset semantic types. Exemplarily, the preset semantic types may include but are not limited to one or more of road edges, lane lines, traffic, poles, signs, road surface markings, etc. Optionally, the first device can also convert the extracted elements expressed in other forms (such as point clouds, images, etc.) into vector form.

[0127] Next, the first device may perform compliance processing operations on the point cloud and / or vector data obtained through the semantic extraction operation. It is understood that the compliance processing operation may refer to the operation of converting a data form that does not comply with surveying and mapping regulations into a data form that complies with surveying and mapping regulations. Exemplarily, the compliance processing operation may include but is not limited to one or more operations of biasing, removing elevation, and compression. Finally, the first device may send the crowdsourced data after the compliance processing operation to the second device. Optionally, the crowdsourced data after the compliance processing may be represented in the form of a point cloud and / or vector.

[0128] It can be understood that the above embodiment takes the first device preprocessing the crowdsourced data as an example. In other implementations, other devices can also perform preprocessing operations on the crowdsourced data, that is, the crowdsourced data obtained by the first device can be preprocessed.

[0129] S402: The first device sends crowdsourced data to the second device. Correspondingly, the second device receives the crowdsourced data from the first device.

[0130] S403: The second device updates the historical map according to the crowdsourcing data.

[0131] For example, the second device may modify the historical map data according to the crowdsourced data to obtain an updated map. Optionally, the second device may also send the updated map to the first device. In some embodiments, step S403 may be specifically implemented as follows: Figure 6 One or more steps in the method shown. Figure 6 The method shown specifically comprises the following steps:

[0132] S601. The second device obtains at least one crowdsourcing sub-map and at least one historical sub-map.

[0133] Among them, at least one crowdsourced submap is obtained by segmenting crowdsourced data, and at least one historical submap is obtained by segmenting historical map data. It can be understood that in the embodiment of the present application, a submap can also be called a submap, and a submap can refer to several local small maps segmented from an overall map.

[0134] The following introduces the process of dividing crowdsourced data into crowdsourced sub-maps and dividing historical map data into historical sub-maps.

[0135] For example, Figure 7 A schematic diagram of a process of dividing a crowd-source sub-map and a historical sub-map provided by an embodiment of the present application is shown. Figure 7As shown, since crowdsourcing data may be represented in the form of point cloud and / or vector. Therefore, the second device may first determine in which form the crowdsourcing data is represented. When the crowdsourcing data is represented in point cloud form (i.e., scalar form), the second device may directly set an anchor point for the crowdsourcing data. When the crowdsourcing data is represented in vector form, the second device may first convert the vector form into a scalar form, for example, by converting the crowdsourcing data in vector form into a scalar form by upsampling, etc., and then setting an anchor point for the crowdsourcing data. The crowdsourcing sub-map is then divided based on the set anchor point.

[0136] In some implementations, an anchor point may be set for crowdsourced data according to the motion trajectory of the first device. Figure 8 As shown, taking the movement trajectory of the first device contained in the crowdsourcing data as trajectory 1 as an example, setting anchor points can refer to marking several anchor points on trajectory 1 according to the preset anchor point distance, such as anchor point A, anchor point B, anchor point C, and so on. Then, with each anchor point as the center, several areas are divided in the crowdsourcing data using a preset size, such as area 1, area 2, area 3, and so on. Each divided area is a crowdsourcing sub-map. It can be understood that the embodiment of the present application does not limit the shape and size of each area.

[0137] In some scenarios, since the crowdsourcing data may contain multiple movement trajectories of the first device, when the crowdsourcing sub-maps are divided based on different movement trajectories, the crowdsourcing sub-maps obtained by division may be repeated. For example, when the overlapping area of ​​two crowdsourcing sub-maps obtained by division based on different movement trajectories is greater than a preset threshold, the two crowdsourcing sub-maps are considered to be repeated. Therefore, in some embodiments, the crowdsourcing sub-maps obtained by division can also be deduplicated, for example, by identifying grids with repeated trajectories by constructing a grid map or by other methods to perform deduplication, so as to remove redundant historical sub-maps.

[0138] Similarly, if Figure 7 As shown, the historical map data may also be divided into one or more historical sub-maps by setting anchor points. In some embodiments, since the historical map data is generally stored in vector form, the vector-based historical map data may be converted into a scalar form by upsampling or other methods. In some scenarios, since the historical map data does not contain the motion trajectory of the first device, as a specific implementation, an anchor point may be set based on the center line of the road. Fig. 9As shown, taking the road 900 with three lanes (such as lane 1, lane 2, and lane 3) in the historical map data as an example, the centerline of the road 900 is centerline 1, and the second device can mark a number of anchor points on centerline 1 according to the preset anchor point distance, such as anchor point D, anchor point E, anchor point F, and so on. The subsequent process is the same as the process of dividing the crowdsourcing sub-map, such as Fig. 9 Region 4, region 5, and region 6 shown are different historical sub-maps obtained by division.

[0139] Similarly, deduplication operation may also be performed on the historical sub-maps obtained by division, which will not be described in detail in the embodiment of the present application.

[0140] Optionally, for different sub-maps (such as crowdsourcing sub-maps, historical sub-maps, etc.), there may be overlapping areas or there may not be overlapping areas, and this embodiment of the application does not limit this. This embodiment of the application takes the existence of overlapping areas as an example, for example: Figure 8 There is an overlapping area between area 1 and area 2, and there is an overlapping area between area 2 and area 3. Fig. 9 As shown, there is an overlapping area between area 4 and area 5, there is an overlapping area between area 5 and area 6, and so on.

[0141] In some embodiments, before the second device divides the historical map data into a plurality of historical sub-maps, Figure 7 As shown, the second device may also first perform an area search in the stored historical map data according to the crowdsourced data, determine the historical map data corresponding to the crowdsourced data, and only divide the historical map data into historical sub-maps, which can reduce the power consumption of the second device. Exemplarily, the second device may determine the historical map data corresponding to the crowdsourced data according to the location information of the crowdsourced data, such as the location information (such as coordinates) of the divided crowdsourced sub-maps. For example: the historical map data corresponding to the crowdsourced data may be the historical map data in the same location range as the crowdsourced data, and the area corresponding to the historical map data may be the same size as or different from the area corresponding to the crowdsourced data.

[0142] S602: The second device determines a road network level corresponding to at least one crowdsourced sub-map according to at least one historical sub-map.

[0143] In some embodiments, the second device can determine the road network level of at least one crowdsourced sub-map based on the road network level of at least one historical sub-map. For example, the road network level corresponding to the historical sub-map can be used as the road network level corresponding to the crowdsourced sub-map. In an embodiment of the present application, the road network level can be used to characterize the relative heights between different sub-maps. In this way, automatic binding of the road network level of crowdsourced sub-maps (such as elevated areas included) can be achieved, without the need for the vehicle end to upload road network information with low update frequency and poor freshness (such as road identity) to perform road network level binding, which can improve the accuracy of road network level binding.

[0144] Combine the following Fig.10 , the process of determining the road network level corresponding to the crowdsourcing sub-map based on the road network level of the historical sub-map is specifically introduced.

[0145] like Fig.10 As shown, the second device can perform loop detection on the historical sub-map and the crowd-sourced sub-map. In the loop detection, for each crowd-sourced sub-map, a historical sub-map that has a matching relationship with the crowd-sourced sub-map can be searched. As a specific implementation, if the center point (i.e., anchor point) of a historical sub-map is within the search range of a crowd-sourced sub-map, it means that the historical sub-map has a matching relationship with the crowd-sourced sub-map. Otherwise, it means that the historical sub-map has no matching relationship with the crowd-sourced sub-map. Fig.11 As shown, based on the public Figure 1 For example, the search range is range 1, the historical sub-region Figure 1 The center point 1. Historical sub-site Figure 2 The center point 2 is located in the crowd Figure 1 Therefore, the historical sub-area Figure 1 、Historical sub-site Figure 2 All with the buns Figure 1 There is a matching relationship. Figure 3 The center point 3 is not located in the crowd Figure 1 If the search scope is within the historical sub-region Figure 3 With all the buns Figure 1 There is no match.

[0146] Optionally, for each crowdsourced submap obtained by segmenting the crowdsourced data, there may be one or more historical submaps that have a matching relationship with it. In this way, the second device can obtain a matching pair set through loop detection, and the matching pair set includes multiple matching pairs between the crowdsourced submap and the historical submap, such as matching pair 1, matching pair 2, etc. Matching pair 1 can be a crowdsourced submap. Figure 1 and the historical sub-maps that have matching relationships (such as historical sub-maps Figure 1 、Historical sub-site Figure 2), the matching pair 2 can be a crowdfunding site Figure 2 and the historical sub-maps that have matching relationships (such as historical sub-maps Figure 3 ), and so on.

[0147] Next, the second device may determine the road network level corresponding to the historical sub-map, and determine the road network level corresponding to the crowdsourcing sub-map that has a matching relationship with the historical sub-map according to the road network level corresponding to the historical sub-map.

[0148] As a specific implementation, Fig.10 As shown, the second device can determine the road network level corresponding to the historical sub-map by clustering the road center lines. Specifically, the second device clusters the road center lines contained in the historical sub-map to determine the number of roads contained in the historical sub-map, and then combines the level corresponding to each road to determine the road network level corresponding to the historical sub-map. Optionally, in a specific implementation, the second device can only determine the road network level corresponding to the historical sub-map contained in the matching pair, that is, the historical sub-map that has a matching relationship with the crowdsourcing sub-map, so as to save power consumption of the second device.

[0149] Further, in combination with the matching pair set described above, after determining the road network level corresponding to the historical sub-map, the second device can obtain two types of sets, namely, a single-level matching pair set and a multi-level matching pair set. Among them, the single-level matching pair set includes matching pairs between the crowdsourced sub-map and the corresponding historical sub-map with a single-level road network level. The multi-level matching pair set includes matching pairs between the crowdsourced sub-map and the corresponding historical sub-map with a multi-level road network level.

[0150] Optionally, in a single-level matching pair set, a crowdsourced sub-map can also have a matching relationship with multiple different historical sub-maps, for example: Figure 1 、Historical sub-site Figure 2 All with the buns Figure 1 There is a matching relationship, historical sub-site Figure 1 and historical sub-site Figure 2 The corresponding road network levels are all single-layer. Similarly, in a multi-level matching pair set, a crowdsourcing sub-map can also have a matching relationship with multiple different historical sub-maps, such as: Figure 4 、Historical sub-site Figure 5 All with the buns Figure 2 There is a matching relationship, historical sub-site Figure 4 and historical sub-site Figure 5The corresponding road network levels are all multi-layer. Optionally, the single-layer matching pair set and the multi-layer matching pair set can also contain the same crowdsourced sub-map, that is, a crowdsourced sub-map may have a matching relationship with a historical sub-map with a single-layer road network level, or may have a matching relationship with a historical sub-map with a multi-layer road network level.

[0151] Furthermore, in some embodiments, Fig.10 As shown, the road network level of the crowdsourced submap can be determined by region growing. As a specific implementation, the second device can obtain a crowdsourced submap (which can be called a second submap) included in the single-level matching pair set, and determine the road network level corresponding to the crowdsourced submap according to the road network level corresponding to the historical submap (which can be called the first submap) corresponding to the crowdsourced submap in the single-level matching pair set. Optionally, the number of the second submap can be one or more.

[0152] Optionally, there may be multiple historical submaps corresponding to the crowdsourced submap in the single-level matching pair set, and the second device may determine the road network level corresponding to the crowdsourced submap based on the road network level corresponding to any one of them. In this embodiment, as a specific implementation, the historical submap and the matching crowdsourced submap may be point cloud aligned based on the point cloud alignment result between the historical submap and the matching crowdsourced submap, and the network level corresponding to the crowdsourced submap may be determined based on the road network level corresponding to the historical submap. Specifically, the second device may determine through point cloud alignment which road in the historical submap corresponds to the motion trajectory of the first device contained in the crowdsourced submap, that is, which road in the historical submap the motion trajectory of the first device belongs to, and use the level corresponding to the road as the road network level corresponding to the crowdsourced submap.

[0153] Then, the crowdsourced sub-map with the determined road network level is used as a seed point to determine the road network level of other crowdsourced sub-maps through region growing. Optionally, the second device can perform region growing according to the road topology, or can use other rules to perform region growing. Figure 8 In the example shown, the crowdsourced sub-map for determining the road network level is region 2. Region 2 is used as a seed point, and regional growth is performed on both sides according to the road topology, thereby determining the road network levels corresponding to other crowdsourced sub-maps such as region 1 and region 3.

[0154] Based on this solution, the road network level corresponding to a crowdsourced submap is determined based on a historical submap with a single-layer road network level, and then the crowdsourced submap is used as a seed point to determine the road network level corresponding to some or all submaps in at least one crowdsourced submap through regional growth. There is no need to determine the road network level corresponding to the crowdsourced submaps with matching relationships based on the road network level corresponding to each historical submap, which can reduce the complexity of the algorithm, reduce the time spent in determining the road network level corresponding to the crowdsourced submap, and reduce the power consumption of the device.

[0155] Optionally, the crowdsourced submaps that determine the road network level by region growing may be included in a single-level matching pair set or in a multi-level matching pair set. That is to say, using the crowdsourced submaps in the single-level matching pair set as seed points, region growing can be used to determine not only the road network level corresponding to other crowdsourced submaps in the single-level matching pair, but also the road network level corresponding to some or all of the crowdsourced submaps in the multi-level matching pair set. That is, it is possible to determine the road network level corresponding to all the crowdsourced submaps obtained by dividing the crowdsourced data by region growing, and it is also possible to determine the road network level corresponding to some or all of the crowdsourced submaps obtained by dividing the crowdsourced data.

[0156] In some specific scenarios, when the crowdsourced submaps in the single-level matching pair set are used as seed points, the road network levels corresponding to all the crowdsourced submaps in the single-level matching pair set and some of the crowdsourced submaps in the multi-level matching pair set may only be determined by region growing. Therefore, after the second device determines the road network levels corresponding to several crowdsourced submaps by region growing, the second device can also determine whether there are crowdsourced submaps whose road network levels have not been determined. If so, that is, there are crowdsourced submaps whose road network levels have not been determined by region growing in the multi-level matching pair set, the second device can also determine the road network levels corresponding to these crowdsourced submaps by level registration. If not, that is, there are no crowdsourced submaps whose road network levels have not been determined by region growing, then the process ends, that is, the second device can no longer determine the road network levels corresponding to the crowdsourced submaps by level registration. In this way, the road network levels of all crowdsourced submaps obtained by dividing the crowdsourced data can be determined, and then the complete road network level information corresponding to the crowdsourced data can be obtained.

[0157] As a specific implementation, for a crowdsourced submap (which may be referred to as the fourth submap) whose road network level is not determined by region growing in a multi-level matching pair set, the road network level corresponding to the crowdsourced submap may be determined based on the road network level corresponding to the historical submap (which may be referred to as the third submap) that matches the crowdsourced submap in the multi-level matching pair set. Specifically, in this implementation, the second device may determine the road network level corresponding to the crowdsourced submap based on the point cloud registration results of each road network layer in the crowdsourced submap and the matched historical submap, and the level corresponding to each road network layer in the historical submap. For example, the second device may perform point cloud registration on the crowdsourced submap and each road network layer contained in the historical submap, and then, based on the point cloud registration results, use the level corresponding to the road network layer with the highest similarity in the historical submap as the level corresponding to the crowdsourced submap. In this way, selecting the level corresponding to the road network layer with the highest similarity as the level corresponding to the crowdsourced submap may make the level of the obtained crowdsourced submap more accurate.

[0158] It can be understood that the above embodiment is based on the example of first executing region growing, and after the execution is completed, determining that there are still crowdsourced sub-maps whose road network levels have not been determined, and then executing hierarchical registration attempts. Compared with the region growing method, the hierarchical registration attempt is more complex and time-consuming to calculate. Therefore, the region growing method is preferably used to determine the road network level corresponding to the crowdsourced sub-map, which can improve the efficiency of determining the road network level corresponding to the crowdsourced sub-map, that is, improve the efficiency of road network level binding and reduce the power consumption overhead of the device.

[0159] Of course, in other embodiments, these two processes may be performed in parallel, or only the level registration attempt may be used to determine the road network levels corresponding to all crowdsourced sub-maps.

[0160] S603: The second device determines a position and posture of at least one crowdsourcing sub-map according to at least one historical sub-map.

[0161] Among them, at least one crowdsourcing sub-map after determining the road network level and posture can be used to optimize the crowdsourcing data.

[0162] Optionally, step S602 and step S603 may be performed sequentially or in parallel. In the case of sequential execution, the embodiment of the present application does not limit the sequence.

[0163] In some embodiments, the second device may obtain a parameter. The parameter includes at least one of a first registration result and a second registration result. The first registration result is a point cloud registration result between the fifth submap and the sixth submap. The fifth submap is a historical submap that has a matching relationship with the sixth submap. The sixth submap may be any one of at least one crowdsourced submap obtained by segmenting crowdsourced data. The second registration result is an image registration result between a first bird's-eye view image and a second bird's-eye view image. The first bird's-eye view image is obtained based on the fifth submap, and the second bird's-eye view image is obtained based on the sixth submap. The second device may determine the position and posture of the sixth submap based on the parameter. Optionally, the fifth submap may be the first submap described above, or the third submap. The sixth submap may be the second submap, or the fourth submap. In this way, the position and posture of the crowdsourced sub-map can be determined based on the point cloud registration results obtained by performing point cloud registration on the crowdsourced sub-map and the historical sub-map with a matching relationship, as well as the image registration results obtained by performing image registration on the bird's-eye view image converted from the crowdsourced sub-map and the bird's-eye view image converted from the historical sub-map. This can achieve registration between crowdsourced data and cloud-based historical map data, and solve the problem of local rigid body deviation between crowdsourced data and cloud-based historical map data.

[0164] The matching relationship described in this embodiment may be the matching relationship obtained in the loop detection described above. Optionally, the number of sixth submaps that have a matching relationship with the fifth submap may be one or more. In this embodiment of the application, different historical submaps that have a matching relationship with the fifth submap are referred to as sixth submaps. Correspondingly, the number of first registration results and / or the number of second registration results included in the parameters obtained by the second device may also be one or more.

[0165] It can be understood that, for each crowdsourcing sub-map obtained by segmenting the crowdsourcing data, the solution described in this embodiment can be used to determine the position and posture.

[0166] In a possible implementation, the first registration result includes a first relative pose between the fifth submap and the sixth submap, and the second registration result includes a second relative pose between the fifth submap and the sixth submap. The second device can adjust the initial pose of the sixth submap according to the first relative pose and the second relative pose to obtain the pose of the sixth submap. In this way, the point cloud registration result includes the first relative pose between the crowdsourcing submap and the historical submap, and the image registration result includes the second relative pose between the crowdsourcing submap and the historical submap. The initial pose of the crowdsourcing submap is adjusted based on the first relative pose and the second relative pose to obtain the pose of the crowdsourcing submap. Point cloud registration belongs to local registration, while image registration belongs to global registration. In this way, the pose obtained by image registration can optimize the pose obtained by image registration, reduce the error of point cloud registration, improve the accuracy of crowdsourcing data registration, and better solve the problem of local rigid body deviation.

[0167] In this embodiment, as a possible implementation, the second device can use the first bird's-eye view image as a template, match the second bird's-eye view image with the template, and determine the second relative posture.

[0168] As a possible implementation, the second device may divide the point cloud contained in the fifth submap into at least one element of points, lines, and surfaces according to the semantic information of the point cloud contained in the fifth submap. According to the semantic information of the point cloud contained in the sixth submap, the point cloud contained in the sixth submap is divided into at least one element of points, lines, and surfaces. The residual between the elements with the same semantic information in the fifth submap and the sixth submap is determined. The first relative pose is determined according to the residual. Optionally, the semantic information of the point cloud contained in the sixth point cloud may be obtained by the first device performing a semantic extraction operation, or may be obtained by the second device performing point cloud semantic segmentation, or geometric feature extraction, etc. And / or, the semantic information of the point cloud contained in the fifth submap may also be obtained by the second device performing point cloud semantic segmentation, or geometric feature extraction, etc. In this way, the result of point cloud registration is determined based on the semantic information of the historical submap and the crowdsourcing submap, which can avoid the interference of outliers and dynamic points and improve the robustness of point cloud registration.

[0169] As a specific implementation, the elements with the same semantic information are points, and the residual between the points can be determined based on the distance between the points. For example, for the elements of semantic information such as lane dashed lines and road surface representations contained in the point cloud, the distance between the points can be calculated to perform point cloud registration. Exemplarily, the residual formula between the points can be expressed as Formula 1.

[0170] ∈ point =||p1-q||2 Formula 1

[0171] In formula 1, p1 and q represent a pair of matching points in two point clouds, ∈ point is the distance between two points.

[0172] Optionally, the parameters included in the formulas involved in the embodiments of the present application may have errors. Therefore, the errors can be eliminated by adding constants in the formulas involved in the embodiments of the present application. However, the embodiments of the present application do not limit the number and position of the added constants. For example: Formula 1 can also be expressed in the form of Formula 1.1.

[0173] ∈ point =||p1-q||2+l Formula 1.1

[0174] In formula 1.1, l is the added constant. For the introduction of other parameters in formula 1, please refer to the introduction of corresponding parameters in formula 1.

[0175] As a specific implementation, the elements of the same semantic information are lines, and the residual between the lines is determined according to the distance from a point on one line to another line. For example, for the elements of the semantic information of the lane solid line, curb, lamp post, and stop line light contained in the point cloud, the distance between the point and the line can be calculated to perform point cloud registration. Exemplarily, the residual formula between lines can be expressed as a combination of Formula 2, Formula 3, and Formula 4.

[0176] o1,v0=SVD(Q1) Formula 2

[0177] f=o1+(p2-o1) T ·v0·v0 Formula 3

[0178] ∈ line =||p2-f||2 Formula 4

[0179] Among them, in formula 2, formula 3, and formula 4, Q1 is a set of linear feature points, and SVD(Q1) means performing singular value decomposition (SVD) on the Q1 point set to obtain the centroid o1 of the Q1 point set and the eigenvector v0 corresponding to the maximum eigenvalue obtained by the singular value decomposition. The geometric meaning represents the direction vector obtained after fitting the Q1 point set into a line segment. The line segment fitted by the Q1 point set and a point p2 outside the line segment, draw a perpendicular line from point p2 to this line segment, and obtain the foot of the perpendicular f according to formula 3. Further, the distance between the point and the line ∈ line It can be determined according to formula 4.

[0180] As another specific implementation, if the elements with the same semantic information are faces, the residual between faces is determined based on the distance from a point on one face to another face. For example, for elements with semantic information such as the ground, traffic signs, and walls contained in the point cloud, the distance between the point and the face can be calculated to perform point cloud registration. Exemplarily, the residual formula between faces can be expressed as a combination of Formula 5 and Formula 6.

[0181] o2,v2=SVD(Q2) Formula 5

[0182] ∈ plane =||(p3-o2) T ·v2||2 Formula 6

[0183] Among them, in formula 5 and formula 6, Q2 is a set of plane feature points, SVD(Q2) means to perform singular value decomposition on the Q2 point set, and obtain the centroid o2 of the Q2 point set and the minimum eigenvector v2 obtained by singular value decomposition. The geometric meaning represents the normal vector obtained after fitting the Q2 point set into a plane. The plane fitted by the Q2 point set and a point p3 outside the plane, then the distance from the point to the plane ∈ plane It can be determined according to Formula 6.

[0184] In combination with the above specific implementation, in some embodiments, the first relative posture can be obtained according to the rigid body transformation matrix between the point cloud in the fifth submap and the point cloud in the sixth submap, and the rigid body transformation matrix can satisfy Formula 7.

[0185]

[0186] In Formula 7, A is the rigid body transformation matrix, (x, y, z, φ, ω, κ) represents the six-degree-of-freedom pose independent variable, (x, y, z) is the translation on the three coordinate axes (such as x-axis, y-axis, and z-axis) in the spatial rectangular coordinate system, (φ, ω, κ) is the Euler angle of rotation around the three coordinate axes in the spatial rectangular coordinate system, and w i 、w j 、w k are the weights corresponding to the residuals between points, lines, and surfaces mentioned above, respectively, i , j , k They are the loss functions corresponding to the residuals between points, between lines, and between surfaces mentioned above, respectively. The loss functions can be used to reduce the influence of outliers. L is the number of points when calculating the residuals between points; M is the number of points sampled on one of the lines when calculating the residuals between lines; N is the number of points sampled on one of the surfaces when calculating the residuals between surfaces. For the introduction of other parameters in Formula 7, please refer to the relevant introduction of the corresponding parameters in Formulas 1 to 6.

[0187] Based on the above scheme, different residual formulas are designed for elements with different semantic information, and the result of point cloud registration is determined based on the combination of different residual formulas. Converting the fusion mapping problem into a least squares problem can not only solve the problem of local rigid body deviation, but also avoid the interference of outliers, dynamic points, etc., and improve the robustness of point cloud registration.

[0188] In some embodiments, the above parameters may also include one or more of prior information and mileage information. Among them, the prior information includes the initial pose of the sixth submap. The mileage information includes the relative pose between the sixth submap and the seventh submap, and the seventh submap is a crowdsourced submap adjacent to the sixth submap. Optionally, the prior information and the mileage information may be determined based on the crowdsourced data obtained by the first device. In some scenarios, due to various reasons such as poor sensor signals, the crowdsourced data obtained by the first device may be inaccurate. Therefore, the prior information may also include the confidence of the initial pose of the sixth submap, and / or the mileage information may also include the confidence of the relative pose between the sixth submap and the seventh submap. In this way, the parameters also include one or more of the initial pose of the crowdsourced submap and the relative pose between adjacent crowdsourced submaps. When the pose of the crowdsourced submap is optimized based on the parameters, a more accurate pose of the crowdsourced submap can be obtained, global optimization can be achieved, and the global non-rigid body deviation between crowdsourced data and historical map data can be better resolved.

[0189] As a possible implementation, the second device can use an adjustment model, input the parameters into the adjustment model, and output the pose of the sixth sub-map through the adjustment model. Of course, the second device can also output the parameters corresponding to each crowdsourced sub-map into the adjustment model, and output the pose corresponding to each crowdsourced sub-map through the adjustment model. This operation can also be called a global optimization process, which can solve the global non-rigid deviation between crowdsourced data and historical map data.

[0190] In a specific implementation, the adjustment model can be implemented in the form of a factor graph. For example, Fig.12 An example of a factor graph provided by an embodiment of the present application is shown. Fig.12 As shown, the factor graph contains two types of vertices and four types of edges.

[0191] The two types of vertices are crowdsourced submap vertices and historical submap vertices. Optionally, in the scenario where crowdsourced data is used to update the historical map, the historical submap nodes are set as fixed nodes, that is, the position of the historical submap is fixed, and the crowdsourced submap nodes are set as floating nodes, that is, the nodes of the crowdsourced submap can adjust their positions during iterative optimization. In the scenario of multi-trip fusion mapping, for example: the vehicle collects multiple trips of road data, and then uploads each trip of road data to the cloud separately, and the cloud builds a map based on these multiple trips of road data, both types of nodes in the factor graph can be set as floating nodes.

[0192] The four types of edges include prior edges, odometry edges, point cloud registration edges, and image registration edges.

[0193] The prior edge records the prior information corresponding to the crowdsourcing submap connected to the prior edge. For the introduction of the prior information corresponding to each crowdsourcing submap, please refer to the relevant introduction of the prior information corresponding to the sixth submap mentioned above. The prior edge can be used to suppress the connected vertices from moving away from the posture recorded in the prior information.

[0194] The odometer edge records the mileage information corresponding to the crowdsourced sub-map to which the odometer edge is connected. For the introduction of the mileage information corresponding to each crowdsourced sub-map, please refer to the relevant introduction of the mileage information corresponding to the sixth sub-map described above. The odometer edge can be used to ensure the smoothness of the motion trajectory of the first device and prevent the motion trajectory from being broken or misaligned during the process of optimizing the position of the crowdsourced sub-map.

[0195] The point cloud registration edge records the point cloud registration results corresponding to the vertices connected by the point cloud registration edge, such as Figure X 1 and the historical sub-map Z1, the point cloud registration result between the crowdsourcing sub-map Figure X Point cloud registration result between 2 and historical sub-map Z1, Figure X 2 and the historical submap Z2, etc. For a detailed introduction to the point cloud registration result, please refer to the above. Optionally, the confidence of the point cloud registration result can also be recorded in the point cloud registration edge. The point cloud registration edge can be used to pull the connected crowdsourcing submap vertex toward the historical submap vertex connected by the point cloud registration edge, and the force is counteracted by the force generated by the prior edge of the connected crowdsourcing submap.

[0196] The image registration edge records the image registration results corresponding to the vertices connected by the image registration edge, such as Figure X 1 and the image registration results between the historical sub-map Z1, the crowd-sourced sub-map Figure X Image registration results between 2 and historical sub-map Z1, crowd-source sub-map Figure X2 and the image registration results between the historical sub-map Z2. Figure X The image registration result between 1 and the historical submap Z1 can be referred to as Figure X 1 is converted into a bird's-eye view image, and the image registration result between the two after the historical submap Z1 is converted into a bird's-eye view image, and so on. For a detailed introduction to the image registration results, please refer to the above. Optionally, the confidence of the image registration result can also be recorded in the image configuration edge. The image registration edge can be used to pull the connected crowdsourcing submap vertex toward the historical submap vertex connected by the image registration edge, and this force counteracts the force generated by the prior edge of the connected crowdsourcing submap.

[0197] It can be understood that, combined with the loop detection link described above, since a crowdsourcing submap may have a matching relationship with one or more historical submaps, a crowdsourcing submap vertex can be connected to one or more historical submap vertices, for example: Figure X 2 is connected with historical submap Z1, historical submap Z2, etc.

[0198] Based on the factor graph mentioned above, image registration belongs to the global registration method. The factor graph also contains image registration edges, that is, the results of image registration, which can not only solve all non-rigid deviation problems, but also reduce the error of point cloud registration, assist in jumping out of erroneous local optimal solutions in the global optimization process, and improve the optimization robustness.

[0199] In some scenarios, during the map construction process, due to various reasons such as changes in road conditions and registration errors, the optimized crowdsourced data may be inconsistent with the historical map data, for example, there may be various problems such as missing lane lines, misaligned lane lines, and scale errors. Therefore, in some embodiments, Figure 6 The illustrated method may include step S604.

[0200] S604: The second device performs quality inspection on the optimized crowdsourcing data based on the historical map data.

[0201] In some embodiments, the second device may input the historical map data and the optimized crowd-sourced data into a quality inspection model, and output areas in the crowd-sourced data where problems exist (eg, low consistency) through the quality inspection model.

[0202] As a specific implementation, Fig.13 FIG. 1 is a flow chart of a method for performing quality inspection on optimized crowd-sourced data based on historical map data provided by an embodiment of the present application. Fig.13 As shown, the method comprises the following steps:

[0203] S1301. The second device obtains historical map data and optimized crowdsourcing data.

[0204] Optionally, the second device may adjust the position and posture of the original crowdsourcing data according to the output of the adjustment model to obtain optimized crowdsourcing data.

[0205] S1302. The second device projects the point cloud in the optimized crowdsourcing data onto the target point cloud according to the semantic information of the point cloud in the optimized crowdsourcing data and the semantic information of the target point cloud in the historical map data.

[0206] The target point cloud matches the point cloud in the optimized crowdsourced data. Optionally, the second device may project the point cloud onto the point cloud with the same semantic information contained in the target point cloud according to the semantic information of the point cloud in the optimized crowdsourced data.

[0207] As a possible implementation, the second device can construct the center point of the historical sub-map obtained by dividing the historical map data into a K-dimensional tree (Kd-tree), search within the neighborhood of the optimized crowdsourcing data based on the Kd-tree, and take the point cloud in the historical map data located in the neighborhood of the optimized crowdsourcing data as the target point cloud.

[0208] Optionally, in this embodiment, the semantic information of the point cloud in the crowdsourced data and the semantic information of the target point cloud in the historical map data may be the semantic information obtained when performing point cloud registration, or may be the semantic information obtained by re-semantic segmentation, etc., and the embodiment of the present application does not limit this. After determining the semantic information of the point cloud in the optimized crowdsourced data, the second device may also perform one or more operations such as spatial downsampling and statistical outlier removal (SOR) filtering before projection to improve the quality of the point cloud in the crowdsourced data being projected. Similarly, after determining the semantic information of the target point cloud, the second device may also perform a spatial downsampling operation before projection, but since the quality of the historical map data is better, the SOR filtering operation may not be performed.

[0209] S1303: The second device outputs a quality inspection label according to the projection result and preset inspection rules.

[0210] The quality inspection tag may include the index of the crowdsourcing sub-map that meets the preset inspection rules, and one or more of the preset inspection rules corresponding to each sub-map (i.e., the preset inspection rules satisfied by each crowdsourcing sub-map). For example, the index of the crowdsourcing sub-map may include but is not limited to the location information of the crowdsourcing sub-map, etc. It can be understood that in this embodiment, the area with problems in the crowdsourcing data output by the quality inspection model is taken as an example in the form of a crowdsourcing sub-map. In other embodiments, the area may also be divided into larger or smaller areas than the crowdsourcing sub-map.

[0211] Exemplarily, the preset inspection rules may include one or more of the following: the average residual exceeds a threshold, the large residual area exceeds a threshold, and the unrelated point area exceeds a threshold. Among them, the average residual exceeds the threshold may refer to: for an optimized crowdsourcing sub-map, the average residual corresponding to the optimized crowdsourcing sub-map is determined to be greater than or equal to a first threshold according to the projection result. Optionally, the average residual corresponding to the optimized crowdsourcing sub-map may refer to the weighted average of the residual items corresponding to each point cloud contained in the optimized crowdsourcing sub-map. In an embodiment of the present application, the residual corresponding to the point cloud contained in the optimized crowdsourcing sub-map may refer to the residual between the point cloud and the corresponding point cloud in the target point cloud (such as a point cloud with the same semantic information). It can be understood that in an embodiment of the present application, the optimized crowdsourcing sub-map can be obtained by segmenting the optimized crowdsourcing data. Optionally, the optimized crowdsourcing sub-map may be the crowdsourcing sub-map after the optimized posture described above.

[0212] It can be understood that the various thresholds described in the embodiments of the present application can be set by developers according to actual needs, and the embodiments of the present application do not limit this.

[0213] The large residual area exceeding the threshold may mean that, for an optimized crowdsourcing sub-map, the area of ​​the point cloud with a large residual in the optimized crowdsourcing sub-map is determined to be greater than or equal to a second threshold according to the projection result, wherein the large residual means that the residual corresponding to the point cloud contained in the optimized crowdsourcing sub-map is greater than or equal to the first threshold.

[0214] The area of ​​unrelated points exceeding the threshold may mean that: the sum of the first area and the second area is greater than or equal to the third threshold, the first area is the sum of the areas of the first type of point cloud contained in an optimized crowdsourcing sub-map, and there is no historical point cloud in the neighborhood of the first type of point cloud, that is, there is no associated historical point cloud, and the historical point cloud is a point cloud in the historical sub-map that has a matching relationship with the optimized crowdsourcing sub-map.

[0215] For example, for each local area included in an optimized crowdsourcing sub-map, it is determined whether the above historical point cloud exists in the neighborhood of the local area. If not, it means that the point cloud of the local area is a first-type point cloud. If it exists, it means that the point cloud of the local area is not a first-type point cloud.

[0216] The second area is the sum of the areas of the second type of point clouds contained in the historical sub-maps that have a matching relationship with the optimized crowdsourcing sub-map. There is no point cloud in the optimized crowdsourcing sub-map in the neighborhood of the second type of point cloud, that is, there is no associated point cloud in the optimized crowdsourcing sub-map.

[0217] For example, for each local area included in the historical sub-map, it is determined whether there is a point cloud in the optimized crowdsourcing sub-map in the neighborhood of the local area. If not, it means that the point cloud of the local area is the second type of point cloud. If so, it means that the point cloud of the local area is not the second type of point cloud.

[0218] Optionally, the historical submap that matches the optimized crowdsourced submap may include one or more historical submaps. When there are multiple historical submaps, the second area may be the sum of the areas of the second type of point clouds contained in the multiple historical submaps. Optionally, the historical submap that matches an optimized crowdsourced submap may be determined based on a factor graph such as that described above. The historical submap that matches an optimized crowdsourced submap may be a historical submap connected to the crowdsourced submap that the optimized crowdsourced submap corresponds to before optimization. For example, the crowdsourced submap that corresponds to the optimized crowdsourced submap before optimization is Fig.12 Taking the crowdsourcing submap X2 shown in the figure as an example, the historical submap that matches the optimized crowdsourcing submap is Fig.12 The historical submap Z1, historical submap Z2, etc. are shown. Accordingly, the historical point cloud mentioned above may refer to the point cloud in the historical submap Z1, historical submap Z2, and the second type of point cloud may also refer to the point cloud in the historical submap Z1, historical submap Z2, etc.

[0219] based on Fig.13 The scheme shown projects the point cloud in the optimized crowdsourcing data onto the point cloud in the historical map data with a matching relationship based on semantic information, and outputs the quality inspection label according to the projection result. It can realize the automatic detection of inconsistent areas between crowdsourcing data and historical map data, which can improve the detection efficiency and reduce the cost compared with the manual detection method. In addition, the detection is based on the semantic information of the point cloud, so that only the elements of interest can be detected. Compared with the detection method based on geometric features, it will not be affected by outliers, dynamic objects or seasonally changing objects, which can improve the accuracy of the detection results.

[0220] The above mainly introduces the solution provided by the embodiment of the present application from the perspective of method. It can be understood that in order to realize the above functions, the map data processing device (such as the second device or the processor in the second device, etc.) includes hardware structures and / or software modules corresponding to the execution of each function. In combination with the units and algorithm steps of each example described in the embodiment disclosed in this application, the embodiment of the present application can be implemented in the form of hardware or a combination of hardware and computer software. Whether a function is executed in hardware or computer-driven hardware depends on the specific application and design constraints of the technical solution. Those skilled in the art may use different methods to implement the described functions for each specific application, but such implementation should not be considered to exceed the scope of the technical solution of the embodiment of the present application.

[0221] The present application is an embodiment that can divide the functional modules of the map data processing device according to the above method example. For example, each functional module can be divided according to each function, or two or more functions can be integrated into one processing unit. The above integrated unit can be implemented in the form of hardware or software functional modules. It should be noted that the division of units in the embodiment of the present application is schematic and is only a logical functional division. There may be other division methods in actual implementation.

[0222] like Fig.14 14 is a schematic diagram of a map data processing device provided in an embodiment of the present application. The map data processing device 1400 can be used to implement the methods described in the above method embodiments. Exemplarily, the map data processing device may specifically include: a processing unit 1401.

[0223] The processing unit 1401 is used to support the map data processing device 1400 to execute Figures 1 to 13 The processing function is performed by the second device described in any one of the above.

[0224] Optional, Fig.14 The map data processing device 1400 shown may further include a communication unit 1402, which is used to support the map data processing device 1400 in executing the steps of communication between the map data processing device and other devices (such as the first device) in the embodiment of the present application.

[0225] Optional, Fig.14 The map data processing device 1400 shown in the figure may also include a storage unit 1403, which stores a program or instruction. When the processing unit 1401 executes the program or instruction, Fig.14 The map data processing device 1400 shown can execute the method described in the above method embodiment.

[0226] Fig.14 The technical effects of the map data processing device 1400 shown can refer to the technical effects described in the above method embodiment, and will not be repeated here. Fig.14 The processing unit 1401 involved in the map data processing device 1400 shown can be implemented by a processor or a processor-related circuit component, which can be a processor or a processing module. The communication unit 1402 can be implemented by a transceiver or a transceiver-related circuit component, which can be a transceiver or a transceiver module.

[0227] The present application also provides a chip system, such as Fig.15 As shown, the chip system includes at least one processor 1501 and at least one interface circuit 1502. The processor 1501 and the interface circuit 1502 can be interconnected through lines. For example, the interface circuit 1502 can be used to receive signals from other devices. For another example, the interface circuit 1502 can be used to send signals to other devices (such as the processor 1501). Exemplarily, the interface circuit 1502 can read instructions stored in the memory and send the instructions to the processor 1501. When the instructions are executed by the processor 1501, the map data processing device can execute the various steps executed by the map data processing device in the above embodiment. Of course, the chip system can also include other discrete devices, which are not specifically limited in the embodiments of the present application.

[0228] Optionally, the processor in the chip system may be one or more. Optionally, the memory in the chip system may also be one or more. The memory may be integrated with the processor or may be separately provided from the processor, which is not limited in the present application.

[0229] The embodiment of the present application further provides a computer storage medium, in which computer instructions are stored. When the computer instructions are executed on a map data processing device, the map data processing device executes the method described in the above method embodiment.

[0230] An embodiment of the present application provides a computer program product, which includes: a computer program or instructions, when the computer program or instructions are executed on a computer, the computer executes the method described in the above method embodiment.

[0231] In addition, an embodiment of the present application also provides a device, which can specifically be a chip, component or module, and the device may include a connected processor and memory; wherein the memory is used to store computer-executable instructions, and when the device is running, the processor can execute the computer-executable instructions stored in the memory so that the device executes the methods in the above-mentioned method embodiments.

[0232] Among them, the map data processing device, computer storage medium, computer program product or chip provided in this embodiment are all used to execute the corresponding methods provided above. Therefore, the beneficial effects that can be achieved can refer to the beneficial effects in the corresponding methods provided above, and will not be repeated here.

[0233] The above contents are only specific implementation methods of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A map data processing method, characterized in that: The method comprises: Obtain at least one crowdsourced sub-map and at least one historical sub-map, wherein the at least one crowdsourced sub-map is obtained by segmenting crowdsourced data, and the at least one historical sub-map is obtained by segmenting historical map data, and the crowdsourced data is used to update the historical map data; Determining the road network level corresponding to the at least one crowdsourced sub-map according to the road network level corresponding to the at least one historical sub-map; Determining the position and posture of the at least one crowdsourced sub-map according to the at least one historical sub-map, and determining the road network level and the at least one crowdsourced sub-map after the position and posture are used for optimizing the crowdsourced data; The optimized crowd-sourced data is quality checked based on the historical map data.

2. The method according to claim 1, characterized in that The determining the road network level corresponding to the at least one crowdsourced sub-map according to the road network level corresponding to the at least one historical sub-map includes: Determining a road network level corresponding to a second submap according to a road network level corresponding to at least one first submap, wherein the first submap is a historical submap that has a matching relationship with the second submap and has a corresponding road network level of a single layer, and the second submap is any one of the at least one crowdsourced submap; The second sub-map for determining the road network level is used as a seed point, and the road network level corresponding to part or all of the sub-maps in the at least one crowdsourced sub-map is determined by region growing.

3. The method according to claim 2, characterized in that The determining the road network level corresponding to the second sub-map according to the road network level corresponding to at least one first sub-map includes: The road network level corresponding to the second submap is determined according to the point cloud registration result between the at least one first submap and the second submap, and the road network level corresponding to the at least one first submap.

4. The method according to claim 2 or 3, characterized in that: After determining the road network level corresponding to some sub-maps in the at least one crowdsourced sub-map by region growing, the method further includes: The road network level corresponding to the fourth submap is determined according to the road network level corresponding to at least one third submap, wherein the at least one third submap is a historical submap that has a matching relationship with the fourth submap and the corresponding road network level is multiple layers, and the fourth submap is a crowdsourced submap whose road network level is not determined by the region growing.

5. The method according to claim 4, characterized in that The determining the road network level corresponding to the fourth sub-map according to the road network level corresponding to at least one third sub-map includes: The road network level corresponding to the fourth submap is determined according to the point cloud registration results of each road network layer in the fourth submap and each third submap, and the level corresponding to each road network layer in each third submap.

6. The method according to claim 4 or 5, characterized in that: Before determining the road network level corresponding to the fourth sub-map according to the road network level corresponding to the at least one third sub-map, the method further includes: Clustering the road centerlines included in the first sub-map and the third sub-map respectively to obtain the number of roads included in the first sub-map and the number of roads included in the third sub-map, wherein the road centerlines are used to divide the historical map data into the at least one historical sub-map; Determining a road network level corresponding to the first sub-map according to the number of roads included in the first sub-map and the level corresponding to each road included in the first sub-map; The road network level corresponding to the third submap is determined according to the number of roads included in the third submap and the level corresponding to each road included in the third submap.

7. The method according to any one of claims 1 to 6, characterized in that: The determining the position of the at least one crowdsourced sub-map according to the at least one historical sub-map includes: Acquire parameters, where the parameters include at least one of a first registration result and a second registration result, where the first registration result is a point cloud registration result between a fifth submap and a sixth submap, where the fifth submap is a historical submap that has a matching relationship with the sixth submap, and where the sixth submap is any one of the at least one crowdsourced submap; The second registration result is an image registration result between a first bird's-eye view image and a second bird's-eye view image, the first bird's-eye view image is obtained according to the fifth sub-map, and the second bird's-eye view image is obtained according to the sixth sub-map; The position and posture of the sixth submap are determined according to the parameters.

8. The method according to claim 7, characterized in that The first registration result includes a first relative position between the fifth submap and the sixth submap, and the second registration result includes a second relative position between the fifth submap and the sixth submap; The determining the position and posture of the sixth submap according to the parameters includes: The initial posture of the sixth submap is adjusted according to the first relative posture and the second relative posture to obtain the posture of the sixth submap.

9. The method according to claim 7 or 8, characterized in that: The parameters also include one or more of prior information and mileage information; wherein the prior information includes an initial position of the sixth submap; the mileage information includes a relative position between the sixth submap and a seventh submap, and the seventh submap is a crowdsourced submap adjacent to the sixth submap.

10. The method according to claim 8 or 9, characterized in that: The parameters include a second registration result; The acquisition parameters include: The first bird's-eye view image is used as a template, and template matching is performed between the second bird's-eye view image and the template to determine the second relative posture.

11. The method according to any one of claims 8 to 10, characterized in that: The parameters include a first registration result; The acquisition parameters include: According to the semantic information of the point cloud contained in the fifth sub-map, the point cloud contained in the fifth sub-map is divided into at least one element of points, lines, and surfaces; According to the semantic information of the point cloud contained in the sixth sub-map, the point cloud contained in the sixth sub-map is divided into at least one element of points, lines, and surfaces; Determining residuals between elements of the same semantic information in the fifth submap and in the sixth submap; The first relative pose is determined according to the residual.

12. The method according to claim 11, characterized in that The determining of the residual between the elements of the same semantic information in the fifth submap and the sixth submap includes: The elements of the same semantic information are points, and the residuals between the points are determined according to the distances between the points; Or, the elements of the same semantic information are lines, and the residual between the lines is determined according to the distance from a point on one of the lines to another line; Alternatively, the elements of the same semantic information are surfaces, and the residual between the surfaces is determined according to the distance from a point on one of the surfaces to another surface.

13. The method according to any one of claims 1 to 12, characterized in that The quality inspection of the optimized crowd-sourced data according to the historical map data includes: Obtaining the optimized crowdsourcing data; According to the semantic information of the point cloud in the optimized crowd-sourced data and the semantic information of the target point cloud in the historical map data, the point cloud in the optimized crowd-sourced data is projected onto the target point cloud, and the target point cloud has a matching relationship with the point cloud in the optimized crowd-sourced data; A quality inspection label is output according to the projection result and the preset inspection rules, wherein the quality inspection label includes: an index of a crowdsourcing sub-map that meets the preset inspection rules, and one or more corresponding preset inspection rules.

14. The method according to claim 13, characterized in that The preset inspection rules include one or more of the following: For an optimized crowdsourcing submap, determining, according to the projection result, that an average residual corresponding to the optimized crowdsourcing submap is greater than or equal to a first threshold; For an optimized crowdsourcing sub-map, determining, according to the projection result, that an area of ​​a point cloud whose corresponding residual in the optimized crowdsourcing sub-map is a large residual is greater than or equal to a second threshold, wherein the large residual means that the residual corresponding to the point cloud included in the optimized crowdsourcing sub-map is greater than or equal to the first threshold; The sum of the first area and the second area is greater than or equal to a third threshold, the first area being the sum of the areas of the first type of point clouds contained in an optimized crowdsourcing sub-map, and there being no historical point clouds in the neighborhood of the first type of point clouds, the historical point clouds being point clouds in historical sub-maps that have a matching relationship with the optimized crowdsourcing sub-map; the second area being the sum of the areas of the second type of point clouds contained in historical sub-maps that have a matching relationship with the optimized crowdsourcing sub-map, and there being no point clouds in the optimized crowdsourcing sub-map in the neighborhood of the second type of point clouds; The optimized crowdsourcing sub-map is obtained by segmenting the optimized crowdsourcing data.

15. A map data processing device, characterized in that: including a processing unit; The processing unit is used for: Obtain at least one crowdsourced sub-map and at least one historical sub-map, wherein the at least one crowdsourced sub-map is obtained by segmenting crowdsourced data, and the at least one historical sub-map is obtained by segmenting historical map data, and the crowdsourced data is used to update the historical map data; Determining the road network level corresponding to the at least one crowdsourced sub-map according to the road network level corresponding to the at least one historical sub-map; Determining the position and posture of the at least one crowdsourced sub-map according to the at least one historical sub-map, and determining the road network level and the at least one crowdsourced sub-map after the position and posture are used for optimizing the crowdsourced data; The optimized crowd-sourced data is quality checked based on the historical map data.

16. The device according to claim 15, characterized in that The processing unit is specifically used for: Determining a road network level corresponding to a second submap according to a road network level corresponding to at least one first submap, wherein the first submap is a historical submap that has a matching relationship with the second submap and has a corresponding road network level of a single layer, and the second submap is any one of the at least one crowdsourced submap; The second sub-map for determining the road network level is used as a seed point, and the road network level corresponding to part or all of the sub-maps in the at least one crowdsourced sub-map is determined by region growing.

17. The device according to claim 16, characterized in that The processing unit is further used to determine the road network level corresponding to the second submap based on the point cloud registration result between the at least one first submap and the second submap, and the road network level corresponding to the at least one first submap.

18. The device according to claim 16 or 17, characterized in that The processing unit is further used to determine the road network level corresponding to the fourth sub-map according to the road network level corresponding to at least one third sub-map, wherein the at least one third sub-map is a historical sub-map that has a matching relationship with the fourth sub-map and the corresponding road network level is a multi-layer historical sub-map, and the fourth sub-map is a crowdsourced sub-map whose road network level is not determined by the regional growth.

19. The device according to claim 18, characterized in that The processing unit is specifically configured to determine the road network level corresponding to the fourth submap according to the point cloud registration results of each road network layer in the fourth submap and each third submap, and the level corresponding to each road network layer in each third submap.

20. The device according to claim 18 or 19, characterized in that The processing unit is also used to: Clustering the road centerlines included in the first sub-map and the third sub-map respectively to obtain the number of roads included in the first sub-map and the number of roads included in the third sub-map, wherein the road centerlines are used to divide the historical map data into the at least one historical sub-map; Determining a road network level corresponding to the first sub-map according to the number of roads included in the first sub-map and the level corresponding to each road included in the first sub-map; The road network level corresponding to the third submap is determined according to the number of roads included in the third submap and the level corresponding to each road included in the third submap.

21. The device according to any one of claims 15 to 20, characterized in that The processing unit is specifically used for: Acquire parameters, where the parameters include at least one of a first registration result and a second registration result, where the first registration result is a point cloud registration result between a fifth submap and a sixth submap, where the fifth submap is a historical submap that has a matching relationship with the sixth submap, and where the sixth submap is any one of the at least one crowdsourced submap; The second registration result is an image registration result between a first bird's-eye view image and a second bird's-eye view image, the first bird's-eye view image is obtained according to the fifth sub-map, and the second bird's-eye view image is obtained according to the sixth sub-map; The position and posture of the sixth submap are determined according to the parameters.

22. The device according to claim 21, characterized in that The first registration result includes a first relative position between the fifth submap and the sixth submap, and the second registration result includes a second relative position between the fifth submap and the sixth submap; The processing unit is specifically configured to adjust the initial posture of the sixth submap according to the first relative posture and the second relative posture to obtain the posture of the sixth submap.

23. The device according to claim 21 or 22, characterized in that The parameters also include one or more of prior information and mileage information; wherein the prior information includes an initial position of the sixth submap; the mileage information includes a relative position between the sixth submap and a seventh submap, and the seventh submap is a crowdsourced submap adjacent to the sixth submap.

24. The device according to claim 22 or 23, characterized in that The parameters include a second registration result; The processing unit is specifically configured to use the first bird's-eye view image as a template, perform template matching between the second bird's-eye view image and the template, and determine the second relative posture.

25. The device according to any one of claims 22 to 24, characterized in that The parameters include a first registration result; The processing unit is specifically used for: According to the semantic information of the point cloud contained in the fifth sub-map, the point cloud contained in the fifth sub-map is divided into at least one element of points, lines, and surfaces; According to the semantic information of the point cloud contained in the sixth sub-map, the point cloud contained in the sixth sub-map is divided into at least one element of points, lines, and surfaces; Determining residuals between elements of the same semantic information in the fifth submap and in the sixth submap; The first relative pose is determined according to the residual.

26. The device according to claim 25, characterized in that The elements of the same semantic information are points, and the processing unit is used to determine the residuals between the points according to the distances between the points; Or, the elements of the same semantic information are lines, and the processing unit is used to determine the residual between the lines according to the distance from a point included in one of the lines to another line; Alternatively, the elements of the same semantic information are surfaces, and the processing unit is used to determine the residual between the surfaces based on the distance from a point included in one of the surfaces to another surface.

27. The device according to any one of claims 15 to 26, characterized in that The processing unit is specifically used for: Obtaining the optimized crowdsourcing data; According to the semantic information of the point cloud in the optimized crowd-sourced data and the semantic information of the target point cloud in the historical map data, the point cloud in the optimized crowd-sourced data is projected onto the target point cloud, and the target point cloud has a matching relationship with the point cloud in the optimized crowd-sourced data; A quality inspection label is output according to the projection result and the preset inspection rules, wherein the quality inspection label includes: an index of a crowdsourcing sub-map that meets the preset inspection rules, and one or more corresponding preset inspection rules.

28. The device according to claim 27, characterized in that The preset inspection rules include one or more of the following: For an optimized crowdsourcing submap, determining, according to the projection result, that an average residual corresponding to the optimized crowdsourcing submap is greater than or equal to a first threshold; For an optimized crowdsourcing sub-map, determining, according to the projection result, that an area of ​​a point cloud whose corresponding residual in the optimized crowdsourcing sub-map is a large residual is greater than or equal to a second threshold, wherein the large residual means that the residual corresponding to the point cloud included in the optimized crowdsourcing sub-map is greater than or equal to the first threshold; The sum of the first area and the second area is greater than or equal to a third threshold, the first area being the sum of the areas of the first type of point clouds contained in an optimized crowdsourcing sub-map, and there being no historical point clouds in the neighborhood of the first type of point clouds, the historical point clouds being point clouds in historical sub-maps that have a matching relationship with the optimized crowdsourcing sub-map; the second area being the sum of the areas of the second type of point clouds contained in historical sub-maps that have a matching relationship with the optimized crowdsourcing sub-map, and there being no point clouds in the optimized crowdsourcing sub-map in the neighborhood of the second type of point clouds; The optimized crowdsourcing sub-map is obtained by segmenting the optimized crowdsourcing data.

29. A map data processing device, characterized in that: The map data processing device comprises a processor coupled to a memory; the processor is used to execute a computer program stored in the memory, so that the map data processing device executes the method according to any one of claims 1 to 14.

30. A computer-readable storage medium, characterized in that: The computer-readable storage medium comprises a computer program, and when the computer program is executed on a map data processing device, the map data processing device is caused to execute the method according to any one of claims 1 to 14.

Images