Method and apparatus for updating a three-dimensional point cloud crowdsourced semantic map

Abstract

The application relates to a method and device for updating a three-dimensional point cloud crowdsourcing semantic map. The method comprises the following steps: matching a newly collected crowdsourcing map and an existing semantic map of the same collection area to obtain at least one of a first unmatched point cloud object set and a second unmatched point cloud object set; judging whether each first unmatched point cloud object in the first unmatched point cloud object set collides with a corresponding point cloud object in a preset semantic distance range of the semantic map; judging whether each second unmatched point cloud object in the second unmatched point cloud object set collides with a corresponding point cloud object in a preset semantic distance range of the crowdsourcing map; and updating the semantic map according to the bounding box collision result of the first unmatched point cloud object and / or the bounding box collision result of the second unmatched point cloud object. The method can reduce object detection errors and improve the map updating accuracy.

Description

Technical Field

[0001] This application relates to the field of autonomous driving map and positioning technology, and in particular to a method and apparatus for updating a crowdsourced semantic map based on 3D point cloud. Background Technology

[0002] In fields such as autonomous driving and high-precision mapping, crowdsourced data collection and mapping, which has a high update frequency and low computational cost, is increasingly favored by industry and academia compared to professional data collection and mapping, which has a low update frequency and relatively high time and labor costs.

[0003] Traditional data collection methods typically require a large number of specialized data collectors, surveying equipment, and collection vehicles. High-precision map collection needs to include road network data, lane network data, road traffic facility data, and more, often requiring multiple data collection runs along a single road to ensure accuracy. After collection, the data must undergo numerous stages, including data fusion, processing, publishing, and delivery.

[0004] Crowdsourced data collection can be understood as users using the sensors of autonomous vehicles themselves, or other low-cost sensors, to collect road data and transmit it to the cloud for data fusion. This fusion method improves data accuracy, enabling the creation of crowdsourced high-precision maps or semantic maps. It mainly includes visual and radar methods. Among these, visual crowdsourced data collection solutions are currently the primary data collection solutions for mapping and autonomous driving companies both domestically and internationally. Crowdsourced data collection solutions based on radar 3D point clouds, however, suffer from larger point cloud data volumes and higher data transmission costs compared to visual image data.

[0005] However, current crowdsourced data collection and mapping methods suffer from inconsistent sensor errors across multiple vehicles, and even the same vehicle collecting data on the same object may have different errors, which can easily lead to a loss of accuracy. Summary of the Invention

[0006] Therefore, it is necessary to provide a method, apparatus, computer equipment, computer-readable storage medium, and computer program product for updating a crowdsourced semantic map based on 3D point cloud, which can improve the accuracy of map updates, in order to address the above-mentioned technical problems.

[0007] Firstly, this application provides a method for updating a crowdsourced semantic map based on 3D point clouds.

[0008] The method includes:

[0009] Match newly collected crowdsourced maps and existing semantic maps in the same collection area to obtain the first set of unmatched point cloud objects and the second set of unmatched point cloud objects.

[0010] Determine whether there is a bounding box collision between each unmatched point cloud object in the first unmatched point cloud object set and the corresponding point cloud object within a preset semantic distance range in the semantic map;

[0011] Determine whether there is a bounding box collision between each unmatched point cloud object in the second unmatched point cloud object set and the corresponding point cloud object within a preset semantic distance range in the crowdsourced map;

[0012] The semantic map is updated based on the bounding box collision results of the first unmatched point cloud object and / or the bounding box collision results of the second unmatched point cloud object.

[0013] In one embodiment, updating the semantic map based on the bounding box collision result of the second unmatched point cloud object includes:

[0014] When the bounding box collision result of the second unmatched point cloud object is that the second unmatched point cloud object does not have a bounding box collision, the second unmatched point cloud object is deleted from the semantic map.

[0015] In one embodiment, updating the semantic map based on the bounding box collision result of the first unmatched point cloud object includes:

[0016] When the bounding box collision result of the first unmatched point cloud object is that the first unmatched point cloud object does not have a bounding box collision, the first unmatched point cloud object is added to the semantic map.

[0017] In one embodiment, the method based on the bounding box collision result of the first unmatched point cloud object and the bounding box collision result of the second unmatched point cloud object includes:

[0018] When the bounding box collision result of the first unmatched point cloud object is that the first unmatched point cloud object does not have a bounding box collision, the first unmatched point cloud object is added to the semantic map; and

[0019] When the bounding box collision result of the second unmatched point cloud object is that the second unmatched point cloud object does not have a bounding box collision, the second unmatched point cloud object is deleted from the semantic map.

[0020] In one embodiment, updating the semantic map based on the bounding box collision result of the first unmatched point cloud object includes:

[0021] When the bounding box collision result indicates that the first unmatched point cloud object has a bounding box collision, the corresponding first colliding point cloud object is determined from the semantic map within a preset distance range.

[0022] For the first unmatched point cloud object and the first colliding point cloud object, the point cloud proportion is obtained;

[0023] The semantic map is updated based on the point cloud proportion and a set threshold.

[0024] In one embodiment, obtaining the point cloud proportion based on the first unmatched point cloud object and the first colliding point cloud object includes:

[0025] Determine the first object point cloud corresponding to the first unmatched point cloud object, and the first point cloud object convex hull corresponding to the first colliding point cloud object;

[0026] Determine the number of points in the first object's point cloud that lie within the convex hull of the first point cloud object, and obtain the point cloud percentage.

[0027] In one embodiment, updating the semantic map based on the point cloud proportion and a set threshold includes:

[0028] When the proportion of the point cloud is greater than or equal to the set threshold, the first unmatched point cloud object and the first colliding point cloud object are determined to be the same object.

[0029] In one embodiment, updating the semantic map based on the point cloud proportion and a set threshold includes:

[0030] When the proportion of the point cloud is less than the set threshold, it is determined that the first unmatched point cloud object and the first colliding point cloud object are not the same object.

[0031] Add the first unmatched point cloud object to the semantic map.

[0032] In one embodiment, before obtaining the point cloud proportion based on the first unmatched point cloud object and the first colliding point cloud object, the method further includes:

[0033] Determine whether there is a convex hull collision between the first unmatched point cloud object and the first colliding point cloud object;

[0034] When a convex hull collision occurs, the step of obtaining the point cloud proportion based on the first unmatched point cloud object and the first colliding point cloud object is executed.

[0035] In one embodiment, the method further includes:

[0036] When there is no convex hull collision, it is determined that the first unmatched point cloud object and the first colliding point cloud object are not the same object;

[0037] Add the first unmatched point cloud object to the semantic map.

[0038] In one embodiment, updating the semantic map based on the bounding box collision result of the second unmatched point cloud object includes:

[0039] When the bounding box collision result of the second unmatched point cloud object is that the second unmatched point cloud object has a bounding box collision, the corresponding second collision point cloud object is determined from the preset distance range in the crowdsourced map.

[0040] Determine whether there is a convex hull collision between the second unmatched point cloud object and the second colliding point cloud object;

[0041] When there is no convex hull collision, it is determined that the second unmatched point cloud object and the second colliding point cloud object are not the same object;

[0042] Remove the second unmatched point cloud object from the semantic map.

[0043] In one embodiment, the method further includes:

[0044] When a convex hull collision occurs, the second unmatched point cloud object and the second colliding point cloud object are determined to be the same object.

[0045] Secondly, this application also provides an update device for a crowdsourced semantic map based on 3D point clouds. The device includes:

[0046] The point cloud object matching module is used to match newly collected crowdsourced maps and existing semantic maps in the same collection area to obtain a first set of unmatched point cloud objects and a second set of unmatched point cloud objects.

[0047] The point cloud object collision module is used to determine whether there is a bounding box collision between each unmatched point cloud object in the first unmatched point cloud object set and the corresponding point cloud object within a preset distance range in the semantic map.

[0048] Determine whether each unmatched point cloud object in the second unmatched point cloud object set has a bounding box collision with the corresponding point cloud object within a preset distance range in the crowdsourced map, and obtain the bounding box collision result.

[0049] The map update module is used to update the semantic map based on the bounding box collision results of the first unmatched point cloud object and / or the bounding box collision results of the second unmatched point cloud object.

[0050] Thirdly, this application also provides a computer device. The computer device includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program to perform the following steps:

[0051] Match newly collected crowdsourced maps and existing semantic maps in the same collection area to obtain the first set of unmatched point cloud objects and the second set of unmatched point cloud objects.

[0052] Determine whether there is a bounding box collision between each unmatched point cloud object in the first unmatched point cloud object set and the corresponding point cloud object within a preset semantic distance range in the semantic map;

[0053] Determine whether there is a bounding box collision between each unmatched point cloud object in the second unmatched point cloud object set and the corresponding point cloud object within a preset semantic distance range in the crowdsourced map;

[0054] The semantic map is updated based on the bounding box collision results of the first unmatched point cloud object and / or the bounding box collision results of the second unmatched point cloud object.

[0055] Fourthly, this application also provides a computer-readable storage medium. The computer-readable storage medium stores a computer program thereon, which, when executed by a processor, performs the following steps:

[0056] Match newly collected crowdsourced maps and existing semantic maps in the same collection area to obtain the first set of unmatched point cloud objects and the second set of unmatched point cloud objects.

[0057] Determine whether there is a bounding box collision between each unmatched point cloud object in the first unmatched point cloud object set and the corresponding point cloud object within a preset semantic distance range in the semantic map;

[0058] Determine whether there is a bounding box collision between each unmatched point cloud object in the second unmatched point cloud object set and the corresponding point cloud object within a preset semantic distance range in the crowdsourced map;

[0059] The semantic map is updated based on the bounding box collision results of the first unmatched point cloud object and / or the bounding box collision results of the second unmatched point cloud object.

[0060] Fifthly, this application also provides a computer program product. The computer program product includes a computer program that, when executed by a processor, performs the following steps:

[0061] Match newly collected crowdsourced maps and existing semantic maps in the same collection area to obtain the first set of unmatched point cloud objects and the second set of unmatched point cloud objects.

[0062] Determine whether there is a bounding box collision between each unmatched point cloud object in the first unmatched point cloud object set and the corresponding point cloud object within a preset semantic distance range in the semantic map;

[0063] Determine whether there is a bounding box collision between each unmatched point cloud object in the second unmatched point cloud object set and the corresponding point cloud object within a preset semantic distance range in the crowdsourced map;

[0064] The semantic map is updated based on the bounding box collision results of the first unmatched point cloud object and / or the bounding box collision results of the second unmatched point cloud object.

[0065] The aforementioned update method, apparatus, computer equipment, storage medium, and computer program product based on a crowdsourced semantic map of 3D point clouds match newly acquired crowdsourced maps and existing semantic maps of the same acquisition area to obtain at least one of a first set of unmatched point cloud objects and a second set of unmatched point cloud objects; bounding box collision is performed on the point cloud objects in the first set of unmatched point cloud objects and / or the point cloud objects in the second set of unmatched point cloud objects; and a second matching detection is performed on the point cloud objects that do not exist in the first matching to improve the accuracy of map update. Attached Figure Description

[0066] Figure 1 This is an application environment diagram of an update method for a crowdsourced semantic map based on 3D point cloud in one embodiment;

[0067] Figure 2 This is a flowchart illustrating an update method for a crowdsourced semantic map based on 3D point clouds in one embodiment.

[0068] Figure 3 This is a flowchart illustrating a method for updating a semantic map based on a first unmatched point cloud object in one embodiment.

[0069] Figure 4 This is a schematic diagram illustrating the determination of the colliding point cloud object of the first unmatched point cloud object in one embodiment.

[0070] Figure 5 This is a flowchart illustrating a method for updating a semantic map based on a first unmatched point cloud object in another embodiment.

[0071] Figure 6a In one embodiment, two point cloud objects are successfully matched.

[0072] Figure 6b In one embodiment, there are two mismatched point cloud objects;

[0073] Figure 7 This is a flowchart illustrating a method for updating a semantic map based on a second unmatched point cloud object in one embodiment.

[0074] Figure 8 This is a schematic diagram illustrating the determination of a collision point cloud object of a second unmatched point cloud object in one embodiment.

[0075] Figure 9 This is a flowchart illustrating a semantic distance determination method in one embodiment;

[0076] Figure 10 This is a flowchart illustrating a map matching method in one embodiment;

[0077] Figure 11 This is a flowchart illustrating an update method for a crowdsourced semantic map based on 3D point clouds in another embodiment.

[0078] Figure 12 This is a structural block diagram of an update device based on a crowdsourced semantic map of 3D point cloud in one embodiment;

[0079] Figure 13 This is an internal structural diagram of a computer device in one embodiment. Detailed Implementation

[0080] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0081] The update method for crowdsourced semantic maps based on 3D point clouds provided in this application embodiment can be applied to, for example... Figure 1In the application environment shown, the acquisition device 102 communicates with the terminal 104 via a network. A data storage system can store the data that the terminal 104 needs to process. The data storage system can be integrated into the terminal 104 or placed in the cloud or on another network server. The terminal 104 matches newly acquired crowdsourced maps and existing semantic maps of the same acquisition area to obtain a first set of unmatched point cloud objects and a second set of unmatched point cloud objects; it determines whether there is a bounding box collision between each unmatched point cloud object in the first set of unmatched point cloud objects and the corresponding point cloud objects within a preset semantic distance range in the semantic map; and it determines whether there is a bounding box collision between each unmatched point cloud object in the second set of unmatched point cloud objects and the corresponding point cloud objects within a preset semantic distance range in the crowdsourced map; based on the bounding box collision results of the first unmatched point cloud objects and / or the second unmatched point cloud objects, it updates the semantic map. The terminal 104 can be, but is not limited to, various personal computers, laptops, smartphones, tablets, IoT devices, or autonomous driving computing platforms. The acquisition device can be a lidar, millimeter-wave radar, or ultrasonic radar, or it can be integrated into the terminal. Understandably, this method can also be applied to servers, and to systems that include both terminals and servers, and can be implemented through the interaction between terminals and servers.

[0082] In one embodiment, such as Figure 2 As shown, a method for updating a crowdsourced semantic map based on 3D point clouds is provided, which is then applied to... Figure 1 Taking the terminal in the example, the explanation includes the following steps:

[0083] Step 202: Match the newly collected crowdsourced map and the existing semantic map in the same collection area to obtain the first set of unmatched point cloud objects and the second set of unmatched point cloud objects.

[0084] Crowdsourced maps refer to maps obtained through crowdsourcing. This can be understood as users collecting road data using sensors on autonomous vehicles or other low-cost sensors, uploading it to the cloud for data fusion, and improving data accuracy through this fusion process to create crowdsourced high-precision maps or semantic maps. Crowdsourced maps or crowdsourced maps refer to maps acquired by terminals via the cloud, collected by other vehicles, and uploaded to the cloud. Semantic maps refer to existing maps.

[0085] The first set of unmatched point cloud objects refers to the set of point cloud objects in the crowdsourced map that do not have a corresponding point cloud object in the semantic map. The point cloud object in the crowdsourced map may be a newly added point cloud object. The second set of unmatched point cloud objects refers to the set of point cloud objects in the semantic map that do not have a corresponding point cloud object in the crowdsourced map. The point cloud object in the semantic map may be a disappeared point cloud object.

[0086] Understandably, in non-ideal conditions, such as environmental occlusion or sensor performance issues (e.g., only half of an object is scanned), objects may be mistakenly identified as two different objects, resulting in the failure to match the point cloud object during the matching process. In the first matching process, newly added point cloud objects and disappeared point cloud objects will be identified. Newly added point cloud objects refer to point cloud objects newly added to the crowdsourced map compared to the semantic map, while disappeared point cloud objects refer to point cloud objects that have disappeared from the crowdsourced map compared to the semantic map.

[0087] The semantic distance is determined based on the semantic information of point cloud objects in the newly collected crowdsourced map and the existing semantic map. The obtained semantic distance is weighted with a pre-set weight to obtain the final semantic distance. The crowdsourced map and the semantic map are then matched using bipartite graph matching based on the semantic distance to obtain matching pairs of point cloud objects that exist and / or matching pairs of point cloud objects that do not exist. Point cloud objects that do not have matching pairs are identified as unmatched point cloud objects.

[0088] The semantic information is predetermined and includes the point cloud object identifier, the point cloud PCA (Principal Components Analysis) coordinate system orientation, the point cloud principal direction, the point cloud object center, the point cloud oriented bounding box (OBB), the point cloud convex hull, and the point cloud histogram. OBB has better compactness, which can significantly reduce the number of bounding boxes participating in intersection tests, resulting in better overall performance than AABB. When the geometric object undergoes rotation, only the OBB needs to be rotated in the same way.

[0089] Furthermore, the point cloud object identifier for each object is different, i.e., unique. Point cloud PCA (Programmatical Convergence Aspect Ratio) uses eigenvalue decomposition of the point cloud covariance matrix to determine the orientation in three dimensions through eigenvectors, defining this coordinate system as the object's local coordinate system. The principal orientation of the point cloud is determined after PCA calculation, selecting the eigenvector corresponding to the smallest eigenvalue as the principal orientation of the point cloud object. For example, a common traffic sign has a point cloud that is approximately planar (of course, a threshold is given during mapping; if it is less than the threshold, a minimum width is given to ensure that the point cloud object is a three-dimensional object). In this case, the eigenvalue corresponding to the plane normal direction is the smallest, and we define this normal direction as the principal orientation.

[0090] The center of the point cloud object is determined based on the maximum and minimum coordinate values of the points in the point cloud. That is, in the local coordinate system, the maximum and minimum coordinate values on different axes are determined. After weighted average processing based on the maximum and minimum coordinate values, the obtained coordinates are converted to the world coordinate system. The bounding box dimensions of the minimum bounding box OBB of the point cloud are width, height, and depth, where width < height < depth, and it can be represented by eight vertices in the world coordinate system; the convex hull of the point cloud is the smallest convex hull that contains all the points of an object; the point cloud histogram stores the histograms of the three dimensions of x, y, and z in a one-dimensional vector.

[0091] Specifically, the semantic distance is determined according to the semantic information of the point cloud objects in the newly collected crowdsourced map and the existing semantic map. The obtained semantic distance is weighted with a preset weight value to obtain the final semantic distance; the crowdsourced map and the semantic map are subjected to bipartite graph matching according to the semantic distance to obtain point cloud object matching pairs and / or non-existent point cloud object matching pairs, and the point cloud objects with non-existent point cloud object matching pairs are determined as unmatched point cloud objects; that is, the first set of unmatched point cloud objects of the newly added point cloud objects in the crowdsourced map recognized in the first matching process is obtained, and / or, the second set of unmatched point cloud objects of the disappeared point cloud objects in the semantic map.

[0092] Step 204: Determine whether there is a bounding box collision between each first unmatched point cloud object in the first set of unmatched point cloud objects and the corresponding point cloud object within the preset semantic distance range in the semantic map.

[0093] Among them, the preset semantic distance range is preset and is used to determine, from the semantic map, the corresponding point cloud objects (which can be understood as the first candidate point cloud objects) within the preset semantic distance range for each first unmatched point cloud object in the first set of unmatched point cloud objects.

[0094] Bounding box collision, that is, OBB collision (Oriented Bounding Box), and the OBB collision processing adopts but is not limited to the separating axis theorem. It can be understood that if an axis can be found where the projections of two convex shapes on this axis do not overlap, then these two shapes do not intersect. If this axis does not exist and those shapes are convex, then it can be determined that the two shapes intersect (concave shapes are not applicable, such as crescent shapes. Even if no separating axis can be found, two crescent shapes may not intersect).

[0095] This can also be understood as follows: if a straight line can be found such that bounding box A is completely on one side of the line and bounding box B is completely on the other side, then the two bounding boxes do not overlap. This straight line is called the separation line (called the separation surface in the 3D world) and is always perpendicular to the separation axis. In this embodiment, OBB collision handling needs to test 15 separation axes to determine the intersection state of the OBBs. Each of the two OBBs has 3 coordinate axes, plus 9 axes perpendicular to each axis. The collision judgment is the same as existing 2D OBB collisions, that is, if the projections of the two polygons overlap on all axes, then a collision is determined; otherwise, no collision occurs, which will not be elaborated further here.

[0096] Specifically, the semantic information of each unmatched point cloud object in the first unmatched point cloud object set and the corresponding point cloud object within a preset semantic distance range is obtained, the semantic distance is calculated, the first candidate point cloud object within the preset semantic distance range is determined, and it is determined whether there is a bounding box collision between each unmatched point cloud object in the first unmatched point cloud object set and the corresponding first candidate point cloud object, so as to obtain the bounding box collision result; the bounding box collision result includes whether there is a bounding box collision or not.

[0097] Step 206: Determine whether there is a bounding box collision between each unmatched point cloud object in the second unmatched point cloud object set and the corresponding point cloud object within the preset semantic distance range in the crowdsourced map.

[0098] The preset semantic distance range is pre-set and is used to determine the point cloud objects (which can be understood as second candidate point cloud objects) corresponding to each second unmatched point cloud object in the second unmatched point cloud object set from the crowdsourced map within the preset semantic distance range.

[0099] Specifically, the semantic information of each second unmatched point cloud object in the second unmatched point cloud object set and the corresponding point cloud object within a preset semantic distance range is obtained, the semantic distance is calculated, the second candidate point cloud object within the preset semantic distance range is determined, and it is determined whether there is a bounding box collision between each second unmatched point cloud object in the second unmatched point cloud object set and the corresponding candidate point cloud object, so as to obtain the bounding box collision result; the bounding box collision result includes whether there is a bounding box collision or not.

[0100] Step 208: Update the semantic map based on the bounding box collision results of the first unmatched point cloud object and / or the bounding box collision results of the second unmatched point cloud object.

[0101] It is understandable that when matching a newly collected crowdsourced map and an existing semantic map of the same collection area, at least one of a first set of unmatched point cloud objects and a second set of unmatched point cloud objects is obtained; that is, in the first matching process, it is determined that there are newly added point cloud objects in the crowdsourced map, and / or that there are disappeared point cloud objects in the semantic map. Updating the semantic map includes adding newly added point cloud objects and / or deleting disappeared point cloud objects.

[0102] Specifically, the semantic map is updated based on the bounding box collision result of the first unmatched point cloud object indicating that the second unmatched point cloud object does not have a bounding box collision, and / or the bounding box collision result of the second unmatched point cloud object indicates that the first unmatched point cloud object does not have a bounding box collision; and the semantic map is updated based on the bounding box collision result of the first unmatched point cloud object indicating that the second unmatched point cloud object has a bounding box collision, and / or the bounding box collision result of the second unmatched point cloud object indicates that the first unmatched point cloud object has a bounding box collision.

[0103] Furthermore, when the bounding box collision result of the first unmatched point cloud object is that the second unmatched point cloud object does not have a bounding box collision, the second unmatched point cloud object is deleted from the semantic map; when the bounding box collision result of the second unmatched point cloud object is that the first unmatched point cloud object does not have a bounding box collision, the first unmatched point cloud object is added to the semantic map.

[0104] In the above-mentioned update method based on 3D point cloud crowdsourced semantic map, the newly collected crowdsourced map and the existing semantic map of the same collection area are matched to obtain at least one of the first unmatched point cloud object set and the second unmatched point cloud object set; bounding box collision is performed on the point cloud objects in the first unmatched point cloud object set and the point cloud objects in the second unmatched point cloud object set; for unmatched point cloud objects that do not exist in the first matching, a second matching detection is performed to improve the map update accuracy.

[0105] In one embodiment, such as Figure 3 As shown, a method for updating a semantic map based on the first unmatched point cloud objects is provided, and this method is applied to... Figure 1 Taking the terminal in the example, the explanation includes the following steps:

[0106] Step 302: When the bounding box collision result of the first unmatched point cloud object is that the first unmatched point cloud object has a bounding box collision, determine the corresponding first colliding point cloud object from the preset distance range in the semantic map.

[0107] Specifically, the newly collected crowdsourced map and the existing semantic map in the same collection area are matched. When the first unmatched point cloud object set of the crowdsourced map is obtained, it is determined whether there is a bounding box collision between each unmatched point cloud object in the first unmatched point cloud object set and the corresponding point cloud object within the preset semantic distance range in the semantic map. When the bounding box collision result of the first unmatched point cloud object is that the first unmatched point cloud object has a bounding box collision, the corresponding first colliding point cloud object is determined from the preset distance range in the semantic map. When the bounding box collision result of the first unmatched point cloud object is that the first unmatched point cloud object does not have a bounding box collision, the first unmatched point cloud object is added to the semantic map.

[0108] like Figure 4 As shown, the first unmatched point cloud object set 1 of the crowdsourced map A includes the first unmatched point cloud object n and the first unmatched point cloud object m. The first unmatched point cloud object n has a bounding box collision with point cloud object d in the semantic map B (including point cloud object d and point cloud object f) within a preset distance range, and point cloud object d is determined to be the first colliding point cloud object.

[0109] Step 304: Obtain the point cloud proportion based on the first unmatched point cloud object and the first colliding point cloud object.

[0110] The point cloud percentage refers to the percentage of points in the object point cloud corresponding to the first unmatched point cloud object within the convex hull of the first colliding point cloud object, relative to the total number of points in the object point cloud. This involves projecting or mapping the object point cloud corresponding to the first unmatched point cloud object and the convex hull of the first colliding point cloud object onto the same coordinate system, determining the number of points in the convex hull of the first colliding point cloud object within the same coordinate system, and then determining the point cloud percentage based on the number of points.

[0111] Specifically, the first object point cloud corresponding to the first unmatched point cloud object and the first point cloud object convex hull corresponding to the first collision point cloud object are determined; the number of points in the first object point cloud in the first point cloud object convex hull is determined, and the point cloud proportion is obtained based on the number of points and the total number of points in the first object point cloud.

[0112] Step 306: Update the semantic map based on the point cloud proportion and the set threshold.

[0113] Specifically, when the proportion of point cloud is greater than or equal to a set threshold, the first unmatched point cloud object and the first colliding point cloud object are determined to be the same object, and the original point cloud object in the semantic map is retained; when the proportion of point cloud is less than the set threshold, the first unmatched point cloud object and the first colliding point cloud object are determined to be different objects, and the first unmatched point cloud object is added to the semantic map.

[0114] In the above method for updating the semantic map based on the first unmatched point cloud object, when the first unmatched point cloud object has a bounding box collision, point cloud detection is performed on the first unmatched point cloud object and the corresponding first colliding point cloud object. The semantic map is updated according to the percentage of the number of points in the object point cloud corresponding to the first unmatched point cloud object within the object convex hull of the first colliding point cloud object to the total number of points in the object point cloud. This avoids errors caused by sensor errors or noise interference during data acquisition and improves the accuracy of map updates.

[0115] In another embodiment, such as Figure 5 As shown, a method for updating a semantic map based on the first unmatched point cloud objects is provided, and this method is applied to... Figure 1 Taking the terminal in the example, the explanation includes the following steps:

[0116] Step 502: When the bounding box collision result of the first unmatched point cloud object is that the first unmatched point cloud object has a bounding box collision, determine the corresponding first colliding point cloud object from the preset distance range in the semantic map.

[0117] Step 504: Determine whether there is a convex hull collision between the first unmatched point cloud object and the first colliding point cloud object. If yes, proceed to step 506; otherwise, proceed to step 510.

[0118] Convex hull collision refers to detecting whether all the convex hull points of one point cloud object are inside the convex hull of another point cloud object. For example, we can find the equation f(x,y,z) = 0 for each face of the convex hull of the point cloud object, and then substitute the judgment point into f, and also substitute other points of the convex hull into f. If they have opposite signs, it means that they are not inside. Here, the judgment point refers to every point in the convex hull of one point cloud object, and we determine whether each point is in the convex hull of another point cloud.

[0119] It's understandable that collisions in OBB and convex hull collision handling don't necessarily mean two point cloud objects are the same object. In reality, collisions can occur due to sensor errors during data acquisition or significant noise during point cloud clustering and object construction. Further point cloud analysis is needed. Figure 6a As shown, these are two point cloud objects that were successfully matched; Figure 6b As shown, there are two mismatched point cloud objects.

[0120] Step 506: Obtain the point cloud percentage for the first unmatched point cloud object and the first colliding point cloud object.

[0121] Step 508: Update the semantic map based on the point cloud proportion and the set threshold.

[0122] Step 510: When there is no convex hull collision, determine that the first unmatched point cloud object and the first colliding point cloud object are not the same object.

[0123] Step 512: Add the first unmatched point cloud object to the semantic map.

[0124] In the aforementioned method for updating the semantic map based on the first unmatched point cloud object, when the first unmatched point cloud object has a bounding box collision, convex hull collision and point cloud detection are performed on the first unmatched point cloud object and the corresponding first colliding point cloud object to update the semantic map. That is, OBB collision detection and convex hull collision detection, as well as point cloud judgment, are performed sequentially to avoid mismatches due to occlusion or sensor performance issues. For unmatched point cloud objects, a second matching detection is performed to improve map update accuracy.

[0125] In one embodiment, such as Figure 7 As shown, a method for updating a semantic map based on a second unmatched point cloud object is provided, which can be applied to... Figure 1 Taking the terminal in the example, the explanation includes the following steps:

[0126] Step 702: When the bounding box collision result of the second unmatched point cloud object is that the second unmatched point cloud object has a bounding box collision, determine the corresponding second collision point cloud object from the preset distance range in the crowdsourced map.

[0127] Specifically, the newly collected crowdsourced map and the existing semantic map of the same collection area are matched. When the second unmatched point cloud object set of the semantic map is obtained, it is determined whether there is a bounding box collision between each second unmatched point cloud object in the second unmatched point cloud object set and the corresponding point cloud object within the preset semantic distance range in the semantic map. When the bounding box collision result of the second unmatched point cloud object is that the second unmatched point cloud object has a bounding box collision, the corresponding second colliding point cloud object is determined from the preset distance range in the semantic map. When the bounding box collision result of the second unmatched point cloud object is that the second unmatched point cloud object does not have a bounding box collision, the second unmatched point cloud object is deleted from the semantic map.

[0128] like Figure 8 As shown, the second unmatched point cloud object set of semantic map A includes the second unmatched point cloud object n and the second unmatched point cloud object m. The second unmatched point cloud object n has a bounding box collision with point cloud object d in crowdsourced map B (including point cloud object d and point cloud object f) within a preset distance range, and point cloud object d is determined to be the second collision point cloud object.

[0129] Step 704: Determine whether there is a convex hull collision between the second unmatched point cloud object and the second colliding point cloud object; if yes, proceed to step 710; otherwise, proceed to step 706.

[0130] Step 706: When there is no convex hull collision, determine that the second unmatched point cloud object and the second colliding point cloud object are not the same object.

[0131] Step 708 removes the second unmatched point cloud object from the semantic map.

[0132] Step 710: When a convex hull collision exists, determine that the second unmatched point cloud object and the second colliding point cloud object are the same object.

[0133] The above-mentioned method for updating the semantic map based on the second unmatched point cloud object, when it is determined that the second unmatched point cloud object has a bounding box collision, that is, when the second colliding point cloud object corresponding to the vanished point cloud object confirmed in the first match is subjected to convex hull collision, the semantic map is updated by determining whether there is a convex hull collision to confirm whether it is a vanished point cloud object or the same point cloud object; this avoids errors caused by sensor errors or noise interference during data acquisition and improves the accuracy of map updates.

[0134] In one embodiment, such as Figure 9 As shown, a semantic distance determination method is provided, which can be applied to... Figure 1 Taking the terminal in the example, the explanation includes the following steps:

[0135] Step 902: Determine the center position coordinate distance difference value based on the point cloud center coordinates of the first point cloud object in the crowdsourced map and the point cloud center coordinates of the second point cloud object in the semantic map.

[0136] The first point cloud object is constructed based on the target object point cloud extracted from the newly collected crowdsourced map. The first semantic information is the physical geometric information of the point cloud extracted from the first point cloud object, including the point cloud object identifier, point cloud center, point cloud convex hull, point cloud minimum bounding box (OBB), point cloud PCA coordinate system direction, point cloud main direction, and point cloud histogram.

[0137] Specifically, the process involves extracting target object point clouds from the crowdsourced map and constructing a corresponding first point cloud object, as well as extracting the semantic information of the first point cloud object; then, it determines a second point cloud object from an existing semantic map, along with its semantic information. For example, in a local map (crowdsourced acquisition), the center coordinates of the first point cloud object are (x1, y1, z1), the main direction of the point cloud is 'a', the bounding box size is (w1, h1, d1), the shape feature is a 30-dimensional vector 's1', and the original number of point clouds is 'n1'. On the other hand, in the semantic map, the center coordinates of the second point cloud object are (x2, y2, z2), the main direction of the point cloud is 'b', the bounding box size is (w2, h2, d2), the shape feature is a 30-dimensional vector 's2', and the original number of point clouds is 'n2'.

[0138] The difference in center position coordinates between the first point cloud object (x1, y1, z1) and the second point cloud object (x2, y2, z2) is determined by the following method:

[0139] Step 904: Determine the point cloud object orientation difference value based on the main point cloud orientation of the first point cloud object and the main point cloud orientation of the second point cloud object.

[0140] The point cloud object orientation difference value can be expressed as:

[0141] cos(θ) = a·b / (|a|·|b|); the larger the angle, the greater the difference, and the smaller the cos value; the smaller the angle, the smaller the difference, and the larger the cos value.

[0142] distance2 = 1 - cos(θ), the smaller the difference, the smaller the directional distance.

[0143] Step 906: Determine the difference value of the calibration frame size based on the calibration frame size of the first point cloud object and the calibration frame size of the second point cloud object.

[0144] The calibration box size of the first point cloud object is (w1, h1, ...). d 1) The calibration box dimensions of the second point cloud object are (w2, h2, d2); the difference in calibration box dimensions can be expressed as:

[0145]

[0146] Step 908: Determine the appearance feature difference value based on the shape elements of the point cloud histogram of the first point cloud object and the shape elements of the point cloud histogram of the second point cloud object.

[0147] The difference in appearance features can be expressed as:

[0148]

[0149] Step 910: By weighting the center position coordinate distance difference value, point cloud object orientation difference value, calibration box size difference value, and appearance feature difference value, the semantic distance between each first point cloud object and the second point cloud object in the semantic map is obtained.

[0150] Specifically, the weight values ​​w1, w2, w3, and w4 corresponding to the center position coordinate distance difference value (distance1), the point cloud object orientation difference value (distance2), the calibration box size difference value (distance3), and the appearance feature difference value (distance4) are obtained. The semantic distance score is obtained by weighting and summing these weights, where each weight ranges from 0 to 1. In other words, each difference value is weighted and then linearly summed to obtain the final semantic distance; that is, semantic distance = w1*distance1 + w2*distance2 + w3*distance3 + w4*distance4.

[0151] Based on the map matching algorithm, the collected crowdsourced map is matched with the existing semantic map according to the semantic distance between each first point cloud object and the second point cloud object in the semantic map. The matching status between point cloud objects in the crowdsourced map and the existing semantic map is determined. The matching status includes the first point cloud object in the crowdsourced map having a matching second point cloud object in the semantic map, the first point cloud object in the crowdsourced map not having a matching second point cloud object in the semantic map (i.e., the first point cloud object is a newly added point cloud object), and the second point cloud object in the semantic map not having a matching first point cloud object in the crowdsourced map (i.e., the second point cloud object disappears).

[0152] In the above method for determining the semantic distance of point cloud objects, the semantic information of the first point cloud object in the crowdsourced map and the semantic information of the second point cloud object in the existing semantic map are extracted. Based on the geometric information of the objects, the semantic distance between the point cloud objects in the crowdsourced map and the point cloud objects in the semantic map is determined, which provides a guarantee for map matching and updating.

[0153] In one embodiment, such as Figure 10 As shown, a map matching method is provided, which is applied to... Figure 1 Taking the terminal in the example, the explanation includes the following steps:

[0154] Step 1002: Extract the point cloud of the target object from the crowdsourced map corresponding to the current collection area.

[0155] Step 1004: Construct a first point cloud object based on the target object point cloud and extract the corresponding first semantic information.

[0156] Specifically, the extracted target object point cloud is denoised and clustered to construct at least one first point cloud object, and the first semantic information of each first point cloud object is extracted.

[0157] Step 1006: Obtain all second point cloud objects in the semantic map and their corresponding second semantic information.

[0158] Step 1008: Determine the semantic distance between the first point cloud object and each of the second point cloud objects based on the first semantic information and the second semantic information.

[0159] Specifically, the center position coordinate distance difference value is determined based on the point cloud center coordinates of the first point cloud object and the point cloud center coordinates of the second point cloud object; the point cloud object orientation difference value is determined based on the point cloud principal direction of the first point cloud object and the point cloud principal direction of the second point cloud object; the calibration box size difference value is determined based on the calibration box size of the first point cloud object and the second point cloud object; the appearance feature difference value is determined based on the shape elements of the point cloud histogram of the first point cloud object and the point cloud histogram of the second point cloud object; and the semantic distance between each first point cloud object and the second point cloud object in the semantic map is obtained by weighting the center position coordinate distance difference score, the point cloud object orientation difference score, the calibration box size difference score, and the appearance feature difference score.

[0160] Step 1010: Obtain the number of first point cloud objects n in the crowdsourced map and the number of second point cloud objects m in the semantic map, and obtain an n*m correlation matrix.

[0161] Step 1012: Use the semantic distance as an element of the association matrix.

[0162] Understandably, each element in the association matrix represents the semantic distance between the first point cloud object in the crowdsourced map and the second point cloud object in the semantic map. For example, if there are 3 first point cloud objects in the crowdsourced map and 3 second point cloud objects in the semantic map, forming a 3*3 association matrix, each element of the association matrix represents the semantic distance between the first point cloud object and the second point cloud object in the semantic map.

[0163] Step 1014: Based on the predetermined threshold and the association matrix, obtain the matching result between the first point cloud object in the crowdsourced map and the second point cloud object in the semantic map.

[0164] If the semantic distance is less than a predetermined threshold, it indicates that there is a correlation between the corresponding first point cloud object and the second point cloud object.

[0165] Specifically, semantic distance is used as an element of the association matrix. When there is a first element in the association matrix that is greater than or equal to a predetermined threshold, it is determined that the first point cloud object and the second point cloud object corresponding to the first element are not a point cloud object matching pair. The first point cloud object and the second point cloud object corresponding to the first element not being a point cloud object matching pair includes the first point cloud object being a newly added point cloud object or the second point cloud object being a disappeared point cloud object.

[0166] When at least one second element in the association matrix is ​​less than a predetermined threshold, it is determined that the first point cloud object corresponding to the second element has at least one matching second point cloud object. The association matrix is ​​then segmented based on the second element to obtain several sub-graphs. Bipartite graph matching is performed on each sub-graph to determine the second point cloud object that matches the first point cloud object. In other words, bipartite graph matching is performed on each sub-graph to determine the matching cost between the first point cloud object and each second point cloud object in the sub-graph. The second point cloud object corresponding to the smallest matching cost is determined as the matching point cloud object of the first point cloud object. The bipartite graph matching is implemented using the Hungarian algorithm, which obtains the object connection pair (object, crowd_object) with the minimum cost.

[0167] For example, the crowdsourced map includes first point cloud objects 1, 2, and 3, and the existing semantic map contains second point cloud objects 4, 5, and 10. Among them, the semantic distance between first point cloud object 2 and second point cloud object 4 and first point cloud object 2 and second point cloud object 5 is greater than a predetermined value, so they are mismatched point cloud objects. The semantic distance between first point cloud object 1 and second point cloud object 4, and first point cloud object 1 and second point cloud object 5 is less than a predetermined threshold. Therefore, it is necessary to divide the semantic distance of first point cloud object 1 and second point cloud object 4, and first point cloud object 1 and second point cloud object 5 into a sub-graph. The existing Hungarian matching algorithm is used for first point cloud object 1 and second point cloud object 4, and first point cloud object 1 and second point cloud object 5 to obtain the matching cost. The one with the smallest cost is determined as the final matched point cloud object of first point cloud object 1.

[0168] In the above map matching method, target object point clouds are extracted from crowdsourced data. Semantic information is further extracted from the extracted object point clouds, compressing the data size while obtaining more information. This makes the overall map data size extremely small while containing all the necessary information. Here, a matching algorithm is added to the map update, instead of simply judging whether they are the same object based on semantic distance. In the local map, when the number of objects exceeds a certain threshold, the semantic distance-based matching algorithm will make the map update more accurate.

[0169] In another embodiment, such as Figure 11 As shown, a method for updating a crowdsourced semantic map based on 3D point clouds is provided, which is then applied to... Figure 1 Taking the terminal in the example, the explanation includes the following steps:

[0170] Step 1102: Match the newly collected crowdsourced map and the existing semantic map in the same collection area to obtain the first set of unmatched point cloud objects and the second set of unmatched point cloud objects.

[0171] Step 1104: Determine whether there is bounding box collision between the point cloud objects in the first unmatched point cloud object set and the second unmatched point cloud object set; if not, proceed to step 1106; if yes, proceed to step 1108.

[0172] Specifically, it is determined whether there is a bounding box collision between each of the first unmatched point cloud objects in the first unmatched point cloud object set and the corresponding point cloud objects within a preset semantic distance range in the semantic map; and / or, it is determined whether there is a bounding box collision between each of the second unmatched point cloud objects in the second unmatched point cloud object set and the corresponding point cloud objects within a preset semantic distance range in the crowdsourcing map.

[0173] Step 1106: Delete the second unmatched point cloud object from the semantic map, and / or add the first unmatched point cloud object to the semantic map.

[0174] Step 1108: If the bounding box collision result is that the first unmatched point cloud object has a bounding box collision, proceed to step 1110; if the bounding box collision result is that the second unmatched point cloud object has a bounding box collision, proceed to step 1122.

[0175] Step 1110: Determine the corresponding first collision point cloud object from the semantic map within a preset distance range.

[0176] Step 1112: Determine whether there is a convex hull collision between the first unmatched point cloud object and the first colliding point cloud object; if yes, proceed to step 1114; otherwise, proceed to step 1118.

[0177] Step 1114: Obtain the point cloud proportion based on the first unmatched point cloud object and the first colliding point cloud object.

[0178] Step 1116: Update the semantic map based on the point cloud proportion and the set threshold.

[0179] Step 1118: When there is no convex hull collision, determine that the first unmatched point cloud object and the first colliding point cloud object are not the same object.

[0180] Step 1120: Add the first unmatched point cloud object to the semantic map.

[0181] Step 1122: Determine the corresponding second collision point cloud object from the preset distance range in the crowdsourced map.

[0182] Step 1124: Determine whether there is a convex hull collision between the second unmatched point cloud object and the second colliding point cloud object. If yes, proceed to step 1130; otherwise, proceed to step 1126.

[0183] Step 1126: When there is no convex hull collision, determine that the second unmatched point cloud object and the second colliding point cloud object are not the same object.

[0184] Step 1128: Remove the second unmatched point cloud object from the semantic map.

[0185] Step 1130: When a convex hull collision exists, determine that the second unmatched point cloud object and the second colliding point cloud object are the same object.

[0186] Optionally, in one embodiment, a newly collected crowdsourced map and an existing semantic map of the same collection area are matched to obtain a first set of unmatched point cloud objects; it is determined whether each of the first unmatched point cloud objects in the first set of unmatched point cloud objects has a bounding box collision with the corresponding point cloud objects within a preset semantic distance range in the semantic map; when the bounding box collision result of the first unmatched point cloud objects is that the second unmatched point cloud objects do not have a bounding box collision, the second unmatched point cloud objects are deleted from the semantic map.

[0187] When the bounding box collision result of the first unmatched point cloud object is that the first unmatched point cloud object has a bounding box collision, the corresponding first colliding point cloud object is determined from the semantic map within a preset distance range; the point cloud proportion is obtained based on the first unmatched point cloud object and the first colliding point cloud object; the semantic map is updated based on the point cloud proportion and the set threshold.

[0188] Furthermore, when the bounding box collision result of the first unmatched point cloud object indicates that the first unmatched point cloud object has a bounding box collision, it is determined whether the first unmatched point cloud object and the first colliding point cloud object have a convex hull collision; when a convex hull collision exists, the corresponding first colliding point cloud object is determined from a preset distance range in the semantic map; the point cloud proportion is obtained based on the first unmatched point cloud object and the first colliding point cloud object; the semantic map is updated based on the point cloud proportion and a set threshold; when no convex hull collision exists, it is determined that the first unmatched point cloud object and the first colliding point cloud object are not the same object; the first unmatched point cloud object is added to the semantic map.

[0189] Optionally, in one embodiment, the newly collected crowdsourced map and the existing semantic map in the same collection area are matched to obtain a second set of unmatched point cloud objects; it is determined whether each of the second unmatched point cloud objects in the second set of unmatched point cloud objects has a bounding box collision with the corresponding point cloud objects within a preset semantic distance range in the crowdsourced map; when the bounding box collision result of the second unmatched point cloud objects is that the first unmatched point cloud objects do not have a bounding box collision, the first unmatched point cloud objects are added to the semantic map.

[0190] When the bounding box collision result of the second unmatched point cloud object is that the second unmatched point cloud object has a bounding box collision, the corresponding second colliding point cloud object is determined from the crowdsourced map within a preset distance range; it is determined whether the second unmatched point cloud object and the second colliding point cloud object have a convex hull collision; when there is no convex hull collision, it is determined that the second unmatched point cloud object and the second colliding point cloud object are not the same object; the second unmatched point cloud object is deleted from the semantic map; when there is a convex hull collision, it is determined that the second unmatched point cloud object and the second colliding point cloud object are the same object.

[0191] Optionally, in one embodiment, a newly collected crowdsourced map and an existing semantic map of the same collection area are matched to obtain a first set of unmatched point cloud objects and a second set of unmatched point cloud objects; it is determined whether each of the first unmatched point cloud objects in the first set of unmatched point cloud objects has a bounding box collision with the corresponding point cloud objects within a preset semantic distance range in the semantic map; it is determined whether each of the second unmatched point cloud objects in the second set of unmatched point cloud objects has a bounding box collision with the corresponding point cloud objects within a preset semantic distance range in the crowdsourced map; when the bounding box collision result of the first unmatched point cloud object is that the second unmatched point cloud object does not have a bounding box collision, the second unmatched point cloud object is deleted from the semantic map; and when the bounding box collision result of the first unmatched point cloud object is that the second unmatched point cloud object does not have a bounding box collision, the second unmatched point cloud object is deleted from the semantic map.

[0192] When the bounding box collision result of the first unmatched point cloud object indicates that the first unmatched point cloud object has a bounding box collision, it is determined whether the first unmatched point cloud object and the first colliding point cloud object have a convex hull collision. If a convex hull collision exists, the corresponding first colliding point cloud object is determined from a preset distance range in the semantic map. The point cloud proportion is obtained based on the first unmatched point cloud object and the first colliding point cloud object. The semantic map is updated based on the point cloud proportion and a set threshold. If no convex hull collision exists, it is determined that the first unmatched point cloud object and the first colliding point cloud object are not the same object. The first unmatched point cloud object is added to the semantic map.

[0193] When the bounding box collision result of the second unmatched point cloud object is that the second unmatched point cloud object has a bounding box collision, the corresponding second colliding point cloud object is determined from the crowdsourced map within a preset distance range; it is determined whether the second unmatched point cloud object and the second colliding point cloud object have a convex hull collision; when there is no convex hull collision, it is determined that the second unmatched point cloud object and the second colliding point cloud object are not the same object; the second unmatched point cloud object is deleted from the semantic map; when there is a convex hull collision, it is determined that the second unmatched point cloud object and the second colliding point cloud object are the same object.

[0194] In the aforementioned update method for crowdsourced semantic maps based on 3D point clouds, the newly acquired crowdsourced map and the existing semantic map of the same acquisition area are matched. When bounding box collisions exist, convex hull collision and point cloud detection are performed on the first unmatched point cloud object and the corresponding first colliding point cloud object; and convex hull collision is performed on the second unmatched point cloud object and the corresponding second colliding point cloud object. By determining whether convex hull collisions exist, it is confirmed whether it is a missing point cloud object or the same point cloud object. This avoids mismatches caused by occlusion or sensor performance issues. For unmatched point cloud objects, a second matching detection is performed to improve the map update accuracy.

[0195] It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.

[0196] Based on the same inventive concept, this application also provides an apparatus for updating a crowdsourced semantic map based on 3D point clouds, which implements the above-described method for updating a crowdsourced semantic map based on 3D point clouds. The solution provided by this apparatus is similar to the solution described in the above-described method. Therefore, the specific limitations of one or more embodiments of the apparatus for updating a crowdsourced semantic map based on 3D point clouds provided below can be found in the limitations of the method for updating a crowdsourced semantic map based on 3D point clouds described above, and will not be repeated here.

[0197] In one embodiment, such as Figure 12 As shown, an update device for a crowdsourced semantic map based on 3D point clouds is provided, including: a point cloud object matching module 1202, a point cloud object collision module 1204, and a map update module 1206, wherein:

[0198] The point cloud object matching module 1202 is used to match newly collected crowdsourced maps and existing semantic maps in the same collection area to obtain a first set of unmatched point cloud objects and a second set of unmatched point cloud objects.

[0199] The point cloud object collision module 1204 is used to determine whether there is a bounding box collision between each unmatched point cloud object in the first unmatched point cloud object set and the corresponding point cloud object within a preset distance range in the semantic map.

[0200] Determine whether each unmatched point cloud object in the second unmatched point cloud object set has a bounding box collision with the corresponding point cloud object within a preset distance range in the crowdsourced map, and obtain the bounding box collision result.

[0201] The map update module 1206 is used to update the semantic map based on the bounding box collision results of the first unmatched point cloud object and / or the bounding box collision results of the second unmatched point cloud object.

[0202] The aforementioned update device based on a crowdsourced semantic map of 3D point clouds matches newly collected crowdsourced maps and existing semantic maps in the same collection area to obtain a first set of unmatched point cloud objects and a second set of unmatched point cloud objects. Bounding box collision is performed on the point cloud objects in the first set of unmatched point cloud objects and the point cloud objects in the second set of unmatched point cloud objects. For point cloud objects that do not exist in the first matching, a second matching detection is performed to improve the accuracy of map updates.

[0203] Optionally, in one embodiment, the map update module 1206 is further configured to delete the second unmatched point cloud object from the semantic map when the bounding box collision result of the second unmatched point cloud object is that the second unmatched point cloud object does not have a bounding box collision.

[0204] Optionally, in one embodiment, the map update module 1206 is further configured to add the first unmatched point cloud object to the semantic map when the bounding box collision result of the first unmatched point cloud object is that the first unmatched point cloud object does not have a bounding box collision.

[0205] Optionally, in one embodiment, the map update module 1206 is further configured to add the first unmatched point cloud object to the semantic map when the bounding box collision result of the first unmatched point cloud object is that the first unmatched point cloud object does not have a bounding box collision; and

[0206] When the bounding box collision result of the second unmatched point cloud object is that the second unmatched point cloud object does not have a bounding box collision, the second unmatched point cloud object is deleted from the semantic map.

[0207] Optionally, in one embodiment, the device further includes a point cloud determination module, configured to determine the corresponding first colliding point cloud object from a preset distance range in the semantic map when the bounding box collision result of the first unmatched point cloud object is that the first unmatched point cloud object has a bounding box collision; and to obtain the point cloud proportion based on the first unmatched point cloud object and the first colliding point cloud object.

[0208] Optionally, in one embodiment, the map update module 1206 is further configured to update the semantic map based on the point cloud proportion and a set threshold.

[0209] The point cloud determination module is also used to determine the first object point cloud corresponding to the first unmatched point cloud object, and the first point cloud object convex hull corresponding to the first colliding point cloud object; determine the number of points in the first object point cloud in the first point cloud object convex hull, and obtain the point cloud proportion.

[0210] Optionally, in one embodiment, the map update module 1206 is further configured to determine that the first unmatched point cloud object and the first colliding point cloud object are the same object when the point cloud proportion is greater than or equal to a set threshold.

[0211] Optionally, in one embodiment, the map update module 1206 is further configured to determine that the first unmatched point cloud object and the first colliding point cloud object are not the same object when the point cloud proportion is less than a set threshold; and add the first unmatched point cloud object to the semantic map.

[0212] Optionally, in one embodiment, the device further includes a convex hull collision module for determining whether there is a convex hull collision between the first unmatched point cloud object and the first colliding point cloud object.

[0213] Optionally, in one embodiment, the map update module 1206 is further configured to determine that the first unmatched point cloud object and the first colliding point cloud object are not the same object when there is no convex hull collision; and add the first unmatched point cloud object to the semantic map.

[0214] The convex hull collision module is also used to determine the corresponding second collision point cloud object from a preset distance range in the crowdsourced map when the bounding box collision result of the second unmatched point cloud object is that the second unmatched point cloud object has a bounding box collision; and to determine whether the second unmatched point cloud object and the second collision point cloud object have a convex hull collision.

[0215] Optionally, in one embodiment, the map update module 1206 is further configured to determine that the second unmatched point cloud object and the second colliding point cloud object are not the same object when there is no convex hull collision; and delete the second unmatched point cloud object from the semantic map.

[0216] Optionally, in one embodiment, the map update module 1206 is further configured to determine that the second unmatched point cloud object and the second colliding point cloud object are the same object when a convex hull collision occurs.

[0217] The modules in the aforementioned 3D point cloud-based crowdsourced semantic map update device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device, or stored in the memory of a computer device as software, so that the processor can call and execute the corresponding operations of each module.

[0218] In one embodiment, a computer device is provided, which may be a terminal, and its internal structure diagram may be as follows: Figure 13 As shown, the computer device includes a processor, memory, communication interface, display screen, and input devices connected via a system bus. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The communication interface is used for wired or wireless communication with external terminals; wireless communication can be achieved through Wi-Fi, mobile cellular networks, NFC (Near Field Communication), or other technologies. When executed by the processor, the computer program implements an update method for a crowdsourced semantic map based on 3D point clouds. The display screen can be an LCD screen or an e-ink display screen. The input devices can be a touch layer covering the display screen, buttons, a trackball, or a touchpad mounted on the computer device casing, or an external keyboard, touchpad, or mouse.

[0219] Those skilled in the art will understand that Figure 13 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0220] In one embodiment, a computer device is also provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps in the above method embodiments.

[0221] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon that, when executed by a processor, implements the steps in the above method embodiments.

[0222] In one embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps in the above method embodiments.

[0223] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties.

[0224] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc., and are not limited to these.

[0225] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0226] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A method for updating a crowdsourced semantic map based on 3D point clouds, characterized in that, The method includes: Match newly collected crowdsourced maps and existing semantic maps in the same collection area to obtain the first set of unmatched point cloud objects and the second set of unmatched point cloud objects. Determine whether there is a bounding box collision between each unmatched point cloud object in the first unmatched point cloud object set and the corresponding point cloud object within a preset semantic distance range in the semantic map; Determine whether there is a bounding box collision between each unmatched point cloud object in the second unmatched point cloud object set and the corresponding point cloud object within a preset semantic distance range in the crowdsourced map; The semantic map is updated based on the bounding box collision result of the first unmatched point cloud object and / or the bounding box collision result of the second unmatched point cloud object. Based on the bounding box collision results of the first unmatched point cloud object, the semantic map is updated, including: When the bounding box collision result of the first unmatched point cloud object is that the first unmatched point cloud object has a bounding box collision, the corresponding first colliding point cloud object is determined from the semantic map within a preset semantic distance range. The point cloud proportion is obtained based on the first unmatched point cloud object and the first colliding point cloud object; The semantic map is updated based on the point cloud proportion and a set threshold. The semantic map is updated based on the bounding box collision results of the second unmatched point cloud object, including: When the bounding box collision result of the second unmatched point cloud object is that the second unmatched point cloud object has a bounding box collision, the corresponding second collision point cloud object is determined from the preset semantic distance range in the crowdsourced map. Determine whether there is a convex hull collision between the second unmatched point cloud object and the second colliding point cloud object; When there is no convex hull collision, it is determined that the second unmatched point cloud object and the second colliding point cloud object are not the same object; Remove the second unmatched point cloud object from the semantic map.

2. The method according to claim 1, characterized in that, The semantic map is updated based on the bounding box collision results of the second unmatched point cloud object, including: When the bounding box collision result of the second unmatched point cloud object is that the second unmatched point cloud object does not have a bounding box collision, the second unmatched point cloud object is deleted from the semantic map.

3. The method according to claim 1, characterized in that, Based on the bounding box collision results of the first unmatched point cloud object, the semantic map is updated, including: When the bounding box collision result of the first unmatched point cloud object is that the first unmatched point cloud object does not have a bounding box collision, the first unmatched point cloud object is added to the semantic map.

4. The method according to claim 1, characterized in that, Based on the bounding box collision results of the first unmatched point cloud object and the second unmatched point cloud object, including: When the bounding box collision result of the first unmatched point cloud object is that the first unmatched point cloud object does not have a bounding box collision, the first unmatched point cloud object is added to the semantic map; and When the bounding box collision result of the second unmatched point cloud object is that the second unmatched point cloud object does not have a bounding box collision, the second unmatched point cloud object is deleted from the semantic map.

5. The method according to claim 1, characterized in that, The step of obtaining the point cloud proportion based on the first unmatched point cloud object and the first colliding point cloud object includes: Determine the first object point cloud corresponding to the first unmatched point cloud object, and the first point cloud object convex hull corresponding to the first colliding point cloud object; Determine the number of points in the first object's point cloud that lie within the convex hull of the first point cloud object, and obtain the point cloud percentage.

6. The method according to claim 5, characterized in that, The step of updating the semantic map based on the point cloud proportion and a set threshold includes: When the proportion of the point cloud is greater than or equal to the set threshold, the first unmatched point cloud object and the first colliding point cloud object are determined to be the same object.

7. The method according to claim 1, characterized in that, The step of updating the semantic map based on the point cloud proportion and a set threshold includes: When the proportion of the point cloud is less than the set threshold, it is determined that the first unmatched point cloud object and the first colliding point cloud object are not the same object. Add the first unmatched point cloud object to the semantic map.

8. The method according to claim 1, characterized in that, Before obtaining the point cloud proportion based on the first unmatched point cloud object and the first colliding point cloud object, the method further includes: Determine whether there is a convex hull collision between the first unmatched point cloud object and the first colliding point cloud object; When a convex hull collision occurs, the step of obtaining the point cloud proportion based on the first unmatched point cloud object and the first colliding point cloud object is executed.

9. The method according to claim 8, characterized in that, The method further includes: When there is no convex hull collision, it is determined that the first unmatched point cloud object and the first colliding point cloud object are not the same object; Add the first unmatched point cloud object to the semantic map.

10. The method according to claim 1, characterized in that, The method further includes: When a convex hull collision occurs, the second unmatched point cloud object and the second colliding point cloud object are determined to be the same object.

11. An update device for a crowdsourced semantic map based on 3D point clouds, characterized in that, The device includes: The point cloud object matching module is used to match newly collected crowdsourced maps and existing semantic maps in the same collection area to obtain a first set of unmatched point cloud objects and a second set of unmatched point cloud objects. The point cloud object collision module is used to determine whether there is a bounding box collision between each unmatched point cloud object in the first unmatched point cloud object set and the corresponding point cloud object within a preset semantic distance range in the semantic map. Determine whether each unmatched point cloud object in the second unmatched point cloud object set has a bounding box collision with the corresponding point cloud object within a preset semantic distance range in the crowdsourced map, and obtain the bounding box collision result. The map update module is used to update the semantic map based on the bounding box collision result of the first unmatched point cloud object and / or the bounding box collision result of the second unmatched point cloud object. The map update module is further configured to determine the corresponding first colliding point cloud object from the semantic map within a preset semantic distance range when the bounding box collision result of the first unmatched point cloud object is that the first unmatched point cloud object has a bounding box collision. The point cloud proportion is obtained based on the first unmatched point cloud object and the first colliding point cloud object; The semantic map is updated based on the point cloud proportion and a set threshold. The map update module is further configured to determine the corresponding second collision point cloud object from the preset semantic distance range in the crowdsourced map when the bounding box collision result of the second unmatched point cloud object is that the second unmatched point cloud object has a bounding box collision. Determine whether there is a convex hull collision between the second unmatched point cloud object and the second colliding point cloud object; When there is no convex hull collision, it is determined that the second unmatched point cloud object and the second colliding point cloud object are not the same object; Remove the second unmatched point cloud object from the semantic map.

12. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 10.

Images