A semantic map incremental update method and system

Abstract

The present invention belongs to the field of map updating technology, and more specifically, relates to a method and system for incrementally updating semantic maps. The method specifically includes: constructing a current local semantic map; aligning the current local semantic map with a previous global semantic map and converting them to the same coordinate system; detecting areas of semantic inconsistency between the point clouds corresponding to the current local semantic map and the previous global semantic map to obtain semantic change regions; calculating and comparing the point cloud distribution categories of the previous global semantic map and the current local semantic map corresponding to the semantic change regions to determine environmental change regions; extracting environmental change regions and updating them to the previous global semantic map to obtain the current global semantic map. In this solution, semantic comparison is used to initially detect change regions, and then point cloud distribution category comparison is used to confirm whether the changed regions are environmental change regions, thereby improving the accuracy of environmental change detection and making semantic map incremental updates more comprehensive.

Description

Technical Field

[0001] The present invention belongs to the technical field of map updating, and more specifically, relates to a semantic map incremental updating method and system. Background Art

[0002] A semantic map is a high-precision semantic map that contains rich semantic information, allowing the autonomous driving system to fully perceive the surrounding environment information and make appropriate driving actions.

[0003] Due to the sheer volume of semantic map data and the limitations of current hardware, the speed of semantic map construction still cannot achieve the real-time performance required by autonomous vehicles. Therefore, semantic maps currently need to be built and stored in advance, and then the autonomous driving system can use them in real time to control the vehicle. This ensures the real-time nature of the autonomous driving system's control decisions. However, the environment can change between the time the semantic map is built and the time it is used by the autonomous driving system. Directly using the previous semantic map would seriously impact driving safety, necessitating an update of the semantic map to accommodate changing circumstances.

[0004] Traditionally, there are two main ways to update maps. The first is a full update, which updates the entire map, but this is computationally intensive and slow. The second is an incremental update, which updates only the changed areas. This method generally determines the changed areas and updates them by matching the features of the point cloud data. Compared with a full update, this method requires less computation and has a faster update speed. However, given the huge amount of semantic information, the computational complexity of point cloud feature matching is still too high, so it is not suitable for incremental updates of semantic maps.

[0005] In the existing technology, a semantic map incremental update method based on feature point detection and segmentation detects environmental changes by comparing the semantic information of point clouds and incrementally updates the semantic map. However, this method relies too much on semantic segmentation, which easily leads to the inability to accurately detect the changed areas, resulting in incomplete incremental updates of the semantic map. Moreover, the detection speed and accuracy of the map change areas in the existing technical solutions are both low. Therefore, how to achieve fast and accurate detection of semantic maps and improve the effectiveness of semantic map incremental updates is a current technical difficulty. Summary of the Invention

[0006] In order to overcome at least one of the above-mentioned defects in the prior art, the present invention provides a semantic map incremental update method and system, which can improve the speed of semantic map update and ensure complete update.

[0007] In order to solve the above technical problems, the technical solution adopted by the present invention is:

[0008] A method for incrementally updating a semantic map is provided, comprising the following steps:

[0009] S1: Obtain the previous global semantic map and construct the current local semantic map;

[0010] S2: align the current local semantic map with the previous global semantic map and transform them into the same coordinate system;

[0011] S3: Detect the semantically inconsistent area between the current local semantic map and the point cloud corresponding to the previous global semantic map, and obtain the semantic change area;

[0012] S4: Calculate the point cloud distribution categories of the previous global semantic map and the current local semantic map corresponding to the semantic change area and compare them to determine the environment change area;

[0013] S5: Extract the environment change area as an increment and update it to the previous global semantic map to obtain the current global semantic map.

[0014] In this solution, the changed areas are preliminarily detected through semantic comparison, and then the point cloud distribution categories are compared to further confirm whether the changed areas are environmental change areas, thereby improving the accuracy of environmental change detection and ensuring the effect of incremental updates of semantic maps.

[0015] Preferably, the construction of the current local semantic map in the above step S1 is specifically as follows:

[0016] S11: Collect information about the current environment and build a point cloud map;

[0017] S12: Assign semantic information to the point cloud map to obtain a local semantic map of the environment;

[0018] S13: Detect and eliminate temporary targets in the local semantic map of the environment to obtain the current local semantic map.

[0019] Preferably, in the above step S2, the current local semantic map is aligned with the previous global semantic map through SAC-IA coarse alignment and ICP fine alignment.

[0020] Preferably, the above-mentioned step S3 detects the area where the semantics of the current local semantic map and the corresponding point cloud of the previous global semantic map are inconsistent, specifically to determine whether there are points in a certain semantic area that have undergone semantic changes exceeding the first set proportion of the current local semantic map. If so, the semantic area is a semantic change area; if not, the semantic area is a non-semantic change area.

[0021] Preferably, the global semantic map in step S1 is voxelized data, and step S4 specifically includes the following steps:

[0022] S41: voxelizing the current local semantic map area corresponding to the semantic change area;

[0023] S42: Calculate the probability density functions of the point clouds in the corresponding voxel cells in the current local semantic map and the global semantic map respectively, and obtain the point cloud distribution categories according to the probability density functions. The point cloud distribution categories include spherical, planar, and linear.

[0024] S43: Confirming the environmental change area according to the point cloud distribution categories within the voxel cells in the current local semantic map and the global semantic map.

[0025] Preferably, in the above step S41, the probability density function is calculated by a three-dimensional normal distribution transformation algorithm. , the specific formula is:

[0026] ,

[0027] ,

[0028] ,

[0029] Where x represents the coordinate within the voxel cell, q is the mean vector of the voxel cell, is the covariance matrix of the voxel unit, T is the transpose, are the coordinates of the points within the voxel cell, k is the number of points within the voxel cell and k=1,...,m.

[0030] Preferably, in the above step S43, the environmental change area is determined according to the point cloud distribution category as follows:

[0031] Determine whether the point cloud distribution category within the voxel cells of the current local semantic map is the same as that of the previous global semantic map; if not, the semantic change area is considered to be an environmental change area; if so, determine whether the point cloud distribution category is spherical. If so, the semantic change area is considered to be a non-environmental change area. If not, calculate the eigenvector of the covariance matrix of the voxel cell. If the direction of the eigenvector is different, it is considered to be an environmental change area, otherwise it is considered to be a non-environmental change area.

[0032] Preferably, the point cloud distribution category obtained according to the probability density function in the above step S42 is specifically:

[0033] Let the three eigenvalues of the covariance matrix be ,when , then the point cloud distribution category is considered to be spherical; when , then the point cloud distribution category is considered to be plane; when , then the point cloud distribution category is considered to be linear.

[0034] Preferably, the above step S5 specifically includes:

[0035] S51: Refine the boundary of the environment change area to obtain the increment;

[0036] S52: deleting the region in the previous global semantic map according to the incremental boundary;

[0037] S53: Update the increment to the previous global semantic map to obtain the current global semantic map.

[0038] This solution also provides a semantic map incremental update system, which stores a voxelized previous global semantic map and includes a mapping module, a registration module, a detection module, and an update module that are sequentially connected in communication;

[0039] The mapping module is used to construct the current local semantic map and input it into the registration module;

[0040] The registration module is used to register the current local semantic map with the previous global semantic map and convert them into the same coordinate system, and input the registered and coordinate-converted data into the detection module;

[0041] The detection module is used to detect the semantic inconsistency area of the point cloud corresponding to the current local semantic map and the previous global semantic map, obtain the semantic change area, determine the environment change area based on the semantic change area, and input the environment change area into the update module;

[0042] The update module is used to extract the environment change area as increments and update it to the previous global semantic map to obtain a global semantic map.

[0043] Compared with the prior art, the beneficial effects are:

[0044] On the one hand, the present invention detects environmental changes through a combination of semantic comparison and three-dimensional normal distribution transformation algorithm, and the obtained environmental change area is more accurate; on the other hand, the present invention only voxelizes the area of semantic change and then calculates the probability density distribution function within the voxel unit cell, which can effectively reduce the amount of calculation and thus improve the overall update speed. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] Figure 1 This is a schematic flow chart of a method for incrementally updating a semantic map according to embodiment 1 of the present invention;

[0046] Figure 2 This is a schematic flow chart of step S4 of the semantic map incremental update method according to embodiment 1 of the present invention;

[0047] Figure 3 This is a structural block diagram of a semantic map incremental update system according to embodiment 3 of the present invention;

[0048] Figure 4 This is a schematic block diagram of the application of the semantic map incremental update system of Example 3 of the present invention in Example 1 or Example 2. DETAILED DESCRIPTION

[0049] The accompanying drawings are for illustrative purposes only and are not to be construed as limiting the present invention. To better illustrate the present embodiment, some components in the accompanying drawings may be omitted, enlarged, or reduced in size, and do not represent actual product dimensions. Those skilled in the art will appreciate that some well-known structures and their descriptions may be omitted from the accompanying drawings. The positional relationships depicted in the accompanying drawings are for illustrative purposes only and are not to be construed as limiting the present invention.

[0050] The same or similar numbers in the drawings of the embodiments of the present invention correspond to the same or similar parts; in the description of the present invention, it should be understood that if there are terms such as "upper", "lower", "left", "right", "long", "short", etc. indicating the orientation or position relationship, they are based on the orientation or position relationship shown in the drawings. They are only for the convenience of describing the present invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operate in a specific orientation. Therefore, the terms describing the position relationship in the drawings are only used for illustrative purposes and cannot be understood as limiting this patent. For ordinary technicians in this field, the specific meanings of the above terms can be understood according to specific circumstances.

[0051] The technical solution of the present invention is further described in detail below through specific embodiments and in conjunction with the accompanying drawings:

[0052] Example 1:

[0053] like Figure 1 、 Figure 2 A first embodiment of a semantic map incremental update method is shown, comprising the following steps:

[0054] S1: Obtain the previous global semantic map and construct the current local semantic map;

[0055] S2: align the current local semantic map with the previous global semantic map and convert them to the same coordinate system to find the specific location of the current environment in the previous global semantic map;

[0056] S3: Detect the semantically inconsistent area between the current local semantic map and the point cloud corresponding to the previous global semantic map, and obtain the semantic change area;

[0057] S4: Calculate the point cloud distribution categories of the previous global semantic map and the current local semantic map corresponding to the semantic change area and compare them to determine the environment change area;

[0058] S5: Extract the environmental change area and update it to the previous global semantic map to obtain the current global semantic map.

[0059] In step S1 of this embodiment, the current local semantic map is constructed as follows:

[0060] S11: Collect information about the current environment and construct a point cloud map. Specifically, a binocular vision camera may be used to collect a sequence of images of the current environment, and the sequence of images may be used to construct a point cloud map through SLAM. SLAM is well known to those skilled in the art and will not be described in detail here.

[0061] S12: Assign semantic information to the point cloud map to obtain a local semantic map of the environment;

[0062] S13: Detect and remove temporary targets in the local semantic map of the environment to obtain the current local semantic map; wherein, temporary targets are objects such as pedestrians and vehicles whose positions and postures will change in a short period of time. Generally, they have left their positions when the semantic map is used. The semantics corresponding to the objects to be removed can be set first, and then when the corresponding semantics appear in the local semantic map of the environment, the objects corresponding to the corresponding semantics are deleted. This can avoid updating the temporary targets when the semantic map is incrementally updated.

[0063] In step S2 of this embodiment, the current local semantic map is aligned with the previous global semantic map through SAC-IA coarse registration and ICP fine registration. The initial value for the ICP fine registration iteration is provided by SAC-IA coarse registration. Of course, the registration method in this embodiment should not be construed as limiting this solution. Those skilled in the art can, based on this inspiration, adopt other registration methods to perform point cloud registration between the current local semantic map and the previous global semantic map.

[0064] In this embodiment, step S3 detects the semantic inconsistency between the current local semantic map and the corresponding point cloud of the previous global semantic map, specifically to determine whether there is a semantic change in a certain semantic area with more than the first set ratio of the points in the current local semantic map. If so, the semantic area is a semantic change area, and if not, the semantic area is a non-semantic change area. Before the start of step S3, the first set ratio is preset to 0.01%. This is because when semantic information is assigned to the point cloud map, the boundary of the semantic area may deviate. At this time, the point cloud on the boundary will be considered as a change area. Therefore, a certain deviation ratio is set. When the number of point cloud changes is very small, the change of the entire area can be ignored. It is worth noting that the first set ratio of 0.01% in this embodiment is only a reference implementation method and cannot be understood as a limitation of this solution. Of course, it can be increased or decreased according to actual conditions during the specific implementation process.

[0065] To maximize the speed of incremental map updates, the previous global semantic map in step S1 of this embodiment is voxelized. This facilitates the subsequent calculation of the probability density function. This should not be construed as a limitation of this solution; in specific implementations, the previous global semantic map can also be voxelized when calculating point cloud distribution categories.

[0066] Step S4 specifically includes the following steps:

[0067] S41: voxelizing the current local semantic map area corresponding to the semantic change area;

[0068] S42: Calculate the probability density functions of the point clouds in the corresponding voxel cells in the current local semantic map and the global semantic map respectively, and obtain the point cloud distribution categories according to the probability density functions. The point cloud distribution categories include spherical, planar, and linear.

[0069] S43: Confirming the environmental change area according to the point cloud distribution categories within the voxel cells in the current local semantic map and the global semantic map.

[0070] In this way, by performing a semantic comparison between the current local semantic map and the previous global semantic map in step S3, a rough semantic change area is detected, and then the semantic change area is further determined. In this way, only the semantic change area is processed and calculated, which can reduce the amount of calculation while improving the detection effect and efficiency.

[0071] In step S41 of this embodiment, the probability density function is calculated by a three-dimensional normal distribution transformation algorithm (3D-NDT) ,Since the three-dimensional point cloud data of the voxelized current local semantic map and the previous global semantic map are evenly divided into voxel cells of regular size, the mean vector and covariance matrix of the point cloud in each voxel cell can be calculated first, as follows:

[0072] ,

[0073] ,

[0074] Then calculate the probability density function at coordinate x through the mean vector and covariance matrix. The specific formula is:

[0075] ,

[0076] ,

[0077] ,

[0078] Among them, among them, is the coordinate of the midpoint of the voxel cell, x represents one of the coordinates of the point in the voxel cell, q is the average vector of the voxel cell, is the covariance matrix of the voxel unit, T is the transpose, are the coordinates of the points within the voxel unit cell, k is the number of points within the voxel unit cell and k=1,...,m, m represents the maximum number of points within the voxel unit cell; cov is the degree of deviation of the corresponding two dimensions from the covariance mean.

[0079] Since the covariance matrix is a real symmetric matrix, so we can Decomposed into eigenvectors and eigenvalues, the specific formula is:

[0080] ,

[0081] in, is an orthogonal matrix composed of eigenvectors, are the three eigenvectors of the covariance matrix, is a diagonal matrix whose diagonal elements are the three eigenvalues ,in, .

[0082] Of course, if the voxel cell does not contain a point cloud, no corresponding calculation is required to ensure calculation speed.

[0083] In step S43 of this embodiment, the environmental change area is determined based on the point cloud distribution category as follows:

[0084] Determine whether the point cloud distribution category within the voxel cells of the current local semantic map is the same as that of the previous global semantic map; if not, the semantic change area is considered to be an environmental change area; if so, determine whether the point cloud distribution category is spherical. If so, the semantic change area is considered to be a non-environmental change area. If not, calculate the eigenvector of the covariance matrix of the voxel cell. If the direction of the eigenvector is different, it is considered to be an environmental change area, otherwise it is considered to be a non-environmental change area.

[0085] In step S42 of this embodiment, the point cloud distribution category obtained according to the probability density function is specifically:

[0086] when , then the point cloud distribution category is considered to be spherical; when , then the point cloud distribution category is considered to be plane; when , then the point cloud distribution category is considered to be linear.

[0087] Among them, when If the absolute value of is less than or equal to zero, then is approximately equal to zero; if Greater than or equal to 10 , then it is believed that Much greater than ,like The absolute value is less than or equal to 0.01 , then it is believed that and Approximately equal, similarly we can deduce Other numerical relationships between them are not listed here one by one.

[0088] Step S5 in this embodiment specifically includes:

[0089] S51: Refine the boundary of the environment change area to obtain the increment;

[0090] S52: deleting the region in the previous global semantic map according to the incremental boundary;

[0091] S53: Update the increment to the previous global semantic map to obtain the current global semantic map.

[0092] This embodiment successively detects changes in the current local semantic map through semantic comparison and three-dimensional normal distribution transformation algorithms, and the obtained environmental change areas are more accurate, so that subsequent map increments are more complete; on the other hand, the present invention only voxelizes the areas with semantic changes, and then calculates the probability density distribution function within the voxel cells, which can effectively reduce the amount of calculation and thus improve the overall update speed.

[0093] Example 2:

[0094] The only difference between this embodiment and embodiment 1 is that the previous global semantic map in this embodiment is non-voxelized data. In step S41 of this embodiment, the current local semantic map area corresponding to the semantic change area and the previous global semantic map area are voxelized. In this way, the previous global semantic map does not need to be voxelized in its entirety, but only the semantic change area is voxelized. This can greatly reduce the data volume of the previous semantic map and further speed up the calculation speed.

[0095] Example 3:

[0096] like Figure 3 、 Figure 4 The figure shows a semantic map incremental update system, which is used to implement the semantic map incremental update method of embodiment 1 or embodiment 2. The system stores a voxelized previous global semantic map and includes a mapping module, a registration module, a detection module, and an update module that are sequentially connected in communication.

[0097] The mapping module is used to construct the current local semantic map and input it into the registration module;

[0098] The registration module is used to register the current local semantic map with the previous global semantic map and convert them into the same coordinate system, and input the registered and coordinate-converted data into the detection module;

[0099] The detection module is used to detect semantically inconsistent areas of the point cloud corresponding to the current local semantic map and the previous global semantic map, obtain semantic change areas, and determine environmental change areas based on the semantic change areas;

[0100] The update module is used to extract the environmental change area as an increment and update it to the previous global semantic map to obtain the global semantic map, and input the environmental change area into the update module.

[0101] The present invention is described with reference to the flowcharts or block diagrams of the methods, devices (systems), and computer program products of the embodiments of the present application. It should be understood that each process or block in the flowchart or block diagram, as well as the combination of processes or blocks in the flowchart or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart or block diagram. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0102] Although the embodiments of the present invention have been shown and described above, it will be understood that the above embodiments are illustrative and are not to be construed as limitations on the present invention. A person skilled in the art may change, modify, replace and modify the above embodiments within the scope of the present invention.

[0103] Obviously, the above embodiments of the present invention are merely examples for the purpose of clearly illustrating the present invention, and are not intended to limit the embodiments of the present invention. Those skilled in the art will appreciate that other variations or modifications can be made based on the above description. It is not necessary and impossible to enumerate all embodiments here. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention shall be included within the scope of protection of the claims of the present invention.

Claims

1. A semantic map incremental update method, characterized in that: The following steps are involved: S1: Acquire a previous global semantic map and construct a current local semantic map; the previous global semantic map is voxelized data; S2: align the current local semantic map with the previous global semantic map and transform them into the same coordinate system; S3: Detect the semantically inconsistent area between the current local semantic map and the point cloud corresponding to the previous global semantic map, and obtain the semantic change area; S4: Calculate the point cloud distribution categories of the previous global semantic map and the current local semantic map corresponding to the semantic change area and compare them to determine the environment change area; The specific steps include: S41: voxelizing the current local semantic map area corresponding to the semantic change area; S42: Calculate probability density functions of point clouds in corresponding voxel cells in the current local semantic map and the global semantic map respectively, and obtain point cloud distribution categories according to the probability density functions, where the point cloud distribution categories include spherical, planar, and linear. S43: confirming the environment change area according to the point cloud distribution categories in the voxel cells in the current local semantic map and the global semantic map; The environmental change area determined according to the point cloud distribution category in step S43 is specifically: Determine whether the point cloud distribution category within the voxel cells of the current local semantic map is the same as that of the previous global semantic map; if not, consider the semantic change area to be an environmental change area; if so, determine whether the point cloud distribution category is spherical. If so, consider the semantic change area to be a non-environmental change area. If not, calculate the eigenvector of the covariance matrix of the voxel cell. If the direction of the eigenvector is different, it is considered to be an environmental change area, otherwise it is considered to be a non-environmental change area. S5: Extract the environment change area as an increment and update it to the previous global semantic map to obtain the current global semantic map.

2. A semantic map incremental update method according to claim 1, characterized in that: The construction of the current local semantic map in step S1 is specifically as follows: S11: Collect information about the current environment and build a point cloud map; S12: Assign semantic information to the point cloud map to obtain a local semantic map of the environment; S13: Detect and eliminate temporary targets in the local semantic map of the environment to obtain the current local semantic map.

3. A semantic map incremental update method according to claim 2, characterized in that: In step S2, the current local semantic map is aligned with the previous global semantic map through SAC-IA coarse alignment and ICP fine alignment.

4. A semantic map incremental update method according to claim 3, characterized in that: The step S3 described in detecting the semantically inconsistent area between the current local semantic map and the corresponding point cloud of the previous global semantic map is specifically to determine whether there are points in a certain semantic area that have undergone semantic changes exceeding the first set proportion of the current local semantic map. If so, the semantic area is a semantic change area; if not, the semantic area is a non-semantic change area.

5. A semantic map incremental update method according to claim 1, characterized in that: In step S41, the probability density function is calculated by the three-dimensional normal distribution transformation algorithm. , the specific formula is: ， ， ， Where x represents the coordinate within the voxel cell, q is the mean vector of the voxel cell, is the covariance matrix of the voxel unit, T is the transpose, are the coordinates of the points within the voxel cell, k is the number of points within the voxel cell and k=1,...,m.

6. A semantic map incremental update method according to claim 5, characterized in that: In step S42, the point cloud distribution category obtained according to the probability density function is specifically: Let the three eigenvalues of the covariance matrix be ,when , then the point cloud distribution category is considered to be spherical; when , then the point cloud distribution category is considered to be plane; when , then the point cloud distribution category is considered to be linear.

7. A semantic map incremental update method according to any one of claims 1 to 6, characterized in that: Step S5 specifically includes: S51: Refine the boundary of the environmental change area to obtain an increment; S52: deleting the region in the previous global semantic map according to the incremental boundary; S53: Update the increment to the previous global semantic map to obtain the current global semantic map.

8. A system for implementing the semantic map incremental update method according to any one of claims 1 to 7, wherein the system stores a voxelized previous global semantic map, characterized in that: The system includes a mapping module, a registration module, a detection module, and an update module that are communicatively connected in sequence; The mapping module is used to construct the current local semantic map and input it into the registration module; The registration module is used to register the current local semantic map with the previous global semantic map and convert them into the same coordinate system, and input the registered and coordinate-converted data into the detection module; The detection module is used to detect semantically inconsistent areas of point clouds corresponding to the current local semantic map and the previous global semantic map, obtain semantic change areas, determine environmental change areas based on the semantic change areas, and input the environmental change areas into the update module; The updating module is used to extract the environment change area as an increment and update the increment to the previous global semantic map to obtain a global semantic map.

Images