A mobile robot continuous semantic mapping method

Abstract

The application discloses a kind of mobile robot continuous semantic mapping methods, it is related to robot technical field, the method can improve the precision and efficiency of the three-dimensional map constructed. Including the following steps: mobile robot is carried with vision camera and three-dimensional laser radar, and the data of two kinds of sensors are fused into semantic point cloud, and three-dimensional reconstruction is carried out;Redundant voxel filtering model is used to classify voxel, to obtain the filtering factor for redundant voxel, and adaptive kernel length model is used to divide the appropriate kernel length for each voxel;Relying on filtering factor and kernel length, construct multi-entropy value kernel function to infer from sparse semantic point cloud containing noise to obtain dense and accurate continuous semantic map.

Description

Technical Field

[0001] This invention relates to the field of robotics, and more specifically to a method for continuous semantic mapping of mobile robots. Background Technology

[0002] Mapping, also known as 3D reconstruction, essentially uses sparse, noisy sensor data to construct a dense and accurate 3D representation of the surrounding scene. It plays a crucial role in a range of robotic tasks, including exploration of unknown environments, behavioral decision-making, and path planning, and is considered an important problem in the field of robotics. Furthermore, when robots are required to perform higher-level tasks, the addition of semantic information can help them distinguish object categories, allowing for a higher level of environmental representation.

[0003] Currently, the most widely used semantic mapping technique is occupancy-grid mapping. Occupancy-grid mapping assumes that voxels are completely independent, thus generating discrete maps that cannot fill in gaps, contradicting the fact that object surfaces are continuous in the real world. Recent successes in Bayesian kernel inference have driven the development of continuous mapping, such as S-BKI. S-BKI incorporates local spatial correlations into its mapping model, enabling the inference of continuous surfaces from sparse sensor data. Specifically, when processing sensor observation data, S-BKI considers not only data falling within the current voxel but also data within a certain local space surrounding it. In this way, voxels without observation data can be filled in using information from surrounding voxels.

[0004] Existing continuous semantic graph construction methods have two main drawbacks:

[0005] First, existing continuous semantic mapping does not fully consider the potential differences between voxels. All voxels are treated the same, so voxels next to objects are incorrectly assigned to occupied states, resulting in objects becoming over-inflated. Such an over-inflated map is unsuitable for robot navigation tasks because normally passable spaces may become blocked.

[0006] Secondly, compared with discrete mapping methods, existing continuous mapping methods require a cubic times the computational cost, making it difficult to guarantee real-time performance and thus failing to meet the requirements of robots working online in unknown environments.

[0007] Therefore, there is an urgent need for a novel continuous semantic mapping method that can improve mapping efficiency while alleviating the problem of map overexpansion. Summary of the Invention

[0008] In view of this, the present invention provides a continuous semantic mapping method for mobile robots, which can improve the accuracy and efficiency of the constructed 3D map.

[0009] To achieve the above objectives, the technical solution of the present invention includes the following steps:

[0010] The mobile robot is equipped with a visual camera and a 3D LiDAR, and it fuses the data from the two sensors into a semantic point cloud for 3D reconstruction.

[0011] A redundant voxel filtering model is used to classify voxels and obtain filtering factors for redundant voxels. An adaptive kernel length model is then used to assign an appropriate kernel length to each voxel.

[0012] Based on the filter factor and kernel length, a multi-entropy kernel function is constructed to infer dense and accurate continuous semantic maps from sparse and noisy semantic point clouds.

[0013] Furthermore, the mobile robot is equipped with a visual camera and a 3D LiDAR, and fuses the data from the two sensors into a semantic point cloud for 3D reconstruction, specifically:

[0014] A mobile robot explores and reconstructs its surroundings in a completely unknown environment. Given a mobile robot r and camera observations I... 1:t That is, the image from time 1 to t, observed by lidar as L. 1:t That is, the original point cloud from time 1 to t, and the robot trajectory O. 1:t That is, the robot pose from time 1 to time t, and the goal is to estimate the continuous semantic map.

[0015]

[0016] in The input value is I 1:t ,L 1:t O 1:t The output is The probability of the solution to the 3D reconstruction problem corresponds to the maximum a posteriori estimate of equation (1); where for the input, O t ∈SE(3), For a two-dimensional set of real numbers, Given a three-dimensional set of real numbers; for the output, a continuous semantic map. Composed of a series of voxels j Composition, where j takes the value The total number of voxels; each voxel v j Includes the three-dimensional coordinates of the center, and an associated tuple. Used to store probabilistic semantic labels, where K is the total number of semantic categories, and has

[0017] At time t, the camera observes image I. tThe input is fed into a semantic segmentation network to obtain a semantic image. The i-th pixel outputs a measurement tuple in one-hot encoded form. i is the pixel index. One-hot encoding of the Kth semantic class is generated for the i-th pixel; then, using the intrinsic parameters of the camera and radar, and the extrinsic parameters between the camera and radar, the original point cloud is projected across dimensions onto the image plane; the semantic information of the image is transferred to the three-dimensional radar points, thus obtaining the semantic point cloud at the current moment. Based on the location information, the semantic point clouds are aligned to ensure that all alignments are performed in the world coordinate system; Equation (1) is rewritten as:

[0018]

[0019] Aligned semantic point cloud Consisting of a series of semantic points p i Composition, quantity is Each semantic point p i Associated with semantic tag c i The problem of 3D reconstruction is further refined to: given semantic points and labels The goal is to estimate v for each voxel j probabilistic semantic label λ j :

[0020]

[0021] in For input The output is The posterior estimated probability; p(λ) j |v j ,p i ,c i () indicates that the input is v j ,p i ,c i The output is λ. j The posterior estimated probability.

[0022] Furthermore, a redundant voxel filtering model is used to classify voxels, resulting in a filtering factor for redundant voxels, specifically:

[0023] Introducing "blocks" as an intermediate layer between the "map" and the "voxels"; each block It is a small semantic octree consisting of several adjacent voxels v j Composition; J is the block index number;

[0024] Redundant voxel filtering is performed on a block-by-block basis, therefore block b J All voxels will inherit b J The attribute; in order to get the current block b JThe state of the surrounding local space is also introduced by the expansion block. It is composed of the current block b J and some surrounding blocks {b x}composition;

[0025] Extract a graph model from the extended block: using blocks in the extended block as nodes, and the current block b J and surrounding blocks {b x The connections between} are treated as edges; the extracted graph model is Dandelion-CRF; given observation D, the context entropy E of the J-th block is... con Described as central node b J Conditional probability:

[0026]

[0027] Among them, b J ~1 represents b J It needs to be filled in to enhance the continuity of the map;

[0028] There are two types of cliques in Dandelion-CRF: one is a single node {b k}, another is a pair of adjacent nodes {b k ,b l}, where k and l are index variables; by choosing the exponential potential function and introducing the characteristic function, the conditional probability is defined as:

[0029]

[0030]

[0031] Where P(b) J ~1|D) is an input of D and an output of b. J ~1 posterior estimated probability; E(b J ~1|D) is The reference to Z(D); Z(D) is the normalized partial function, and the state characteristic function ψ(b) is the reference to the state characteristic function ψ(b). k ) and the transition characteristic function ψ(b) k ,b l Describe the effects of the observed sequence and adjacent nodes respectively; formulate the following rule, ψ(b k Different values ​​are obtained depending on whether the block has observation data; ψ(b) k ,b l The radial basis function of the Euclidean distance between blocks is used; in equations (7) and (8), ω1 and ω2 are hyperparameters controlling the amount of transmitted information, s is the block resolution, and p i ∈b k Represents semantic point p i Falling on block b k Inside;

[0032]

[0033]

[0034] Context entropy E con It revealed the potential differences between voxels and used them as a distinguishing indicator; with T con The entropy threshold is E. con Less than T con The voxels contained in the block are redundant; these redundant voxels will be filtered out during continuous inference, while the observed voxels and the voxels to be activated will be retained to estimate a more accurate map; this operation is performed by passing a filtering factor f to the subsequent multi-entropy kernel inference module. J This is achieved;

[0035]

[0036] in The context entropy corresponding to the block with index J; Indicates conformity Then f J The value is 1, which does not meet the requirements. Then f J It is 0.

[0037] Furthermore, an adaptive kernel length model is used to assign a suitable kernel length to each voxel, specifically including the following steps:

[0038] The adaptive kernel length model is used to assign the most suitable kernel length to voxels, and the processing unit is still "block". Voxels in the same block are assigned the same kernel length.

[0039] Category Entropy E cla Used to measure the overall uncertainty of voxels; category entropy E cla It is composed of two sub-entropies: one is the probability entropy E. p The other is semantic entropy E s ;

[0040] Probability Entropy E p This reflects the proportion of quantity, defined as:

[0041]

[0042] Where n max It is the number of semantic points that constitute the largest number of all semantic classes, n all It's a block of b J The total number of semantic points in;

[0043] Semantic entropy E s Describes block b JThe diversity of semantic tags in Chinese; definition To represent block b J The number of semantic tags in the semantic entropy E s Defined as:

[0044]

[0045] Where K is the total number of semantic categories;

[0046] Category Entropy E cla The combination of the two sub-entropies is shown in equation (12);

[0047]

[0048] Substituting equations (10) and (11) into equation (12), the category entropy E cla for:

[0049]

[0050] The category entropy corresponding to the block with index J;

[0051] For block b J voxel v j The kernel length with upper and lower boundaries is adjusted as follows:

[0052]

[0053] Furthermore, using a filter factor and kernel length, a continuous semantic map is obtained from the semantic point cloud through kernel function reasoning, specifically including the following steps:

[0054] According to Bayes' rule, equation (3) can be decomposed into the product of prior probability and conditional probability:

[0055] p(λ j |v j ,p i ,c i )∝p(λ j )p(c i |v j ,p i ,λ j (15)

[0056] Where p(λ) j |v j ,p i ,c i p(λ) represents the posterior probability obtained after observation. j p(c) represents the prior probability without any information; i |v j ,p i,λ j To obtain the likelihood probability of the corresponding observation; for incremental Bayesian inference, the likelihood probability is modeled as a classification distribution. The prior probability follows a Dirichlet distribution Dir(K,σ0), and the posterior probability follows a Dirichlet distribution Dir(K,σ0). j ),in and These are distribution parameters;

[0057] Using a three-dimensional space The kernel function k(v) that operates j ,p i To create the extended likelihood probability, we get:

[0058]

[0059] After simplification, equation (16) yields two Dirichlet distribution parameters σ0 and σ0. j Relationship between them:

[0060] σ j =σ0+k(v j ,p i )c i (17)

[0061] Ignoring the value of σ0, σ j voxel v j semantic count tuples; To represent in voxel v j The count value of semantic points with semantic label k in the middle;

[0062] Equation (17) shows that the semantic count tuple σ j Not only calculating the value falling into the current voxel v j The semantic points are calculated, and neighboring semantic points are weighted by a kernel function; a sparse kernel function k0(v) is selected. j ,p i (As a template)

[0063]

[0064] Where I represents the indicator function, d = ||v j -p i || is the Euclidean distance between the voxel and the point cloud, ε0 is the scaling factor, and L is the kernel length;

[0065] By integrating equation (9) of the redundant voxel filtering model and equation (14) of the adaptive kernel length model into equation (18), a multi-entropy kernel function k is derived. e (v j ,p iAs shown in equation (19); multi-entropy kernel function k e (v j ,p i The adaptive kernel function k0(v) was improved by fusing the two models. j ,p i ):

[0066]

[0067] Semantic point cloud of insertion alignment map Each voxel v j semantic counting tuple σ j It can be obtained as follows:

[0068]

[0069] in Indicates in voxel v j The count value of semantic points with semantic label k.

[0070] Therefore, voxel v j The probabilistic semantic label is the expected value of the posterior Dirichlet distribution:

[0071]

[0072] The probability that the semantic label of the voxel with index j is k;

[0073] In summary, the mapping problem defined in equation (3) has been transformed into a probabilistic problem as shown in equation (21); when new sensor observations are obtained, the final continuous semantic map is updated by incrementally calculating equations (20) and (21).

[0074] Beneficial effects:

[0075] 1. This invention provides a continuous semantic mapping method for mobile robots. It fuses visual camera data and 3D radar data into a semantic point cloud for 3D reconstruction. A redundant voxel filtering model classifies voxels to obtain filtering factors for redundant voxels. An adaptive kernel length model assigns an appropriate kernel length to each voxel to improve efficiency. Finally, multi-entropy kernel inference utilizes the filtering factors and kernel lengths provided by the two models to infer a continuous semantic map from the semantic point cloud using a kernel function. This method can improve mapping efficiency while mitigating the problem of map overexpansion.

[0076] 2. In the process of constructing a continuous semantic map, this invention proposes a redundant voxel filtering model, which distinguishes different voxels into observation voxels (which have observation data), activation voxels (which do not have observation data but need to be filled in), and redundant voxels (which do not have observation data and do not need to be filled in). The redundant voxel filtering model filters out those redundant voxels before performing continuous map inference, thereby ensuring that the map is not overfitted and objects are not inflated, thus alleviating the over-inflation problem.

[0077] 3. In the process of constructing a continuous semantic map, the present invention proposes an adaptive kernel length model. This adaptive kernel length model uses the specific information contained in the observed voxels to determine the size of the local spatial range that the current voxel needs to process. It assigns an appropriate kernel length to each voxel through category entropy. Since most voxels are located inside objects, the observation of these voxels is relatively accurate. It is not necessary to over-consider the surrounding information, thereby greatly reducing the computational complexity and improving the efficiency of map construction. Attached Figure Description

[0078] Figure 1 This is a flowchart of the continuous semantic mapping method for mobile robots according to the present invention.

[0079] Figure 2 For expansion blocks and dandelion-CRF. Detailed Implementation

[0080] The present invention will now be described in detail with reference to the accompanying drawings and embodiments.

[0081] To address the problems of excessive map expansion and poor real-time performance in current continuous semantic mapping methods, this invention proposes a novel continuous semantic mapping method for mobile robots to improve the accuracy and efficiency of the constructed 3D maps.

[0082] Figure 1 (The attached diagram shows the method flowchart.) A mobile robot equipped with a vision camera and a 3D LiDAR fuses data from both sensors into a semantic point cloud for 3D reconstruction. During this process, a redundant voxel filtering model classifies voxels to obtain filtering factors for redundant voxels, and an adaptive kernel length model assigns an appropriate kernel length to each voxel to improve efficiency. Finally, multi-entropy kernel inference utilizes the filtering factors and kernel lengths provided by the two models to infer a continuous semantic map from the semantic point cloud using a kernel function.

[0083] Specifically, the embodiments of the present invention employ the following three steps to construct a continuous semantic map:

[0084] Step 1: The mobile robot is equipped with a visual camera and a 3D LiDAR, and the data from the two sensors are fused into a semantic point cloud for 3D reconstruction;

[0085] Consider a robot exploring and attempting to reconstruct its surroundings in a completely unknown environment. The problem can be defined as: given a robot r, camera observation I 1:t (i.e., images from time 1 to t), lidar observation of L 1:t (Original point cloud at times 1 to t) and robot trajectory O 1:t (Robot pose from time 1 to t), the goal is to estimate a continuous semantic map.

[0086]

[0087] in The input is I 1:t ,L 1:t O 1:t The output is The probability of the solution to the 3D reconstruction problem corresponds to the maximum a posteriori estimate of equation (1); where for the input, O t ∈SE(3), For a two-dimensional set of real numbers, Given a three-dimensional set of real numbers; for the output, a continuous semantic map. Composed of a series of voxels j Composition, where j takes the value The total number of voxels; each voxel v j Includes the three-dimensional coordinates of the center, and an associated tuple. Used to store probabilistic semantic labels, where K is the total number of semantic categories, and has

[0088] At time t, the camera observes image I. t The input is fed into a semantic segmentation network to obtain a semantic image. The i-th pixel outputs a measurement tuple in one-hot encoded form. i is the pixel index. One-hot encoding of the Kth semantic class is generated for the i-th pixel. Then, using the intrinsic parameters of the camera and radar, and the extrinsic parameters between the camera and radar, the original point cloud is projected across dimensions onto the image plane. The semantic information of the image is transferred to the 3D radar points, thus obtaining the semantic point cloud at the current time. Based on the location information, the semantic point clouds can be aligned to ensure that all alignments are performed in the world coordinate system. Therefore, equation (1) can be rewritten as:

[0089]

[0090] Aligned semantic point cloud Consisting of a series of semantic points p i Composition, quantity is Each semantic point is associated with the semantic tag c.i Therefore, the problem of 3D reconstruction can be refined to: given semantic points and labels The goal is to estimate v for each voxel j probabilistic semantic label λ j .

[0091]

[0092] in For input The output is The posterior estimated probability; p(λ) j |v j ,p i ,c i () indicates that the input is v j ,p i ,c i The output is λ. j The posterior estimated probability.

[0093] Step 2: Classify voxels using a redundant voxel filtering model to obtain filtering factors for redundant voxels, and assign an appropriate kernel length to each voxel using an adaptive kernel length model.

[0094] Existing continuous mapping methods fail to clearly distinguish voxels, leading to overfitting of the final continuous map. Redundant voxel filtering models aim to address this problem by distinguishing different types of voxels and filtering out redundant voxels.

[0095] To improve semantic accuracy and reasoning efficiency, "blocks" are introduced as an intermediate layer between the "map" and the "voxels". Each block... It is a small semantic octree consisting of several adjacent voxels v j Composition. Considering time costs, filtering of redundant voxels is performed on a block-by-block basis, therefore block b J All voxels will inherit b J The attributes of the current block b. J The state of the surrounding local space is also introduced by the expansion block. It is composed of the current block b J and some surrounding blocks {b x}composition.

[0096] like Figure 2 (As shown in the attached diagram) a graph model is extracted from the extended block: using blocks in the extended block as nodes, and the current block b... J and surrounding blocks {b x The connections between nodes are called edges. This graph is called "Dandelion-CRF" due to its unique structure. It allows for arbitrary modification of the style of the expanded blocks to suit map requirements and sensor resolution. Given observation D, context entropy Econ Described as central node b J Conditional probability:

[0097]

[0098] Among them, b J ~1 represents b J It needs to be filled in to enhance the continuity of the map.

[0099] There are two types of cliques in Dandelion-CRF: one is a single node {b k}, another is a pair of adjacent nodes {b k ,b l}, where k and l are index variables. By choosing the exponential potential function and introducing the characteristic function, the conditional probability is defined as:

[0100]

[0101]

[0102] Where P(b) J ~1|D) is an input of D and an output of b. J ~1 posterior estimated probability; E(b J ~1|D) is The reference to Z(D); Z(D) is the normalized partial function, and the state characteristic function ψ(b) is the reference to the state characteristic function ψ(b). k ) and the transition characteristic function ψ(b) k ,b l Describe the effects of the observed sequence and adjacent nodes respectively. The following rule is established: ψ(b) k Different values ​​are obtained depending on whether the block has observation data. ψ(b) k ,b l The radial basis function of the Euclidean distance between blocks is used. In equations (7) and (8), ω1 and ω2 are hyperparameters controlling the amount of transmitted information, s is the block resolution, and p i ∈b k This indicates that the semantic point pi falls within block b. k Inside.

[0103]

[0104]

[0105] Context entropy E con This reveals potential differences between voxels, which can be used as a distinguishing indicator. (T) con The entropy threshold is E. con Less than T conThe voxels contained in the blocks are redundant; they are typically located in gaps between two objects or outside object edges. These redundant voxels will be filtered out during continuous inference, while observed voxels (voxels with observation data) and activation voxels (voxels without observation data but not requiring filtering) will be retained to estimate a more accurate map. This operation is performed by passing a filtering factor f to the subsequent multi-entropy kernel inference module. J This is achieved through [the means].

[0106]

[0107] in The context entropy corresponding to the block with index J; Indicates conformity Then f J The value is 1, which does not meet the requirements. Then f J It is 0.

[0108] Adaptive kernel length model:

[0109] Kernel length is crucial for graph construction efficiency because it determines the query range. The adaptive kernel length model aims to assign the most appropriate kernel length to voxels. Continuing to process in "blocks," voxels within the same block are assigned the same kernel length.

[0110] Category Entropy E cla It is used to measure the overall uncertainty of voxels. It consists of two sub-entropies: one is the probability entropy E. p The other is semantic entropy E s On the one hand, probability entropy E p This reflects the proportion of quantity, defined as:

[0111]

[0112] Where n max It is the number of semantic points that constitute the largest number of all semantic classes, n all It's a block of b J The total number of semantic points in the text. On the other hand, semantic entropy E s Describes block b J The diversity of semantic tags in Chinese. Definition To represent block b J The number of semantic tags in the semantic entropy E s Defined as:

[0113]

[0114] Category Entropy E cla The combination of the two sub-entropies is shown in equation (12). The probability entropy E is... p It accounts for a larger proportion because it includes semantic entropy E. sThe information is partially derived from semantic entropy, and the weight of semantic entropy should be inversely proportional to the total number of semantic classes K.

[0115]

[0116] This is the category entropy corresponding to the block with index J.

[0117] It will have the maximum category entropy when there is no observation data in the block. Or it will also have the maximum category entropy when all labeled points fall evenly into the block. Substituting equations (10) and (11) into equation (12), the category entropy E cla Can be written as:

[0118]

[0119] Larger category entropy E cla This implies higher uncertainty, requiring a larger query range to ensure map accuracy. Therefore, for block b... J voxel v j The kernel length with upper and lower boundaries is adjusted as follows:

[0120]

[0121] Step 3: Using the filter factor and kernel length, use the kernel function to infer a continuous semantic map from the semantic point cloud.

[0122] The effectiveness of the above two models needs to be achieved through multi-entropy kernel inference, as this step is the actual process of converting sensor observations into an updated map. Unlike the classic voxel probability update model, the multi-entropy kernel inference model is derived from the counting sensor model. According to Bayes' rule, equation (3) can be decomposed into:

[0123] p(λ j |v j ,p i ,c i )∝p(λ j )p(c i |v j ,p i ,λ j (15)

[0124] For incremental Bayesian inference, the likelihood probability is modeled as a classification distribution. The prior probability follows a Dirichlet distribution Dir(K,σ0), and the posterior probability follows a Dirichlet distribution Dir(K,σ0). j ),in and It is a distribution parameter.

[0125] To truly break the independence between voxels, this method uses a three-dimensional space The kernel function k is used to create the extended likelihood probability, thus yielding:

[0126]

[0127] After simplification, equation (16) yields two Dirichlet distribution parameters σ0 and σ0. j Relationship between them:

[0128] σ j =σ0+k(v j ,p i )c i (17)

[0129] σ0 typically takes a very small value due to a lack of prior knowledge, therefore σ j This can be viewed as a weighted count of semantic points, referred to as voxel v. j Semantic counting tuples. To represent in voxel v j The count value of semantic points with semantic label k.

[0130] Equation (17) shows that the semantic count tuple σ j Not only calculating the value falling into the current voxel v j The semantic graph is constructed by first identifying semantic points and then calculating neighboring semantic points weighted by a kernel function. Therefore, the choice of kernel function has a crucial impact on the quality and efficiency of semantic graph construction. To reduce computational complexity, this method selects a sparse kernel function k0(v... j ,p i (As a template)

[0131]

[0132] Where I represents the indicator function, d = ||v j -p i || is the Euclidean distance between the voxel and the point cloud, ε0 is the scaling factor, and L is the kernel length.

[0133] By combining equation (9) of the redundant voxel filtering model and equation (14) of the adaptive kernel length model into equation (18), a multi-entropy kernel function k can be derived. e (v j ,p i As shown in equation (19). The multi-entropy kernel function k e (v j ,p i The fusion of the two models improved k0(v) j ,p i This greatly improved the quality of the mapping.

[0134]

[0135] Semantic point cloud of insertion alignment map Each voxel v j semantic counting tuple σ j It can be obtained as follows:

[0136]

[0137] in Indicates in voxel v j The count value of semantic points with semantic label k.

[0138] Therefore, voxel v j The probabilistic semantic label is the expected value of the posterior Dirichlet distribution:

[0139]

[0140] Let be the probability that the semantic label of the voxel with index j is k.

[0141] In summary, the mapping problem defined in equation (3) has been transformed into a probabilistic problem as shown in equation (21). When new sensor observations are obtained, the final continuous semantic map can be updated by incrementally calculating equations (20) and (21).

[0142] In summary, the above are merely preferred embodiments of the present invention and are not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for continuous semantic mapping of mobile robots, characterized in that, Includes the following steps: The mobile robot is equipped with a visual camera and a 3D LiDAR, and it fuses the data from the two sensors into a semantic point cloud for 3D reconstruction. A redundant voxel filtering model is used to classify voxels, obtaining a filtering factor for redundant voxels. An adaptive kernel length model is then used to assign an appropriate kernel length to each voxel. Specifically, the process of using the redundant voxel filtering model to classify voxels and obtain the filtering factor for redundant voxels is as follows: Introducing "blocks" as an intermediate layer between the "map" and the "voxels"; each block It is a small semantic octree consisting of several adjacent voxels. composition; J The index number of the block; Redundant voxels are filtered on a block-by-block basis, therefore blocks All voxels will inherit The attribute; in order to get the current block The state of the surrounding local space is also introduced by the expansion block. It is composed of the current block and some surrounding blocks composition; Extract a graph model from the extended block: using blocks in the extended block as nodes, and the current block as the node. and surrounding blocks The connections between them are used as edges; the extracted graph model is Dandelion-CRF; given observations The context entropy of the Jth block Described as the central node Conditional probability: (4) in, express It needs to be filled in to enhance the continuity of the map; There are two types of cliques in Dandelion-CRF: one is a single node Another type is a pair of adjacent nodes. ,in and It is an index variable; by choosing the exponential potential function and introducing the characteristic function, the conditional probability is defined as: (5) (6) in Given input D, the output is The posterior estimated probability; for The reference; It is a normalized partial function, a state characteristic function. and transition characteristic function Describe the effects of the observed sequence and adjacent nodes respectively; formulate the following rules, Different values ​​are obtained depending on whether the block has observation data; Radial basis functions of the Euclidean distance between blocks are used; in equations (7) and (8), and It is a hyperparameter that controls the amount of information transmitted. It is the resolution of the block. Representing semantic points p i Falling on the block Inside; (7) (8) Context entropy This reveals the potential differences between voxels, which can be used as a distinguishing indicator; Entropy threshold, context entropy Less than The voxels contained in the block are redundant; these redundant voxels will be filtered out during continuous inference, while the observed voxels and the voxels to be activated will be retained to estimate a more accurate map; this operation is performed by a filter factor passed to subsequent multi-entropy kernel inference modules. This is achieved; (9) in For index J The context entropy corresponding to the block; Indicates conformity but The value is 1, which does not meet the requirements. but =0; The process of assigning a suitable kernel length to each voxel using an adaptive kernel length model specifically includes the following steps: The adaptive kernel length model is used to assign the most suitable kernel length to voxels, and the processing unit is still "block". Voxels in the same block are assigned the same kernel length. Category entropy Used to measure the overall uncertainty of voxels; category entropy It is composed of two sub-entropies: one is probability entropy. The other is semantic entropy. ; Probability Entropy This reflects the proportion of quantity, defined as: (10) in It represents the largest number of semantic points across all semantic classes. It is a piece The total number of semantic points in; Semantic entropy Describes the block The diversity of semantic tags in Chinese; definition To represent a block The number of semantic tags and semantic entropy Defined as: (11) in It represents the total number of semantic categories; Category entropy The combination of the two sub-entropies is shown in equation (12); (12) Substituting equations (10) and (11) into equation (12), the category entropy... for: (13) The category entropy corresponding to the block with index J; For blocks voxels in The kernel length with upper and lower boundaries is adjusted as follows: (14) Based on the filtering factor and kernel length, a multi-entropy kernel function is constructed to infer a continuous semantic map from the semantic point cloud.

2. The method for continuous semantic mapping of a mobile robot as described in claim 1, characterized in that, The mobile robot is equipped with a visual camera and a 3D LiDAR, and fuses the data from the two sensors into a semantic point cloud for 3D reconstruction, specifically: The mobile robot explores and reconstructs its surroundings in a completely unknown environment, i.e., given a mobile robot... Camera observation is That is, the image from time 1 to t, observed by lidar. That is, the original point cloud from time 1 to t, and the robot trajectory. That is, the robot pose from time 1 to time t, and the goal is to estimate the continuous semantic map. ; (1) in The input is The output is The probability of the solution to the 3D reconstruction problem corresponds to the maximum a posteriori estimate of equation (1); where for the input, , , , For a two-dimensional set of real numbers, Given a three-dimensional set of real numbers; for the output, a continuous semantic map. Composed of a series of voxels Composition, where j takes values ​​from 1 to... ; The total number of voxels; each voxel Includes the three-dimensional coordinates of the center, and an associated tuple. , used to store probabilistic semantic labels, where It is the total number of semantic categories, and has k is a semantic tag; At any moment Camera observation images The data is input into a semantic segmentation network to obtain a semantic image. i Each pixel outputs a measurement tuple in one-hot encoded form. , i For pixel index, For the first i The pixel generates information about the first pixel. One-hot encoding of semantic class is performed; then, using the intrinsic parameters of the camera and radar, and the extrinsic parameters between the camera and radar, the original point cloud is projected across dimensions onto the image plane; the semantic information of the image is transferred to the three-dimensional radar points, thus obtaining the semantic point cloud at the current moment. Based on the location information, the semantic point clouds are aligned to ensure that all alignments are performed in the world coordinate system; Equation (1) is rewritten as: (2) Aligned semantic point cloud Consisting of a series of semantic points Composition, quantity is Each semantic point Associated with semantic tags The problem of 3D reconstruction is further refined to: given semantic points and labels The goal is to estimate each voxel probabilistic semantic tags : (3) in For input The output is The posterior estimated probability; The input is The output is The posterior estimated probability.

3. The continuous semantic mapping method for mobile robots as described in claim 2, characterized in that, Using the aforementioned filter factor and kernel length, a continuous semantic map is obtained from the semantic point cloud through reasoning using a kernel function, specifically including the following steps: According to Bayes' rule, equation (3) can be decomposed into the product of prior probability and conditional probability: (15) in To obtain the posterior probability after observation; This represents the prior probability when there is no information available. To obtain the likelihood probability of the corresponding observation; for incremental Bayesian inference, the likelihood probability is modeled as a classification distribution. The prior probability follows a Dirichlet distribution. The posterior probability follows a Dirichlet distribution. ,in and These are distribution parameters; Using a three-dimensional space Operating kernel function To create the extended likelihood probability, we get: (16) Equation (16) is simplified to obtain two Dirichlet distribution parameters. and Relationship between them: (17) neglect The value of , voxels semantic counting tuples; To represent voxels semantic tags The count value of semantic points; Equation (17) shows the semantic counting tuple Not only calculating the current voxel The semantic points are calculated, and neighboring semantic points are also weighted by a kernel function; Choosing a sparse kernel function As a template: (18) in Represents an indicator function. It is the Euclidean distance between voxels and point clouds. It is a scaling factor. L For the length of the core; By integrating equation (9) of the redundant voxel filtering model and equation (14) of the adaptive kernel length model into equation (18), a multi-entropy kernel function is derived. As shown in equation (19); multi-entropy kernel function The adaptive kernel function was improved by fusing the two models. : (19) Semantic point cloud of insertion alignment ,map Each voxel semantic counting tuples It can be obtained as follows: (20) in Indicated in voxels semantic tags The count value of semantic points; Therefore, voxels The probabilistic semantic label is the expected value of the posterior Dirichlet distribution: (21) The semantic label for the voxel with index j is k The probability of; In summary, the mapping problem defined in equation (3) has been transformed into a probabilistic problem as shown in equation (21); when new sensor observations are obtained, the final continuous semantic map is updated by incrementally calculating equations (20) and (21).

Images