Global positioning method for mobile robot in indoor scene

Abstract

The invention belongs to the technical field of mobile robot positioning and navigation, and discloses a mobile robot global positioning method for an indoor scene, which comprises the following steps: firstly, creating a semantic laser radar point cloud by fusing a laser radar point cloud with a visual camera, and constructing a semantic grid map Mapsemic, a semantic position map Mapposition and a semantic region map Maparea; then, on the basis of a semantic position map Map and a semantic region map Map, by means of semantic laser radar point cloud, a distance search algorithm and a region search algorithm are adopted for pose screening, and a particle set of poses to be selected is obtained; and finally, for the to-be-selected particle set, adopting a random extraction and likelihood value screening mode to obtain a sample particle set, carrying out iterative calculation on each particle in the sample particle set by adopting a gradient positioning algorithm to carry out positioning, and taking a positioning pose with the maximum likelihood value in all positioning results as a global positioning pose.

Description

Technical Field

[0001] This invention relates to the field of mobile robot positioning and navigation technology, specifically a global positioning method for mobile robots in indoor scenes. Background Technology

[0002] With continuous innovation in sensor and information control technologies, and the impact of the new industrial revolution, robotics is experiencing rapid development. Mobile robot technology has become one of the fastest-growing fields in scientific research. Leveraging their superior capabilities, mobile robots can replace human labor in numerous sectors. Mobile robots have already created significant economic benefits in laboratories, industry, warehousing and logistics, transportation, shopping, and entertainment. They can replace manual tasks such as cargo transportation, monitoring and patrolling, hazardous operations, and repetitive labor, enabling 24 / 7 automated and unmanned operation in factories and laboratories. Their applications cover a wide range of industrial and non-industrial sectors, including monitoring, planetary exploration, patrolling, emergency rescue, reconnaissance, petrochemical applications, industrial automation, construction, entertainment, museum guidance, personal services, extreme environment intervention, transportation and logistics, and healthcare. Many of these applications have already been commercialized.

[0003] Achieving high-precision mobile robot positioning technology to support the aforementioned functions has always been a pressing challenge. Positioning refers to estimating the location and orientation of a mobile robot within its movement area, and it is the foundation of robot navigation. Positioning problems in outdoor environments have been solved by the ubiquitous Global Navigation Satellite System (GNSS). However, due to the complex and variable conditions of indoor environments (factories, laboratories, hospitals, shopping malls, etc.), as well as issues such as signal attenuation, multipath effects, and non-line-of-sight distance, GNSS is no longer applicable indoors. Therefore, industry professionals have proposed various indoor positioning systems. In practical work, mobile robots typically need to move, pick up and place objects, patrol, pass through access control systems, and ride elevators in their work environment. High-precision indoor positioning is a prerequisite for ensuring that mobile robots complete their tasks and for protecting human-machine interaction and environmental safety. The need to achieve centimeter-level positioning accuracy has driven researchers to continuously explore positioning technologies for indoor mobile robots.

[0004] Accurate navigation in complex environments is a fundamental challenge for indoor mobile robots, drawing significant interest from researchers and engineers. Efficient and accurate navigation of mobile robots is a complex task, especially in dynamic and complex spaces such as factories and homes. Despite significant progress in indoor autonomous robot navigation, challenges remain when navigating in environments lacking predefined maps and recognizable geometric landmarks. This is particularly evident in environments such as warehouses, office buildings, manufacturing plants, and structures with extensive corridor networks, where classic localization methods may struggle.

[0005] Mobile robot localization and navigation primarily rely on two key components: accurate mapping and reliable localization. Mapping involves creating a representation of the environment, typically in the form of a grid map or semantic map, to help the robot understand its surroundings. Localization, on the other hand, is the process of precisely locating the robot's exact position within this mapped environment. LiDAR, with its high distance measurement accuracy, is widely used in indoor scenarios. Maps built using LiDAR are typically grid maps, storing only relevant distance information. For global localization, many classic methods employ particle swarm optimization (PSO) Monte Carlo localization algorithms. However, in large indoor scenes, insufficient particle counts can cause the PSO to converge to incorrect positions. To ensure accuracy, using a sufficiently large number of particles for localization significantly increases the computational cost of PSO convergence, resulting in low computational efficiency. Summary of the Invention

[0006] This invention provides a global localization method for mobile robots in indoor scenes, which helps to solve the problems mentioned in the background art.

[0007] This invention provides the following technical solution: a method for global localization of a mobile robot in an indoor scene, comprising:

[0008] Step 1: Create a semantically meaningful LiDAR point cloud by projecting the LiDAR point cloud onto a semantic image from a visual camera;

[0009] Based on the original LiDAR point cloud, an occupancy raster map is constructed, and the location of each semantic point cloud is plotted onto the raster map to build a semantic raster map. semantic ;

[0010] Constructing a semantic location map based on semantic raster maps position and semantic region map area ;

[0011] Step 2: Based on semantic location map position and semantic region map area Using semantic lidar point clouds, distance search algorithm and region search algorithm are used to filter poses and obtain a set of particles with candidate poses;

[0012] Step 3: Randomly select an appropriate number of particles from the candidate particle set as the initial sample particle set, and perform likelihood value screening on the initial sample particle set to obtain the final sample particle set.

[0013] For each particle in the sample particle set, the gradient localization algorithm is used to iteratively calculate the localization, and the localization pose with the largest likelihood value among all localization results is taken as the global localization pose.

[0014] Optionally, the method of creating a semantically meaningful lidar point cloud by projecting the lidar point cloud onto a semantic image from a visual camera includes:

[0015] Semantic information is obtained by processing visual images through semantic segmentation technology, and LiDAR point clouds are clustered to eliminate outliers and distinguish instances.

[0016] The clustered point cloud is projected onto the corresponding position of the semantic image using the projection matrix obtained by calibration, thereby assigning the semantic labels in the image to the LiDAR point cloud and forming point cloud data with semantic information.

[0017] Optionally, the occupancy grid map constructed based on the original LiDAR point cloud and the location of each semantic point cloud are projected onto the grid map to construct a semantic grid map. semantic ,include:

[0018] The semantic consistency of point clouds is determined by setting a clustering threshold.

[0019] When the proportion of specific semantic points in a clustered point cloud exceeds a threshold, the entire point cloud is assigned a unified semantic meaning.

[0020] By combining a pre-built raster map, semantic point clouds are projected onto raster cells, and semantic annotations are performed on raster cells that meet the occupancy conditions, ultimately forming a semantic raster map.

[0021] Optionally, the construction of a semantic location map based on a semantic raster map... position and semantic region map area ,include:

[0022] Valid semantically labeled regions are filtered by setting thresholds to avoid semantically labeling unoccupied and unknown regions;

[0023] A semantic statistical filtering mechanism is used to resolve semantic boundary conflicts, and the final semantic labels are determined by calculating the proportion of semantic annotation times.

[0024] Optionally, the construction of a semantic location map based on a semantic raster map... position and semantic region map area Specifically, it also includes:

[0025] Extract various semantic regions from the semantic raster map and perform geometric simplification on them;

[0026] A semantic circular region set is obtained by calculating the minimum circumcircle, and a semantic location map is constructed.

[0027] Extract the convex hull boundary of the semantic region to form a semantic region map;

[0028] Generate a likelihood map and its gradient map based on the original raster map.

[0029] Optionally, the semantic location map (Map) position and semantic region map area Using semantic lidar point clouds, distance search and region search algorithms are employed for pose selection to obtain a set of particles with candidate poses, including:

[0030] Generate a set of semantic location points based on semantic point clouds;

[0031] Candidate circular regions that meet the criteria are filtered in the semantic location map using a distance matching algorithm;

[0032] Geometric constraint methods are used to calculate the possible pose regions and angle ranges of the robot, and pose verification is performed by combining semantic region maps, ultimately forming a set of particles for candidate poses.

[0033] Optionally, the step of randomly selecting an appropriate number of particles from the candidate particle set as an initial sample particle set, and performing likelihood value screening on the initial sample particle set to obtain the final sample particle set, includes:

[0034] An initial particle set is selected from the candidate pose set by random sampling. The likelihood value of each particle is calculated using a high standard deviation likelihood map. Valid particles are selected to form a sample particle set by setting a likelihood threshold.

[0035] Optionally, the step of iteratively calculating the localization of each particle in the sample particle set using a gradient localization algorithm, and taking the localization pose with the highest likelihood value among all localization results as the global localization pose, includes:

[0036] Gradient descent is used to iteratively optimize the sample particles, and the pose adjustment is calculated based on the likelihood gradient map so that each particle converges to a local optimum pose.

[0037] By comparing the likelihood values ​​of each candidate particle, the pose corresponding to the maximum value is selected as the final global localization result.

[0038] The present invention has the following beneficial effects:

[0039] 1. This invention creates a semantically information-rich LiDAR point cloud by fusing it with a visual camera's semantic image, and then constructs a semantic raster map, a semantic location map, and a semantic region map based on this. Specifically, semantic segmentation technology is used to process the visual image to obtain semantic labels, while the LiDAR point cloud is clustered to eliminate outliers and distinguish instances. The clustered point cloud is projected onto the corresponding positions in the semantic image using a calibrated projection matrix, thereby assigning semantic labels from the image to the LiDAR point cloud, forming point cloud data with semantic information. Based on this, an occupancy raster map constructed from the original LiDAR point cloud and the position of each semantic point cloud are used. A clustering threshold is set to determine the semantic consistency of the point cloud. When the proportion of a specific semantic point in the clustered point cloud exceeds the threshold, the entire point cloud is assigned a unified semantic meaning. Combined with the pre-constructed raster map, the semantic point cloud is projected onto raster cells, and raster cells that meet the occupancy conditions are semantically labeled, ultimately forming a semantic raster map. Furthermore, by setting thresholds to filter effective semantically labeled regions, semantic labeling of unoccupied and unknown areas is avoided. A semantic statistical filtering mechanism is employed to resolve semantic boundary conflicts, and the final semantic labels are determined by calculating the proportion of semantic labeling frequency. Various semantic regions are extracted from the semantic grid map and geometrically simplified. A semantic location map is constructed by calculating the minimum circumcircle to obtain a set of semantic circular regions, while the convex hull boundaries of the semantic regions are extracted to form a semantic region map. The beneficial effect of this series of claims is that traditional mobile robot localization relies on grid maps constructed by LiDAR. These maps only contain distance information and lack semantic context, resulting in low localization efficiency in complex indoor environments. This invention integrates visual semantic information to endow LiDAR point clouds with rich semantic labels, thereby constructing a semantically enhanced grid map. This map not only identifies the physical structure of the environment but also labels key semantic features such as doors, windows, and furniture, providing the robot with a more comprehensive understanding of the environment. During global localization, semantic information can significantly reduce location ambiguity because semantic landmarks have unique recognition characteristics. For example, in corridors or areas with similar geometric structures, geometric information alone may not be enough to distinguish different locations. However, semantic labels such as "elevator door" or "office entrance" can serve as key anchor points, precisely constraining the robot's possible positions. Furthermore, semantic location maps and semantic region maps, through geometric simplification (such as minimum circumcircle and convex hull boundaries), transform complex semantic regions into easily tractable geometric shapes, further optimizing the location search algorithm. This semantically enhanced map allows the robot to quickly narrow down the candidate location range during global initialization, reducing unnecessary computational overhead and significantly improving localization efficiency. In large indoor scenes, this method can greatly reduce localization initialization time while improving localization success rate, especially in dynamically changing or feature-sparse environments.In summary, by introducing semantic information, this invention solves the problem of insufficient expression of traditional grid maps in semantically rich environments, laying a solid foundation for efficient global localization. This invention combines the semantic recognition function of a visual camera to propose a novel semantic grid map and its construction method. This map can express semantic information on traditional grid maps, enabling the map to express richer and more precise location information, thus helping to improve the efficiency of global localization for mobile robots.

[0040] 2. This invention, based on semantic location maps and semantic region maps, utilizes semantic LiDAR point clouds and employs distance search and region search algorithms for pose selection to obtain a particle set of candidate poses. Specifically, it includes generating a set of semantic location points based on the semantic point cloud; selecting candidate circular regions that meet certain conditions in the semantic location map using a distance matching algorithm; calculating the possible pose regions and angle ranges of the robot using geometric constraint methods; and verifying the pose using the semantic region map, ultimately forming a particle set of candidate poses. The beneficial effect of this claim is that in global localization of mobile robots, traditional methods such as Monte Carlo localization typically require a large number of particles to cover the entire state space to ensure convergence to the correct position, which leads to a heavy computational burden, especially in large indoor scenes. This invention efficiently narrows down the candidate position range through a semantically guided pose selection mechanism. The distance search algorithm uses circular regions represented in the semantic location map to quickly match the current semantic point cloud with semantic landmarks in the map, selecting candidate regions with consistent distances. The region search algorithm further utilizes the convex hull boundary of the semantic region map to perform geometric verification of the candidate poses, ensuring they conform to environmental constraints. This two-stage screening strategy significantly reduces the number of particles to be processed, from tens of thousands to hundreds, thereby greatly reducing computational complexity. For example, in a warehouse environment, semantic point clouds identify multiple shelves and aisles. Distance search algorithms can quickly eliminate areas far from these landmarks, while region search algorithms verify whether the robot is within a walkable area. This not only improves computational efficiency but also avoids the risk of particle swarms converging to incorrect positions. Compared with traditional methods, this algorithm shortens computation time while maintaining the same positioning accuracy, enabling the robot to respond to environmental changes in real time. Furthermore, this method is robust and has a certain tolerance to sensor noise and dynamic environmental changes because it relies on the stability of semantic information rather than simple geometric features. Therefore, this invention solves the problem of low computational efficiency of particle filtering in large-scale scenes through intelligent pose screening, providing a feasible solution for real-time global localization.

[0041] 3. This invention randomly selects an appropriate number of particles from the candidate particle set as an initial sample particle set. The initial sample particle set is then filtered by likelihood value to obtain a final sample particle set. Each particle in the sample particle set is then iteratively located using a gradient descent algorithm. The pose with the highest likelihood value among all localization results is taken as the global localization pose. Specifically, this involves selecting an initial particle set from the candidate pose set through random sampling, calculating the likelihood value of each particle using a high standard deviation likelihood map, and filtering effective particles to form the sample particle set by setting a likelihood threshold. Subsequently, the sample particles are iteratively optimized using gradient descent, and the pose adjustment is calculated based on the likelihood gradient map, causing each particle to converge to a local optimum pose. Finally, by comparing the likelihood values ​​of each candidate particle, the pose corresponding to the maximum value is selected as the final global localization result. The beneficial effect of this claim is that a key challenge in global localization is how to avoid getting trapped in local optima while ensuring accuracy. Traditional Monte Carlo localization methods rely on the random propagation and resampling of particle swarms, which are prone to converging to incorrect positions in complex environments and involve large computational costs. This invention introduces a gradient localization algorithm combined with semantically guided particle filtering to achieve efficient and robust accurate localization. First, low-probability particles are removed from the candidate particle set through likelihood value filtering, ensuring the initial sample set has high potential and reducing unnecessary computation. Then, gradient descent is applied to each sample particle, iteratively adjusting its pose using a likelihood gradient map to quickly converge to a local extremum. This method is similar to a refined local search, but based on a global filtering starting point, it avoids the sensitivity to initial values ​​inherent in traditional gradient descent. Since gradient descent directly utilizes the gradient information of the map, the localization process is more efficient, typically requiring only a few iterations to achieve centimeter-level accuracy. Furthermore, by comparing the likelihood values ​​of multiple candidate particles and selecting the global maximum, interference from local optima is effectively prevented. In environments lacking obvious geometric features, such as wide corridors or symmetrical rooms, this method can accurately identify the robot's true position, while significantly improving computational efficiency because the iterative computation of gradient descent is far less than the large-scale sampling of particle filtering. Overall, this invention combines global search with local optimization, utilizing both global guidance from semantic information and the precise convergence capability of gradient descent. This solves the problem of unrobust localization in complex indoor scenes using traditional methods, providing reliable position awareness for mobile robots. Attached Figure Description

[0042] Figure 1 This invention illustrates the overall framework of the map preprocessing steps;

[0043] Figure 2 This forms the overall framework for the global positioning steps of the present invention;

[0044] Figure 3 Example of a raster map and its corresponding likelihood map:

[0045] (a) Original raster map grid ;

[0046] (b) High standard deviation likelihood map likelihood ;

[0047] (c) x-direction likelihood gradient map gradient-x ;

[0048] (d)y-direction likelihood gradient map gradient-y ;

[0049] Figure 4 Examples of semantically relevant maps:

[0050] (a) Semantic Raster Map semantic ;

[0051] (b) Semantic Region Map area ;

[0052] (c) Semantic Location Map position ;

[0053] Figure 5 Example of steps related to LiDAR point cloud generation:

[0054] (a) Example of mobile robot pose;

[0055] (b) Example of a lidar point cloud;

[0056] (c) Example of semantic lidar point cloud;

[0057] (d) Example of semantic location point;

[0058] Figure 6 Examples of distance search and region search algorithms:

[0059] (a) Example of distance search algorithm results;

[0060] (b) Example of region search algorithm results;

[0061] Figure 7 For particle set gradient localization algorithm:

[0062] (a) Example of the initial sample particle set;

[0063] (b) Example of a sample particle set;

[0064] (c) Example of a candidate particle set;

[0065] Figure 8 This is a schematic diagram of the process of the present invention. Detailed Implementation

[0066] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0067] Example 1, see Figures 1 to 8 A method for global localization of a mobile robot in an indoor scene, comprising:

[0068] Step 1: Create a semantically meaningful LiDAR point cloud by projecting the LiDAR point cloud onto a semantic image from a visual camera;

[0069] Based on the original LiDAR point cloud, an occupancy raster map is constructed, and the location of each semantic point cloud is plotted onto the raster map to build a semantic raster map. semantic ;

[0070] Constructing a semantic location map based on semantic raster maps position and semantic region map area ;

[0071] Step 2: Based on semantic location map position and semantic region map area Using semantic lidar point clouds, distance search algorithm and region search algorithm are used to filter poses and obtain a set of particles with candidate poses;

[0072] Step 3: Randomly select an appropriate number of particles from the candidate particle set as the initial sample particle set, and perform likelihood value screening on the initial sample particle set to obtain the final sample particle set.

[0073] For each particle in the sample particle set, the gradient localization algorithm is used to iteratively calculate the localization, and the localization pose with the largest likelihood value among all localization results is taken as the global localization pose.

[0074] The method of creating semantically meaningful lidar point clouds by projecting lidar point clouds onto a semantic image from a visual camera includes:

[0075] Semantic information is obtained by processing visual images through semantic segmentation technology, and LiDAR point clouds are clustered to eliminate outliers and distinguish instances.

[0076] The clustered point cloud is projected onto the corresponding position of the semantic image using the projection matrix obtained by calibration, thereby assigning the semantic labels in the image to the LiDAR point cloud and forming point cloud data with semantic information.

[0077] The occupancy grid map constructed based on the original LiDAR point cloud and the position of each semantic point cloud are projected onto the grid map to construct a semantic grid map. semantic ,include:

[0078] The semantic consistency of point clouds is determined by setting a clustering threshold.

[0079] When the proportion of specific semantic points in a clustered point cloud exceeds a threshold, the entire point cloud is assigned a unified semantic meaning.

[0080] By combining a pre-built raster map, semantic point clouds are projected onto raster cells, and semantic annotations are performed on raster cells that meet the occupancy conditions, ultimately forming a semantic raster map.

[0081] The semantic location map constructed based on semantic raster map position and semantic region map area ,include:

[0082] Valid semantically labeled regions are filtered by setting thresholds to avoid semantically labeling unoccupied and unknown regions;

[0083] A semantic statistical filtering mechanism is used to resolve semantic boundary conflicts, and the final semantic labels are determined by calculating the proportion of semantic annotation times.

[0084] The semantic location map constructed based on semantic raster map position and semantic region map area Specifically, it also includes:

[0085] Extract various semantic regions from the semantic raster map and perform geometric simplification on them;

[0086] A semantic circular region set is obtained by calculating the minimum circumcircle, and a semantic location map is constructed.

[0087] Extract the convex hull boundary of the semantic region to form a semantic region map;

[0088] Generate a likelihood map and its gradient map based on the original raster map.

[0089] The semantic location map position and semantic region map area Using semantic lidar point clouds, distance search and region search algorithms are employed for pose selection to obtain a set of particles with candidate poses, including:

[0090] Generate a set of semantic location points based on semantic point clouds;

[0091] Candidate circular regions that meet the criteria are filtered in the semantic location map using a distance matching algorithm;

[0092] Geometric constraint methods are used to calculate the possible pose regions and angle ranges of the robot, and pose verification is performed by combining semantic region maps, ultimately forming a set of particles for candidate poses.

[0093] The step of randomly selecting an appropriate number of particles from the candidate particle set as an initial sample particle set, and then performing likelihood value screening on the initial sample particle set to obtain the final sample particle set includes:

[0094] An initial particle set is selected from the candidate pose set by random sampling. The likelihood value of each particle is calculated using a high standard deviation likelihood map. Valid particles are selected to form a sample particle set by setting a likelihood threshold.

[0095] The gradient localization algorithm is used to iteratively calculate the localization of each particle in the sample particle set, and the localization pose with the highest likelihood value among all localization results is taken as the global localization pose, including:

[0096] Gradient descent is used to iteratively optimize the sample particles, and the pose adjustment is calculated based on the likelihood gradient map so that each particle converges to a local optimum pose.

[0097] By comparing the likelihood values ​​of each candidate particle, the pose corresponding to the maximum value is selected as the final global localization result.

[0098] Example 2: A method for global localization of a mobile robot in an indoor scene, comprising:

[0099] Step 1: Create a semantically meaningful LiDAR point cloud by projecting the LiDAR point cloud onto a semantic image from a visual camera;

[0100] Based on the original LiDAR point cloud, an occupancy raster map is constructed, and the location of each semantic point cloud is plotted onto the raster map to build a semantic raster map. semantic ;

[0101] Constructing a semantic location map based on semantic raster maps position and semantic region map area ;

[0102] Step 2: Based on semantic location map position and semantic region map area Using semantic lidar point clouds, distance search algorithm and region search algorithm are used to filter poses and obtain a set of particles with candidate poses;

[0103] Step 3: Randomly select an appropriate number of particles from the candidate particle set as the initial sample particle set, and perform likelihood value screening on the initial sample particle set to obtain the final sample particle set.

[0104] For each particle in the sample particle set, the gradient localization algorithm is used to iteratively calculate the localization, and the localization pose with the largest likelihood value among all localization results is taken as the global localization pose.

[0105] The method of creating semantically meaningful lidar point clouds by projecting lidar point clouds onto a semantic image from a visual camera includes:

[0106] Semantic images are obtained by extracting semantic information from images acquired by a visual camera based on semantic segmentation methods.

[0107] To remove outliers and address the issue that semantic segmentation cannot distinguish between different instances of the same concept, the original LiDAR point cloud is clustered.

[0108] The LiDAR and visual camera are calibrated using a calibration algorithm to obtain the projection matrices of the LiDAR point cloud and the camera image.

[0109] The projection matrix projects a cluster of point clouds from the lidar point cloud onto the corresponding semantic image.

[0110] Based on the semantic information corresponding to the projection position, the semantic information in the image is assigned to the LiDAR point cloud.

[0111] The occupancy grid map constructed based on the original LiDAR point cloud and the position of each semantic point cloud are projected onto the grid map to construct a semantic grid map. semantic ,include:

[0112] Considering the errors in the projection matrix and the deviations at the boundaries of semantic regions in the semantic image, for each group of clustered point clouds, if the ratio of the number of points with specific semantic meaning in the clustered point cloud to the total number of points in the clustered point cloud exceeds the clustering threshold (Threshold), then... cluster Then all points in the clustered point cloud are assigned that semantic meaning;

[0113] For each cluster of point clouds from the lidar point cloud, a semantic point cloud is obtained by projection and semantic information assignment operations.

[0114] Occupation Raster Map Constructed Based on Original LiDAR Point Cloud grid Based on the location of each semantic point cloud, the semantic point cloud is projected onto a raster map, and semantic information is assigned to each occupied raster to construct a semantic raster map. semantic .

[0115] The semantic location map constructed based on semantic raster map position and semantic region map area ,include:

[0116] To avoid assigning semantic information to unoccupied or unknown grid cells in a raster map, it is necessary to determine whether to assign semantic information to each pixel.

[0117] If the occupancy probability of this pixel is greater than the occupancy grid threshold (Threshold) occupied Then, the corresponding semantic information is assigned to that pixel.

[0118] Otherwise, the pixel is not assigned corresponding semantic information;

[0119] Due to the ambiguity of semantic segmentation boundaries, semantic information conflicts may occur at the connection points of different semantic objects. To avoid this interference with the semantic information of the raster map, a semantic information filtering method is adopted.

[0120] On a grid map, for each pixel that has been projected with semantic information, record the number of times each piece of semantic information is projected onto that pixel;

[0121] If the ratio of the largest occurrence to the total occurrences is greater than the semantic raster threshold Then the pixel will be assigned the semantic information corresponding to the maximum number of times;

[0122] Otherwise, no semantic information is assigned to this pixel; the semantic information is calculated using the following formula:

[0123] ;

[0124] in , i=1,2,…,s represents the number of times each semantic information of the pixel corresponds to.

[0125] The semantic location map constructed based on semantic raster map position and semantic region map area Specifically, it also includes:

[0126] From semantic raster map semanti Extract each set of semantic grids;

[0127] Based on such Figure 3 Example semantic raster maps constructed from the example original raster map shown in (a) are as follows: Figure 4 As shown in (a);

[0128] The center C and radius R of the minimum circumcircle corresponding to each group of semantic graticles are calculated.

[0129] Using the center C and radius R as the center and radius of the approximate circular region of each group of semantic grids, we simplify them into semantic circular regions.

[0130] Simplify and categorize all semantic rasters to obtain a set of semantic circular regions. , where k represents the number of different semantic information types in the semantic raster map;

[0131] For a certain type of semantic circular region Ei= , where a+1 represents the index of the first circle center among all circular semantic regions with the i-th type of semantic information, and b represents the number of circular semantic regions with the i-th type of semantic information;

[0132] Save all the above semantic circular regions as a semantic region location map. position Example semantic region location diagram as follows Figure 4 As shown in (c);

[0133] For each set of semantic rasters, the corresponding convex bounding regions are extracted to obtain a semantic region map. area Example semantic region graph as follows Figure 4 As shown in (b);

[0134] To improve the efficiency and accuracy of global positioning calculations, based on the original raster map Map grid Generate a likelihood map with a high standard deviation. likelihood Example likelihood map such as Figure 3 As shown in (b);

[0135] Based on the Map likelihood Calculate the gradients of the likelihood values ​​in the x and y directions respectively, and obtain the likelihood gradient map. gradient-x and Map gradient-y The likelihood gradient maps in the x and y directions are as follows: Figure 3 As shown in (c) and (d), blue indicates a positive gradient and red indicates a negative gradient.

[0136] This invention creates a semantically information-rich LiDAR point cloud by fusing it with a visual camera's semantic image, and then constructs a semantic raster map, a semantic location map, and a semantic region map based on this. Specifically, semantic segmentation technology is used to process the visual image to obtain semantic labels, while the LiDAR point cloud is clustered to eliminate outliers and distinguish instances. The clustered point cloud is projected onto the corresponding positions in the semantic image using a calibrated projection matrix, thus assigning semantic labels from the image to the LiDAR point cloud, forming point cloud data with semantic information. Based on this, an occupancy raster map constructed from the original LiDAR point cloud and the position of each semantic point cloud are used. A clustering threshold is set to determine the semantic consistency of the point cloud. When the proportion of a specific semantic point in the clustered point cloud exceeds the threshold, the entire point cloud is assigned a unified semantic meaning. Combined with the pre-constructed raster map, the semantic point cloud is projected onto raster cells, and raster cells that meet the occupancy conditions are semantically labeled, ultimately forming a semantic raster map. Furthermore, by setting thresholds to filter effective semantically labeled regions, semantic labeling of unoccupied and unknown areas is avoided. A semantic statistical filtering mechanism is employed to resolve semantic boundary conflicts, and the final semantic labels are determined by calculating the proportion of semantic labeling frequency. Various semantic regions are extracted from the semantic grid map and geometrically simplified. A semantic location map is constructed by calculating the minimum circumcircle to obtain a set of semantic circular regions, while the convex hull boundaries of the semantic regions are extracted to form a semantic region map. The beneficial effect of this series of claims is that traditional mobile robot localization relies on grid maps constructed by LiDAR. These maps only contain distance information and lack semantic context, resulting in low localization efficiency in complex indoor environments. This invention integrates visual semantic information to endow LiDAR point clouds with rich semantic labels, thereby constructing a semantically enhanced grid map. This map not only identifies the physical structure of the environment but also labels key semantic features such as doors, windows, and furniture, providing the robot with a more comprehensive understanding of the environment. During global localization, semantic information can significantly reduce location ambiguity because semantic landmarks have unique recognition characteristics. For example, in corridors or areas with similar geometric structures, geometric information alone may not be enough to distinguish different locations. However, semantic labels such as "elevator door" or "office entrance" can serve as key anchor points, precisely constraining the robot's possible positions. Furthermore, semantic location maps and semantic region maps, through geometric simplification (such as minimum circumcircle and convex hull boundaries), transform complex semantic regions into easily tractable geometric shapes, further optimizing the location search algorithm. This semantically enhanced map allows the robot to quickly narrow down the candidate location range during global initialization, reducing unnecessary computational overhead and significantly improving localization efficiency. In large indoor scenes, this method can greatly reduce localization initialization time while improving localization success rate, especially in dynamically changing or feature-sparse environments.In summary, by introducing semantic information, this invention solves the problem of insufficient expression of traditional grid maps in semantically rich environments, laying a solid foundation for efficient global localization. This invention combines the semantic recognition function of a visual camera to propose a novel semantic grid map and its construction method. This map can express semantic information on traditional grid maps, enabling the map to express richer and more precise location information, thus helping to improve the efficiency of global localization for mobile robots.

[0137] The semantic location map position and semantic region map area Using semantic lidar point clouds, distance search and region search algorithms are employed for pose selection to obtain a set of particles with candidate poses, including:

[0138] In the example map, the pose of the mobile robot is as follows: Figure 5 As shown in (a), the blue triangle represents the mobile robot and the red point cloud represents the lidar point cloud;

[0139] Based on the single-frame lidar semantic point cloud, for each cluster of semantic point cloud, its average position point relative to the mobile robot is taken as the semantic position point N(x,y) of that cluster of semantic point cloud.

[0140] The set Z = all semantic location points n represents the number of semantic location points;

[0141] The distance of each location point from the origin of the mobile robot's coordinate system is represented by d, and these distances form a set of location points. ;

[0142] For example Figure 5 (b) shows a lidar point cloud example. Semantic clustering and semantic processing are performed to obtain a clustered semantic point cloud, as shown in the figure. Figure 5 As shown in (c), the calculated semantic location points are as follows: Figure 5 As shown in (d);

[0143] For each pair of semantic point clouds Nm and Nn, calculate the distance between them. and in the semantic region location map position Find the set of semantic circular regions Ea whose semantic information is the same as Nm and Nn respectively. and Eb= , where f+1 and g+1 represent the center numbers of the circles whose semantic information is the same as Nm and Nn respectively and whose order is first, and h and z represent the number of circular semantic regions whose semantic information is the same as Nm and Nn respectively;

[0144] distance The calculation formula is as follows:

[0145] ;

[0146] in, , This represents the coordinates of point Nm. , Represents the coordinates of point Nn;

[0147] For each semantic circular region in the semantic circular region sets Ea and Eb, a filtering process is performed, and if the conditions are met, the circular region is retained.

[0148] Otherwise, the circular region is not retained, and the filtering formula is as follows:

[0149] ;

[0150] in, , Indicates the two centers and The distance between them The center of each circle and The radius of the corresponding circular region;

[0151] All circular regions that meet the conditions are retained to form a set of circular regions E. can ;

[0152] In semantic raster map semantic At position I corresponding to each pixel, for each semantic location point N, determine the distance from position I to the circular region set E. can Is the difference between the distance d between the center point of any semantic circular region corresponding to the semantic location and the distance d between the two semantic locations less than the radius of the circular region? The formula for judgment is as follows:

[0153] ;

[0154] in, , Indicate position I and The distance between them;

[0155] If the condition is met for all semantic location points, then location I is within the location area of ​​the mobile robot.

[0156] Using the above method in semantic raster maps semantic A global search is performed to calculate the range of the mobile robot's location, and the angle range is calculated for each location within the range. The formula for calculating the range is as follows:

[0157] ;

[0158] in R represents the distance from the center C to position I, and R corresponds to the radius of the semantically circular region. and This represents the x and y coordinates of the center of the corresponding semantically circular region. This indicates the deflection angle of the mobile robot at position I;

[0159] At position I, for each semantic location point, within the semantic circular region set E can The corresponding semantic circular regions within each region are all calculated for their angle range, and together they form the angle range for that position.

[0160] The calculated location range of the mobile robot in the example is as follows: Figure 6 As shown in (a), the white area represents the range of the mobile robot's location;

[0161] Within the obtained location and angle ranges, a search is performed at each location with a fixed angular resolution to determine whether the semantic location point in that pose falls within the semantic region map. area Within their respective semantic regions;

[0162] If all semantic location points fall within the corresponding semantic region, then the pose is saved into the set of candidate poses;

[0163] Each pose is treated as a particle to obtain a set of candidate particles. In the example, the calculated set of candidate particles is as follows: Figure 6 As shown in (b), the red dots represent candidate particles;

[0164] This invention, based on semantic location maps and semantic region maps, utilizes semantic LiDAR point clouds and employs distance search and region search algorithms for pose selection to obtain a particle set of candidate poses. Specifically, it includes generating a set of semantic location points based on the semantic point cloud; selecting candidate circular regions that meet certain conditions in the semantic location map using a distance matching algorithm; calculating the possible pose regions and angle ranges of the robot using geometric constraint methods; and verifying the pose using the semantic region map, ultimately forming a particle set of candidate poses. The beneficial effect of this claim is that in global localization of mobile robots, traditional methods such as Monte Carlo localization typically require a large number of particles to cover the entire state space to ensure convergence to the correct position, which leads to a heavy computational burden, especially in large indoor scenes. This invention, through a semantically guided pose selection mechanism, efficiently narrows down the range of candidate positions. The distance search algorithm uses circular regions represented in the semantic location map to quickly match the current semantic point cloud with semantic landmarks in the map, selecting candidate regions with consistent distances. The region search algorithm further utilizes the convex hull boundary of the semantic region map to perform geometric verification of the candidate poses, ensuring they conform to environmental constraints. This two-stage screening strategy significantly reduces the number of particles to be processed, from tens of thousands to hundreds, thereby greatly reducing computational complexity. For example, in a warehouse environment, semantic point clouds identify multiple shelves and aisles. Distance search algorithms can quickly eliminate areas far from these landmarks, while region search algorithms verify whether the robot is within a walkable area. This not only improves computational efficiency but also avoids the risk of particle swarms converging to incorrect positions. Compared with traditional methods, this algorithm shortens computation time while maintaining the same positioning accuracy, enabling the robot to respond to environmental changes in real time. Furthermore, this method is robust and has a certain tolerance to sensor noise and dynamic environmental changes because it relies on the stability of semantic information rather than simple geometric features. Therefore, this invention solves the problem of low computational efficiency of particle filtering in large-scale scenes through intelligent pose screening, providing a feasible solution for real-time global localization.

[0165] The step of randomly selecting an appropriate number of particles from the candidate particle set as an initial sample particle set, and then performing likelihood value screening on the initial sample particle set to obtain the final sample particle set includes:

[0166] To improve computational efficiency, the candidate poses obtained above are randomly selected, and an appropriate number of particles are chosen as the initial sample particle set. The initial sample particle set calculated in the example is as follows: Figure 7 As shown in (a);

[0167] For the initial sample particle set, for each pose in the high standard deviation likelihood map (Map)... likelihood The likelihood value is calculated, and poses with a likelihood value less than the likelihood threshold are deleted using a filtering formula.

[0168] The selection formula is as follows:

[0169] ;

[0170] in Let be the likelihood function. For the robot's pose, x represents the x-axis coordinate, y represents the y-axis coordinate, and φ represents the heading angle coordinate. This represents the likelihood threshold.

[0171] The sample particle set is composed of all the new candidate poses obtained after filtering and updating. The sample particle set calculated in the example is as follows: Figure 7 As shown in (b).

[0172] The gradient localization algorithm is used to iteratively calculate the localization of each particle in the sample particle set, and the localization pose with the highest likelihood value among all localization results is taken as the global localization pose, including:

[0173] Each particle in the sample particle set is used as the initial pose, and the local maximum likelihood value is iteratively calculated using the gradient descent method based on the likelihood map.

[0174] The update formula is as follows:

[0175] ;

[0176] in, , Let be the likelihood function. Find the function The minimum value can be used to obtain the likelihood function. The maximum value;

[0177] Likelihood function The calculation formula is as follows:

[0178] ;

[0179] in LiDAR point cloud aggregation The coordinates of the i-th point in the point cloud, where M represents the number of particles in the lidar point cloud. This indicates the robot's pose. The likelihood value at the location of the i-th lidar point;

[0180] Based on this likelihood map, generate likelihood gradient maps in the x and y directions. gradient-x and Map gradient-y , respectively and Differentiating the J formula yields:

[0181] ;

[0182] Taking the partial derivatives with respect to x, y, and y respectively, we get:

[0183] ;

[0184] ;

[0185] ;

[0186] in They represent Partial derivatives in the x, y, and y directions, These represent the lidar points. The gradient of the likelihood value at a given location in the x and y directions. Indicates the angular resolution of the lidar;

[0187] The update formulas for x, y, and y are as follows:

[0188] ;

[0189] in , representing the magnitude of the gradient in the x, y plane. This represents the average distance of the lidar point cloud. This represents the convergence threshold.

[0190] The above method is used to calculate the likelihood of each particle in the initial sample particle set. Each particle converges to a local maximum of its likelihood value, and each local maximum pose is used as a candidate particle. In the example, the candidate particle set calculated is as follows: Figure 7 As shown in (c);

[0191] The likelihood values ​​of each candidate particle are compared, and the pose of the particle with the highest likelihood value is taken as the global localization result.

[0192] This invention randomly selects an appropriate number of particles from a candidate particle set as an initial sample particle set. The initial sample particle set is then filtered by likelihood value to obtain a final sample particle set. Each particle in the sample particle set is then iteratively located using a gradient descent algorithm. The pose with the highest likelihood value among all localization results is taken as the global localization pose. Specifically, this involves selecting an initial particle set from the candidate pose set through random sampling, calculating the likelihood value of each particle using a high standard deviation likelihood map, and filtering effective particles to form a sample particle set by setting a likelihood threshold. Subsequently, the sample particles are iteratively optimized using gradient descent, and the pose adjustment is calculated based on the likelihood gradient map, causing each particle to converge to a local optimum pose. Finally, by comparing the likelihood values ​​of each candidate particle, the pose corresponding to the maximum value is selected as the final global localization result. The beneficial effect of this claim lies in addressing a key challenge of global localization: how to avoid getting trapped in local optima while maintaining accuracy. Traditional Monte Carlo localization methods rely on random propagation and resampling of particle swarms, which can easily converge to incorrect positions in complex environments and involve large computational costs. This invention introduces a gradient localization algorithm combined with semantically guided particle filtering to achieve efficient and robust accurate localization. First, low-probability particles are removed from the candidate particle set through likelihood value filtering, ensuring the initial sample set has high potential and reducing unnecessary computation. Then, gradient descent is applied to each sample particle, iteratively adjusting its pose using a likelihood gradient map to quickly converge to a local extremum. This method is similar to a refined local search, but based on a global filtering starting point, it avoids the sensitivity to initial values ​​inherent in traditional gradient descent. Since gradient descent directly utilizes the gradient information of the map, the localization process is more efficient, typically requiring only a few iterations to achieve centimeter-level accuracy. Furthermore, by comparing the likelihood values ​​of multiple candidate particles and selecting the global maximum, interference from local optima is effectively prevented. In environments lacking obvious geometric features, such as wide corridors or symmetrical rooms, this method can accurately identify the robot's true position, while significantly improving computational efficiency because the iterative computation of gradient descent is far less than the large-scale sampling of particle filtering. Overall, this invention combines global search with local optimization, utilizing both global guidance from semantic information and the precise convergence capability of gradient descent. This solves the problem of unrobust localization in complex indoor scenes using traditional methods, providing reliable position awareness for mobile robots.

[0193] In the algorithm described above, step 1 is the map preprocessing step. For the same scene, the mobile robot only needs to perform this step once during global localization. The overall framework of the map preprocessing step is as follows: Figure 1 As shown. After completing step 1, the mobile robot only needs to perform steps 2 and 3 each time it performs global localization. The overall framework for the mobile robot's global localization is as follows. Figure 2 As shown.

[0194] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus.

[0195] The above description is only a preferred embodiment of the present invention. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the technical principles of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.

Claims

1. A method for global localization of a mobile robot in an indoor scene, characterized in that, include: Step 1: Create a semantically meaningful LiDAR point cloud by projecting the LiDAR point cloud onto a semantic image from a visual camera; Based on the original LiDAR point cloud, an occupancy raster map is constructed, and the location of each semantic point cloud is plotted onto the raster map to build a semantic raster map. semantic ; Constructing a semantic location map based on semantic raster maps position and semantic region map area ; Step 2: Based on semantic location map position and semantic region map area Using semantic lidar point clouds, distance search algorithm and region search algorithm are used to filter poses and obtain a set of particles with candidate poses; Step 3: Randomly select an appropriate number of particles from the candidate particle set as the initial sample particle set, and perform likelihood value screening on the initial sample particle set to obtain the final sample particle set. For each particle in the sample particle set, the gradient localization algorithm is used to iteratively calculate the localization, and the localization pose with the largest likelihood value among all localization results is taken as the global localization pose.

2. The method for global localization of a mobile robot in an indoor scene according to claim 1, characterized in that: The method of creating semantically meaningful lidar point clouds by projecting lidar point clouds onto a semantic image from a visual camera includes: Semantic information is obtained by processing visual images through semantic segmentation technology, and LiDAR point clouds are clustered to eliminate outliers and distinguish instances. The clustered point cloud is projected onto the corresponding position of the semantic image using the projection matrix obtained by calibration, thereby assigning the semantic labels in the image to the LiDAR point cloud and forming point cloud data with semantic information.

3. The method for global localization of a mobile robot in an indoor scene according to claim 1, characterized in that: The occupancy grid map constructed based on the original LiDAR point cloud and the position of each semantic point cloud are projected onto the grid map to construct a semantic grid map. semantic ,include: The semantic consistency of point clouds is determined by setting a clustering threshold. When the proportion of specific semantic points in a clustered point cloud exceeds a threshold, the entire point cloud is assigned a unified semantic meaning. By combining a pre-built raster map, semantic point clouds are projected onto raster cells, and semantic annotations are performed on raster cells that meet the occupancy conditions, ultimately forming a semantic raster map.

4. The method for global localization of a mobile robot in an indoor scene according to claim 1, characterized in that: The semantic location map constructed based on semantic raster map position and semantic region map area ,include: Valid semantically labeled regions are filtered by setting thresholds to avoid semantically labeling unoccupied and unknown regions; A semantic statistical filtering mechanism is used to resolve semantic boundary conflicts, and the final semantic labels are determined by calculating the proportion of semantic annotation times.

5. The method for global localization of a mobile robot in an indoor scene according to claim 1, characterized in that: The semantic location map constructed based on semantic raster map position and semantic region map area Specifically, it also includes: Extract various semantic regions from the semantic raster map and perform geometric simplification on them; A semantic circular region set is obtained by calculating the minimum circumcircle, and a semantic location map is constructed. Extract the convex hull boundary of the semantic region to form a semantic region map; Generate a likelihood map and its gradient map based on the original raster map.

6. The method for global localization of a mobile robot in an indoor scene according to claim 1, characterized in that: The semantic location map position and semantic region map area Using semantic lidar point clouds, distance search and region search algorithms are employed for pose selection to obtain a set of particles with candidate poses, including: Generate a set of semantic location points based on semantic point clouds; Candidate circular regions that meet the criteria are filtered in the semantic location map using a distance matching algorithm; Geometric constraint methods are used to calculate the possible pose regions and angle ranges of the robot, and pose verification is performed by combining semantic region maps, ultimately forming a set of particles for candidate poses.

7. The method for global localization of a mobile robot in an indoor scene according to claim 1, characterized in that: The step of randomly selecting an appropriate number of particles from the candidate particle set as an initial sample particle set, and then performing likelihood value screening on the initial sample particle set to obtain the final sample particle set includes: An initial particle set is selected from the candidate pose set by random sampling. The likelihood value of each particle is calculated using a high standard deviation likelihood map. Valid particles are selected to form a sample particle set by setting a likelihood threshold.

8. The method for global localization of a mobile robot in an indoor scene according to claim 1, characterized in that: The gradient localization algorithm is used to iteratively calculate the localization of each particle in the sample particle set, and the localization pose with the highest likelihood value among all localization results is taken as the global localization pose, including: Gradient descent is used to iteratively optimize the sample particles, and the pose adjustment is calculated based on the likelihood gradient map so that each particle converges to a local optimum pose. By comparing the likelihood values ​​of each candidate particle, the pose corresponding to the maximum value is selected as the final global localization result.

Images