Mobile robot positioning method based on multi-source environment perception and venue semantic partitioning

Abstract

The application discloses a mobile robot positioning method and system based on multi-source environment perception and site semantic zoning, a computer device and a storage medium. The method comprises the following steps: establishing a global coordinate system according to the standard size specification of a work site, and pre-dividing a physical space into a plurality of logical semantic areas to construct a site semantic zoning model; acquiring environment data in real time through an environment perception sensor and extracting real-time site features; matching the real-time site features with standard skeleton features in the model to determine the logical semantic area where the robot is currently located; calling the corresponding standard skeleton feature geometric parameters as absolute position constraints according to the current area, calculating the pose deviation between the real-time site features and the standard skeleton features, and correcting the positioning coordinates of the robot in the global coordinate system by using the pose deviation. The application eliminates the cumulative drift by using the inherent fixed geometric constraints of the site, and realizes high-precision drift-free positioning under long-time operation.

Description

Technical Field

[0001] This application relates to the field of mobile robot localization technology, specifically a mobile robot localization method, system, computer equipment, and storage medium based on multi-source environmental perception and site semantic partitioning. Background Technology

[0002] With the widespread application of mobile robot technology in sports training, venue cleaning, and competition assistance, the demand for high-precision real-time positioning of robots in standardized work environments (such as tennis courts and peak courts) is becoming increasingly urgent. Currently, the mainstream positioning methods mainly include Global Positioning System (GPS) and Simultaneous Localization and Mapping (SLAM) technology. However, GPS suffers from severe signal attenuation indoors or in obstructed environments, making it difficult to meet positioning accuracy requirements. Meanwhile, the single SLAM method is prone to odometry drift in open, similar-textured sports fields due to a lack of salient features. Furthermore, the cumulative error is further amplified by dynamic interference such as personnel movement, leading to a significant decrease in robot positioning accuracy over time, failing to meet the requirements for long-term stable operation. Summary of the Invention

[0003] Therefore, it is necessary to provide a mobile robot localization method, system, computer equipment, and storage medium based on multi-source environmental perception and site semantic partitioning that can eliminate cumulative drift and achieve high-precision positioning, in order to address the above-mentioned technical problems.

[0004] Firstly, a mobile robot localization method based on multi-source environmental perception and site semantic partitioning is provided, the method comprising: Based on the standard size specifications corresponding to the mobile robot's work site, a global coordinate system is established, and the physical space of the work site is pre-divided into multiple logical semantic regions with unique geometric coordinate ranges to construct a site semantic partitioning model. The site semantic partitioning model stores the geometric parameters of the standard skeleton features contained in each logical semantic region. The standard skeleton features include line segments or points with geometric parameters. By using environmental perception sensors mounted on a mobile robot, environmental data of the surrounding environment can be acquired in real time, and real-time site features can be extracted from the environmental data. The extracted real-time site features are matched with the standard skeleton features of each logical semantic region in the site semantic partitioning model to determine the logical semantic region where the mobile robot is currently located. Based on the current logical semantic region, the geometric parameters of the standard skeleton feature corresponding to that region in the site semantic partitioning model are retrieved as absolute position constraints. The pose deviation between the real-time site feature and the standard skeleton feature is calculated, and the pose deviation is used to correct the positioning coordinates of the mobile robot in the global coordinate system.

[0005] In one embodiment, correcting the positioning coordinates of the mobile robot in the global coordinate system using the pose deviation includes: The calculated pose deviation is decomposed into rotational and translational components; The rotation component and the translation component are respectively input to the angular velocity Kalman filter and the acceleration Kalman filter for filtering. The process noise covariance matrix of the angular velocity Kalman filter is set according to the angular velocity measurement noise of the mobile robot, and the process noise covariance matrix of the acceleration Kalman filter is set according to the acceleration measurement noise of the mobile robot. The filtered rotation and translation components are superimposed on the current positioning coordinates of the mobile robot to generate the corrected positioning coordinates.

[0006] In one embodiment, determining the logical semantic region where the mobile robot is currently located includes: The extracted real-time site features are used to construct a local topology map, which includes the relative distances and connectivity relationships between features. The local topology map is matched with the preset topology map of each region in the site semantic partitioning model, and the region with the highest matching degree is selected as the current logical semantic region.

[0007] In one embodiment, calculating the pose deviation between the real-time site features and the standard skeleton features includes: For the feature elements in the real-time site features, point-to-point matching is performed between the endpoint coordinates of the line segments extracted from the feature elements and the endpoint coordinates of the corresponding line segments in the standard skeleton features to obtain the local pose deviation corresponding to the feature element; wherein, if no line segment endpoints are extracted from a feature element, the local pose deviation corresponding to the feature element is obtained by least squares fitting based on the point cloud or image data corresponding to the observable line segment portion in the feature element, and based on the known line segment length and direction constraints in the standard skeleton features. The pose deviation is determined based on the number of feature elements in the real-time site features and the local pose deviation corresponding to each feature element.

[0008] In one embodiment, determining the pose deviation based on the number of feature elements in the real-time site features and the local pose deviation corresponding to each feature element includes: When the real-time site features contain multiple feature elements, the local pose deviations corresponding to each feature element are weighted and fused according to the confidence level of each feature element to obtain the pose deviation; wherein the confidence level is pre-calibrated based on the measurement noise of the environmental perception sensor, the observation completeness of the feature element, and the degree of dynamic occlusion. When there is only one feature element in the real-time site features, the local pose deviation corresponding to that feature element is taken as the pose deviation.

[0009] In one embodiment, the environmental perception sensor includes a lidar and a vision sensor; determining the logical semantic region where the mobile robot is currently located includes: The line features extracted by the LiDAR and the corner features extracted by the visual sensor are fused at the feature level to construct a fused real-time site feature vector. The real-time site feature vector is then matched with the standard feature vector of the corresponding region in the site semantic partitioning model to determine the logical semantic region where the mobile robot is currently located.

[0010] In one embodiment, the method further includes: Odometer data measured by a wheeled odometer mounted on a mobile robot, as well as angular velocity and acceleration data measured by an inertial measurement unit, are acquired. A filtering algorithm is then used to fuse the odometer data, the angular velocity data, the acceleration data, and the corrected positioning coordinates to update the positioning coordinates of the mobile robot in the global coordinate system.

[0011] Secondly, a mobile robot localization system based on multi-source environmental perception and site semantic partitioning is provided, the system comprising: The semantic modeling module is used to establish a global coordinate system based on the standard size specifications corresponding to the working site of the mobile robot, and to pre-divide the physical space of the working site into multiple logical semantic regions with unique geometric coordinate ranges to construct a site semantic partitioning model. The site semantic partitioning model stores the geometric parameters of the standard skeleton features contained in each logical semantic region. The standard skeleton features include line segments or points with geometric parameters. The environmental perception module is used to acquire environmental data of the surrounding environment in real time through environmental perception sensors mounted on the mobile robot, and extract real-time site features from the environmental data. The region discrimination module is used to match the extracted real-time site features with the standard skeleton features of each logical semantic region in the site semantic partitioning model to determine the logical semantic region where the mobile robot is currently located. The positioning correction module is used to retrieve the geometric parameters of the standard skeleton features corresponding to the current logical semantic region as absolute position constraints in the site semantic partitioning model, calculate the pose deviation between the real-time site features and the standard skeleton features, and use the pose deviation to correct the positioning coordinates of the mobile robot in the global coordinate system.

[0012] Thirdly, a computer device is provided, including a memory and a processor, wherein the memory is communicatively connected to the processor, and the memory stores a computer program that can run on the processor, wherein when the processor executes the computer program, it implements the mobile robot localization method based on multi-source environmental perception and site semantic partitioning as described above.

[0013] Fourthly, a computer-readable storage medium is provided, on which a computer program is stored, wherein when the computer program is executed by a processor, it implements the mobile robot localization method based on multi-source environmental perception and site semantic partitioning as described above.

[0014] The aforementioned mobile robot localization method, system, computer equipment, and storage medium based on multi-source environmental perception and site semantic partitioning establish a global coordinate system and pre-divide logical semantic regions according to the standard size specifications of the work site. This constructs a site semantic partitioning model containing standard skeleton feature geometric parameters, providing a priori reference with absolute constraints for subsequent localization. Environmental data is acquired in real time by environmental perception sensors, and real-time site features are extracted. These real-time site features are matched with the standard skeleton features in the model to determine the current logical semantic region, achieving coarse localization and region identification of the robot. Based on the discrimination result, the standard skeleton feature geometric parameters of the corresponding region are retrieved as absolute position constraints. The pose deviation between the real-time site features and the standard skeleton features is calculated, and the robot's global positioning coordinates are corrected accordingly. This allows the robot to utilize the inherent fixed geometric constraints of the site to eliminate the cumulative drift of the odometry, thereby achieving high-precision, drift-free localization over long periods in indoor environments where GPS signals are missing or attenuated, and in scenarios with dynamic interference. Attached Figure Description

[0015] Figure 1 This is an application environment diagram of a mobile robot localization method based on multi-source environmental perception and site semantic partitioning in one embodiment; Figure 2 This is a flowchart illustrating a mobile robot localization method based on multi-source environmental perception and site semantic partitioning in one embodiment. Figure 3 This is a structural block diagram of a mobile robot localization system based on multi-source environmental perception and site semantic partitioning in one embodiment; Figure 4 This is an internal structural diagram of a computer device in one embodiment. Detailed Implementation

[0016] To facilitate understanding of the technical solutions provided in the embodiments of this application, the background technology involved in the embodiments of this application will be described below.

[0017] With the widespread application of mobile robot technology in sports training, venue cleaning, and competition assistance, the demand for high-precision real-time positioning of robots in standardized work environments is becoming increasingly urgent. Taking tennis courts and peak performance courts as examples, these venues have clearly defined international standard dimensions and fixed functional zones. Robots need to operate stably within these areas for extended periods to complete predetermined tasks. Currently, the mainstream positioning methods mainly include the Global Positioning System (GPS) and real-time positioning and mapping (RTM) technology.

[0018] While the Global Positioning System (GPS) provides relatively stable positioning services in open outdoor environments, standardized work sites often include indoor venues or areas with obstructed ceilings. In these conditions, GPS signals attenuate significantly, leading to a sharp drop in positioning accuracy and failing to meet the requirements for continuous robot operation. Simultaneous localization and mapping (SLAM) technology, which does not rely on external signals, perceives the environment through sensors and simultaneously builds a map and estimates its own position, performing well in typical indoor environments. However, in standardized venues such as tennis courts and peak courts, two prominent problems arise: first, the surface textures are highly similar, such as large areas of uniform ground and repetitive white line patterns, making it difficult for SLAM algorithms to extract effective feature points and easily leading to feature mismatches; second, in training or competition scenarios, dynamic interference such as personnel movement and ball movement frequently occur, and these dynamic objects are misclassified as static environmental features, further exacerbating the cumulative drift of the odometry. These factors combined cause the positioning error of traditional SLAM methods to gradually increase over time when running in standardized venues for extended periods, eventually exceeding the allowable accuracy range for the operation.

[0019] To address the aforementioned issues, this application provides a mobile robot localization method, system, computer device, and storage medium based on multi-source environmental perception and site semantic zoning. By utilizing the inherent dimensional specifications and functional zoning of a standardized site, a semantic model with absolute constraints is constructed. Real-time perceived environmental features are matched with the model for deviation correction, achieving drift-free and precise localization over extended periods.

[0020] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description, in conjunction with the accompanying drawings and embodiments, further illustrates this application. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application. Furthermore, the data collection, processing, and use involved in this application are all conducted under the premise of obtaining relevant authorizations and complying with laws and regulations, ensuring that personal privacy and legitimate rights are not infringed.

[0021] The mobile robot localization method based on multi-source environmental perception and site semantic partitioning provided in this application can be applied to, for example... Figure 1 In the application environment shown, the mobile robot 102 communicates with the server 104 via a network. The mobile robot 102 is equipped with environmental perception sensors to acquire environmental data of its surroundings in real time and perform steps such as real-time site feature extraction, matching with a site semantic partitioning model, identification of the current logical semantic region, pose deviation calculation, and positioning coordinate correction. The server 104 can be used to store the site semantic partitioning model pre-built according to standard size specifications and to provide necessary computing resource support or data synchronization services. The mobile robot 102 can be various types of mobile robot platforms, such as a court training robot, a site cleaning robot, or a competition assistance robot. The server 104 can be implemented using a standalone server or a server cluster consisting of multiple servers.

[0022] Firstly, in the process of mobile robot localization, traditional methods are prone to cumulative drift in standardized venues with similar textures (such as tennis courts and peak courts), making it difficult to meet the requirements of long-term high-precision localization.

[0023] Therefore, in one embodiment, a mobile robot localization method based on multi-source environmental perception and site semantic partitioning is provided, which is then applied to... Figure 1 Let's take the mobile robot in the example of this as an illustration. Figure 2 As shown, the method specifically includes the following steps: Step S1: Based on the standard size specifications corresponding to the mobile robot's work site, establish a global coordinate system and pre-divide the physical space of the work site into multiple logical semantic regions with unique geometric coordinate ranges to construct a site semantic partitioning model; the site semantic partitioning model stores the geometric parameters of the standard skeleton features contained in each logical semantic region, and the standard skeleton features include line segments or points with geometric parameters.

[0024] In this step, a standard tennis court is used as an example, with official dimensions of 23.77 meters × 10.97 meters and fixed positions for the baseline, sidelines, etc. In this embodiment, a global coordinate system is established with one corner of the court as the origin, the baseline direction as the X-axis, the sideline direction as the Y-axis, and the vertical upward direction as the Z-axis. Then, the physical space of the court is pre-divided into multiple logical semantic regions, such as the service area, baseline area, buffer zone, and sideline area, each with a unique geometric coordinate range. For example, the baseline area can be defined as the area extending a certain distance from the baseline into the court. After these regions are divided, a semantic partitioning model of the court is constructed. This model stores the geometric parameters of the standard skeleton features contained in each region. Standard skeleton features refer to line segments or points with fixed geometric shapes in the court; for example, the baseline is a line segment of fixed length (23.77 meters), and the center point is a fixed point. Geometric parameters include the coordinates of the line segment endpoints in the global coordinate system, the line segment length, direction angle, and point coordinates. These parameters are directly derived from the standard dimensions of the field. For example, the coordinates of the two endpoints of the baseline are (0,0,0) and (23.77,0,0).

[0025] Step S2: Use environmental perception sensors mounted on the mobile robot to acquire environmental data of the surrounding environment in real time, and extract real-time site features from the environmental data.

[0026] In this step, the environmental perception sensor can be any one or more of the existing technologies. In this embodiment, it can be, for example, a lidar and a vision sensor. From the point cloud data acquired by the lidar, high-reflectivity white line point clouds can be extracted through reflectivity threshold segmentation. For example, point clouds with reflectivity higher than a preset threshold (e.g., 80%) are marked as candidate points. From the image data acquired by the vision sensor, white line edges and corners can be extracted using edge detection algorithms (e.g., the Canny operator) and Hough transform. Real-time site features are extracted from these data, such as the coordinates of the endpoints of detected line segments, line segment length, and direction. In this embodiment, real-time site features are represented in the form of feature elements, each corresponding to a perceived line segment or point. For example, if a portion of the baseline is perceived, a line segment feature element is formed, containing the endpoint coordinates and direction of the line segment in the sensor coordinate system.

[0027] Step S3: Match the extracted real-time site features with the standard skeleton features of each logical semantic region in the site semantic partitioning model to determine the logical semantic region where the mobile robot is currently located.

[0028] In this step, the matching process between real-time site features and standard skeleton features can employ feature descriptors or geometric constraints. For example, for line segments extracted in real-time, their length and direction are calculated and compared with the line segment features of each region in the model to find the best-matching region. The purpose of matching is to determine which logical semantic region the robot is currently located in. In this embodiment, this can be achieved by calculating the distance and angular similarity between real-time line segments and standard line segments.

[0029] Step S4: Based on the current logical semantic region, retrieve the geometric parameters of the standard skeleton features corresponding to that region in the site semantic partitioning model as absolute position constraints, calculate the pose deviation between the real-time site features and the standard skeleton features, and use the pose deviation to correct the positioning coordinates of the mobile robot in the global coordinate system.

[0030] In this step, for example, if it is determined that the robot is currently located in the baseline area, the coordinates and length of the standard endpoints of the baseline are retrieved. Then, the real-time site features (such as the perceived baseline segments) are compared with the standard baseline to calculate the pose deviation. The pose deviation includes position deviation (translation vector) and attitude deviation (rotation vector). The pose deviation can be calculated using a point set registration algorithm, such as the iterative nearest point algorithm, which obtains the rotation matrix and translation vector by matching the endpoints of real-time line segments with the endpoints of standard line segments. After obtaining the pose deviation, the robot's current positioning coordinates (e.g., coordinates given by the odometry) are added to this deviation to obtain the corrected global positioning coordinates.

[0031] Based on the above, this method first establishes a semantic zoning model of the site based on standard size specifications, providing a priori benchmark with absolute constraints for subsequent localization, overcoming the limitations of traditional methods that rely on relative estimation due to a lack of prior information. Secondly, by extracting site features in real time using environmental perception sensors and matching them with the model, coarse localization and region identification of the robot's position are achieved, solving the problem that single SLAM methods struggle to effectively localize in environments with similar textures. Finally, using the standard skeleton features corresponding to the identified regions as absolute position constraints, pose deviations are calculated and corrected, enabling the robot to reverse-correct accumulated drift using the inherent fixed geometric constraints of the site (such as the fixed length of the baseline and the perpendicular relationship of the edges). This achieves high-precision, drift-free localization over long periods, even in indoor environments where positioning signals are missing or attenuated, and in scenarios with dynamic interference such as people moving around.

[0032] After obtaining the pose deviation, directly superimposing it may introduce measurement noise; therefore, filtering of the deviation is necessary to improve correction accuracy. In one embodiment, the pose deviation is used to correct the positioning coordinates of the mobile robot in the global coordinate system, specifically including the following steps: The calculated pose deviation is decomposed into rotational and translational components. The rotational and translational components are respectively input to the angular velocity Kalman filter and the acceleration Kalman filter for filtering. The process noise covariance matrix of the angular velocity Kalman filter is set according to the angular velocity measurement noise of the mobile robot, and the process noise covariance matrix of the acceleration Kalman filter is set according to the acceleration measurement noise of the mobile robot. The filtered rotation and translation components are superimposed on the current positioning coordinates of the mobile robot to generate the corrected positioning coordinates.

[0033] Specifically, the rotation matrix can be converted into a rotation angle or a rotation vector, and the translation vector is the translation component. In this embodiment, the rotation component is represented as a rotation angle, and the translation component is represented as a translation vector. An angular velocity Kalman filter is used to filter the rotation component; its state vector can be the rotation angle, and the observed value is the rotation component calculated by feature matching. The process noise covariance matrix of the filter is set according to the angular velocity measurement noise of the mobile robot, which can be obtained from IMU calibration data. Similarly, an acceleration Kalman filter is used for the translation component, and its process noise covariance matrix is ​​set according to the acceleration measurement noise. The prediction and update equations of the Kalman filter can be implemented using standard forms in the prior art, such as predicting by establishing a constant velocity motion model and correcting using the observed values. The filtered rotation and translation components are smoother, reducing the influence of measurement noise. Finally, the filtered rotation and translation are superimposed on the current positioning coordinates to obtain the corrected positioning coordinates.

[0034] Based on the above, by using separate filtering methods, it is possible to independently optimize different dynamic characteristics of rotation and translation, avoid coupling interference, improve the accuracy of correction, and effectively suppress the disturbance of measurement noise on the positioning results.

[0035] When determining the logical semantic region where a robot is currently located, traditional methods may rely solely on the position of a single feature, making them susceptible to local interference or missing features. Therefore, in one embodiment, determining the logical semantic region where the mobile robot is currently located specifically includes the following steps: The extracted real-time site features are used to construct a local topology map, which includes the relative distances and connectivity relationships between features. The local topology map is matched with the preset topology maps of each region in the site semantic partitioning model, and the region with the highest matching degree is selected as the current logical semantic region.

[0036] Specifically, feature elements (such as line segments and points) extracted from real-time site features constitute nodes, and the relative distances and connections between nodes constitute edges. The construction of the local topology graph can be achieved using any existing method, such as establishing connections between feature points based on Delaunay triangulation, or establishing adjacency relationships based on feature distance thresholds. Each node contains the geometric attributes of the feature elements (such as line segment length and direction), and each edge contains the distance and angle between two feature elements. In the site semantic partitioning model, each logical semantic region also predefines its standard skeleton feature topology graph. During matching, the similarity between the local topology graph and each predefined topology graph is calculated. Similarity calculation can use existing graph matching algorithms, such as graph edit distance-based algorithms or spectral clustering-based graph matching methods, to obtain a matching score. The region with the highest score is selected as the current region.

[0037] Based on the above, by utilizing the geometric relationships between features, even if some features are occluded or missing, as long as the retained topological structure is sufficiently distinguishable, the region can still be correctly identified, thereby improving the robustness of region identification and laying the foundation for subsequent invocation of accurate absolute constraints.

[0038] When calculating the pose deviation between real-time site features and standard skeleton features, it is necessary to handle cases where feature data is incomplete, such as occluded line segment endpoints. Therefore, in one embodiment, calculating the pose deviation between real-time site features and standard skeleton features specifically includes the following steps: For feature elements in real-time site features, point-to-point matching is performed between the endpoint coordinates of the line segments extracted from the feature elements and the endpoint coordinates of the corresponding line segments in the standard skeleton features to obtain the local pose deviation corresponding to the feature elements. If no line segment endpoints are extracted from a feature element, the local pose deviation corresponding to the feature element is obtained by fitting the point cloud or image data corresponding to the observable line segment portion in the feature element, based on the known line segment length and direction constraints in the standard skeleton features, using the least squares method. The pose deviation is determined based on the number of feature elements in the real-time site features and the local pose deviation corresponding to each feature element.

[0039] Specifically, for each feature element in the real-time site features, let it be denoted as the i-th. Feature elements First, attempt to extract the coordinates of the line segment endpoints. If two endpoints can be extracted, perform point-to-point matching between the coordinates of the line segment endpoints and the corresponding line segment endpoint coordinates in the standard skeleton features. Point-to-point matching can be achieved using point set registration algorithms, such as the iterative nearest-point algorithm, to calculate the rotation matrix. Translation vector This represents the local pose deviation corresponding to the feature element.

[0040] If, due to occlusion or sensor noise, the complete endpoint cannot be extracted (e.g., only a portion of the point cloud or image pixels in the middle of the line segment are extracted), then the translation deviation is solved using the point cloud or image data of the observable portion, combined with the known length and orientation constraints of the standard line segment, and employing a least-squares fitting method. Let the orientation of the standard line segment be a unit vector. The center point of the standard line segment is The set of points observed in real time is These points should fall on or near the standard line segment. The fitted line equation is: ,in Let be a point on the straight line. Let be scalar parameters. The least squares objective function is: in By solving this optimization problem, we can obtain... The optimal estimate. Center point of standard line segment By comparison, the translation vector is obtained. The rotational deviation is considered zero in this case. This yields the local pose deviation corresponding to the feature element, i.e., the translation vector. .

[0041] After obtaining the local pose deviations corresponding to all feature elements, the final pose deviation needs to be determined by combining them. If there is only one feature element, then that local pose deviation is the final pose deviation. If there are multiple feature elements, then subsequent fusion processing is required.

[0042] Based on the above, precise point-to-point matching is used when the endpoint information is complete, and fitting is performed using known geometric constraints when the endpoints are missing. This enables the effective use of incomplete observation data, improves the robustness and accuracy of pose deviation calculation, and further enhances the reliability of positioning correction.

[0043] When multiple feature elements exist, the reliability of the local pose deviation provided by each feature element may vary due to differences in observation quality. Therefore, weighted fusion based on confidence level is required. In one embodiment, the pose deviation is determined based on the number of feature elements in the real-time site features and the local pose deviation corresponding to each feature element, specifically including the following steps: When there are multiple feature elements in the real-time site features, the local pose deviations corresponding to each feature element are weighted and fused according to the confidence of each feature element to obtain the pose deviation. The confidence is based on the measurement noise of the environmental perception sensor, the observation completeness of the feature element, and the dynamic occlusion degree, which are pre-calibrated. The observation completeness is the ratio of the length of the observable part to the length of the complete line segment in the feature element, and the dynamic occlusion degree is the ratio of the length of the part of the feature element occluded by dynamic objects to the length of the complete line segment. When there is only one feature element in the real-time site features, the local pose deviation corresponding to that feature element is taken as the pose deviation.

[0044] Specifically, for the first Each feature element has a confidence level. The environmental sensing sensor's measurement noise level is determined by a combination of factors. (Can be obtained from sensor calibration parameters, normalized to the [0,1] interval), observation completeness and the degree of dynamic occlusion , For the first The actual observed line segment lengths among the feature elements. This represents the total length of the standard line segments corresponding to this feature element. This represents the length of the line segment in this feature element that is occluded by a dynamic object. Confidence level. Calculated by weighted summation: in , , For the preset weighting coefficients, satisfy The weighting coefficient can be set according to the actual needs of the scenario; for example, it can be appropriately increased in dynamic scenarios. The weight.

[0045] Obtain the confidence level of each feature element. Then, the local pose deviations are weighted and fused. For the translation component, the fused translation deviation... The calculation is as follows: in, For the first The translation component of the local pose deviation of each feature element. For the rotation component, the fused rotation deviation. The calculation is as follows: in For the first The rotational component of the local pose deviation of a feature element (if the feature element is obtained by fitting with missing endpoints, then...) The resulting fused pose error incorporates information from multiple features, with features of higher confidence contributing more, thus improving the reliability of pose error estimation. If there is only one feature element, its local pose error is directly used as the final pose error.

[0046] Based on the above, it is possible to effectively utilize multi-feature information, reduce the impact of poor single-feature observation quality on positioning correction, and further improve positioning accuracy and robustness.

[0047] To enrich the dimensions of real-time site features and improve the accuracy of region identification, one embodiment determines the logical semantic region where the mobile robot is currently located, specifically including the following steps: The line features extracted by the LiDAR and the corner features extracted by the visual sensor are fused at the feature level to construct the fused real-time site feature vector. The real-time site feature vector is then matched with the standard feature vector of the corresponding area in the site semantic partitioning model to determine the logical semantic region where the mobile robot is currently located.

[0048] Specifically, the line features extracted by LiDAR include line segment length, direction, and endpoint coordinates, while the corner features extracted by the visual sensor include corner coordinates and gradient direction. In feature-level fusion, the two types of features are combined into a single feature vector. The specific method for feature-level fusion can be any existing technology, such as feature concatenation or canonical correlation analysis. Each feature component is normalized and then concatenated into a vector. In the site semantic partitioning model, each region also corresponds to a standard feature vector, which is composed of the standard skeleton features of that region. During matching, the Euclidean distance or cosine similarity between the real-time feature vector and each standard feature vector is calculated, and the region with the smallest distance or the highest similarity is selected as the current region.

[0049] Based on the above, by making full use of the advantages of different sensors, namely, LiDAR provides accurate geometric scale information and visual sensors provide rich texture and corner information, the two complement each other, improving the accuracy and robustness of region discrimination, and providing a more reliable basis for subsequent invocation of accurate absolute constraints.

[0050] Furthermore, the method of determining the current logical semantic region through multi-sensor feature-level fusion, compared to the aforementioned topology-based matching method, can be implemented in practical applications by choosing one method based on sensor configuration and environmental characteristics, or by combining the two methods. For example, candidate regions can be obtained first through topology-based matching, and then verified through feature vector matching to improve the accuracy of the determination. This application does not limit this approach.

[0051] To further improve the continuity and robustness of positioning, in one embodiment, the method further includes: The system acquires odometer data from a wheeled odometer mounted on a mobile robot, as well as angular velocity and acceleration data from an inertial measurement unit. A filtering algorithm is then used to fuse the odometer data, angular velocity data, acceleration data, and corrected positioning coordinates to update the mobile robot's positioning coordinates in the global coordinate system.

[0052] Specifically, wheeled odometry provides the robot's relative motion displacement and rotation increments, but it suffers from cumulative errors caused by wheel slippage, etc.; inertial measurement units (IMUs) provide angular velocity and acceleration, which can be integrated to obtain attitude and position, but drift increases over time. In this embodiment, existing filtering algorithms, such as extended Kalman filtering or unscented Kalman filtering, are used for multi-sensor data fusion. The state vector typically includes the robot's position, attitude, velocity, and sensor bias terms. The prediction phase uses wheeled odometry and IMU data for kinematic recursion; the update phase uses corrected positioning coordinates as observations to correct the predicted state. Both the system model and observation model in the filtering algorithm can adopt standard forms from existing technologies. The state estimate obtained after filtering and updating is the updated global positioning coordinates.

[0053] Based on the above, even when feature matching temporarily fails (e.g., due to complete occlusion by a dynamic object), short-term positioning can be maintained using the wheel odometer and IMU, preventing complete loss of positioning. When feature matching is effective, the corrected positioning coordinates can suppress the drift of the odometer and IMU, achieving complementary advantages. This further improves the continuity and robustness of the entire positioning system.

[0054] It should be understood that, although Figure 2 The steps in the flowchart are shown sequentially as indicated by the arrows, but these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order in which these steps are executed, and they can be performed in other orders. Figure 2 At least some of the steps in the process may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily executed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be executed in turn or alternately with other steps or at least some of the sub-steps or stages of other steps.

[0055] Secondly, this embodiment provides a mobile robot localization system based on multi-source environmental perception and site semantic partitioning, such as... Figure 3As shown, it includes a semantic modeling module, an environment perception module, a region discrimination module, and a positioning correction module.

[0056] The semantic modeling module establishes a global coordinate system based on the standard dimensions of the mobile robot's work area. It pre-divides the physical space of the work area into multiple logical semantic regions with unique geometric coordinate ranges to construct a site semantic partitioning model. This model stores the geometric parameters of the standard skeleton features contained in each logical semantic region. These standard skeleton features include line segments or points with geometric parameters. This module can be a software module running on a server or robot controller, or it can be a standalone hardware unit. Its working principle is based on offline modeling according to known site specifications, independent of real-time sensing data, thus providing a priori benchmarks with absolute constraints for subsequent localization.

[0057] The environmental perception module is used to acquire real-time environmental data of the surrounding environment through environmental perception sensors mounted on the mobile robot, and to extract real-time site features from the environmental data. This module consists of environmental perception sensors mounted on the mobile robot and their associated data processing unit, which can be an embedded processor, FPGA, or GPU, etc.

[0058] The region discrimination module matches the extracted real-time site features with the standard skeleton features of each logical semantic region in the site semantic partitioning model to determine the logical semantic region where the mobile robot is currently located. This module can be a software algorithm module running on the robot controller and communicates with the semantic modeling module and the environment perception module.

[0059] The positioning correction module retrieves the geometric parameters of the standard skeleton features corresponding to the current logical semantic region from the site semantic partitioning model as absolute position constraints. It then calculates the pose deviation between the real-time site features and the standard skeleton features, and uses this deviation to correct the mobile robot's positioning coordinates in the global coordinate system. This module outputs the corrected positioning coordinates to the robot's control system for subsequent motion control.

[0060] The connections between the modules are as follows: the output of the semantic modeling module is connected to the input of the region discrimination module and the positioning correction module to provide a semantic zoning model of the site; the output of the environment perception module is connected to the input of the region discrimination module to provide real-time site features; the output of the region discrimination module is connected to the input of the positioning correction module to provide the current region discrimination result; and the output of the positioning correction module is connected to the robot's control system to output the corrected positioning coordinates.

[0061] During system operation, the semantic modeling module pre-builds and stores a semantic zoning model of the site; the environmental perception module collects environmental data in real time and extracts real-time site features; the region discrimination module matches real-time features with the model to determine the robot's current region; and the positioning correction module retrieves absolute constraints based on the region, calculates pose deviations, and corrects the positioning coordinates. These modules work collaboratively to form a complete positioning closed loop from prior modeling, real-time perception, region discrimination, to pose correction.

[0062] In one embodiment, the positioning correction module includes a coordinate correction unit.

[0063] The coordinate correction unit is used to decompose the calculated pose deviation into rotation and translation components; it is used to input the rotation and translation components into the angular velocity Kalman filter and the acceleration Kalman filter respectively for filtering, wherein the process noise covariance matrix of the angular velocity Kalman filter is set according to the angular velocity measurement noise of the mobile robot, and the process noise covariance matrix of the acceleration Kalman filter is set according to the acceleration measurement noise of the mobile robot; and it is used to superimpose the filtered rotation and translation components onto the current positioning coordinates of the mobile robot to generate the corrected positioning coordinates.

[0064] In one embodiment, the region discrimination module includes a first discrimination unit.

[0065] The first discrimination unit is used to construct a local topology map from the extracted real-time site features. The local topology map includes the relative distance and connection relationship between the features. It is also used to match the local topology map with the preset topology map of each region in the site semantic partitioning model and select the region with the highest matching degree as the current logical semantic region.

[0066] In one embodiment, the positioning correction module includes a deviation calculation unit.

[0067] The deviation calculation unit is used to perform point-to-point matching between the endpoint coordinates of the line segments extracted from the feature elements in the real-time site features and the endpoint coordinates of the corresponding line segments in the standard skeleton features to obtain the local pose deviation corresponding to the feature elements. If no line segment endpoints are extracted from a feature element, the local pose deviation corresponding to the feature element is obtained by least squares fitting based on the point cloud or image data corresponding to the observable line segment portion of the feature element and the known line segment length and direction constraints in the standard skeleton features. The unit is also used to determine the pose deviation based on the number of feature elements in the real-time site features and the local pose deviation corresponding to each feature element.

[0068] In one embodiment, the deviation calculation unit further includes a classification calculation subunit.

[0069] The classification calculation subunit is used to perform weighted fusion of the local pose deviations corresponding to each feature element based on the confidence level of each feature element when there are multiple feature elements in the real-time site features, to obtain the pose deviation. The confidence level is pre-calibrated based on the measurement noise of the environmental perception sensor, the observation completeness of the feature element, and the degree of dynamic occlusion. It is also used to take the local pose deviation corresponding to the feature element as the pose deviation when there is only one feature element in the real-time site features.

[0070] In one embodiment, the environmental perception sensor includes a lidar sensor and a vision sensor. Based on this, the region discrimination module includes a second discrimination unit.

[0071] The second discrimination unit is used to perform feature-level fusion of the line features extracted by the lidar and the corner features extracted by the visual sensor to construct the fused real-time site feature vector, and to match the real-time site feature vector with the standard feature vector of the corresponding area in the site semantic partitioning model to determine the logical semantic region where the mobile robot is currently located.

[0072] In one embodiment, the system also includes a coordinate update module.

[0073] The coordinate update module is used to acquire odometer data measured by the wheeled odometer mounted on the mobile robot, as well as angular velocity and acceleration data measured by the inertial measurement unit. A filtering algorithm is used to fuse the odometer data, angular velocity data, acceleration data and the corrected positioning coordinates to update the positioning coordinates of the mobile robot in the global coordinate system.

[0074] Specific limitations regarding the mobile robot localization system based on multi-source environmental perception and site semantic partitioning can be found in the limitations of the mobile robot localization method based on multi-source environmental perception and site semantic partitioning mentioned above, and will not be repeated here. Each module in the aforementioned mobile robot localization system based on multi-source environmental perception and site semantic partitioning can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device in hardware form, or stored in the memory of a computer device in software form, so that the processor can call and execute the corresponding operations of each module.

[0075] In one embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be as follows. Figure 4As shown, the computer device includes a processor, memory, network interface, and database connected via a system bus. The processor provides computational and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and database. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage media. The database stores mobile robot localization data based on multi-source environmental perception and site semantic partitioning. The network interface communicates with external terminals via a network connection. When the computer program is executed by the processor, it implements a mobile robot localization method based on multi-source environmental perception and site semantic partitioning.

[0076] Those skilled in the art will understand that Figure 4 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0077] Thirdly, a computer device is provided, including a memory and a processor. The memory and the processor are communicatively connected, and the memory stores a computer program that can run on the processor. When the processor executes the computer program, it implements the mobile robot localization method based on multi-source environmental perception and site semantic partitioning as described above. Furthermore, the specific limitations of the computer device in implementing the mobile robot localization method based on multi-source environmental perception and site semantic partitioning can be found in the limitations of the mobile robot localization method based on multi-source environmental perception and site semantic partitioning described above, and will not be repeated here.

[0078] Fourthly, a computer-readable storage medium is provided, on which a computer program is stored. When executed by a processor, the computer program implements the mobile robot localization method based on multi-source environmental perception and site semantic partitioning as described above. Furthermore, the specific limitations of the computer-readable storage medium in implementing the mobile robot localization method based on multi-source environmental perception and site semantic partitioning can be found in the limitations of the mobile robot localization method based on multi-source environmental perception and site semantic partitioning described above, and will not be repeated here.

[0079] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments of the above methods. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in a variety of forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), RAMbus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0080] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0081] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the invention patent. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this patent application should be determined by the appended claims.

Claims

1. A mobile robot localization method based on multi-source environmental perception and site semantic partitioning, characterized in that, The method includes: Based on the standard size specifications corresponding to the mobile robot's work site, a global coordinate system is established, and the physical space of the work site is pre-divided into multiple logical semantic regions with unique geometric coordinate ranges to construct a site semantic partitioning model. The site semantic partitioning model stores the geometric parameters of the standard skeleton features contained in each logical semantic region. The standard skeleton features include line segments or points with geometric parameters. By using environmental perception sensors mounted on a mobile robot, environmental data of the surrounding environment can be acquired in real time, and real-time site features can be extracted from the environmental data. The extracted real-time site features are matched with the standard skeleton features of each logical semantic region in the site semantic partitioning model to determine the logical semantic region where the mobile robot is currently located. Based on the current logical semantic region, the geometric parameters of the standard skeleton feature corresponding to that region in the site semantic partitioning model are retrieved as absolute position constraints. The pose deviation between the real-time site feature and the standard skeleton feature is calculated, and the pose deviation is used to correct the positioning coordinates of the mobile robot in the global coordinate system.

2. The method according to claim 1, characterized in that, The step of correcting the positioning coordinates of the mobile robot in the global coordinate system using the pose deviation includes: The calculated pose deviation is decomposed into rotational and translational components; The rotation component and the translation component are respectively input to the angular velocity Kalman filter and the acceleration Kalman filter for filtering. The process noise covariance matrix of the angular velocity Kalman filter is set according to the angular velocity measurement noise of the mobile robot, and the process noise covariance matrix of the acceleration Kalman filter is set according to the acceleration measurement noise of the mobile robot. The filtered rotation and translation components are superimposed on the current positioning coordinates of the mobile robot to generate the corrected positioning coordinates.

3. The method according to claim 1, characterized in that, The determination of the logical semantic region where the mobile robot is currently located includes: The extracted real-time site features are used to construct a local topology map, which includes the relative distances and connectivity relationships between features. The local topology map is matched with the preset topology map of each region in the site semantic partitioning model, and the region with the highest matching degree is selected as the current logical semantic region.

4. The method according to claim 1, characterized in that, The calculation of the pose deviation between the real-time site features and the standard skeleton features includes: For the feature elements in the real-time site features, point-to-point matching is performed between the endpoint coordinates of the line segments extracted from the feature elements and the endpoint coordinates of the corresponding line segments in the standard skeleton features to obtain the local pose deviation corresponding to the feature element; wherein, if no line segment endpoints are extracted from a feature element, the local pose deviation corresponding to the feature element is obtained by least squares fitting based on the point cloud or image data corresponding to the observable line segment portion in the feature element, and based on the known line segment length and direction constraints in the standard skeleton features. The pose deviation is determined based on the number of feature elements in the real-time site features and the local pose deviation corresponding to each feature element.

5. The method according to claim 4, characterized in that, The determination of the pose deviation based on the number of feature elements in the real-time site features and the local pose deviation corresponding to each feature element includes: When the real-time site features contain multiple feature elements, the local pose deviations corresponding to each feature element are weighted and fused according to the confidence level of each feature element to obtain the pose deviation; wherein the confidence level is pre-calibrated based on the measurement noise of the environmental perception sensor, the observation completeness of the feature element, and the degree of dynamic occlusion. When there is only one feature element in the real-time site features, the local pose deviation corresponding to that feature element is taken as the pose deviation.

6. The method according to claim 1, characterized in that, The environmental perception sensors include lidar and vision sensors; determining the logical semantic region where the mobile robot is currently located includes: The line features extracted by the LiDAR and the corner features extracted by the visual sensor are fused at the feature level to construct a fused real-time site feature vector. The real-time site feature vector is then matched with the standard feature vector of the corresponding region in the site semantic partitioning model to determine the logical semantic region where the mobile robot is currently located.

7. The method according to any one of claims 1 to 6, characterized in that, The method also includes: Odometer data measured by a wheeled odometer mounted on a mobile robot, as well as angular velocity and acceleration data measured by an inertial measurement unit, are acquired. A filtering algorithm is then used to fuse the odometer data, the angular velocity data, the acceleration data, and the corrected positioning coordinates to update the positioning coordinates of the mobile robot in the global coordinate system.

8. A mobile robot localization system based on multi-source environmental perception and site semantic partitioning, characterized in that, The system includes: The semantic modeling module is used to establish a global coordinate system based on the standard size specifications corresponding to the working site of the mobile robot, and to pre-divide the physical space of the working site into multiple logical semantic regions with unique geometric coordinate ranges to construct a site semantic partitioning model. The site semantic partitioning model stores the geometric parameters of the standard skeleton features contained in each logical semantic region. The standard skeleton features include line segments or points with geometric parameters. The environmental perception module is used to acquire environmental data of the surrounding environment in real time through environmental perception sensors mounted on the mobile robot, and extract real-time site features from the environmental data. The region discrimination module is used to match the extracted real-time site features with the standard skeleton features of each logical semantic region in the site semantic partitioning model to determine the logical semantic region where the mobile robot is currently located. The positioning correction module is used to retrieve the geometric parameters of the standard skeleton features corresponding to the current logical semantic region as absolute position constraints in the site semantic partitioning model, calculate the pose deviation between the real-time site features and the standard skeleton features, and use the pose deviation to correct the positioning coordinates of the mobile robot in the global coordinate system.

9. A computer device comprising a memory and a processor, the memory being communicatively connected to the processor, and the memory storing a computer program executable on the processor, characterized in that, When the processor executes the computer program, it implements the mobile robot localization method based on multi-source environmental perception and site semantic partitioning as described in any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the mobile robot localization method based on multi-source environmental perception and site semantic partitioning as described in any one of claims 1 to 7.

Images