Wheel robot passability prediction method and system in complex scene

Abstract

This application provides a method and system for predicting the accessibility of wheeled robots in complex scenarios, relating to the field of robotics technology. The method includes: pre-collecting indoor 3D point cloud data using a 3D laser scanner to generate an indoor point cloud map; processing the indoor point cloud map using a sparse convolution-based 3D semantic segmentation network to generate a multi-layer semantic map; extracting the target point coordinates of the task command; searching for a global reference path on the multi-layer semantic map using a path planning engine; constructing a local real-time dynamic environment model; obtaining a comprehensive feasibility score; and performing hierarchical decision-making based on the comprehensive feasibility score to generate a motion adjustment scheme. This method solves the technical problems of insufficient environmental representation capabilities, limited perception dimensions, and unintelligent decision-making mechanisms in complex indoor scenarios, which hinder the robot's ability to cope with dynamic and changing accessibility challenges.

Description

Technical Field

[0001] This application relates to the field of robotics technology, and in particular to a method and system for predicting the accessibility of wheeled robots in complex scenarios. Background Technology

[0002] Wheeled robots, with their flexible mobility and mature drive technology, have been widely used in indoor scenarios such as warehousing and logistics, intelligent inspection, and service reception. The core of achieving autonomous robot navigation lies in its motion planning technology, that is, how to plan a safe and efficient path according to the task objectives and control the robot to execute stably.

[0003] However, existing motion planning methods for wheeled robots have significant drawbacks. First, most systems rely on traditional two-dimensional grid maps for global path planning. These maps only express planar occupancy information and lack descriptions of the environment's three-dimensional structure and semantic attributes. This results in planned paths potentially traversing areas the robot cannot actually pass through, such as steep slopes, steps, or slippery surfaces. Second, at the local planning level, existing methods typically rely solely on geometric collision detection for obstacle avoidance, failing to integrate multimodal sensor data for refined assessment of ground accessibility. Robots cannot distinguish between smooth surfaces, gravel roads, and liquid spill areas, thus hindering intelligent passage decisions and frequently stalling or failing to plan due to small obstacles or ground anomalies. Finally, the system's decision-making logic is often binary, lacking a quantitative and comprehensive evaluation metric to guide hierarchical recursive planning, leading to insufficient adaptability and robustness in complex dynamic scenarios.

[0004] In summary, existing technologies struggle to address the dynamic and ever-changing accessibility challenges in complex indoor scenarios. Their shortcomings primarily lie in insufficient environmental representation capabilities, limited perception dimensions, and unintelligent decision-making mechanisms. A solution to these problems is urgently needed. Summary of the Invention

[0005] This disclosure provides a method and system for predicting the accessibility of wheeled robots in complex scenarios, which addresses the technical problems in the prior art, such as difficulty in coping with dynamic and changing accessibility challenges in complex indoor scenarios, insufficient environmental representation capabilities, single perception dimensions, and unintelligent decision-making mechanisms.

[0006] According to a first aspect of this disclosure, a method for predicting the accessibility of a wheeled robot in complex scenarios is provided, comprising:

[0007] Indoor 3D point cloud data is pre-collected using a 3D laser scanner to generate an indoor point cloud map. The indoor point cloud map is then processed using a sparse convolution-based 3D semantic segmentation network to generate a multi-layer semantic map.

[0008] Extract the target point coordinates of the task instruction, and use a path planning engine to search for a global reference path on a multi-layer semantic map. The global reference path consists of a series of ordered waypoints.

[0009] A sensor array is mounted on the top of the wheeled robot to collect information about the surrounding environment and the wheeled robot itself in real time as the wheeled robot moves along the global reference path, and to build a local real-time dynamic environment model.

[0010] Based on a local real-time dynamic environment model, the mobility of wheeled robots is analyzed, and ground mobility scores and spatial mobility scores are calculated. The ground mobility scores and spatial mobility scores are then fused to obtain a comprehensive feasibility score.

[0011] Based on a comprehensive feasibility score, a tiered decision-making process is implemented to generate motion adjustment plans.

[0012] According to a second aspect of this disclosure, a system for predicting the accessibility of a wheeled robot in complex scenarios is provided, comprising:

[0013] The prior semantic environment modeling module is used to pre-collect indoor 3D point cloud data through a 3D laser scanner, generate an indoor point cloud map, and process the indoor point cloud map using a sparse convolution-based 3D semantic segmentation network to generate a multi-layer semantic map.

[0014] A global path planning module is used to extract the target point coordinates of the task instruction and use a path planning engine to search for a global reference path on a multi-layer semantic map. The global reference path consists of a series of ordered waypoints.

[0015] A real-time multimodal perception and fusion module is used to mount a sensor array on the top of a wheeled robot to collect information about the surrounding environment and the wheeled robot itself in real time during the movement of the wheeled robot along a global reference path, and to construct a local real-time dynamic environment model.

[0016] The dynamic accessibility assessment module is used to perform accessibility analysis of wheeled robots based on a local real-time dynamic environment model, calculate ground accessibility score and spatial accessibility score, and fuse the ground accessibility score and spatial accessibility score to obtain a comprehensive feasibility score.

[0017] A hierarchical autonomous decision-making and control module is used to execute hierarchical decisions based on a comprehensive feasibility score and generate motion adjustment plans.

[0018] One or more technical solutions provided in this disclosure have at least the following technical effects or advantages: Indoor 3D point cloud data is pre-collected using a 3D laser scanner to generate an indoor point cloud map; the indoor point cloud map is processed using a sparse convolution-based 3D semantic segmentation network to generate a multi-layer semantic map; the target point coordinates of the task instructions are extracted, and a path planning engine is used to search for a global reference path on the multi-layer semantic map, the global reference path consisting of a series of ordered waypoints; a sensor array is mounted on the top of a wheeled robot to collect information about the surrounding environment and the wheeled robot itself in real time during the robot's movement along the global reference path, constructing a local real-time dynamic environment model; based on the local real-time dynamic environment model, the drivability of the wheeled robot is analyzed, calculating ground drivability scores and spatial drivability scores, fusing the ground drivability scores and spatial drivability scores to obtain a comprehensive feasibility score; based on the comprehensive feasibility score, hierarchical decision-making is performed to generate a motion adjustment scheme. This solves the technical problems in existing technologies, such as difficulty in coping with dynamic and changing drivability challenges in complex indoor scenes, insufficient environmental representation capabilities, single perception dimensions, and unintelligent decision-making mechanisms. It has achieved significant technical effects in improving the accuracy of wheeled robots' passability prediction in complex scenarios, the intelligence level of motion planning, and the reliability and efficiency of the overall navigation system.

[0019] The above description is merely an overview of the technical solution of this application. In order to better understand the technical means of this application and to implement it in accordance with the contents of the specification, and to make the above and other objects, features and advantages of this application more obvious and understandable, specific embodiments of this application are given below. Attached Figure Description

[0020] To more clearly illustrate the technical solutions in this disclosure or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are merely exemplary. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0021] Figure 1 A flowchart illustrating a method for predicting the accessibility of a wheeled robot in a complex scenario, provided in an embodiment of this application;

[0022] Figure 2 This is a schematic diagram of the structure of a wheeled robot accessibility prediction system in a complex scenario, provided as an embodiment of this application.

[0023] Figure labeling: 11 Prior semantic environment modeling module, 12 Global path planning module, 13 Real-time multimodal perception fusion module, 14 Dynamic drivability assessment module, 15 Hierarchical autonomous decision-making and control module. Detailed Implementation

[0024] The exemplary embodiments of this disclosure are described below with reference to the accompanying drawings, including various details of the embodiments to aid understanding, and should be considered merely exemplary. Therefore, those skilled in the art will recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of this disclosure. Similarly, for clarity and brevity, descriptions of well-known functions and structures are omitted in the following description.

[0025] Example 1: This disclosure provides a method for predicting the accessibility of a wheeled robot in a complex scenario, which is referred to below. Figure 1 The methods include:

[0026] S1: Indoor 3D point cloud data is pre-collected using a 3D laser scanner to generate an indoor point cloud map. The indoor point cloud map is then processed using a sparse convolution-based 3D semantic segmentation network to generate a multi-layer semantic map.

[0027] Furthermore, step S1 also includes:

[0028] Based on the real-scene survey planning scanning path, a 3D laser scanner is used to perform scanning according to the scanning path, collect raw point cloud data, and generate multi-station segmented raw point clouds;

[0029] The original point cloud from multiple stations is preprocessed to obtain an indoor point cloud map. The preprocessing includes point cloud registration, point cloud denoising, and point cloud compression.

[0030] Obtain an indoor point cloud map, perform semantic segmentation using a sparse convolution-based 3D semantic segmentation network, and assign semantic labels to the indoor point cloud map.

[0031] A multi-layer semantic map for wheeled robots is constructed based on indoor point cloud maps with added semantic tags. The multi-layer semantic map includes a navigation layer and a semantic layer.

[0032] Specifically, based on the real-world site conditions, key areas, obstacles, and feature points requiring focused scanning are identified. Spherical or planar targets are placed around these key locations as common matching points. A scanning path is designed using all common matching points, with stations arranged in a zigzag or circular pattern to ensure at least 30% overlap between adjacent stations. This ensures that each station can capture targets or significant features from other stations. Subsequently, 3D laser scanners are sequentially deployed at each station to perform scans, recording data points containing 3D coordinates and reflection intensity, generating a multi-station segmented raw point cloud.

[0033] After scanning, point cloud processing software is used to preprocess the multi-station segmented raw point clouds. Specific operations include point cloud registration, point cloud denoising, and point cloud compression. By identifying the centers of identical spherical or planar targets in the point clouds from different stations, the rotation and translation matrices between common targets are calculated, precisely aligning the point clouds to a unified global coordinate system. For areas without targets or with missing targets, the ICP algorithm is used to iteratively optimize the transformation matrix, minimizing the distance between corresponding points in two point clouds, thus achieving point cloud registration. Point cloud denoising is achieved using a statistical outlier removal filter, analyzing the average distance between each point and its K nearest neighbors. Assuming these distances follow a Gaussian distribution, all points whose distances exceed twice the standard deviation of the average distance are considered noise and removed, eliminating discrete noise points caused by dust, moving shadows, and reflections. A voxel grid downsampling method is used to compress the number of point clouds. The 3D space is divided into tiny cubic grids of a specified resolution, such as 5mm. Within each non-empty voxel, the coordinates of the center point of all points represent all points within that voxel. This significantly reduces the number of points while preserving the original point cloud shape characteristics to the greatest extent possible. Finally, the preprocessed site data is fused to generate a complete and clean indoor point cloud map.

[0034] Obtain an indoor point cloud map and perform semantic segmentation using a sparse convolution-based 3D semantic segmentation network to assign semantic labels to the indoor point cloud map. Input the indoor point cloud map into a 3D semantic segmentation network, such as Cylinder3D or MinkowskiNet. The network first discretizes the input point cloud data into a defined 3D voxel grid. Then, it uses sparse convolutional layers to extract features from each voxel layer by layer. Through multiple layers of convolution and pooling operations, the network progressively learns and extracts highly discriminative voxel features from the local geometric context. Subsequently, trilinear interpolation is used to map the learned voxel features back to each original point location, generating a feature vector for each point. Finally, the feature vector of each point is processed by a classifier, outputting a probability distribution vector representing the probability that the point belongs to each semantic category, such as ground, wall, shelf, door, restricted area, etc. The label with the highest probability is assigned to the point, ultimately producing point cloud data with accurate semantic labels.

[0035] Finally, a multi-layer semantic map for the wheeled robot is constructed based on the semantically segmented indoor point cloud map. This map consists of a navigation layer representing a two-dimensional occupied grid map and a semantic layer representing a 3D semantic volumetric map. The navigation layer is generated primarily by projecting the 3D semantic point cloud vertically along the Z-axis onto the XY plane and generating grid state labels according to rules. The specific rules are: if a grid is projected by any non-ground obstacle point, it is marked as "occupied"; if only ground points are projected, it is marked as "free"; if there are no points, it is marked as "unknown". This ultimately generates a two-dimensional map for use by the global path planning algorithm. The semantic layer is generated by converting the labeled semantic point cloud into an octree map data structure. This structure recursively subdivides the space into eight sub-cube voxels. Each voxel node not only stores its occupancy probability but, more importantly, its dominant semantic label. For example, if 80% of the points in a voxel are labeled as "shelf," then its semantic label is "shelf." This ultimately forms a dense knowledge base that supports fast multi-resolution queries and 3D spatial semantic reasoning. By integrating the navigation layer and the semantic layer, a multi-layered semantic map is finally obtained, which combines low-level geometric information with high-level semantic knowledge.

[0036] S2: Extract the target point coordinates of the task instruction, and use a path planning engine to search for a global reference path on a multi-layer semantic map. The global reference path consists of a series of ordered waypoints.

[0037] Furthermore, step S2 also includes:

[0038] Receive task instructions and multi-layer semantic map, obtain a two-dimensional occupied grid map and extract the target point coordinates;

[0039] Using A The algorithm searches for the optimal path on a two-dimensional occupied grid map;

[0040] By smoothing the path, jagged path segments in the optimal path are eliminated, generating a global reference path that conforms to the kinematic constraints of the wheeled robot.

[0041] Specifically, the system receives task instructions and extracts the navigation layer from a multi-layer semantic map to obtain a two-dimensional occupied grid map representing the distribution of static obstacles in the environment. Each grid cell in this map has three states: free, occupied, and unknown, where free areas represent walkable areas and occupied areas represent permanent obstacles. Based on the two-dimensional occupied grid map, the coordinates of the task target point in the global coordinate system are extracted.

[0042] Path planning is performed based on the obtained target point coordinates and a two-dimensional occupied grid map. The path planning process is executed by a built-in path planning engine, which employs classic graph search-based algorithms, such as A*. Algorithm. A The algorithm designates the starting position of the wheeled robot as the starting node and the target point as the target node. It's important to note that all wheeled robots start at the site entrance with defined coordinates. The algorithm maintains an open list and a closed list, finding the optimal path by calculating the cost function f(n) = g(n) + h(n) for each node. Here, g(n) is the actual cost from the starting node to the current node, and h(n) is the heuristically estimated cost from the current node to the target node. Both costs are obtained using Manhattan distance or Euclidean distance. Specifically, the planning engine begins with initialization, adding the starting point to the open list. Then, a cyclical search phase begins: the algorithm continuously expands the node with the smallest f(n) value from the open list, checking all its adjacent grids. If an adjacent grid is empty and unvisited, its cost is calculated and added to the open list. This process iterates until the target point is successfully found. Finally, by tracing back the parent node pointers of each node from the target point, an optimal path containing the complete coordinate sequence from the starting point to the destination is reconstructed.

[0043] Due to the discrete nature of the grid, the optimal path often contains unnecessary turns and jagged edges. Therefore, it is necessary to further smooth the optimal path using gradient descent or B-spline curve fitting algorithms to eliminate sharp corners, ensure the path is continuous and smooth, and meet the minimum turning radius constraint of the wheeled robot. The final generated global reference path consists of a series of dense, equally spaced waypoints, each containing (x,y) coordinates and orientation information, forming the trajectory reference for the wheeled robot.

[0044] The global reference path is output as the planning result. This path consists of an ordered sequence of waypoints {P0, P1, P2, ..., Pn}, where P0 is the starting point and Pn is the target point. This path provides the theoretically optimal route for the wheeled robot's macroscopic motion, and more importantly, it serves as the benchmark framework for all subsequent local real-time obstacle avoidance, behavior decision-making, and motion control.

[0045] S3: Equipped with a sensor array on the top of the wheeled robot, it collects information about the surrounding environment and the wheeled robot itself in real time as the wheeled robot moves along the global reference path, and builds a local real-time dynamic environment model.

[0046] Furthermore, step S3 also includes:

[0047] A sensor array is deployed on the top of a wheeled robot to acquire a multi-source heterogeneous dataset. The sensor array includes a 3D LiDAR, an RGB-D camera, and an IMU. The multi-source heterogeneous dataset includes laser point cloud data, visual image data, and IMU data.

[0048] The RANSAC plane fitting algorithm based on normal vector analysis is used to process laser point cloud data and extract the geometric contours of the surrounding environment.

[0049] Visual image data is processed using a deep learning-based segmentation network to obtain semantic labels for specific terrain features;

[0050] The body posture of the wheeled robot is obtained by solving the IMU data through Kalman filtering;

[0051] Based on the obtained geometric contours of the surrounding environment, semantic labels of special terrain, and the body posture of the wheeled robot, a local real-time dynamic environment model is constructed by fusing multi-source feature information through the extended Kalman filter algorithm.

[0052] Specifically, a multi-source heterogeneous dataset is constructed by continuously capturing environmental data from the surrounding environment through a sensor array mounted on the wheeled robot as it moves along a global reference path. This process begins with the deployment of the sensor array and data acquisition during movement. First, the sensor array is integrated onto the top of the wheeled robot to maximize its field of view. The sensor array includes a 3D LiDAR, an RGB-D camera, and an IMU. The 3D LiDAR, as the primary sensor, rotates at a high speed of 10-20Hz, emitting laser beams and measuring their return time to acquire laser point cloud data of the surrounding environment, thus obtaining the geometric contours of the environment. The RGB-D camera, as an important auxiliary sensor, simultaneously captures color images of the environment and corresponding depth information, compensating for the inherent limitations of LiDAR in recognizing colorless, transparent objects or semantic elements dependent on texture, such as water surfaces and oil stains. The IMU measures the three-axis angular velocity and acceleration of the wheeled robot at a high frequency. After acquiring the multi-source heterogeneous dataset through the sensor array, data synchronization is performed. Timestamp alignment ensures that all sensor data have a unified time reference. Next, coordinate system transformation is performed, converting the laser point cloud data from the radar coordinate system, the visual image data from the camera coordinate system, and the IMU data from the body coordinate system to the wheeled robot base coordinate system through a pre-calibrated extrinsic parameter matrix, and finally integrating them to obtain a multi-source heterogeneous dataset.

[0053] Feature extraction was performed on heterogeneous data from three different sources within a multi-source heterogeneous dataset to construct local real-time dynamic environment models. For laser point cloud data, the core objective was to segment the ground, cluster and identify obstacles, and extract the geometric contours of the surrounding environment. Specifically, a RANSAC plane fitting algorithm based on normal vector analysis was used for ground segmentation. A planar model was estimated by randomly sampling a subset of the point cloud, and then the distances of all points to this model were calculated. Intra-points that conformed to the model were counted. Through multiple iterations, the model with the most intra-points was selected as the optimal ground plane. The segmented points were labeled as ground points, and the rest as non-ground points. Next, Euclidean clustering was used to cluster obstacles among the non-ground points. This algorithm uses a given point as the core and searches for all points within its specified radius neighborhood. If a point is found, it is grouped into a cluster, and the cluster continues to expand using new points within the cluster as cores until no more points can be added, thus identifying individual obstacle clusters.

[0054] Secondly, for visual image data captured by RGB-D cameras, the focus of processing is on identifying special terrain features that are difficult to distinguish in point cloud data, such as water surfaces and oil stains. This task is accomplished using a deep learning-based segmentation network, such as Mask R-CNN. This network can perform pixel-level semantic segmentation on the input visual image data, outputting a probability mask for each pixel belonging to a specific category. Subsequently, these segmentation results on the two-dimensional image are aligned with the simultaneously acquired depth image. Using camera intrinsics and a pre-calibrated camera-radar extrinsic matrix, the two-dimensional pixel coordinates are converted into points in three-dimensional space, thereby accurately mapping and attaching the identified semantic labels such as "water surface" and "oil stain" to their corresponding three-dimensional locations in the local environment model.

[0055] Finally, the core function of IMU data is to provide the wheeled robot's real-time body posture. The raw three-axis acceleration and angular velocity data are used for posture calculation through Kalman filtering. The filter continuously optimizes the state estimation through two steps: prediction (calculating the current posture based on the previous posture and angular velocity) and updating (correcting pitch and roll using accelerometer data, and correcting yaw using magnetometer or global observation). Finally, it outputs a stable and reliable current pitch angle of the robot, while retaining the previously measured acceleration as part of the body posture information.

[0056] After extracting the geometric contours of the surrounding environment, semantic labels for specific terrain features, and the robot's pose from three types of data, a tightly coupled multi-source feature information fusion is performed using an Extended Kalman Filter (EKF). The filter's state vector contains the robot's pose, while the observation vector incorporates feature points from vision and laser. Through a continuous "prediction-update" process, the EKF optimally fuses all observation information, continuously correcting and optimizing the robot's state estimation.

[0057] Ultimately, all processing results are integrated and output as a unified local real-time dynamic environment model. This model is based on a 3D point cloud, where each point not only contains its 3D coordinates but is also endowed with rich attribute information. This environment model, rich in multi-dimensional features, provides a complete data foundation for subsequent comprehensive accessibility assessment.

[0058] S4: Based on the local real-time dynamic environment model, perform accessibility analysis on the wheeled robot, calculate the ground accessibility score and the spatial accessibility score, and fuse the ground accessibility score and the spatial accessibility score to obtain a comprehensive feasibility score.

[0059] Furthermore, step S4 also includes:

[0060] The ground slope angle is obtained based on the geometric contour of the surrounding environment and the body posture in the local real-time dynamic environment model.

[0061] The maximum ramp angle that the wheeled robot can traverse is determined based on its own parameters. The specific formula is as follows:

[0062] ;

[0063] in This represents the maximum slope angle that a wheeled robot can traverse. This represents the maximum gradient based on the motor torque, with a value of [value missing]. ,in represents the maximum output torque of the motor, m represents the total mass of the wheeled robot, g represents the acceleration due to gravity, r represents the wheel radius, and arcsin represents the arcsine function. Represents the maximum slope based on stability, with a value of Where h represents the center of gravity height, w represents the wheelbase width, and arctan represents the arctangent function. Represents the maximum slope based on ground friction, with a value of ,in The coefficient of static friction between the wheel and the ground is represented by , arctan represents the arctangent function, and the maximum slope angle that the wheeled robot can ultimately traverse is obtained. The minimum value among the three is selected using the min function;

[0064] A slope feasibility scoring function is defined, which combines the ground slope angle and the maximum passable slope angle of the wheeled robot to calculate the slope feasibility score. The specific calculation formula is as follows:

[0065] ;

[0066] in, The slope feasibility score is represented by θ, which represents the ground slope angle. This represents the maximum slope angle that a wheeled robot can traverse. This represents the optimal slope angle threshold, which is typically set to 50%. ;

[0067] Based on the point cloud data of all ground points in the local real-time dynamic environment model, the roughness index is calculated using the following formula:

[0068] ;

[0069] Where R represents the roughness index, sqrt represents the square root, N is the total number of ground points in the point cloud data, and d i This represents the vertical distance from each ground point to the fitted plane;

[0070] The maximum permissible ground roughness for a wheeled robot is determined based on its own parameters, using the following formula:

[0071] ;

[0072] in This represents the maximum permissible ground roughness for wheeled robots. This represents the maximum permissible surface roughness based on the wheel radius, with a value of [value missing]. ,in The safety factor is represented by r, which represents the wheel radius. This represents the maximum permissible ground roughness considering suspension system corrections, and the value is... ,in This represents the static compression of the suspension system. Represents the suspension utilization factor. The maximum travel of the suspension system and the maximum permissible ground roughness Rmax of the wheeled robot are selected as the minimum value between the two by the min function.

[0073] A ground roughness scoring function is defined, and the ground roughness score is calculated by combining the roughness index with the maximum permissible ground roughness of the wheeled robot. The specific formula is as follows:

[0074] ;

[0075] in, R represents the surface roughness score, and R represents the roughness index. max This represents the maximum permissible ground roughness for the wheeled robot, and k represents the attenuation coefficient. Here, the value of the attenuation coefficient k is set to... ;

[0076] The system directly obtains the special terrain semantic labels from the local real-time dynamic environment model, determines whether there is a liquid hazard area within the target region, outputs the hazard source presence marker "isWater", and inputs the result into the defined hazard source scoring function.

[0077] ;

[0078] in, This represents the hazard rating; isWater indicates the presence of the hazard.

[0079] The ground accessibility score is obtained based on the slope feasibility score, ground roughness score, and hazard source score:

[0080] S ground = f slope (θ) · f roughness (R) · f water (isWater);

[0081] Among them, S ground Represents ground accessibility score, Represents the slope feasibility score. Represents the surface roughness score. Represents a hazard rating;

[0082] The 3D point cloud data obtained from a single scan is discretized into a regular voxel grid. Occupied voxels are selected, and the proportion of each voxel occupied within its 26-neighborhood is calculated. The specific calculation formula is as follows:

[0083] ;

[0084] in, This represents the proportion occupied by the 26 neighboring voxels. This represents the number of occupies in the 26 neighborhood voxels, where i, j, and k represent the index values ​​in the three dimensions, respectively.

[0085] Based on this ratio, the local entropy value of each occupied voxel is calculated using the following formula:

[0086] ;

[0087] in, This represents the local entropy value of the voxel. This represents the proportion of voxels that are occupied in the 26 neighboring voxels of this voxel;

[0088] After obtaining the local entropy values ​​of all occupied voxels, the overall spatial complexity index is calculated using the following formula:

[0089] ;

[0090] in, Represents an indicator of overall spatial complexity. The total number of occupied voxels. Represents an indicator function. Represents the local entropy value;

[0091] The overall spatial complexity index is mapped to the interval [0,1] to obtain the spatial traversability score. The specific calculation formula is as follows:

[0092] ;

[0093] in, Represents spatial traversability score, Represents an indicator of overall spatial complexity. Represents the theoretical maximum entropy value, and tanh is the hyperbolic tangent function;

[0094] The ground accessibility score and the spatial accessibility score are weighted and linearly fused to obtain the final comprehensive feasibility score:

[0095] S = w g ·S ground + w s ·S space ;

[0096] Where S represents the comprehensive feasibility score, w g and w s Let w be the weighting coefficient, and satisfy w g + w s = 1, S ground S represents the ground accessibility score. space This represents the spatial traversability score.

[0097] Specifically, the first step is to conduct a ground mobility analysis. This analysis aims to accurately quantify and evaluate the physical properties of the ground area in front of the wheeled robot, and the output is a ground mobility score S. ground .

[0098] Point cloud data labeled as ground points are extracted from a local real-time dynamic environment model. This point cloud data contains a series of three-dimensional coordinates (x, y, z). i , y i , z iThe least squares method is used to fit a three-dimensional plane to the above ground points, finding an optimal plane equation Ax + By + Cz + D = 0, which minimizes the sum of the squares of the vertical distances from all ground points to this plane. Based on the four coefficients A, B, C, and D of the fitted optimal plane equation, the normal vector n = (A, B, C) of the plane is obtained. This normal vector is perpendicular to the fitted plane and explicitly defines the plane's orientation in three-dimensional space—whether the plane is steep or gentle, and in which direction it tilts. Simultaneously, the body's attitude information is extracted from the local real-time dynamic environment model. Based on the correspondence between the world coordinate system and the gravity direction established when calculating the pitch angle, a reference gravity vector pointing towards the Earth's center is obtained. This reference gravity vector is typically simplified to g = (0, 0, -1). Subsequently, the ground slope angle is obtained by calculating the cosine of the angle between these two vectors. The calculation formula is:

[0099] ;

[0100] in, The slope angle represents the ground slope angle; across represents the inverse cosine function, which converts the cosine values ​​of two vector directions back to their corresponding angle values; n represents the normal vector; and g represents the reference gravity vector. This represents the dot product of vectors n and g. and These represent the magnitudes of vectors n and g, respectively.

[0101] The calculated ground slope angle is compared and verified with the pitch angle in the body's attitude. If the difference between the ground slope angle and the pitch angle is within 5 degrees, the calculated ground slope angle is considered to be consistent with reality, and the deviation angle may be due to sensor accuracy; therefore, no further optimization adjustment of the ground slope angle is performed. If the difference between the ground slope angle and the pitch angle is greater than 5 degrees, the ground slope angle is optimized using the following formula:

[0102] ;

[0103] in, This represents the optimized ground slope angle, characterizing the current angle of inclination of the ground relative to the horizontal plane. This represents the ground slope angle previously calculated using point cloud data. This represents the robot's pitch angle directly measured by the IMU. Representing the confidence level of point cloud data, it is a quantitative assessment value indicating the reliability of slope measurement results obtained from point cloud data. Represents the confidence level of IMU data, indicating a quantitative assessment of the reliability of IMU pitch angle measurement results.

[0104] Specifically, the formula for obtaining the confidence level of point cloud data is as follows: ,in The standard deviation of the residuals representing the point cloud plane fitting reflects the degree of dispersion between the point cloud data and the fitted plane. The standard deviation of the lidar measurement noise is calibrated by the technical parameters provided by the sensor manufacturer and characterizes the basic accuracy of laser ranging. exp(x) represents an exponential function. To ensure that the confidence level of the point cloud data is within the range of 0 to 1, therefore... Adding a negative sign, the final confidence value of the point cloud data is 1, which represents a perfect fit with no fitting error, and the point cloud data can be completely trusted. The closer the value is to 0, the worse the fit quality is, and the point cloud data cannot be trusted.

[0105] Specifically, the formula for obtaining IMU data confidence is as follows: ,in The magnitude of the current resultant acceleration is calculated using acceleration information from the body's attitude. The threshold for acceleration interference is usually set to 1 m / s². The confidence level of the final IMU data will also be in the range of 0 to 1. When the value is 1, it means that the pitch angle data provided by the IMU is trustworthy. The closer it is to 0, the less trustworthy it is.

[0106] A slope feasibility scoring function is defined, and the slope feasibility score is calculated based on the final obtained ground slope angle. The slope feasibility scoring function is specifically defined as follows:

[0107] ;

[0108] in, This represents the slope feasibility score, where θ represents the ground slope angle; the specific value may not be optimized or adjusted. It may also be after optimization and adjustment. This is determined by the aforementioned steps. The maximum ramp angle that a wheeled robot can traverse is represented by this parameter. This parameter is not calculated but rather obtained through calculation. It is a fundamental indicator for measuring the robot's climbing ability. This represents the optimal slope angle threshold, which is typically set to 50%. .

[0109] Specifically, the maximum slope angle that the wheeled robot can traverse is... The formula for obtaining it is ,in This represents the maximum gradient based on the motor torque, with a value of [value missing]. ,in represents the maximum output torque of the motor, m represents the total mass of the wheeled robot, g represents the acceleration due to gravity, r represents the wheel radius, and arcsin represents the arcsine function. Represents the maximum slope based on stability, with a value of Where h represents the center of gravity height, w represents the wheelbase width, and arctan represents the arctangent function. Represents the maximum slope based on ground friction, with a value of ,in The coefficient of static friction between the wheel and the ground is represented by , arctan represents the arctangent function, and the maximum slope angle that the wheeled robot can ultimately traverse is obtained. The minimum value among the three can be selected using the min function.

[0110] After completing the slope feasibility assessment, ground roughness calculation is performed. Ground roughness is obtained to assess the smoothness of the ground surface and the presence of scattered small obstacles. First, point cloud data of all ground points is acquired from a local real-time dynamic environment model, and P is calculated for each ground point. i Vertical distance d to the previously fitted plane i Then, the root mean square error of all distance values ​​is calculated as the overall roughness index R. The specific calculation formula is as follows:

[0111] ;

[0112] Where sqrt represents the square root, N is the total number of ground points, and d i R represents the vertical distance from each ground point to the fitted plane. This index R is also derived from statistical calculations of real-time point cloud data. The final calculated roughness index R is fed into a ground roughness scoring function, which is defined as:

[0113] ;

[0114] in, R represents the ground roughness score, and R represents the roughness index. Its value is obtained through statistical analysis of the ground point cloud and is an input variable of the algorithm. maxrepresents the maximum permissible ground roughness, a preset constant whose value is usually directly related to the robot's wheel radius and the shock absorption capacity of the suspension system. It is used to define the absolute boundary between smooth and uneven surfaces. k represents the attenuation coefficient, a preset constant that controls the rate of score decrease. The larger the k value, the more sensitive the score is to roughness exceeding the threshold, and the faster the score decreases. The ground roughness scoring function gives full marks within a safe range. Once the roughness exceeds the critical value, the score does not immediately return to zero, but decays exponentially with increasing roughness. This is more in line with the physical intuition that passability gradually deteriorates in the real world. Here, the value of the attenuation coefficient k is set to [value missing]. .

[0115] Specifically, the formula for obtaining the maximum permissible ground roughness Rmax is as follows: ,in This represents the maximum permissible surface roughness based on the wheel radius, with a value of [value missing]. ,in The safety factor is set to 0.2, and r represents the wheel radius. This represents the maximum permissible ground roughness considering suspension system corrections, and the value is... ,in This represents the static compression of the suspension system. This represents the suspension utilization factor, typically taken as 0.3. Represents the maximum travel of the suspension system, and the final maximum permissible ground roughness Rmax is selected as the minimum value between the two through the min function.

[0116] In addition, specific terrain semantic labels are directly obtained from the local real-time dynamic environment model to determine whether there are liquid hazard areas such as "water surface" and "oil pollution" within the target area. The Boolean value "isWater" is output, and this result is input into the defined hazard source scoring function. The specific expression of the hazard source scoring function is as follows:

[0117] ;

[0118] in, The hazard rating function assigns a veto to liquid hazards because such hazards typically pose a fatal threat to the mobility of wheeled robots and are difficult to detect reliably with lidar. Therefore, it must rely on visual information for specialized processing. isWater represents the presence of the hazard, and its Boolean value comes from the output of a vision-based semantic segmentation model.

[0119] Finally, the ground accessibility score is obtained by multiplying the slope feasibility score, ground roughness score, and hazard score:

[0120] S ground = fslope (θ) · f roughness (R) · f water (isWater);

[0121] Among them, S ground This represents the number of points that allow passage on the ground. Represents the slope feasibility score. Represents the surface roughness score. Representing the hazard score, the formula uses multiplication for fusion, meaning that any sub-score of zero will result in a total score of zero. This fusion mechanism mandates that the ground must simultaneously meet conditions in three dimensions: slope, roughness, and hazard source, reflecting the system's highest priority safety principle. ground The output is a scalar value between 0 and 1.

[0122] While completing the ground accessibility analysis, a spatial complexity analysis process based on information entropy is simultaneously carried out. This process focuses on quantifying the structural characteristics of the wheeled robot's forward passage from the perspective of spatial distribution patterns, and its output is the spatial accessibility score S. space The process first discretizes the 3D point cloud data obtained from a single scan into a regular voxel mesh, with each voxel having a physical size of d. x ×d y ×d z For example, 0.1m × 0.1m × 0.1m forms an N × M × K voxel unit, where N, M, and K represent the number of voxels in the X, Y, and Z directions, respectively, and d x d y d z These represent the physical dimensions of a single voxel. A voxel, as the smallest unit in three-dimensional space, is similar to a three-dimensional extension of a two-dimensional pixel. The occupancy state of a voxel is determined based on point cloud data: if a voxel contains at least one point cloud data point, it is considered to be in an "occupied" state; otherwise, it is in an "idle" state.

[0123] Subsequently, for each occupied voxel v ijk The occupancy status of other voxels within its 26-neighborhood is analyzed. The 26-neighborhood includes all adjacent voxels surrounding the central voxel. For each occupied voxel, the proportion of occupied voxels in its 26-neighborhood is calculated using the following formula:

[0124] ;

[0125] in, This represents the proportion occupied by the 26 neighboring voxels. This represents the number of occupies in the 26 neighboring voxels, where i, j, and k represent the index values ​​in the three dimensions, and this proportion. It reflects the occupancy density of a local area.

[0126] Based on this ratio, the local entropy value of each occupied voxel is calculated using the following formula:

[0127] ;

[0128] in, This represents the proportion of voxels occupied in the 26-neighborhood of that voxel. This represents the local entropy value of the voxel, if ,but Local entropy quantifies the uncertainty of the neighborhood occupancy state, when At this time, the entropy value is at its maximum, indicating that the occupied state is most uncertain. When the entropy value is close to 0 or 1, it is relatively small, indicating that the occupied state is relatively certain.

[0129] After obtaining the local entropy values ​​of all occupied voxels, the overall spatial complexity index is calculated using the following formula:

[0130] ;

[0131] in, Represents an indicator of overall spatial complexity. The total number of occupied voxels. It is an indicator function, indicating when The value is 1 if the space is occupied, and 0 otherwise. Representing the local entropy value, this index reflects the degree of irregularity in the distribution of obstacles throughout the scanned area.

[0132] Finally, the overall spatial complexity index is mapped to the interval [0,1] to obtain the spatial traversability score. The specific calculation formula is as follows:

[0133] ;

[0134] in, Represents spatial traversability score, Represents an indicator of overall spatial complexity. This is the theoretical maximum entropy value, set to 1 here. tanh is the hyperbolic tangent function used for smoothing the mapping. (Scoring) The closer a value is to 1, the more regular and predictable the spatial pattern, and the easier it is to pass through. The closer a value is to 0, the more complex and uncertain the spatial structure, and the more difficult it is to pass through.

[0135] Finally, the comprehensive score fusion and decision-making stage begins. The scores from the two different dimensions are weighted and linearly fused to obtain the final comprehensive feasibility score.

[0136] S = wg ·S ground + w s ·S space ;

[0137] Where S represents the comprehensive feasibility score, w g and w s Let w be the weighting coefficient, and satisfy w g + w s = 1, S ground S represents the ground accessibility score. space Representing spatial accessibility score, this fusion formula, by adjusting the weighting coefficients, can flexibly reflect different emphases on ground and spatial accessibility in different application scenarios. For example, for a robot operating in a flat warehouse but with narrow aisles between shelves, a higher weighting can be set. s For robots operating in confined spaces, a higher w can be set, while for robots operating in open outdoor environments with rugged terrain, a higher w can be set. g To focus more on ground undulations and roughness.

[0138] S5: Based on the comprehensive feasibility score, implement hierarchical decision-making and generate motion adjustment plans.

[0139] Furthermore, step S5 also includes:

[0140] The comprehensive feasibility score S and the preset safety threshold S are combined. threshold Perform real-time comparison;

[0141] If S ≥ S threshold Determine if the current path is feasible, and generate control instructions to track the original global reference path;

[0142] If S threshold If this happens, the local trajectory optimizer is triggered, attempting to generate a locally feasible trajectory that bypasses the current obstacle, guided by the global reference path.

[0143] If the local trajectory optimizer fails multiple times, it is determined to be a persistence obstacle, and then the global rollback mechanism is activated to update the multi-layer semantic map and replan a completely new global reference path.

[0144] Specifically, a comprehensive feasibility score is received, which is a normalized value between 0 and 1, with a preset safety threshold S. threshold The value is typically set between 0.65 and 0.75, and can be adjusted depending on the conservatism of the wheeled robot. The core decision-making process involves adjusting S and S... threshold Perform real-time comparisons and branch to different decision flows based on the results.

[0145] If S ≥ S threshold ​If the score is deemed satisfactory, path tracking is executed, and control commands are generated to track the original global reference path. If the overall feasibility score is higher than the safety threshold, it is determined that the path ahead meets the passage requirements in all dimensions of dynamic feasibility. At this point, the wheeled robot uses the global reference path as its ideal trajectory and its current pose as input. By minimizing the tracking error, it calculates the optimal linear velocity and angular velocity commands and sends them directly to the robot's underlying drive unit to control it to move smoothly along the original path.

[0146] If S threshold If the overall feasibility score is below a safety threshold, the path ahead is deemed impassable. At this point, the wheeled robot does not immediately abandon the global reference path but first initiates a diagnostic and local replanning sub-process. First, the root cause of the impassable path is analyzed. The decision-maker backtracks and analyzes the two sub-scores that constitute the overall feasibility score—ground accessibility score and spatial accessibility score—and even the three factors within the ground accessibility score, to determine the specific reasons for the low score. This analysis aims to provide intelligent guidance for the replanning strategy. If the spatial accessibility score is significantly low, it indicates that the main problem is narrow space or obstruction by large obstacles. The local trajectory optimizer will prioritize obstacle avoidance. If the ground accessibility score is significantly low, its sub-components need further analysis. A low slope feasibility score indicates an excessively steep slope. The local trajectory optimizer may attempt to find a transition area in the slope or calculate whether it can pass through by accelerating uphill. A low ground roughness score indicates an excessively rugged surface or a large number of small obstacles. The local planner will attempt to find a relatively flat area to traverse or generate a slow path that reduces bumps. A low hazard score indicates the detection of a liquid hazard. This is a very high-priority warning, typically meaning the area must be completely avoided. The local trajectory optimizer is likely to fail immediately, triggering a rapid global replanning.

[0147] Based on the above analysis, the local trajectory optimizer is activated. It defines a local optimization window near the current global reference path, using real-time perceived obstacle information, ground attributes, and the robot's kinematic and dynamic constraints as optimization conditions to quickly generate one or more smooth, collision-free local trajectories. Crucially, the system immediately re-executes the previous trajectories' feasibility analysis on this newly generated local trajectory, calculating its new comprehensive feasibility score S. new Only when S new ≥ S threshold Only then is the local trajectory adopted and executed. The wheeled robot will switch to this new local path, complete the detour, and then return to the global path.

[0148] ​If local replanning fails, a global rollback and replanning are triggered. If the local trajectory optimizer cannot find any satisfactory local trajectory within the set time or number of attempts—for example, if the passage is completely blocked, there is a large obstacle that cannot be bypassed, or a hazard that must be absolutely avoided—then the current obstacle is determined to be a persistent obstacle. The system then initiates a global rollback and replanning mechanism, which is a crucial feedback loop. The decision-maker packages the precise location, geometric features, and semantic information of the currently blocked area, such as "temporary obstacle - cargo box," and sends it back to the prior semantic environment modeling module. This module temporarily updates the navigation layer's two-dimensional occupied grid map, changing the grid status of the corresponding area from "idle" to "occupied."

[0149] Based on this updated global map, the path planning engine re-searches for a global path from the starting point to the target point, starting from the robot's current position. Since previously blocked passages have been marked as impassable, this planning will generate a completely new global path, fundamentally bypassing the obstruction area. Finally, the wheeled robot begins to follow this newly planned global path and continues to perform its navigation task.

[0150] Example 2: Based on the same inventive concept as the wheeled robot accessibility prediction method in a complex scenario described in the previous example, this application also provides a wheeled robot accessibility prediction system in a complex scenario. Please refer to the appendix. Figure 2 The system includes:

[0151] The prior semantic environment modeling module 11 is used to pre-collect indoor three-dimensional point cloud data through a three-dimensional laser scanner, generate an indoor point cloud map, and process the indoor point cloud map using a sparse convolution-based three-dimensional semantic segmentation network to generate a multi-layer semantic map.

[0152] The global path planning module 12 is used to extract the target point coordinates of the task instruction and use a path planning engine to search for a global reference path on a multi-layer semantic map. The global reference path consists of a series of ordered waypoints.

[0153] A real-time multimodal perception and fusion module 13 is used to mount a sensor array on the top of a wheeled robot. During the wheeled robot's movement along a global reference path, it collects real-time information about the surrounding environment and the robot itself to construct a local real-time dynamic environment model. The construction of the local real-time dynamic environment model further includes:

[0154] A sensor array is deployed on the top of a wheeled robot to acquire a multi-source heterogeneous dataset. The sensor array includes a 3D LiDAR, an RGB-D camera, and an IMU. The multi-source heterogeneous dataset includes laser point cloud data, visual image data, and IMU data.

[0155] The RANSAC plane fitting algorithm based on normal vector analysis is used to process laser point cloud data and extract the geometric contours of the surrounding environment.

[0156] Visual image data is processed using a deep learning-based segmentation network to obtain semantic labels for specific terrain features;

[0157] The body posture of the wheeled robot is obtained by solving the IMU data through Kalman filtering;

[0158] Based on the obtained geometric contours of the surrounding environment, semantic labels of special terrain, and the body posture of the wheeled robot, a local real-time dynamic environment model is constructed by fusing multi-source feature information through the extended Kalman filter algorithm.

[0159] The dynamic mobility assessment module 14 is used to perform mobility analysis of the wheeled robot based on a local real-time dynamic environment model, calculate the ground mobility score and the spatial mobility score, and fuse the ground mobility score and the spatial mobility score to obtain a comprehensive feasibility score. Obtaining the comprehensive feasibility score also includes:

[0160] The ground slope angle is obtained based on the geometric contour of the surrounding environment and the body posture in the local real-time dynamic environment model.

[0161] The maximum ramp angle that the wheeled robot can traverse is determined based on its own parameters. The specific formula is as follows:

[0162] ;

[0163] in This represents the maximum slope angle that a wheeled robot can traverse. This represents the maximum gradient based on the motor torque, with a value of [value missing]. ,in represents the maximum output torque of the motor, m represents the total mass of the wheeled robot, g represents the acceleration due to gravity, r represents the wheel radius, and arcsin represents the arcsine function. Represents the maximum slope based on stability, with a value of Where h represents the center of gravity height, w represents the wheelbase width, and arctan represents the arctangent function. Represents the maximum slope based on ground friction, with a value of ,in The coefficient of static friction between the wheel and the ground is represented by , arctan represents the arctangent function, and the maximum slope angle that the wheeled robot can ultimately traverse is obtained. The minimum value among the three is selected using the min function;

[0164] A slope feasibility scoring function is defined, which combines the ground slope angle and the maximum passable slope angle of the wheeled robot to calculate the slope feasibility score. The specific calculation formula is as follows:

[0165] ;

[0166] in, The slope feasibility score is represented by θ, which represents the ground slope angle. This represents the maximum slope angle that a wheeled robot can traverse. This represents the optimal slope angle threshold, which is typically set to 50%. ;

[0167] Based on the point cloud data of all ground points in the local real-time dynamic environment model, the roughness index is calculated using the following formula:

[0168] ;

[0169] Where R represents the roughness index, sqrt represents the square root, N is the total number of ground points in the point cloud data, and d i This represents the vertical distance from each ground point to the fitted plane;

[0170] The maximum permissible ground roughness for a wheeled robot is determined based on its own parameters, using the following formula:

[0171] ;

[0172] in This represents the maximum permissible ground roughness for wheeled robots. This represents the maximum permissible surface roughness based on the wheel radius, with a value of [value missing]. ,in The safety factor is represented by r, which represents the wheel radius. This represents the maximum permissible ground roughness considering suspension system corrections, and the value is... ,in This represents the static compression of the suspension system. Represents the suspension utilization factor. The maximum travel of the suspension system and the maximum permissible ground roughness Rmax of the wheeled robot are selected as the minimum value between the two by the min function.

[0173] A ground roughness scoring function is defined, and the ground roughness score is calculated by combining the roughness index with the maximum permissible ground roughness of the wheeled robot. The specific formula is as follows:

[0174] ;

[0175] in, R represents the surface roughness score, and R represents the roughness index. max This represents the maximum permissible ground roughness for the wheeled robot, and k represents the attenuation coefficient. Here, the value of the attenuation coefficient k is set to... ;

[0176] The system directly obtains the special terrain semantic labels from the local real-time dynamic environment model, determines whether there is a liquid hazard area within the target region, outputs the hazard source presence marker "isWater", and inputs the result into the defined hazard source scoring function.

[0177] ;

[0178] in, This represents the hazard rating; isWater indicates the presence of the hazard.

[0179] The ground accessibility score is obtained based on the slope feasibility score, ground roughness score, and hazard source score:

[0180] S ground = f slope (θ) · f roughness (R) · f water (isWater);

[0181] Among them, S ground Represents ground accessibility score, Represents the slope feasibility score. Represents the surface roughness score. Represents a hazard rating;

[0182] The 3D point cloud data obtained from a single scan is discretized into a regular voxel grid. Occupied voxels are selected, and the proportion of each voxel occupied within its 26-neighborhood is calculated. The specific calculation formula is as follows:

[0183] ;

[0184] in, This represents the proportion occupied by the 26 neighboring voxels. This represents the number of occupies in the 26 neighborhood voxels, where i, j, and k represent the index values ​​in the three dimensions, respectively.

[0185] Based on this ratio, the local entropy value of each occupied voxel is calculated using the following formula:

[0186] ;

[0187] in, This represents the local entropy value of the voxel. This represents the proportion of voxels that are occupied in the 26 neighboring voxels of this voxel;

[0188] After obtaining the local entropy values ​​of all occupied voxels, the overall spatial complexity index is calculated using the following formula:

[0189] ;

[0190] in, Represents an indicator of overall spatial complexity. The total number of occupied voxels. Represents an indicator function. Represents the local entropy value;

[0191] The overall spatial complexity index is mapped to the interval [0,1] to obtain the spatial traversability score. The specific calculation formula is as follows:

[0192] ;

[0193] in, Represents spatial traversability score, Represents an indicator of overall spatial complexity. Represents the theoretical maximum entropy value, and tanh is the hyperbolic tangent function;

[0194] The ground accessibility score and the spatial accessibility score are weighted and linearly fused to obtain the final comprehensive feasibility score:

[0195] S = w g ·S ground + w s ·S space ;

[0196] Where S represents the comprehensive feasibility score, w g and w s Let w be the weighting coefficient, and satisfy w g + w s = 1, S ground S represents the ground accessibility score. space Represents spatial traversability score;

[0197] The hierarchical autonomous decision-making and control module 15 is used to perform hierarchical decision-making based on the comprehensive feasibility score and generate motion adjustment schemes.

[0198] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0199] The above description of the disclosed embodiments enables those skilled in the art to make or use this application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of this application. Therefore, this application is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A wheeled robot passability prediction method in complex scenes, characterized in that, The method comprises: Pre-acquire indoor three-dimensional point cloud data by a three-dimensional laser scanner, generate an indoor point cloud map, process the indoor point cloud map by a three-dimensional semantic segmentation network based on sparse convolution, and generate a multi-layer semantic map; Extract the target point coordinates of the task instruction, search for a global reference path on the multi-layer semantic map by using a path planning engine, and the global reference path is composed of a series of ordered waypoints; Mount a sensor array on the top of the wheeled robot, and acquire surrounding environment and wheeled robot body information in real time during the movement of the wheeled robot along the global reference path, and construct a local real-time dynamic environment model, wherein the local real-time dynamic environment model further comprises: Lay out a sensor array on the top of the wheeled robot, and acquire a multi-source heterogeneous data set, the sensor array comprises a 3D laser radar, an RGB-D camera and an IMU, and the multi-source heterogeneous data set comprises laser point cloud data, visual image data and IMU data; Process the laser point cloud data by using a RANSAC plane fitting algorithm based on normal vector analysis, and extract the geometric profile of the surrounding environment; Process the visual image data by using a segmentation network based on deep learning, and obtain special terrain semantic labels; Solve the IMU data by Kalman filtering, and acquire the body attitude of the wheeled robot; Fuse the multi-source feature information by an extended Kalman filtering algorithm according to the obtained geometric profile of the surrounding environment, special terrain semantic labels and the body attitude of the wheeled robot, and construct a local real-time dynamic environment model; Perform passability analysis of the wheeled robot based on the local real-time dynamic environment model, calculate the ground passability score and the space passability score, fuse the ground passability score and the space passability score, and obtain a comprehensive feasibility score, wherein the comprehensive feasibility score further comprises: Obtain the ground slope angle according to the geometric profile of the surrounding environment and the body attitude in the local real-time dynamic environment model; Set the maximum passable slope angle of the wheeled robot based on the parameters of the wheeled robot, and the specific formula is: ； wherein represents the maximum passable slope angle of the wheeled robot, represents the maximum slope based on the motor torque, the value of which is wherein represents the maximum output torque of the motor, m represents the total mass of the wheeled robot, g represents the acceleration of gravity, r represents the wheel radius, arcsin represents the inverse sine function, represents the maximum slope based on stability, the value of which is wherein h represents the height of the center of gravity, w represents the track width, arctan represents the inverse tangent function, represents the maximum slope based on the ground friction, the value of which is wherein represents the static friction coefficient between the wheel and the ground, arctan represents the inverse tangent function, and the finally obtained maximum passable slope angle of the wheeled robot is the minimum value among the three is selected by the min minimum function; Set a slope feasibility score function, calculate the slope feasibility score by combining the ground slope angle and the maximum passable slope angle of the wheeled robot, and the specific calculation formula is: ； wherein, represents the slope feasibility score, represents the ground slope angle, represents the maximum passable slope angle of the wheeled robot, represents the optimal slope angle threshold, typically set to 50% ; Calculate the roughness index according to all ground point cloud data in the local real-time dynamic environment model, and the specific calculation formula is: ； where R represents roughness index, sqrt represents square root, N is the total number of ground points in the point cloud data, d i represents the vertical distance of each ground point to the fitted plane; Set the maximum allowable ground roughness of the wheeled robot based on the parameters of the wheeled robot, and the specific formula is: ； wherein Rmax represents the maximum allowed ground roughness for the wheeled robot, Rmax represents the maximum allowed ground roughness based on the wheel radius, the value being wherein Rmax represents the safety factor, r represents the wheel radius, Rmax represents the maximum allowed ground roughness taking into account the suspension system correction, the value being wherein Rmax represents the static compression of the suspension system, Rmax represents the suspension utilization factor, Rmax represents the maximum travel of the suspension system, the maximum allowed ground roughness Rmax of the final wheeled robot being chosen by the min minimum function between the two; Set a ground roughness score function, calculate the ground roughness score by combining the roughness index and the maximum allowable ground roughness of the wheeled robot, and the specific formula is: ； wherein, represents a ground roughness score, R represents a roughness index, R max represents a maximum allowable ground roughness for the wheeled robot, k represents an attenuation coefficient, and the value of the attenuation coefficient k is set to ; Directly obtain the special terrain semantic labels in the local real-time dynamic environment model, judge whether there is a liquid danger area in the target area, output a danger source existence flag isWater, and input the result to a set danger source score function: ； wherein, represents a hazard source score, and isWater represents a hazard source existence flag. Obtain the ground passability score according to the slope feasibility score, the ground roughness score and the danger source score: S ground = f slope (θ) · f roughness (R) · f water (isWater); S ground representing a ground passability score, representing a slope feasibility score, representing a ground roughness score, representing a hazard source score; Discretize the three-dimensional point cloud data obtained by a single scan into a regular voxel grid, select the occupied voxels and calculate the proportion of occupied voxels in the 26-neighborhood voxels, and the specific calculation formula is: ； wherein, represents the proportion of the 26-neighborhood voxels that are occupied, represents the number of occupied voxels in the 26-neighborhood, i, j, k represent the index values in three dimensions, respectively; Based on the ratio, the local entropy value of each occupied voxel is calculated, and the specific calculation formula is: ； wherein, a local entropy value representing the voxel, a proportion of occupied voxels in the neighborhood of the voxel 26; After obtaining the local entropy value of all occupied voxels, the overall spatial complexity index is calculated, and the specific formula is: ； wherein, represents an overall spatial complexity indicator, is the total number of occupied voxels, represents an indicator function, represents a local entropy value; Map the overall spatial complexity index to the interval [0, 1] to obtain the spatial passability score, and the specific calculation formula is: ； wherein, represents the spatial navigability score, represents the overall spatial complexity index, represents the theoretical maximum entropy value, tanh is the hyperbolic tangent function; Weighted linear fusion of ground passability score and spatial passability score to obtain the final comprehensive feasibility score: S = w g • S ground + w s • S space ; Wherein, S represents the comprehensive feasibility score, w g and w s are weight coefficients, and satisfy w g + w s = 1, S ground represents the ground passability score, S space represents the space passability score; Based on the comprehensive feasibility score, hierarchical decision is executed to generate a motion adjustment scheme.

2. The method for predicting the passability of a wheeled robot in a complex scenario as described in claim 1, characterized in that, Generate a multi-layer semantic map, including: Based on the planning of the real scene survey, the scanning path is scanned using a three-dimensional laser scanner according to the scanning path, the original point cloud data is collected, and the multi-station fragmented original point cloud is generated; Pretreatment of multi-station fragmented original point cloud to obtain indoor point cloud map, the pretreatment includes point cloud registration, point cloud denoising, point cloud compression; Obtain the indoor point cloud map, and perform semantic segmentation using a three-dimensional semantic segmentation network based on sparse convolution to assign semantic labels to the indoor point cloud map; Based on the indoor point cloud map with added semantic labels, a multi-layer semantic map for a wheeled robot is constructed, which includes a navigation layer and a semantic layer.

3. The method for predicting the passability of a wheeled robot in a complex scenario as described in claim 1, characterized in that, Use the path planning engine to search for a global reference path on the multi-layer semantic map, including: Receive task instructions and multi-layer semantic map, obtain two-dimensional occupancy grid map and extract target point coordinates; A The algorithm searches for the optimal path on a two-dimensional occupancy grid map; Eliminate the jagged path segment in the optimal path through path smoothing processing to generate a global reference path that meets the kinematic constraints of the wheeled robot.

4. The method for predicting the passability of a wheeled robot in a complex scenario as described in claim 1, characterized in that, Generate a motion adjustment scheme, including: comprehensive feasibility score S with a preset safety threshold S threshold real-time comparison is made; If S ≥ S threshold , determine that the current path is feasible, and generate control instructions that follow the original globally referenced path. If S < S threshold then trigger the local trajectory optimizer to attempt to generate a local feasible trajectory that bypasses the current obstacle under the guidance of the global reference path; If the local trajectory optimizer fails multiple attempts, it is determined to be a persistent obstacle, and then the global rollback mechanism is started, the multi-layer semantic map is updated and a brand new global reference path is planned.

5. A wheeled robot traversability prediction system in complex scenes, characterized by, The system is used to implement the wheeled robot passability prediction method in a complex scene according to any one of claims 1 to 4, and the system comprises: The prior semantic environment modeling module is used to pre-collect indoor three-dimensional point cloud data by a three-dimensional laser scanner, generate an indoor point cloud map, and process the indoor point cloud map using a three-dimensional semantic segmentation network based on sparse convolution to generate a multi-layer semantic map; The global path planning module is used to extract the target point coordinates of the task instructions, and use the path planning engine to search for a global reference path on the multi-layer semantic map, which is composed of a series of ordered waypoints; The real-time multi-modal perception fusion module is used to mount a sensor array on the top of the wheeled robot, and collect surrounding environment and wheeled robot body information in real time during the movement of the wheeled robot along the global reference path, and construct a local real-time dynamic environment model, wherein the local real-time dynamic environment model further comprises: Mount the sensor array on the top of the wheeled robot to obtain a multi-source heterogeneous data set, the sensor array includes a 3D laser radar, an RGB-D camera and an IMU, and the multi-source heterogeneous data set includes laser point cloud data, visual image data and IMU data; The laser point cloud data is processed by using a RANSAC plane fitting algorithm based on normal vector analysis to extract the geometric profile of the surrounding environment; The visual image data is processed by using a segmentation network based on deep learning to obtain a special terrain semantic label; The IMU data is solved by using Kalman filtering to obtain the body pose of the wheeled robot; The geometric profile of the surrounding environment, the special terrain semantic label and the body pose of the wheeled robot are obtained, and multi-source feature information is fused by using an extended Kalman filtering algorithm to construct a local real-time dynamic environment model; The dynamic passability evaluation module is used for passability analysis of the wheeled robot based on the local real-time dynamic environment model, calculation of a ground passability score and a space passability score, fusion of the ground passability score and the space passability score, and obtaining of a comprehensive feasibility score, wherein the obtaining of the comprehensive feasibility score further comprises: a ground slope angle is obtained according to the geometric profile of the surrounding environment and the body pose in the local real-time dynamic environment model; a maximum passable slope angle of the wheeled robot is set based on the parameters of the wheeled robot, and a specific formula is as follows: ； wherein represents the maximum passable slope angle of the wheeled robot, represents the maximum slope based on the motor torque, the value of which is wherein represents the maximum output torque of the motor, m represents the total mass of the wheeled robot, g represents the acceleration of gravity, r represents the wheel radius, arcsin represents the inverse sine function, represents the maximum slope based on stability, the value of which is wherein h represents the height of the center of gravity, w represents the track width, arctan represents the inverse tangent function, represents the maximum slope based on ground friction, the value of which is wherein represents the static friction coefficient between the wheel and the ground, arctan represents the inverse tangent function, and the finally obtained maximum passable slope angle of the wheeled robot is the minimum value among the three is selected by the min minimum value function; a slope feasibility score function is set, and a slope feasibility score is calculated by combining the ground slope angle and the maximum passable slope angle of the wheeled robot, and a specific calculation formula is as follows: ； wherein, represents the slope feasibility score, represents the ground slope angle, represents the maximum passable slope angle of the wheeled robot, represents the optimal slope angle threshold, typically set to 50% ; a roughness index is calculated according to all ground point cloud data in the local real-time dynamic environment model, and a specific calculation formula is as follows: ； where R represents roughness index, sqrt represents square root, N is the total number of ground points in the point cloud data, d i represents the vertical distance of each ground point to the fitted plane; a maximum allowable ground roughness of the wheeled robot is set based on the parameters of the wheeled robot, and a specific formula is as follows: ； wherein Rmax represents the maximum allowed ground roughness for the wheeled robot, Rmax represents the maximum allowed ground roughness based on the wheel radius, the value being wherein Rmax represents the maximum allowed ground roughness for the wheeled robot, Rmax represents the maximum allowed ground roughness considering the suspension system correction, the value being wherein Rmax represents the maximum allowed ground roughness for the wheeled robot, Rmax represents the maximum allowed ground roughness for the wheeled robot, Rmax represents the maximum allowed ground roughness for the wheeled robot, a ground roughness score function is set, and a ground roughness score is calculated by combining the roughness index and the maximum allowable ground roughness of the wheeled robot, and a specific formula is as follows: ； wherein, represents a ground roughness score, R represents a roughness index, R max represents a maximum allowable ground roughness for the wheeled robot, k represents an attenuation coefficient, and the value of the attenuation coefficient k is set to ; a special terrain semantic label in the local real-time dynamic environment model is directly obtained, it is judged whether a liquid danger area exists in a target region, an existence flag isWater of a danger source is output, and the result is input into a set danger source score function: ； wherein, represents a hazard source score, and isWater represents a hazard source existence flag. a ground passability score is obtained according to the slope feasibility score, the ground roughness score and the danger source score: S ground = f slope (θ) · f roughness (R) · f water (isWater); S ground representing a ground passability score, representing a slope feasibility score, representing a ground roughness score, representing a hazard source score; three-dimensional point cloud data obtained by a single scan is discretized into a regular voxel grid, an occupied voxel is selected, and a proportion of occupied voxels in a 26-neighborhood voxel is calculated, and a specific calculation formula is as follows: ； wherein, represents the proportion of the 26-neighborhood voxels being occupied, represents the number of the 26-neighborhood voxels being occupied, i, j, k represent the index values in three dimensions, respectively; based on the proportion, a local entropy value of each occupied voxel is calculated, and a specific calculation formula is as follows: ； wherein, a local entropy value representative of the voxel, a proportion of occupied voxels in the neighborhood of the voxel 26; after the local entropy values of all occupied voxels are obtained, a whole space complexity index is calculated, and a specific formula is as follows: ； wherein, represents an overall spatial complexity indicator, is the total number of occupied voxels, represents an indicator function, represents a local entropy value; the whole space complexity index is mapped to an interval of [0, 1] to obtain a space passability score, and a specific calculation formula is as follows: ； wherein, represents the spatial navigability score, represents the overall spatial complexity index, represents the theoretical maximum entropy value, tanh is the hyperbolic tangent function; the ground passability score and the space passability score are linearly fused by weighting to obtain a final comprehensive feasibility score: S = w g • S ground + w s • S space ; Wherein, S represents the comprehensive feasibility score, w g and w s are weight coefficients, and satisfy w g + w s = 1, S ground represents the ground passability score, S space represents the space passability score; the hierarchical autonomous decision and control module is used for executing hierarchical decision based on the comprehensive feasibility score to generate a motion adjustment scheme.

Images