Robot automatic obstacle avoidance system and method based on multi-source sensors

Abstract

The application provides a robot automatic obstacle avoidance system and method based on a multi-source sensor, relates to the technical field of mobile robots, and collects environment data through the multi-source sensor, obtains obstacle information, and converts the environment data to a local coordinate system of the robot; an improved dynamic window method is used to construct a time-space potential field between the robot and the obstacle, to calculate a safe channel around the robot, to introduce an adaptive weight factor to dynamically adjust the safe channel, to generate an obstacle avoidance track of the robot, and to use a gradient descent method to optimize the speed and steering angle of the robot, so that the robot moves along the obstacle avoidance track. In this way, the problems of insufficient environment perception accuracy, weak dynamic prediction ability and low obstacle avoidance decision efficiency can be solved.

Description

Technical Field

[0001] This invention relates to the field of mobile robot technology, and more specifically, to a robot automatic obstacle avoidance system and method based on multi-source sensors. Background Technology

[0002] Obstacle avoidance technology for service robots has evolved from simple avoidance to intelligent avoidance. Early obstacle avoidance systems relied primarily on single sensors for basic collision prevention. With the development of multi-source sensor technology, sensor fusion perception has gradually become the mainstream solution. In dynamic indoor scenarios, service robots need to simultaneously handle various types of obstacles, including static facilities, mobile devices, and crowds, which places higher demands on obstacle avoidance systems. Currently, mainstream obstacle avoidance methods include path planning based on artificial potential fields, dynamic windowing based on sampling, and behavioral decision-making based on probabilistic reasoning. However, when dealing with densely populated and spatially varied indoor scenarios such as hospitals, shopping malls, and hotels, these methods often struggle to accurately identify and predict the motion characteristics and behavioral patterns of various obstacles, resulting in insufficient accuracy and real-time performance in obstacle avoidance decisions. Especially in human-robot hybrid scenarios, due to a lack of in-depth understanding of human motion characteristics and group behavior patterns, existing methods struggle to achieve natural and smooth obstacle avoidance behavior.

[0003] Existing obstacle avoidance systems for indoor service robots in dynamic scenarios suffer from the following prominent problems: Sensor data fusion strategies are overly mechanical, typically employing simple superposition of preset fixed weights, failing to dynamically adjust the reliability of each sensor based on environmental characteristics and obstacle states; motion prediction for dynamic obstacles relies excessively on historical trajectory fitting, lacking the extraction and utilization of high-level semantic information such as human gait characteristics and group interaction patterns; obstacle avoidance decisions are often limited to local information processing at a single spatiotemporal scale, failing to effectively integrate multi-scale environmental features to achieve global collaborative decision-making; existing path planning algorithms have high computational loads when handling multi-constrained dynamic scenarios, making it difficult to meet the stringent real-time requirements in complex environments. These technical problems are particularly pronounced in densely populated, space-constrained indoor environments, severely limiting the practical application effectiveness of service robots. Summary of the Invention

[0004] To address the aforementioned technical problems, this invention is proposed. This invention provides a robot automatic obstacle avoidance system and method based on multi-source sensors, which can, to some extent, solve the problems of existing service robots being unable to accurately identify and predict the motion characteristics and behavior patterns of various obstacles in dynamic indoor scenes, lacking a deep understanding of human gait characteristics and group interaction patterns, and struggling to simultaneously achieve real-time performance and accuracy in obstacle avoidance decisions.

[0005] According to one aspect of the present invention, a method for automatic obstacle avoidance of a robot based on multi-source sensors is provided, comprising:

[0006] The robot collects environmental data using multi-source sensors installed on it, obtains the position coordinates, direction of motion, and velocity information of obstacles, and converts the environmental data into the robot's local coordinate system.

[0007] Based on the environmental data, an improved dynamic window method is used to construct the spatiotemporal potential field between the robot and the obstacles;

[0008] The safety passage around the robot is calculated based on the spatiotemporal potential field, and an adaptive weighting factor is introduced to dynamically adjust the safety passage.

[0009] An obstacle avoidance trajectory for the robot is generated within the safety passage. The robot's speed and turning angle are optimized using the gradient descent method, enabling the robot to move along the obstacle avoidance trajectory.

[0010] Furthermore, in the local coordinate system, a spatiotemporal potential field is constructed for each detected obstacle, including:

[0011] Obstacles are classified based on environmental semantic information: if the obstacle is a human target, feature point sequences are obtained through human key point detection, and its movement trend is predicted by combining gait features and movement patterns.

[0012] If the obstacle is a mobile device, the corresponding kinematic model is matched according to the device type identification result, and its motion trajectory is predicted by combining angular velocity, acceleration and velocity constraints.

[0013] If multiple pedestrian targets are detected moving together, their group behavior characteristics are extracted, the spatial relationship and motion characteristics between the targets are analyzed, a group motion model is constructed, and the influence weight of the group structure is dynamically adjusted.

[0014] Furthermore, if the obstacle is a human target, feature point sequences of the torso center position, hip joint position, and foot position are obtained through human key point detection, and prediction model parameters are constructed, expressed by the formula:

[0015]

[0016]

[0017] in, This indicates the predicted coordinates of the next position. Indicates the current position coordinates. Indicates step size, This represents the gait adjustment factor, reflecting the effects of acceleration and deceleration. Indicates the angle of the torso. This represents the support phase coefficient, with a value range of [0,1], and represents the normalized progress of the gait period. Indicates the direction angle of the supporting leg. Indicates the vertical reference angle. Indicates the characteristic angle, Indicates the current walking speed. This indicates normal walking speed.

[0018] Furthermore, if the obstacle is a motorized device, a device type parameter is introduced to classify the device type into wheeled devices and omnidirectional devices, and the constraint models of angular velocity, acceleration and speed are adjusted accordingly.

[0019] Furthermore, if the obstacle is a group of pedestrians moving together, an influence coefficient for inter-group interaction is introduced to adjust the influence weight when a new member joins or leaves the group.

[0020] Furthermore, the spatiotemporal potential field employs a three-layer structure to represent the influence of obstacles:

[0021] The underlying potential field adopts a Gaussian distribution function based on the feature size of the obstacle;

[0022] The distribution pattern of the intermediate potential field is adjusted according to scene elements;

[0023] The top-level potential field establishes an envelope surface based on the population density distribution.

[0024] Furthermore, based on the constructed three-layer spatiotemporal potential field structure, the safe passage around the robot is calculated, and the passage characteristics are dynamically adjusted by introducing an adaptive weighting factor;

[0025] The adaptive weighting factors include: obstacle state influence factor, robot state influence factor, and environmental feature influence factor.

[0026] Furthermore, the feasibility assessment value of the safe passage is calculated through a two-level weight adjustment mechanism;

[0027] First, the comprehensive potential field value is obtained by weighting the bottom layer weight, the middle layer weight, and the top layer weight. Then, the overall feasibility assessment value is obtained by adjusting the weight factor.

[0028] Furthermore, based on the distribution of the feasibility assessment values, the search space for trajectory generation is determined, and the gradient descent method is used to jointly optimize the robot's speed and steering angle, where the baseline values ​​for speed and steering angle are the current speed and heading angle, respectively.

[0029] According to another aspect of the present invention, a robot automatic obstacle avoidance system based on multi-source sensors is provided, comprising:

[0030] The data acquisition module is used to collect environmental data based on the multi-source sensors installed on the robot, obtain information about obstacles, and convert the environmental data into the robot's local coordinate system.

[0031] The construction module, based on the environmental data, uses an improved dynamic window method to construct the spatiotemporal potential field between the robot and obstacles;

[0032] The adjustment module calculates the safety passage around the robot based on the spatiotemporal potential field and introduces an adaptive weighting factor to dynamically adjust the safety passage;

[0033] The optimization module generates an obstacle avoidance trajectory for the robot within the safety passage and uses the gradient descent method to optimize the robot's speed and turning angle, enabling the robot to move along the obstacle avoidance trajectory.

[0034] Compared with existing technologies, this invention achieves accurate environmental perception by constructing an adaptive multi-layer potential field model and employs a multi-scale optimization strategy to jointly optimize robot speed and steering angle, ensuring smooth and continuous trajectory while guaranteeing obstacle avoidance safety. In particular, when handling human-robot hybrid scenarios, it can deeply understand human gait characteristics and group interaction patterns. Through dynamic evaluation and real-time optimization mechanisms, the robot can complete obstacle avoidance tasks more naturally and smoothly, significantly improving obstacle avoidance performance and service efficiency in dynamic and complex indoor environments. Attached Figure Description

[0035] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort. In the drawings:

[0036] Figure 1 This is a flowchart of a robot automatic obstacle avoidance method based on multi-source sensors according to an embodiment of the present invention;

[0037] Figure 2 A flowchart for constructing a spatiotemporal potential field according to an embodiment of the present invention. Detailed Implementation

[0038] Hereinafter, exemplary embodiments according to the present invention will be described in detail with reference to the accompanying drawings. Obviously, the described embodiments are merely some embodiments of the present invention, and not all embodiments of the present invention. It should be understood that the present invention is not limited to the exemplary embodiments described herein.

[0039] As mentioned in the background section, existing technologies have three main problems: First, the sensor data fusion methods of existing service robots are too simple. The linear combination with fixed weights cannot dynamically adjust the reliability of each sensor according to scene characteristics, resulting in insufficient environmental perception accuracy. Second, the motion prediction of dynamic obstacles relies too much on historical trajectories and lacks a deep understanding of human gait characteristics and group behavior patterns, making it difficult to accurately assess potential risks. Third, obstacle avoidance decisions are often limited to local information processing at a single spatiotemporal scale, and the computational load is high when dealing with multi-constrained dynamic scenes, making it difficult to simultaneously meet the requirements of real-time performance and accuracy.

[0040] Figure 1 This is a system block diagram of a robot automatic obstacle avoidance method based on multi-source sensors according to an embodiment of the present invention. Figure 1 As shown, the automatic obstacle avoidance method for robots based on multi-source sensors includes:

[0041] S1: Based on the multi-source sensors installed on the robot, environmental data is collected to obtain the position coordinates, movement direction and speed information of obstacles, and the environmental data is converted into the robot's local coordinate system.

[0042] The robot's sensor configuration adopts a "three-in-one" multi-source sensor solution: a 360-degree LiDAR is installed on the top of the robot to obtain information on obstacles on a large horizontal plane; a depth camera is installed at the front of the robot to obtain three-dimensional structural information of obstacles in front; and eight ultrasonic sensors are evenly installed around the bottom of the robot to supplement the detection of low obstacles.

[0043] The lidar has a scanning frequency of 20Hz, a scanning angle resolution of 0.25 degrees, and a ranging range of 0.1 meters to 30 meters; the depth camera has a field of view of 85 degrees, a frame rate of 30fps, and a depth accuracy of less than 1 centimeter within a 2-meter range; the ultrasonic sensor has a detection range of 0.02 meters to 4 meters and a transmission frequency of 40kHz.

[0044] During data acquisition, the horizontal plane contour point cloud data of the obstacle is first obtained using LiDAR. The DBSCAN clustering algorithm is then used to segment the point cloud data. The search radius of the DBSCAN clustering algorithm is one-eighth of the robot chassis width, and the minimum number of sampling points is 5% of the number of sampling points in a single LiDAR scan. During clustering, a neighborhood is constructed for each point with the search radius as its range. When the number of points in the neighborhood is greater than the minimum number of sampling points, it is marked as a core point. Based on the density reachability principle, all density-connected points are grouped into the same category, resulting in multiple independent point cloud clusters. Each point cloud cluster represents a potential obstacle. For each point cloud cluster, the centroid position of the obstacle is obtained by calculating the mean coordinates of all points in the cluster.

[0045] Secondly, using the depth images acquired by the depth camera, the moving target in front is extracted by the background subtraction method, and the three-dimensional spatial coordinates of the target are calculated; at the same time, the ultrasonic sensor detects nearby obstacles around the robot in real time.

[0046] For the motion information of dynamic obstacles, the instantaneous velocity and direction of motion of the obstacle are calculated by the difference between two consecutive frames of data. Specifically, the displacement vector of the obstacle is obtained by calculating the position change of the obstacle's center of gravity at adjacent moments, and then the velocity magnitude and direction angle are obtained.

[0047] Because the various sensors operate in different coordinate systems, all sensor data needs to be uniformly transformed into the robot's local coordinate system. The transformation process is based on the installation position parameters of each sensor, using a rigid body transformation matrix to convert data from different coordinate systems to a right-handed coordinate system with the robot's geometric center as the origin and the forward direction as the positive X-axis.

[0048] To ensure the timing synchronization and accuracy of data fusion across multiple sensors, all sensor data acquisition is based on the unified clock of the robot's main controller. During data acquisition, timestamps are recorded for the LiDAR, depth camera, and ultrasonic sensors, and all data is aligned to a 50ms control cycle. The LiDAR point cloud data is time-stamped using linear interpolation to maintain consistency with the depth camera's frame rate, while the ultrasonic sensor selects the latest valid data within each control cycle.

[0049] In terms of spatial alignment, calibration experiments were conducted to obtain the installation position and orientation parameters of each sensor, including 3D position offset (x, y, z) and attitude deviation (pitch angle, yaw angle, roll angle), and a rigid body transformation matrix was constructed. The horizontal point cloud data of the lidar was mapped to the local coordinate system based on the rigid body transformation matrix; the 3D structural information of the depth camera was transformed into coordinates in the same way; the single-point distance data of the ultrasonic sensor was calculated by combining its installation angle and robot geometric parameters, and converted into a 2D position in the local coordinate system.

[0050] For the fusion of multi-sensor data, obstacle information unified to a local coordinate system is processed using a weighted fusion method. LiDAR provides horizontal point clouds with high long-range detection confidence, while depth cameras perform excellently in close-range 3D structure detection, and ultrasonic sensors supplement information on low-lying obstacles. During fusion, the multi-source information on the same obstacle is weighted and averaged based on the detection confidence of each sensor. The confidence level is calculated based on the ranging accuracy of the sensors and the distance to the obstacle.

[0051] Finally, the state information of the obstacle in the robot's local coordinate system is obtained, including: position coordinates (x, y), motion velocity v, and motion direction angle θ.

[0052] Furthermore, considering the potential noise in the data, Kalman filtering is used to smooth the acquired obstacle state information to improve data reliability. Meanwhile, to ensure real-time performance, the entire data acquisition and processing cycle is controlled within 50ms.

[0053] S2: Based on the environmental data, an improved dynamic window method is used to construct the spatiotemporal potential field between the robot and the obstacles.

[0054] In the robot's local coordinate system, a spatiotemporal potential field is constructed for each detected obstacle.

[0055] First, obstacles are classified based on environmental semantic information. When the obstacle is a human target, a dynamic model considering individual movement intentions is constructed, and its movement trend is predicted based on human gait characteristics and movement patterns. When the obstacle is a motorized device, a constraint model is constructed based on its kinematic characteristics. When multiple pedestrian targets are detected moving together, their group behavior characteristics are extracted, and a prediction model considering group movement patterns is established.

[0056] Specifically, when the obstacle is a human body, the system first obtains feature point sequences such as the center position of the torso, the position of the hip joint, and the position of the feet through human key point detection, and then constructs the prediction model parameters, expressed by the formula:

[0057]

[0058]

[0059] in, This indicates the predicted coordinates of the next position. Indicates the current position coordinates. Indicates step size, This represents the gait adjustment factor, reflecting the effects of acceleration and deceleration. Indicates the angle of the torso. This represents the support phase coefficient, with a value range of [0,1], and represents the normalized progress of the gait period. Indicates the direction angle of the supporting leg. Indicates the vertical reference angle. Indicates the characteristic angle, Indicates the current walking speed. This indicates normal walking speed.

[0060] By introducing gait correction coefficients It can dynamically reflect the gait characteristics of a human target under different movement states. Specifically, Incorporating human body orientation deviation Ratio of speed change This effectively solves the problem of ignoring dynamic gait features in human target prediction. Furthermore, it addresses this issue through orientation correction parameters. Direction of the supporting leg and orientation angle By incorporating deviations into the modeling, the dynamic behavior of the human body when turning or changing direction can be predicted more accurately. This improves the prediction accuracy for complex gaits. It also makes the prediction results more adaptable to individual characteristics (such as stride length and direction changes), especially demonstrating higher accuracy in non-linear motion scenarios such as pedestrian turning and obstacle avoidance.

[0061] When the stance phase begins, the next movement direction is predicted based on the position of the supporting leg and the trunk tilt angle. In the middle of the stance phase, stride length is predicted using the hip joint trajectory. As the swing phase approaches, the next foot placement is predicted by combining current gait parameters and historical gait data. If a sudden acceleration or deceleration is detected, the prediction model parameters are adjusted promptly based on changes in the trunk forward tilt angle.

[0062] When the obstacle is a mobile device, the device type is first identified and the corresponding kinematic model is matched, as expressed by the formula:

[0063]

[0064]

[0065] in, This indicates the device's position coordinates at the predicted time. Indicates the initial position coordinates of the device. Indicates the magnitude of the equipment's linear velocity. The predicted time is indicated by λ, which is a parameter for the equipment type: λ=1 for wheeled equipment and λ=0 for omnidirectional platforms. Indicates the initial heading angle of the equipment. Indicates the angular velocity of the device. Indicates the magnitude of the device's acceleration. Maximum steering angle, Wheelbase , It is linear velocity Components in Cartesian coordinate system , It is acceleration The components in the Cartesian coordinate system, where μ is the coefficient of ground friction and g is the acceleration due to gravity. Indicates the maximum acceleration of the equipment. Indicates the weight of the equipment. Indicates the load quality.

[0066] The introduction of the device type parameter λ addresses the issue of insufficient prediction accuracy caused by uniform modeling of different obstacle types. Specifically, this parameter distinguishes between wheeled devices and omnidirectional devices, and adjusts the constraint models for angular velocity, acceleration, and speed accordingly, improving adaptability to the motion patterns of different device types. Simultaneously, it retains the ability to accurately model specific kinematic characteristics of the devices. For example, the angular velocity of wheeled devices is limited by wheelbase and maximum steering angle, while the angular velocity of omnidirectional devices incorporates the maximum angular velocity, enhancing the prediction accuracy for diverse devices.

[0067] For wheeled mobile equipment, minimum turning radius constraints are established based on wheelbase and steering angle; for omnidirectional mobile platforms, acceleration limits in all directions are considered. When the equipment is in uniform linear motion, a constant speed model is used for prediction; when the equipment is turning, the trajectory is predicted based on Ackerman steering geometry; when the equipment speed changes, acceleration constraints are incorporated into the prediction model. If the equipment is carrying cargo, the motion constraint parameters are adjusted accordingly based on the cargo's mass and dimensions.

[0068] When multiple pedestrian targets are detected moving together, the spatial relationships and motion characteristics between the targets are first analyzed to construct a prediction model, expressed by the formula:

[0069]

[0070] in, Representing a group Predicted location coordinates at time [time]. Indicates the coordinates of the group's center position. Indicates the average speed of the group. Indicates the predicted time. Member and The actual distance between them Indicates the desired comfortable spacing. Member point to The unit direction vector, This represents the influence coefficient of interactions between groups. For structural weights, Member The moment of joining the group The characteristic time represents the time scale at which new members integrate into the group.

[0071] By adjusting the influence weights of new members joining or leaving the group through the influence coefficients of inter-group interactions, the system can dynamically adapt to changes in group structure, exhibiting higher robustness, especially in dynamic scenarios with dense crowds, such as pedestrian walkways or squares. Furthermore, the formula... By capturing the actual distance between individuals relative to desired comfortable distance The deviation describes the dynamic interaction among group members, significantly improving the ability to describe the behavioral characteristics within a group.

[0072] If adjacent targets are spaced less than a normal walking distance and have similar speed directions, they are classified into the same group. For identified groups, a motion model of the group center is first established, and the overall positional changes of the group are tracked by calculating the weighted average position of all members and the average speed of the group. Secondly, the internal structure of the group is analyzed. When the group exhibits a leader-follower pattern, the focus is on predicting the movement intention of the leader target, i.e., increasing the structural weight to highlight the leader's influence. When the group exhibits a parallel pattern, the weight coefficient is reduced to extract lateral spacing features. If the group structure changes, such as when a new member joins, their joining time is recorded, and their influence is adjusted using a time decay term. When a member leaves, their weight is automatically reduced until they disappear using the same time decay mechanism, and the group center position, average speed, and structural features are recalculated to achieve dynamic updates to the prediction model.

[0073] The predicted results are transformed into a spatiotemporal potential field for expression. In the construction of the spatiotemporal potential field, a three-layer structure is used to express the influence of obstacles.

[0074] The bottom potential field adopts a Gaussian distribution function based on the feature size of the obstacle, and the distribution range extends along the direction of motion to the distance of the velocity multiplied by the prediction time domain; the middle potential field adjusts the distribution shape according to the scene elements, extending the potential field to the entire width of the doorway in the lobby and compressing the distribution along the wall direction in the corridor; the top potential field establishes an envelope surface based on the population density distribution, and when the population density is greater than two people per square meter, the entire population area is regarded as a continuous potential field.

[0075] Specifically, the underlying potential field adopts a Gaussian distribution function based on the feature size of obstacles, and the distribution range is determined by the output of various obstacle prediction models: for human targets, the step size and gait adjustment coefficients are centered on the predicted next position coordinates. The basic distribution range is determined, with the potential field extending along the predicted trunk-oriented angle. For mobile equipment, the distribution pattern is differentiated based on the equipment type parameters, centered on the predicted location. Wheeled equipment forms a fan-shaped distribution considering the minimum turning radius, while omnidirectional platforms form a uniform circular distribution. For group targets, the distribution range is determined by the group's average speed and the predicted time domain, centered on the group's predicted location. The distribution characteristics are adjusted according to structural weights: leader-follower targets extend along the direction of movement, while parallel targets maintain width laterally. All distribution ranges extend to the distance of their respective predicted speed multiplied by the predicted time domain.

[0076] The distribution pattern of the mid-level potential field is adjusted according to scene elements. In the lobby, the potential field is extended to the entire width of the doorway to accommodate the passage needs of the group. In the passageway, the potential field is compressed and distributed along the wall direction according to the predicted motion characteristics of obstacles: for human targets, the motion direction predicted by gait is considered; for motorized equipment, its kinematic constraints are combined; for group targets, it is adjusted based on the group structure characteristics and overall motion trend.

[0077] The top-level potential field establishes an envelope surface based on the population density distribution. When the population density is greater than two people per square meter, the discrete individual predictions are integrated into a continuous potential field with the predicted position of the population center as the reference. The range of the potential field is dynamically adjusted by the influence coefficient of the interaction between the populations as the population structure changes.

[0078] For the calculation of the underlying potential field, a basic distribution is established with the current position of the obstacle as the center. When the obstacle is a single target, the potential field range is no less than twice the distance required for the robot to decelerate from its current speed to a stop; when the obstacles are a group of targets, the potential field range is the sum of the diagonal length of the rectangle surrounding the group and the robot's safe distance. The boundary of the influence range is determined by the superposition region of the obstacle's velocity vector and the robot's reachable velocity domain.

[0079] Environmental semantic constraints are introduced into the mid-level potential field. If the obstacle is located in a passageway such as a doorway, the width of the passageway is detected and the potential field is stretched along the center line of the passageway to half the length of the passageway. If the obstacle is in an open area, the passage area marked by ground markings is extracted and the potential field distribution is aligned with the direction of the main pedestrian passageway. If the distance between the obstacle and the wall is less than the width of a normal wheelchair or the distance between surrounding obstacles is less than the shoulder width of an adult, the potential field strength is increased in the direction perpendicular to the wall or the line connecting the obstacles.

[0080] The top-level potential field reflects the characteristics of group movement. If the velocity direction deviation of multiple pedestrian targets is less than 30 degrees and the velocity difference is less than one-quarter of the normal walking speed of an adult, the overall movement trend is established based on the average velocity of the group. If the rate of change of distance between group targets exceeds 20% of the current spacing, the dispersion or convergence trend of the group is predicted, and the potential field range is expanded or compressed accordingly. If a new target is less than twice the shoulder width from the group and its velocity direction is similar, it is included in the group potential field range, and the smooth transition of the potential field is completed within three control cycles.

[0081] In the calculation of potential field strength, if the robot speed exceeds 80% of the cruising speed, the potential field sampling range is extended along the direction of motion to the distance of three control cycles at the current speed; if there are maneuver-restricted areas such as downhill or turning in the environment, the potential field strength is enhanced in advance within the range of braking distance not less than the robot's maximum deceleration; if the predicted time of possible intersection is less than five control cycles, the potential field strength is increased by 20% in each cycle until it reaches the maximum value.

[0082] The dynamic update of the spatiotemporal potential field employs a multi-scale strategy. On a timescale of one second, the semantic information of the environment and the characteristics of group behavior are updated; on a timescale of 0.5 seconds, the overall distribution structure of the potential field is adjusted; and on a timescale of one control cycle, local updates are performed based on the real-time state of obstacles. If a change in scene semantic labels or a shift in group behavior patterns is detected, the weights of each potential field layer are redistributed within two control cycles.

[0083] S3: Calculate the safety passage around the robot based on the spatiotemporal potential field, and introduce an adaptive weighting factor to dynamically adjust the safety passage.

[0084] Based on the constructed three-layer spatiotemporal potential field structure, the safe passage around the robot is calculated, and the passage characteristics are dynamically adjusted through adaptive weights.

[0085] Specifically, firstly, in the robot's local coordinate system, a reference sampling sector is set along the current direction of motion, with a sampling radius no less than three times the distance required to brake from the current speed to a stop. In each sampling direction, the comprehensive potential field value is calculated as the basis for the channel feasibility assessment.

[0086] The underlying potential field reflects the basic motion prediction of various obstacles. If a human target is present, a Gaussian distribution is projected onto the sampling direction centered on its predicted position. The projection intensity is adjusted by the step size and gait coefficients. Related to the predicted location of mobile equipment, the projection value is calculated based on its predicted position. Wheeled equipment forms a stronger projection within the minimum turning radius, while omnidirectional platforms exhibit a uniformly decaying projection distribution. If a group of targets exists, the projection is calculated with the predicted position of the group as the center, and the projection intensity is affected by the average speed and structural characteristics of the group. When multiple obstacles exist simultaneously, the maximum value of their respective projections is taken as the component value of the underlying potential field in that direction.

[0087] The middle-layer potential field reflects the influence of environmental constraints. The projection value of the lower layer is adjusted according to environmental factors: if the sampling direction passes through the lobby area, the projection value is expanded along the width of the doorway to ensure a similar potential field intensity in the doorway area; if the sampling direction is in the corridor space, the projection distribution is compressed based on the predicted motion characteristics of obstacles. For human targets, the predicted motion direction is based on gait; for motorized equipment, its kinematic constraints are considered; and for group targets, their overall motion trend is taken into account. When the angle between the sampling direction and the wall is less than a preset threshold, the projection value in that direction is increased to reflect environmental boundary constraints.

[0088] The top-level potential field represents the characteristics of group movement, with a focus on the influence of group characteristics: if the sampling direction passes through the group's activity area and the local density exceeds two people per square meter, then this area is considered a continuous potential field and the projection value is calculated, with the projection intensity modulated by the influence coefficient; if the group exhibits a leader-follower structure, the projection value in the leader's predicted movement direction is larger; if it exhibits a parallel structure, a larger projection range is maintained laterally. When the group structure changes, the changes in projection values ​​are smoothly transitioned through a time decay term.

[0089] Finally, the combined potential field value in the sampling direction is obtained by weighted summation of the three potential field components. The sum of the potential field weights of each layer is always kept at 1, and the baseline values ​​are set as follows: bottom layer weight 0.4, middle layer weight 0.35, top layer weight 0.25, and dynamically adjusted according to the current state.

[0090] For the bottom-level weights, if a drastic change in obstacle movement is detected, the bottom-level weights are increased cycle by cycle. For human targets, if the gait adjustment coefficient... If the change exceeds 30%, the underlying weight is increased to 1.5 times the baseline value; if sudden acceleration or deceleration causing gait disorder is detected, it is further increased to 2 times. For maneuvering equipment, when the steering angular velocity exceeds 60% of the maximum steering angular velocity, or the acceleration exceeds 70% of the maximum acceleration, the underlying weight is increased to 1.8 times the baseline value. For group targets, if the rate of change in the spacing between group members exceeds 20% of the current spacing, the underlying weight is increased to 1.6 times the baseline value. All increased weights linearly decay back to the baseline value within 3 control cycles.

[0091] For mid-level weights, if the robot is in a special environmental area, the mid-level weights are adjusted according to the area's characteristics. In a lobby, if the robot's deviation from the center line of the doorway exceeds 1 / 4 of the doorway width, the mid-level weight is increased to 1.7 times the baseline value; in a corridor, if the distance to the wall is less than 1.5 times the width of a wheelchair, the weight is increased to 1.6 times the baseline value; in areas with limited mobility, such as downhill slopes or turns, the mid-level weight is increased to 2 times the baseline value. All weight increases are restored to the baseline value through exponential decay over 5 control cycles after leaving the special area.

[0092] For the top-level weights, if there is dense group activity nearby, the top-level weights are adjusted based on group characteristics. If the local group density exceeds 2 people per square meter, the top-level weight is increased to 1.4 times the baseline value; if the density continues to increase, the weight is increased by an additional 0.2 times for every 0.5 people / square meter increase, up to a maximum of 2.5 times the baseline value. When the group exhibits a leader-follower structure, the top-level weight is increased to 1.8 times the baseline value in the sampling sector along the leader's predicted movement direction; when exhibiting a parallel structure, it is increased to 1.5 times the baseline value in the lateral sampling direction. If a change in group structure is detected, such as a new member joining or an existing member leaving, the weight changes are smoothly adjusted within 2 seconds using the influence coefficient.

[0093] When the weight of one layer is increased, the weights of other layers are decreased accordingly to ensure that the sum of the weights of the three layers remains 1. Weight adjustments are calculated in real-time: if the weight of a layer is increased to k times its original value, the remaining weight values ​​are distributed to the other two layers proportionally. For example, when the weight of the bottom layer increases from 0.4 to 0.6, the remaining 0.4 weight is distributed between the middle and top layers in a ratio of 0.35:0.25. This yields the comprehensive potential field value in that sampling direction.

[0094] After obtaining the comprehensive potential field value, an adaptive weighting factor is introduced for a second-level adjustment. This weighting factor consists of three components: First, a dynamic weight related to the obstacle's motion state. When the obstacle's motion is unstable (such as human acceleration / deceleration, turning of motorized equipment, or changes in group structure), the weight is increased accordingly to reflect higher safety margin requirements. Second, a constraint weight related to the robot's maneuverability. When the speed is high or the load is large, the weight is increased to reflect the impact of limited maneuverability. Finally, a scene weight related to environmental characteristics. In passageways such as lobbies or densely populated areas, the weight is appropriately adjusted to balance passage efficiency and safety.

[0095] Specifically, when calculating the obstacle state influence factor, the basic influence values ​​of various obstacles along the sampling direction are first obtained. When a human target is present, the influence value is calculated based on gait characteristics: if the gait adjustment coefficient... If the change exceeds 30%, set the influence value within the range of 1.4-1.8 based on the magnitude of the change; the larger the change, the higher the influence value. If the torso's forward tilt angle exceeds 15 degrees, set the influence value within the range of 1.6-2.0 based on the tilt angle; the larger the tilt angle, the higher the influence value. If it is in the transition phase of the support phase, set the influence value to 1.4. The maximum value is taken when multiple conditions are met simultaneously. When there is mobile equipment, calculate the influence value based on motion characteristics: if the turning angular velocity exceeds 60% of the maximum value, set the influence value within the range of 1.5-1.9 based on the excess ratio; if the acceleration exceeds 70% of the maximum value, set the influence value within the range of 1.7-2.1 based on the excess ratio; if the load changes, set the influence value within the range of 1.3-1.7 based on the magnitude of the change. When there is a group of targets, calculate the influence value based on group characteristics: if the density exceeds 2 people per square meter, increase the influence value by 0.2 within the range of 1.4-2.0 for every additional 0.5 people / square meter; if the structure changes, set the influence value within the range of 1.3-1.7 based on the degree of change. Finally, the maximum value of all obstacle influence values ​​is taken as the obstacle state influence factor for that direction.

[0096] When calculating the robot state influence factor, the basic influence value is first calculated based on the robot's current motion state. When the speed exceeds 80% of the cruising speed, the influence value in the forward sector is set within the range of 1.4-1.8 according to the excess ratio, and the influence value in adjacent sectors decreases by 0.2. When the turning angular velocity exceeds 70% of the maximum value, the influence value in the outer turning sector is set within the range of 1.6-2.0 according to the excess ratio. When the load rate exceeds 60%, the basic influence value is set to 1.5, and it increases by 0.1 for every 10% increase in load rate, with a maximum not exceeding 2.0. Next, the robot's dynamic performance is considered: the minimum braking distance is calculated based on the current speed, and the influence value is increased by 0.2-0.4 according to the distance ratio within this range; the minimum turning radius is calculated based on the current turning angular velocity, and the influence value is increased by 0.1-0.3 according to the radius ratio within this range. Finally, the basic influence value and the dynamic influence value are added together, and the result is limited to within 2.5 to obtain the robot state influence factor.

[0097] When calculating the environmental characteristic impact factor, the environmental characteristics along the sampling direction are first identified. When the direction passes through the lobby area, the impact value is calculated based on the angle with the center line of the doorway: 1.0 for angles less than 15 degrees, 1.3 for angles between 15 and 30 degrees, and increasing from 1.3 to 1.8 for angles greater than 30 degrees. When the direction is in a corridor, the impact value is calculated based on the angle with the main direction of the corridor: increasing by 0.15 for every 10-degree increase, up to a maximum of 2.0. When the direction points towards a densely populated area, the impact value is calculated based on density: 1.4 for more than 2 people per square meter, increasing by 0.2 for every additional 0.5 people / square meter, up to a maximum of 2.0. When approaching walls or fixed obstacles, the impact value is calculated based on distance: 2.0 for distances less than the safety threshold, decreasing by 0.2 for every 0.5 times the safety threshold distance for distances greater than the threshold, until it reaches the baseline value of 1.0. Finally, the maximum value of each environmental characteristic impact value is taken as the environmental characteristic impact factor.

[0098] When calculating the final adaptive weighting factor, a weighted combination method is used to integrate the three influencing factors. The weight range for the obstacle state influence factor is 0.35-0.50, the weight range for the robot state influence factor is 0.30-0.45, and the weight range for the environmental feature influence factor is 0.20-0.35. The sum of the weights of the three is always 1. When any influence factor changes by more than 30%, a smooth transition is performed within 3 control cycles. The difference in adaptive weighting factors between adjacent sampling directions must not exceed 0.5; if it does, interpolation smoothing is performed. The final weighting factor value is limited to the range of 1.0-3.0 to avoid excessive suppression or amplification.

[0099] Ultimately, the feasibility assessment value of the safe passage is calculated through a two-level weight adjustment mechanism: first, the comprehensive potential field value is obtained by weighting the bottom layer weight, the middle layer weight, and the top layer weight; then, the overall value is adjusted by an adaptive weight factor. The calculation formula is: Assessment value = Adaptive weight factor × (Bottom layer weight × Bottom layer potential field + Middle layer weight × Middle layer potential field + Top layer weight × Top layer potential field).

[0100] A higher assessment value indicates that the direction is significantly affected by obstacles or environmental constraints, making it unsuitable as a movement path. The direction with the lower assessment value is selected as the feasible safe path, and the assessment results are continuously updated during the movement.

[0101] If the assessed value suddenly increases by more than 30% compared to the previous period, the direction of movement should be adjusted in a timely manner to maintain safety.

[0102] Ensure that the safety passage can flexibly adapt to changes in the movement state of obstacles, constraints on the robot's own state, and the influence of environmental characteristics, so as to maintain good movement efficiency while ensuring safety.

[0103] S4: Generate an obstacle avoidance trajectory for the robot within the safe passage, and optimize the robot's speed and turning angle using the gradient descent method so that the robot moves along the obstacle avoidance trajectory.

[0104] The search space for trajectory generation is determined based on the distribution of the current evaluation values: trajectory search is performed within the sectors formed by sampling directions where the evaluation values ​​are less than a threshold. Gradient descent is used to jointly optimize the robot's velocity and steering angle, where the baseline values ​​for velocity and steering angle are the current velocity and heading angle, respectively, and the initial threshold is 60% of the maximum evaluation value. Optimization objectives include: minimizing the evaluation value, speed smoothing, and steering continuity.

[0105] The evaluation value minimization term considers the weighted sum of evaluation values ​​at all points on the trajectory, with a sampling interval of 0.1 meters. The weighting coefficients employ an exponential decay function. The evaluation value at the starting point is assigned a weight of 1.0, and the weight at the ending point decays to 0.3. When the evaluation values ​​of adjacent sampling points differ by more than 30%, a penalty coefficient of 1.5 is applied to that segment of the trajectory to avoid crossing high-risk areas.

[0106] The speed smoothing term is expressed as the sum of squares of the rate of change of speed between adjacent sampling points. Speed ​​variation constraints are dynamically adjusted based on the robot's state: when the load factor exceeds 60%, the maximum acceleration is limited to 70% of the rated acceleration; when the obstacle state influence factor is greater than 1.5, the maximum deceleration is allowed to increase to 90% of the rated value; and when the robot is in the lobby area, as determined by environmental characteristic influence factors, the speed limit is reduced to 75% of the cruising speed. The coupling between speed and the evaluation value uses a piecewise linear mapping: when the evaluation value is in the range [0, 0.5], the speed limit linearly decreases to 85% of the rated speed; when the evaluation value is in the range [0.5, 1.0], the speed limit further linearly decreases to 65% of the rated speed.

[0107] The steering continuity term is expressed as the sum of squares of the rate of change of steering angle between adjacent sampling points. Steering constraints are linearly related to the current speed: for every 0.5 times the cruise speed increase, the maximum steering angular velocity decreases by 10%. When the speed exceeds 80% of the cruise speed, the maximum steering angular velocity is limited to 65% of the rated value; when the adaptive weighting factor is greater than 1.8, indicating approach to an obstacle, the maximum steering angular velocity is allowed to increase by 30%. In the corridor area, the maximum steering angle does not exceed 1.2 times the angle with the main direction of the corridor; in the lobby area, for every 15-degree increase in the angle with the centerline of the doorway, the allowed maximum steering angular velocity decreases by 15%.

[0108] The optimization process employs a three-level scale strategy: 10 candidate trajectories are searched at a 1-meter scale, and the 3 with the smallest evaluation values ​​are selected; these 3 trajectories are optimized at a 0.5-meter scale, with the number of sampling points doubled; and the optimal trajectory is locally smoothed at a 0.1-meter scale, with a smoothing window of 5 sampling points.

[0109] The optimization results must meet the following requirements: the evaluated value on the trajectory is always less than 85% of the maximum evaluated value; the speed change meets the dynamic constraints under each working condition; and the rate of change of the steering angle between adjacent sampling points is less than 95% of the maximum allowable value.

[0110] When the robot moves along the optimized obstacle avoidance trajectory, an evaluation value on the trajectory is calculated every 0.1 seconds. If a certain evaluation value suddenly increases by more than 30% of its original value, the trajectory is re-optimized in the next cycle. If a trajectory that meets the constraints cannot be found for three consecutive cycles, emergency obstacle avoidance is triggered: the speed is reduced to 50% of its current value, while the maximum steering angular velocity is allowed to increase to 1.5 times the original constraint. After the evaluation value returns to normal, the speed and steering constraints are linearly restored to their standard values ​​over 5 control cycles, ensuring that the robot can safely and smoothly complete the obstacle avoidance task.

[0111] In summary, the multi-source sensor-based automatic obstacle avoidance system and method for robots based on embodiments of the present invention have been elucidated. It achieves accurate environmental perception by constructing an adaptive multi-layer potential field model and employs a multi-scale optimization strategy to jointly optimize robot speed and steering angle, ensuring smooth and continuous trajectory while guaranteeing obstacle avoidance safety. In particular, when handling human-robot hybrid scenarios, it can deeply understand human gait characteristics and group interaction patterns. Through dynamic evaluation and real-time optimization mechanisms, the robot can complete obstacle avoidance tasks more naturally and smoothly, significantly improving obstacle avoidance performance and service efficiency in dynamic and complex indoor environments.

Claims

1. A robot automatic obstacle avoidance method based on multi-source sensors, characterized in that, include: The robot collects environmental data using multi-source sensors installed on it, obtains the position coordinates, movement direction, and velocity information of obstacles, and converts the environmental data into the robot's local coordinate system. Based on the environmental data, an improved dynamic window method is used to construct the spatiotemporal potential field between the robot and the obstacles; The safety passage around the robot is calculated based on the spatiotemporal potential field, and an adaptive weighting factor is introduced to dynamically adjust the safety passage. An obstacle avoidance trajectory for the robot is generated within the safety passage. The robot's speed and turning angle are optimized using the gradient descent method, so that the robot moves along the obstacle avoidance trajectory. In the local coordinate system, a spatiotemporal potential field is constructed for each detected obstacle, including: Obstacles are classified based on environmental semantic information: if the obstacle is a human target, feature point sequences are obtained through human key point detection, and its movement trend is predicted by combining gait features and movement patterns. If the obstacle is a mobile device, the corresponding kinematic model is matched according to the device type identification result, and its motion trajectory is predicted by combining angular velocity, acceleration and velocity constraints. If multiple pedestrian targets are detected moving together, their group behavior characteristics are extracted. Analyze the spatial relationships and motion characteristics between targets, construct a group motion model, and dynamically adjust the influence weights of the group structure; The spatiotemporal potential field uses a three-layer structure to represent the influence of obstacles: The underlying potential field adopts a Gaussian distribution function based on the feature size of the obstacle; The distribution pattern of the intermediate potential field is adjusted according to scene elements; The top-level potential field establishes an envelope surface based on the population density distribution.

2. The automatic obstacle avoidance method for robots based on multi-source sensors according to claim 1, characterized in that, If the obstacle is a human target, feature point sequences of the torso center position, hip joint position, and foot position are obtained through human key point detection, and prediction model parameters are constructed, expressed by the formula: ， in, This indicates the predicted coordinates of the next position. Indicates the current position coordinates. Indicates step size, This represents the gait adjustment factor, reflecting the effects of acceleration and deceleration. Indicates the angle of the torso. This represents the support phase coefficient, with a value range of [0,1], and represents the normalized progress of the gait period. Indicates the direction angle of the supporting leg. Indicates the vertical reference angle. Indicates the characteristic angle, Indicates the current walking speed. This indicates normal walking speed.

3. The automatic obstacle avoidance method for robots based on multi-source sensors according to claim 1, characterized in that, If the obstacle is a motorized device, a device type parameter is introduced to classify the device type into wheeled devices and omnidirectional devices, and the constraint models of angular velocity, acceleration and speed are adjusted accordingly.

4. The automatic obstacle avoidance method for robots based on multi-source sensors according to claim 1, characterized in that, If the obstacle is a group of pedestrians moving together, an influence coefficient for inter-group interaction is introduced to adjust the influence weight when a new member joins or leaves the group.

5. The automatic obstacle avoidance method for robots based on multi-source sensors according to claim 4, characterized in that, Based on the constructed three-layer spatiotemporal potential field structure, the safe passage around the robot is calculated, and the passage characteristics are dynamically adjusted by introducing an adaptive weighting factor. The adaptive weighting factors include: obstacle state influence factor, robot state influence factor, and environmental feature influence factor.

6. The automatic obstacle avoidance method for robots based on multi-source sensors according to claim 5, characterized in that, The feasibility assessment value of the safe passage is calculated through a two-level weight adjustment mechanism; First, the comprehensive potential field value is obtained by weighting the bottom layer weight, the middle layer weight, and the top layer weight. Then, the overall feasibility assessment value is obtained by adjusting the weight factor.

7. The automatic obstacle avoidance method for robots based on multi-source sensors according to claim 6, characterized in that, Based on the distribution of the feasibility assessment values, the search space for trajectory generation is determined, and the gradient descent method is used to jointly optimize the robot's speed and steering angle, where the baseline values ​​for speed and steering angle are the current speed and heading angle, respectively.

8. A robot automatic obstacle avoidance system based on multi-source sensors, wherein the robot obstacle avoidance method based on multi-source sensors according to any one of claims 1-7 is used to optimize robot obstacle avoidance, characterized in that, include: The data acquisition module is used to collect environmental data based on the multi-source sensors installed on the robot, obtain information about obstacles, and convert the environmental data into the robot's local coordinate system. The construction module, based on the environmental data, uses an improved dynamic window method to construct the spatiotemporal potential field between the robot and obstacles; The adjustment module calculates the safety passage around the robot based on the spatiotemporal potential field and introduces an adaptive weighting factor to dynamically adjust the safety passage; The optimization module generates an obstacle avoidance trajectory for the robot within the safety passage and uses the gradient descent method to optimize the robot's speed and turning angle, enabling the robot to move along the obstacle avoidance trajectory.

Images