A mobile robot dynamic obstacle avoidance planning system and method based on visual navigation

Abstract

The application belongs to the technical field of robot control, and discloses a mobile robot dynamic obstacle avoidance planning system and method based on visual navigation, real-time motion parameters such as obstacle position, speed and acceleration are extracted based on pose semantic perception results, which are input into a social LSTM network together with obstacle semantic types and robot motion parameters, interaction behaviors such as avoidance and following among multiple obstacles are captured through a social pooling layer, the influence of robot motion on obstacle trajectory is quantified through an interaction modeling layer, and a trajectory probability distribution and a prediction confidence of 3-5 time steps are output, while a collision probability, a minimum collision time and a minimum control amount required for collision avoidance are accurately calculated; through a lightweight Transformer double-branch shared network, a pose solution and obstacle semantic segmentation share feature extraction backbones, and robot motion constraints are integrated to realize multi-task parallel output of single-frame data, and perception link delay is shortened.

Description

Technical Field

[0001] This invention belongs to the field of robot control technology, specifically a dynamic obstacle avoidance planning system and method for mobile robots based on vision navigation. Background Technology

[0002] Vision-based navigation solutions, with their advantages of low sensor cost, rich environmental information perception dimensions, and no need for pre-deployment of auxiliary positioning facilities, have become the mainstream research and development direction for autonomous navigation technology in mobile robots. Existing vision-based dynamic obstacle avoidance technologies for mobile robots mostly adopt a sequential architecture of visual SLAM mapping, global path planning, dynamic obstacle perception, and local obstacle avoidance fine-tuning. However, in real-world complex dynamic scenarios, the following technical problems still exist: Existing methods mostly use simplified motion models with constant velocity / acceleration, without considering the social interaction behavior and nonlinear motion characteristics between dynamic obstacles. In scenarios with dense pedestrians and moving vehicles, the predicted trajectory deviates greatly from the actual trajectory, which can easily lead to collision risks and cannot meet the core requirements for the safety of mobile robot operation.

[0003] Existing technologies separate visual pose calculation, obstacle semantic perception, path planning, and motion control into independent serial modules. The data processing link for a single frame is long, and in dynamic scenes, the planned path is prone to mismatch with the real-time environment, resulting in obstacle avoidance failure.

[0004] Existing obstacle avoidance methods only aim at obstacle avoidance and path tracking, without considering the visibility requirements of environmental features for visual navigation. When robots avoid obstacles, they are very likely to deviate from the effective observation area of ​​visual features, resulting in pose calculation failure and navigation loss of control.

[0005] Existing solutions mostly adopt an architecture of fixed global path and local obstacle avoidance fine-tuning. When dynamic obstacles cover the global path, local planning can only be adjusted within a small range, which can easily lead to local dead zones. Furthermore, frequent obstacle avoidance can cause the actual path to deviate too much from the global optimal path, significantly reducing navigation efficiency. Summary of the Invention

[0006] The purpose of this invention is to provide a dynamic obstacle avoidance planning system and method for mobile robots based on vision navigation, so as to solve one or more problems mentioned in the background art.

[0007] To achieve the above objectives, the present invention provides the following technical solution: a dynamic obstacle avoidance planning method for mobile robots based on vision navigation, comprising the following specific steps: Preferably, in the map anchor point initialization stage, a continuous image sequence of the target scene is acquired through visual SLAM technology to construct a topology and grid fusion map with semantic information. The grid map is used to describe the obstacle occupancy status, and the topology map is used for the node-based planning of the global path. In the scene, ORB feature points and fixed semantic features that meet the preset threshold of robustness are selected as global visual feature anchors. The anchors are bound to the topology map nodes one by one to generate the visual field constraint range of each anchor. At the same time, combined with the kinematic parameters of the mobile robot, the corresponding motion control adaptation parameters are associated with each anchor. The failure determination rule for the global visual feature anchor point is as follows: if the anchor point is not detected for 10 consecutive control cycles, the anchor point is completely occluded by an obstacle, or the semantic features or ORB feature points of the anchor point undergo irreversible changes, the anchor point is determined to be in failure. Irreversible changes refer to the ORB feature point corner response value dropping below the threshold (<200) or the pixel contour / global position of the semantic features undergoing permanent changes, such as equipment removal or wall renovation. The dynamic update rule for anchor points is as follows: during robot navigation, new feature points in the scene are detected in real time. If the new feature point meets the robustness screening criteria, it is automatically added as a temporary anchor point. If the detection is stable for 30 consecutive control cycles, it is upgraded to a global anchor point. Invalid global anchor points are automatically removed from the topology map nodes, and the visible field of view and motion control adaptation parameters of the anchor point are updated.

[0008] Based on the navigation start and end points, an initial globally optimal path is generated in the topology map, and the navigation system initialization and motion control parameter pre-configuration are completed simultaneously.

[0009] Preferably, in the pose semantic perception stage, the robot uses a binocular vision camera to collect environmental image sequences in real time, and inputs adjacent frame images synchronously into a lightweight Transformer dual-branch shared network. The first branch is the pose calculation branch, which matches real-time image features with global visual feature anchor points to calculate the robot's real-time pose and heading angle in the global map, and outputs the pose deviation rate synchronously. The second branch is the dynamic semantic segmentation branch, which outputs pixel-level semantic masks of obstacles in the environment synchronously, distinguishes between static obstacles, dynamic obstacles and semi-dynamic obstacles, and outputs the outline, global position information and preliminary motion state judgment results of each obstacle. The output precision of the pixel-level semantic mask is 1:4 of the pixel ratio of the input image. The pixel labeling accuracy of the obstacle region of the mask is not less than 95%, and the region where no obstacle is detected is labeled as the background. The category labeling of the mask adopts a single-layer labeling rule, and a dedicated label is configured for static, dynamic and semi-dynamic obstacles. Dynamic obstacles are further subdivided into sub-labels such as pedestrians and moving vehicles. Semi-dynamic obstacles are labeled with their specific semantic types. When the mask is output, the pixel contour coordinates of each obstacle and the global position mapping relationship are simultaneously attached.

[0010] The dual-branch shared feature extraction backbone network incorporates robot motion state constraints during feature extraction, enabling parallel output of pose calculation, semantic perception, and motion control pre-adaptation for a single frame image. Motion control pre-adaptation refers to matching the robot's basic motion control parameters in advance based on the pose and semantic perception results.

[0011] Preferably, the trajectory probability prediction stage extracts the real-time motion state of each dynamic obstacle based on the dynamic obstacle semantic information, continuous frame position changes, and preliminary motion state judgment results output by the pose semantic perception stage. The motion state and semantic type of all dynamic obstacles are combined with the robot's real-time motion parameters and input into the social LSTM network. The social pooling layer captures the interaction behavior between multiple obstacles. The robot-obstacle interaction modeling layer quantifies the impact of robot motion on the trajectory of dynamic obstacles, outputs the probability distribution of future motion trajectories of each dynamic obstacle, and simultaneously outputs the trajectory prediction confidence. The trajectory prediction confidence level is divided into three levels: 0.7 and above is high confidence, 0.4-0.7 is medium confidence, and below 0.4 is low confidence. At high confidence, the predicted trajectory is used directly to calculate the collision parameters. At medium confidence, the obstacle safety domain is expanded by 10% before calculating the collision parameters, and the original collision probability judgment threshold is 70%. At low confidence, the trajectory prediction time step is shortened and the collision probability judgment threshold is reduced to 50%, while the tracking sampling frequency of the obstacle is increased to ensure obstacle avoidance safety at low confidence.

[0012] Simultaneously, based on trajectory probability distribution, prediction confidence, and robot kinematic constraints, the collision probability, minimum collision time, and minimum motion control amount required to avoid collisions between each obstacle and the robot's global path are calculated.

[0013] Preferably, the global path replanning stage is based on the collision probability, minimum collision time, and minimum motion control input output by the trajectory probability prediction stage. When it is determined that a high probability collision risk covers the core section of the current global path, global path replanning is triggered. The replanning has the dual optimization objectives of the shortest path length and the lowest motion control energy consumption, while setting insurmountable hard constraints. The replanned global path must pass through a preset proportion of the global visual feature anchor point's visible range. The curvature and slope of the replanned path must match the robot's kinematic parameters, and it must avoid high-probability movement areas of dynamic obstacles and reserve redundancy for motion control adjustment. After the replanning is completed, the global path, the corresponding anchor point constraints, and the motion control adaptation parameters are updated synchronously.

[0014] Preferably, the obstacle avoidance safety domain construction stage is based on the obstacle semantic type output by the pose semantic perception stage, the motion state, collision probability, minimum collision time, and minimum motion control amount output by the trajectory probability prediction stage, to construct an adaptive obstacle avoidance safety domain decision model driven by both obstacle characteristics and robot motion state; combining the motion characteristics and collision risk of dynamic obstacles, as well as the robot's own real-time motion parameters, to adaptively construct differentiated obstacle avoidance safety domains for dynamic obstacles, static obstacles, and semi-dynamic obstacles in different motion states and associate them with corresponding motion control strategies; For dynamic obstacles that move at high speeds and have a high risk of collision, an elliptical dynamic safety domain is set, with the major axis along the direction of obstacle movement. The safety radius is linearly and collaboratively adjusted with the obstacle speed and the robot's own speed, and deceleration and emergency steering control strategies are matched synchronously. For dynamic and semi-dynamic obstacles that move at low speeds or have a low risk of collision, a circular fixed safety domain is set with a standard safety radius, and uniform speed detour and small-amplitude steering control strategies are matched. For static obstacles, only a minimum collision distance constraint is set without an additional expansion radius, and the passing speed is dynamically adjusted in combination with the robot's current speed.

[0015] Preferably, the local obstacle avoidance planning stage uses the updated global path from the global path replanning stage as the tracking benchmark, and the obstacle avoidance safety domain constructed in the obstacle avoidance safety domain construction stage and the associated motion control strategy as hard constraints to construct a model predictive control optimization objective function that integrates motion control performance. The optimization objective function includes four core weight terms: minimum global path tracking error, robot motion smoothness, visual feature anchor visibility guarantee, and minimum motion control energy consumption. At the same time, insurmountable hard constraints are set: the robot's planned path must not enter the obstacle avoidance safety domain of each obstacle and must meet the robot's kinematic constraints. At each moment on the planned path, the robot camera must observe at least a preset number of global visual feature anchors, and the motion control commands corresponding to the planned path must be continuous and stable. By using rolling time-domain optimization, the optimal local obstacle avoidance path and corresponding velocity and angular velocity control quantities for multiple future control cycles are output, and the adjustment gradient of the control quantities is output simultaneously.

[0016] The adjustment gradient of the control quantity is used to guide the smooth adjustment of velocity and angular velocity in adjacent control cycles. The application rule is to adjust gradually according to the deviation between the current control quantity and the planned value, and it is not allowed to jump directly to the target value across cycles. The gradient ratio refers to the ratio of the actual adjusted value of the control quantity to the planned difference, and the value range is 0.2-0.5. It is dynamically adjusted according to the collision risk level, with 0.5 for high risk and 0.2 for low risk. The adjustment range is limited to a single velocity adjustment gradient of no more than 0.1 m / s and a single angular velocity adjustment gradient of no more than 2° / s. If an emergency adjustment is required due to a high collision risk, the maximum adjustment range shall not exceed the threshold of the robot's kinematic parameters, and the control command must still be continuous and stable after the adjustment.

[0017] Preferably, the closed-loop feedback correction stage sends the control quantity and control quantity adjustment gradient output from the local obstacle avoidance planning stage to the robot's motion actuator to complete the tracking and execution of the obstacle avoidance path; at the same time, it collects the robot's actual operating pose, visual positioning deviation, deviation between the actual trajectory and the predicted trajectory of the obstacle, and motion control execution error in real time, and executes differentiated correction strategies for different types of deviations. The motion actuator responds to control commands in a hard real-time manner, with a command reception delay of no more than 0.03 seconds. It executes speed and angular velocity adjustment commands according to the control cycle, maintaining the continuity of commands during execution and not interrupting the already executed stable control commands. For execution errors, a real-time compensation method is adopted. Based on the deviation between the actual execution value and the planned value fed back by the parameter acquisition unit, the gradient is adjusted according to the control quantity to compensate cycle by cycle. When the deviation is large, a step-by-step compensation strategy is adopted to avoid the robot's motion being unstable due to excessive single compensation amplitude.

[0018] When the visual positioning deviation exceeds a preset threshold, the weight of the visual feature visibility term in subsequent planning is dynamically increased, forcing the robot to return to the anchor point's visual field, while fine-tuning the motion control speed. When the deviation between the actual obstacle movement and the predicted trajectory exceeds a preset threshold, the time step of trajectory prediction is dynamically shortened, the update frequency of planning and prediction is increased, and the safety domain parameters and corresponding motion control strategies are adjusted synchronously. When the path tracking error or motion control execution error exceeds a preset threshold, the rolling time domain length and control quantity adjustment gradient of the model predictive control optimization are dynamically adjusted, and the speed and angular velocity control commands are optimized. When the motion control energy consumption exceeds a preset threshold, the weight allocation of the model predictive control optimization objective function is dynamically adjusted.

[0019] This invention also provides a visual navigation-based dynamic obstacle avoidance planning system for mobile robots, which, based on the above method, consists of hardware modules and software functional modules: The hardware module includes a binocular vision camera, a main control unit, a motion actuator, and a parameter acquisition unit. The binocular vision camera is used to acquire environmental image sequences in real time. The main control unit is used to run various software function modules, process image data, execute planning algorithms, and output control commands. The motion actuator receives control commands and completes the adjustment of the robot's speed and angular velocity. The parameter acquisition unit is used to acquire the robot's kinematic parameters and operating status data. The software functional modules include a map anchor point initialization module, a pose semantic perception module, a trajectory probability prediction module, a global path replanning module, an obstacle avoidance safety domain construction module, a local obstacle avoidance planning module, and a closed-loop feedback correction module. The map anchor point initialization module constructs a topology and grid fusion map using visual SLAM technology, selects global visual feature anchor points and associates them with motion control adaptation parameters to generate an initial global optimal path. The pose semantic perception module uses a lightweight Transformer dual-branch shared network to simultaneously complete the robot's global pose calculation, obstacle semantic segmentation, and initial motion state judgment. The trajectory probability prediction module, based on perception data, captures obstacle interaction behavior through a social LSTM network and outputs trajectory probability distribution and collision-related parameters. The global path replanning module triggers replanning and updates the global path and motion control adaptation parameters based on the collision prediction results. The obstacle avoidance safety domain construction module constructs an adaptive obstacle avoidance safety domain driven by both obstacle characteristics and robot motion state, and associates it with differentiated motion control strategies. The local obstacle avoidance planning module constructs a model predictive control optimization objective function and outputs the optimal local obstacle avoidance path and control quantity. The closed-loop feedback correction module collects various deviation data and performs differentiated corrections.

[0020] The beneficial effects of this invention are as follows: 1. This invention extracts real-time motion parameters such as obstacle position, velocity, and acceleration based on pose semantic perception results. These parameters, along with the obstacle semantic type and the robot's own motion parameters, are input into a social LSTM network. The social pooling layer captures interactive behaviors such as obstacle avoidance and following among multiple obstacles. Furthermore, the interaction modeling layer quantifies the impact of robot motion on obstacle trajectories, outputting the trajectory probability distribution and prediction confidence over 3-5 time steps. Simultaneously, it accurately calculates the collision probability, minimum collision time, and minimum control input required to avoid collisions. This reduces trajectory prediction bias in dense pedestrian and vehicle scenarios, thus avoiding collision risks.

[0021] 2. This invention utilizes a lightweight Transformer dual-branch shared network to enable pose calculation and obstacle semantic segmentation to share a feature extraction backbone. It incorporates robot motion constraints to achieve multi-task parallel output of single-frame data, thus shortening the perception link latency. At the same time, it abandons the old architecture of fixed global paths. When a high collision risk is detected covering the core road segment, global path replanning is triggered. Combined with local rolling temporal optimization, fine planning is performed, avoiding the problem of getting stuck in dead zones during local obstacle avoidance fine-tuning, and enabling the planned path to match the real-time environment.

[0022] 3. This invention binds visual anchor points and topological nodes and sets the visible field of view during the map anchor point initialization stage, then globally replans the path to pass through a preset proportion of the anchor point's visible field of view, and then incorporates anchor point visibility into the optimization objective function during local planning. In the closed-loop feedback correction stage, the visibility weight can be dynamically increased for positioning deviations. This multi-stage process ensures the robot's effective observation of visual features. At the same time, it constructs an adaptive obstacle avoidance safety domain driven by both obstacle characteristics and robot motion state. Differentiated safety domains and control strategies are matched for different types of obstacles. Combined with closed-loop differentiated correction and energy consumption optimization objectives, it avoids navigation loss of control caused by obstacle avoidance and detours, and can dynamically optimize motion control energy consumption while ensuring safety, thus improving the overall robustness and practicality of robot visual navigation and obstacle avoidance. Attached Figure Description

[0023] Figure 1 This is a flowchart of the visual navigation-based dynamic obstacle avoidance planning system for mobile robots according to the present invention. Figure 2 This is a flowchart of the accurate prediction process for dynamic obstacle trajectories in this invention; Figure 3 This is a flowchart of the global-local collaborative obstacle avoidance planning process of the present invention. Detailed Implementation

[0024] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0025] like Figures 1 to 2 As shown, this embodiment of the invention provides a dynamic obstacle avoidance planning method for mobile robots based on vision navigation, including the following specific steps: In one embodiment, the map anchor initialization phase acquires a continuous image sequence of the target scene using visual SLAM technology to construct a topology and grid fusion map with semantic information. The grid map is used to describe the obstacle occupancy status, and the topology map is used for the node-based planning of the global path. Visual SLAM employs a mapping method that combines feature point analysis and direct mapping. The grid map has a grid resolution of 0.1m x 0.1m. It uses binocular visual depth information to label the grid with three states: occupied, free, and unknown. It also adds obstacle semantic labels to occupied grids. The topology map uses corners, passageways, and anchor points in the scene as topology nodes, and the connecting paths between nodes as topology edges. The edge attributes include path length, direction of travel, and curvature information. The fused map is associated with the topology nodes through coordinate mapping with the grid map, ensuring coordinate consistency between global path planning and local obstacle avoidance.

[0026] In the scene, ORB feature points with robustness meeting the preset threshold are selected, along with fixed semantic features such as corners, device identifiers, and doorplates, as global visual feature anchors. The anchors are then bound to topology map nodes one by one to generate the visible field constraint range of each anchor. The robustness screening criteria for ORB feature points are: corner response value not less than 200, scale invariance error less than 5%, rotation invariance coverage of 0-360°, and feature points without occlusion or motion blur in the scene; fixed semantic features must meet the following requirements: pixel area not less than 50×50, clear edge features, and fixed position in the scene; the visible field of view of the anchor point is constrained to be within the field of view of the robot's binocular vision camera of 60°×45°, and the straight-line distance between the camera and the anchor point does not exceed 15 meters. If it exceeds this range, the anchor point is judged to be unobservable.

[0027] Simultaneously, by combining the kinematic parameters of the mobile robot, such as its turning radius, maximum speed, and acceleration threshold, corresponding motion control adaptation parameters are associated with each anchor point, including the optimal observation speed and the turning angle range. The motion control adaptation parameters also include the robot's minimum turning radius, acceleration limit range, and camera pitch angle adjustment range during anchor point observation. The configuration rules are determined based on the spatial location of the anchor points. For planar anchor points, the optimal observation speed in the horizontal heading is configured to be 0.5-1.5 m / s, and the turning angle range is -15° to 15°. For high-altitude anchor points, the camera pitch angle is configured to be 5° to 20°, and the robot's movement speed is reduced to 0.3-0.8 m / s. For anchor points in narrow areas, the minimum turning radius is reduced, and the maximum acceleration value is limited.

[0028] The specific scope of robot kinematic parameters also includes minimum velocity, maximum deceleration, angular velocity range, vehicle dimensions, wheelbase and track width, and motion response delay; the calibration standards for these parameters are that the turning radius calibration error should not exceed 0.05 meters, and the velocity and acceleration calibration errors should not exceed 0.05 m / s² and 0.05 m / s², respectively. 2 The angular velocity calibration error does not exceed 1° / s, the motion response delay is calibrated to within 0.05 seconds, and the robot automatically completes the self-calibration of kinematic parameters after each power-on start-up, ensuring the consistency between planning and execution.

[0029] Based on the navigation start and end points, an initial global optimal path is generated in the topology map, and the navigation system initialization and motion control parameter pre-configuration are completed simultaneously to ensure that the initial path matches the robot's motion capabilities.

[0030] The optimality of the initial global optimal path is determined by the core criteria of the shortest total path length, the most anchor points, and the smallest change in path curvature, with the weights of the three decreasing in that order. The path generation uses the A* algorithm, which searches for all nodes in the topology map, avoiding grid areas that are set as no-go zones. After generation, it is necessary to verify whether the curvature and slope of the path are within the allowable range of the robot's kinematic parameters. If the verification fails, the path is searched and generated again until the requirements of optimality and motion adaptability are met.

[0031] The initialization of the navigation system includes global map coordinate calibration, binocular vision camera intrinsic and extrinsic parameter calibration, parameter acquisition unit sampling frequency initialization, and communication link detection and activation of each software module. The motion control parameter pre-configuration includes the robot's initial motion speed, basic angular velocity, initial acceleration value, and basic weight configuration for visual anchor point observation. At the same time, the initial values ​​of the trigger thresholds of each module are pre-configured. After all initialization and pre-configuration are completed, the system performs a self-check. Only after the self-check passes can the robot navigation and obstacle avoidance process be started.

[0032] In one embodiment, the pose semantic perception stage acquires environmental image sequences in real time through a binocular vision camera mounted on the robot, and synchronously inputs adjacent frame images into a lightweight Transformer dual-branch shared network; the first branch is the pose calculation branch, which matches real-time image features with global visual feature anchor points, calculates the robot's real-time pose and heading angle in the global map, and synchronously outputs the pose deviation rate to provide real-time feedback for motion control. The pose deviation rate is the ratio of the robot's real-time calculated pose to the planned path pose. The reference standard for its judgment is that a pose deviation rate of no more than 5% is within the normal range, 5%-10% is a slight deviation, and more than 10% is a serious deviation. If the pose deviation rate is within the slight deviation range for two consecutive control cycles, a small adjustment of motion control is triggered. If it is within the serious deviation range for two consecutive control cycles, the local obstacle avoidance planning is immediately re-optimized, and the visibility weight of the visual anchor point is increased.

[0033] Formula for calculating core position pose deviation rate: In the formula: The deviation between the robot's actual position and the planned path position is the proportion of the planned path step length, which is the core calculation index of pose deviation. , The actual global position coordinates of the robot are calculated in real time, in meters, and are output by the binocular vision pose calculation branch. , The global position coordinates at the corresponding time on the robot's planned path are given by the global path planning module. The unit is meters. The single-step length of the planned path is in meters, and the straight-line distance between two adjacent path points in the planned path is matched with the robot control cycle.

[0034] Formula for calculating heading angle deviation rate: In the formula: This refers to the heading angle deviation rate; This is the robot's actual heading angle, in degrees, output from the pose calculation branch; The expected heading angle at the time corresponding to the robot's path planning, in degrees; This represents the robot's maximum allowable heading angle adjustment range, expressed in degrees (°), typically set to 30°, to match the steering angle constraint.

[0035] The binocular vision camera is mounted in a parallel binocular configuration with a baseline distance of 0.15 meters. The acquired images are output simultaneously as color RGB images and depth maps. The effective depth detection range is from 0.5 meters to 20 meters, and depth information outside this range is marked as invalid. The images captured by the camera are not distorted, the exposure mode is automatic, the frame rate is stable at 30fps, and the time interval between adjacent frame acquisitions is fixed to ensure the consistency of motion parameter extraction time.

[0036] The second branch is the dynamic semantic segmentation branch, which synchronously outputs pixel-level semantic masks of obstacles in the environment, distinguishes between static obstacles, dynamic obstacles (pedestrians, moving vehicles, etc.) and semi-dynamic obstacles (movable shelves, etc.), and outputs the outline, global position information and preliminary motion state judgment results of each obstacle. The preliminary motion state judgment results include uniform speed, variable speed, linear motion, curvilinear motion, etc. The dual-branch shared feature extraction backbone network incorporates robot motion state constraints during the feature extraction process. These constraints include current speed and turning angle, enabling parallel output of pose calculation, semantic perception, and motion control pre-adaptation for single-frame images, thus shortening the perception link latency.

[0037] The lightweight Transformer dual-branch shared network's feature extraction backbone is a lightweight ViT-Base architecture, comprising 12 shared self-attention layers and a multilayer perceptron. The input is a 640×480 pixel RGB-D neighboring frame image feature map acquired through binocular vision, with the feature dimension compressed to 256 dimensions. The pose calculation branch adds 3 convolutional layers and 1 fully connected layer, outputting a 6-dimensional pose vector (x, y, z, α, β, γ), a 1-dimensional heading angle, and a 1-dimensional pose deviation rate. The semantic segmentation branch adds 2 deconvolutional layers and a Softmax classification layer, outputting a 128×128 pixel semantic mask and a 10-dimensional obstacle state feature vector. Both branches are connected to a robot motion state constraint layer, whose input consists of two-dimensional parameters: the robot's real-time velocity and angular velocity, enabling parallel output of multi-task feature fusion.

[0038] The initial motion state judgment is based on obstacle position change data of 5 consecutive frames: if the deviation of displacement change between adjacent frames is less than 10%, it is judged as uniform motion, and if the deviation is greater than 10%, it is judged as variable motion; if the deviation of the motion trajectory fitting straight line in 5 consecutive frames is less than 8%, it is judged as linear motion, and if the deviation is greater than 8%, it is judged as curvilinear motion; if no obstacle position change is detected in 3 consecutive frames, it is temporarily judged as a static obstacle, and if there is still no change after 6 consecutive frames, it is confirmed as a static obstacle.

[0039] Semi-dynamic obstacles are defined as obstacles that have no active movement capability, can be displaced by external factors, and have a movement speed of less than 0.2 m / s and a low movement frequency. Specific scenario classifications include movable shelves, turnover boxes, temporary roadblocks, and push-type equipment. The determination of this type of obstacle requires combining semantic features and continuous frame position changes. If external force is detected to cause displacement, it is treated as a low-speed dynamic obstacle; if there is no displacement, it is treated as a static obstacle, and it is labeled with a unique semantic tag for safety domain adaptation. The low-speed dynamic obstacle refers to a dynamic obstacle with a movement speed ≤ 1 m / s; the high-speed dynamic obstacle refers to a dynamic obstacle with a movement speed > 1 m / s. When the displacement change of a semi-dynamic obstacle is > 0.05 m in 3 consecutive frames, it is determined to have generated displacement and is treated as a low-speed dynamic obstacle; otherwise, it is treated as a static obstacle.

[0040] In one embodiment, the trajectory probability prediction stage extracts the real-time motion state of each dynamic obstacle, such as its position, velocity, acceleration, and direction of motion, based on the dynamic obstacle semantic information, continuous frame position changes, and initial motion state judgment results output by the pose semantic perception stage. The motion state and semantic type of all dynamic obstacles, combined with the robot's real-time motion parameters such as current speed, turning angular velocity, and trajectory, are input into a social LSTM network. The social pooling layer captures the interactive behaviors between multiple obstacles, including pedestrian avoidance, following, and merging. The obstacle interaction neighborhood determination of the social pooling layer is a 5-meter circular area centered on a single obstacle. Other obstacles within this area are considered to have interaction relationships, while those beyond 5 meters are considered to have no interaction and do not participate in feature fusion. The feature weighting rule assigns weights based on the distance and movement direction between obstacles. The closer the distance, the higher the weight. Obstacles moving in the same direction have a higher weight than obstacles moving in opposite directions. Obstacles with no interaction have a weight of 0. The weighted feature fusion result is used to update the motion state parameters of obstacles and improve the accuracy of trajectory prediction.

[0041] The robot-obstacle interaction modeling layer quantifies the impact of robot motion on the trajectory of dynamic obstacles, such as the interference of robot deceleration and turning on the movement of pedestrians in the surrounding area. It outputs the probability distribution of the motion trajectory of each dynamic obstacle in the next 3-5 time steps and outputs the trajectory prediction confidence at the same time. The robot-obstacle interaction modeling layer adopts a feature-by-element fusion rule, which performs dimensional matching between robot motion features and obstacle motion features and then performs element-by-element weighted fusion. The fusion weight is dynamically adjusted according to the relative distance between the robot and the obstacle, with higher weights for closer distances. The quantification method for the impact on the obstacle trajectory is to correct the obstacle's motion speed and direction parameters based on the fused feature values. The corrected parameters serve as new inputs for trajectory prediction. At the same time, the degree of impact is divided into three levels: no impact, slight impact, and significant impact, providing a level reference for collision probability calculation.

[0042] Simultaneously, based on trajectory probability distribution, prediction confidence, and robot kinematic constraints, the collision probability, minimum collision time, and minimum motion control quantities required to avoid collisions are calculated between each obstacle and the robot's global path, including deceleration amplitude and steering angle.

[0043] The social LSTM network comprises an input layer, four stacked LSTM hidden layers, a social pooling layer, a robot-obstacle interaction modeling layer, and an output layer. The input layer consists of normalized 8-dimensional obstacle motion features and 6-dimensional robot motion features. The embedding layer maps the 8-dimensional obstacle features and 6-dimensional robot features to 128 dimensions respectively, and then performs feature fusion to ensure that the input dimensions are consistent. The obstacle motion features include position, velocity, acceleration, and direction of motion, while the robot motion features include pose, velocity, and angular velocity. The social pooling layer employs spatial attention pooling to weightedly fuse features of other obstacles within a 5m neighborhood centered on each obstacle, capturing interactive behaviors such as avoidance and following. The interaction modeling layer is a single fully connected layer that quantifies the trajectory influence weights through the dot product operation of robot motion features and obstacle features. The output layer is a Gaussian distribution fitting layer, where the trajectory probability distribution is represented by a 5-dimensional Gaussian feature vector. The 5-dimensional Gaussian feature vector includes the mean x, mean y, variance x, variance y, correlation coefficient, and a normalized value with a prediction confidence of 0-1. The collision-related parameters are quantized numerical parameters.

[0044] The trajectory prediction time step is dynamically selected based on the obstacle's speed: 3 time steps are taken when the obstacle speed is >1m / s, 4 time steps are taken when the speed is 0.5-1m / s, and 5 time steps are taken when the speed is <0.5m / s, with each time step being 0.5s; the collision probability is calculated by the proportion of the spatial intersection area between the trajectory probability distribution and the robot's global path, and the result is a normalized value of 0-1. The minimum collision time is calculated by the relative distance / relative velocity between the robot and the obstacle, in seconds; the minimum motion control quantity is calculated by a linear fitting model of the collision probability and the robot's kinematic constraints, and the output is the deceleration amplitude (0 to the robot's maximum deceleration, in m / s²). 2 ), and the quantified values ​​of the steering angle (-30°~30°).

[0045] In one embodiment, the global path replanning stage is based on the collision probability, minimum collision time, and minimum motion control amount output by the trajectory probability prediction stage. When it is determined that a high probability collision risk covers the core section of the current global path, global path replanning is triggered. The core section refers to the path segment from the robot's current position to 20 meters ahead, path turns, areas with dense visual anchor points, and narrow path segments. The threshold for determining high collision risk is that the probability of collision between an obstacle and the robot's global path is not less than 70% and the minimum collision time is not more than 3 seconds. Meeting either condition is considered a high collision risk. The core sections of the global path are defined as the path segment from the robot's current position to 20 meters ahead, path bends, areas with dense anchor points, and narrow path segments. When the above-mentioned sections are covered by high collision risk, global path replanning is immediately triggered.

[0046] The replanning aims to optimize both the shortest path length and the lowest motion control energy consumption. At the same time, it sets insurmountable hard constraints. The global path after replanning must pass through the visual field of a preset proportion of global visual feature anchor points to ensure that the camera can always observe the number of visual feature anchor points that meet the positioning requirements during the robot's entire navigation process. The pre-set proportion of the visible area of ​​the global visual feature anchor points that the replanned global path needs to pass through is no less than 80%, that is, the proportion of road segments within the visible area of ​​the anchor points in the total path length is no less than 80%; the reserved motion control adjustment redundancy space for high probability movement areas of dynamic obstacles is no less than 1.5 meters, that is, the shortest straight distance between the replanned path and the high probability movement area of ​​dynamic obstacles is no less than 1.5 meters, and the redundancy space for turning sections needs to be expanded to 2 meters. The curvature and slope of the replanned path need to match the robot's kinematic parameters to avoid path segments that exceed the robot's turning ability and acceleration range, thereby reducing the difficulty of motion control. It is also necessary to avoid high-probability movement areas of dynamic obstacles and reserve redundancy for motion control adjustment, such as turning buffer sections and deceleration transition sections. After the replanning is completed, the global path, corresponding anchor point constraints and motion control adaptation parameters are updated synchronously.

[0047] The parameters after global path replanning are updated using a master-slave synchronous update rule. The global path replanning module acts as the master module, synchronously sending the updated global path, anchor point constraints, and motion control adaptation parameters to all downstream modules, including obstacle avoidance safety domain construction, local obstacle avoidance planning, and closed-loop feedback correction. Upon receiving the parameters, each downstream module immediately stops its current planning calculation and restarts the corresponding process based on the new parameters. The parameter synchronization employs a data verification mechanism to ensure that the parameters received by each module are consistent. If the verification fails, the master module is triggered to resend the parameters, ensuring the consistency of the linkage between modules.

[0048] In one embodiment, the obstacle avoidance safety domain construction stage is based on the obstacle semantic type output by the pose semantic perception stage, the motion state, collision probability, minimum collision time, and minimum motion control quantity output by the trajectory probability prediction stage, to construct an adaptive obstacle avoidance safety domain decision model driven by both obstacle characteristics and robot motion state; combining the motion characteristics and collision risk of dynamic obstacles, as well as the robot's own real-time motion parameters such as speed and turning angular velocity, to adaptively construct differentiated obstacle avoidance safety domains for dynamic obstacles, static obstacles, and semi-dynamic obstacles in different motion states and associate them with corresponding motion control strategies. For dynamic obstacles that move at high speed and have a high risk of collision, an elliptical dynamic safety domain is set, with the major axis along the direction of obstacle movement and the safety radius linearly and collaboratively adjusted with the speed of the obstacle and the speed of the robot itself, and simultaneously matched with deceleration and emergency steering control strategies. The dynamic adjustment of obstacle avoidance safety domain parameters is triggered when the obstacle's speed changes by more than 0.3 m / s, the robot's own speed changes by more than 0.5 m / s, or the relative distance between the obstacle and the robot changes by more than 1 meter. Adjustment is triggered when any one of these conditions is met. The update frequency of the safety domain parameters is consistent with the visual acquisition frame rate, which is 30 times / second. The adjusted parameters are immediately synchronized to the local obstacle avoidance planning module to ensure the real-time performance and accuracy of the planning. For dynamic and semi-dynamic obstacles with low speed or low collision risk, a fixed circular safety zone is set with a standard safety radius, and a uniform speed detour and small-amplitude steering control strategy is matched. For static obstacles, only a minimum collision distance constraint is set without an additional expansion radius, and the passing speed is dynamically adjusted in combination with the robot's current speed.

[0049] Major axis of elliptical dynamic safety domain short axis The unit is m, where The real-time velocity of the obstacle is (m / s). The real-time speed of the robot (m / s), coefficient =0.8、 =0.5, the safety radius is linearly and collaboratively adjusted with the speed of both. The standard radius of the circular fixed safety zone is 0.8m; the minimum collision distance constraint for static obstacles is 0.5m, and the robot's speed is linearly adjusted according to the obstacle distance, using the following formula: , This represents the actual distance (in meters) between the robot and the static obstacle. This represents the robot's maximum speed (m / s).

[0050] In one embodiment, the local obstacle avoidance planning stage uses the updated global path from the global path replanning stage as the tracking benchmark, and the obstacle avoidance safety domain constructed in the obstacle avoidance safety domain construction stage and the associated motion control strategy as hard constraints to construct a model predictive control optimization objective function that integrates motion control performance. The optimization objective function includes four core weight terms: minimum global path tracking error, robot motion smoothness, visual feature anchor visibility guarantee, and minimum motion control energy consumption. At the same time, insurmountable hard constraints are set: the robot's planned path must not enter the obstacle avoidance safety domain of each obstacle, and must meet the robot's turning radius, maximum speed, acceleration threshold, and other kinematic constraints. At each moment on the planned path, the robot camera must observe at least a preset number of global visual feature anchors, and the motion control commands (velocity, angular velocity) corresponding to the planned path must be continuous and stable to avoid sudden changes in control quantities such as sudden increases in angular velocity or sudden decreases in speed, thereby preventing the robot from losing control. The robot camera needs to observe at least 3 global visual feature anchor points, of which at least 1 is a fixed semantic feature anchor point. The anchor points need to be distributed in different areas of the camera's field of view to avoid being concentrated in a single field of view. If the number of anchor points observed in 3 consecutive control cycles is lower than this value, it is determined that the visual positioning features are insufficient.

[0051] The criteria for determining the continuity and stability of the motion control commands are that the velocity change between adjacent control cycles does not exceed 0.2 m / s, the angular velocity change does not exceed 5° / s, and there are no abrupt changes in the control quantity within a single control cycle. If the planned control command exceeds this range, it is determined that the command is unstable, and the local obstacle avoidance planning is immediately re-optimized. The control quantity is then smoothed by interpolation and output to the motion actuator to ensure the continuous transition of the robot's motion state.

[0052] By using rolling time-domain optimization, the optimal local obstacle avoidance path and corresponding velocity and angular velocity control quantities for multiple future control cycles are output, and the adjustment gradient of the control quantities is output synchronously to avoid local planning getting stuck in dead zones and ensure the tracking consistency of the global path and the smoothness of motion control.

[0053] The mathematical expression for the objective function of model predictive control optimization is: In the formula: For global path tracking error terms, For the smoothness of motion term, For visual anchor visibility items, This is the energy consumption term for motion control; the weight term is initialized to a value of [value missing]. =0.4、 =0.2、 =0.25、 =0.15, and each weight can be dynamically adjusted within the range of 0.05-0.5 according to the robot's operating status; the rolling time domain length is set to 5-10 control cycles, with a single control cycle of 0.2s. The optimized output is the linear velocity, angular velocity, and two-dimensional control quantities and control quantity adjustment gradient for each cycle.

[0054] The rolling time-domain optimization uses gradient descent for iterative solution, with 20 iterations per cycle. Only the control input for the current control cycle is executed, and the control input for the remaining cycles is used as a pre-planning reference. The optimization update rule is that after each control cycle is completed, the data of the executed cycle is discarded and a new data for the future control cycle is added. If the obstacle trajectory deviation exceeds the threshold or the pose deviation rate is abnormal, the rolling time-domain optimization is immediately triggered to re-iterate, shortening the time-domain length of a single optimization and increasing the iteration frequency.

[0055] In one embodiment, the closed-loop feedback correction stage sends the control quantity and control quantity adjustment gradient output from the local obstacle avoidance planning stage to the robot's motion actuator to complete the tracking and execution of the obstacle avoidance path; at the same time, it collects the robot's actual operating posture, visual positioning deviation, deviation between the actual motion trajectory of the obstacle and the predicted trajectory, and motion control execution error (deviation between actual speed / angular velocity and the planned value) in real time, and executes differentiated correction strategies for different types of deviations. The preset thresholds for various deviations are as follows: for visual positioning deviation, the position error should not exceed 0.5 meters and the heading angle error should not exceed 5°. Exceeding these values ​​is considered a positioning deviation exceeding the threshold. For obstacle trajectory deviation, the position error should not exceed 0.3 meters and the movement direction error should not exceed 10°. Exceeding these values ​​is considered a trajectory deviation exceeding the threshold. For path tracking error, the position error should not exceed 0.4 meters. For motion control execution error, the speed deviation should not exceed 0.2 m / s and the angular velocity deviation should not exceed 3° / s. Exceeding these values ​​is considered an execution error exceeding the threshold. The motion control energy consumption threshold is when the energy consumption value for 10 consecutive control cycles exceeds 80% of the robot's rated energy consumption. Exceeding this value is considered an energy consumption exceeding the threshold.

[0056] The calculation dimensions of motion control energy consumption include three core dimensions: the operating power of the robot drive motor, the motion energy consumption of the steering mechanism, and the additional energy consumption of speed and angular velocity adjustment. The statistical rule is to accumulate the energy consumption value according to the control cycle. The energy consumption of a single control cycle is the sum of the energy consumption of each dimension within that cycle. The energy consumption data is collected in real time and continuously statistically analyzed. The energy consumption data of the most recent 50 control cycles is retained for the determination of energy consumption exceeding the threshold and the dynamic adjustment of the weight of the optimization objective function.

[0057] When the visual positioning deviation exceeds a preset threshold, the weight of the visual feature visibility term in subsequent planning is dynamically increased, forcing the robot to return to the anchor point's visual field. At the same time, the motion control speed is fine-tuned to reduce the positioning deviation. When the deviation between the actual movement of the obstacle and the predicted trajectory exceeds a preset threshold, the time step of trajectory prediction is dynamically shortened, the update frequency of planning and prediction is increased, and the safety domain parameters and corresponding motion control strategies are adjusted synchronously. When the path tracking error or motion control execution error exceeds a preset threshold, the rolling time domain length and control quantity adjustment gradient of the model predictive control optimization are dynamically adjusted to optimize the speed and angular velocity control commands. When the motion control energy consumption exceeds a preset threshold, the weight allocation of the objective function of the model predictive control optimization is dynamically adjusted to reduce energy consumption while ensuring safety.

[0058] This invention also provides a visual navigation-based dynamic obstacle avoidance planning system for mobile robots, which, based on the above method, consists of hardware modules and software functional modules: The hardware module includes a binocular vision camera, a main control unit, a motion actuator, and a parameter acquisition unit. The binocular vision camera is used to acquire environmental image sequences in real time. The main control unit is used to run various software function modules, process image data, execute planning algorithms, and output control commands. The motion actuator receives control commands and completes the adjustment of the robot's speed and angular velocity. The parameter acquisition unit is used to acquire the robot's kinematic parameters and operating status data. The main control unit is equipped with an edge computing chip with a computing power of no less than 10 TOPS. It supports forward inference acceleration of lightweight Transformer dual-branch shared network and social LSTM network, with a total model inference latency of ≤20ms for a single frame image. The image acquisition frame rate of the binocular vision camera is 30fps, and the output format is RGB-D. It communicates with the main control unit via Gigabit Ethernet to ensure low-latency real-time transmission of image data. The parameter acquisition unit has a sampling frequency of 100Hz, providing high refresh rate robot kinematic parameters for AI model input to ensure the real-time performance of model input. The depth information acquired by binocular vision is processed using a 3×3 pixel neighborhood interpolation method to handle depth anomalies. A single pixel depth value with a change of more than 50% from the neighborhood mean is identified as an invalid point, and a number of consecutive invalid points exceeding 8 is identified as an invalid region. Invalid points and invalid regions are filled in by using the mean of the surrounding valid depth values. At the same time, the depth information is smoothed and filtered to remove depth noise caused by ambient lighting and object reflections. Only the filled and smoothed depth information is used for obstacle position calculation and pose determination to ensure the validity of environmental perception data.

[0059] The software functional modules include a map anchor point initialization module, a pose semantic perception module, a trajectory probability prediction module, a global path replanning module, an obstacle avoidance safety domain construction module, a local obstacle avoidance planning module, and a closed-loop feedback correction module. The map anchor point initialization module constructs a topology and grid fusion map using visual SLAM technology, selects global visual feature anchor points and associates them with motion control adaptation parameters to generate an initial global optimal path. The pose semantic perception module uses a lightweight Transformer dual-branch shared network to simultaneously complete the robot's global pose calculation, obstacle semantic segmentation, and initial motion state judgment. The trajectory probability prediction module, based on perception data, captures obstacle interaction behavior through a social LSTM network and outputs trajectory probability distribution and collision-related parameters. The global path replanning module triggers replanning and updates the global path and motion control adaptation parameters based on the collision prediction results. The obstacle avoidance safety domain construction module constructs an adaptive obstacle avoidance safety domain driven by both obstacle characteristics and robot motion state, and associates it with differentiated motion control strategies. The local obstacle avoidance planning module constructs a model predictive control optimization objective function and outputs the optimal local obstacle avoidance path and control quantity. The closed-loop feedback correction module collects various deviation data and performs differentiated corrections.

[0060] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus.

[0061] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. A dynamic obstacle avoidance planning method for mobile robots based on vision navigation, characterized in that, The specific steps include the following: Map anchor point initialization stage: Visual SLAM technology is used to collect continuous image sequences of the target scene, construct a topology and grid fusion map containing semantic information, select visual features with robustness requirements as global anchor points and bind them to topology nodes, associate robot kinematic parameters, configure motion control adaptation parameters, and generate an initial global optimal path based on navigation start and end points; In the pose semantic perception stage: the robot uses its binocular vision device to collect environmental images in real time. Through a lightweight Transformer dual-branch shared network, the robot's global pose calculation, environmental obstacle semantic segmentation, and initial motion state judgment are completed simultaneously. The robot's motion state constraints are incorporated to achieve multi-task parallel output. Trajectory probability prediction stage: Based on the semantic and motion state information of dynamic obstacles, extract the real-time motion parameters of obstacles, combine the robot's own motion state with the input of the social LSTM network, capture the interaction behavior between obstacles and between the robot and obstacles, and output the probability distribution of the future motion trajectory of obstacles, prediction confidence and collision-related parameters. Global path replanning phase: Based on collision prediction results, replanning is triggered when there is a high collision risk in the global path. The goal is to optimize both path length and motion control energy consumption, while satisfying constraints including anchor point visibility and motion parameter matching, and updating the global path and related control parameters. Obstacle avoidance safety domain construction phase: Combine obstacle characteristics and robot motion state to build an adaptive obstacle avoidance safety domain decision model, construct differentiated safety domains for different types of obstacles and match corresponding motion control strategies, and dynamically adjust safety domain parameters and passing speed; Local obstacle avoidance planning stage: Based on the updated global path and obstacle avoidance safety domain, a model predictive control optimization objective function that integrates motion control performance is constructed to meet multiple requirements including path tracking, motion constraints, and visual anchor point visibility. The optimal local obstacle avoidance path and motion control quantity are output through rolling temporal optimization. Closed-loop feedback correction stage: The control quantity output by the local planning is sent to the motion actuator, and navigation, prediction and control related deviation parameters are collected in real time. Differentiated correction strategies are executed for different deviations to optimize control accuracy and energy consumption.

2. The method for dynamic obstacle avoidance planning of a mobile robot based on vision navigation according to claim 1, characterized in that, In the map anchor point initialization phase, the grid map is used to describe the obstacle occupancy status, and the topology map is used for the node-based planning of the global path. The global visual feature anchor points include ORB feature points with robustness satisfying a preset threshold and fixed semantic features, and are bound one-to-one with the topology map nodes. A visible field constraint range is generated for each global visual feature anchor point, and each anchor point is associated with corresponding motion control adaptation parameters. When the initial global optimal path is generated, the navigation system initialization and motion control parameter pre-configuration are completed simultaneously.

3. The method for dynamic obstacle avoidance planning of a mobile robot based on vision navigation according to claim 2, characterized in that, In the pose semantic perception stage, adjacent frame images acquired by the binocular vision camera are synchronously input into a lightweight Transformer dual-branch shared network. The first branch is used to match real-time image features with global visual feature anchor points to calculate the robot's global real-time pose, heading angle, and pose deviation rate. The second branch is used to output pixel-level semantic masks of obstacles, distinguish between static, dynamic, and semi-dynamic obstacles, and output the contours, global position information, and preliminary motion state judgment results of each obstacle. The dual-branch shared feature extraction backbone network is integrated with robot motion state constraints to achieve multi-task parallel output.

4. The method for dynamic obstacle avoidance planning of a mobile robot based on vision navigation according to claim 3, characterized in that, In the trajectory probability prediction stage, the real-time motion state of dynamic obstacles is extracted from the semantic information output from the pose semantic perception stage, the position changes of continuous frames, and the preliminary judgment results of motion state; the social LSTM network captures multi-obstacle interaction behavior through the social pooling layer, and quantifies the influence of robot motion on obstacle trajectory through the robot-obstacle interaction modeling layer. Collision-related parameters include the probability of collision between each dynamic obstacle and the robot's global path, the minimum collision time, and the minimum motion control required to avoid collisions.

5. The method for dynamic obstacle avoidance planning of a mobile robot based on vision navigation according to claim 4, characterized in that, In the global path replanning phase, the criteria for judging high collision risk are the collision probability, minimum collision time, and minimum motion control amount output in the trajectory probability prediction phase. Replanning is triggered when a high-probability collision risk covers a critical segment of the current global path. The critical segment refers to the path segment from the robot's current position to a preset distance ahead, path turns, dense areas of visual anchor points, and narrow path segments. The hard constraints of replanning include that the path must pass through a preset proportion of the visible range of global visual feature anchor points, the path curvature and slope must match the robot's kinematic parameters, and high-probability movement areas of dynamic obstacles must be avoided while reserving redundant space for motion control adjustment. After replanning, the global path, anchor point constraints, and motion control adaptation parameters are updated synchronously.

6. The method for dynamic obstacle avoidance planning of a mobile robot based on vision navigation according to claim 5, characterized in that, In the obstacle avoidance safety domain construction phase, the adaptive obstacle avoidance safety domain decision model is driven by both obstacle characteristics and robot motion state; differentiated safety domains and corresponding motion control strategies are adapted to different types of obstacles: elliptical dynamic safety domains are set for high-speed, high-collision-risk dynamic obstacles, and matched with deceleration and emergency turning motion control strategies; circular fixed safety domains and uniform speed bypass and small-amplitude turning strategies are adopted for low-speed, low-collision-risk dynamic obstacles and semi-dynamic obstacles; only minimum collision distance constraints are set for static obstacles, and the passing speed of static obstacles is dynamically adjusted according to the robot's own motion state.

7. The method for dynamic obstacle avoidance planning of a mobile robot based on vision navigation according to claim 6, characterized in that, In the local obstacle avoidance planning stage, the model predictive control optimization objective function includes four core weight terms: minimizing global path tracking error, ensuring robot motion smoothness, ensuring visual feature anchor point visibility, and minimizing motion control energy consumption. Hard constraints include ensuring the planned path does not enter the obstacle avoidance safety domain, satisfying robot kinematic constraints, ensuring the robot camera observes at least a preset number of global visual feature anchor points at each moment, and ensuring continuous and smooth motion control commands. The rolling time domain optimization outputs the optimal local obstacle avoidance path for the next 5-10 control cycles, the corresponding velocity and angular velocity control quantities, and simultaneously outputs the adjustment gradient of each control quantity.

8. The method for dynamic obstacle avoidance planning of a mobile robot based on vision navigation according to claim 7, characterized in that, In the closed-loop feedback correction phase, the deviation parameters collected in real time include the robot's actual operating posture, visual positioning deviation, deviation between the actual and predicted trajectory of obstacles, and motion control execution error. At the same time, the robot's motion control energy consumption index is collected. The differentiated correction strategy includes: when the visual positioning deviation exceeds the preset threshold, increasing the weight of the visual feature anchor point visibility guarantee item and fine-tuning the robot's motion speed; when the obstacle trajectory deviation exceeds the threshold, shortening the prediction time step and adjusting the safety domain and control strategy; when the path tracking or control execution error exceeds the threshold, adjusting the rolling time domain length and the control quantity adjustment gradient; and when the energy consumption exceeds the threshold, adjusting the weight allocation of the objective function and increasing the weight ratio of the lowest motion control energy consumption item.

9. A visual navigation-based dynamic obstacle avoidance planning system for mobile robots, based on the method described in any one of claims 1-8, characterized in that, It consists of hardware modules and software functional modules: The hardware module includes a binocular vision camera, a main control unit, a motion actuator, and a parameter acquisition unit. The binocular vision camera is used to acquire environmental image sequences in real time. The main control unit is used to run various software function modules, process image data, execute planning algorithms, and output control commands. The motion actuator receives control commands and completes the adjustment of the robot's speed and angular velocity. The parameter acquisition unit is used to acquire the robot's kinematic parameters and operating status data. The software functional modules include a map anchor point initialization module, a pose semantic perception module, a trajectory probability prediction module, a global path replanning module, an obstacle avoidance safety domain construction module, a local obstacle avoidance planning module, and a closed-loop feedback correction module. The map anchor point initialization module constructs a topology and grid fusion map using visual SLAM technology, selects global visual feature anchor points and associates them with motion control adaptation parameters to generate an initial global optimal path. The pose semantic perception module uses a lightweight Transformer dual-branch shared network to simultaneously complete the robot's global pose calculation, obstacle semantic segmentation, and initial motion state judgment. The trajectory probability prediction module, based on perception data, captures obstacle interaction behavior through a social LSTM network and outputs trajectory probability distribution and collision-related parameters. The global path replanning module triggers replanning and updates the global path and motion control adaptation parameters based on the collision prediction results. The obstacle avoidance safety domain construction module constructs an adaptive obstacle avoidance safety domain driven by both obstacle characteristics and robot motion state, and associates it with differentiated motion control strategies. The local obstacle avoidance planning module constructs a model predictive control optimization objective function and outputs the optimal local obstacle avoidance path and control quantity. The closed-loop feedback correction module collects various deviation data and performs differentiated corrections.

Images