Methods, devices, equipment and media for dynamic obstacle avoidance navigation of mobile robots

By combining inertial and visual sensing data, using physical motion models and machine learning models to predict obstacle trajectories, and dynamically adjusting the fusion weights, the problem of mobile robots quickly navigating around dynamic obstacles is solved, achieving efficient and reliable obstacle avoidance decisions.

CN123103703APending Publication Date: 2026-11-17HAOCUN TECHNOLOGY (HANGZHOU) CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202611612992.2
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2026-10-19
Publication Date
2026-11-17

AI Technical Summary

Technical Problem

Existing technologies cannot quickly identify and avoid dynamic obstacles when mobile robots are driving autonomously, resulting in a high risk of collision, especially in scenarios such as pedestrians suddenly changing direction or vehicles braking suddenly, where obstacle avoidance decisions are delayed.

Method used

By acquiring inertial and visual sensor data, and combining physical motion models and machine learning models, obstacle trajectory prediction is performed, the fusion weights are dynamically adjusted, collision risks are detected in real time, and detour paths are planned.

Benefits of technology

It significantly improves the accuracy of dynamic obstacle trajectory prediction, reduces the probability of missed detections and false detections, enhances the real-time performance and reliability of obstacle avoidance decisions, and ensures that robots can safely and efficiently avoid obstacles in complex environments.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN123103703A_ABST
    Figure CN123103703A_ABST
Patent Text Reader

Abstract

This application discloses a method, apparatus, device, and medium for dynamic obstacle avoidance navigation of a mobile robot, relating to the field of robot navigation and control technology. The method includes: acquiring inertial and visual sensing data of the mobile robot to determine target obstacles and motion state vectors around the robot, inputting these data into a first predictor of a physical model and a second predictor of a machine learning model for trajectory prediction; dynamically determining the respective fusion weights based on the output trajectory prediction results to obtain trajectory fusion information; detecting collision risks at a preset frequency based on the trajectory fusion information; and planning an obstacle avoidance path and controlling the robot's movement. This method can improve the accuracy of dynamic obstacle trajectory prediction, effectively reduce the probability of missed and false detections, improve the real-time performance and reliability of obstacle avoidance decisions, and ensure that the mobile robot safely and efficiently avoids obstacles in complex dynamic environments.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This application relates to the field of robot navigation and control technology, and in particular to methods, devices, equipment and media for dynamic obstacle avoidance navigation of mobile robots. Background Technology

[0002] When mobile robots autonomously navigate in complex and dynamic environments such as warehousing and logistics, park delivery, and home services, they must perceive and safely avoid dynamic obstacles such as pedestrians and vehicles in real time. Because the movement of obstacles is sudden and uncertain, if the robot cannot complete the perception in a very short time, collision accidents are very likely to occur.

[0003] Current methods employ visual sensors combined with deep learning object detection algorithms to identify dynamic obstacles and generate detour paths using obstacle avoidance algorithms. However, these methods suffer from significant latency, meaning the robot has already moved before the perception results are output. This can easily lead to delayed dynamic obstacle avoidance decisions, and the risk of collisions increases dramatically in scenarios such as pedestrians suddenly changing direction or vehicles braking abruptly. Therefore, how to quickly achieve detour navigation around dynamic obstacles has become an urgent problem to be solved. Summary of the Invention

[0004] The main objective of this application is to provide a method, apparatus, device, and medium for dynamic obstacle avoidance navigation of a mobile robot, aiming to solve the technical problem of how to quickly achieve obstacle avoidance navigation.

[0005] To achieve the above objectives, this application proposes a dynamic obstacle avoidance navigation method for a mobile robot, the method comprising: Acquire inertial and visual sensor data from the mobile robot; Based on the inertial sensing data and the visual sensing data, the target obstacles around the mobile robot and the motion state vectors of the target obstacles are determined. Based on the target obstacle and the motion state vector, a first predictor of a preset physical motion model and a second predictor of a preset machine learning model are input to predict the motion trajectory, obtain the trajectory prediction result, and determine the fusion weight of the first predictor and the fusion weight of the second predictor based on the trajectory prediction result. Based on the fusion weights of the first predictor and the fusion weights of the second predictor, the motion state vector is fused to obtain trajectory fusion information. Based on the trajectory fusion information, the collision risk between the mobile robot and the target obstacle is detected at a preset frequency, and if the collision risk exists, a detour path to avoid the target obstacle is determined based on the trajectory fusion information. The mobile robot is controlled to travel along the detour path using a preset independent safety passage.

[0006] In one embodiment, the step of determining the target obstacles around the mobile robot and the motion state vectors of the target obstacles based on the inertial sensing data and the visual sensing data includes: The pose information of the mobile robot is determined based on the inertial sensing data; The target obstacles around the mobile robot and their location information are determined based on the visual sensing data. Based on the pose information, the coordinate system of the position information is transformed to the corresponding global coordinate system to obtain the global position information; The motion state vector of the target obstacle is determined by fusing the global position information with the motion increment of the inertial sensing data. The motion state vector includes the position, velocity and / or acceleration of the target obstacle in the global coordinate system.

[0007] In one embodiment, the step of predicting the motion trajectory by inputting a first predictor of a preset physical motion model and a second predictor of a preset machine learning model based on the target obstacle and the motion state vector, obtaining a trajectory prediction result, and determining the fusion weights of the first predictor and the second predictor based on the trajectory prediction result includes: Based on the motion state vector, the first predictor of the preset physical motion model is input to predict the motion trajectory, and the first trajectory prediction result is obtained from the trajectory prediction result. The first predictor contains multiple motion models, and the prediction outputs of the multiple motion models are weighted and fused through an interactive multi-model method to obtain the first trajectory prediction result. Based on the target obstacle and the motion state vector, a second predictor of a preset machine learning model is input to predict the motion trajectory, thereby obtaining a second trajectory prediction result in the trajectory prediction results. The second predictor predicts the multimodal trajectory distribution of the target obstacle based on the trajectory sequence of the target obstacle within a preset historical window and environmental context features, thereby obtaining the second trajectory prediction result. Obtain a time window, and determine the fusion weight of the first predictor corresponding to the first trajectory prediction result and the fusion weight of the second predictor corresponding to the second trajectory prediction result based on the time window.

[0008] In one embodiment, the first predictor of the preset physical motion model includes a uniform motion model, a uniformly accelerated motion model, and a constant velocity motion model; The step of weightedly fusing the prediction outputs of the multiple motion models through an interactive multi-model approach to obtain the first trajectory prediction result includes: The motion state vector is predicted by the uniform motion model, the uniformly accelerated motion model and the constant velocity motion model to obtain the prediction output of each motion model; The model probability of each predicted output is determined based on the interactive multi-model approach represented by the preset Markov transition probability matrix. The predicted outputs are weighted and fused according to the probabilities of each model to obtain the first trajectory prediction result.

[0009] In one embodiment, the step of determining the fusion weights of the first predictor corresponding to the first trajectory prediction result and the fusion weights of the second predictor corresponding to the second trajectory prediction result based on the time window includes: The time window is compared with a preset time threshold to obtain the comparison result; When the comparison result is that the time window is less than or equal to a preset time threshold, the fusion weight corresponding to the first trajectory prediction result is greater than the fusion weight corresponding to the second trajectory prediction result. If the comparison result is that the time window is greater than the preset time threshold, the fusion weight corresponding to the second trajectory prediction result is greater than the fusion weight corresponding to the first trajectory prediction result.

[0010] In one embodiment, the step of detecting the collision risk between the mobile robot and the target obstacle at a preset frequency based on the trajectory fusion information includes: Based on the trajectory fusion information, the probabilistic reachable area of ​​the target obstacle within a preset frequency is determined; The current motion state of the mobile robot is obtained, and a collision index between the mobile robot and the target obstacle is calculated based on the probable reachable area and the current motion state. The collision index includes collision time, collision probability, obstacle category priority, and detour space cost. The detour safety distance is calculated based on the collision time, the collision probability, the obstacle category priority, and the detour space cost. If the detour safety distance is less than the preset safety buffer zone, it is determined that there is a collision risk.

[0011] In one embodiment, the step of determining a detour path to avoid the target obstacle based on the trajectory fusion information when the collision risk exists includes: In the event of the collision risk, the trajectory fusion information is mapped to a dynamic obstacle cost region in a preset two-dimensional grid map, wherein the cost of the dynamic obstacle cost region is proportional to the collision probability of the target obstacle. Based on the two-dimensional grid map, a preset spatiotemporal algorithm is used to search for an initial detour path from the current position of the mobile robot to the preset target point while avoiding the dynamic obstacle cost area; Based on the initial detour path input to the preset optimizer model, the motion trajectory at the current moment is predicted to obtain the current trajectory prediction information; The current position of the mobile robot is obtained, and the initial detour path is locally optimized in real time based on the current trajectory prediction information to obtain a detour path that avoids the target obstacle from the current position of the mobile robot.

[0012] Furthermore, to achieve the above objectives, this application also proposes a dynamic obstacle avoidance navigation device for a mobile robot, the mobile robot dynamic obstacle avoidance navigation device comprising: The acquisition module is used to acquire inertial and visual sensor data of the mobile robot. The determination module is used to determine the target obstacles around the mobile robot and the motion state vector of the target obstacles based on the inertial sensing data and the visual sensing data. The prediction module is used to predict the motion trajectory based on the target obstacle and the motion state vector, by inputting a first predictor of a preset physical motion model and a second predictor of a preset machine learning model, to obtain the trajectory prediction result, and to determine the fusion weight of the first predictor and the fusion weight of the second predictor based on the trajectory prediction result. The fusion module is used to fuse the motion state vector according to the fusion weights of the first predictor and the fusion weights of the second predictor to obtain trajectory fusion information; The planning module is used to detect the collision risk between the mobile robot and the target obstacle at a preset frequency based on the trajectory fusion information, and determine a detour path to avoid the target obstacle based on the trajectory fusion information if the collision risk exists. An execution module is used to control the mobile robot to travel along the bypass path via a preset independent safety channel.

[0013] In addition, to achieve the above objectives, this application also proposes a mobile robot dynamic obstacle avoidance navigation device, the device comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, the computer program being configured to implement the steps of the mobile robot dynamic obstacle avoidance navigation method described above.

[0014] In addition, to achieve the above objectives, this application also proposes a storage medium, which is a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, it implements the steps of the mobile robot dynamic obstacle avoidance navigation method described above.

[0015] One or more technical solutions proposed in this application have at least the following technical effects: This embodiment proposes a dynamic obstacle avoidance navigation method for a mobile robot. The method involves acquiring inertial and visual sensing data of the mobile robot; determining target obstacles around the mobile robot and their motion state vectors based on the inertial and visual sensing data; inputting a first predictor of a preset physical motion model and a second predictor of a preset machine learning model to predict the motion trajectory based on the target obstacles and the motion state vectors, obtaining a trajectory prediction result, and determining the fusion weights of the first and second predictors based on the trajectory prediction result; fusing the motion state vectors based on the fusion weights of the first and second predictors to obtain trajectory fusion information; detecting the collision risk between the mobile robot and the target obstacle at a preset frequency based on the trajectory fusion information, and determining a detour path to avoid the target obstacle if the collision risk exists; and controlling the mobile robot to travel along the detour path using a preset independent safety channel. This application acquires inertial and visual sensor data from a mobile robot to determine target obstacles and motion state vectors around the robot. These data are then input into a first predictor of a physical model and a second predictor of a machine learning model for trajectory prediction. Based on the output trajectory prediction results, the respective fusion weights are dynamically determined to obtain trajectory fusion information. Collision risks are detected at a preset frequency based on the trajectory fusion information. If a risk exists, an alternative path is planned and the robot is controlled to travel along the path. This combines the generalization ability of the physical model with the data fitting ability of the machine learning model. Furthermore, the fusion weights can be adaptively adjusted based on real-time trajectory prediction results, significantly improving the accuracy of dynamic obstacle trajectory prediction, effectively reducing the probability of missed and false detections, improving the real-time performance and reliability of obstacle avoidance decisions, and ensuring that the mobile robot can safely and efficiently avoid obstacles in complex dynamic environments. Attached Figure Description

[0016] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0017] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0018] Figure 1 A flowchart illustrating an embodiment of the mobile robot dynamic obstacle avoidance navigation method of this application; Figure 2 This is a schematic diagram of the dual-channel detection process for the mobile robot dynamic obstacle avoidance navigation method of this application; Figure 3 This is a schematic diagram of the dual-model parallel architecture of the mobile robot dynamic obstacle avoidance navigation method of this application; Figure 4 This is a schematic diagram illustrating the obstacle avoidance navigation method for mobile robots according to this application; Figure 5 This is a timing diagram of the emergency braking safety passage for the mobile robot dynamic obstacle avoidance navigation method of this application; Figure 6 This is a schematic diagram of the overall architecture of the mobile robot dynamic obstacle avoidance navigation method of this application; Figure 7 This is a flowchart illustrating Embodiment 2 of the mobile robot dynamic obstacle avoidance navigation method of this application; Figure 8 This is a schematic diagram of the module structure of the mobile robot dynamic obstacle avoidance navigation device according to an embodiment of this application; Figure 9 This is a schematic diagram of the device structure of the hardware operating environment involved in the mobile robot dynamic obstacle avoidance navigation method in the embodiments of this application.

[0019] The purpose, features, and advantages of this application will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0020] It should be understood that the specific embodiments described herein are merely illustrative of the technical solutions of this application and are not intended to limit this application.

[0021] To better understand the technical solution of this application, a detailed description will be provided below in conjunction with the accompanying drawings and specific implementation methods.

[0022] The main solution of this application embodiment is as follows: acquiring inertial sensing data and visual sensing data of a mobile robot; determining target obstacles around the mobile robot and their motion state vectors based on the inertial sensing data and the visual sensing data; inputting a first predictor of a preset physical motion model and a second predictor of a preset machine learning model to predict the motion trajectory based on the target obstacles and the motion state vectors, obtaining a trajectory prediction result, and determining the fusion weights of the first predictor and the second predictor based on the trajectory prediction result; fusing the motion state vectors based on the fusion weights of the first predictor and the second predictor to obtain trajectory fusion information; detecting the collision risk between the mobile robot and the target obstacle at a preset frequency based on the trajectory fusion information, and determining a detour path to avoid the target obstacle based on the trajectory fusion information if the collision risk exists; and controlling the mobile robot to travel along the detour path using a preset independent safety channel.

[0023] In this embodiment, for ease of description, the following description uses a mobile robot as the execution subject.

[0024] Because the existing technology has a long overall delay, it means that the robot has already moved before the perception results are output, which can easily lead to a delay in dynamic obstacle avoidance decision-making. In scenarios such as pedestrians suddenly changing direction or vehicles braking suddenly, the risk of collision increases sharply.

[0025] This application provides a solution that acquires inertial and visual sensor data from a mobile robot to determine target obstacles and motion state vectors around the robot. These vectors are then input into a first predictor of a physical model and a second predictor of a machine learning model for trajectory prediction. Based on the output trajectory prediction results, the respective fusion weights are dynamically determined to obtain trajectory fusion information. Collision risks are detected at a preset frequency based on the trajectory fusion information. If a risk exists, an alternative path is planned and the robot is controlled to travel along the path. This combines the generalization ability of the physical model with the data fitting ability of the machine learning model, and the fusion weights can be adaptively adjusted according to the real-time trajectory prediction results. This significantly improves the accuracy of dynamic obstacle trajectory prediction, effectively reduces the probability of missed and false detections, improves the real-time performance and reliability of obstacle avoidance decisions, and ensures that the mobile robot can safely and efficiently avoid obstacles in complex dynamic environments.

[0026] Based on this, embodiments of this application provide a method for dynamic obstacle avoidance navigation of a mobile robot, referring to... Figure 1 , Figure 1 This is a flowchart illustrating the first embodiment of the mobile robot dynamic obstacle avoidance navigation method of this application.

[0027] In this embodiment, the mobile robot dynamic obstacle avoidance navigation method includes steps S10 to S60: Step S10: Acquire inertial sensing data and visual sensing data of the mobile robot; In a specific embodiment, the inertial and visual sensing data of the mobile robot can be acquired by the EMS-Vision Multimodal Heterogeneous Perception Fusion Module. This module serves as the foundational perception layer, responsible for real-time detection and tracking of dynamic obstacles. Simultaneously, the system operates two perception channels: the MEMS inertial sensing channel and the visual sensing channel. The MEMS inertial sensing channel acquires inertial data, which is the raw output collected at a frequency of 1000Hz by a high-precision MEMS inertial measurement unit mounted on the robot. After attitude calculation, the robot's six-degree-of-freedom pose and motion increments can be obtained. Furthermore, abnormal ground vibration characteristics can be extracted through chassis vibration spectrum analysis to assist in determining the existence of dynamic objects in blind spots. The high-precision MEMS inertial measurement unit integrates a three-axis gyroscope, a three-axis accelerometer, a three-axis magnetometer, and a high-precision IMU module. The visual sensing channel acquires visual sensing data, which is generated by an RGB-D depth camera (such as Intel RealSense). Environmental image data acquired by a D455 (90fps depth frame rate) and / or a wide-angle RGB camera is processed by a lightweight object detection neural network (such as YOLOv8-nano, with an inference latency of approximately 8ms) to perform 2D / 3D detection and classification of dynamic obstacles such as pedestrians, vehicles, and other robots within the field of view. The network can output category labels, 2D bounding boxes, 3D spatial positions (x, y, z), and confidence scores for dynamic obstacles such as pedestrians and vehicles, containing rich semantic information for identifying the category of obstacles and their precise spatial positions.

[0028] Step S20: Based on the inertial sensing data and the visual sensing data, determine the target obstacles around the mobile robot and the motion state vector of the target obstacles; It should be noted that the target obstacles around the mobile robot are dynamic objects identified in the robot's environment that may affect the robot's movement, including pedestrians, bicycles, and other mobile robots. The motion state vector of the target obstacle is a set of parameters used to describe the obstacle's motion state in the global coordinate system, specifically a 9-dimensional state vector: x=[px, py, pz, vx, vy, vz, ax, ay, az] Where px, py, and pz are the three-dimensional positions of the target obstacle in the global coordinate system, vx, vy, and vz are the three-dimensional motion velocities, and ax, ay, and az are the three-dimensional motion accelerations. Each target obstacle maintains an independent motion state vector. When there are multiple target obstacles at the same time, multiple independent 9-dimensional state vectors are maintained in parallel.

[0029] In a specific embodiment, the pose information of the mobile robot is determined based on the inertial sensing data, that is, the six-degree-of-freedom pose of the robot in the global coordinate system is calculated by a neural network matrix.

[0030] The target obstacles around the mobile robot and their location information are determined based on the visual sensing data. Specifically, environmental images are acquired by an RGB-D depth camera, and the three-dimensional spatial position of the obstacles in the camera coordinate system is output by a target detection network.

[0031] Based on the pose information, the coordinate system of the position information is transformed to the corresponding global coordinate system to obtain the global position information. That is, based on the timestamp of the MEMS sensor hardware, the timestamp of the visual detection result is aligned to the MEMS time axis through linear interpolation to eliminate the time deviation between the two channels. Combined with the pre-calibrated extrinsic parameter matrix (rotation matrix R∈SO(3) and translation vector t∈R³), the obstacle position is transformed from the camera coordinate system to the robot body coordinate system. The MEMS real-time pose transformation is used to transform it to the global world coordinate system to obtain the global position information.

[0032] The motion state vector of the target obstacle is determined by fusing the global position information with the motion increment of the inertial sensing data. The motion state vector includes the position, velocity and / or acceleration of the target obstacle in the global coordinate system. That is, the global position information and the high-frequency incremental motion estimation provided by the MEMS channel are input into an adaptive Kalman filter framework, the visual channel provides position observations, the MEMS channel provides process model constraints, and the fusion weights are dynamically adjusted according to the real-time confidence of each channel, thereby outputting the motion state vector of the target obstacle.

[0033] It is understandable that, such as Figure 2 As shown, Figure 2This is a schematic diagram of the dual-channel detection process of the mobile robot dynamic obstacle avoidance navigation method of this application. The first channel is for the acquisition of raw data from the MEMS sensor at 1000Hz, the neural network matrix for attitude calculation, and the global pose calculation. The second channel is for the acquisition of images from the RGB-D camera, the YOLOv8-nano for target detection, the 3D position estimation, and the transformation from the camera coordinate system to the global coordinate system. This process aligns the timestamps of the two channels, unifies the spatial coordinates, performs adaptive Kalman weighted fusion, and outputs the obstacle status at 200Hz. For example, key time indicators are marked in the figure (MEMS delay 3ms, visual detection delay 8ms, fusion delay 5ms).

[0034] In one feasible implementation, step S20 may include steps A11 to A13: Step A11: Determine the pose information of the mobile robot based on the inertial sensing data; It should be noted that the pose information of the mobile robot is the six-degree-of-freedom pose of the mobile robot body in the global coordinate system, including three-dimensional position coordinates (x, y, z) and three-dimensional attitude angles (roll angle, pitch angle, yaw angle), which can characterize the spatial position and orientation of the mobile robot in the global coordinate system at the current moment.

[0035] Step A12: Determine the target obstacles around the mobile robot and the location information of the target obstacles based on the visual sensing data; It should be noted that the location information of the target obstacle refers to its three-dimensional spatial position in the camera coordinate system, specifically the three-dimensional coordinates (x, y, z) of the obstacle obtained by the RGB-D depth camera. These coordinates represent the spatial coordinates of the target obstacle relative to the location of the vision sensor.

[0036] Step A13: Transform the coordinate system of the position information to the corresponding global coordinate system based on the pose information to obtain the global position information; It should be noted that the global position information refers to the three-dimensional spatial position of the target obstacle in the global world coordinate system, which is used to uniformly describe the absolute coordinates of the obstacle in the environment.

[0037] Understandably, the global coordinate system is a unified world coordinate system used for mobile robot navigation. The robot's own pose, obstacle positions, target point positions, and planned paths are all represented in this coordinate system.

[0038] Step A14: Based on the fusion processing of the global position information and the motion increment of the inertial sensing data, the motion state vector of the target obstacle is determined. The motion state vector includes the position, velocity and / or acceleration of the target obstacle in the global coordinate system.

[0039] It should be noted that the position, velocity, and / or acceleration in the global coordinate system are all components of the motion state vector of the target obstacle, wherein the motion state vector of the target obstacle is a 9-dimensional state vector x=[px, py, pz,vx, vy, vz, ax, ay, az] At this point, px, py, and pz are the three-dimensional positions in the global coordinate system, vx, vy, and vz are the three-dimensional motion velocities, and ax, ay, and az are the three-dimensional motion accelerations.

[0040] Understandably, fusion processing is the process of combining the position observations provided by the visual channel with the high-frequency incremental motion estimations provided by the MEMS channel to obtain a more accurate and real-time estimate of the motion state of the target obstacle.

[0041] Step S30: Based on the target obstacle and the motion state vector, input the first predictor of the preset physical motion model and the second predictor of the preset machine learning model to predict the motion trajectory, obtain the trajectory prediction result, and determine the fusion weight of the first predictor and the fusion weight of the second predictor based on the trajectory prediction result. It should be noted that the first predictor of the preset physical motion model is a predictor that predicts the future trajectory of the target obstacle based on physical motion laws (such as uniformly accelerated motion, uniform turning motion, etc.). This predictor takes physical quantities such as position, velocity, and acceleration in the motion state vector as input, and calculates the state of the target obstacle at various future moments through kinematic or dynamic equations, thereby generating a predicted trajectory that conforms to physical constraints. The second predictor of the preset machine learning model is a predictor that predicts the future trajectory of the target obstacle based on machine learning algorithms (such as long short-term memory networks, graph neural networks, Transformer, etc.). This predictor learns the motion patterns and interaction rules of the obstacle from a large amount of historical trajectory data, and takes the historical state sequence or current state of the target obstacle as input, and outputs the trajectory prediction result for a period of time in the future.

[0042] Understandably, a motion trajectory is a sequence of locations that a target obstacle is expected to pass through over a period of time in the future. It is represented by coordinates (x, y, z) in a global coordinate system and corresponding timestamps, and is used to characterize the future motion path and trend of the target obstacle.

[0043] Additionally, it should be noted that the fusion weight of the first predictor is the proportion or contribution of the physical motion model's prediction result in the final fused trajectory, representing the credibility or effectiveness of the predictor's output result in the current scene. The higher the weight value, the greater the influence of the physical motion model's prediction result on the final trajectory. Similarly, the fusion weight of the second predictor is the proportion or contribution of the machine learning model's prediction result in the final fused trajectory, representing the credibility or effectiveness of the predictor's output result in the current scene. The higher the weight value, the greater the influence of the machine learning model's prediction result on the final trajectory.

[0044] In a specific embodiment, the dynamic obstacle trajectory prediction module can predict the short-term (0-500ms) and medium-term (500ms-2s) trajectory of each detected dynamic obstacle based on the target obstacle and the motion state vector. The dynamic obstacle trajectory prediction module adopts a dual-model parallel architecture, namely a first predictor of the physical motion model and a second predictor of the preset machine learning model.

[0045] Based on the motion state vector, a first predictor of a preset physical motion model is input to predict the motion trajectory, resulting in a first trajectory prediction result. The first predictor contains multiple motion models, and the prediction outputs of the multiple motion models are weighted and fused through an interactive multi-model approach to obtain the first trajectory prediction result. Based on the target obstacle and the motion state vector, a second predictor of a preset machine learning model is input to predict the motion trajectory, resulting in a second trajectory prediction result. The second predictor predicts the multimodal trajectory distribution of the target obstacle based on the trajectory sequence of the target obstacle within a preset historical window and environmental context features, resulting in the second trajectory prediction result. A time window is obtained, and the fusion weights of the first predictor corresponding to the first trajectory prediction result and the second predictor corresponding to the second trajectory prediction result are determined based on the time window.

[0046] It is understandable that, such as Figure 3 As shown, Figure 3 This diagram illustrates the dual-model parallel architecture of the mobile robot dynamic obstacle avoidance navigation method of this application. It shows the parallel architecture of the physical model predictor (IMM filter containing three sub-models: CV / CA / CTRV) and the learning predictor (ST-GNN intention-aware model), as well as the short-term / medium-to-long-term weight switching mechanism of the adaptive prediction fusion unit. The diagram also includes a visualization of the probabilistic reachable region of the prediction result.

[0047] Step S40: Based on the fusion weights of the first predictor and the fusion weights of the second predictor, the motion state vector is fused to obtain trajectory fusion information; It should be noted that the trajectory fusion information is a comprehensive sequence of motion states of the target obstacle at various moments in the future prediction time domain, used to characterize the most likely motion trend that the target obstacle will follow.

[0048] Understandably, fusion processing is the process of integrating the trajectory states output by different prediction models. That is, it uses their respective confidence weights to merge the state estimates at the same time, thereby combining the advantages of different predictors and compensating for their respective limitations, forming a continuous, smooth and more reliable unified trajectory estimation result.

[0049] In a specific embodiment, the dynamic obstacle trajectory prediction module can perform weighted synthesis based on the outputs of the first predictor and the second predictor to obtain trajectory fusion information. The trajectory fusion information is output in the form of a probabilistic reachable set, which highlights the stability and interpretability of the physical model in short-term prediction. Combined with the adaptability and multimodal expression capability of the machine learning model in long-term prediction of complex dynamic scenes, it significantly improves the accuracy of obstacle trajectory prediction by mobile robots in mixed environments such as crowds and vehicles.

[0050] Step S50: Based on the trajectory fusion information, detect the collision risk between the mobile robot and the target obstacle at a preset frequency, and if the collision risk exists, determine a detour path to avoid the target obstacle based on the trajectory fusion information; It should be noted that the collision risk refers to the collision risk between the mobile robot and the target obstacle. It can be quantified by indicators such as collision probability, minimum relative distance, and time margin (such as collision time TTC) to measure the likelihood and urgency of a collision between the two in the future prediction time domain. For example, when the mobile robot is traveling according to the currently planned movement path, if the distance between the predicted position of the target obstacle at a certain future time based on the trajectory fusion information and the predicted position of the mobile robot at that time is less than a preset safe distance threshold, then it is determined that there is a collision risk between the mobile robot and the target obstacle.

[0051] It is understood that the detour path is a safe and feasible trajectory that the mobile robot re-searches and generates in the current local planning space when the collision risk is detected. It starts from the current position, bypasses the target obstacle, and finally returns to the original navigation path, ensuring that the robot can safely and smoothly continue to perform the navigation task without colliding with the target obstacle or other environmental obstacles.

[0052] In a specific embodiment, the real-time detour path planning module can generate a safe and smooth detour path in real time based on the trajectory fusion information. For example, based on the trajectory fusion information, it determines the probabilistically accessible region of the target obstacle within a preset frequency; it acquires the current motion state of the mobile robot and calculates the collision index between the mobile robot and the target obstacle based on the probabilistically accessible region and the current motion state. The collision index includes collision time, collision probability, obstacle category priority, and detour space cost; it calculates the detour safety distance based on the collision time, collision probability, obstacle category priority, and detour space cost, and determines that there is a collision risk if the detour safety distance is less than a preset safety buffer. That is, based on the trajectory fusion information, it uses Gaussian process or kernel density estimation to probabilistically model the position uncertainty of the target obstacle in the future prediction time domain, and determines the probabilistically accessible region of the target obstacle at a preset frequency (synchronized with the 200Hz control cycle of the local detour layer) (e.g., 9). (5% confidence ellipse or contour line). At this time, the current motion state of the mobile robot (including position, speed, heading angle and acceleration) is obtained. Combined with the probable reachable area, the collision index is obtained by calculating the relative distance, relative speed and position distribution overlap. The collision index includes collision time (TTC), collision probability, risk priority based on obstacle category (e.g., pedestrians, vehicles, strollers, etc. are assigned different weights), and detour space cost (measuring the path offset and maneuvering difficulty required to bypass the area from the current state). Based on the collision time, the collision probability, the obstacle category priority and the detour space cost, the detour safety distance is dynamically calculated using weighted fusion or fuzzy inference. Thus, the detour strategy in multi-obstacle scenarios can also be globally optimized. That is, the decision-maker maintains an obstacle threat assessment matrix. Based on this matrix, the decision-maker selects the detour direction (left detour / right detour / decelerate and wait) and determines the detour safety distance. For example, the threat score is calculated based on the parameters in the obstacle threat assessment matrix, expressed as: S_i = w1·f_TTC(TTC_i) + w2·P_col,i + w3·C_i + w4·f_space(d_avail,i) Where S_i is the overall threat score of the i-th obstacle, a higher value indicates a greater threat to the vehicle; w1, w2, w3, w are the weighting coefficients of each threat factor, used to adjust the relative importance of collision time, collision probability, obstacle category, and available space to the overall score, satisfying normalization constraints; and TTC_i is the estimated collision time (Time To) between the i-th obstacle and the vehicle. Collision, i.e., the time required for the distance between the two vehicles to shrink to zero according to the current relative motion trend. The smaller the TTC_i, the more urgent the collision risk. f_TTC is a normalized mapping function that converts the collision time into a dimensionless score in the interval [0,1]. P_col,i is the probability of the i-th obstacle colliding with the vehicle. C_i is the category priority coefficient of the i-th obstacle, used to distinguish the danger level of different obstacle types (such as pedestrians, non-motorized vehicles, small cars, large vehicles, stationary obstacles, etc.). d_avail,i is the available space around the i-th obstacle that the vehicle can safely pass through (such as lateral distance, gap width, etc.), which represents the passability near the obstacle. f_space is a normalized mapping function that converts the size of the available space into a score. The smaller the available space, the higher the threat score after mapping, indicating that the avoidance is more difficult.

[0053] The safe detour distance is calculated based on the parameters in the obstacle threat assessment moment and is expressed as: d_safe,i = d_min + α·S_i + β·V_rel,i + d_buffer Wherein, d_safe,i is the safe distance to be maintained for the i-th obstacle, that is, the minimum distance that the vehicle should maintain with the obstacle during driving; d_min is the basic minimum safe distance, the static safe distance that must be maintained even when the threat is zero; α is the threat score gain coefficient, which is a scaling factor that converts the comprehensive threat score SiS_iSi into an additional safe distance. The higher the threat score, the larger the additional safe distance required; β is the relative speed gain coefficient, which is used to measure the contribution of relative speed to the safe distance; V_rel,i is the relative speed between the i-th obstacle and the vehicle. The larger the relative speed, the larger the space required for braking or avoidance, and therefore the safe distance increases accordingly; d_buffer is the safety buffer reserved for MEMS (Micro-Electro-Mechanical Systems) blind spot perception, which is used to compensate for potential errors caused by sensor blind spots, measurement noise, or perception uncertainties, and ensure that the safe distance has an additional margin.

[0054] The detour decision is calculated based on the parameters in the obstacle threat assessment moments, and is expressed as follows: Triggering detour when max(S_i)>S_th Where max(S_i) is the maximum value of all obstacle threat scores S_i in the current environment, representing the most pressing threat faced by the vehicle in front. S_th is the detour decision threshold. When the maximum threat score exceeds this threshold, it means that the danger level of the obstacle in front has exceeded the safety tolerance range. Continuing to go straight or follow the vehicle is unsafe, so detour behavior is triggered.

[0055] The direction selection is based on the parameters in the obstacle threat assessment moment, and is expressed as follows: argmin(S_left, S_right, S_wait) Here, argmin is the strategy that minimizes the cost function, S_left is the total cost of choosing to detour to the left, S_right is the total cost of choosing to detour to the right, and S_wait is the total cost of choosing to wait in place or slow down. Thus, the direction with the lowest cost among the three is selected as the detour decision, achieving optimal obstacle avoidance while ensuring safety.

[0056] If the detour safety distance is less than the preset safety buffer, it can be determined that there is a collision risk. For blind zone obstacles that are sensed by the MEMS channel but not yet confirmed by the visual channel, the decision-maker adopts a conservative strategy, that is, it reserves a safety buffer in the detour path in advance (by default, the safety distance is extended by 30%) to ensure that the 3ms low latency sensing advantage is not diluted by subsequent processing.

[0057] In the presence of collision risk, the trajectory fusion information is mapped to a dynamic obstacle cost region in a preset two-dimensional grid map, where the cost of the dynamic obstacle cost region is proportional to the collision probability of the target obstacle. Based on the two-dimensional grid map, a preset spatiotemporal algorithm is used to search for an initial detour path from the mobile robot's current position to a preset target point while avoiding the dynamic obstacle cost region. The initial detour path is input into a preset optimizer model to predict the current trajectory, obtaining current trajectory prediction information. The current position of the mobile robot is obtained, and the initial detour path is locally optimized in real-time based on the current trajectory prediction information to obtain a detour path from the mobile robot's current position to avoid the target obstacle. That is, when collision risk exists, the global guidance layer uses an improved spatiotemporal algorithm... The algorithm (Time-Enhanced A) searches for the globally optimal path from the current position to the target point on a 5cm×5cm resolution 2D grid map. It maps the probabilistically reachable region to a dynamic obstacle cost region where the cost is proportional to the collision probability, updating asynchronously at a frequency of 5Hz. This allows the global path to actively avoid high-risk areas. Simultaneously, the local detour layer, based on a Model Predictive Control (MPC) framework, operates at a frequency of 200Hz and is strictly synchronized with the MEMS fusion output. Within N=40 steps (corresponding to 200ms) in the prediction time domain, it minimizes a multi-objective cost function composed of path tracking error, control input smoothness, obstacle collision risk, and kinematic constraint violation penalties. The collision risk term is calculated based on the probabilistically reachable region output from Module 2, using a Signed Distance Field (SDF) for efficient lookup. The MPC solver employs the ACADOS framework based on the interior-point method, achieving a solution latency of no more than 3ms on Jetson Orin NX. The generated local trajectory is a 5th-order polynomial spline (ensuring the continuity of position, velocity, and acceleration), containing a position sequence p(t), a velocity sequence v(t), and an angular velocity sequence ω(t). This ensures the continuity of position, velocity, and acceleration. Therefore, through the collaboration of the global guidance layer and the local detour layer, the uncertainty of trajectory prediction, obstacle category attributes, and robot detour costs are uniformly incorporated into the collision risk assessment, avoiding missed or false detections caused by a single distance threshold. This reduces unnecessary sudden stops and detours while ensuring safety, significantly improving the mobile robot's passage efficiency, path smoothness, and motion safety in densely populated and dynamically complex environments.

[0058] It is understandable that, such as Figure 4 As shown, Figure 4 This is a schematic diagram illustrating the obstacle avoidance navigation method for mobile robots in this application, showing the global guidance layer (spatiotemporal A). The hierarchical collaboration between the path search (5Hz) and the local detour layer (MPC trajectory optimization, 200Hz), as well as the threat assessment and detour direction decision-making logic of the detour strategy decision-maker in multi-obstacle scenarios, are shown in the figure in the form of a two-dimensional planar diagram, which presents the robot's detour path planning results in multiple dynamic obstacle environments.

[0059] In one feasible implementation, step S50 may include steps B11 to B13: Step B11: Based on the trajectory fusion information, determine the probabilistic reachable area of ​​the target obstacle within a preset frequency. It should be noted that the probabilistic reachable region of the target obstacle within the preset frequency is a description of the uncertainty of the future position of the target obstacle. That is, at the preset frequency, the probability distribution of the spatial range in which the target obstacle may appear in the future prediction time domain is modeled and a dynamic envelope region is generated with a set confidence level.

[0060] It is understandable that the probabilistic reachable area of ​​the target obstacle within the preset frequency can dynamically evolve as the prediction time domain progresses, the target motion mode switches, and the sensor measurements are updated. Its area directly represents the level of prediction uncertainty. The greater the uncertainty, the wider the area, and the more safety margin the robot needs to reserve.

[0061] Step B12: Obtain the current motion state of the mobile robot, and calculate the collision index between the mobile robot and the target obstacle based on the probable reachable area and the current motion state. The collision index includes collision time, collision probability, obstacle category priority, and detour space cost. It should be noted that the collision index is a multi-dimensional feature vector used to comprehensively quantify the urgency, probability, severity of consequences, and difficulty of avoidance when a mobile robot collides with a target obstacle.

[0062] Understandably, the time to collision (TTC) represents the time required for the robot to enter the probable reachable area or reach the minimum safe distance while maintaining its current motion trend. The smaller the value, the closer the danger. The collision probability is obtained by integrating the degree of spatiotemporal overlap between the robot's predicted trajectory and the target probable reachable area, representing the likelihood of a collision. The obstacle category priority assigns different risk weights based on the semantic attributes of the target obstacle (such as pedestrians, children, vehicles, bicycles, etc.) to represent the difference in the severity of collision consequences for different obstacle categories. The detour space cost measures the path offset, speed adjustment, and loss of maneuverability required to avoid the probable reachable area from the current state and heading. The higher the cost, the more difficult the avoidance.

[0063] Step B13: Calculate the safe detour distance based on the collision time, the collision probability, the obstacle category priority, and the detour space cost; and determine that there is a collision risk if the safe detour distance is less than a preset safety buffer zone.

[0064] Understandably, the safe detour distance is the minimum dynamic spatial distance threshold that a mobile robot should maintain to safely avoid a target obstacle under the current collision risk level. The preset safety buffer is the minimum spatial margin set to ensure physical safety. It is determined by a combination of factors such as the robot's own size, maximum braking distance, sensor noise level, and control delay. When the dynamically calculated safe detour distance is less than the preset safety buffer, it indicates that the currently planned path or motion state can no longer provide sufficient safety margin, and therefore a collision risk is judged to exist.

[0065] In another feasible implementation, step S50 may further include steps C11 to C14: Step C11: In the event of the collision risk, the trajectory fusion information is mapped to a dynamic obstacle cost region in a preset two-dimensional grid map, wherein the cost of the dynamic obstacle cost region is proportional to the collision probability of the target obstacle. It should be noted that the dynamic obstacle cost region is a set of grids formed by superimposing the probable reachable area of ​​the target obstacle in the future prediction time domain onto a preset two-dimensional grid map (resolution 5cm×5cm) after detecting collision risk. The cost value of each grid is not a simple binary occupancy mark, but is continuously assigned according to the probability that the grid is occupied by the target obstacle. The higher the collision probability, the greater the cost value, and vice versa. In this way, the probability distribution in the trajectory fusion information is transformed into a spatial risk gradient field.

[0066] Step C12: Based on the two-dimensional grid map, use a preset spatiotemporal algorithm to search for an initial detour path from the current position of the mobile robot to the preset target point while avoiding the dynamic obstacle cost area; It should be noted that the aforementioned time-space algorithm is an improved search algorithm that introduces a time dimension and dynamic cost information on the basis of the traditional A algorithm, namely Time-Enhanced A (Time-Space A). This algorithm extends a time axis onto a two-dimensional grid map, transforming the search space from a static (x,y) plane into a three-dimensional spatiotemporal grid of (x,y,t). Each node contains spatial coordinates and the corresponding arrival time. The algorithm also integrates the cost value of dynamic obstacle cost regions at that moment during cost evaluation. During the search process, the algorithm treats the predicted obstacle probability reachable region at the current moment as a dynamic cost layer. As time progresses, the cost region migrates and evolves based on the prediction results of trajectory fusion information, thereby guiding the search path to actively avoid high-risk spatiotemporal points in the time dimension. Furthermore, the preset spatiotemporal algorithm considers robot kinematic constraints (such as maximum speed, acceleration, and minimum turning radius) to limit the transfer cost of adjacent spatiotemporal nodes, ensuring that the searched path is kinematically executable. Thus, starting from the current position and ending at a preset target point, it solves for an initial detour path that starts from the current moment, avoids high-cost grids in spatiotemporal space, and minimizes the total cost. This path is updated asynchronously at a frequency of 5Hz, enabling rapid response to changes in obstacle movement trends in dynamic environments.

[0067] Step C13: Based on the initial detour path input to the preset optimizer model, predict the motion trajectory at the current moment to obtain the current trajectory prediction information; It should be noted that the current trajectory prediction information is a high-resolution trajectory prediction generated by forward simulation of the mobile robot's motion state in the current moment and in the short future time domain, with the initial detour path as a reference clue and a preset optimizer model.

[0068] Step C14: Obtain the current position of the mobile robot, and perform local real-time optimization of the initial detour path based on the current trajectory prediction information to obtain a detour path that avoids the target obstacle from the current position of the mobile robot.

[0069] Understandably, the current position of a mobile robot is its real-time spatial position in the global map coordinate system, obtained through the positioning system. Local real-time optimization is the process of using the current position as the initial condition and the current trajectory prediction information as the warm-start input in each control cycle, and then substituting it into the local detour trajectory optimizer based on model predictive control to solve the problem. This allows the robot to make rapid and smooth motion adjustments to sudden obstacles or prediction errors while smoothly tracking the global path, forming a feasible detour path. This avoids the computational overhead caused by frequent replanning of the global path and ensures real-time obstacle avoidance capability and driving comfort in complex dynamic environments.

[0070] Step S60: Control the mobile robot to travel along the detour path using a preset independent safety channel.

[0071] It is understandable that the preset independent safety channel can be a 1kHz independent safety channel. When the mobile robot is controlled to travel along the detour path using the preset independent safety channel, it is also necessary to dynamically compensate for deviations caused by changes in ground friction, load disturbances, or sensor noise to ensure that the robot always stays within the allowable error range of the detour path.

[0072] In a specific embodiment, the navigation strategy execution and safety monitoring module is responsible for converting the detour path into low-level motion control commands for the robot, and for real-time safety monitoring of the entire navigation process. The navigation strategy execution and safety monitoring module may include a trajectory tracking controller, a real-time collision monitor, and a system health monitor.

[0073] The trajectory tracking controller adopts a passive time-varying feedback linearization controller. The bypass path (position sequence p(t), velocity sequence v(t), and angular velocity sequence ω(t)) generated by MPC in the local optimization layer is used as the desired input. After being solved by the feedforward term (dynamic inverse solution based on the reference trajectory) and the feedback term (PID correction based on the current pose error), the output is the left and right wheel speed command (vL, vR) suitable for differential drive or Ackerman steering robot. At this time, there are two modes: differential drive chassis drive and Ackerman steering chassis drive.

[0074] For differential drive chassis, the feedforward term calculation is expressed as: vL_ff = v_ref ω_ref·L / 2 vR_ff = v_ref + ω_ref·L / 2 Where v_ref is the reference linear velocity, which is the forward velocity that the robot center should have when planning the tracking trajectory; ω_ref is the reference angular velocity, which is the rotational angular velocity that the robot center should have; L is the wheelbase, which is the lateral distance between the left and right wheels; vL_ff and vR_ff are the feedforward velocity commands for the left and right wheels, respectively. This part uses the inverse kinematics relationship of the differential drive model to directly decompose the reference linear velocity and angular velocity into the left and right wheel velocities, which serve as the open-loop reference values ​​under error-free conditions. If the robot's current pose is completely consistent with the reference pose, the reference trajectory motion can be maintained by using only this feedforward term.

[0075] The feedback item calculation is expressed as follows: v_cmd = v_ref·cos(e_θ) + kx·e_x ω_cmd = ω_ref + ky·v_ref·e_y + kθ·e_θ Where e_x, e_y, and e_θ represent the pose error in the robot's body coordinate system, e_x represents the longitudinal distance error along the robot's current forward direction, e_y represents the lateral distance error perpendicular to the forward direction, and e_θ represents the heading angle error. kx, ky, and kθ are positive control gains, which determine the correction strength for the longitudinal, lateral, and heading angle errors, respectively. v_cmd is the corrected desired linear velocity, ω_cmd is the corrected desired angular velocity, and v_ref·cos(e_θ) projects the reference linear velocity onto the robot's actual heading direction to avoid large heading deviations. When the speed command changes abruptly, kx·e_x provides proportional feedback for the longitudinal error, gradually converging the position error in the robot's forward direction. In ω_cmd, ω_ref is the reference angular velocity feedforward, and ky·v_ref·e_y converts the lateral error into angular velocity correction through the reference linear velocity. That is, the greater the lateral deviation and the higher the vehicle speed, the greater the steering correction required. kθ·e_θ provides proportional feedback for the heading error, gradually aligning the robot's orientation with the reference heading. The function of this feedback term is to adjust the robot's speed and turning angle in real time when there is a pose deviation, achieving stable trajectory tracking.

[0076] The final output is represented as: vL = v_cmd ω_cmd·L / 2 vR = v_cmd + ω_cmd·L / 2 Here, vL and vR are the desired speed commands that are finally output to the left and right wheels, respectively. This part decomposes the linear velocity v_cmd and angular velocity ω_cmd after feedback correction into left and right wheel speeds through differential inverse kinematics, thereby converting the continuous speed / angular velocity control quantity into wheel speed commands that can actually be executed by the two-wheel differential chassis.

[0077] For Ackermann steering chassis drives, the feedforward steering angle is calculated and expressed as: δ_ff = arctan(L_wb·ω_ref / v_ref) Where L_wb is the wheelbase, ω_ref is the reference angular velocity, and v_ref is the reference linear velocity.

[0078] The speeds of the left and right wheels are calculated and expressed as follows: vL / vR = v_cmd·(1 tanδ·L / (2·L_wb)) Where L is the track width and δ is the actual front wheel steering angle.

[0079] The calculation of the driving speed v_cmd is the same as that of the differential chassis, which is synthesized from the feedforward speed and the feedback correction. The front wheel steering angle can be obtained from the reference angular velocity through inverse kinematics.

[0080] At this time, the controller performs linear interpolation of the reference trajectory within the MPC update interval, and then performs PID correction with a 1ms cycle to ensure that the robot travels smoothly and accurately along the detour path. The real-time collision monitor runs at a frequency of 1kHz in an independent safety thread, directly reading the raw high-frequency data stream of the MEMS sensor. Combined with the emergency braking algorithm, it continuously assesses the collision risk. Once the collision time TTC is detected to be less than the safety threshold of 200ms, the emergency braking signal is triggered directly without going through the perception fusion and planning modules. The braking response delay is less than 5ms (including the 3ms delay of the MEMS sensor and the 2ms response of the controller). Meanwhile, the system health monitor continuously monitors the data quality of each sensor channel (such as the frame loss rate of the vision channel and the vibration anomaly index of the MEMS channel), the operating status of each algorithm module (processing delay, error count), and the integrity of the communication link in the background. When any sensor channel is detected to be abnormal, it automatically switches to a degraded operation mode (such as pure MEMS blind zone navigation mode) and notifies the operator through the system log and visualization interface to ensure that even if some perception or computing resources fail, the mobile robot can still safely complete the detour or stop smoothly.

[0081] It is understandable that, such as Figure 5 As shown, Figure 5 This is a timing diagram illustrating the emergency braking safety channel of the mobile robot dynamic obstacle avoidance navigation method of this application. The diagram shows the process of the MEMS sensor's 1kHz raw data stream, collision risk assessment, TTC calculation, emergency braking signal generation, and motor driver response. The complete delay chain is 3ms for MEMS sensing + 2ms for controller response = a total of 5ms. A comparison with the timing of a conventional obstacle avoidance channel highlights the rapid response advantage of the MEMS independent safety channel.

[0082] This embodiment proposes a dynamic obstacle avoidance navigation method for mobile robots. This application acquires inertial and visual sensor data from the mobile robot to determine the target obstacles and motion state vectors around the robot. These data are then input into a first predictor of a physical model and a second predictor of a machine learning model for trajectory prediction. Based on the output trajectory prediction results, the respective fusion weights are dynamically determined to obtain trajectory fusion information. Collision risks are detected at a preset frequency based on the trajectory fusion information. If a risk exists, an avoidance path is planned and the robot is controlled to travel along the path. This method combines the generalization ability of the physical model with the data fitting ability of the machine learning model. Furthermore, the fusion weights can be adaptively adjusted based on real-time trajectory prediction results, significantly improving the accuracy of dynamic obstacle trajectory prediction, effectively reducing the probability of missed and false detections, and enhancing the real-time performance and reliability of obstacle avoidance decisions. This ensures that the mobile robot can safely and efficiently avoid obstacles in complex dynamic environments.

[0083] It is understandable that, such as Figure 6 As shown, Figure 6 This diagram illustrates the overall architecture of the mobile robot dynamic obstacle avoidance navigation method of this application. The overall architecture of the mobile robot dynamic obstacle avoidance navigation strategy software system includes the data flow relationship between four core technology modules (MEMS-vision multimodal heterogeneous perception fusion module, dynamic obstacle trajectory prediction module, real-time obstacle avoidance path planning module, and navigation strategy execution and safety monitoring module), the input and output interfaces of each module, and the connection relationship with the robot hardware layer (MEMS sensor array, RGB-D camera, motor driver). In the diagram, different line types are used to distinguish high-frequency data flow (1kHz MEMS channel, solid line), mid-frequency data flow (200Hz fusion output, dashed line), and low-frequency data flow (5Hz global planning, dotted line).

[0084] Based on the first embodiment of this application, in the second embodiment of this application, the same or similar content as the first embodiment can be referred to the above description, and will not be repeated hereafter.

[0085] In this embodiment, refer to Figure 7 , Figure 7 This is a flowchart illustrating Embodiment 2 of the mobile robot dynamic obstacle avoidance navigation method of this application. Step S30 specifically includes steps S31 to S33: Step S31: Based on the motion state vector, input the first predictor of the preset physical motion model to predict the motion trajectory and obtain the first trajectory prediction result in the trajectory prediction result. The first predictor contains multiple motion models, and the prediction outputs of the multiple motion models are weighted and fused through an interactive multi-model method to obtain the first trajectory prediction result. It should be noted that the first trajectory prediction result is the optimal comprehensive estimate of the target obstacle's trajectory over a future period of time. It can output the target state vector (including position, velocity, acceleration, and heading angle, etc.) at a series of discrete time points in the prediction time domain with a fixed sampling period. At the same time, it outputs the matching probability and fusion covariance matrix of each model to quantify the uncertainty of trajectory prediction and the confidence of model switching.

[0086] Understandably, a motion model is a mathematical expression representing the state evolution of an obstacle under a specific motion pattern. The interactive multi-model approach, on the other hand, first inputs the state estimates of each model based on the model probability of the previous moment into each sampling period, and then executes the prediction and update of each model in parallel. By calculating and updating the weights of each model, the state estimates of each model are summed according to their weights, so that the first trajectory prediction result can adaptively follow the dynamic switching of the obstacle's motion pattern, taking into account both the smoothness of steady-state prediction and the response speed of maneuvering changes, and significantly improving the accuracy of trajectory prediction in complex dynamic environments.

[0087] In a specific embodiment, the motion state vector is predicted using the uniform motion model, the uniform acceleration motion model, and the constant velocity motion model to obtain the predicted output of each motion model; the model probability of each predicted output is determined according to the interactive multi-model method represented by the preset Markov transition probability matrix; the predicted outputs are weighted and fused according to the model probabilities to obtain the first trajectory prediction result. That is, when the motion state vector is input into the first predictor, the first predictor contains an interactive multi-model (IMM) filter of a constant acceleration model (CA model) and a constant rotational speed and velocity model (CTRV model). The IMM filter maintains three sub-models, namely, a uniform linear motion model (CV), a uniform acceleration linear motion model (CTRV), a constant acceleration linear motion model (CMV ... Multiple physical motion models, such as the linear motion model (CA) and the constant velocity motion model (CTRV), are used to extrapolate the trajectory in parallel. The model switching is achieved through the Markov transition probability matrix. The example values ​​of the IMM Markov transition probability matrix (e.g., 0.95 on the main diagonal and switching probabilities of 0.025 / 0.025) are used to adaptively adjust the fusion weights of the model outputs based on the posterior probabilities of each model. The output is a trajectory prediction value with model probability weighting, which is the first trajectory prediction result that takes into account both short-term linear motion and sudden maneuvering scenarios. The first predictor of the physical motion model is suitable for scenarios where obstacles maintain their current motion mode (e.g., pedestrians walking at a constant speed along a straight line). The prediction update cycle is 5ms (synchronized with the MEMS fusion output).

[0088] In one feasible implementation, the first predictor of the preset physical motion model includes a uniform motion model, a uniformly accelerated motion model, and a constant velocity motion model. It is understandable that the constant velocity model (CV) assumes that the target moves in uniform linear motion, the constant acceleration model (CA) assumes that the acceleration is constant, and the cooperative turning model (CT) assumes that the angular velocity and velocity maintain a specific relationship. Each model is defined by a state transition equation and a corresponding process noise covariance matrix, which are used to perform prior deduction of the target's state at the next moment.

[0089] Step S31 may include steps D11 to D13: Step D11: Predict the motion state vector using the uniform motion model, the uniform acceleration motion model, and the constant velocity motion model to obtain the prediction output of each motion model; It should be noted that the prediction outputs of each of the aforementioned motion models are state estimation sequences for several future moments obtained by extrapolating the state state vector of the current obstacle (such as position, velocity, acceleration, heading angle, etc.) as initial values ​​under their respective motion assumptions through state transition equations, based on the uniform motion model, uniform acceleration model, and constant velocity motion model. The uniform motion model provides a predicted trajectory with constant velocity and linear position change over time; the uniform acceleration model provides a predicted trajectory with constant acceleration, linear velocity change, and quadratic position change; and the constant velocity model provides a circular arc-shaped predicted trajectory with constant linear velocity and turning rate. The prediction output of each model not only includes the position, velocity, acceleration, and heading angle sequence in the prediction time domain, but also includes a covariance matrix quantized from process noise and model error to characterize the uncertainty of the model's prediction results. Since each model describes a different motion mode, these prediction outputs are essentially multiple candidate hypotheses of the obstacle's possible motion modes.

[0090] Step D12: Determine the model probability of each predicted output according to the interactive multi-model method represented by the preset Markov transition probability matrix; It should be noted that the model probabilities of the predicted outputs are obtained at each sampling time by interacting and predicting the probabilities of each model at the previous time step through a preset Markov transition probability matrix, calculating the likelihood function corresponding to each model using the current sensor observation data, and then performing Bayesian normalization to obtain the posterior probability. This probability can characterize the degree of matching between the actual motion of the obstacle and the assumptions of each motion model. The higher the probability, the more reasonably the model can explain the current and recent observation data, and the higher the credibility of its predicted output. The Markov transition probability matrix describes the statistical law of the switching between motion models, such as the probability of the obstacle changing from uniform motion to uniform acceleration or constant rotation rate motion. This allows the interactive multi-model algorithm to dynamically adjust the weights of each model and quickly transfer the probability to a more matching model when the motion mode changes abruptly, thereby ensuring the adaptive tracking capability of the obstacle's maneuvering state.

[0091] Step D13: Weight and fuse the prediction outputs according to the probabilities of each model to obtain the first trajectory prediction result.

[0092] It is understandable that the essence of weighted fusion is to make soft decisions among multiple motion hypotheses. The model with the higher probability contributes more to the final result, while the model with the lower probability still participates in the fusion with a lower weight, so as to follow the current dominant motion mode of the obstacle and maintain a smooth trajectory during the mode switching process, avoiding jumps or delays caused by hard switching between models.

[0093] Step S32: Based on the target obstacle and the motion state vector, input the second predictor of the preset machine learning model to predict the motion trajectory and obtain the second trajectory prediction result in the trajectory prediction result. The second predictor predicts the multimodal trajectory distribution of the target obstacle based on the trajectory sequence of the target obstacle in the preset historical window and the environmental context features to obtain the second trajectory prediction result. It should be noted that the second trajectory prediction result is a multimodal probability distribution of the future motion trajectory of the target obstacle, which can be composed of multiple candidate trajectories and their corresponding probabilities, such as a trajectory set or probability distribution parameters, used to characterize the multiple possibilities and uncertainties of the future behavior of the target obstacle.

[0094] Understandably, the preset history window is the length of the time interval used to extract the input sequence from the historical state of the target obstacle. For example, it is the time span covered by the position, speed, heading, and other states recorded at a uniform sampling period within the past 3 or 5 seconds. The trajectory sequence is a set of state vectors of the target obstacle arranged in chronological order within the preset history window, including the position, speed, acceleration, heading angle, etc. at each sampling moment. It is used to characterize the obstacle's own motion dynamics and recent behavior patterns. The environmental context features are the surrounding scene information closely related to the movement of the target obstacle, such as road topology, lane markings, traffic light status, the position and speed of other traffic participants, road surface conditions, and obstacle category attributes. The multimodal trajectory distribution of the target obstacle is a multi-hypothetical probability description of the model's output for the future movement of the target obstacle. That is, it simultaneously gives several physically meaningful and mutually distinct motion trajectories (such as going straight, turning left, turning right, changing lanes, decelerating and stopping, etc.) and assigns a probability value to each trajectory. These trajectories and probabilities together constitute a multi-peak distribution to express the uncertainty and diversity of the obstacle's future behavior.

[0095] In a specific embodiment, the real-time detour path planning module inputs the trajectory sequence of the target obstacle within a preset historical window, along with environmental context features (such as the relative position of the obstacle to the passable area and the spatial relationship between the obstacle and the target point), into the second predictor of the machine learning model. Through an encoder-decoder structure, it outputs multiple candidate trajectories and their probability distributions within the future time window, forming a multimodal second trajectory prediction result. For example, it utilizes a lightweight spatiotemporal graph neural network (ST-GNN) intent-aware trajectory prediction model. The lightweight spatiotemporal graph neural network structure consists of two spatial graph convolutional layers, a temporal GRU layer, and a GMM parameter output header (each mixed component output...). The model consists of a trajectory mean sequence, a covariance matrix, and mixing weights. The loss function is a Gaussian mixture negative log-likelihood. It takes the obstacle trajectory sequence of the past 50 frames (corresponding to a 250ms history window) as input, combines environmental context features (such as the relative position of obstacles and passable areas, and the spatial relationship between obstacles and target points), and outputs the multimodal trajectory distribution of the next 40 frames (corresponding to a 2s prediction window) (represented by a two-dimensional Gaussian mixture model, K=5 mixing components, each component containing a trajectory mean sequence and a covariance matrix) to obtain the second trajectory prediction result. At this time, the model has about 0.8M parameters and an inference latency of about 4ms on an embedded GPU (such as NVIDIA Jetson Orin NX).

[0096] Step S33: Obtain a time window, and determine the fusion weight of the first predictor corresponding to the first trajectory prediction result and the fusion weight of the second predictor corresponding to the second trajectory prediction result based on the time window.

[0097] It should be noted that the time window is a fusion decision variable used to weigh the reliability of the outputs of the first predictor and the second predictor. That is, it is the length of the future prediction time domain that the current prediction task is concerned with or the current prediction time scale. When the time window is short, the motion state of the target obstacle is strongly constrained by physical dynamics. The first predictor has high accuracy and fast response in a short time based on the extrapolation of the motion model, so its fusion weight increases accordingly. When the time window is long, the behavior of the target obstacle is more affected by driving intention, scene semantics and interaction rules. The second predictor can better capture the multimodal trajectory distribution and long-term motion trend, so its fusion weight gradually increases.

[0098] In a specific embodiment, the time window is compared with a preset time threshold to obtain a comparison result. If the comparison result indicates that the time window is less than or equal to the preset time threshold, the fusion weight corresponding to the first trajectory prediction result is greater than the fusion weight corresponding to the second trajectory prediction result. If the comparison result indicates that the time window is greater than the preset time threshold, the fusion weight corresponding to the second trajectory prediction result is greater than the fusion weight corresponding to the first trajectory prediction result. That is, a preset time window (e.g., a prediction time domain length of 3 seconds) is obtained, and the fusion weight is dynamically determined based on the position of the prediction time point within this time window. Because the first predictor has high short-term position accuracy, low computational latency, and interpretability, it can achieve high accuracy in time... In the first half of the window (e.g., 0-1 seconds), the fusion weight of the first predictor is increased. Since the second predictor can effectively suppress the cumulative error caused by long-term extrapolation by utilizing the learned interaction rules and scene semantics, and output the multimodal uncertainty distribution, the fusion weight of the second predictor can be increased in the later half of the time window (e.g., 1-3 seconds). For example, in short-term prediction (<500ms), the physical model predictor is the main one (weight 0.7-0.85), making full use of the high-speed update characteristics of MEMS sensors to ensure the accuracy and real-time performance of short-term prediction. In medium- and long-term prediction (>500ms), the weight of the learning predictor is gradually increased (to 0.5-0.65), using its semantic understanding of motion intentions to improve the rationality of medium- and long-term trajectory prediction.

[0099] In one feasible implementation, step S33 may include steps E11 to E13: Step E11: Compare the time window with a preset time threshold to obtain the comparison result; It should be noted that the comparison result is a qualitative or quantitative judgment result obtained by comparing the time window with the preset time threshold, that is, judging whether the future time scale of the current prediction task falls within the short-term prediction interval or the long-term prediction interval with the preset time threshold as the boundary.

[0100] Understandably, the preset time threshold is a pre-defined time scale boundary value used to distinguish between applicable scenarios for short-term and long-term predictions. The size of this threshold can be determined based on actual application requirements, sensor update frequency, scene dynamics, and the performance characteristics of the two types of predictors. For example, in autonomous driving scenarios, it may be set to 1 second, 2 seconds, or 3 seconds. When the window is shorter than this threshold, the extrapolation accuracy of the physical motion model is higher and the computational latency is lower. When the time window is longer than this threshold, the influence of obstacle behavior intent and scene interaction gradually becomes dominant, and the predictive advantage of the machine learning model becomes more obvious.

[0101] Step E12: When the comparison result is that the time window is less than or equal to a preset time threshold, the fusion weight corresponding to the first trajectory prediction result is greater than the fusion weight corresponding to the second trajectory prediction result. Understandably, when the time window is less than or equal to the preset time threshold, it indicates that the current prediction target is within the short-term prediction range. At this time, the motion state of the target obstacle is strongly constrained by its own dynamics and motion inertia. Physical quantities such as speed, acceleration, and turning rate will not undergo drastic changes in the short term. The uniform motion model, uniform acceleration motion model, and constant speed motion model relied upon by the first predictor can extrapolate the short-term trajectory relatively accurately. At the same time, the prediction results based on the physical model have clear dynamic interpretability and low sensitivity to observation noise. Therefore, they have high credibility in the short term. Although the second predictor can capture multimodal behavioral intentions, its advantages require a long historical context and scene feature accumulation to be fully realized. In short-term prediction, its uncertainty description may be too broad, and the error introduced by model inference may not be smaller than that of physical extrapolation. Therefore, its contribution is relatively limited when the time window is short. Therefore, the fusion weight is allocated to give a larger weight to the first trajectory prediction result, so that it occupies a dominant position in the final fused trajectory to ensure the accuracy and stability of short-term trajectory prediction.

[0102] Step E13: If the comparison result is that the time window is greater than the preset time threshold, the fusion weight corresponding to the second trajectory prediction result is greater than the fusion weight corresponding to the first trajectory prediction result.

[0103] Understandably, when the time window exceeds a preset time threshold, it indicates that the current prediction target is within the long-term prediction range. At this point, as the prediction time domain extends, the movement of the target obstacle is no longer solely constrained by its current physical state, but depends more on its driving intentions, understanding of traffic rules, reactions to changes in the surrounding environment, and interactions with other traffic participants. The first predictor, based on the extrapolation of the motion model, will accumulate dynamic biases over a long time scale. For example, it cannot predict when the obstacle will turn, decelerate, or change its motion mode, leading to a gradual increase in prediction error and a significant decrease in the reliability of its long-term prediction. In contrast, the second predictor, through historical trajectory sequences and environmental context features, can learn the behavior patterns of the target obstacle in similar scenarios and output a multimodal trajectory distribution containing multiple possible behaviors (such as going straight, changing lanes, turning, stopping, etc.), better covering the uncertainties in long-term prediction. Therefore, it has higher prediction confidence and scene adaptability within a long time window. Consequently, the fusion weight allocation gives a larger weight to the second trajectory prediction result, making the machine learning prediction result dominate in long-term trajectory fusion, thereby improving the modeling ability for complex scenarios.

[0104] This embodiment proposes a dynamic obstacle avoidance navigation method for mobile robots. This application achieves accurate prediction of the trajectory of the target obstacle through a dual predictor fusion framework of physical model and machine learning model, and dynamically determines the fusion weight of the two predictors according to the time window to achieve complementarity between short-term and medium-to-long-term predictions, thereby taking into account scene semantics and multimodal randomness and improving the accuracy of trajectory prediction.

[0105] This application also provides a dynamic obstacle avoidance navigation device for a mobile robot. Please refer to [link / reference]. Figure 8 The mobile robot dynamic obstacle avoidance navigation device includes: The acquisition module 10 is used to acquire inertial sensing data and visual sensing data of the mobile robot; The determining module 20 is used to determine the target obstacles around the mobile robot and the motion state vector of the target obstacles based on the inertial sensing data and the visual sensing data. The prediction module 30 is used to predict the motion trajectory based on the target obstacle and the motion state vector, by inputting a first predictor of a preset physical motion model and a second predictor of a preset machine learning model, to obtain a trajectory prediction result, and to determine the fusion weight of the first predictor and the fusion weight of the second predictor based on the trajectory prediction result. The fusion module 40 is used to perform fusion processing on the motion state vector according to the fusion weight of the first predictor and the fusion weight of the second predictor to obtain trajectory fusion information; The planning module 50 is used to detect the collision risk between the mobile robot and the target obstacle at a preset frequency based on the trajectory fusion information, and to determine a detour path to avoid the target obstacle based on the trajectory fusion information if the collision risk exists. The execution module 60 is used to control the mobile robot to travel along the detour path through a preset independent safety channel.

[0106] The determining module 20 is further configured to determine the pose information of the mobile robot based on the inertial sensing data; The target obstacles around the mobile robot and their location information are determined based on the visual sensing data. Based on the pose information, the coordinate system of the position information is transformed to the corresponding global coordinate system to obtain the global position information; The motion state vector of the target obstacle is determined by fusing the global position information with the motion increment of the inertial sensing data. The motion state vector includes the position, velocity and / or acceleration of the target obstacle in the global coordinate system.

[0107] The prediction module 30 is further configured to predict the motion trajectory based on the motion state vector and input a first predictor of a preset physical motion model to obtain a first trajectory prediction result in the trajectory prediction result. The first predictor contains multiple motion models and performs weighted fusion of the prediction outputs of the multiple motion models through an interactive multi-model approach to obtain the first trajectory prediction result. Based on the target obstacle and the motion state vector, a second predictor of a preset machine learning model is input to predict the motion trajectory, thereby obtaining a second trajectory prediction result in the trajectory prediction results. The second predictor predicts the multimodal trajectory distribution of the target obstacle based on the trajectory sequence of the target obstacle within a preset historical window and environmental context features, thereby obtaining the second trajectory prediction result. Obtain a time window, and determine the fusion weight of the first predictor corresponding to the first trajectory prediction result and the fusion weight of the second predictor corresponding to the second trajectory prediction result based on the time window.

[0108] The prediction module 30 is further configured to predict the motion state vector using the uniform motion model, the uniformly accelerated motion model, and the constant velocity motion model, and obtain the prediction output of each of the motion models; The model probability of each predicted output is determined based on the interactive multi-model approach represented by the preset Markov transition probability matrix. The predicted outputs are weighted and fused according to the probabilities of each model to obtain the first trajectory prediction result.

[0109] The prediction module 30 is also used to compare the time window with a preset time threshold to obtain a comparison result; When the comparison result is that the time window is less than or equal to a preset time threshold, the fusion weight corresponding to the first trajectory prediction result is greater than the fusion weight corresponding to the second trajectory prediction result. If the comparison result is that the time window is greater than the preset time threshold, the fusion weight corresponding to the second trajectory prediction result is greater than the fusion weight corresponding to the first trajectory prediction result.

[0110] The planning module 50 is further configured to determine the probabilistic reachable area of ​​the target obstacle within a preset frequency based on the trajectory fusion information; The current motion state of the mobile robot is obtained, and a collision index between the mobile robot and the target obstacle is calculated based on the probable reachable area and the current motion state. The collision index includes collision time, collision probability, obstacle category priority, and detour space cost. The detour safety distance is calculated based on the collision time, the collision probability, the obstacle category priority, and the detour space cost. If the detour safety distance is less than the preset safety buffer zone, it is determined that there is a collision risk.

[0111] The planning module 50 is further configured to map the trajectory fusion information into a dynamic obstacle cost region in a preset two-dimensional grid map when the collision risk exists, wherein the cost of the dynamic obstacle cost region is proportional to the collision probability of the target obstacle. Based on the two-dimensional grid map, a preset spatiotemporal algorithm is used to search for an initial detour path from the current position of the mobile robot to the preset target point while avoiding the dynamic obstacle cost area; Based on the initial detour path input to the preset optimizer model, the motion trajectory at the current moment is predicted to obtain the current trajectory prediction information; The current position of the mobile robot is obtained, and the initial detour path is locally optimized in real time based on the current trajectory prediction information to obtain a detour path that avoids the target obstacle from the current position of the mobile robot.

[0112] The mobile robot dynamic obstacle avoidance navigation device provided in this application, employing the mobile robot dynamic obstacle avoidance navigation method in the above embodiments, can solve the technical problem of how to quickly achieve obstacle avoidance navigation. Compared with the prior art, the beneficial effects of the mobile robot dynamic obstacle avoidance navigation device provided in this application are the same as those of the mobile robot dynamic obstacle avoidance navigation method provided in the above embodiments, and other technical features in the mobile robot dynamic obstacle avoidance navigation device are the same as those disclosed in the methods of the above embodiments, and will not be repeated here.

[0113] This application provides a mobile robot dynamic obstacle avoidance navigation device, which includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the mobile robot dynamic obstacle avoidance navigation method in Embodiment 1 above.

[0114] The following is for reference. Figure 9This document illustrates a structural schematic diagram of a mobile robot dynamic obstacle avoidance navigation device suitable for implementing embodiments of this application. The mobile robot dynamic obstacle avoidance navigation device in the embodiments of this application may include, but is not limited to, mobile terminals such as mobile phones, laptops, digital broadcast receivers, PDAs (Personal Digital Assistants), PMPs (Portable Media Players), and in-vehicle terminals (e.g., in-vehicle navigation terminals), as well as fixed terminals such as digital TVs and desktop computers. Figure 9 The illustrated mobile robot dynamic obstacle avoidance navigation device is merely an example and should not impose any limitations on the functionality and scope of use of the embodiments of this application.

[0115] like Figure 9 As shown, the mobile robot dynamic obstacle avoidance navigation device may include a processing unit 1001 (e.g., a central processing unit, a graphics processing unit, etc.), which can perform various appropriate actions and processes according to a program stored in ROM (Read Only Memory) 1002 or a program loaded from storage device 1003 into RAM (Random Access Memory) 1004. RAM 1004 also stores various programs and data required for the operation of the mobile robot dynamic obstacle avoidance navigation device. The processing unit 1001, ROM 1002, and RAM 1004 are interconnected via bus 1005. Input / output (I / O) interface 1006 is also connected to the bus. Typically, the following systems can be connected to I / O interface 1006: input devices 1007 including, for example, touchscreens, touchpads, keyboards, mice, image sensors, microphones, accelerometers, gyroscopes, etc.; output devices 1008 including, for example, liquid crystal displays (LCDs), speakers, vibrators, etc.; storage devices 1003 including, for example, magnetic tapes, hard disks, etc.; and communication devices 1009. Communication device 1009 allows the mobile robot dynamic obstacle avoidance navigation device to exchange data wirelessly or via wired communication with other devices. Although the figure shows a mobile robot dynamic obstacle avoidance navigation device with various systems, it should be understood that it is not required to implement or possess all the systems shown. More or fewer systems can be implemented alternatively.

[0116] Specifically, according to the embodiments disclosed in this application, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments disclosed in this application include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via a communication device, or installed from storage device 1003, or installed from ROM 1002. When the computer program is executed by processing device 1001, it performs the functions defined in the methods of the embodiments disclosed in this application.

[0117] The mobile robot dynamic obstacle avoidance navigation device provided in this application, employing the mobile robot dynamic obstacle avoidance navigation method in the above embodiments, can solve the technical problem of how to quickly achieve obstacle avoidance navigation. Compared with the prior art, the beneficial effects of the mobile robot dynamic obstacle avoidance navigation device provided in this application are the same as those of the mobile robot dynamic obstacle avoidance navigation method provided in the above embodiments, and other technical features in this mobile robot dynamic obstacle avoidance navigation device are the same as those disclosed in the previous embodiment method, and will not be repeated here.

[0118] It should be understood that the various parts disclosed in this application can be implemented using hardware, software, firmware, or a combination thereof. In the description of the above embodiments, specific features, structures, materials, or characteristics can be combined in any suitable manner in one or more embodiments or examples.

[0119] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

[0120] This application provides a computer-readable storage medium having computer-readable program instructions (i.e., a computer program) stored thereon, which are used to execute the mobile robot dynamic obstacle avoidance navigation method in the above embodiments.

[0121] The computer-readable storage medium provided in this application may be, for example, a USB flash drive, but is not limited to, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices, or any combination thereof. More specific examples of computer-readable storage media may include, but are not limited to: electrical connections having one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. In this embodiment, the computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, system, or device. The program code contained on the computer-readable storage medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (Radio Frequency), etc., or any suitable combination thereof.

[0122] The aforementioned computer-readable storage medium may be included in the mobile robot dynamic obstacle avoidance navigation device; or it may exist independently and not assembled into the mobile robot dynamic obstacle avoidance navigation device.

[0123] The aforementioned computer-readable storage medium carries one or more programs. When these programs are executed by the mobile robot dynamic obstacle avoidance navigation device, the mobile robot dynamic obstacle avoidance navigation device performs the following actions: acquires inertial sensing data and visual sensing data of the mobile robot; determines target obstacles around the mobile robot and their motion state vectors based on the inertial sensing data and the visual sensing data; inputs a first predictor of a preset physical motion model and a second predictor of a preset machine learning model to predict the motion trajectory based on the target obstacles and the motion state vectors, obtains a trajectory prediction result, and determines the fusion weights of the first predictor and the second predictor based on the trajectory prediction result; performs fusion processing on the motion state vectors based on the fusion weights of the first predictor and the second predictor to obtain trajectory fusion information; detects the collision risk between the mobile robot and the target obstacle at a preset frequency based on the trajectory fusion information, and determines a detour path to avoid the target obstacle based on the trajectory fusion information if the collision risk exists; and controls the mobile robot to travel along the detour path using a preset independent safety channel.

[0124] Computer program code for performing the operations of this application can be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, and C++, and conventional procedural programming languages ​​such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a Local Area Network (LAN) or a Wide Area Network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0125] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0126] The modules described in the embodiments of this application can be implemented in software or hardware. The names of the modules do not necessarily limit the functionality of the unit itself.

[0127] The readable storage medium provided in this application is a computer-readable storage medium that stores computer-readable program instructions (i.e., a computer program) for executing the above-described mobile robot dynamic obstacle avoidance navigation method, thereby solving the technical problem of how to quickly achieve obstacle avoidance navigation. Compared with the prior art, the beneficial effects of the computer-readable storage medium provided in this application are the same as those of the mobile robot dynamic obstacle avoidance navigation method provided in the above embodiments, and will not be repeated here.

[0128] The above description is only a part of the embodiments of this application and does not limit the patent scope of this application. All equivalent structural transformations made under the technical concept of this application and using the contents of the specification and drawings of this application, or direct / indirect applications in other related technical fields, are included in the patent protection scope of this application.

Claims

1. A method for dynamic obstacle avoidance navigation of a mobile robot, characterized in that, The method includes: Acquire inertial and visual sensor data from the mobile robot; Based on the inertial sensing data and the visual sensing data, the target obstacles around the mobile robot and the motion state vectors of the target obstacles are determined. Based on the target obstacle and the motion state vector, a first predictor of a preset physical motion model and a second predictor of a preset machine learning model are input to predict the motion trajectory, obtain the trajectory prediction result, and determine the fusion weight of the first predictor and the fusion weight of the second predictor based on the trajectory prediction result. Based on the fusion weights of the first predictor and the fusion weights of the second predictor, the motion state vector is fused to obtain trajectory fusion information. Based on the trajectory fusion information, the collision risk between the mobile robot and the target obstacle is detected at a preset frequency, and if the collision risk exists, a detour path to avoid the target obstacle is determined based on the trajectory fusion information. The mobile robot is controlled to travel along the detour path using a preset independent safety passage.

2. The method as described in claim 1, characterized in that, The step of determining the target obstacles around the mobile robot and the motion state vectors of the target obstacles based on the inertial sensing data and the visual sensing data includes: The pose information of the mobile robot is determined based on the inertial sensing data; The target obstacles around the mobile robot and their location information are determined based on the visual sensing data. Based on the pose information, the coordinate system of the position information is transformed to the corresponding global coordinate system to obtain the global position information; The motion state vector of the target obstacle is determined by fusing the global position information with the motion increment of the inertial sensing data. The motion state vector includes the position, velocity and / or acceleration of the target obstacle in the global coordinate system.

3. The method as described in claim 1, characterized in that, The steps of predicting the motion trajectory by inputting a first predictor of a preset physical motion model and a second predictor of a preset machine learning model based on the target obstacle and the motion state vector, obtaining a trajectory prediction result, and determining the fusion weights of the first predictor and the second predictor based on the trajectory prediction result include: Based on the motion state vector, the first predictor of the preset physical motion model is input to predict the motion trajectory, and the first trajectory prediction result is obtained from the trajectory prediction result. The first predictor contains multiple motion models, and the prediction outputs of the multiple motion models are weighted and fused through an interactive multi-model method to obtain the first trajectory prediction result. Based on the target obstacle and the motion state vector, a second predictor of a preset machine learning model is input to predict the motion trajectory, thereby obtaining a second trajectory prediction result in the trajectory prediction results. The second predictor predicts the multimodal trajectory distribution of the target obstacle based on the trajectory sequence of the target obstacle within a preset historical window and environmental context features, thereby obtaining the second trajectory prediction result. Obtain a time window, and determine the fusion weight of the first predictor corresponding to the first trajectory prediction result and the fusion weight of the second predictor corresponding to the second trajectory prediction result based on the time window.

4. The method as described in claim 3, characterized in that, The first predictor of the preset physical motion model includes a uniform motion model, a uniformly accelerated motion model, and a constant velocity motion model. The step of weightedly fusing the prediction outputs of the multiple motion models through an interactive multi-model approach to obtain the first trajectory prediction result includes: The motion state vector is predicted by the uniform motion model, the uniformly accelerated motion model and the constant velocity motion model to obtain the prediction output of each motion model; The model probability of each predicted output is determined based on the interactive multi-model approach represented by the preset Markov transition probability matrix. The predicted outputs are weighted and fused according to the probabilities of each model to obtain the first trajectory prediction result.

5. The method as described in claim 3, characterized in that, The step of determining the fusion weights of the first predictor corresponding to the first trajectory prediction result and the fusion weights of the second predictor corresponding to the second trajectory prediction result based on the time window includes: The time window is compared with a preset time threshold to obtain the comparison result; When the comparison result is that the time window is less than or equal to a preset time threshold, the fusion weight corresponding to the first trajectory prediction result is greater than the fusion weight corresponding to the second trajectory prediction result. If the comparison result is that the time window is greater than the preset time threshold, the fusion weight corresponding to the second trajectory prediction result is greater than the fusion weight corresponding to the first trajectory prediction result.

6. The method as described in claim 1, characterized in that, The step of detecting the collision risk between the mobile robot and the target obstacle at a preset frequency based on the trajectory fusion information includes: Based on the trajectory fusion information, the probabilistic reachable area of ​​the target obstacle within a preset frequency is determined; The current motion state of the mobile robot is obtained, and a collision index between the mobile robot and the target obstacle is calculated based on the probable reachable area and the current motion state. The collision index includes collision time, collision probability, obstacle category priority, and detour space cost. The detour safety distance is calculated based on the collision time, the collision probability, the obstacle category priority, and the detour space cost. If the detour safety distance is less than the preset safety buffer zone, it is determined that there is a collision risk.

7. The method as described in claim 1, characterized in that, The step of determining a detour path to avoid the target obstacle based on the trajectory fusion information when the collision risk exists includes: In the event of the collision risk, the trajectory fusion information is mapped to a dynamic obstacle cost region in a preset two-dimensional grid map, wherein the cost of the dynamic obstacle cost region is proportional to the collision probability of the target obstacle. Based on the two-dimensional grid map, a preset spatiotemporal algorithm is used to search for an initial detour path from the current position of the mobile robot to the preset target point while avoiding the dynamic obstacle cost area; Based on the initial detour path input to the preset optimizer model, the motion trajectory at the current moment is predicted to obtain the current trajectory prediction information; The current position of the mobile robot is obtained, and the initial detour path is locally optimized in real time based on the current trajectory prediction information to obtain a detour path that avoids the target obstacle from the current position of the mobile robot.

8. A dynamic obstacle avoidance navigation device for a mobile robot, characterized in that, The device includes: The acquisition module is used to acquire inertial and visual sensor data of the mobile robot. The determination module is used to determine the target obstacles around the mobile robot and the motion state vector of the target obstacles based on the inertial sensing data and the visual sensing data. The prediction module is used to predict the motion trajectory based on the target obstacle and the motion state vector, by inputting a first predictor of a preset physical motion model and a second predictor of a preset machine learning model, to obtain the trajectory prediction result, and to determine the fusion weight of the first predictor and the fusion weight of the second predictor based on the trajectory prediction result. The fusion module is used to fuse the motion state vector according to the fusion weights of the first predictor and the fusion weights of the second predictor to obtain trajectory fusion information; The planning module is used to detect the collision risk between the mobile robot and the target obstacle at a preset frequency based on the trajectory fusion information, and determine a detour path to avoid the target obstacle based on the trajectory fusion information if the collision risk exists. An execution module is used to control the mobile robot to travel along the bypass path via a preset independent safety channel.

9. A dynamic obstacle avoidance navigation device for a mobile robot, characterized in that, The device includes: a memory, a processor, and a computer program stored in the memory and executable on the processor, the computer program being configured to implement the steps of the mobile robot dynamic obstacle avoidance navigation method as described in any one of claims 1 to 7.

10. A storage medium, characterized in that, The storage medium is a computer-readable storage medium, and a computer program is stored on the storage medium. When the computer program is executed by a processor, it implements the steps of the mobile robot dynamic obstacle avoidance navigation method as described in any one of claims 1 to 7.