A dynamic obstacle intention prediction method for robot autonomous navigation and a system thereof

Abstract

This invention discloses a method and system for predicting the intention of dynamic obstacles in robot autonomous navigation. It collects multimodal perception data of dynamic obstacles in a home setting in real time. These obstacles include people or pets. The multimodal perception data is acquired using Video SLAM (Visual Simultaneous Localization and Mapping) combined with video AI intelligent analysis technology, specifically including motion trajectory, instantaneous velocity, posture features, and relative position to the robot. Based on the multimodal perception data, a pre-trained intention prediction model extracts the motion pattern features of the dynamic obstacles and outputs their short-term behavioral intention categories. The smooth obstacle avoidance path avoids sudden braking or abrupt detours. The robot is controlled to navigate along the smooth obstacle avoidance path, achieving smooth and safe autonomous navigation in a scenario where people, pets, and robots coexist.

Description

Technical Field

[0001] This invention relates to the field of autonomous navigation technology for home robots, and in particular to a dynamic obstacle intention prediction method and system for autonomous robot navigation based on Video SLAM and video AI intelligent analysis. Background Technology

[0002] With the rapid development of smart home technology, home service robots, such as robotic vacuum cleaners and companion robots, have been widely used in daily life. Autonomous navigation is one of the core functions for home robots to achieve autonomous operation. However, the home environment contains numerous dynamic obstacles, such as walking people and running pets. The unpredictable movement trajectories and behaviors of these obstacles pose a significant challenge to the safety and smoothness of the robot's autonomous navigation.

[0003] Most existing autonomous navigation and obstacle avoidance technologies for home robots employ passive obstacle avoidance. This involves the robot using sensors to detect the current position of obstacles in real time, and then adjusting its navigation path as an obstacle approaches to avoid it. However, without combining the scene mapping capabilities of Video SLAM and the proactive behavior analysis capabilities of video AI, this passive obstacle avoidance method has significant drawbacks: Firstly, because it doesn't anticipate the behavioral intentions of dynamic obstacles, the robot often needs to brake suddenly or abruptly dodge when an obstacle approaches, affecting navigation smoothness and potentially leading to collisions that could damage the robot or endanger the safety of people or pets. Secondly, passive obstacle avoidance relies solely on the real-time position information of obstacles and cannot handle sudden behaviors of dynamic obstacles such as sudden acceleration or turning, resulting in poor obstacle avoidance robustness and difficulty adapting to complex home living scenarios. Therefore, how to combine Video SLAM and video AI technologies to predict the behavioral intent of dynamic obstacles, enabling robots to upgrade from passive obstacle avoidance to active obstacle avoidance, and improving the safety, smoothness, and robustness of autonomous navigation of home robots, has become an urgent technical problem to be solved in the field of home robot navigation technology.

[0004] Therefore, the existing technology of autonomous navigation for home robots needs further improvement. Summary of the Invention

[0005] The purpose of this invention is to provide a dynamic obstacle intent prediction method and system for robot autonomous navigation based on Video SLAM and video AI intelligent analysis. By predicting the short-term behavioral intent of dynamic obstacles, a smooth obstacle avoidance path can be planned in advance, enabling the robot to actively avoid obstacles and improving the safety and smoothness of navigation for human-robot and robot-pet coexistence in home scenarios.

[0006] To achieve the above objectives, the present invention adopts the following solution: A method for predicting dynamic obstacle intent during autonomous robot navigation includes the following steps: Step S1: Collect multimodal perception data of dynamic obstacles in the home scene in real time. The dynamic obstacles include people or pets. The scene localization and obstacle spatial position perception are completed by using Video SLAM visual simultaneous localization and mapping technology. At the same time, the collected video stream is analyzed at the frame level by video AI intelligent analysis technology to extract and fuse multimodal perception data including motion trajectory, instantaneous speed, posture features and relative position with the robot. Step S2: Based on the multimodal perception data, extract the motion pattern features of the dynamic obstacle through a pre-trained intention prediction model, and output its short-term behavioral intention category, which includes going straight, turning, turning, sudden acceleration or sudden running. Step S3: Based on the behavioral intent category, and combined with the robot's current position and target navigation point, generate a predictive-driven smooth obstacle avoidance path. The smooth obstacle avoidance path avoids sudden braking or abrupt detours. Step S4: Control the robot to navigate along the smooth obstacle avoidance path to achieve smooth, safe, and autonomous navigation in scenarios where people, pets, and robots coexist.

[0007] Furthermore, in step S2, the intent prediction model is a deep learning model based on temporal convolutional networks and attention mechanisms. The model takes multi-frame multimodal perception data from Video SLAM and video AI as input and outputs the probability distribution of the dynamic obstacle's behavioral intent within a preset time window in the future.

[0008] Further, in step S3, the generation of the smooth obstacle avoidance path includes: Based on the aforementioned behavioral intent category, and combining the 3D map of the home scene constructed by Video SLAM with the real-time motion parameters of obstacles analyzed by video AI, the predicted motion area of ​​dynamic obstacles is calculated. By combining the predicted motion area with the robot's motion constraints, an improved heuristic path search algorithm or elastic band algorithm is used to generate a smooth obstacle avoidance path that adapts to the obstacle's motion trend.

[0009] Furthermore, it also includes: The pre-executed step S0 involves collecting scene spatial topology information through Video SLAM in a home setting, collecting and labeling obstacle behavior link data through video AI, constructing a labeled dynamic obstacle behavior dataset, and using it to train the intent prediction model. The labeling includes behavioral intent categories and corresponding multimodal perception data sequences.

[0010] Furthermore, in step S4, if the video AI real-time frame parsing detects a sudden change in the behavioral intent of the dynamic obstacle, the intent prediction result is updated in real time, and a smooth obstacle avoidance path is regenerated by combining the real-time scene localization map of Video SLAM.

[0011] A prediction system includes a perception acquisition unit, an intent prediction unit, a path planning unit, and a motion control unit; The output of the perception acquisition unit is connected to the intention prediction unit, the output of the intention prediction unit is connected to the path planning unit, and the output of the path planning unit is connected to the motion control unit. Perception and Acquisition Unit: It has a built-in Video SLAM module and a video AI analysis module. The Video SLAM module realizes real-time positioning, map building and obstacle spatial location perception in the home scene. The video AI analysis module intelligently analyzes the video stream and extracts the motion trajectory, speed change and posture features of obstacles. The two work together to output multimodal perception data. The dynamic obstacles include moving people and moving pets. Intent prediction unit: Based on the multimodal perception data, it extracts the motion pattern features of dynamic obstacles through a preset intent prediction model and outputs their short-term behavioral intent, including behaviors such as going straight, turning, turning, and suddenly running. Path planning unit: used to generate a smooth obstacle avoidance path based on the behavioral intention, combined with the robot's current pose and the target navigation point. The smooth obstacle avoidance path is used to avoid sudden braking or abrupt detours. Motion control unit: Used to control the robot to perform navigation along the smooth obstacle avoidance path, so as to achieve smooth and safe navigation for people, robots and pets to live together.

[0012] Furthermore, the intent prediction unit includes a deep learning subunit, which adopts a model combining a temporal convolutional network and an attention mechanism. It takes multi-frame multimodal perception data output by the Video SLAM module and the video AI analysis module as input and outputs the probability distribution of the behavioral intent of dynamic obstacles within a preset time window.

[0013] Furthermore, the path planning unit includes: Predictive region calculation subunit: used to calculate the predicted motion region of dynamic obstacles based on behavioral intent, combined with the scene 3D map built by the Video SLAM module and the real-time obstacle motion analysis data output by the video AI analysis module; Path generation subunit: Used to combine the predicted motion area with the robot motion constraints, and use an improved heuristic path search algorithm or elastic band algorithm to generate a smooth obstacle avoidance path that adapts to the obstacle's motion trend.

[0014] Furthermore, it also includes a model training unit, which is used to collect scene spatial information through the Video SLAM module and obstacle behavior information through the video AI analysis module in a home setting, and construct an annotated dynamic obstacle behavior dataset. The annotation includes the behavior intention category and the corresponding multimodal perception data sequence, and an intention prediction model is trained based on the dataset.

[0015] Furthermore, the intent prediction unit is also used to receive real-time detection signals from the video AI analysis module. If a sudden change in the dynamic obstacle's behavior intent is detected, the path planning unit is immediately triggered to regenerate a smooth obstacle avoidance path in conjunction with the real-time scene map of the Video SLAM module.

[0016] In summary, the advantages of this invention over the prior art are: This invention addresses the shortcomings of existing autonomous navigation technologies for home robots. Through the deep integration of Video SLAM and video AI technologies, it achieves proactive prediction of dynamic obstacle intentions, upgrading from passive obstacle avoidance to active obstacle avoidance. It anticipates short-term behavioral intentions and plans obstacle avoidance paths based on the prediction results, avoiding the problems of sudden braking and abrupt detours in traditional passive obstacle avoidance, thus improving navigation smoothness. This invention analyzes obstacle motion parameters using video AI and calculates the predicted motion area of ​​dynamic obstacles using VideoSLAM scene maps, generating smooth obstacle avoidance paths adapted to obstacle movement trends. Simultaneously, it uses video AI to detect sudden changes in obstacle behavior intentions in real time and updates the path promptly using the VideoSLAM real-time map, effectively reducing the risk of collisions between the robot and dynamic obstacles, and adapting to the complex movement states of people and pets in home scenarios. The method and system of this invention cover the complete process of model training, Video SLAM, video AI data acquisition, intent prediction, path planning, and navigation execution. Each unit and step is logically clear and verifiable. At the same time, this technology can be directly reused in various home robot products such as robotic vacuum cleaners and companion robots, and has good technical versatility and practicality. Attached Figure Description

[0017] Figure 1 This is a schematic diagram of the prediction method of the present invention; Figure 2 This is one of the connection diagrams of the prediction system of the present invention; Figure 3 This is the second schematic diagram of the prediction system connection of the present invention. Detailed Implementation

[0018] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0019] Please refer to Figures 1-3. This invention provides a dynamic obstacle intent prediction method for robot autonomous navigation based on Video SLAM and video AI intelligent analysis, including the following steps: Step S1: Collect multimodal perception data of dynamic obstacles in the home scene in real time. The dynamic obstacles include people or pets. The scene localization and obstacle spatial position perception are completed by using Video SLAM visual simultaneous localization and mapping technology. At the same time, the collected video stream is analyzed at the frame level by video AI intelligent analysis technology to extract and fuse multimodal perception data including motion trajectory, instantaneous speed, posture features and relative position with the robot. Step S2: Based on the multimodal perception data, extract the motion pattern features of the dynamic obstacle through a pre-trained intention prediction model, and output its short-term behavioral intention category, which includes going straight, turning, turning, sudden acceleration or sudden running. Step S3: Based on the behavioral intent category, and combined with the robot's current position and target navigation point, generate a predictive-driven smooth obstacle avoidance path. The smooth obstacle avoidance path avoids sudden braking or abrupt detours. Step S4: Control the robot to navigate along the smooth obstacle avoidance path to achieve smooth, safe, and autonomous navigation in scenarios where people, pets, and robots coexist.

[0020] In step S2 of this invention, the intent prediction model is a deep learning model based on temporal convolutional networks and attention mechanisms. The model takes multi-frame multimodal perception data from Video SLAM and video AI as input and outputs the probability distribution of the behavioral intent of dynamic obstacles within a preset time window in the future.

[0021] In step S3 of this invention, the generation of the smooth obstacle avoidance path includes: Based on the aforementioned behavioral intent category, and combining the 3D map of the home scene constructed by Video SLAM with the real-time motion parameters of obstacles analyzed by video AI, the predicted motion area of ​​dynamic obstacles is calculated. By combining the predicted motion area with the robot's motion constraints, an improved heuristic path search algorithm or elastic band algorithm is used to generate a smooth obstacle avoidance path that adapts to the obstacle's motion trend.

[0022] The present invention also includes: The pre-executed step S0 involves collecting scene spatial topology information through Video SLAM in a home setting, collecting and labeling obstacle behavior link data through video AI, constructing a labeled dynamic obstacle behavior dataset, and using it to train the intent prediction model. The labeling includes behavioral intent categories and corresponding multimodal perception data sequences.

[0023] In step S4 of this invention, if the video AI real-time frame parsing detects a sudden change in the behavioral intent of a dynamic obstacle, the intent prediction result is updated in real time, and a smooth obstacle avoidance path is regenerated by combining the real-time scene localization map of Video SLAM.

[0024] A prediction system includes: a perception acquisition unit, an intent prediction unit, a path planning unit, and a motion control unit; The output of the perception acquisition unit is connected to the intention prediction unit, the output of the intention prediction unit is connected to the path planning unit, and the output of the path planning unit is connected to the motion control unit. Perception and Acquisition Unit: It has a built-in Video SLAM module and a video AI analysis module. The Video SLAM module realizes real-time positioning, map building and obstacle spatial location perception in the home scene. The video AI analysis module intelligently analyzes the video stream and extracts the motion trajectory, speed change and posture features of obstacles. The two work together to output multimodal perception data. The dynamic obstacles include moving people and moving pets. Intent prediction unit: Based on the multimodal perception data, it extracts the motion pattern features of dynamic obstacles through a preset intent prediction model and outputs their short-term behavioral intent, including behaviors such as going straight, turning, turning, and suddenly running. Path planning unit: used to generate a smooth obstacle avoidance path based on the behavioral intention, combined with the robot's current pose and the target navigation point. The smooth obstacle avoidance path is used to avoid sudden braking or abrupt detours. Motion control unit: Used to control the robot to perform navigation along the smooth obstacle avoidance path, so as to achieve smooth and safe navigation for people, robots and pets to live together.

[0025] The intent prediction unit of the present invention includes a deep learning subunit. The deep learning subunit adopts a model that combines a temporal convolutional network with an attention mechanism. It takes multi-frame multimodal perception data output by the Video SLAM module and the video AI analysis module as input and outputs the probability distribution of the behavioral intent of dynamic obstacles within a preset time window.

[0026] The path planning unit of the present invention includes: Predictive region calculation subunit: used to calculate the predicted motion region of dynamic obstacles based on behavioral intent, combined with the scene 3D map built by the Video SLAM module and the real-time obstacle motion analysis data output by the video AI analysis module; Path generation subunit: Used to combine the predicted motion area with the robot motion constraints, and use an improved heuristic path search algorithm or elastic band algorithm to generate a smooth obstacle avoidance path that adapts to the obstacle's motion trend.

[0027] The present invention also includes a model training unit, which is used to collect scene spatial information through the VideoSLAM module and obstacle behavior information through the video AI analysis module in a home setting, and construct an annotated dynamic obstacle behavior dataset. The annotation includes the behavior intention category and the corresponding multimodal perception data sequence, and an intention prediction model is trained based on the dataset.

[0028] The intent prediction unit described in this invention is also used to receive real-time detection signals from the video AI analysis module. If a sudden change in the behavior intent of a dynamic obstacle is detected, the path planning unit is immediately triggered to regenerate a smooth obstacle avoidance path in conjunction with the real-time scene map of the Video SLAM module.

[0029] This embodiment provides a dynamic obstacle intent prediction method for autonomous navigation of a home robot, applied to a robotic vacuum cleaner. The specific steps are as follows: Step S0, Model Pre-training: In typical scenarios such as the living room, bedroom, and kitchen, the high-definition camera on the robot vacuum cleaner serves as the core perception hardware for Video SLAM and video AI. It collects video streams and spatial location data of various behaviors, including people walking, turning, and suddenly accelerating, and pets (such as cats and dogs) running and turning. Video SLAM is used to analyze the scene's spatial topology and obstacle spatial coordinates, while video AI performs frame-level analysis of the video stream to extract behavioral features. These are then fused to obtain a multimodal perception data sequence, with multiple sets of data sequences collected. Each data sequence is manually labeled, including the behavioral intent category (straight ahead, turning, turning, sudden acceleration, sudden running) and the corresponding motion trajectory, instantaneous speed, posture features, and relative position to the robot. An initial deep learning model combining a temporal convolutional network and an attention mechanism is used. The labeled dataset is used as training data, and training batches, training epochs, and learning rates are set. After training, a pre-trained intent prediction model is obtained. This model can output the probability distribution of the behavioral intent of dynamic obstacles within the next 2 seconds, improving prediction accuracy.

[0030] Step S1: Real-time acquisition of multimodal perception data: The robot vacuum cleaner starts its autonomous navigation operation, using the VideoSLAM module to complete the localization and map update of the home scene in real time, and perceive the relative spatial position of obstacles and the robot; the video AI analysis module performs frame-level analysis on the video stream captured by the high-definition camera at a frequency of 10 frames / second, extracts the posture features of dynamic obstacles, fits the motion trajectory, and calculates the instantaneous velocity; the two data are fused in real time to output multimodal perception data, ensuring the real-time performance and accuracy of the data.

[0031] Step S2, Predicting short-term behavioral intentions of dynamic obstacles: Input the 10 consecutive frames of multimodal perception data output by Video SLAM and video AI in step S1 into the pre-trained intention prediction model. The model extracts the motion pattern features of the dynamic obstacles and outputs the probability distribution of behavioral intentions in the next 2 seconds. If the probability of going straight is 85% and the probability of other behavioral intentions is less than 10%, then the short-term behavioral intention category of the dynamic obstacle is determined to be going straight.

[0032] Step S3: Generating a smooth obstacle avoidance path: Based on the straight-moving behavior intention category, combined with the 3D map of the home scene constructed by Video SLAM and the instantaneous speed and direction of movement of obstacles analyzed by video AI, calculate the predicted movement area of ​​dynamic obstacles in the next 2 seconds (i.e., a rectangular area with the current position as the starting point and the instantaneous speed × 2 seconds as the length in the current movement direction); combined with the motion constraints of the robot vacuum cleaner (maximum driving speed 0.5m / s, minimum turning radius 0.3m), use an improved heuristic path search algorithm to search for a path that avoids the predicted movement area and is closest to the target navigation point (such as the corner of the bedroom), and then use the elastic band algorithm to smooth the path to generate a smooth obstacle avoidance path. The turning angle of the path inflection point does not exceed 30° to avoid abrupt detours.

[0033] Step S4, Navigation Execution and Intent Sudden Change Handling: Control the robot vacuum cleaner to travel along a smooth obstacle avoidance path at a speed of 0.3m / s to achieve autonomous navigation; during navigation, if the video AI analysis module detects a dynamic obstacle suddenly turning around (behavioral intent sudden change) in real-time frame parsing, the intent prediction result is immediately updated, and the predicted motion area is recalculated in combination with the real-time scene localization map of the Video SLAM module to generate a new smooth obstacle avoidance path, and the robot is controlled to adjust its driving direction to avoid collisions.

[0034] The foregoing has shown and described the basic principles and main features of the present invention, as well as its advantages. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The embodiments and descriptions in the specification are merely illustrative of the principles of the invention. Various changes and modifications can be made to the present invention without departing from its spirit and scope, and all such changes and modifications fall within the scope of the present invention as claimed. The scope of protection of the present invention is defined by the appended claims and their equivalents.

Claims

1. A method for dynamic obstacle intent prediction for robot autonomous navigation, the method comprising: Includes the following steps: Step S1: Collect multimodal perception data of dynamic obstacles in the home scene in real time. The dynamic obstacles include people or pets. The scene localization and obstacle spatial position perception are completed by using Video SLAM visual simultaneous localization and mapping technology. At the same time, the collected video stream is analyzed at the frame level by video AI intelligent analysis technology to extract and fuse multimodal perception data including motion trajectory, instantaneous speed, posture features and relative position with the robot. Step S2: Based on the multimodal perception data, extract the motion pattern features of the dynamic obstacle through a pre-trained intention prediction model, and output its short-term behavioral intention category, which includes going straight, turning, turning, sudden acceleration or sudden running. Step S3: Based on the behavioral intent category, and combined with the robot's current position and target navigation point, generate a predictive-driven smooth obstacle avoidance path. The smooth obstacle avoidance path avoids sudden braking or abrupt detours. Step S4: Control the robot to navigate along the smooth obstacle avoidance path to achieve smooth, safe, and autonomous navigation in scenarios where people, pets, and robots coexist.

2. The method of claim 1, wherein: In step S2, the intent prediction model is a deep learning model based on temporal convolutional networks and attention mechanisms. The model takes multi-frame multimodal perception data from Video SLAM and video AI as input and outputs the probability distribution of the dynamic obstacle's behavioral intent within a preset time window in the future.

3. The method for predicting dynamic obstacle intent in autonomous robot navigation according to claim 2, characterized in that: In step S3, the generation of the smooth obstacle avoidance path includes: Based on the aforementioned behavioral intent category, and combining the 3D map of the home scene constructed by Video SLAM with the real-time motion parameters of obstacles analyzed by video AI, the predicted motion area of ​​dynamic obstacles is calculated. By combining the predicted motion area with the robot's motion constraints, an improved heuristic path search algorithm or elastic band algorithm is used to generate a smooth obstacle avoidance path that adapts to the obstacle's motion trend.

4. The method for predicting dynamic obstacle intent in autonomous robot navigation according to claim 1, characterized in that, Also includes: The pre-executed step S0 involves collecting scene spatial topology information through Video SLAM in a home setting, collecting and labeling obstacle behavior link data through video AI, constructing a labeled dynamic obstacle behavior dataset, and using it to train the intent prediction model. The labeling includes behavioral intent categories and corresponding multimodal perception data sequences.

5. The method for predicting dynamic obstacle intent in autonomous robot navigation according to claim 4, characterized in that: In step S4, if the video AI real-time frame parsing detects a sudden change in the behavioral intent of a dynamic obstacle, the intent prediction result is updated in real time, and a smooth obstacle avoidance path is regenerated from the real-time scene localization map of Video SLAM.

6. A prediction system, characterized in that, include: The system comprises a perception and acquisition unit, an intent prediction unit, a path planning unit, and a motion control unit. The output of the perception acquisition unit is connected to the intention prediction unit, the output of the intention prediction unit is connected to the path planning unit, and the output of the path planning unit is connected to the motion control unit. Perception and Acquisition Unit: It has a built-in Video SLAM module and a video AI analysis module. The Video SLAM module realizes real-time positioning, map building and obstacle spatial location perception in the home scene. The video AI analysis module intelligently analyzes the video stream and extracts the motion trajectory, speed change and posture features of obstacles. The two work together to output multimodal perception data. The dynamic obstacles include moving people and moving pets. Intent prediction unit: Based on the multimodal perception data, it extracts the motion pattern features of dynamic obstacles through a preset intent prediction model and outputs their short-term behavioral intent, including behaviors such as going straight, turning, turning, and suddenly running. Path planning unit: used to generate a smooth obstacle avoidance path based on the behavioral intention, combined with the robot's current pose and the target navigation point. The smooth obstacle avoidance path is used to avoid sudden braking or abrupt detours. Motion control unit: Used to control the robot to perform navigation along the smooth obstacle avoidance path, so as to achieve smooth and safe navigation for people, robots and pets to live together.

7. A prediction system according to claim 6, characterized in that... The intent prediction unit includes a deep learning subunit. The deep learning subunit adopts a model that combines a temporal convolutional network with an attention mechanism. It takes multi-frame multimodal perception data output by the Video SLAM module and the video AI analysis module as input and outputs the probability distribution of the behavioral intent of dynamic obstacles within a preset time window.

8. A prediction system according to claim 7, characterized in that: The path planning unit includes: Predictive region calculation subunit: used to calculate the predicted motion region of dynamic obstacles based on behavioral intent, combined with the scene 3D map built by the Video SLAM module and the real-time obstacle motion analysis data output by the video AI analysis module; Path generation subunit: Used to combine the predicted motion area with the robot motion constraints, and use an improved heuristic path search algorithm or elastic band algorithm to generate a smooth obstacle avoidance path that adapts to the obstacle's motion trend.

9. A prediction system according to claim 8, characterized in that: It also includes a model training unit, which is used to collect scene spatial information through the Video SLAM module and obstacle behavior information through the video AI analysis module in a home setting, and construct an annotated dynamic obstacle behavior dataset. The annotation includes the behavior intention category and the corresponding multimodal perception data sequence. An intention prediction model is trained based on the dataset.

10. A prediction system according to claim 9, characterized in that: The intent prediction unit is also used to receive real-time detection signals from the video AI analysis module. If a sudden change in the behavior intent of a dynamic obstacle is detected, the path planning unit is immediately triggered to regenerate a smooth obstacle avoidance path in conjunction with the real-time scene map of the Video SLAM module.

Images