Path planning guide for the blind based on a robot dog

Abstract

A path planning guide method based on a robot dog, in the method, a navigation system, a path planning module and a voice prompt system for user interaction are arranged in the robot dog, after the robot dog is started, the sensors carried by the robot dog collect surrounding environment information in real time, an environment map is constructed by using a SLAM technology, and a current position of a user is determined, the user sets a path point and a destination based on the environment map through voice interaction or a mobile device, according to the environment map, the current position of the user, the path point and the destination, the robot dog executes a path planning algorithm through the built-in path planning module, a planning path from the current position of the user to the destination is generated, the voice prompt system generates navigation instructions through voice, and is used for providing the navigation instructions and environment information to the user, the sensors detect and avoid obstacles in real time, through identification of a camera and a machine learning algorithm, the robot dog predicts a motion trajectory of the obstacles and adjusts the planning path, and guides the user to walk along the planning path to guide the user to the destination.

Description

Technical Field

[0001] This invention relates to the field of guide dog technology, and in particular to a path planning guide dog method based on a robot dog. Background Technology

[0002] Visually impaired individuals face numerous challenges in their daily lives, with independent travel being one of the biggest hurdles. While traditional guide canes, guide dogs, and electronic navigation devices can assist visually impaired individuals to some extent, they still have many limitations. For example, guide canes have a limited detection range, cannot provide comprehensive environmental information, and require extensive training to become proficient in their use; training and maintaining guide dogs is expensive, guide dogs have a limited working lifespan, and some places may not allow guide dogs, and the limited number of guide dogs cannot meet the needs of all visually impaired individuals; electronic devices often have lower accuracy indoors or in densely populated high-rise areas, and some devices are cumbersome to operate and not user-friendly for visually impaired individuals.

[0003] In recent years, with the rapid development of robotics technology, robot dogs, as a mobile robot platform, possess flexible movement capabilities, advanced sensors, and artificial intelligence algorithms. They can perceive their surroundings in real time, identifying obstacles, traffic signals, and pedestrian flow, thus providing safe and reliable navigation services for visually impaired individuals. Their quadrupedal design allows them to walk stably on complex terrains, such as stairs, slopes, and uneven surfaces, greatly expanding the application range of traditional guide dogs. With continuous technological advancements, robot dogs are expected to become an important tool in the future of guide dogs, helping visually impaired individuals better integrate into social life and improve their independence and quality of life.

[0004] The information disclosed in the background section is only for enhancing the understanding of the background of this invention, and therefore may contain information that does not constitute prior art known to those skilled in the art. Summary of the Invention

[0005] This invention provides a path planning and guidance method for the visually impaired based on a robot dog, offering a safer, more convenient, and intelligent travel experience for visually impaired individuals.

[0006] A path planning and guidance method for the blind based on robot dogs includes: The robot dog is equipped with a navigation system, a path planning module, and a voice prompt system for user interaction. The robot dog is equipped with sensors for detecting the surrounding environment, including cameras, lidar, and inertial measurement units. After the robot dog is started, the onboard sensors collect information about the surrounding environment in real time. The system utilizes SLAM technology to construct an environmental map. Kalman filtering fuses surrounding environmental information collected by LiDAR, cameras, and inertial measurement units to generate a real-time environmental map with dynamic obstacle markers and determine the user's current location. The user can then set waypoints and destinations based on the environmental map through voice interaction or mobile devices. Based on the environmental map, the user's current location, waypoints, and destination, the robot dog executes a path planning algorithm through its built-in path planning module to generate a planned path from the user's current location to the destination. The voice prompt system generates navigation instructions through voice to provide the user with navigation instructions and environmental information. The sensors detect and avoid obstacles in real time. Through camera recognition and machine learning algorithms, the robot dog predicts the movement trajectory of obstacles and adjusts the planned path. It guides the user to walk along the planned path through voice prompts, vibration feedback, or traction control to guide the user to the destination.

[0007] In the aforementioned path planning and guidance method for the blind based on a robot dog, a remote monitoring and control module is deployed on the robot dog. The guardian can view the robot dog's location, status, and surrounding environment information in real time and remotely control it through a mobile device or computer. The remote monitoring and control module includes a data acquisition unit, a communication unit, and an application unit on the guardian's end. The data acquisition unit acquires the robot dog's location, posture, environmental map, and user status in real time and transmits them to the application unit on the guardian's end through encryption via the communication unit. The application unit displays the robot dog's real-time location, environmental map, and user deviation warnings through a visual interface.

[0008] In the aforementioned path planning and guidance method for the blind based on a robot dog, after setting waypoints and destinations, the guardian can select and modify waypoints and destinations via a mobile device or computer.

[0009] In the aforementioned path planning and guidance method for the blind based on a robot dog, the camera mounted on the robot dog collects video data of the surrounding environment in real time and transmits the video data to a remote monitoring and control module for the guardian to view. When obstacles or low battery are detected, the robot dog sends a warning message to the remote monitoring and control module and takes safety measures, including issuing voice prompts through the built-in speaker to remind the user to pay attention to obstacles or low battery, activating a low battery mode to extend the battery life, sending a warning message to the guardian through the remote monitoring and control module, and replanning the path based on environmental data to avoid obstacles or guide the user to the nearest charging point or safe area.

[0010] In the aforementioned path planning and guidance method for the blind based on a robot dog, the robot dog constructs an environmental map using SLAM technology. The environmental map includes obstacles, walkable areas, stairs, ramps, tactile paving, entrances and exits, and potential danger zones. The robot dog uses a positioning module to locate itself on the environmental map.

[0011] In the aforementioned path planning and guidance method for the blind based on a robot dog, the positioning algorithm of the positioning module includes Kalman filtering, particle filtering, and visual odometry. Kalman filtering is fused with an inertial measurement unit to achieve pose noise-resistant estimation, particle filtering calculates the optimal position in the environment through particle swarm dynamic weights, and visual odometry is used to achieve indoor displacement tracking through image feature analysis.

[0012] In the aforementioned path planning and guidance method for the blind based on a robot dog, the path planning algorithm includes the A* search algorithm, the Dijkstra algorithm, and a deep reinforcement learning algorithm. The A* search algorithm combines a heuristic function to search for the optimal path from the starting point to the target point in a static environment. The Dijkstra algorithm provides a safe and smooth walking route for blind users by traversing all possible paths and selecting the path with the minimum cost. The deep reinforcement learning algorithm optimizes the path planning strategy by simulating the interaction between the robot dog and the environment and using a reward mechanism, adjusting the path in real time to avoid moving obstacles.

[0013] In the aforementioned path planning and guidance method for the blind based on a robot dog, the robot dog generates multiple feasible paths according to user needs and optimizes the selection based on path length, safety, and comfort. The path planning is adjusted in real time according to environmental changes and user needs to ensure the user safely reaches their destination. Specifically, path length is optimized through geometric distance, prioritizing shorter paths to reduce walking time; safety is optimized by assessing obstacle density, dangerous area distribution, and dynamic obstacle probability, prioritizing paths with fewer obstacles and lower risk; and comfort is optimized by analyzing path flatness, turning frequency, and compliance with tactile paving.

[0014] In the aforementioned path planning and guidance method based on a robot dog, the robot dog transmits directional information to the user through a tactile feedback device. The user perceives the robot dog's navigation commands through touch. The tactile feedback device includes a vibration motor or a movable handle, which transmits navigation commands through different vibration modes or the tilt direction of the handle.

[0015] In the aforementioned path planning and guidance method based on a robot dog, the robot dog classifies and identifies obstacles using camera recognition and machine learning algorithms, and optimizes avoidance strategies based on the type and trajectory of the obstacles.

[0016] Compared with the prior art, the present invention has the following advantages: High safety: The robot dog has strong environmental perception and intelligent decision-making capabilities, which can effectively identify and avoid obstacles, ensuring user safety.

[0017] Highly practical: The robot dog is agile and can adapt to various complex environments, providing users with a more convenient travel experience.

[0018] High interactivity: The robot dog supports multiple interaction methods such as voice interaction and mobile APP control, making it convenient for users to operate. It can also be remotely connected, allowing guardians to share the robot dog's view through VR devices and control the robot dog through a mobile APP.

[0019] High scalability: This invention can be combined with other auxiliary technologies, such as facial recognition and voice recognition, to provide users with more comprehensive services. Attached Figure Description

[0020] Various other advantages and benefits of the present invention will become apparent to those skilled in the art upon reading the detailed description of the preferred embodiments below. The accompanying drawings are for illustrative purposes only and are not intended to limit the invention. It is obvious that the drawings described below are merely some embodiments of the invention, and those skilled in the art can obtain other drawings based on these drawings without any inventive effort. Furthermore, the same reference numerals denote the same parts throughout the drawings.

[0021] In the attached diagram: Figure 1 This is a flowchart of a path planning and guidance method for the blind based on a robot dog, according to an embodiment of the present invention.

[0022] The present invention will be further explained below with reference to the accompanying drawings and embodiments. Detailed Implementation

[0023] Specific embodiments of the invention will now be described in more detail with reference to the accompanying drawings. While specific embodiments of the invention are shown in the drawings, it should be understood that the invention may be implemented in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that this invention will be thorough and complete, and will fully convey the scope of the invention to those skilled in the art.

[0024] It should be noted that certain terms are used in the specification and claims to refer to specific components. Those skilled in the art will understand that different terms may be used to refer to the same component. This specification and claims do not distinguish components based on differences in terminology, but rather on differences in function. The terms "comprising" or "including" used throughout the specification and claims are open-ended and should be interpreted as "comprising but not limited to." The following descriptions are preferred embodiments for carrying out the invention; however, these descriptions are for the purpose of understanding the general principles of the specification and are not intended to limit the scope of the invention. The scope of protection of this invention is determined by the appended claims.

[0025] To facilitate understanding of the embodiments of the present invention, further explanations and descriptions will be provided below with reference to the accompanying drawings and specific embodiments. The accompanying drawings do not constitute a limitation on the embodiments of the present invention.

[0026] like Figure 1 As shown, the path planning and guidance method for the blind based on robot dogs includes the following steps: The robot dog is equipped with a navigation system, a path planning module, and a voice prompt system for user interaction. The robot dog is equipped with sensors for detecting the surrounding environment, including cameras, lidar, and inertial measurement units. After the robot dog is started, the onboard sensors collect information about the surrounding environment in real time. The system utilizes SLAM technology to construct an environmental map. Kalman filtering fuses surrounding environmental information collected by LiDAR, cameras, and inertial measurement units to generate a real-time environmental map with dynamic obstacle markers and determine the user's current location. The user can then set waypoints and destinations based on the environmental map through voice interaction or mobile devices. Based on the environmental map, the user's current location, waypoints, and destination, the robot dog executes a path planning algorithm through its built-in path planning module to generate a planned path from the user's current location to the destination. The voice prompt system generates navigation instructions through voice to provide the user with navigation instructions and environmental information. The sensors detect and avoid obstacles in real time. Through camera recognition and machine learning algorithms, the robot dog predicts the movement trajectory of obstacles and adjusts the planned path. It guides the user to walk along the planned path through voice prompts, vibration feedback, or traction control to guide the user to the destination.

[0027] In a preferred embodiment of the path planning and guidance method for the blind based on a robot dog, the robot dog is equipped with a remote monitoring and control module. The guardian can view the robot dog's location, status, and surrounding environment information in real time and remotely control it through a mobile device or computer. The remote monitoring and control module includes a data acquisition unit, a communication unit, and an application unit at the guardian's end. The data acquisition unit acquires the robot dog's location, posture, environmental map, and user status in real time and transmits them to the application unit at the guardian's end through encryption via the communication unit. The application unit displays the robot dog's real-time location, environmental map, and user deviation warnings through a visual interface.

[0028] In a preferred embodiment of the path planning and guidance method for the blind based on a robot dog, after setting waypoints and destinations, the guardian can modify the selection of waypoints and destinations through a mobile device or computer.

[0029] In a preferred embodiment of the path planning and guidance method for the blind based on a robot dog, the camera mounted on the robot dog collects video data of the surrounding environment in real time and transmits the video data to a remote monitoring and control module for the guardian to view. When obstacles or low battery are detected, the robot dog sends a warning message to the remote monitoring and control module and takes safety measures, including issuing voice prompts through the built-in speaker to remind the user to pay attention to obstacles or low battery, activating a low battery mode to extend the battery life, sending a warning message to the guardian through the remote monitoring and control module, and replanning the path based on environmental data to avoid obstacles or guide the user to the nearest charging point or safe area.

[0030] In a preferred embodiment of the path planning and guidance method for the blind based on a robot dog, the robot dog constructs an environmental map using SLAM technology. The environmental map includes obstacles, walkable areas, stairs, ramps, tactile paving, entrances and exits, and potential danger zones. The robot dog uses a positioning module to locate itself on the environmental map.

[0031] In a preferred embodiment of the path planning and guidance method for the blind based on a robot dog, the positioning algorithm of the positioning module includes Kalman filtering, particle filtering, and visual odometry. Kalman filtering is fused with an inertial measurement unit to achieve pose noise-resistant estimation, particle filtering calculates the optimal position in the environment through particle swarm dynamic weights, and visual odometry is used to achieve indoor displacement tracking through image feature analysis.

[0032] In a preferred embodiment of the path planning and guidance method for the blind based on a robot dog, the path planning algorithm includes the A* search algorithm, the Dijkstra algorithm, and a deep reinforcement learning algorithm. The A* search algorithm combines a heuristic function to search for the optimal path from the starting point to the target point in a static environment. The Dijkstra algorithm provides a safe and smooth walking route for blind users by traversing all possible paths and selecting the path with the lowest cost. The deep reinforcement learning algorithm optimizes the path planning strategy by simulating the interaction between the robot dog and the environment and using a reward mechanism, adjusting the path in real time to avoid moving obstacles.

[0033] In one embodiment, the A-star search algorithm includes the following steps: Initialization: Add the starting point to the open list.

[0034] Loop: Select the node with the minimum cost from the open list (based on the heuristic function f(n) = g(n) + h(n), where g(n) is the actual cost from the starting point to the current node, and h(n) is the estimated cost from the current node to the target node).

[0035] Extended: Move the current node into the closed list and check its neighboring nodes.

[0036] Update: If a neighboring node is not visited or a better path is found, update its cost and add it to the open list.

[0037] Termination: When the target node is added to the close list, backtrack the path.

[0038] The nodes obtained by backtracking are the nodes traversed by the optimal path planned by the A* search algorithm. Outputting the nodes in the order of the list yields the optimal path under the conditions of this algorithm.

[0039] Dijkstra's algorithm: Initialization: Add the starting point to the priority queue and set its cost to 0. Set the cost of other nodes to infinity.

[0040] Loop: Take the node with the lowest cost from the priority queue.

[0041] Extension: Check its neighboring nodes. If the path to the neighboring node through the current node is better, update its cost and add it to the priority queue.

[0042] Termination: When the target node is retrieved, backtrack the path.

[0043] The nodes obtained by backtracking are the nodes traversed by the optimal path planned by Dijkstra's algorithm. Outputting the nodes in the order of the list yields the optimal path under the conditions of this algorithm.

[0044] Quickly expand random tree: Initialization: Define the starting and target regions, initialize a tree, with the root node of the tree as the starting point.

[0045] Random sampling: A target point (usually called a "random point" or "sampling point") is randomly generated in the configuration space. If the target point falls within the target area, an attempt is made to directly connect the target point.

[0046] Search: Find the node in the current tree that is closest to the random point (usually using Euclidean distance or other distance metrics).

[0047] Expanding the tree: Extend a fixed length from the nearest node toward a random point, generating a new node. Check if the path between the new node and the nearest node is collision-free (i.e., without obstacles). If there is no collision, add the new node to the tree. If a collision occurs, discard the node.

[0048] Repeat: Repeat the three steps of random sampling, searching, and expanding the tree until a new node enters the target region or the maximum number of iterations is reached.

[0049] Path extraction: If a new node enters the target area, backtrack from the root node (starting point) to the target node to extract a complete path.

[0050] In a preferred embodiment of the path planning and guidance method for the blind based on a robot dog, the robot dog generates multiple feasible paths according to user needs and optimizes the selection based on path length, safety, and comfort. The path planning is adjusted in real time according to environmental changes and user needs to ensure the user safely reaches their destination. Specifically, path length is optimized through geometric distance, prioritizing shorter paths to reduce walking time; safety is optimized by assessing obstacle density, dangerous area distribution, and dynamic obstacle probability, prioritizing paths with fewer obstacles and lower risk; and comfort is optimized by analyzing path flatness, turning frequency, and compliance with tactile paving.

[0051] In a preferred embodiment of the path planning and guidance method for the blind based on a robot dog, the robot dog transmits directional information to the user through a tactile feedback device. The user perceives the robot dog's navigation instructions through touch. The tactile feedback device includes a vibration motor or a movable handle, which transmits navigation instructions through different vibration modes or the tilt direction of the handle.

[0052] In a preferred embodiment of the path planning and guidance method for the blind based on a robot dog, the robot dog classifies and identifies obstacles using camera recognition and machine learning algorithms. It optimizes avoidance strategies based on the type and trajectory of the obstacles to ensure user safety. Specifically, for static obstacles, it pre-plans detour paths; for moving pedestrians or vehicles, it predicts their trajectories and calculates safe distances, dynamically adjusting the robot dog's walking speed and direction, prioritizing paths away from dangerous areas. Simultaneously, during the avoidance process, the robot dog provides real-time voice prompts and tactile feedback to inform the user of changes in the surrounding environment, ensuring the user safely avoids obstacles and successfully reaches their destination.

[0053] In one embodiment, the camera acquires RGB images and depth data to identify and label static and dynamic obstacles; the lidar generates a 3D point cloud, accurately measures the distance to obstacles, constructs a high-resolution map, and fuses it with camera data to eliminate misjudgment of dynamic targets; the inertial measurement unit (IMU) outputs acceleration, angular velocity, and attitude angle data in real time to compensate for sensor positioning drift.

[0054] In one embodiment, a path planning and guidance method for the blind based on a robot dog includes the following steps: Once the robot dog is started, its onboard sensors begin to work, collecting information about the surrounding environment in real time.

[0055] Use SLAM technology to build an environmental map and determine the user's current location.

[0056] Users can set destinations via voice interaction or a mobile app, and guardians can select route points and destinations via the mobile app.

[0057] The route planning module plans a safe and efficient optimal route based on the environmental map, user location, target point information, and route point information set by the guardian.

[0058] The robot dog guides users along a planned path through voice prompts, vibration feedback, and traction control.

[0059] The system monitors user status and environmental changes in real time and makes dynamic adjustments to ensure users arrive at their destination safely.

[0060] In one embodiment, a path planning and guidance method for the blind based on a robot dog includes the following major steps: User location and target point setting: Determine the user's current location using GPS and other positioning technologies, and set the target point through voice interaction or a mobile app.

[0061] Path planning: Based on the map and target point information, the blind user's guardian selects path points on the map to generate a fixed route. If the blind user changes their destination temporarily during the journey, they can change the destination through voice interaction or contact the user's guardian through the robot dog to reset the route.

[0062] Environmental perception and map building: The robot dog is equipped with sensors (such as cameras, LiDAR, depth cameras, etc.) to collect information about the surrounding environment in real time, including the location, size, and type of obstacles, and uses simultaneous localization and mapping (SLAM) technology to build an environmental map.

[0063] Safety features: The robot dog is equipped with an emergency stop button, obstacle avoidance sensors, and other safety devices to ensure user safety.

[0064] Emergency Response Measures: In case of emergencies during the blind user's walk, such as the user accidentally dropping their belongings, the robot dog can be instructed to retrieve the items via voice interaction. In more complex situations, the robot dog can also contact the user's guardian, who can then wear VR glasses. The robot dog will share the guardian's field of vision, and the guardian can remotely control the robot dog via a mobile app to perform more complex actions to handle unexpected situations.

[0065] Path guidance: The robot dog guides users along a planned path through voice prompts, vibration feedback, and traction control, and monitors user status and environmental changes in real time to make dynamic adjustments.

[0066] In one embodiment, the method includes: In the robot dog system, a ROS (Robot Operating System) environment is deployed. The LiDAR driver node is configured to subscribe to the ` / raw_points` topic of type `sensor_msgs / PointCloud2`, with the data acquisition frequency set to 10Hz and NTP time synchronization protocol enabled. The LiDAR outputs raw point clouds of XYZ coordinates (accuracy ±2cm) and intensity values ​​(8-bit quantization). Simultaneously, the IMU driver node is started to subscribe to the ` / imu_data` topic of type `sensor_msgs / Imu` (200Hz sampling rate). Spatiotemporal synchronization is achieved through the `message_filters` library, with timestamp alignment error controlled within ±5ms. In the preprocessing stage, the RadiusOutlierRemoval filter from the PCL library is used, with the search radius set to 0. With a minimum neighborhood size of 6 points and a 0.15m radius, a voxel grid filter (LeafSize=0.05m) is applied after noise filtering. For motion distortion calibration, a six-dimensional state vector of IMU angular velocity (rad / s) and linear acceleration (m / s²) is established. A rigid transformation matrix from the IMU to the LiDAR coordinate system is calculated based on the mechanical installation position of the LiDAR. A B-spline interpolation algorithm is used to generate 20 interpolated poses within a 100ms point cloud acquisition cycle, and an inverse coordinate transformation is performed on each laser point. In the LOAM algorithm implementation, the single-frame point cloud is sorted by curvature value, with the top 2% of curvature points selected as edge points (Sharp) and the bottom 4% as flat points (Flat). A KD-Tree is constructed for feature matching. Edge points are identified using the point-to-line distance residual (residual formula: |(X_i)|). - X_j) × (X_i - X_k)| / |X_j - X_k|, the plane points are represented by the point-to-plane distance residuals (residual formula: |(X_i - X_j)·( (X_j - X_k) × (X_j - X_l) )| / |(X_j - X_k) × (X_j - X_l)|), the SE(3) pose transformation is solved iteratively by the Levenberg-Marquardt optimizer, with the convergence conditions set as Δθ<0.01rad and Δt<0.005m; the global map is maintained by storing it in an octree structure, and NDT registration is performed every 5 frames (grid resolution 0.2m, Newton method iteration 30 times). At the same time, a factor map is established to optimize the pose map (implemented by g2o library, with IMU pre-integration constraints and loop closure detection constraints added). Finally, 6-DoF pose (accuracy: translation ±3cm, rotation ±0.5°) and dense 3D map (point cloud density 500 points / m²) are output. The whole process meets the real-time requirements (single frame processing time <80ms).

[0067] A Basler acA2440-75uc camera (global shutter, 2448×2048 resolution, 30fps) was deployed in the robot dog's vision system. The camera subscribed to the ` / rgb_image` topic of type `sensor_msgs / Image` via the ROS `usb_cam` driver node. Hardware-level synchronization with LiDAR data was achieved using the PTP precise clock protocol (time deviation <1ms). The acquisition environment covered ISO [unclear]. The 22048 standard defines six illumination levels (0-200,000 lux) and CIE haze levels (Haze ≥ 90%). OpenCV's cv::VideoCapture is used to capture H.264 encoded video streams (bitrate 20 Mbps). A hybrid dataset is constructed by combining the Chinese Academy of Sciences Traffic Signs Dataset (CCTSDB: 46,000 annotated images, containing 4 major categories and 58 subcategories of signs) and the BDD100K dataset (100,000 images of driving scenes in various weather conditions). LabelImg is used for polygon annotation (occluded targets are preserved when the minimum visible area is ≥ 15 pixels). An XML tag structure is defined, including category ID, rotated bounding box coordinates (cx, cy, w, h, θ), and visibility flags (0-1 levels). The annotation results were cross-validated three times (error box ratio <2%) and then converted to COCO format (annotations.json containing segmentation, area, and iscrowd fields). During the data augmentation phase, the Albumentations library was used to implement spatial transformations (RandomRotateLimit ±15°, SafeHorizontalFlip probability 0.5), photometric distortion (RandomGammaLimit(80,120), GaussianBlur(σ=3)), and weather simulation (RainEffect parameters: drop_length=15, drop_width=3, drop_angle=45). The model was augmented using a hybrid augmentation method (rain_type='heavy') and CutMix (β=1.0), resulting in a dataset 5 times larger than the original (200,000 original samples → 1 million augmented samples). The model was built on the Ultralytics YOLOv5s architecture, replacing the backbone with EfficientNet-B4 (which includes the MBConv module and SE attention mechanism, and adjusts the input size to 1024×1024). A BiFPN cross-scale feature fusion layer was added to the Neck part (with a weight learning rate of 0.01), and the Head part was decoupled into three independent detection branches (the traffic sign branch uses 4).The training dataset used small anchor boxes (12×12, 16×18, 22×25) of 5×10^-3, medium anchor boxes (36×72, 48×96) for pedestrian branches, and large anchor boxes (128×256, 256×512) for obstacle branches. The loss function was a weighted GIOU (traffic sign weight 2.0, pedestrian weight 1.5, obstacle weight 1.0). Training was performed using an NVIDIA A100×4 processor. Distributed training was performed using GPUs, with the AdamW optimizer configured (lr=3e-4, weight_decay=1e-4), batch_size=128, and Cosine annealing scheduling (T_0=100 cycles). Backbone was frozen for fine-tuning after achieving mAP@0.5:0.95≥0.78 on the validation set. TensorRT acceleration (FP16 quantization) was used during inference, deploying three parallel detection heads and setting class-specific NMS thresholds (traffic sign IoU=0.4, pedestrian IoU=0.5, obstacle IoU=0.6). Output results were mapped to the LiDAR coordinate system using cv::projectPoints (calibration error <0.1°). Real-time detection performance was achieved (FPS≥25 at 1080p input), with a detection rate of 85% Recall@0.9 Precision for small traffic signs (50×50 pixels). A false detection feedback mechanism was also established (triggering re-detection when confidence fluctuation >15%).

[0068] Ultrasonic sensors are used to detect nearby obstacles, and IMU is used to monitor the robot dog's posture and movement. The voice recognition module collects the user's voice commands (such as "go forward" or "turn left") through a microphone, and also receives simple operation commands from the user through touch sensors or buttons. The voice commands are parsed into specific operation commands (such as "turn left" being parsed into a command to turn left) through natural language processing (NLP) technology, and feedback is generated through the voice synthesis module (such as "There is an obstacle ahead, please be careful").

[0069] An NVIDIA Jetson AGX Xavier embedded processor (512-core Volta GPU, 64GB eMMC) was deployed in the robot dog system. A multi-sensor synchronous acquisition framework was established using ROS Melodic: a Basler acA2440-75uc camera (global shutter, 2448×2048@30fps) was connected to a USB 3.0 interface to output raw Bayer RG8 images; a Velodyne VLP-16 LiDAR (16 lines, 300,000 points / second) output XYZIR point clouds via Ethernet; and a hardware-triggered synchronization signal (PPS pulse accuracy ±20ns) was configured, using IEEE 1588. The PTP protocol enables cross-device microsecond-level time synchronization; in the data preprocessing stage, camera data is processed by the ISP pipeline (de-mosaic → gamma correction → automatic white balance) to generate RGB images, and the LiDAR point cloud is filtered by the PCL library's PassThrough filter (Z-axis range 0.1-3m) to remove ground clutter. Both are then transformed using a TF tree (calibration error <0.05°) and input into the multimodal feature extraction module—the visual stream uses OpenCV's FAST detector (threshold 20, non-maximum suppression NMS radius 3 pixels) to extract corner features, while the laser stream uses LOAM's feature selection method (curvature threshold 0.1) to extract edge points. The feature descriptor uses a combination of BRIEF-512 (visual) and FPFH (laser); in the tightly coupled SLAM implementation based on factor graphs, visual inertial odometry ( The VIO (Visual IoV) module uses the VINS-Fusion framework (sliding window length 10 frames, reprojection error threshold 2.5 pixels), and the laser odometry (LO) module uses LIO-SAM (keyframe interval 0.3m / 10°). IMU data (BMI088, 800Hz) is fused using error state Kalman filtering (ESKF). Visual reprojection constraints (Huber kernel function δ=5), laser ICP constraints (maximum matching distance 0.2m), and IMU pre-integration constraints (bias random walk noise 1e-4) are established in the pose graph. The 3D mapping module uses OctoMap (resolution 0.05m) to dynamically update probabilistic voxels and synchronously generate a 2D occupancy grid map (0.02m / pixel). The path planning layer deploys the ROS navigation stack—the global planner uses TimedElastic. The Band (TEB) algorithm (time resolution 0.5s, maximum acceleration 2m / s²) uses the Dynamic Window Method (DWA) to calculate the optimal control values ​​(v_max=1.5m / s, ω_max=1.57rad / s) for local obstacle avoidance, and the path tracking error exceeds 0.Replanning is triggered at a distance of 15m; the user interface integrates the iFlytek XFS5153CE voice module (95% Chinese recognition rate), defining a semantic command set. The mobile terminal publishes NavGoal messages (JSON format including target pose and maximum speed) via WebSocket. The system's real-time visualization uses RViz to display multi-layer maps (point cloud / grid / topology overlay). In typical application scenarios (corridor width 1.2m, dynamic pedestrian density 0.3 people / ㎡), it achieves positioning accuracy of ±3cm (static) and ±8cm (dynamic), path planning response time <200ms, and overall power consumption <25W.

[0070] The system takes as input a 3D environment map (such as a raster map or point cloud map) constructed by SLAM, the user's current location obtained in real time by the SLAM positioning module, the target location specified by the user or automatically generated by the system, and information on static obstacles (such as walls and furniture) and dynamic obstacles (such as pedestrians and vehicles). Then, based on the map type and specific requirements, it selects an appropriate path planning algorithm (such as A* search algorithm, Dijkstra's algorithm, and fast expanding random tree) to generate a planned path from the user's current location to the destination. The voice prompt system generates navigation instructions through voice to provide the user with navigation instructions and environmental information.

[0071] The sensors detect and avoid obstacles in real time. Through camera recognition and machine learning algorithms, the robot dog predicts the movement trajectory of obstacles and adjusts the planned path. It guides the user along the planned path to the destination through voice prompts, vibration feedback, or traction control. High-resolution sensors (such as LiDAR and depth cameras) are used to build a more accurate environmental map to guide the user. Semantic information (such as doors, elevators, and stairs) is added to the map to help the user better understand the environment and prevent the user from going the wrong way.

[0072] In the robot dog remote monitoring system, the embedded main controller adopts a collaborative architecture of NVIDIA Jetson AGX Xavier (equipped with Carmel ARMv8.2 64-bit CPU + 512-core Volta GPU) and Raspberry Pi 4B (Broadcom BCM2711 Cortex-A72). The communication module integrates Intel AX200 WiFi 6 (dual-band 2.4 / 5GHz, MIMO 2×2, peak rate 1.2Gbps) and Nordic nRF52840 Bluetooth 5.2 (supporting Mesh networking). A layered communication framework is established through ROS2 Galactic: 1) The video transmission layer uses the WebRTC protocol (H.265 encoding, GOP=15, bitrate adaptive 5-20Mbps) to achieve 1080P@30fps real-time streaming, and uses the SRT protocol to ensure transmission in weak networks (anti-packet loss rate 30%); 2) The control command layer is based on MQTT. 5.0 (QoS=2) Transmits JSON format messages (including pose coordinates / velocity commands / alarm status), defining control protocol fields such as {"cmd":"nav","target":{"x":5.3,"y":2.7,"theta":0},"speed":0.8,"mode":"safe"}; 3) The VR interaction layer connects to a Raspberry Pi High Quality Camera (12 megapixels, CS-Mount) via Raspberry Pi GPIO. The video stream is encoded with h264_v4l2 and transmitted to the Oculus Quest 2 headset using the NDI|HX protocol (latency <50ms). Synchronized IMU data (100Hz) enables 6DoF view control; The map building module runs the Google Cartographer SLAM algorithm (2D / 3D hybrid mode), and the LiDAR uses RPLIDAR A3 (sampling rate 16kHz, ranging 40m), combined with Intel RealSense. A D455 depth camera (global shutter, depth accuracy ±2%) was used in the mapping phase to classify terrain in real time using a semantic segmentation model (DeepLabv3+, Backbone=MobileNetV3, input size 512×512): tactile paving areas (semantic label ID=1, point cloud coloring is green), flat roads (ID=2, blue), and dangerous areas (slope >15° or height difference >5cm, ID=3, red), generating a multi-layer OctoMap (resolution 3cm, probability update threshold 0.7). Path planning employed the TimeElastic Band algorithm (time dispersion 0.25s, maximum radius of curvature 0).3m), when the guardian modifies the target point through the control APP developed by React Native (supports gesture drawing path points, coordinate transformation accuracy ±5cm), the system triggers dynamic replanning (response time <300ms). If dynamic obstacles are encountered (YOLOv5s detection confidence >0.6), an avoidance path is generated using the artificial potential field method (repulsion coefficient η=0.8). The safety monitoring module integrates the TI BQ40Z80 battery management system (4S lithium battery, capacity 10Ah). When the battery level is <20%, a three-level warning is triggered: 1) Local alarm (Falmec linear motor ERM3036 vibration frequency 5Hz, piezoelectric buzzer emits 85dB@1kHz prompt tone); 2) Remote push Telegram Bot alarm message (including remaining battery level / estimated range); 3) Autonomous return to the charging station (based on AprilTag 36h11 marker positioning, alignment accuracy ±1cm); The positioning module integrates AMCL algorithm (particle number 2000, laser matching weight 0.7) and UWB positioning (Decawave). The DW1000, with 3 base stations and a TDoA accuracy of ±10cm, maintains a positioning error of <8cm (static) / 15cm (dynamic) in complex environments. The system establishes a VPN tunnel (AES-256-GCM encryption) via WireGuard. In case of communication interruption, a local buffer queue is activated (data is cached for up to 15 seconds), triggering a safety stop mechanism (the hub motor gradually decelerates to a stop, with a time constant τ=0.5s). The robot dog generates multiple feasible paths based on user needs and optimizes the selection based on path length, safety, and comfort. It adjusts the path planning in real-time according to environmental changes and user requirements to ensure the user safely reaches their destination. The robot dog uses haptic feedback devices, including vibration motors or a movable handle, to transmit navigation commands through different vibration modes (e.g., short vibration for straight ahead, long vibration for stop) or handle tilt direction (e.g., tilting left for left turn, tilting right for right turn). The robot dog converts the direction information generated by the path planning algorithm into control signals, driving the haptic feedback devices to execute corresponding actions. Users perceive tactile changes through the handheld device, receive navigation commands in real-time, and adjust their walking direction, achieving efficient and intuitive guide interaction for the visually impaired. The robot dog uses camera recognition and machine learning algorithms to classify and identify obstacles. Based on the obstacle type and its movement trajectory, it optimizes avoidance strategies. For static obstacles, it pre-plans detour paths; for moving pedestrians or vehicles, it predicts their trajectories and calculates safe distances, dynamically adjusting the robot dog's walking speed and direction, prioritizing paths away from dangerous areas. Simultaneously, considering the special needs of guide dogs for the visually impaired, the robot dog provides real-time voice prompts and tactile feedback to inform the user of changes in the surrounding environment during obstacle avoidance, ensuring the user's safe avoidance and successful arrival at their destination.

[0073] The robot dog integrates a navigation system, path planning module, and voice prompt system, and is equipped with a camera, LiDAR, and inertial measurement unit (IMU) for environmental perception. The camera acquires RGB images and depth data to identify and label static and dynamic obstacles; the LiDAR generates a 3D point cloud, accurately measures obstacle distances, constructs a high-resolution map, and fuses it with camera data to eliminate misjudgments of dynamic targets; the IMU outputs real-time acceleration, angular velocity, and attitude angle data to compensate for sensor positioning drift and improve SLAM mapping accuracy. A real-time environmental map with dynamic obstacle markers is generated by fusing LiDAR, camera, and IMU data through Kalman filtering.

[0074] The system receives user voice commands, generates the optimal path based on the environmental map and the A* search algorithm, and dynamically adjusts it to avoid moving obstacles. Based on the path planning results, it generates movement, obstacle avoidance, and voice interaction commands. The robot dog executes these commands through wheeled or legged movements, continuously updating the map as it moves. If the user changes their target, the system replans the path and updates the guidance commands. When the lidar detects that the user has deviated from the safe distance, it triggers a voice prompt, "Please come closer to me." By using "semantic segmentation + dynamic obstacle prediction" to achieve dedicated path planning for the blind, it forms a complete technology chain of "sensor data acquisition → environmental modeling → user command parsing → dynamic path generation → multimodal feedback," solving the problem of data acquisition being disconnected from guidance functions in existing technologies, and significantly improving the safety and convenience of travel for the blind.

[0075] This solution utilizes SLAM technology to construct an environmental map and determine the user's current location. Environmental data is simultaneously collected via LiDAR, cameras, and IMU. A multi-sensor fusion SLAM algorithm generates a dynamic map with semantic labels, marking both static and dynamic obstacles. Combining IMU data and LiDAR scan results, a particle filter algorithm achieves centimeter-level positioning. The user sets a target point via voice commands or a mobile device, and the robot dog matches the target in the SLAM map and generates a navigation path. During travel, the SLAM system continuously updates the map, detects temporary obstacles, and generates obstacle avoidance paths. The robot dog guides the user in real time through voice prompts and tactile feedback, ensuring safe travel. This solution achieves high-precision environmental perception and dynamic path planning through SLAM technology, combined with multimodal interaction, solving the problems of real-time obstacle avoidance and user guidance in guide scenarios for the visually impaired, significantly improving the safety and convenience of travel for blind people.

[0076] The robot dog uses a built-in path planning module to generate the optimal path based on a 3D environment map generated by SLAM, the user's current location and destination determined by LiDAR and IMU, and executes an improved A* search algorithm. Preprocessing includes predicting dynamic obstacle trajectories to update the map and setting path weights according to user preferences. The path planning steps are: initialization (current location as the starting point, destination as the ending point), node expansion (searching for nodes based on feasible areas), cost calculation (considering path length, dynamic obstacle risk, and user preferences), and path generation (selecting the sequence of nodes with the minimum cost). Dynamic adjustments include replanning when new obstacles are detected and reinitializing and generating a new path when the user changes their destination. The voice prompt system converts the path into navigation commands, generating voice prompts (direction, obstacle warnings, destination reminders) through a TTS engine and combining haptic feedback to enhance user perception.

[0077] In the specific implementation of the robot dog guide system, the embedded main control uses an NVIDIA Jetson Xavier NX (8-core NVIDIA Carmel ARMv8.2 64-bit CPU + 384-core Volta GPU) and an Intel RealSense D455 depth camera (global shutter, RGB 1280×720@30fps, depth resolution 848×480, accuracy ±2%±2mm) to form the perception unit. A multi-threaded processing framework is established through ROS2 Humble: 1) The obstacle detection thread runs a TensorRT-accelerated YOLOv5m model (input size 640×640, category definition: pedestrian=0 / vehicle=1 / pet=2 / static obstacle=3, confidence threshold 0.65, NMS IoU=0.5), and outputs the detection box and depth point cloud coordinates (converted by cv::reprojectImageTo3D); 2) The trajectory prediction module is based on a PyTorch-deployed LSTM network (structure: input layer [Δx,Δy,vx,vy]×10 frames of historical data → 64-unit hidden layer → output (x,y) coordinates of the next 30 frames). The training data uses the ETH / UCY pedestrian trajectory dataset (with added noise σ=0.1m). Dynamic obstacle avoidance is triggered when the prediction error is <0.3m. 3) The path planning layer integrates the RRT* algorithm of the OMPL library (state space dimension SE(2), sets the collision detection radius expansion coefficient to 0.3m, and generates a detour path at a 50ms cycle when an obstacle is detected to enter the safe area (0.5m buffer zone) (maximum curvature limit 0.25m⁻¹). If there is no feasible path within 1 second, the emergency stop protocol is started (hub motor Kollmorgen). The RGM+EtherCAT bus sends the 0x6041 QuickStop command; the guide vanity feature recognition uses DeepLabv3+ (Backbone=ResNet-18-DeepStem, input 512×512, output semantic segmentation map), the training dataset is a fusion of Cityscapes (labeled 19 classes) and self-collected tactile paving data (5000 frames, LabelMe labeled tactile paving / stairs / ramp), quantized to INT8 precision on Jetson, with an inference speed of 22FPS; the multimodal feedback system includes: a) the voice module uses iFlytek XFS5152CE (Chinese TTS latency <200ms), with a pre-built prompt audio library ("There is a pedestrian 1.5 meters ahead, detouring"); b) tactile feedback uses a TDK Tronics linear motor (GPIO controls PWM duty cycle, frequency 80Hz corresponds to "approach warning", 120Hz corresponds to "emergency stop"); c) traction control is achieved through MaxonEC-4pole. A 30W motor (encoder resolution 2048PPR) adjusts the tension of the guide strap in real time (PID parameter Kp=2).5 / Ki=0.01 / Kd=0.1), applying a reverse force of 3-5N when stairs are detected; SLAM mapping uses ORB-SLAM3 (RGB-D mode + IMU tight coupling), constructing an octree map (OctoMap resolution 5cm, probability update threshold 0.85), the dynamic path planning layer uses the Timed Elastic Band algorithm (time dispersion 0.2 seconds, maximum acceleration 1.5m / s²), combined with a user preference weight matrix (tactile paving priority coefficient 1.5 / flat road surface 1.2 / dangerous area 0.3); maintaining a safe distance is achieved by calculating the Euclidean distance between the user's feet (wearable UWB tag Decawave DWM1001, positioning frequency 50Hz) and the robot dog's centroid in real time using the ICP algorithm, triggering a three-level response when deviating from the threshold ±0.3m: 1) 0.5-0.8 meters: voice prompt "Please keep your distance"; 2) 0.3-0.5 meters: Motor vibration (2 times / s); 3) <0.3 meters: Emergency braking (deceleration slope 2m / s³); The system achieves state machine management through ROS2's LifecycleNode, realizing a dynamic obstacle avoidance response time ≤150ms in complex shopping mall environments (people density 0.5 people / ㎡), a guide path planning success rate ≥98%, and a total power consumption ≤18W (dual-cell 18650 7.4V 6000mAh).

[0078] In the specific implementation of the robot dog remote monitoring and control module, the data acquisition unit adopts a multi-sensor fusion architecture: 1) The positioning data consists of a hybrid positioning system composed of a UWB module (Decawave DWM3000, 6-channel TOF ranging, accuracy ±10cm) and RTK-GPS (ublox ZED-F9P, L1 / L2 band, HPP accuracy ±2cm), which outputs 10Hz pose data through extended Kalman filtering (process noise Q=diag[0.1,0.1,0.05]); 2) The attitude perception integrates a BMI088 six-axis IMU (accelerometer ±16g, gyroscope ±2000dps) and a BMM150 magnetometer, which outputs quaternion attitude through the Madgwick filtering algorithm (β=0.1); 3) The environmental map construction uses an Intel RealSense L515 LiDAR camera (1024×768 depth map) running RTAB-Map. SLAM algorithm (loop detection threshold 0.3, point cloud downsampling voxel 0.05m) generates 3DOctoMap with semantic annotation in real time (classifier uses PointNet++, recognizing 15 features such as tactile paving / stairs / manhole covers); 4) User status monitoring detects user gait cycle (FFT analysis of step frequency / stride length) and UWB wearable tag (5Hz distance measurement) through millimeter wave radar (TI IWR6843, 60GHz, resolution 4cm) and triggers three-level warnings (thresholds: 0.5m / yellow, 1m / orange, 1.5m / red) when there is abnormal deviation. The communication unit constructs dual-link redundancy: the main link uses a 5G module (Quectel RM500Q, Sub-6GHz, peak rate 3.5Gbps) to transmit H.265 video streams (1080P@25fps, GOP=12), and the backup link uses LoRa (Semtech SX1280, 915MHz, SF=10) to transmit critical status data (100ms cycle). Data encryption adopts dual encryption of TLS 1.3 protocol (ECDHE-ECDSA-AES256-GCM-SHA384) and national cryptographic SM4-CBC mode. When the network is interrupted, a local ring buffer (RocksDB storage engine, capacity 8GB, maximum cache of 30 minutes of data) is activated.The monitoring application is developed based on the Flutter framework to create a cross-platform control interface. Its core functions include: a) 3D map visualization using the CesiumJS engine overlaid with real-time SLAM point clouds (Three.js rendering pipeline); AR annotations using ARKit 5.0 for spatial anchoring (plane detection accuracy ±2cm); and hazardous area annotations using red semi-transparent cubes (size adaptive); b) The path planning interface supports gesture-based Bézier curve drawing (control points are draggable, coordinate transformation uses a Homography matrix); path parameter settings include maximum speed (adjustable in stages from 0.1-1.5m / s), safety distance (adjustable from 0.3-1m), and obstacle avoidance modes (conservative / balanced / aggressive levels); c) Control mode switching logic: When emergency obstacle avoidance mode is activated, D* Lite global replanning is triggered (update frequency 5Hz), and the maximum angular velocity is limited to 0.5rad / s. In low-speed following mode (0.3m / s), a human body following algorithm (DeepSORT tracking + Kalman prediction) is enabled. Two-way voice communication uses Opus encoding (8kHz sampling, 6-16kbps adaptive bit rate) to achieve end-to-end latency <300ms. Echo cancellation is achieved in conjunction with the WebAudio API (AEC algorithm convergence time 20ms). The system establishes a two-way data channel through gRPC-Web (protobuf protocol defines ControlCommand message body with 32-bit CRC check). When network jitter is detected (packet loss rate >15% for 2 seconds), it automatically degrades to instruction buffer mode (local execution queue depth 50 lines) and activates watchdog timer (period 1s) to ensure process liveness. The security mechanism includes multiple layers of protection: 1) Hardware-level secure boot (TrustZone technology verifies the boot image using SHA-256); 2) Data integrity verification (HMAC-SHA256 signature for each frame of data); 3) Operation audit log (WAL mode records all control commands); In typical urban road scenarios (5G signal strength -85dBm, 30% obstruction), the system achieves end-to-end control command latency ≤180ms, video stream keyframe recovery time <500ms, emergency braking response time ≤80ms, and a packet loss compensation success rate ≥99.5% in complete guide tasks for the blind. The robot dog's camera collects real-time video data of the surrounding environment and transmits the video data to a remote monitoring and control module for the guardian to view. When obstacles or low battery are detected, the robot dog sends a warning message to the remote monitoring and control module and takes safety measures, including: the robot dog immediately stops to avoid collisions, issues a voice prompt through the built-in speaker to remind the user to pay attention to obstacles or low battery, activates a low battery mode to extend battery life, sends a warning message to the guardian through the remote monitoring and control module, and replans the route based on environmental data to avoid obstacles or guide the user to the nearest charging point or safe area.

[0079] The robot dog uses SLAM technology to build an environmental map. The environmental map not only includes obstacles and walkable areas, but also marks key information that blind users may be interested in, such as stairs, ramps, tactile paving, entrances and exits, and potential danger zones. At the same time, the robot dog uses a positioning module to accurately locate itself on the environmental map and dynamically optimizes the path planning based on the user's actual needs (such as walking speed and obstacle avoidance priority), ensuring the safety, convenience, and comfort of the path. This is different from conventional navigation, which only aims at the shortest path or the fastest arrival, and is more in line with the special needs of the blind community.

[0080] The positioning module employs Kalman filtering to fuse inertial measurement and encoder data for pose noise-resistant estimation. It combines particle filtering with dynamic weighted particle swarm analysis to calculate the optimal position in complex environments and utilizes visual odometry to track indoor displacement through image feature analysis. The algorithm synchronously associates with pre-stored environmental map data (including markings of tactile paving, obstacles, etc.), prioritizing accessible paths and optimizing positioning accuracy based on guidance needs. When an obstacle is detected, it automatically switches to safe tactile paving navigation and triggers voice prompts to ensure user safety.

[0081] The robot dog generates multiple feasible paths based on user needs and comprehensively evaluates path length, safety, and comfort using a multi-objective optimization algorithm. Path length is optimized through geometric distance, prioritizing shorter paths to reduce walking time; safety is optimized by assessing obstacle density, dangerous area distribution, and dynamic obstacle probability, prioritizing paths with fewer obstacles and lower risk; comfort is optimized by analyzing path flatness, turning frequency, and compliance with tactile paving, prioritizing flat, straight paths that suit the habits of blind people.

[0082] The robot dog monitors environmental changes in real time using sensors and dynamically adjusts its path planning. For example, it replans the path when it detects a moving obstacle and reminds the user through voice prompts, ensuring the user reaches their destination safely and comfortably. This fully integrates with the needs of guide dogs, improving the practicality and adaptability of path planning. The robot dog transmits directional information to the user through haptic feedback devices. These devices include vibration motors or movable handles, which transmit navigation commands through different vibration modes (e.g., short vibrations indicate going straight, long vibrations indicate stopping) or handle tilt directions (e.g., tilting left indicates turning left, tilting right indicates turning right). The robot dog converts the directional information generated by the path planning algorithm into control signals, driving the haptic feedback devices to execute corresponding actions. The user perceives tactile changes through the handheld device, receives navigation commands in real time, and adjusts their walking direction, achieving efficient and intuitive guide dog interaction. The robot dog uses camera recognition and machine learning algorithms to classify and identify obstacles. Based on the type and trajectory of the obstacle, it optimizes avoidance strategies. For static obstacles, it pre-plans detour paths. For moving pedestrians or vehicles, it predicts their trajectories and calculates safe distances, dynamically adjusting the robot dog's walking speed and direction, prioritizing paths away from dangerous areas. At the same time, considering the special needs of guide dogs, the robot dog uses voice prompts and tactile feedback to inform users of changes in the surrounding environment in real time during the avoidance process, ensuring that users can safely avoid obstacles and reach their destination smoothly.

[0083] Although embodiments of the present invention have been described above in conjunction with the accompanying drawings, the present invention is not limited to the specific embodiments and application fields described above. The specific embodiments described above are merely illustrative and instructive, and not restrictive. Those skilled in the art can make many other forms based on the guidance of this specification and without departing from the scope of protection of the claims of the present invention, and all of these are within the scope of protection of the present invention.

Claims

1. A path planning and guidance method for the blind based on a robot dog, characterized in that, Includes the following steps: The robot dog is equipped with a navigation system, a path planning module, and a voice prompt system for user interaction. The robot dog is equipped with sensors for detecting the surrounding environment, including cameras, lidar, and inertial measurement units. After the robot dog is started, the onboard sensors collect information about the surrounding environment in real time. The system utilizes SLAM technology to construct an environmental map. Kalman filtering fuses surrounding environmental information collected by LiDAR, cameras, and inertial measurement units to generate a real-time environmental map with dynamic obstacle markers and determine the user's current location. The user can then set waypoints and destinations based on the environmental map through voice interaction or mobile devices. Based on the environmental map, the user's current location, waypoints, and destination, the robot dog executes a path planning algorithm through its built-in path planning module to generate a planned path from the user's current location to the destination. The voice prompt system generates navigation instructions through voice to provide the user with navigation instructions and environmental information. The sensors detect and avoid obstacles in real time. Through camera recognition and machine learning algorithms, the robot dog predicts the movement trajectory of obstacles and adjusts the planned path. It guides the user to walk along the planned path through voice prompts, vibration feedback, or traction control to guide the user to the destination.

2. The path planning and guidance method for the blind based on a robot dog according to claim 1, characterized in that, Preferably, the robot dog is equipped with a remote monitoring and control module.

3. The path planning and guidance method for the blind based on a robot dog according to claim 2, characterized in that, After setting waypoints and destinations, guardians can modify the selection of waypoints and destinations via mobile devices or computers.

4. The path planning and guidance method for the blind based on a robot dog according to claim 3, characterized in that, The robot dog is equipped with a camera that collects video data of the surrounding environment in real time.

5. A path planning and guidance method for the blind based on a robot dog according to claim 1, characterized in that, The robot dog uses SLAM technology to build an environmental map, which includes obstacles, walkable areas, stairs, ramps, tactile paving, entrances and exits, and potential danger zones. The robot dog uses a positioning module to locate itself on the environmental map.

6. A path planning and guidance method for the blind based on a robot dog according to claim 5, characterized in that, The positioning algorithm of the positioning module includes Kalman filtering, particle filtering, and visual odometry.

7. A path planning and guidance method for the blind based on a robot dog according to claim 1, characterized in that, Path planning algorithms include A* search algorithm, Dijkstra algorithm, and deep reinforcement learning algorithm.

8. A path planning and guidance method for the blind based on a robot dog according to claim 1, characterized in that, The robot dog generates multiple feasible paths based on user requirements.

9. A path planning and guidance method for the blind based on a robot dog according to claim 1, characterized in that, The robot dog transmits directional information to the user through a haptic feedback device.

10. A path planning and guidance method for the blind based on a robot dog according to claim 1, characterized in that, The robot dog uses camera recognition and machine learning algorithms to classify and identify obstacles, and optimizes avoidance strategies based on the type and trajectory of the obstacles.

Images