Working method and system of blind guiding robot based on laser radar and camera

Abstract

The invention provides a working method of a blind guiding robot based on a laser radar and a camera, and belongs to the technical field of mobile service robots. Accurate space structure information is provided through three-dimensional point cloud data, dynamic traffic characteristics such as traffic lights and zebra crossings are captured through visual images, and comprehensiveness of environmental perception is achieved through complementation of the space structure information and the visual images; through combination of environment information and traffic characteristics, passable areas and barrier distribution in a map are dynamically adjusted, dynamic changes such as traffic light switching and pedestrian movement are adapted, and a navigation route can be updated in real time; the characteristics of traffic light states, zebra crossing and the like are fused into path planning, it is ensured that the robot complies with traffic rules, real-time position and speed constraint, dynamic adjustment of motion trails and efficient obstacle avoidance can be carried out on the robot, and the robot can make a decision and optimize a path autonomously according to human social rules.

Description

Technical Field

[0001] The present invention belongs to the technical field of mobile service robots, and particularly relates to a working method and system of a guide robot based on lidar and a camera. Background Art

[0002] With the continuous improvement of the scientific and technological level, the demand of visually impaired people for safe and convenient travel is showing a significant growth trend. Against this background, guide robot technology has gradually become a research hotspot and focus in the scientific research field due to its unique advantage of being able to provide real-time environment perception and navigation assistance for visually impaired people.

[0003] Currently, the guide devices on the market mainly rely on relatively traditional and single technical means such as ultrasonic sensors, infrared sensors or monocular cameras for environmental perception. However, these devices have exposed a series of problems that need to be solved urgently in practical applications: Traditional guide devices, such as common ultrasonic guide canes, are only capable of detecting obstacles within a short distance due to their working principles. When facing complex and critical traffic signs such as traffic lights and zebra crossings, such devices often seem powerless and are unable to accurately identify and give corresponding prompts to users. More seriously, when facing dynamic obstacles such as pedestrians and vehicles, the tracking ability of traditional guide devices is significantly insufficient, and it is difficult to continuously and stably monitor their dynamic changes, thus bringing great potential safety hazards to the travel safety of users.

[0004] In terms of positioning technology, most of the existing guide solutions adopt inertial navigation or GPS positioning methods. However, both of these positioning methods have certain limitations in practical applications. During long-term operation, the inertial navigation system is prone to positioning drift due to the accumulation of sensor errors; while GPS positioning has a significant decrease in positioning accuracy or even may result in positioning failure in special environments such as indoors or urban canyons due to signal occlusion or interference. These positioning problems will not only affect the accurate judgment of the position of the guide device for users, but also further seriously affect the accuracy of path planning, making it difficult for users to reach the destination safely and efficiently according to the expected route.

[0005] In addition, traditional path planning algorithms have also shown obvious deficiencies when dealing with complex and changing dynamic environments. Such algorithms usually have a high computational complexity, consuming a large amount of computing resources and time, making it difficult to achieve real-time and efficient path planning in practical applications. More critically, in the case of dynamic obstacles, the external environment changes rapidly, which may cause the decision-making device to have difficulty making decisions and thus fall into a state of stagnation. They often cannot make reasonable decisions based on traffic rules, which may lead to the guide robot violating traffic rules, such as running red lights and going against the traffic flow, thus bringing great safety risks to users.

[0006] Therefore, there is an urgent need for a guide robot solution that integrates multi-sensor data, has high-precision semantic environment understanding ability, and can optimize the path in real time. Summary of the Invention

[0007] The purpose of the present invention is to overcome the above deficiencies and provide a working method and system for a guide robot based on lidar and camera.

[0008] To achieve the above purpose, the present invention adopts the following technical solutions: In the first aspect, the present invention provides a working method for a guide robot based on lidar and camera, including the following steps: Obtain environmental information, including three-dimensional point cloud data and visual image data; Use the YOLOv5 model to extract traffic features of traffic lights and zebra crossings from visual image data in real time; Based on the obtained environmental information and traffic features, construct a dynamic environmental map model; Determine the real-time position of the robot through the Adaptive Monte Carlo Localization algorithm, and generate a global path based on the dynamic environmental map model and the real-time position of the robot; Based on the global path and the real-time position of the robot, combine the traffic features extracted in real time to optimize obstacle avoidance for the global path and generate a local path; Generate the robot's motion path based on the local path and the dynamic window method.

[0009] In the step of obtaining environmental information, including three-dimensional point cloud data and visual image data, obtain the three-dimensional point cloud data of the environment through lidar, and obtain the visual image data of the environment based on a depth camera.

[0010] In the step of constructing a dynamic environmental map model based on the obtained environmental information and traffic features, the specific method is as follows: Construct the motion equation and observation equation of the robot through the Visual SLAM model; (1) Wherein, Indicates the pose of the robot at moment, Indicates the motion input, Indicates the noise, and the motion equation Indicates the pose of the robot at The pose at the moment is determined by The pose at the moment and The motion input at the moment , and is described by the mapping of the general function ; Indicates the observation data of the robot's observation of the environmental feature at the moment, and Indicates the noise generated during the observation.

[0011] Introduce maximum likelihood estimation, and infer the robot's pose and environmental feature based on the observation data and motion input , and obtain the optimal value of the dynamic environment map model parameters through maximum likelihood estimation.

[0012] In the step of determining the real-time position of the robot through the adaptive Monte Carlo localization algorithm and generating a global path based on the dynamic environment map model and the real-time position of the robot, the specific method is as follows: Obtain the state data of the robot, including motion speed, acceleration, and motion direction; Take the obtained robot state data as the initial particle swarm, and predict the pose distribution of the initial particle swarm at the next moment through the state transition equation to obtain the predicted particle swarm; Utilize the real-time traffic features of the lidar and depth camera, combine with the pre-constructed dynamic environment map model, calculate the matching probability of each particle in the predicted particle swarm, and update the particle weights using the feature correlation information in the environmental information to reflect their position credibility; Perform importance sampling according to the weight distribution of the current observed particle swarm, retain the high-weight particles and eliminate the low-weight particles, and at the same time introduce an adaptive mechanism to dynamically adjust the number of particles to balance the calculation efficiency and positioning accuracy; The concentrated area of the high-weight particles is used as the position of the robot; Based on the position of the robot, use the A* algorithm in the global path planner to plan a path within 3 to 5 meters as the global path, and update the global path at frequency; Among them, when the predicted position distribution of the initial particle swarm at the next moment is exactly the same as the actual operation result of the robot, each particle in the particle swarm can reflect the true position of the robot, and the error is 0. At this time, each particle has the same weight, which is 1 / M, where M is the total number of particles. In the case of observation, high-weight particles refer to particles with weights greater than or equal to 2 / M, and particles with weights less than 2 / M are low-weight particles and are eliminated.

[0013] In the step of optimizing obstacle avoidance for the global path based on the global path and the real-time position of the robot, and generating a local path by combining the traffic features extracted in real time, the local path planner uses the D* algorithm to optimize obstacle avoidance for the global path in combination with the traffic features extracted in real time to obtain the local path.

[0014] In the step of generating the robot motion path based on the local path and the dynamic window method, the specific method is as follows: Using the dynamic window method, the speed space of the robot's motion is divided into safe and unsafe regions. The safe region is the reachable speed range of the robot. The following evaluation function is defined: (8) Among them, is a smoothing function; is a weighting coefficient; represents the difference in azimuth angles between the actual orientation of the robot and the target orientation; represents the distance between the robot and the target point; represents the distance difference between the end of the local path planning and the global path planning. The smaller this distance is, the more the local path planning conforms to the global path planning; represents the distance between the end of the local path planning and the obstacle. If the distance from the robot's trajectory to the obstacle is greater than the radius of the robot, there is no danger of collision; otherwise, this trajectory is discarded; Through weighted calculation, make value the smallest, obtain the optimal path, linear velocity scheme and angular velocity scheme, and the robot moves based on the linear velocity scheme and angular velocity scheme.

[0015] In the second aspect, the present invention provides a guide dog robot system based on a lidar and a camera, including: An environmental information acquisition module: used to acquire environmental information, including three-dimensional point cloud data and visual image data; A traffic feature extraction module, used to perform real-time extraction of traffic features such as traffic lights and zebra crossings on the visual image data through the YOLOv5 model; A dynamic environmental map model construction module, used to construct a dynamic environmental map model based on the acquired environmental information and traffic features; A global path generation module, which is used to determine the real-time position of the robot through an adaptive Monte Carlo localization algorithm, and generate a global path based on the dynamic environment map model and the real-time position of the robot; A local path generation module, which optimizes obstacle avoidance for the global path based on the global path and the real-time position of the robot, and combines the traffic features extracted in real time to generate a local path; A motion path generation module, which generates a robot motion path based on the local path and the dynamic window method.

[0016] It includes a retractable flexible tow rope, on which a photosensitive sensor and a force sensor are arranged. The photosensitive sensor is used to detect whether the blind person holds the tow rope tightly, and the force sensor is used to detect the pulling force of the tow rope.

[0017] It includes a voice control module, which is used to start and stop the robot and receive instructions issued by the blind person.

[0018] Compared with the prior art, the present invention has the following beneficial effects: The present invention provides a working method for a blind guiding robot based on lidar and a camera, including the following steps: obtaining environmental information, including three-dimensional point cloud data and visual image data; extracting traffic features of traffic lights and zebra crossings in real time from the visual image data through the YOLOv5 model; constructing a dynamic environment map model based on the obtained environmental information and traffic features; determining the real-time position of the robot through an adaptive Monte Carlo localization algorithm, and generating a global path based on the dynamic environment map model and the real-time position of the robot; optimizing obstacle avoidance for the global path based on the global path and the real-time position of the robot, and combining the traffic features extracted in real time to generate a local path; generating a robot motion path based on the local path and the dynamic window method. Precise spatial structure information is provided through the three-dimensional point cloud data, and dynamic traffic features (such as traffic lights and zebra crossings) are captured by the visual image. The two complement each other to achieve the comprehensiveness of environmental perception; combining the environmental information and traffic features, the passable areas and obstacle distributions in the map are dynamically adjusted to adapt to dynamic changes such as traffic light switching and pedestrian movement, and the navigation route can be updated in real time; features such as traffic light states and zebra crossings are incorporated into the path planning to ensure that the robot complies with traffic rules, the real-time position and speed of the robot can be constrained, and the motion trajectory can be dynamically adjusted to achieve efficient obstacle avoidance, so that the robot can make autonomous decisions and autonomously optimize the route in accordance with the rules of human society.

[0019] Furthermore, the present invention relies on the point cloud map of the surrounding environment constructed by the lidar and makes decisions by integrating the special environmental information extracted by the depth camera. The lidar point cloud can quickly update the positions of obstacles, and the depth camera can track dynamic targets in real time to achieve autonomous navigation and obstacle avoidance, guiding the blind to travel.

[0020] Furthermore, the dynamic window method selects the optimal trajectory by evaluating the collision risks of different trajectories (such as obstacle distance, speed direction), and is applicable to scenarios where multiple obstacles are parallel or intersecting. The dynamic window method combines local path planning to ensure that the robot always faces the global target direction and avoids getting lost due to obstacle avoidance. Brief Description of the Drawings

[0021] Figure 1 It is a flowchart of the method of the present invention; Figure 2 It is a diagram of the initial state setting of the Matlab simulation in the present invention; Figure 3 It is a simulation trajectory diagram of the adaptive Monte Carlo localization algorithm in the present invention; Figure 4 It is an error curve of the adaptive Monte Carlo localization algorithm in the present invention; Figure 5 It is a system structure diagram of the present invention. Detailed Embodiment

[0022] To further understand the content of the present invention, the following describes the present invention in detail with reference to the accompanying drawings and specific embodiments. It should be understood that the embodiments are only for explaining the present invention and not for limiting it.

[0023] Embodiment 1 As Figure 1 shown, a working method of a guide robot based on lidar and camera includes the following steps: S1: Obtain environmental information, including three-dimensional point cloud data and visual image data; S2: Use the YOLOv5 model to extract traffic features of traffic lights and zebra crossings from the visual image data in real time; S3: Based on the obtained environmental information and traffic features, construct a dynamic environmental map model; S4: Determine the real-time position of the robot through the adaptive Monte Carlo localization algorithm, and generate a global path based on the dynamic environmental map model and the real-time position of the robot; S5: Based on the global path and the real-time position of the robot, optimize obstacle avoidance for the global path by combining the traffic features extracted in real time, and generate a local path; S6: Generate a robot motion path based on the local path and the dynamic window method.

[0024] Specifically, in S1, three-dimensional point cloud data of the environment is obtained based on lidar, and visual image data of the environment is obtained based on a depth camera.

[0025] Specifically, in S2, the YOLOv5 model is used to extract traffic features of traffic lights and zebra crossings from the visual image data in real time.

[0026] Specifically, in S3, based on the obtained environmental information and traffic characteristics, a dynamic environmental map model is constructed through the SLAM model. The pose and motion state of the robot are reflected by the motion equation and the pose equation, which serves as the dynamic environmental map model. The specific method is as follows: Construct the motion equation and the observation equation of the SLAM model: (1) Among them, represents the pose of the robot at moment, represents the motion input, represents the noise. The motion equation represents that the pose of the robot at moment can be described by the mapping of the pose at and the motion input at through the general function represents the observation data of the robot observing the environmental feature at moment, represents the noise generated during the observation process.

[0027] Introduce the maximum likelihood estimation. According to the observation data and the motion input infer the robot pose and the environmental feature , and obtain the optimal value of the dynamic environmental map model parameters through the maximum likelihood estimation. The specific method is as follows: Assume that the noise term follows a Gaussian distribution, that is , and the conditional probability of the observation data is: (2) Among them, represents that the noise follows a Gaussian distribution with a mean of 0 and a variance of , represents the variance of the noise distribution.

[0028] Calculate the maximized by minimizing the negative logarithm equation. Considering any high-dimensional Gaussian distribution , its probability density function expansion form is: (3) Take the negative logarithm of Equation (3) to get: (4) Among them, Nis the number of dimensions, representing the dimensionality of the space where the Gaussian distribution lies, i.e., the length of the random variable vector ; is the determinant of the covariance matrix, which is used to reflect the influence of the correlation between dimensions of multi-dimensional data on the distribution range; is the covariance matrix. The diagonal elements of the matrix represent the covariance of the same dimension, and the non-diagonal elements represent the covariance between different dimensions; is the mean vector, which is a N -dimensional vector representing the expected value of the Gaussian distribution.

[0029] Substitute Equation (4) into the observation equation to obtain (5) Equation (5) is equivalent to minimizing the square of the noise term. Therefore, for all motions and any observations, the error between the actual value and the estimated value of the data is defined as: (6) The sum of the squares of the errors in Equation (6) is: (7) where represents the optimal estimated value after observation by the observation equation and maximum likelihood estimation, represents the inverse matrix of the variance matrix of the multi-dimensional Gaussian distribution, represents the error between the actual value and the estimated value of the observed data, represents the inverse matrix of the covariance matrix, represents the observation residual, which reflects the inconsistency between the actual measurement value and the model prediction value.

[0030] Thus, a least squares estimate in the overall sense is obtained, and its optimal solution is equivalent to the maximum likelihood estimate of the state.

[0031] Specifically, in S4, the real-time position of the robot is determined by the adaptive Monte Carlo localization algorithm, and a global path is generated based on the dynamic environment map model and the real-time position of the robot. The specific method is as follows: Obtain the state data of the robot, including motion speed, acceleration, and motion direction; Take the obtained robot state data as the initial particle swarm, and predict the position distribution of the initial particle swarm at the next moment through the state transition equation to obtain the predicted particle swarm; Utilize the real-time traffic features of the lidar and depth camera, combine with the pre-constructed dynamic environment map model, calculate the matching probability of each particle in the predicted particle swarm, and adopt the feature correlation information in the environmental information to update the particle weights to reflect their position credibility; Perform importance sampling according to the weight distribution of the currently observed particle swarm, retain high-weight particles and eliminate low-weight particles, and at the same time introduce an adaptive mechanism to dynamically adjust the number of particles to balance the computational efficiency and positioning accuracy; The concentrated area of high-weight particles is used as the position of the robot; Based on the position of the robot, use the A* algorithm through the global path planner to plan a path within 3 to 5 meters as the global path, and Update the global path at a certain frequency.

[0032] Among them, when the predicted position distribution of the initial particle swarm at the next moment is exactly the same as the actual operation result of the robot (that is, the matching probability is 100%), each particle in the particle swarm can reflect the true position of the robot, the error is 0, and each particle has the same weight, that is, the weights of the particles in the particle swarm should be evenly distributed, which is 1 / M, where M is the total number of particles. In the case of observation, high-weight particles refer to particles with weights greater than or equal to 2 / M, and particles with weights less than 2 / M are low-weight particles and are eliminated.

[0033] High-weight particles are obtained through the following methods: 1. Sort the weights of the particles in all observed particle swarms from high to low; 2. By selecting M / 2 points among them (or any other number of points, but preferably between 1 / 2 and 3 / 5 of the total), in addition to using the method of taking out a fixed total number to obtain high-weight particles, it is also possible to select according to the weight size of the particles. Usually, 2-3 times of 1 / M is used as the threshold. Particles higher than this threshold are considered high-weight particles, and the rest of the particles are eliminated.

[0034] To verify the performance of the algorithm, this study constructed a robot kinematics and observation model on the MATLAB platform, carried out multi-scenario positioning simulation experiments, and obtained the true trajectory and positioning trajectory of the robot and its error image, as Figures 2 - 4 shown. Through the above simulation, the positioning error is within .

[0035] Specifically, in S5, based on the global path and the real-time position of the robot, the D* algorithm is used through the local path planner to optimize the obstacle avoidance of the global path by combining the traffic features extracted in real time to obtain the local path.

[0036] Specifically, in S6, the method for generating the robot motion path based on the local path and the dynamic window method is as follows: Using the dynamic window method, divide the speed space of the robot's motion into safe and unsafe areas. The safe area is the reachable speed range of the robot, and define the evaluation function: (8) Among them, is a smoothing function. Set to 1, that is, do not smooth the evaluation function; is a weighting coefficient; represents the azimuth difference between the actual orientation of the current robot and the target orientation (the orientation of the robot's motion path). The smaller the difference, the smaller will be; represents the distance between the robot and the target point (the end point of the robot's motion path). The smaller the distance, the smaller will be; represents the distance difference between the end of the current local path planning and the global path planning. The smaller this distance, the more the local path planning conforms to the global path planning. At this time,

[0037] the value of will be minimized, that is, the optimal path, linear velocity scheme, and angular velocity scheme. The robot moves based on the linear velocity scheme and angular velocity scheme.

[0038] Embodiment 2 As Figure 5 shown, a blind guiding robot system based on lidar and camera includes: An environmental information acquisition module: used to acquire environmental information, including three-dimensional point cloud data and visual image data; A traffic feature extraction module, used to extract traffic features such as traffic lights and zebra crossings from visual image data in real time through the YOLOv5 model; A dynamic environmental map model construction module, used to construct a dynamic environmental map model based on the acquired environmental information and traffic features; A global path generation module, used to determine the real-time position of the robot through the adaptive Monte Carlo localization algorithm, and generate a global path based on the dynamic environmental map model and the real-time position of the robot; A local path generation module, used to optimize obstacle avoidance for the global path based on the global path and the real-time position of the robot, and combine the traffic features extracted in real time to generate a local path; A motion path generation module, used to generate the robot's motion path based on the local path and through the dynamic window method.

[0039] Furthermore, the robot system includes a retractable flexible tow rope. A photosensitive sensor and a force sensor are arranged on the flexible tow rope. The photosensitive sensor is used to detect whether the blind person holds the tow rope, and the force sensor is used to detect the pulling force of the tow rope.

[0040] Furthermore, the robot system includes a voice control module for starting and stopping the robot and receiving instructions issued by the blind.

[0041] Furthermore, the robot adopts a four-legged wheeled structure chassis, which converts the angular velocity and linear velocity calculated by the motion path generation module into the rotational speed and angular velocity of the four-legged wheel type, driving the robot to move along the motion path and complete the navigation and obstacle avoidance tasks.

[0042] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit them. Although the present invention has been described in detail with reference to the above embodiments, those of ordinary skill in the art should understand that: the specific implementation manners of the present invention can still be modified or equivalently replaced, and any modification or equivalent replacement that does not depart from the spirit and scope of the present invention shall be covered by the protection scope of the claims of the present invention.

Claims

1. A working method of a guide robot based on lidar and camera, characterized in that, It includes the following steps: Obtain environmental information, including 3D point cloud data and visual image data; Use the YOLOv5 model to extract traffic features of traffic lights and zebra crossings from visual image data in real time; Construct a dynamic environmental map model based on the obtained environmental information and traffic features; Determine the real-time position of the robot through the adaptive Monte Carlo localization algorithm, and generate a global path based on the dynamic environmental map model and the real-time position of the robot; Optimize and avoid obstacles for the global path based on the global path and the real-time position of the robot, combined with the traffic features extracted in real time, to generate a local path; Generate the robot's motion path based on the local path and the dynamic window method.

2. The working method of a blind guiding robot based on lidar and camera according to claim 1, characterized in that, In the step of obtaining environmental information, including 3D point cloud data and visual image data, obtain the 3D point cloud data of the environment through a lidar, and obtain the visual image data of the environment based on a depth camera.

3. The working method of a blind guiding robot based on lidar and camera according to claim 1, characterized in that, In the step of constructing a dynamic environmental map model based on the obtained environmental information and traffic features, the specific method is as follows: Construct the motion equation and observation equation of the robot through a visual SLAM model; （1） Among them, represents the pose of the robot at moment, represents the motion input, represents the noise, and the motion equation represents that the pose of the robot at moment is determined by the pose at and the motion input at moment, through the mapping of the general function ; represents the observation data of the robot's observation of the environmental feature at moment, represents the noise generated during the observation.

4. The working method of a guide robot based on lidar and camera according to claim 3, characterized in that, Introduce the maximum likelihood estimation, and infer the robot pose and the motion input to infer the robot pose and environmental features , and obtain the optimal values of the dynamic environment map model parameters through the maximum likelihood estimation.

5. The working method of a guiding blind robot based on lidar and camera according to claim 1, characterized in that, In the step of determining the real-time position of the robot through the adaptive Monte Carlo localization algorithm and generating a global path based on the dynamic environmental map model and the real-time position of the robot, the specific method is as follows: Obtain the state data of the robot, including motion speed, acceleration, and motion direction; Use the obtained state data of the robot as the initial particle swarm, and predict the position distribution of the initial particle swarm at the next moment through the state transition equation to obtain the predicted particle swarm; Utilize the real-time traffic features of the lidar and depth camera, combined with the pre-constructed dynamic environmental map model, calculate the matching probability of each particle in the predicted particle swarm, and use the feature correlation information in the environmental information to update the particle weights to reflect their position credibility; Perform importance sampling according to the weight distribution of the current observed particle swarm, retain high-weight particles and eliminate low-weight particles, and at the same time introduce an adaptive mechanism to dynamically adjust the number of particles to balance the calculation efficiency and positioning accuracy; The concentrated area of high-weight particles is used as the position of the robot; Based on the position of the robot, the global path planner uses the A* algorithm to plan a path within 3 to 5 meters as the global path, and updates the global path at frequency; Among them, when the position distribution of the predicted initial particle swarm at the next moment is completely consistent with the actual operation result of the robot, each particle in the particle swarm can reflect the real position of the robot, and the error is 0. At this time, each particle has the same weight, which is 1 / M, where M is the total number of particles. In the case of observation, high-weight particles refer to particles with weights greater than or equal to 2 / M, and particles with weights less than 2 / M are low-weight particles and are eliminated.

6. The working method of a guiding blind robot based on lidar and camera according to claim 1, characterized in that, In the step of optimizing and avoiding obstacles for the global path based on the global path and the real-time position of the robot, combined with the traffic features extracted in real time, to generate a local path, use a local path planner to optimize and avoid obstacles for the global path through the D* algorithm combined with the traffic features extracted in real time to obtain a local path.

7. The working method of a guiding blind robot based on lidar and camera according to claim 1, characterized in that, In the step of generating the robot's motion path based on the local path and the dynamic window method, the specific method is as follows: Adopt the dynamic window method to divide the speed space of the robot's motion into safe and unsafe regions. The safe region is the reachable speed range of the robot, and define the following evaluation function: （8） Among them, is a smoothing function; is a weighting coefficient; represents the azimuth difference between the actual orientation of the robot and the target orientation; represents the distance between the robot and the target point; represents the distance difference between the end of the local path planning and the global path planning. The smaller this distance is, the more the local path planning conforms to the global path planning; represents the distance between the end of the local path planning and the obstacle. If the distance from the robot's trajectory to the obstacle is greater than the radius of the robot, there is no danger of collision; otherwise, discard this trajectory; Through weighted calculation, make the value minimum, obtain the optimal path, linear velocity scheme and angular velocity scheme, and the robot moves based on the linear velocity scheme and angular velocity scheme.

8. A blind guiding robot system based on lidar and camera, characterized in that, It includes: Environmental information acquisition module: used to acquire environmental information, including 3D point cloud data and visual image data; Traffic feature extraction module, used to extract traffic features of traffic lights and zebra crossings in real time from visual image data through the YOLOv5 model; Dynamic environment map model construction module, used to construct a dynamic environment map model based on the acquired environmental information and traffic features; Global path generation module, used to determine the real-time position of the robot through the adaptive Monte Carlo localization algorithm, and generate a global path based on the dynamic environment map model and the real-time position of the robot; Local path generation module, based on the global path and the real-time position of the robot, combines the traffic features extracted in real time to optimize obstacle avoidance for the global path and generate a local path; Motion path generation module, generates the robot motion path based on the local path and the dynamic window method.

9. The blind guiding robot system based on lidar and camera according to claim 8, wherein, It includes a retractable flexible towing rope, on which a photosensitive sensor and a force sensor are arranged. The photosensitive sensor is used to detect whether the blind person holds the towing rope tightly, and the force sensor is used to detect the pulling force of the towing rope.

10. The blind guiding robot system based on lidar and camera according to claim 8, characterized in that, It includes a voice control module, used to start and stop the robot and receive instructions issued by the blind person.

Images