Robot high-speed navigation method based on event camera and laser radar

Abstract

The invention belongs to the field of navigation and obstacle avoidance in the field of robots, and discloses a robot high-speed navigation method based on an event camera and a laser radar. The robot carries an event camera, a two-dimensional laser radar and a speedometer sensor for sensing the environment; an event camera obtains an event frame in a current time window, robot motion state information is introduced, and a motion compensation event frame only containing a dynamic obstacle is obtained through compensation calculation; inputting the motion compensation event frame into a pre-trained event frame encoder to obtain a feature vector, splicing the feature vector with laser radar data and robot motion state data, and inputting the spliced feature vector into a reinforcement learning network; and the reinforcement learning network outputs a two-dimensional vector which represents the linear speed of the left and right wheels of the robot as a decision action. According to the invention, static objects and moving objects in the environment are distinguished, dynamic environment information is introduced for a traditional navigation scheme based on laser radar, and the system can still run well in an unfamiliar environment.

Description

Technical Field

[0001] The present invention relates to the field of navigation and obstacle avoidance in the field of robots, and in particular to a robot high-speed navigation method based on an event camera and a laser radar. Background Art

[0002] Robot navigation tasks refer to the robot moving to the target position autonomously without collision in known or unknown scenarios through its own sensors or external information, which has great practical application value. Robot navigation tasks can be divided into routing, behavior decision-making, and motion planning. Different tasks include different optimization goals such as safety, stability, and robustness. Different constraints such as acceleration and deceleration constraints, non-integrity constraints, and dynamic constraints should be considered for different scenarios.

[0003] At present, navigation methods based on traditional sensors such as LiDAR and RGB cameras are widely used in the field of autonomous driving. These methods have good and stable performance in low-speed environments with fixed obstacles. However, in real environments, intelligent bodies need to be able to make quick decisions. High speed includes both the high speed of the robot itself and the high speed of obstacles in the environment, such as people crossing the road and thrown objects. At this time, sensors need to be able to respond to changes quickly and feed back to the control center, which means that sensors have to run at a higher frame rate, occupy a higher bandwidth, and may even seriously affect the stability of the navigation algorithm. Therefore, how to safely navigate to the target point while moving at high speed is an urgent task in the field of robot navigation.

[0004] Event cameras are bionic sensors with microsecond response times that recognize objects by simulating the imaging principle of the human retina. Compared with traditional cameras, they are more suitable for use in high-dynamic and high-speed environments, and have significant advantages such as high dynamic range, high temporal resolution, and no motion blur. Therefore, they are widely used in drones, robots, computer vision, fluorescent imaging, and other fields. In the same period of time, the amount of data occupied by a set of event streams is often much less than the data transmitted by traditional cameras. In addition, the temporal resolution of event cameras is extremely low, and their small amounts of data are output in a stream, making their latency much lower than that of traditional cameras. However, their focus on identifying dynamic moving objects limits their ability to perceive the overall environment, making it difficult to use them directly as a single sensor.

[0005] LiDAR sensors have been widely used in the field of robot navigation. Their simple data format, easy deployment, and low data processing difficulty make them the preferred sensor for algorithm simulation and real machine application. However, their ability to recognize moving objects is poor, and their use scenarios are often limited to low-speed or static scenes.

[0006] Based on the investigation and analysis of existing navigation technologies, the present invention combines the advantages of LiDAR and event cameras, compensates event camera data through the robot's own state information, detects moving objects in the environment, detects static fixed obstacles in the environment through LiDAR, and jointly inputs reinforcement learning networks to guide robot navigation. This method can effectively utilize the advantages of low power consumption and high temporal resolution of event cameras, and move to the target location at high speed while avoiding moving obstacles. Summary of the invention

[0007] The present invention provides a robot high-speed navigation method based on event camera and laser radar. The method can separate fast-moving obstacles in the environment according to the event camera and the robot's own state, and then extract features by pre-trained event frame encoder, enhance the basic environmental information provided by the laser radar, and finally combine the reinforcement learning framework training based on proximal strategy optimization to complete the robot's high-speed navigation task in a simulated unfamiliar dynamic environment.

[0008] The technical solution of the present invention is as follows: a robot high-speed navigation method based on event camera and laser radar, the robot is equipped with event camera, two-dimensional laser radar and odometer sensor for perceiving the environment; the event camera obtains the event frame in the current time window, introduces the robot motion state information, and obtains the motion compensated event frame containing only dynamic obstacles through compensation calculation; the motion compensated event frame is input into the pre-trained event frame encoder to obtain the feature vector, which is spliced ​​with the laser radar data and the robot motion state data and input into the reinforcement learning network; the reinforcement learning network adopts the proximal optimization strategy to be trained in the simulation environment, and finally outputs a two-dimensional vector, which represents the linear speed of the left and right wheels of the robot as the decision action, thereby guiding the movement of the robot.

[0009] The motion compensation event frame is based on the original event frame for motion compensation. The current posture state of the robot is obtained through the open source ROS robot operating system. According to the trigger frequency of the laser radar, within the interval between two triggers, the angular velocity in the posture state information is used to calculate the degree of motion compensation required in the original event frame. When the picture contains static obstacles and dynamic obstacles, the formula of the motion compensation event frame is shown in formula (1).

[0010]

[0011] Among them, x i represents the xth event frame, K is the calibration matrix, I is the identity matrix, is the average angular velocity; t i is the current time, and t0 is the initial time.

[0012] The pre-trained event frame encoder adopts a variational autoencoder structure; the variational autoencoder structure is a 9-layer structure, and the dimensions of each layer are 128, 64, 32, 8, 8, 8, 32, 64, and 128, respectively. The event frame encoder structure is the first five layers, and the fifth layer is a separate expansion layer of 8*8, which is used to extract the latent vector layer of the feature; the loss function used to train the encoder and decoder structures in the variational autoencoder is shown in formula (2);

[0013]

[0014] The first term represents the original input image x i，j and reconstruct the image The mean square error MSE between them corresponds to the process of reconstructing the image by relying on the latent vector z extracted from the fifth layer of the event frame encoder structure; the second term represents the gap between the probability distribution p(x) obtained by the decoder and the KL divergence, which corresponds to making the approximate posterior distribution of the latent vector z as close as possible to the prior distribution of z; q(x) represents the probability distribution of x obtained by the encoder; m and n represent the width and height of the event picture respectively; the dimension of the event picture after the event frame encoder changes from 128*128*1 to 8*8*1, and then expands to a one-dimensional latent vector z of 64*1.

[0015] The observation space s = {Gdis, Gdir, v, ω, S, E} of the reinforcement learning network is mapped to the action space a = {v1, Vr} of the robot; Gdis and Gdir are the relative distance and direction from the current position of the robot to the target position; v and ω are the linear velocity and angular velocity of the robot respectively; S is the observation distance of the laser radar; E is the latent vector z obtained by the event frame encoder of the event picture; v1 and vr are the left wheel speed and right wheel speed of the robot respectively;

[0016] The basic framework of the reinforcement learning network adopts the proximal strategy optimization algorithm. According to the difference between the new and old strategies in reinforcement learning, the objective function is set as shown in formula (3);

[0017]

[0018] Where clip(r t (θ), 1-ε, 1+ε) means clipping the strategy ratio rt(θ) to [1-ε, 1+ε]. If the deviation of rt(θ) exceeds the range, clipping is performed, the gradient is changed to 0, and the training of the parameters of the reinforcement learning network is suspended.

[0019] The reinforcement learning network is a 5-layer fully connected layer with dimensions of 88, 256, 256, 256, and 2 respectively. The reward and punishment mechanism of the reinforcement learning network takes two factors into consideration, namely, the distance between the robot and the end point, and the distance between the robot and the obstacle. The robot is encouraged to move toward the target position and will not choose to move away from all obstacles in order to avoid collision.

[0020] The laser radar module uses 18 lines of radar data, evenly spread out within a 180-degree range in front of the robot, to obtain 18*1 one-dimensional radar data.

[0021] The robot is a URDF model of the urtleBot-ROS robot.

[0022] Beneficial effects of the present invention:

[0023] (1) Introducing event cameras to obtain dynamic environment information

[0024] Event cameras have the advantage of high temporal resolution, but the sparse nature of event streams makes it difficult to directly apply them to robot navigation tasks. This paper proposes a robot high-speed navigation method based on event cameras and laser radars. The image is compensated by the motion of the event camera and the robot body, thereby distinguishing static objects from moving objects in the environment, and introducing dynamic environmental information into the traditional laser radar-based navigation scheme.

[0025] (2) Wide applicability

[0026] The present invention is applicable to robot navigation tasks in different navigation scenarios. The constructed training scenarios set up complex venues such as straight lines, rings, curves, and Z shapes, while the environment in the test scenario contains more static obstacles of unknown shapes and more dynamic obstacles. After testing, the algorithm can still run well in unfamiliar environments. The effectiveness and applicability of this method in different navigation scenarios are proved. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] Figure 1 It is a flow chart of the robot high-speed navigation method based on event camera and laser radar of the present invention;

[0028] Figure 2 Diagram of training network for event frame encoder;

[0029] Figure 3 Schematic diagram of the fully connected layer. DETAILED DESCRIPTION

[0030] The specific implementation of the present invention is further described below in conjunction with the accompanying drawings and technical solutions.

[0031] A specific embodiment is used to illustrate a proposed robot high-speed navigation method based on an event camera and a laser radar, and the method comprises the following steps:

[0032] Step 1: Load the robot simulation model and build a training and testing simulation environment;

[0033] This experiment uses the URDF model of the TurtleBot-ROS robot as the experimental robot; it is equipped with an event camera, a two-dimensional lidar, and an odometer sensor to perceive the environment; it uses the static Block obstacles in ROS-Stage to build a training environment, and sets up multiple training scenes of different difficulty levels, such as straight lines, curves, and circles; and it sets moving obstacles that move along fixed trajectories for the training and test environments in ROS-Gazebo.

[0034] Step 2: Introduce robot state compensation event frame;

[0035] In the simulation environment, the current posture state of the robot is obtained through ROS subscription. According to the trigger frequency of the laser radar, the angular velocity obtained within the interval between two triggers is used to calculate the degree of motion compensation required in the original event frame. When the picture contains static obstacles and dynamic obstacles, the relative movement of the static obstacles is very small, and the impact in the event camera picture is very small, and it will be easily compensated. On the contrary, dynamic obstacles will not have much impact. The formula of the motion compensation event frame is shown in formula (1).

[0036]

[0037] Among them, x i represents the xth event frame, K is the calibration matrix, I is the identity matrix, is the average angular velocity, t i is the current time, and t0 is the initial time. After the above processing, the dynamic obstacles that are close to the screen can be distinguished from the background.

[0038] The robot is run in the simulation environment to collect the raw data of the event stream, and the event data in the corresponding time is accumulated according to the trigger frequency of the laser radar to obtain the (128, 128)-dimensional event frame, where the value of the triggered point is 1, and the other positions are 0.

[0039] Step 3: training the event frame encoder module;

[0040] In order to effectively extract event data features, a variational autoencoder structure is used to process event data. The variational autoencoder structure is a 9-layer structure of (128-64-32-8-8-8-32-64-128), in which the fifth layer is a separate 8*8 expansion layer, representing the latent vector layer used to extract features. The loss function used to train the encoder and decoder structures is shown in formula (2).

[0041]

[0042] The first term represents the original input image x i，j and reconstruct the image The mean square error (MSE) between q(x) and n(x) corresponds to the process of reconstructing image x by relying on latent vector Z. The second term represents the difference between the probability distribution p(x) obtained by the decoder and the KL divergence, which corresponds to making the approximate posterior distribution of the latent vector z as close as possible to the prior distribution of z. q(x) represents the probability distribution of x obtained by the encoder; m and n represent the width and height of the event screen respectively; the dimension of the event screen after the event frame encoder changes from 128*128*1 to 8*8*1, and then expands to a one-dimensional latent vector z of 64*1.

[0043] Step 4: Reinforcement Learning Training

[0044] The lidar module uses 18 lines of radar data, evenly spread out within a 180-degree range in front of the intelligent body, thereby obtaining 18*1 one-dimensional radar data.

[0045] The observation space s = {Gdis, Gdir, ν, ω, S, E} of reinforcement learning is mapped to the action space a = {vl, vr} of the robot. Gdis and Gdir are the relative distance and direction from the current position of the robot to the target position; ν and ω are the linear velocity and angular velocity of the robot respectively; S is the observation distance of the laser radar; E is the latent vector z obtained by the encoder of the event picture; vl and vr are the left wheel speed and right wheel speed of the robot respectively.

[0046] The basic framework of reinforcement learning adopts the proximal policy optimization algorithm. This method sets the objective function as shown in formula (3) based on the difference between the new and old strategies in reinforcement learning.

[0047]

[0048] Among them, clip(rt(θ),1-ε,1+ε) means clipping the policy ratio rt(θ) to [1-ε,1+ε]. If the deviation of rt(θ) exceeds the range, clipping is performed, the gradient becomes 0, and the network training of parameters is suspended. This limits the amplitude of policy updates and prevents the policy update from deviating too far.

[0049] The reinforcement learning network is a 5-layer fully connected network with a network structure of 88-256-256-256-2. In order to ensure that the agent can make reasonable plans to reach the target location, the reward and punishment setting of reinforcement learning takes into account two factors, namely the distance between the agent and the end point, and the distance between the agent and the obstacle. This can encourage the agent to move to the target location and will not choose to stay away from all obstacles to avoid collisions. This can optimize the final trajectory to a certain extent and facilitate training.

[0050] like Figure 1-Figure 3 As shown, a robot high-speed navigation method based on event camera and lidar includes event data preprocessing, event frame encoder training, simulation environment setting and reinforcement learning setting.

[0051] The event camera uses DVS128 as the benchmark, with a resolution of 128×128 pixels. The URDF model and event camera plug-in of the TurtleBot-ROS robot are installed in the Gezabo simulation environment to simulate the random motion trajectory for five minutes, and several rows of four-tuple data in the format of (x, y, p, t) per row are captured. The value range of t is nanoseconds, and the event data is divided into several parts according to the trigger frequency of the laser radar. The present invention does not consider the polarity p of the event data, and maps each x and y position to a 128×128 empty matrix as a frame of event frame for subsequent processing.

[0052] The event frame encoder obtains the event frame (dimension 128×128) of the previous step, and reduces the event frame (whose resolution is 128×128 pixels) by 128-64-32-16-8 to obtain a 64×1 latent vector. Then it is reversely restored to the original size of 128×128. The loss function is used to control the similarity of the previous and next pictures to ensure that the latent vector contains enough feature information.

[0053] The simulation environment includes a training environment and a test environment. The training environment is a linear, circular, and other shaped venue. Each venue includes a starting area and an end area, which are used to control the generation of the starting and end positions of each round. The venue contains 40-50 rectangular columns as static obstacles, including a certain proportion of moving obstacles that move back and forth along a fixed trajectory. As the difficulty increases, the proportion of moving obstacles and the speed of moving obstacles will increase, so that the robot can have a good learning gradient.

[0054] The reinforcement learning algorithm was implemented in Pytorch. The proximal optimization strategy algorithm was introduced using the Stable_baselines3 library. The number of iterations was set to 300, the learning rate was initialized to 0.0005, and the decay factor of the learning rate was set to 0.1, which decayed every 20 iterations.

[0055] The overall workflow is as follows: the event camera obtains the event frame within the current time window, introduces the robot's motion state information, and performs compensation calculations to obtain event frames that only contain dynamic obstacles. The event frame is input into the pre-trained event frame encoder to obtain a feature vector, which is then concatenated with the lidar data and the robot's motion state data to obtain reinforcement learning data input. The system is trained in a simulation environment using a proximal optimization strategy, and finally outputs a two-dimensional vector representing the linear speed of the robot's left and right wheels, i.e., the decision action, thereby guiding the robot's movement.

Claims

1. A robot high-speed navigation method based on event camera and laser radar, characterized in that: The robot is equipped with an event camera, a two-dimensional laser radar and an odometer sensor for perceiving the environment; the event camera obtains the event frame within the current time window, introduces the robot's motion state information, and performs compensation calculation to obtain a motion compensated event frame containing only dynamic obstacles; the motion compensated event frame is input into a pre-trained event frame encoder to obtain a feature vector, which is spliced ​​with the laser radar data and the robot's motion state data and input into a reinforcement learning network; the reinforcement learning network is trained in a simulation environment using a proximal optimization strategy, and finally outputs a two-dimensional vector representing the linear speed of the robot's left and right wheels as a decision action, thereby guiding the robot to move.

2. The robot high-speed navigation method based on event camera and laser radar according to claim 1 is characterized in that: The motion compensation event frame is based on the original event frame for motion compensation. The current posture state of the robot is obtained through the open source ROS robot operating system. According to the trigger frequency of the laser radar, within the interval between two triggers, the angular velocity in the posture state information is used to calculate the degree of motion compensation required in the original event frame. When the picture contains static obstacles and dynamic obstacles, the formula of the motion compensation event frame is shown in formula (1). Among them, x i represents the xth event frame, K is the calibration matrix, I is the identity matrix, is the average angular velocity; t i is the current time, and t0 is the initial time.

3. The robot high-speed navigation method based on event camera and laser radar according to claim 1, characterized in that: The pre-trained event frame encoder adopts a variational autoencoder structure; the variational autoencoder structure is a 9-layer structure, and the dimensions of each layer are 128, 64, 32, 8, 8, 8, 32, 64, and 128, respectively. The event frame encoder structure is the first five layers, and the fifth layer is a separate expansion layer of 8*8, which is used to extract the latent vector layer of the feature; the loss function used to train the encoder and decoder structures in the variational autoencoder is shown in formula (2); The first term represents the original input image x i,j and reconstruct the image The mean square error MSE between them corresponds to the process of reconstructing the image based on the latent vector z extracted from the fifth layer of the event frame encoder structure; The second term represents the difference between the probability distribution p(x) obtained by the decoder and the KL divergence, which corresponds to making the approximate posterior distribution of the latent vector z as close as possible to the prior distribution of z; q(x) represents the probability distribution of x obtained by the encoder; m and n represent the width and height of the event picture respectively; the dimension of the event picture after the event frame encoder changes from 128*128*1 to 8*8*1, and then expands to a one-dimensional latent vector z of 64*1.

4. The robot high-speed navigation method based on event camera and laser radar according to claim 1, characterized in that: The observation space s = {Gdis, Gdir, ν, ω, S, E} of the reinforcement learning network is mapped to the action space a = {vl, vr} of the robot; Gdis and Gdir are the relative distance and direction from the current position of the robot to the target position; ν and ω are the linear velocity and angular velocity of the robot respectively; S is the observation distance of the laser radar; E is the latent vector z obtained by the event frame encoder of the event picture; vl and vr are the left wheel speed and right wheel speed of the robot respectively; The basic framework of the reinforcement learning network adopts the proximal strategy optimization algorithm. According to the difference between the new and old strategies in reinforcement learning, the objective function is set as shown in formula (3); Where clip(r t (θ), 1-ε, 1+ε) means clipping the strategy ratio rt(θ) to [1-ε, 1+ε]. If the deviation of rt(θ) exceeds the range, clipping is performed, the gradient becomes 0, and the training of the parameters of the reinforcement learning network is suspended.

5. The robot high-speed navigation method based on event camera and laser radar according to claim 4 is characterized in that: The reinforcement learning network is a 5-layer fully connected layer with dimensions of 88, 256, 256, 256, and 2 respectively. The reward and punishment mechanism of the reinforcement learning network takes two factors into consideration, namely, the distance between the robot and the end point, and the distance between the robot and the obstacle. The robot is encouraged to move toward the target position and will not choose to move away from all obstacles in order to avoid collision.

6. The robot high-speed navigation method based on event camera and laser radar according to claim 1, characterized in that: The laser radar module uses 18 lines of radar data, evenly spread out within a 180-degree range in front of the robot, to obtain 18*1 one-dimensional radar data.

7. The robot high-speed navigation method based on event camera and laser radar according to claim 1, characterized in that: The robot is a URDF model of the urtleBot-ROS robot.

Images