A robot mobile obstacle avoidance method based on safety training imitation learning

Through security training, imitation learning methods and neural network loss functions, combined with dataset symmetry enhancement technology, the problem of insufficient safety in robot mobile obstacle avoidance training is solved, training efficiency and obstacle avoidance effect are improved, and safety in real environments is achieved Ability to avoid obstacles.

CN114942636BActive Publication Date: 2024-11-26UNIV OF SCI & TECH OF CHINA
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202210485390.0
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2022-05-06
Publication Date
2024-11-26
Estimated Expiration
2042-05-06

AI Technical Summary

Technical Problem

The existing robot mobile obstacle avoidance technology lacks security during training, which makes it impossible to implement on real robots. The obstacle avoidance method based on machine learning requires more computing resources and hardware computing power.

Method used

Using a hybrid exploration of security training imitation learning method, the neural network is trained to avoid the collision between robots and obstacles during training and improve training efficiency and performance performance through safely enhanced neural network loss function and data set symmetry enhancement technology.

Benefits of technology

It realizes the safety of avoiding collision between robots and obstacles during training, improves the effect of obstacle avoidance algorithms and the efficiency of training data, and reduces the cost of hardware computing power.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN114942636B_ABST
    Figure CN114942636B_ABST
Patent Text Reader

Abstract

The invention relates to the field of robot mobile obstacle avoidance, and discloses a robot mobile obstacle avoidance method based on safety training imitation learning; the invention enables the robot to learn mobile obstacle avoidance ability during exploration, while avoiding the collision between the robot and obstacles during training; a safety-enhanced neural network loss function is proposed, which can make network training pay more attention to expert demonstration information under dangerous conditions, thereby improving the obstacle avoidance effect of the obstacle avoidance algorithm; in addition, through the symmetrically flipped data set enhancement technology, a left-right flipped training data copy is generated, thereby doubling the size of the training data set and improving the use efficiency of the training data.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of robot movement and obstacle avoidance, and in particular to a robot movement and obstacle avoidance method based on safety training imitation learning. Background Art

[0002] Robot mobile obstacle avoidance technology refers to the technology that allows a robot to perceive obstacle information in an obstacle environment by processing the data collected by its own sensors, thereby safely avoiding obstacles and finally reaching a given target point. In recent years, machine learning technology has developed rapidly, and the application of neural networks and machine learning methods in robot navigation has also increased. The machine learning methods for training robots to acquire navigation capabilities are divided into reinforcement learning and imitation learning. The former acquires navigation capabilities by continuously exploring and trial-and-error in the environment, while the latter directly acquires capabilities by imitating experts (manual or traditional navigation methods).

[0003] Existing robot mobile obstacle avoidance technology usually uses ultrasonic, lidar, camera and other sensors to collect environmental sensing information, and constructs the surrounding environment through SLAM mapping technology to obtain an environmental map. On this map, path planning algorithms such as artificial potential field method and grid method are run to obtain a navigation path, and then move to the target point along the path.

[0004] Traditional mobile obstacle avoidance technology requires real-time mapping of the terrain environment, which requires a lot of computing resources. However, after completing the training of the neural network, the actual operating overhead of obstacle avoidance technology based on machine learning is relatively small, and it does not require mapping of the environment, thus reducing the hardware computing cost.

[0005] Existing reinforcement learning and online imitation learning algorithms require robots to repeatedly try and error during the learning and training process. This method lacks safety guarantees and cannot be implemented on real robots. Summary of the invention

[0006] To solve the above technical problems, the present invention proposes a hybrid exploration safety training imitation learning method to ensure the safety of robot obstacle avoidance training. This method enables the robot to learn the ability to move and avoid obstacles during exploration, while avoiding collisions between the robot and obstacles during training. The present invention also proposes a safety-enhanced neural network loss function and a data set symmetry enhancement technology, both of which can significantly improve the robot training efficiency and performance.

[0007] In order to solve the above technical problems, the present invention adopts the following technical solutions:

[0008] A robot mobile obstacle avoidance method based on safety training imitation learning includes the following steps:

[0009] Step A: Train the neural network π via secure training imitation learning methodθ , which can prevent the robot from colliding with obstacles during training; the neural network π θ The input is the state information s, and the output is the robot's manipulation instruction u and the risk coefficient λ(s) of the state information s; the state information s includes the relative coordinates s of the navigation target g , the robot's current speed s v And lidar scanning information l ;

[0010] Step B: Transform the trained neural network π θ Deployed on the robot, the control command u output by the neural network enables the robot to move and avoid obstacles;

[0011] The process of training a neural network through the secure training imitation learning method includes:

[0012] Step A1: Input the robot’s current state information s into the expert strategy π * and neural network π θ In the expert strategy π * Output expert manipulation instruction u * , neural network π θ Output neural network manipulation instruction u′;

[0013] Step A2: Transform the state-action pair (s,u * ) is stored as training data into the training demonstration data set D;

[0014] Step A3: Randomly extract a set of training data from the training demonstration data set D, and use the following security enhancement loss function to train the neural network π θ To train:

[0015]

[0016] The loss weight coefficient α(s) = (1+k·λ(s)) is used to indicate the importance of the state information s in the training data; k is the laser radar scanning information s l The number of elements in ;

[0017] Step A4: Calculate the risk factor λ(s) of the current state information s, and use λ(s) as the mixing ratio to convert the expert control instruction u * Mixed with the neural network control instruction u′, the final control instruction u=λ(s)u * +(1-λ(s))u′;

[0018] Repeat steps A1-A4 to obtain the trained neural network π θ .

[0019] Specifically, the expert strategy π* The A* algorithm and TEB algorithm in the robot operating system ROS navigation library are selected.

[0020] Specifically, the network structure of the neural network is as follows:

[0021] The neural network consists of four fully connected layers. The number of neurons in each fully connected layer is 1024, 512, 256, and 2, respectively. Each fully connected layer is followed by a batch of standardization layers. Except for the last layer, each fully connected layer is followed by a RELU activation function. The risk coefficient λ(s) output by the neural network is determined by the closest distance d(s) between the robot and the obstacle:

[0022]

[0023] Among them, d max and d min are pre-set parameters.

[0024] Specifically, the data preprocessing process of the neural network is as follows:

[0025] LiDAR scan information l Before inputting the network, perform min-pooling to reduce s l The dimension of the simplified lidar data s l ′, the i-th simplified lidar data

[0026] Relative coordinates of the navigation target g Perform a limiting operation, the maximum target distance is r max , the relative coordinates of the navigation target after clipping

[0027] After clipping, the relative coordinates of the navigation target and the current velocity of the robot are normalized and mapped to [-1, 1], and the lidar data is mapped to [0, 1].

[0028] Specifically, in step A2, the state-action pair (s, u * ) is input into the training demonstration data set D, the (s,u * ) The symmetric data is also input into the training demonstration data set D as training data;

[0029] The method of obtaining symmetric data through symmetry enhancement operation specifically includes:

[0030] For relative coordinates s g To perform a left-right symmetrical flip:

[0031] The current speed of the robot s vTo perform a left-right symmetrical flip:

[0032] Scanning information of laser radar l To perform a left-right symmetrical flip:

[0033] For the control command u * To perform a left-right symmetrical flip:

[0034] where x g ,y g The relative coordinates s are g The horizontal and vertical coordinates, s v =(v0,ω0), v0 and ω0 are the current linear velocity and angular velocity of the robot respectively; u * =(v,ω), v and ω are the control instructions u * Set the linear and angular velocities.

[0035] Compared with the prior art, the beneficial technical effects of the present invention are:

[0036] 1. The present invention proposes a hybrid exploration safety training imitation learning method to ensure the safety of robot obstacle avoidance training. This method enables the robot to learn the ability of mobile obstacle avoidance during exploration while avoiding collisions between the robot and obstacles during training.

[0037] 2. The present invention proposes a safety-enhanced neural network loss function, which can make network training pay more attention to expert demonstration information under dangerous conditions, thereby improving the obstacle avoidance effect of the obstacle avoidance algorithm.

[0038] 3. The present invention proposes a symmetrically flipped data set enhancement technology, which generates left-right flipped copies of training data, thereby doubling the size of the training data set and greatly improving the efficiency of training data utilization. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] Figure 1 It is a flow chart of the robot movement obstacle avoidance method based on safety training imitation learning of the present invention;

[0040] Figure 2 The algorithm structure diagram of the robot mobile obstacle avoidance method based on safety training imitation learning of the present invention;

[0041] Figure 3 A network structure diagram of the neural network strategy proposed by the present invention;

[0042] Figure 4 This is a schematic diagram of the data symmetry enhancement algorithm proposed in the present invention. DETAILED DESCRIPTION

[0043] A preferred embodiment of the present invention is described in detail below with reference to the accompanying drawings.

[0044] The present invention trains the neural network π through secure imitation learning θ , so that the neural network has the ability to control the robot to move and avoid obstacles. The robot in the present invention is a two-wheel differential wheel chassis, and the neural network π θ The input is the state information s, which includes the relative coordinates s of the navigation target. g , the robot's current speed s v And the scanning information of the lidar sensor l ; The output of the neural network is the robot's manipulation instruction u. The expert strategy π that provides training data for the neural network * The A* algorithm and TEB algorithm in the robot operating system ROS navigation library are selected.

[0045] The data preprocessing process of the neural network is as follows:

[0046] The robot's 320-line laser radar scan information l Before formally inputting the neural network, a min-pooling operation is performed (i.e., selecting the minimum value of the same dimension) to make the 320-dimensional lidar scanning information s l Reduced to 32-dimensional simplified lidar data s l ′, the calculation formula for the i-th data of the simplified lidar data is:

[0047]

[0048] The relative coordinates of the navigation target s g Limiting operation is required, the maximum target distance is r max , the relative coordinate after clipping s g The calculation formula of ′ is:

[0049]

[0050] After clipping, the relative coordinates of the navigation target and the current velocity of the robot are normalized and mapped to [-1, 1], and the lidar data is mapped to [0, 1].

[0051] After data preprocessing is completed, the lidar data s l ′, relative coordinates of the navigation target s g ′ and current speed s v It is input into a set of fully connected neural networks. The neural network consists of four fully connected layers, with the number of neurons in each layer being 1024, 512, 256, and 2, respectively. Each fully connected layer is followed by a batch of standardization layers. Except for the last layer, each fully connected layer is followed by a RELU activation function.

[0052] The neural network has two outputs. The first output is the robot manipulation instruction u, and the second output is the risk factor λ(s). The risk factor indicates the degree of danger of the state information s. It is determined by the closest distance d(s) between the robot and the obstacle. The closer the distance, the higher the value. The calculation formula of d(s) is:

[0053]

[0054] Among them, d max and d min are pre-set parameters.

[0055] The hazard factor λ(s) plays an important role in both neural network optimization and the training of imitation learning methods for safety training.

[0056] The safety training imitation learning method of the present invention can prevent the robot from colliding with obstacles during the training stage.

[0057] Input to the Imitation Learning Method for Security Training: Expert Policy π * , the obstacle avoidance neural network strategy π to be trained θ .

[0058] Perform the following operations for each moment in a loop:

[0059] Input the robot's current state information s into the expert strategy π * and neural network π θ In the above example, each of them outputs the corresponding control instruction u * and u′.

[0060] The state-action pair (s,u * ) and its symmetric data are stored in the training demonstration data set D. The symmetric data is a copy of the training data that is symmetrically flipped left and right, generated by the symmetric enhancement operation of the data, thereby doubling the size of the training data set. The symmetric enhancement operation of the data greatly improves the efficiency of the use of training data. The specific implementation formula is:

[0061] For the original training data (s,u * ), relative coordinates s g The left-right symmetrical flip is:

[0062] Current speed v The left-right symmetrical flip is:

[0063] LiDAR scan information l The left-right symmetrical flip is: Where k is the laser radar s lThe number of elements in ;

[0064] Operation command u * The left-right symmetrical flip is:

[0065] From the training demonstration dataset A set of training data is randomly selected from the neural network π θ The present invention uses a new security enhancement loss function to train the neural network:

[0066]

[0067] The loss weight coefficient α(s) is used to indicate the importance of the state information s in the training data, α(s) = (1 + k · λ(s)s, λ(s) is the risk factor of the state information s. This loss function can make the network training pay more attention to the expert demonstration information under dangerous conditions, thereby improving the obstacle avoidance effect of the obstacle avoidance method. The current state information s of the robot is the expert strategy π * and neural network π θ The parameter of the expert strategy is expressed as π * (s), the neural network parameterization is π θ (s).

[0068] The final safe execution action is obtained through hybrid exploration: the risk coefficient λ(s) of the current state information s is calculated; then, the expert control instruction u is mixed with λ(s) as the mixing ratio. * Mixed with the neural network manipulation instruction u′, the final execution operation u is obtained. The specific formula is: u=λ(s)u * +(1-λ(s))u′. This hybrid exploration method can automatically increase the control weight of the expert strategy when the robot is too close to an obstacle, resulting in a large risk factor λ(s), so as to avoid collisions between the robot and obstacles during training.

[0069] By iteratively running the above security training imitation learning method, the trained neural network π will eventually be obtained. θ , the neural network can be directly deployed on the robot for control.

[0070] It is obvious to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above, and that the present invention can be implemented in other specific forms without departing from the spirit or essential features of the present invention. Therefore, from any point of view, the embodiments should be regarded as exemplary and non-limiting, and the scope of the present invention is defined by the appended claims rather than the above description, and it is intended that all changes falling within the meaning and scope of the equivalent elements of the claims are included in the present invention, and any reference numerals in the claims should not be regarded as limiting the claims involved.

[0071] In addition, it should be understood that although this specification is described in terms of implementation methods, not every implementation method contains only one independent technical solution. This narrative method of the specification is only for the sake of clarity. Those skilled in the art should regard the specification as a whole. The technical solutions in each embodiment may also be appropriately combined to form other implementation methods that can be understood by those skilled in the art.

Claims

1. A robot mobile obstacle avoidance method based on safety training imitation learning, comprising the following steps: Step A: Train the neural network π via secure training imitation learning method θ , which can prevent the robot from colliding with obstacles during training; the neural network π θ The input is the state information s, and the output is the robot's manipulation instruction u and the risk coefficient λ(s) of the state information s; the state information s includes the relative coordinates s of the navigation target g , the robot's current speed s v And lidar scanning information l ; Step B: Transform the trained neural network π θ Deployed on the robot, the control command u output by the neural network enables the robot to move and avoid obstacles; The process of training a neural network through the secure training imitation learning method includes: Step A1: Input the robot’s current state information s into the expert strategy π * and neural network π θ In the expert strategy π * Output expert manipulation instruction u * , neural network π θ Output neural network manipulation instruction u′; Step A2: Transform the state-action pair (s,u * ) as training data to the training demonstration dataset middle; Step A3: From the training demonstration dataset A set of training data is randomly selected from the neural network π through the following security enhancement loss function θ To train: The loss weight coefficient α(s) = (1+k·λ(s)) is used to indicate the importance of the state information s in the training data; k is the laser radar scanning information s l The number of elements in ; Step A4: Calculate the risk factor λ(s) of the current state information s, and use λ(s) as the mixing ratio to convert the expert control instruction u * Mixed with the neural network control instruction u′, the final control instruction u=λ(s)u * +(1-λ(s))u′; Repeat steps A1-A4 to obtain the trained neural network π θ .

2. The robot mobile obstacle avoidance method based on safety training imitation learning according to claim 1 is characterized in that: Expert Strategy π * The A* algorithm and TEB algorithm in the robot operating system ROS navigation library are selected.

3. The robot mobile obstacle avoidance method based on safety training imitation learning according to claim 1 is characterized in that: The network structure of the neural network is as follows: The neural network consists of four fully connected layers. The number of neurons in each fully connected layer is 1024, 512, 256, and 2, respectively. Each fully connected layer is followed by a batch of standardization layers. Except for the last layer, each fully connected layer is followed by a RELU activation function. The risk coefficient λ(s) output by the neural network is determined by the closest distance d(s) between the robot and the obstacle: Among them, d max and d min are pre-set parameters.

4. The robot mobile obstacle avoidance method based on safety training imitation learning according to claim 1 is characterized in that: The data preprocessing process of the neural network is as follows: LiDAR scan information l Before inputting the network, perform min-pooling to reduce s l The dimension of the simplified lidar data s l ′, the i-th simplified lidar data Relative coordinates of the navigation target g Perform a limiting operation, the maximum target distance is r max , the relative coordinates of the navigation target after clipping After clipping, the relative coordinates of the navigation target and the current velocity of the robot are normalized and mapped to [-1, 1], and the lidar data is mapped to [0, 1].

5. The robot mobile obstacle avoidance method based on safety training imitation learning according to claim 1 is characterized in that: In step A2, the state-action pair (s, u * ) is input into the training demonstration dataset When (s, u * ) Symmetric data is also input into the training demonstration data set as training data middle; The method of obtaining symmetric data through symmetry enhancement operation specifically includes: For relative coordinates s g To perform a left-right symmetrical flip: The current speed of the robot s v To perform a left-right symmetrical flip: Scanning information of laser radar l To perform a left-right symmetrical flip: For the control command u * To perform a left-right symmetrical flip: where x g ,y g Relative coordinates s g The horizontal and vertical coordinates, s v =(v0,ω0), v0 and ω0 are the current linear velocity and angular velocity of the robot respectively; u * =(v,ω), v and ω are the control instructions u * Set the linear and angular velocities.