A robot autonomous obstacle avoidance method
The robot's obstacle avoidance network is trained through long-term pulse neural networks, which solves the problem of decision-making difficulties in the face of interference, and improves the robustness and computing efficiency of the robot's autonomous obstacle avoidance.
Patent Information
- Application Number
- CN202311631739.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2023-11-30
- Publication Date
- 2024-12-03
- Estimated Expiration
- 2043-11-30
AI Technical Summary
When traditional neural networks face sudden interference such as strong light, the robot cannot make decisions about the current state, and the problem of large obstacle avoidance calculations and high energy consumption.
A long and short-term pulse neural network is used to train the robot's autonomous obstacle avoidance network by obtaining the robot's obstacle avoidance reward function, cumulative reward function and loss function, and build the trained robot's autonomous obstacle avoidance network, and use the network to complete the obstacle avoidance task.
It improves the robustness of the robot in the face of interference, reduces the calculation amount and energy consumption, ensures that the robot can make decisions according to the movement trend of the previous moment, and enhances the efficiency of autonomous obstacle avoidance navigation.
Smart Images

Figure CN117519192B_ABST
Abstract
Description
Technical Field
[0001] The invention relates to a robot autonomous obstacle avoidance method, belonging to the technical field of robot obstacle avoidance and navigation. Background Art
[0002] Robotic autonomous obstacle avoidance and navigation technology is one of the core technologies of all mobile robots, such as household robots, food delivery robots, cleaning robots, etc., which must have the ability to autonomously avoid obstacles and navigate. The endurance of robots poses a challenge to the energy consumption of robots in all aspects. Traditional neural networks have a high amount of computing, and with high computing comes high power consumption. In addition, robots that use traditional neural networks for obstacle avoidance calculations will be unable to make decisions on the current state when there is sudden strong light or other noise that interferes with the robot's perception. Summary of the invention
[0003] The purpose of the present invention is to overcome the deficiencies in the prior art and provide a robot autonomous obstacle avoidance method with a smaller amount of computation than traditional neural networks and stronger robustness in the face of noise. It solves the problem that when traditional neural networks face sudden strong light and other noise that interferes with the robot's perception, the robot cannot make decisions on the current state, the obstacle avoidance has a large amount of computation and high energy consumption.
[0004] In order to achieve the above purpose / solve the above technical problems, the present invention is implemented by adopting the following technical solutions:
[0005] A robot autonomous obstacle avoidance method, comprising:
[0006] Obtain the robot's obstacle avoidance reward function, cumulative reward function, first loss function, and second loss function;
[0007] The robot's preset autonomous obstacle avoidance network is trained using the obtained obstacle avoidance reward function, cumulative reward function, first loss function and second loss function to obtain a trained robot autonomous obstacle avoidance network; wherein the preset autonomous obstacle avoidance network adopts a long-short time pulse neural network;
[0008] The initial point, target point and obstacle position information of the target robot are obtained, and the trained robot autonomous obstacle avoidance network is used to complete the obstacle avoidance task.
[0009] Furthermore, the robot obstacle avoidance network is trained by using the obtained robot obstacle avoidance reward function, cumulative reward function and loss function to obtain a trained robot autonomous obstacle avoidance network, including:
[0010] Build a robot obstacle avoidance training environment and start navigation tasks;
[0011] Initialize the robot's preset autonomous obstacle avoidance network and preset value network;
[0012] Initialize the robot's initial state information;
[0013] Normalize the initial state information of the robot;
[0014] Get the neuron input of the preset autonomous obstacle avoidance network;
[0015] According to the neuron input of the preset autonomous obstacle avoidance network, the robot's current state information and the state information of the next moment are obtained;
[0016] Based on the current state information of the robot, the current moment reward is obtained through the obstacle avoidance reward function;
[0017] Determine whether the robot collides or reaches the preset end point during operation;
[0018] If not, record done = 0, initialize the robot's status information and re-acquire the robot's current status information and the status information at the next moment;
[0019] Based on the current state information reacquired by the robot, the current moment reward is obtained through the obstacle avoidance reward function;
[0020] Determine whether the robot collides or reaches the preset end point during operation until done = 1;
[0021] Input the robot's status information at different times into the preset value network to obtain the robot's status evaluation;
[0022] Based on the obtained rewards and the robot's status evaluation, the cumulative rewards are obtained through the cumulative reward function;
[0023] Based on the state evaluation, a first loss value is obtained through a first loss function, and the first loss value is back-propagated to update a preset value network;
[0024] Based on the cumulative reward function, a second loss value is obtained through a second loss function, the second loss value is back-propagated, and a preset autonomous obstacle avoidance network is updated until the number of updates meets a threshold;
[0025] Get the trained robot autonomous obstacle avoidance network.
[0026] Furthermore, the robot initial state information includes: a radar signal, a linear velocity and an angular velocity of the robot's initial position.
[0027] Furthermore, the obtaining of neuron input of a preset autonomous obstacle avoidance network includes:
[0028] Calculate the output value f of the forget gate t , the neuron membrane potential mem t-1 The pulse s input at the current momentt As input:
[0029] f t =σ(W if s t +b if +W hf mem t-1 +b hf );
[0030] Where: W if is the weight of the forget gate; b if is the bias of the forget gate; W hf is the weight of the hidden state of the forget gate; b if is the bias of the hidden state of the forget gate; σ is the sigmoid function;
[0031] Calculate the memory gate output value i t , the neuron membrane potential mem t-1 The pulse s input at the current moment t As input:
[0032] i t =σ(W ii x t +b ii +W hi mem t-1 +b hi );
[0033] Where: W ii is the weight of the memory gate; b ii is the bias of the memory gate; W hi is the weight of the hidden state of the memory gate; b hi is the bias of the implicit state of the memory gate;
[0034] Calculate the temporary state g t , the neuron membrane potential mem t-1 The pulse s input at the current moment t As input:
[0035] g t =tanh(W ig x t +b ig +W hm mem t-1 +b hg );
[0036] Where: W ig is the weight of the temporary state; b ig is the bias of the temporary state; W hg is the weight of the temporary state implied state; bhg is the bias of the implied state of the temporary state;
[0037] Calculate the saliency current syn of the neuron at the current moment t :
[0038] syn t =f t *syn t-1 +i t *g t ;
[0039] Among them: syn t-1 is the synaptic current of the neuron at the previous moment;
[0040] Calculate the output value o of the output gate t :
[0041] o t =sigma(W io x t +b io +W go mem t-1 +b ho );
[0042] Where: W io is the weight of the output gate; b io is the bias of the output gate; W ho is the weight of the hidden state of the output gate; b ho is the bias of the implicit state of the output gate;
[0043] Calculate the membrane potential mem of the neuron at the current moment t :
[0044] mem t =o t *tann(syn t );
[0045] The syn t and the current neuron membrane potential mem t As the input of the neuron at the next moment.
[0046] Furthermore, the obtaining of the current state information of the robot according to the neuron input of the preset autonomous obstacle avoidance network includes:
[0047] Get the mean and variance of the robot at the current moment;
[0048] According to the mean and variance at the current moment, the current state information of the robot is obtained, wherein the current state information includes the linear velocity and angular velocity at the current moment.
[0049] Furthermore, it is determined whether the robot collides or reaches a preset end point during operation. If so, done=1 is recorded, the robot obstacle avoidance training environment is refreshed, and a new round of navigation tasks is started.
[0050] Furthermore, the obstacle avoidance reward function of the robot includes:
[0051]
[0052] in:
[0053] d th is the minimum distance between the robot and the target or obstacle;
[0054] d ob is the distance between the robot and the obstacle;
[0055] d tar is the distance of the robot from the destination.
[0056] Furthermore, the cumulative reward function includes:
[0057] R=[R[0], R[1],..., R[T]];
[0058] R[t]=reward[t]+γ*reward[t+1]+γ 2 *reward[t+2]+...+γ T-t+1 *reward[T-1]+γ T-t *post_value;
[0059] in:
[0060] Post_value is the state evaluation of the robot at the next moment. The acquisition of the state evaluation of the robot at the next moment includes:
[0061] Input the robot's state information at the next moment into the preset value network to obtain the robot's state evaluation at the next moment;
[0062] R is the cumulative reward from 0 to T, where 0 is the current moment and T is the last moment;
[0063] R[0] is the reward at the current moment, and R[T] is the reward at the last moment;
[0064] R[t] is the expanded expression of the R sub-term;
[0065] γ is a discount factor in (0, 1); reward[t] is the reward at time t.
[0066] Furthermore, the first loss function includes:
[0067] advantage = R-Value;
[0068] Where: R is the cumulative reward;
[0069] Value is a cumulative status evaluation, and obtaining the cumulative status evaluation includes: accumulating all obtained status evaluations;
[0070] The mean square error of advantage is used as the first loss value.
[0071] Furthermore, the second loss function includes:
[0072] Loss=mean(min(ratio*advantage, advantage*clip(ratio, 1-ξ, ξ);
[0073] in:
[0074] ratio=P 2 / P 1 , synthesize the actions corresponding to the robot's state information at different times, and input the synthesized actions into the preset autonomous obstacle avoidance network at the previous moment and the preset autonomous obstacle avoidance network at the current moment, respectively, to obtain the mean μ 1 , variance σ 1 , mean μ 2 and variance σ 2 , according to the mean μ 1 , variance σ 1 , mean μ 2 and variance σ 2 Find the corresponding probability P 1 and P 2 ;
[0075] clip(,,) means truncation. clip(ratio, 1-ξ, ξ) means that if ratio is less than 1-ξ, then ratio takes the value of 1-ξ. If ratio is greater than ξ, then ratio takes the value of ξ. If ratio is within the range of (1-ξ, ξ), then ratio remains unchanged.
[0076] ξ is a hyperparameter, which indicates the proportion of updated parameters retained, and its value is in the range of (0, 1).
[0077] Beneficial Effects
[0078] Compared with the prior art, the present invention has the following beneficial effects:
[0079] The present invention utilizes the constructed robot's obstacle avoidance reward function, cumulative reward function, first loss function and second loss function to train the robot's preset long-short time pulse neural network, and then obtains the trained robot autonomous obstacle avoidance network. The long-short time pulse neural network retains historical memory of the robot's state information, and short-term interference does not affect the operation of the network. The robot can still make decisions according to the movement trend of the previous moment, and the long-short time structure can improve the memory capacity of the pulse neural network, which ensures the robustness of the robot's autonomous obstacle avoidance navigation while reducing the amount of calculation and improving the calculation efficiency. It solves the problem that when traditional neural networks face sudden strong light and other noises that interfere with the robot's perception, the robot cannot make decisions on the current state, and the obstacle avoidance calculation is large and the energy consumption is high. BRIEF DESCRIPTION OF THE DRAWINGS
[0080] Figure 1 The present invention provides a flowchart of a robot autonomous obstacle avoidance method. DETAILED DESCRIPTION
[0081] The present invention will be further described below in conjunction with the accompanying drawings. The following embodiments are only used to more clearly illustrate the technical solution of the present invention, and cannot be used to limit the protection scope of the present invention.
[0082] Example
[0083] like Figure 1 As shown, a robot autonomous obstacle avoidance method includes:
[0084] Build a robot obstacle avoidance training environment and start navigation tasks;
[0085] Initialize the robot's preset autonomous obstacle avoidance network and preset value network; the preset autonomous obstacle avoidance network adopts a long-short time pulse neural network;
[0086] Initialize the initial state information of the robot, which includes: radar signal, linear velocity and angular velocity of the robot's initial position;
[0087] Normalize the initial state information of the robot and obtain the neuron input of the preset autonomous obstacle avoidance network. Specifically:
[0088] Calculate the output value f of the forget gate t , the neuron membrane potential mem t-1 The pulse s input at the current moment t As input:
[0089] f t =σ(W if s t +b if +W hf memt-1 +b hf );
[0090] Where: W if is the weight of the forget gate; b if is the bias of the forget gate; W hf is the weight of the hidden state of the forget gate; b if is the bias of the hidden state of the forget gate; σ is the sigmoid function;
[0091] Calculate the memory gate output value i t , the neuron membrane potential mem t-1 The pulse s input at the current moment t As input:
[0092] i t =σ(W ii x t +b ii +W hi mem t-1 +b hi );
[0093] Where: W ii is the weight of the memory gate; b ii is the bias of the memory gate; W hi is the weight of the hidden state of the memory gate; b hi is the bias of the implicit state of the memory gate;
[0094] Calculate the temporary state g t , the neuron membrane potential mem t-1 The pulse s input at the current moment t As input:
[0095] g t =tanh(W ig x t +b ig +W hg mem t-1 +b hg );
[0096] Where: W ig is the weight of the temporary state; b ig is the bias of the temporary state; W hg is the weight of the temporary state implied state; b hg is the bias of the implied state of the temporary state;
[0097] Calculate the saliency current syn of the neuron at the current moment t :
[0098] syn t=f t *syn t-1 +i t *g t ;
[0099] Among them: syn t-1 is the synaptic current of the neuron at the previous moment;
[0100] Calculate the output value o of the output gate t :
[0101] o t =sigma(W io x t +b io +W ho mem t-1 +b ho );
[0102] Where: W io is the weight of the output gate; b io is the bias of the output gate; W ho is the weight of the hidden state of the output gate; b ho is the bias of the implicit state of the output gate;
[0103] Calculate the membrane potential mem of the neuron at the current moment t :
[0104] mem t =o t *tann(syn t );
[0105] The syn t and the current neuron membrane potential mem t As the input of the neuron at the next moment.
[0106] According to the neuron input of the preset autonomous obstacle avoidance network, the robot's current state information and next state information are obtained. Specifically:
[0107] Get the mean and variance of the robot at the current moment;
[0108] According to the mean and variance at the current moment, the current state information of the robot is obtained, where the current state information includes the linear velocity and angular velocity at the current moment, where:
[0109] Normalize the robot's initial state information to [0, 1], and then encode the neuron's current input into pulses through Poisson coding;
[0110] The pulse is used as the current state information of the robot and input into the preset autonomous obstacle avoidance network to calculate the mean and variance at the current moment. A new normal distribution function can be obtained based on the mean and variance at the current moment. The linear velocity and angular velocity of the robot at the current moment can be obtained by sampling the normal distribution function.
[0111] Based on the current state information of the robot, the current moment reward is obtained through the obstacle avoidance reward function. Specifically, the robot's obstacle avoidance reward function includes:
[0112]
[0113] in:
[0114] d th is the minimum distance between the robot and the target or obstacle;
[0115] d ob is the distance between the robot and the obstacle;
[0116] d tar is the distance of the robot from the destination.
[0117] After obtaining the current linear velocity and angular velocity, the robot moves in the training environment to determine whether the robot collides or reaches the preset end point during operation; if so, record done = 1, refresh the robot obstacle avoidance training environment, and start a new round of navigation tasks;
[0118] If not, record done = 0, initialize the robot's status information and re-acquire the robot's current status information and the robot's status information at the next moment; based on the robot's current status information, obtain the reward at the current moment through the obstacle avoidance reward function, and determine whether the robot collides or reaches the preset end point during operation, until done = 1.
[0119] Input the robot's status information at different times into the preset value network to obtain the robot's status evaluation;
[0120] Based on the obtained rewards and the status evaluation of the robot, the cumulative rewards are obtained through the cumulative reward function. Specifically, the cumulative reward function includes:
[0121] R=[R[0], R[1],…, R[T]];
[0122] R[t]=reward[t]+γ*reward[t+1]+γ 2 *reward[t+2]+…+γ T-t+1 *reward[T-1]+γ T-t *post_value;
[0123] in:
[0124] Post_value is the robot's status evaluation at the next moment;
[0125] The acquisition of the state evaluation of the robot at the next moment includes: inputting the state information of the robot at the next moment into the preset value network to obtain the state evaluation of the robot at the next moment;
[0126] R is the cumulative reward from 0 to T, where 0 is the current moment and T is the last moment;
[0127] R[0] is the reward at the current moment, and R[T] is the reward at the last moment;
[0128] R[t] is the expanded expression of the R sub-term;
[0129] γ is the discount factor in (0, 1);
[0130] reward[t] is the reward at time t.
[0131] Based on the state evaluation, a first loss value is obtained through a first loss function, and the first loss value is back-propagated to update the preset value network. Specifically, the first loss function includes:
[0132] advantage = R-Value;
[0133] Where: R is the cumulative reward;
[0134] Value is the cumulative status evaluation. The acquisition of the cumulative status evaluation includes: adding up all the acquired status evaluations;
[0135] The mean square error of advantage is used as the first loss value.
[0136] Based on the cumulative reward function, the second loss value is obtained through the second loss function, the second loss value is back-propagated, and the preset autonomous obstacle avoidance network is updated until the number of updates meets the threshold. Specifically, the second loss function includes:
[0137] Loss=mean(min(ratio*advantage, advantage*clip(ratio, 1-ξ, ξ);
[0138] in:
[0139] ratio=P 2 / P 1 , the normal distribution can be obtained from the mean and variance, the probability distribution is obtained by sampling the normal distribution function, and then the probability distribution is exponentially taken to obtain the probabilities P1 and P2. Specifically:
[0140] The actions corresponding to the robot's state information at different times are synthesized, and the synthesized actions are input into the preset autonomous obstacle avoidance network at the previous moment and the preset autonomous obstacle avoidance network at the current moment, respectively, to obtain the mean μ 1 , variance σ 1 , mean μ 2 and variance σ 2 , according to the mean μ 1 , variance σ 1 , mean μ 2 and variance σ 2 Find the corresponding probability P 1 and P 2 ;
[0141] clip(,,) means truncation. clip(ratio, 1-ξ, ξ) means that if ratio is less than 1-ξ, then ratio takes the value of 1-ξ. If ratio is greater than ξ, then ratio takes the value of ξ. If ratio is within the range of (1-ξ, ξ), then ratio remains unchanged.
[0142] ξ is a hyperparameter, which indicates the proportion of updated parameters retained, and its value is in the range of (0, 1).
[0143] Get the trained robot autonomous obstacle avoidance network;
[0144] The initial point, target point and obstacle position information of the target robot are obtained, and the trained robot autonomous obstacle avoidance network is used to complete the obstacle avoidance task.
[0145] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the technical principles of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.
Claims
1. A robot autonomous obstacle avoidance method, characterized in that: include: Obtain the robot's obstacle avoidance reward function, cumulative reward function, first loss function, and second loss function; The robot's preset autonomous obstacle avoidance network is trained using the obtained robot's obstacle avoidance reward function, cumulative reward function, first loss function, and second loss function to obtain a trained robot autonomous obstacle avoidance network, including: Build a robot obstacle avoidance training environment and start navigation tasks; Initialize the robot's preset autonomous obstacle avoidance network and preset value network; Initialize the robot's initial state information; Normalize the initial state information of the robot; Get the neuron input of the preset autonomous obstacle avoidance network; According to the neuron input of the preset autonomous obstacle avoidance network, the robot's current state information and the state information of the next moment are obtained; Based on the current state information of the robot, the current moment reward is obtained through the obstacle avoidance reward function; Determine whether the robot collides or reaches the preset end point during operation; If not, record done=0, initialize the robot's status information and re-acquire the robot's current status information and the status information at the next moment; Based on the current state information reacquired by the robot, the current moment reward is obtained through the obstacle avoidance reward function; Determine whether the robot collides or reaches the preset end point during operation until done=1; Input the robot's status information at different times into the preset value network to obtain the robot's status evaluation; Based on the obtained rewards and the robot's status evaluation, the cumulative rewards are obtained through the cumulative reward function; Based on the state evaluation, a first loss value is obtained through a first loss function, and the first loss value is back-propagated to update a preset value network; Based on the cumulative reward function, a second loss value is obtained through a second loss function, the second loss value is back-propagated, and a preset autonomous obstacle avoidance network is updated until the number of updates meets a threshold; Get the trained robot autonomous obstacle avoidance network; Among them, the preset autonomous obstacle avoidance network adopts long and short time pulse neural network; The initial point, target point and obstacle position information of the target robot are obtained, and the trained robot autonomous obstacle avoidance network is used to complete the obstacle avoidance task.
2. The robot autonomous obstacle avoidance method according to claim 1, characterized in that: The robot initial state information includes: a radar signal, a linear velocity and an angular velocity of the robot's initial position.
3. The robot autonomous obstacle avoidance method according to claim 1, characterized in that: The step of obtaining the neuron input of the preset autonomous obstacle avoidance network includes: Calculate the output value of the forget gate f t , the neuron membrane potential at the previous moment The pulse input at the current moment As input: ; Where: W if is the weight of the forget gate; b if is the bias of the forget gate; W hf is the weight of the hidden state of the forget gate; b if is the bias of the hidden state of the forget gate; It is the sigmoid function; Calculate the memory gate output value i t , the neuron membrane potential at the previous moment The pulse input at the current moment As input: ; Where: W ii is the weight of the memory gate; b ii is the bias of the memory gate; W hi is the weight of the hidden state of the memory gate; b hi is the bias of the implicit state of the memory gate; Calculate temporary status g t , the neuron membrane potential at the previous moment The pulse input at the current moment As input: ; Where: W ig is the weight of the temporary state; b ig is the bias of the temporary state; W hg is the weight of the temporary state implied state; b hg is the bias of the implied state of the temporary state; Calculate the saliency current syn of the neuron at the current moment t : ; Among them: syn t-1 is the synaptic current of the neuron at the previous moment; Calculate the output value o of the output gate t : ; Where: W io is the weight of the output gate; b io is the bias of the output gate; W ho is the weight of the hidden state of the output gate; b ho is the bias of the implicit state of the output gate; Calculate the membrane potential mem of the neuron at the current moment t : ; The syn t and the current neuron membrane potential mem t As the input of the neuron at the next moment.
4. The robot autonomous obstacle avoidance method according to claim 1, characterized in that: The obtaining of the current state information of the robot according to the neuron input of the preset autonomous obstacle avoidance network includes: Get the mean and variance of the robot at the current moment; According to the mean and variance at the current moment, the current state information of the robot is obtained, wherein the current state information includes the linear velocity and angular velocity at the current moment.
5. The robot autonomous obstacle avoidance method according to claim 1, characterized in that: The robot is judged whether a collision occurs or reaches a preset end point during operation. If so, done=1 is recorded, the robot obstacle avoidance training environment is refreshed, and a new round of navigation tasks is started.
6. The robot autonomous obstacle avoidance method according to claim 1, characterized in that: The robot's obstacle avoidance reward function includes: ;in: is the minimum distance between the robot and the target or obstacle; is the distance between the robot and the obstacle; is the distance of the robot from the destination.
7. The robot autonomous obstacle avoidance method according to claim 1, characterized in that: The cumulative reward function includes: ; ; Among them: Post - value is the state evaluation of the robot at the next moment, and obtaining the state evaluation of the robot at the next moment includes: inputting the state information of the robot at the next moment into a preset value network to obtain the state evaluation of the robot at the next moment; R is the cumulative reward from 0 to T, where 0 is the current moment and T is the last moment; R[0] is the reward at the current moment, R[T] is the reward at the last moment; R[t] is the expanded expression of the R sub-item; γ is the discount factor in (0, 1); reward[t] is the reward at moment t.
8. The robot autonomous obstacle avoidance method according to claim 4, characterized in that: The first loss function includes: advantage=R-Value; Where: R is the cumulative reward; Value is the cumulative status evaluation, and the acquisition of the cumulative status evaluation includes: accumulating all acquired status evaluations; The mean square error of advantage is used as the first loss value.
9. The robot autonomous obstacle avoidance method according to claim 8, characterized in that: The second loss function includes: Among them: ratio = P2 / P1, synthesize the actions corresponding to the state information of the robot at different times, and input the synthesized actions into the preset autonomous obstacle avoidance network at the previous moment and the preset autonomous obstacle avoidance network at the current moment, and get the mean μ1 and variance 1. Mean μ2 and variance 2. According to the mean μ1 and variance 1. Mean μ2 and variance 2 Obtain the corresponding probabilities P1 and P2; clip(,,) means truncation, clip(ratio, ) means if ratio is less than , then the value of ratio is , if ratio is greater than , then the value of ratio is , if ratio is located at ( ) range, the value of ratio remains unchanged; is a hyperparameter, indicating the proportion of updated parameters to be retained, The value is in the range of (0, 1).
