Robot motion planning and obstacle avoidance decision-making method inspired by parallel reasoning mechanism

Abstract

The present invention relates to the field of robot control technology, and in particular to a robot motion planning and obstacle avoidance decision-making method inspired by a parallel reasoning mechanism, the method comprising: decomposing and modeling the parallel reasoning brain mechanism of the human brain; designing a parallel arbitration module and embedding it into a planning method, while evaluating the binary operations of utilization and exploration; constructing an inference buffer, and coordinating the behavior strategies therein based on the designed reliability inference function; multiple behavior strategies guide the robot motion planning to generate example samples, and dynamically adjust the behavior strategies to adapt to complex environments; in the process of collecting example samples, the robot uses deep reinforcement learning to perform self-updating learning based on previous successful experiences, and guides the robot to explore more potential environments. The present invention enables the robot to accurately reach the target position in a complex environment, and to perform efficient planning and precise obstacle avoidance for complex terrain without large-scale data and large computing power.

Description

Technical Field

[0001] The invention relates to the field of robot control technology, and in particular to a robot motion planning and obstacle avoidance decision method inspired by a parallel reasoning mechanism. Background Art

[0002] With the rapid development and widespread deployment of autonomous robot systems, the role of robots in various fields has become increasingly prominent. Robots are now widely used to replace humans in complex and dangerous tasks, such as intelligence reconnaissance, assisting medical surgery, and logistics transportation, due to their diverse functions, safety, and efficiency. Many robots are involved in performing tasks in constrained environments, such as fast traversal of complex environments, emergencies during autonomous driving, and assembly of parts on production lines. These scenarios place extremely high demands on the robot's motion planning capabilities, including how to safely and efficiently plan paths, quickly avoid obstacles, and adjust and control motion trajectories in real time. For work scenarios that require high efficiency and high precision, the operating environment is usually high-dimensional, continuous, and highly uncertain, involving a large number of state variables, complex decision spaces, and dynamic obstacles.

[0003] Therefore, in dynamic high-dimensional continuous environments, sampling-based planning methods are a key means. Robots face the following challenges in achieving efficient planning and rapid obstacle avoidance through action decision-making based on sampling:

[0004] First, how to overcome the exploration-exploitation trade-off dilemma and make fast and accurate maneuver decisions in high-dimensional continuous environments. Robots learn to exploit strategies by exploring the environment to collect more information, but learning exploration strategies requires good utilization to evaluate exploration. This requires a flexible trade-off mechanism to determine when the robot exploits existing information and when to explore the environment. However, traditional sampling planning methods usually use a uniform sampling strategy, that is, indiscriminately exploring the entire space to obtain new information, and fail to fully utilize the structural information of the actual problem itself. Although heuristic biased sampling methods are conducive to balancing exploration and utilization, they are manually designed to solve specific tasks, so they are not effective when dealing with new problems and cannot be effectively generalized.

[0005] Second, in a dynamic and uncertain environment, how can the robot continuously update its experience knowledge and achieve accurate obstacle avoidance in complex terrain. When faced with obstacles that are fast or close, the robot must be able to make decisions quickly and accurately to avoid the obstacles. Traditional sampling planning methods handle each task independently and fail to make full use of past experience and established data models. Learning-based planning methods have become the main way to enhance motion planning performance.

[0006] To address these two major challenges, it is necessary to explore smarter sampling methods that balance the exploration-exploitation dilemma and dynamically respond to uncertainties in the environment.

[0007] At present, planning methods based on learning can be mainly divided into: reinforcement learning, imitation learning, and brain-like learning. Planning methods based on reinforcement learning are usually random, especially for complex tasks, which usually takes a lot of time. Therefore, some studies have introduced imitation learning for human guidance in the learning process. However, robots require a large number of example samples and manual labels during the learning process, and when facing new environments, more data needs to be collected. This makes it impossible for robots to learn from very limited example samples. In recent years, brain-like learning has made remarkable progress, such as the imitation of episodic memory and the application of attention mechanisms. Inspired by the reasonable allocation of human cognitive resources, RR-RL2 can make the most reasonable decision under given constraints. Compared with traditional planning methods, single-strategy guided brain-like learning methods have certain advantages. However, they fail to consider the coordinated learning guided by multiple strategies, and ignore the collection of optimized example samples; the learned model still finds it difficult to balance the dilemma of exploration and utilization when facing maneuvering decisions.

[0008] The shortcomings of existing methods are: 1) Robot learning requires a large amount of data and can only be used for specific environments and tasks, failing to fully utilize experiential knowledge; 2) It is difficult to balance the dilemma of exploration and utilization; 3) In the process of collecting example samples, their optimization is neglected; 4) It fails to draw on the multi-strategy coordination mechanism of the human brain and introduce multi-strategy guided coordinated learning. Summary of the invention

[0009] In order to solve the technical problems of the prior art that a large number of samples are required and the accuracy of planning and obstacle avoidance in complex terrain is low, the embodiments of the present invention provide a robot motion planning and obstacle avoidance decision method inspired by a parallel reasoning mechanism, a robot motion planning and obstacle avoidance decision device and a storage medium. The technical solution is as follows:

[0010] On the one hand, a robot motion planning and obstacle avoidance decision method inspired by a parallel reasoning mechanism is provided, the method is implemented by a robot motion planning and obstacle avoidance decision device, and the method comprises:

[0011] S1, the robot control end obtains the current position and target position of the robot, according to the current position and target position of the robot;

[0012] S2, the robot determines a reasoning buffer and a strategy set, wherein the reasoning buffer and the strategy set both include a plurality of non-learning planning methods and a learning-based planning method b u ;

[0013] S3, the robot control end calculates the prior reliability of each planning method in the reasoning buffer;

[0014] S4, the robot control end selects a local optimal behavior strategy according to the prior reliability of each planning method, and sends the local optimal behavior strategy to the robot;

[0015] S5. After receiving the local optimal behavior strategy, the robot obtains the target path under the guidance of the local optimal behavior strategy, executes the target path, and fills the target path into the training buffer as a training sample. When the training buffer reaches a certain number, the robot u Conduct an update study;

[0016] S6: If the robot receives b u As a local optimal behavior strategy, modeling is performed according to the candidate states to obtain a correlation state model; the robot designs a neural constraint observation function based on the correlation state model;

[0017] S7, calculating the observed values ​​of the candidate states according to the neural constraint observation function, and determining the maximum observed value;

[0018] S8, if the maximum observed value is greater than a preset threshold, the candidate state corresponding to the maximum observed value is expanded to the next state; if all observed values ​​are less than or equal to the preset threshold, the robot samples a new temporary detection state and repeats S7-S8 until the next state is determined;

[0019] S9, the robot in strategy b u Under the guidance of , new states are continuously expanded until the end is reached, the target path is obtained, and the target path is executed.

[0020] On the other hand, a robot motion planning and obstacle avoidance decision-making device is provided, which includes: a processor; a memory, on which computer-readable instructions are stored, and when the computer-readable instructions are executed by the processor, any one of the robot motion planning and obstacle avoidance decision-making methods inspired by the parallel reasoning mechanism is implemented.

[0021] On the other hand, a computer-readable storage medium is provided, in which at least one instruction is stored, and the at least one instruction is loaded and executed by a processor to implement any one of the above-mentioned robot motion planning and obstacle avoidance decision methods inspired by the parallel reasoning mechanism.

[0022] The beneficial effects brought about by the technical solution provided by the embodiment of the present invention include at least:

[0023] In the embodiment of the present invention, the robot control end obtains the current position and the target position of the robot, and according to the current position and the target position of the robot, the robot determines the inference buffer and the strategy set, and the inference buffer and the strategy set both include multiple non-learning planning methods and a learning-based planning method b u ; The robot control end calculates the prior reliability of each planning method in the inference buffer; The robot control end selects the local optimal behavior strategy according to the prior reliability of each planning method and sends the local optimal behavior strategy to the robot; After receiving the local optimal behavior strategy, the robot obtains the target path under the guidance of the local optimal behavior strategy, executes the target path, and fills the target path into the training buffer as a training sample. When the training buffer reaches a certain number, the robot performs b u Perform an update learning; if the robot receives b u As the local optimal behavior strategy, the model is built according to the candidate states to obtain the correlation state model; the robot designs a neural constraint observation function based on the correlation state model; the observation value of the candidate state is calculated according to the neural constraint observation function, and the maximum observation value is determined; if the maximum observation value is greater than the preset threshold, the candidate state corresponding to the maximum observation value is expanded to the next state; if all observation values ​​are less than or equal to the preset threshold, the robot samples a new temporary detection state and repeats the above steps until the next state is determined; the robot in strategy b u Under the guidance of , the new state is continuously expanded until the end is reached, the target path is obtained, and the target path is executed. The present invention makes full use of empirical knowledge, does not require a large amount of sample data, draws on the multi-strategy coordination mechanism of the human brain, and introduces multi-strategy guided coordination learning, so that the robot can accurately reach the target position in a complex environment, and perform efficient planning and precise obstacle avoidance for complex terrain without large-scale data and high computing power. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0025] Figure 1 It is a flow chart of a robot motion planning and obstacle avoidance decision method inspired by a parallel reasoning mechanism provided by an embodiment of the present invention;

[0026] Figure 2 is a schematic diagram of the structure of a robot learning environment provided by an embodiment of the present invention;

[0027] Figure 3It is a structural diagram of a parallel reasoning mechanism modeling provided by an embodiment of the present invention;

[0028] Figure 4 It is a flow chart of a robot motion planning and obstacle avoidance decision method inspired by a parallel reasoning mechanism provided by an embodiment of the present invention;

[0029] Figure 5 is a schematic diagram of an environment visualization interface provided by an embodiment of the present invention;

[0030] Figure 6 This is a schematic diagram of a Hard2 data set visualization result interface provided by an embodiment of the present invention;

[0031] Figure 7 This is a schematic diagram of a Maze3 data set visualization result interface provided by an embodiment of the present invention;

[0032] Figure 8 It is a structural schematic diagram of a robot motion planning and obstacle avoidance decision-making device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0033] The technical solution of the present invention is described below in conjunction with the accompanying drawings.

[0034] In the embodiments of the present invention, words such as "exemplarily" and "for example" are used to indicate examples, illustrations or explanations. Any embodiment or design described as "example" in the present invention should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of the word "example" is intended to present the concept in a specific way. In addition, in the embodiments of the present invention, the meaning expressed by "and / or" can be both, or it can be either of the two.

[0035] In the embodiments of the present invention, "image" and "picture" can sometimes be used interchangeably. It should be noted that when the difference between them is not emphasized, the meanings they intend to express are the same. "of", "corresponding, relevant" and "corresponding" can sometimes be used interchangeably. It should be noted that when the difference between them is not emphasized, the meanings they intend to express are the same.

[0036] In the embodiments of the present invention, sometimes a subscript such as W1 may be mistakenly written as a non-subscript such as W1. When the difference is not emphasized, the meanings to be expressed are consistent.

[0037] In order to make the technical problems, technical solutions and advantages to be solved by the present invention more clear, a detailed description will be given below with reference to the accompanying drawings and specific embodiments.

[0038] The embodiment of the present invention provides a robot motion planning and obstacle avoidance decision method inspired by a parallel reasoning mechanism, the method is implemented by a robot motion planning and obstacle avoidance decision system based on a parallel reasoning mechanism, and the robot motion planning and obstacle avoidance decision system based on a parallel reasoning mechanism includes a robot control terminal and a robot. Figure 1 The flowchart of the robot motion planning and obstacle avoidance decision method inspired by the parallel reasoning mechanism is shown in FIG. The processing flow of the method may include the following steps:

[0039] S11, the robot control end obtains the current position and target position of the robot, according to the current position and target position of the robot;

[0040] S12, the robot determines the reasoning buffer and the strategy set, the reasoning buffer and the strategy set both include multiple non-learning planning methods and a learning-based planning method b u ;

[0041] S13, the robot control end calculates the prior reliability of each planning method in the reasoning buffer;

[0042] S14, the robot control end selects the local optimal behavior strategy according to the prior reliability of each planning method, and sends the local optimal behavior strategy to the robot;

[0043] S15. After receiving the local optimal behavior strategy, the robot obtains the target path under the guidance of the local optimal behavior strategy, executes the target path, and fills the target path into the training buffer as a training sample. When the training buffer reaches a certain number, the robot u Conduct an update study;

[0044] S16, if the robot receives b u As a local optimal behavior strategy, a model is built based on the candidate states to obtain a correlation state model; the robot designs a neural constraint observation function based on the correlation state model;

[0045] S17, calculating the observed value of the candidate state according to the neural constraint observation function, and determining the maximum observed value;

[0046] S18, if the maximum observed value is greater than the preset threshold, the candidate state corresponding to the maximum observed value is expanded to the next state; if all observed values ​​are less than or equal to the preset threshold, the robot samples a new temporary detection state and repeats S17-S18 until the next state is determined;

[0047] S19, robot in strategy b u Under the guidance of , new states are continuously expanded until the end is reached, the target path is obtained, and the target path is executed.

[0048] Optionally, the robot in S12 determines a reasoning buffer and a strategy set, wherein the reasoning buffer and the strategy set both include a plurality of non-learning planning methods and a learning-based planning method b u ,include:

[0049] Based on multiple non-learning planning methods b1, b2 and a learning-based planning method b u , construct the inference buffer B as follows (1): I :

[0050] B I = {b1, b2, b u} (1)

[0051] Based on multiple non-learning planning methods b1, b2, b3 and a learning-based planning method b u , construct the inference buffer B as follows (2): S :

[0052] B S ={b1, b2, b3, b4} (2)

[0053] Among them, b1, b2, b3 are non-learning planning methods, b u It is a planning method based on learning.

[0054] Optionally, the robot control end of S13 calculates the prior reliability of each planning method in the inference buffer, including:

[0055] For a planning problem U l , according to the following formula (3), calculate the method b in the inference buffer i The ex post reliability η i，l :

[0056]

[0057] Among them, i represents the order of the planning method, t represents the current time of the robot, st-1 represents the current state of the robot, and s t Represents based on state s t-1 The next state, Indicates following behavioral strategy b i The state transition function, f colli (s t ,s t-1 ) is from s t-1 Move to s t The number of collision detections, c([s t ,s t-1 ]) is from s t-1 Move to st The path cost, is the first normalization factor;

[0058] According to the following formula (4), the planning method b is calculated i The ex ante reliability τ i，l :

[0059]

[0060] Among them, B I represents the inference buffer, b represents the planning method, and the parameter α is the perception volatility constant. Represents the first l-1 planning problem method b i The average ex post reliability, is the second normalization factor.

[0061] Optionally, the robot control end of S14 selects a local optimal behavior strategy according to the prior reliability of each planning method, including:

[0062] The robot control end infers each planning method based on the prior reliability and selects the method with the maximum prior reliability as the local optimal behavior strategy b * .

[0063] Optionally, after receiving the local optimal behavior strategy, the robot of S15 obtains the target path under the guidance of the local optimal behavior strategy, executes the target path, and fills the target path into the training buffer as a training sample, including:

[0064] Construct the planning problem U l = {S,S f , S w ,s s ,s g}, which involves the configuration space Including all degrees of freedom of the robot, obstacle-free space S f =S\S o , is the obstacle space set, S w is the working space, S s , S g ∈S starting state and goal state;

[0065] According to the following formula (5), in the local optimal behavior strategy b * Under the guidance of U l , get the target path, and execute the target path ξ l :

[0066] ξ l ={e t :[st ,s t-1 ]} te[1,T] ,s t ∈S,s0=s s ,s T =s g

[0067]

[0068] Among them, e t Indicates t to t-1 The edge of c(s0,s g ) is the path cost of the path, ||s i ,s i-1 ||2 is state s t to t-1 Costs;

[0069] According to the following formula (6), the target path is filled into the training buffer as a training sample:

[0070] D=D∪ξ l (6)

[0071] Where D represents the training buffer.

[0072] Optionally, S16 performs modeling according to the candidate states to obtain a correlation state model, including:

[0073] The model is built according to the following formula (7):

[0074] in

[0075] Among them, σ t (s) represents the variance estimate after expanding t actions, s′ represents the next state of candidate state s, k(s′,s) is the kernel function, and K(s) represents the kernel function between adjacent states.

[0076] Optionally, the S16 robot designs a neural constraint observation function based on the correlation state model, including:

[0077] The neural constraint observation function is designed according to the following formula (8):

[0078]

[0079] Where ψ(s) represents the neural constraint observation function; is the average reward after t expansions, which is calculated by the value function V(s) of deep reinforcement learning; σ t(s) is the variance estimator after t times of expansion, ε represents the weight, c([s′, s g ]) represents the path cost from the next state to the target state.

[0080] Optionally, the calculation formula of the value function V(s) is as follows:

[0081]

[0082] in, represents the fully connected layer with parameter W2, Features extracted by the neural network.

[0083] The embodiment of the present invention provides a robot motion planning and obstacle avoidance decision method inspired by a parallel reasoning mechanism, the method is implemented by a robot motion planning and obstacle avoidance decision system based on a parallel reasoning mechanism, and the robot motion planning and obstacle avoidance decision system based on a parallel reasoning mechanism includes a robot control terminal and a robot. In order to verify the proposed method, considering safety and cost factors, the embodiment of the present invention can build a simulated robot operation platform (or virtual simulation environment), including the robot's learning environment, working environment and task scene settings.

[0084] The following is an introduction to the platform building and modeling process:

[0085] 1. Build a simulated robot operation platform;

[0086] like Figure 2 As shown, the platform can be used to learn and output policy networks and value networks according to the current environment, and can also be used to verify the effectiveness of the learned model in executing tasks. The constructed environment simulates the physical properties of the real environment to a certain extent, such as collisions. The embodiment of the present invention can build a simulation environment based on pybullet, mainly including task scene settings, virtual robots with different degrees of freedom, virtual cameras, etc.

[0087] The robot's learning environment serves as the control end of the simulation operation platform, processing input data and implementing learning algorithms. The robot learning environment controls the robot's maneuvering decisions by sending instructions to the simulation environment, and receives feedback information from the simulation environment to evaluate the value of the execution decision.

[0088] The robot's learning environment includes a coordination module, a planning module, a collision detection module, and a neural network module. The coordination module is used to select a behavior strategy from the inference buffer according to the current task environment; the planning module is used to follow the current behavior strategy to guide the robot to make a maneuver decision; the collision detection module is used to detect whether a collision occurs after the robot executes a maneuver decision; the network module is used to load the network model according to the task information and continuously update the network model through learning. The neural network can include CNN, feature fusion module, and multi-layer perceptron. Its specific use method and principle are commonly used methods in the prior art, and the embodiments of the present invention are not described in detail here.

[0089] 2. Model the brain mechanism of parallel reasoning in the human brain;

[0090] Model the parallel reasoning mechanism in PFC, such as Figure 3 Assume that the inference buffer can accommodate up to 4 behavior strategies, and use Bayesian inference to infer the buffer B. I The monitored strategy evaluates its reliability, including the reliability of the behavioral strategy being executed τ k , other alternative behavior strategy reliability τ i and τ j , temporary behavior strategy reliability τ p The reasoning process can be summarized into the following steps:

[0091] (1) If τ k is reliable, then the ongoing policy will choose the action with the maximum reward.

[0092] (2) If all strategies in the inference buffer become unreliable, the robot will explore a temporary behavior strategy τ p .

[0093] (3) The robot will resume the utilization state in two situations:

[0094] (a) When B I When any monitoring strategy in becomes reliable, it is used and the temporary behavior strategy is rejected.

[0095] (b) When the temporary behavior strategy is reliable and other monitoring strategies are unreliable, the temporary behavior strategy is used and stored in B I middle.

[0096] like Figure 4 The flowchart of the robot motion planning and obstacle avoidance decision method inspired by the parallel reasoning mechanism is shown in FIG. The processing flow of the method may include the following steps:

[0097] S21. The robot control end obtains the current position and target position of the robot, based on the current position and target position of the robot.

[0098] S22, the robot determines a reasoning buffer and a strategy set, where the reasoning buffer and the strategy set both include multiple non-learning planning methods and a learning-based planning method b u ,include:

[0099] Based on multiple non-learning planning methods b1, b2 and a learning-based planning method b u , construct the inference buffer B as follows (1): I :

[0100] B I = {b1, b2, b u} (1)

[0101] Based on multiple non-learning planning methods b1, b2, b3 and a learning-based planning method b u , construct the inference buffer B as follows (2): S :

[0102] B S = {b1, b2, b3, b u} (2)

[0103] Among them, b1, b2, b3 are non-learning planning methods, b u It is a planning method based on learning.

[0104] In a feasible implementation, a reasoning buffer B is designed by imitating the monitoring mechanism of the human brain. I and strategy set B S , can simultaneously monitor four different behavioral strategies, namely B I = {b1, b2, b u} and B S = {b1, b2, b3, b u}, where b1, b2, b3 are planning methods based on sampling, b u It is a learning-based plan.

[0105] S23. The robot control end calculates the prior reliability of each planning method in the reasoning buffer.

[0106] Optionally, the robot control end of S23 calculates each planning method b in the reasoning buffer i Ex ante reliability, including:

[0107] For a planning problem U l , according to the following formula (3), calculate the method b in the inference buffer i The ex post reliability η i，l :

[0108]

[0109] Where i represents the order of the planning method, t represents the current time of the robot, and s t-1 Indicates the current state of the robot, s t Represents based on state s t-1 The next state, Indicates following behavioral strategy b i The state transition function, f colli (s t ,s t-1 ) is from s t-1 Move to s t The number of collision detections, c([s t ,s t-1 ]) is from s t-1 Move to s t The path cost, is the first normalization factor;

[0110] According to the following formula (4), the planning method b is calculated i The ex ante reliability τ i，1 :

[0111]

[0112] Among them, B I represents the inference buffer, b represents the planning method, and the parameter α is the perception volatility constant. Represents the first l-1 planning problem method b i The average ex post reliability, is the second normalization factor.

[0113] In one feasible implementation, for any given behavior strategy b i , its reliability evaluation when dealing with the lth task scenario involves two different stages. The first stage occurs before taking action and is called ex ante reliability τ i，l , which reflects the confidence level assigned to each behavior strategy before executing an action, and represents the expected evaluation of the reliability of the behavior strategy in guiding the robot. In contrast, the ex post reliability η i，l Occurs after action has been taken and is determined by the actual observed outcomes through evaluation. η i，l Taking into account the feedback received from the environment and providing a measure of the reliability of the behavior policy based on its actual performance, this dual estimate enhances the robot’s ability to dynamically adjust its planning in complex environments.

[0114] Based on the designed reliability inference function, B IAccording to the behavior strategy in the current environment, a suitable behavior strategy is selected to guide the robot to plan the task, and the generated example samples are filled into the training buffer D, and the behavior strategy is dynamically adjusted to adapt to the complex environment.

[0115] S24. The robot control end selects the local optimal behavior strategy according to the prior reliability of each planning method, and sends the local optimal behavior strategy to the robot.

[0116] Optionally, the robot control end of S24 selects a local optimal behavior strategy according to the prior reliability of each planning method, including:

[0117] The robot control end infers each planning method based on the prior reliability and selects the method with the maximum prior reliability as the local optimal behavior strategy b * .

[0118] S25. After receiving the local optimal behavior strategy, the robot obtains the target path under the guidance of the local optimal behavior strategy, executes the target path, and fills the target path into the training buffer as a training sample. When the training buffer reaches a certain number, the robot performs b u Conduct an update study.

[0119] Optionally, after receiving the local optimal behavior strategy, the robot of S25 obtains the target path under the guidance of the local optimal behavior strategy, executes the target path, and fills the target path into the training buffer as a training sample, including:

[0120] Construct the planning problem U l = {S,S f , S w ,s s ,s g}, which involves the configuration space Including all degrees of freedom of the robot, obstacle-free space S f =S\S o , is the obstacle space set, S w is the working space, s s ,s g ∈S starting state and goal state;

[0121] According to the following formula (5), in the local optimal behavior strategy b * Under the guidance of U l , get the target path, and execute the target path ξ l :

[0122] ξ l ={e t :[s t ,st-1 ]} t∈[1,T] ,s t ∈S,s0=s s ,s T =s g

[0123]

[0124] Among them, e t Indicates t to t-1 The edge of c(s0,s g ) is the path ξ l The path cost, ||s i ,s i-1 ||2 is state s t to t-1 Costs;

[0125] According to the following formula (6), the target path is filled into the training buffer as a training sample:

[0126] D=D∪ξ l (6)

[0127] Where D represents the training buffer.

[0128] In one possible implementation, the robot selects a parent state s in the current search tree based on a tree sampling algorithm. t-1 , if it is the initial moment, the starting point is the parent state. Then, following the guidance of the deep reinforcement learning policy function, a series of candidate states are generated in the neighborhood of the parent state.

[0129] S26, if the robot receives b u As a local optimal behavior strategy, the candidate states are modeled to obtain a correlation state model. Based on the correlation state model, the robot designs a neural constraint observation function.

[0130] In a feasible implementation, due to the correlation between adjacent states and the number of states gradually increasing as the algorithm proceeds, the traditional confidence interval upper bound algorithm is not directly applicable to state selection. It is necessary to model these correlation states to obtain a correlation state model. The corresponding specific process can be as follows:

[0131] The model is built according to the following formula (7):

[0132] in

[0133] Among them, σ t(s) represents the variance estimate after expanding t actions, s′ represents the next state of candidate state s, k(s′,s) is the kernel function, and K(s) represents the kernel function between adjacent states.

[0134] Alternatively, the generalized structure of the task scenario can be captured more accurately using prior and environmental information. Combining the prior and target state, a neural constraint observation function can be designed based on the correlation state model. The specific steps can be as follows:

[0135] The neural constraint observation function is designed according to the following formula (8):

[0136]

[0137] Where ψ(S) represents the neural constraint observation function; is the average reward after t expansions, which is calculated by the value function V(s) of deep reinforcement learning; σ t (s) is the variance estimator after t times of expansion, ε represents the weight, c([s′, s g ]) represents the path cost from the next state to the target state.

[0138] Optionally, the calculation formula of the value function V(s) in the above steps is as follows:

[0139]

[0140] in, represents the fully connected layer with parameter W2, Features extracted by the neural network.

[0141] S27. Calculate the observation value of the candidate state according to the neural constraint observation function, and determine the maximum observation value.

[0142] S28. If the maximum observation value is greater than the preset threshold, the candidate state corresponding to the maximum observation value is expanded to the next state; if all observation values ​​are less than or equal to the preset threshold, the robot samples a new temporary detection state and repeats S27-S28 until the next state is determined.

[0143] In one feasible implementation, if the maximum observed value of a candidate state is better than a threshold β, the candidate state with the maximum observed value will be expanded to s new In contrast, if all observations are below β, the robot will explore new behavioral strategies and sample a series of temporary exploration states.

[0144] The robot switches to exploit mode in the following two scenarios:

[0145] When all exploration rounds are completed and no state with an observation value better than the threshold β is found, the robot will stop the exploration mode and select the state with the highest observation value among all the states found so far.

[0146] When the maximum observed value of the detection state exceeds β, the state will be expanded to s new , the robot switches to exploit mode.

[0147] S29, robot in strategy b u Under the guidance of , new states are continuously expanded until the end is reached, the target path is obtained, and the target path is executed.

[0148] It should be noted that in the above process of collecting sample examples, the robot can use deep reinforcement learning to perform self-updating learning based on previous successful experiences, as illustrated by the following example:

[0149] It is assumed that every 200 sample samples are collected for training, for a total of 10 training cycles. In the nth training cycle, the training buffer is represented by D n As the training buffer is optimized, the learning of the value function V(s) and the strategy function π(a|s) is continuously iterated and updated. The specific learning steps are as follows:

[0150] For each mission planning scenario, follow step 4 from B I Select the behavior strategy b with the highest current reliability * , then b * Guide the generated sample to fill in D n middle.

[0151] Once the 200 sample samples are completed, D n All the books in the book are used for training b u The value function V(s) and the policy function π(a|s) in the equation are:

[0152]

[0153] Among them, λ is a regularization factor, v t According to D n It is calculated from the empirical trajectory stored in W are all the learnable parameters of the neural network.

[0154] The value function V(s) and the policy function π(a|s) share parameters and are continuously learned and updated in the process of accumulating experience. Since the policy function is trained based on previous successful experiences under different problems, it can guide the robot to explore more promising spaces. At the same time, this optimization method of the training buffer enables the network to perform self-updating learning and dynamically adjust the model to adapt to complex environments according to different planning problems.

[0155] The following is an explanation of the experimental results of the experiment according to the above steps:

[0156] 1. Experimental description

[0157] This experiment uses a high-dimensional continuous planning task to verify the proposed method. The goal of the task is to find a collision-free path between the starting state and the target state. A total of 8 different data sets are used in this experiment. Each data is described in detail as follows. All environment visualizations are shown in the following figure. Figure 5 The visualization results of Hard2 dataset and Maze3 dataset are shown in Figure 6 and Figure 7 shown.

[0158] (1) Maze2: A 2-DOF robot operating in a 2D workspace. It is divided into Easy2 and Hard2, where the obstacle density of Hard2 is not less than 46%.

[0159] (2) Maze3: A 3-DOF stick robot operating in a 2D workspace.

[0160] (3) UR5: A 6-DOF robot working in a 3D workspace. The obstacles are some randomly generated objects, namely two sets of boxes, poles and mats, which are generated in two different size ranges.

[0161] (4) Snake7: A 7-DOF snake robot working in a 2D workspace. The snake robot has five sticks, and the positions of their ends have 2 degrees of freedom.

[0162] (5) Kuka7: A 7-DOF robot operating in a 3D workspace. This robot is located in a fixed base position.

[0163] (6) Kuka13: A 13-DOF robot operating in a 3D workspace.

[0164] (7) Kuka14: A robot with two 7-DOF Kuka arms working in a 3D workspace. Each arm must successfully reach a target area while avoiding collisions with any obstacles in the environment. In Kuka7 and Kuka14, all boxes in each problem workspace are randomly generated.

[0165] 2. Experimental results

[0166] This experiment uses three key metrics to evaluate the performance, namely the success rate (SR) of finding a collision-free path, the path cost (CO), and the time cost quantified by the number of collision detections (CK). The success rate (SR) represents the percentage of problems that successfully find a collision-free path among all the test problems in 1000. The path cost (CO) is the sum of the Euclidean path lengths of the collision-free path found for each problem. Finally, the number of collision detections (CK) is counted by the collision detection module during the execution of the planning algorithm. The performance comparison of the proposed model and the most advanced methods on the above 8 datasets is shown in Tables 1, 2 and 3.

[0167] Table 1 Success rate (SR)

[0168]

[0169]

[0170] Table 2 Average collision check (CK)

[0171]

[0172] Table 3 Average path cost (CO)

[0173]

[0174] The present invention has shown remarkable performance in all tested simulation environments, achieving a 100% success rate. In particular, in high-dimensional environments, the present invention significantly reduces the number of collisions; and can achieve optimal or near-optimal path costs in all simulation environments. Experimental results show that the present invention is robust and stable for decision-making tasks in various complex environments. At the same time, it also shows that exploring the decision-making mechanism in the human brain may help develop more advanced and reliable decision-making systems, such as simulating human cognitive abilities.

[0175] In the embodiment of the present invention, the robot control end obtains the current position and the target position of the robot, and according to the current position and the target position of the robot, the robot determines the inference buffer and the strategy set, and the inference buffer and the strategy set both include multiple non-learning planning methods and a learning-based planning method b u; The robot control end calculates the prior reliability of each planning method in the inference buffer; The robot control end selects the local optimal behavior strategy according to the prior reliability of each planning method and sends the local optimal behavior strategy to the robot; After receiving the local optimal behavior strategy, the robot obtains the target path under the guidance of the local optimal behavior strategy, executes the target path, and fills the target path into the training buffer as a training sample. When the training buffer reaches a certain number, the robot performs b u Perform an update learning; if the robot receives b u As the local optimal behavior strategy, the model is built according to the candidate states to obtain the correlation state model; the robot designs a neural constraint observation function based on the correlation state model; the observation value of the candidate state is calculated according to the neural constraint observation function, and the maximum observation value is determined; if the maximum observation value is greater than the preset threshold, the candidate state corresponding to the maximum observation value is expanded to the next state; if all observation values ​​are less than or equal to the preset threshold, the robot samples a new temporary detection state and repeats the above steps until the next state is determined; the robot in strategy b u Under the guidance of , the new state is continuously expanded until the end is reached, the target path is obtained, and the target path is executed. The present invention makes full use of empirical knowledge, does not require a large amount of sample data, draws on the multi-strategy coordination mechanism of the human brain, and introduces multi-strategy guided coordination learning, so that the robot can accurately reach the target position in a complex environment, and perform efficient planning and precise obstacle avoidance for complex terrain without large-scale data and high computing power.

[0176] Figure 8 is a schematic diagram of the structure of a robot motion planning and obstacle avoidance decision-making device provided by an embodiment of the present invention, such as Figure 8 As shown, the robot motion planning and obstacle avoidance decision device can be used to implement the above-mentioned robot motion planning and obstacle avoidance decision method inspired by the parallel reasoning mechanism. Optionally, the robot motion planning and obstacle avoidance decision device 810 can include a processor 2001.

[0177] Optionally, the robot motion planning and obstacle avoidance decision-making device 810 may also include a memory 2002 and a transceiver 2003 .

[0178] The processor 2001, the memory 2002 and the transceiver 2003 may be connected via a communication bus.

[0179] Combine the following Figure 8 The components of the robot motion planning and obstacle avoidance decision-making device 810 are specifically introduced:

[0180] The processor 2001 is the control center of the robot motion planning and obstacle avoidance decision device 810, and can be a processor or a general term for multiple processing elements. For example, the processor 2001 is one or more central processing units (CPUs), or an application specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of the present invention, such as one or more microprocessors (digital signal processors, DSPs), or one or more field programmable gate arrays (field programmable gate arrays, FPGAs).

[0181] Optionally, the processor 2001 can perform various functions of the robot motion planning and obstacle avoidance decision-making device 810 by running or executing a software program stored in the memory 2002 and calling data stored in the memory 2002.

[0182] In a specific implementation, as an embodiment, the processor 2001 may include one or more CPUs, such as Figure 8 CPU0 and CPU1 are shown in FIG.

[0183] In a specific implementation, as an embodiment, the robot motion planning and obstacle avoidance decision device 810 may also include multiple processors, such as Figure 8 2001 and processor 2004 are shown in FIG. Each of these processors may be a single-core processor (single-CPU) or a multi-core processor (multi-CPU). A processor herein may refer to one or more devices, circuits, and / or processing cores for processing data (e.g., computer program instructions).

[0184] The memory 2002 is used to store the software program for executing the solution of the present invention, and the execution is controlled by the processor 2001. The specific implementation method can refer to the above method embodiment, which will not be repeated here.

[0185] Optionally, the memory 2002 may be a read-only memory (ROM) or other types of static storage devices that can store static information and instructions, a random access memory (RAM) or other types of dynamic storage devices that can store information and instructions, or an electrically erasable programmable read-only memory (EEPROM), a compact disc read-only memory (CD-ROM) or other optical disc storage, optical disc storage (including compressed optical disc, laser disc, optical disc, digital versatile disc, Blu-ray disc, etc.), a magnetic disk storage medium or other magnetic storage device, or any other medium that can be used to carry or store the desired program code in the form of instructions or data structures and can be accessed by a computer, but is not limited thereto. The memory 2002 may be integrated with the processor 2001, or may exist independently, and may be accessed through the interface circuit ( Figure 8 2001) is coupled to the processor 2001, which is not specifically limited in the embodiment of the present invention.

[0186] The transceiver 2003 is used to communicate with a network device or a terminal device.

[0187] Optionally, the transceiver 2003 may include a receiver and a transmitter ( Figure 8 The receiver is used to implement a receiving function, and the transmitter is used to implement a sending function.

[0188] Optionally, the transceiver 2003 may be integrated with the processor 2001, or may exist independently and communicate with the robot through the interface circuit ( Figure 8 2001) is coupled to the processor 2001, which is not specifically limited in the embodiment of the present invention.

[0189] It should be noted that Figure 8 The structure of the robot motion planning and obstacle avoidance decision device 810 shown in the figure does not constitute a limitation on the router. The actual knowledge structure recognition device may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently.

[0190] In addition, the technical effects of the robot motion planning and obstacle avoidance decision-making device 810 can refer to the technical effects of the robot motion planning and obstacle avoidance decision-making method inspired by the parallel reasoning mechanism described in the above method embodiment, and will not be repeated here.

[0191] It should be understood that the processor 2001 in the embodiment of the present invention may be a central processing unit (CPU), and the processor may also be other general-purpose processors, digital signal processors (DSP), application specific integrated circuits (ASIC), field programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.

[0192] It should also be understood that the memory in the embodiments of the present invention may be a volatile memory or a non-volatile memory, or may include both volatile and non-volatile memories. Among them, the non-volatile memory may be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory may be a random access memory (RAM), which is used as an external cache. By way of example and not limitation, many forms of random access memory (RAM) are available, such as static RAM (SRAM), dynamic random access memory (DRAM), synchronous DRAM (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SLDRAM), and direct rambus RAM (DR RAM).

[0193] The above embodiments can be implemented in whole or in part by software, hardware (such as circuits), firmware or any other combination. When implemented by software, the above embodiments can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions or computer programs. When the computer instructions or computer programs are loaded or executed on a computer, the process or function described in the embodiment of the present invention is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from one website, computer, server or data center to another website, computer, server or data center by wired (such as infrared, wireless, microwave, etc.). The computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server or data center that contains one or more available media sets. The available medium can be a magnetic medium (for example, a floppy disk, a hard disk, a tape), an optical medium (for example, a DVD), or a semiconductor medium. The semiconductor medium can be a solid-state hard disk.

[0194] It should be understood that the term "and / or" in this article is only a description of the association relationship of associated objects, indicating that there can be three relationships. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. A and B can be singular or plural. In addition, the character " / " in this article generally indicates that the associated objects before and after are in an "or" relationship, but it may also indicate an "and / or" relationship. Please refer to the context for specific understanding.

[0195] In the present invention, "at least one" means one or more, and "plurality" means two or more. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single items or plural items. For example, at least one of a, b, or c can be represented by: a, b, c, ab, ac, bc, or abc, where a, b, c can be single or multiple.

[0196] It should be understood that in various embodiments of the present invention, the size of the serial numbers of the above-mentioned processes does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0197] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.

[0198] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the above-described equipment, devices and units can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0199] In the several embodiments provided by the present invention, it should be understood that the disclosed devices, apparatuses and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another device, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0200] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0201] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0202] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention can be essentially or partly embodied in the form of a software product that contributes to the prior art. The computer software product is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the methods described in various embodiments of the present invention. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0203] The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art can easily think of changes or substitutions within the technical scope disclosed by the present invention, which should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention should be based on the protection scope of the claims.

Claims

1. A robot motion planning and obstacle avoidance decision method based on parallel reasoning mechanism, characterized in that: The method is implemented by a robot motion planning and obstacle avoidance decision system based on a parallel reasoning mechanism, and the robot motion planning and obstacle avoidance decision system based on a parallel reasoning mechanism includes a robot control terminal and a robot; The method comprises: S1, the robot control end obtains the current position and target position of the robot, according to the current position and target position of the robot; S2, the robot determines a reasoning buffer and a strategy set, wherein the reasoning buffer and the strategy set both include a plurality of non-learning planning methods and a learning-based planning method b u ; S3, the robot control end calculates the prior reliability of each planning method in the reasoning buffer; S4, the robot control end selects a local optimal behavior strategy according to the prior reliability of each planning method, and sends the local optimal behavior strategy to the robot; S5. After receiving the local optimal behavior strategy, the robot obtains the target path under the guidance of the local optimal behavior strategy, executes the target path, and fills the target path into the training buffer as a training sample. When the training buffer reaches a certain number, the robot u Conduct an update study; S6: If the robot receives b u As a local optimal behavior strategy, modeling is performed according to the candidate states to obtain a correlation state model; the robot designs a neural constraint observation function based on the correlation state model; S7, calculating the observed values ​​of the candidate states according to the neural constraint observation function, and determining the maximum observed value; S8, if the maximum observed value is greater than a preset threshold, the candidate state corresponding to the maximum observed value is expanded to the next state; if all observed values ​​are less than or equal to the preset threshold, the robot samples a new temporary detection state and repeats S7-S8 until the next state is determined; S9, the robot in strategy b u Under the guidance of , new states are continuously expanded until the end is reached, the target path is obtained, and the target path is executed; The robot of S2 determines a reasoning buffer and a strategy set, wherein the reasoning buffer and the strategy set both include a plurality of non-learning planning methods and a learning-based planning method b u ,include: Based on multiple non-learning planning methods b1, b2 and a learning-based planning method b u , construct the inference buffer B as follows (1) I : B I ={b1,b2,b u } (1) Based on multiple non-learning planning methods b1, b2, b3 and a learning-based planning method b u , construct the inference buffer B as follows (2): S : <h2 style=";text-align:left;direction:ltr">B<h2 style=";text-align:left;direction:ltr"> S <h2 style=";text-align:left;direction:ltr"> (b1, b2, b3, b)<h2 style=";text-align:left;direction:ltr"> u <h2 style=";text-align:left;direction:ltr">} (2) Among them, b1, b2, b3 are non-learning planning methods, b u It is a learning-based planning approach; The robot control end of S3 calculates the prior reliability of each planning method in the reasoning buffer, including: For a planning problem U l , according to the following formula (3), calculate the method b in the inference buffer i The ex post reliability η i,l : Among them, i represents the order of the planning method, t represents the current moment of the robot, and S t-1 Indicates the current state of the robot, s t Represents based on state s t-1 The next state, Indicates following behavioral strategy b i The state transition function, f colli (s t ,s t-1 ) is from s t-1 Move to s t The number of collision detections, c([s t ,s t-1 ]) is from s t-1 Move to s t The path cost, is the first normalization factor; According to the following formula (4), the planning method b is calculated i Ex ante reliability τ i，1 : Among them, B I represents the inference buffer, b represents the planning method, and the parameter α is the perception volatility constant. Represents the first l-1 planning problem method b i The average ex post reliability, is the second normalization factor; The step S6 of modeling the candidate states to obtain a correlation state model includes: The model is built according to the following formula (7): in Among them, σ t (s) represents the variance estimate after expanding t actions, s′ represents the next state of candidate state s, k(s′, s) is the kernel function, and K(s) represents the kernel function between adjacent states; The S6 robot designs a neural constraint observation function based on the correlation state model, including: The neural constraint observation function is designed according to the following formula (8): Where ψ(s) represents the neural constraint observation function; is the average reward after t expansions, which is calculated by the value function V(s) of deep reinforcement learning; σ t (s) is the variance estimator after t times of expansion, ε represents the weight, c([s′, s g ] represents the path cost from the next state to the target state.

2. The method according to claim 1, characterized in that The robot control end of S4 selects the local optimal behavior strategy according to the prior reliability of each planning method, including: The robot control end infers each planning method based on the prior reliability and selects the method with the maximum prior reliability as the local optimal behavior strategy b * .

3. The method according to claim 1, characterized in that After receiving the local optimal behavior strategy, the robot of S5 obtains the target path under the guidance of the local optimal behavior strategy, executes the target path, and fills the target path into the training buffer as a training sample, including: Construct the planning problem U l ={S,S f , S w , S s ,s g }, which involves the configuration space Including all degrees of freedom of the robot, obstacle-free space Sf = S\So, is the obstacle space set, S w is the working space, S s ,s g ∈S starting state and goal state; According to the following formula (5), in the local optimal behavior strategy b * Under the guidance of U l , get the target path, and execute the target path ξ l : ξ l ={e t :[s t ,s t-1 ]} t∈[1,T] ,s t ∈S,s0=s s ,s T =s g Among them, e t Indicates t to t-1 The edge of c(s0,s g ) is the path ξ l The path cost, ||s i ,s i-1 ||2 is state s t to t-1 Costs; According to the following formula (6), the target path is filled into the training buffer as a training sample: D=D∪ξ l (6) Where D represents the training buffer.

4. The method according to claim 3, characterized in that The calculation formula of the value function V(s) is as follows: in, represents the fully connected layer with parameter W2, Features extracted by the neural network.

5. A robot motion planning and obstacle avoidance decision-making device, characterized in that: The robot motion planning and obstacle avoidance decision-making device includes: processor; A memory having computer-readable instructions stored thereon, wherein when the computer-readable instructions are executed by the processor, the method according to any one of claims 1 to 4 is implemented.

6. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores program codes, which can be called by a processor to execute the method according to any one of claims 1 to 4.

Images