Unlock instant, AI-driven research and patent intelligence for your innovation.

Path planning method for single robot based on q-learning algorithm

A technology of path planning and robotics, applied in computer parts, instruments, computing, etc., can solve problems such as missing the optimal solution, affecting the effect of path planning, and insufficient exploration completeness

Active Publication Date: 2021-03-30
CHONGQING UNIV OF TECH
View PDF8 Cites 0 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

The cost of this approach is: the learning system will miss the optimal solution due to insufficient exploration of the environment (the value of the exploration factor ε is too small), and can only converge to a suboptimal solution, and may even converge to a Ordinary solution, this defect will affect the effect of path planning

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Path planning method for single robot based on q-learning algorithm
  • Path planning method for single robot based on q-learning algorithm
  • Path planning method for single robot based on q-learning algorithm

Examples

Experimental program
Comparison scheme
Effect test

Embodiment 1

[0094] Such as figure 1 Shown: A single-robot path planning method based on the Q-Learning algorithm, including the following steps:

[0095] S1: Initialize the action set A, the state set S, the maximum number of iterations n, the maximum number of exploration steps m, the minimum number of paths MinPathNum, the maximum number of successful pathfinding MaxSuccessNum, the exploration factor ε, the single update step size of the exploration factor eSize, and the change of the exploration factor Period eCycle, maximum counting threshold h, start update time B(s, a), finish update time, action value function Q(s, a), state-action access times C(s, a), reward function storage U( s, a), the number of times of successful pathfinding SuccessNum, the number of successful paths PathNum, the PathList of successful paths, the storage table List of successful paths, the number of iterations i, the current time t and the target state parameters.

[0096] S2: Determine whether the number o...

Embodiment 2

[0121] This embodiment discloses a simulation experiment of path planning for a single robot.

[0122] 1. Description of the simulation experiment

[0123] 1) During the simulation experiment, the software platform uses Windows 10 operating system, the CPU uses Inter Core I5-8400, and the size of the running memory is 16GB. The path planning algorithm of the single robot system will use Python language and TensorFlow deep learning tool to complete the simulation experiment, and the multi-robot path planning algorithm will be written on the matlab2016a simulation software using matlab language.

[0124] 2) This paper will use the grid method to describe the environment, divide the working space of the robot system into small grids one by one, and each small grid can represent a state of the robot system. In the map, the white grid indicates the safe area, and the black grid indicates the existence of obstacles.

[0125] The target state and obstacles in the environment are st...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The present invention relates to the technical field of road robot path planning, in particular to a single robot path planning method based on the Q-Learning algorithm, comprising: initializing the parameters of the algorithm; selecting an action command, and calculating and generating running state parameters and reward functions according to the action command; If the state parameter is equal to the termination state parameter and is equal to the target state parameter, then the successful path is stored in the successful path storage table; otherwise, when the start update time is less than or equal to the current time and the number of visits of the state-action pair is equal to the maximum count threshold, update the action value function, and store the operating state parameters in the successful path; repeat the above steps until the maximum number of iterations is reached; repeat the selection of action instructions and the generation of state parameters according to the action value function, and obtain the optimal path for a single robot. The invention can better improve the update learning speed and path planning effect of the learning system when the Q-Learning algorithm is used for path planning of a single robot.

Description

technical field [0001] The invention relates to the technical field of robot path planning, in particular to a single robot path planning method based on a Q-Learning algorithm. Background technique [0002] Mobile robots have a wide range of applications, such as home, agriculture, industry, military and other fields that have mobile robots. The three cores in the research field of controlling robot movement are robot positioning, task assignment and path planning technology. Among them, path planning is the primary condition for the mobile robot to reach the task goal and complete the task content. For example: household service cleaning robots need reasonable path planning for the indoor environment to complete cleaning tasks; agricultural picking robots need path planning to walk between crops to complete picking tasks; industrial robots also need path planning to work in shared workspaces Complete the given task. [0003] Single-robot systems are widely used in house...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
Patent Type & Authority Patents(China)
IPC IPC(8): G06K9/62G06Q10/04G05D1/02
CPCG06Q10/047G05D1/0221G06F18/29G06F18/214
Inventor 李波易洁梁宏斌
Owner CHONGQING UNIV OF TECH