Robot dynamic target search and tracking method based on adaptive particle filtering tree

Abstract

The application discloses a robot dynamic target searching and tracking method based on adaptive particle filtering tree. The method can quickly and accurately calculate mutual information based on a mutual information estimation method of sigma points. In Monte Carlo tree search, a motion primitive meeting a kinematic model is used to form an action space, so that a smooth path meeting the kinematic constraint is generated. An adaptive stop strategy in the adaptive particle filtering tree can dynamically adjust a planning time domain according to reward information, so as to reduce the operation time. The application selectively searches the feasible paths in the future, and the adaptive property can dynamically adjust the planning time domain according to the current reward information, so as to save the computing resources. In combination with the action space formed by the motion primitive, the application can efficiently generate a long-time-domain smooth path meeting the kinematic model considering the influence of future observation, so as to efficiently complete the target searching and tracking problem.

Description

TECHNICAL FIELD

[0001] The present application relates to a method for searching and tracking a dynamic target in an environment by an autonomous mobile robot, in particular to an adaptive particle filter tree method, and belongs to the technical field of robot target search and tracking. BACKGROUND

[0002] Searching and tracking a dynamic target in an environment by an autonomous mobile robot is a problem that attracts much attention in the field of robots, which has wide application in environmental detection, disaster rescue and other scenarios. The target search and tracking problem is mainly divided into two stages, namely target search and target tracking. The target search stage means that the robot does not know the specific location of the target at the beginning, and needs to use sensors with limited sensing range such as cameras, laser radars, etc. to explore the environment to find the target; after finding the target, it enters the target tracking stage, that is, the robot performs motion control to try to ensure that the target is within the robot's sensing range, reducing the probability of losing the target view.

[0003] The target search and tracking problem can usually be considered as an information collection problem, that is, the robot autonomously plans the motion according to the current known information to obtain more informative environmental observations, thereby reducing the uncertainty of the robot's target estimation and improving the accuracy of target positioning. In the information collection problem, calculating the objective function and path planning are two main contents, but there are still some defects in the existing work. First, in terms of the objective function, mutual information, as a measure of the degree of correlation between variables in information theory, is an effective objective function commonly used in information collection problems. However, in its calculation process, it usually needs to calculate the entropy of the mixed distribution, which involves complex integral calculation and often has no analytical expression. Existing methods either estimate the integral through a large number of random sampling, which introduces a high computational cost, or approximate the integral through a Taylor expansion-based method, which introduces a non-negligible estimation error. These problems of calculation accuracy and efficiency of the objective function will affect the subsequent planning stage. Secondly, in terms of path planning, in the case of a series of limitations of traditional planning schemes, the planning scheme based on Monte Carlo tree search has become a widely used method in recent years. However, this method needs to fix the planning time domain in advance, and in the problem of target search with a long planning time domain, it may encounter the dilemma of planning time domain setting: if the planning time domain is too short, the robot may not be able to effectively discover and locate the target, while a too long planning time domain will make it difficult to make quick decisions. In view of the above two problems, the present application proposes an adaptive particle filter tree algorithm, which combines the mutual information estimation method based on sigma point sampling and the adaptive path planning method based on Monte Carlo tree search to obtain a complete scheme that can solve the dynamic target search and tracking problem in real time and efficiently.

[0004] In the current commonly used methods, 1, the control method based on maximizing mutual information is used to estimate the target position, so as to realize the search and tracking of the target. The method generates a series of trajectories through the motion primitive, predicts the future state of the target according to the target motion model, and obtains the optimal robot trajectory by maximizing the mutual information between the target state and the observation in the planning time domain. In order to solve the problem that the mixed entropy in the mutual information cannot be explicitly calculated, the integral estimation method based on Taylor series expansion is used to approximate the calculation of the mutual information. Specifically, the estimation method carries out low-order Taylor series expansion on the logarithmic term in the entropy, estimates the integrand, so that the integral can be calculated analytically. The mutual information estimation method based on Taylor expansion used in the method has an estimation error that cannot be ignored in the result, and the control law obtained is an open-loop control strategy, that is, only the target state is predicted according to the target motion model in the planning process, without considering the influence of future possible observation results on the planning effect. Moreover, the method assumes that the robot can obtain the observation results of the target at any time, without considering the influence of the limited sensing range of the robot.

[0005] 2, the sampling-based motion planning method is used to solve the information collection problem. The method is based on the rapid expansion random tree algorithm, samples in the feasible space to generate a tree structure, and extracts a planning scheme with information from the tree that meets the constraints. Specifically, the current robot position is taken as the root node, random sampling is carried out in the task environment, new leaf nodes are added according to the sampling results and the existing tree structure, and the information gain of the new nodes is calculated. Under the pre-specified budget constraint, the above steps are repeatedly performed to build a search tree, and finally an effective path without collision and with maximum information is extracted from the tree according to the node information gain. The sampling-based path planning method has great randomness, and the algorithm effect is greatly affected by the sampling number and sampling distribution, and the stability is poor. At the same time, the generated trajectory is relatively tortuous, and the smoothness is poor; there are many redundant points, and the path efficiency is low.

[0006] 3, a greedy planning method is used to search and track the target. The method models the problem as an optimization problem with one-step planning time domain and constraints, considers maximizing the mutual information of the next step for motion planning, and uses the gradient descent-based optimization method to solve it. The greedy planning scheme can only obtain a local optimal solution, and cannot consider the planning effect in a long time domain, so it cannot obtain a global optimal solution. For example, in the target search stage, the information amount obtained by each action in the initial search process is usually small and has little difference, which makes it difficult for the robot to find an effective path that can complete the task in a short time domain. SUMMARY

[0007] In view of the defects of the existing mutual information estimation method in the prior art background technology, such as high computational complexity and low estimation accuracy, the method proposes a mutual information estimation method based on sigma points to solve the above problems. In view of the shortcomings of the existing path planning method, such as short planning time domain, inability to obtain a global optimal solution, unstable planning effect, zigzag and non-smooth planning path, and low path efficiency. The adaptive planning scheme based on the Monte Carlo tree search method is proposed to solve the above problems.

[0008] The technical purpose of the present application is to propose a robot dynamic target search and tracking method based on adaptive particle filtering tree, which can quickly and accurately calculate mutual information based on the mutual information estimation method based on sigma points. In the Monte Carlo tree search, the motion primitives conforming to the kinematic model are used to compose the action space to generate a smooth path that meets the kinematic constraints. The adaptive stopping strategy in the adaptive particle filtering tree can dynamically adjust the planning time domain according to the reward information to reduce the computation time.

[0009] To achieve the above purpose, the technical scheme adopted by the present method is a robot dynamic target search and tracking method based on adaptive particle filtering tree, comprising the following steps:

[0010] Step 1: The mobile robot uses a sensor with a limited field of view, including a camera, a radar, etc., to obtain the surrounding environment information at the current location. The observation result of the target by the mobile robot through multiple sensors is defined as z. If the target is within the observation range of the mobile robot, the mobile robot obtains the observation information of the target, i.e. the distance and angle information of the target relative to the robot; otherwise, the observation result of the target by the mobile robot is empty. The mobile robot uses particle filtering technology to filter the observation result z to obtain the estimation result of the target position.

[0011] Step 2: Based on the current particle-based target position estimation result and the state of the mobile robot itself, the tree is built based on the Monte Carlo tree search method, and the four steps of selection-expansion-simulation-backtracking are iteratively performed to incrementally build the adaptive particle filtering tree, and the information of the nodes is constantly updated and maintained in the adaptive particle filtering tree for subsequent mobile robot decision-making. The nodes in the adaptive particle filtering tree are used to store the information of the mobile robot in the planning time domain, including the position that the mobile robot may reach in the planning time domain and the corresponding reward, and the adaptive particle filtering tree is used to explore the possible trajectory of the mobile robot in the future to provide the optimal motion decision scheme for the mobile robot. The adaptive particle filtering tree contains two types of nodes: action nodes and state nodes, which are alternately generated according to the action-observation sequence to simulate the actions taken by the planning mobile robot during the movement in the planning time domain and the observation results that may be obtained. The tree building process includes four steps of selection-expansion-simulation-backtracking:

[0012] Step 2.1, Selection: Based on the current policy tree information, the mobile robot selects the most valuable planning path in the current tree according to the UCB criterion, which balances exploration and exploitation. The UCB criterion is used to select the optimal child node n' of node n * The specific selection formula is as follows:

[0013]

[0014] where Q is the cumulative reward of the node, N(n) is the number of times the node n is visited, N(n') is the number of times the node n' is visited, C is the set of child nodes of node n, and c is the parameter that balances exploration and exploitation. Starting from the root node, the mobile robot continuously selects the optimal child node according to the criterion to find the current optimal planning path. Further, according to the selected series of action-state nodes, the action-observation sequence is used and the particle filtering technique is used to predict and update the particle estimation results.

[0015] Step 2.2, Expansion: After the mobile robot finds the end point of the selected planning path, the action-observation sequence is sampled from the action set of the state node corresponding to the end point and the environmental information, the reward brought by the action is calculated using the mutual information estimation method based on sigma points, and the particle filtering is used to update the particle estimation results, and the new action node and state node are expanded.

[0016] The calculation process of the mutual information calculation method based on sigma points is as follows:

[0017] In the calculation process of mutual information, the entropy H(z) of the observation result z of the mobile robot to the target is involved, that is, the calculation of

[0018]

[0019] When the observation noise is Gaussian distribution, the probability distribution P(z) of the observation result z of the mobile robot to the target satisfies a Gaussian mixture distribution where μ i and V i are the mean and variance of the i-th Gaussian component. For the i-th Gaussian distribution The sigma points and corresponding weights are defined as j = 0, …, 2m, where the dimension of the observation result is defined as m. The specific calculation formula is

[0020]

[0021] where λ is used to measure the dispersion degree of the sigma points. According to the obtained sigma points and weights, each Gaussian component is approximated, that is,

[0022]

[0023] Thus, the formula for approximating the entropy of the Gaussian mixture model is obtained

[0024]

[0025] The mutual information result is quickly calculated using the formula of the entropy of the Gaussian mixture model, which is used to measure the reward of the nodes in the planning path, laying the foundation for subsequent mobile robot decision-making.

[0026] Further, the state space is regularly divided into multiple small squares, and the weighted average of the particles in each square is obtained to obtain new particles. In this way, the number of particles required for calculation can be reduced, thereby improving the calculation efficiency of the estimation method and achieving efficient and accurate estimation of mutual information.

[0027] Step 2.3, simulation: for the newly expanded state node in the expansion process, an adaptive rollout process is performed, i.e. according to the pre-set action strategy, the mobile robot of the newly expanded node is simulated to move, the cumulative discounted reward of the process is calculated, until the termination state is reached or the pre-set time budget is reached, which is used to estimate the value of the new node. If the information gain obtained before reaching the termination state or the pre-set time budget is greater than the pre-set threshold, the rollout process is terminated in advance, which is used to adaptively adjust the planning horizon.

[0028] Step 2.4, backtracking: according to the reward information obtained by the adaptive rollout process in the simulation step, all visited action nodes are backtracked, and the node visit count and reward information are updated. The specific process is to take the newly expanded node as the current node n', find its parent node n, and update the parent node n, the formula is as follows:

[0029] N(n) = N(n) + 1

[0030]

[0031] where r is the immediate reward, V is the total discounted reward in the planning horizon, and γ is the discount factor. After updating the information of node n, its parent node is selected as the current node and the above steps are iteratively performed until the root node is reached. By repeatedly selecting, expanding, simulating and backtracking the four steps, an updated policy tree is iteratively established for the selection of the optimal strategy of the mobile robot.

[0032] Step 3: based on the established adaptive particle filter tree, the mobile robot selects the node with the maximum reward in the root node, i.e. the optimal decision for the current position of the mobile robot. Subsequently, the mobile robot executes the motion control corresponding to the node with the maximum reward, generating a motion trajectory.

[0033] Step 4: After the mobile robot finishes the motion trajectory and reaches the new position, steps 1-3 are repeated, that is, the mobile robot re-plans according to the newly obtained observation results until the target search and tracking task is completed. In the process of motion of the mobile robot, the target object in the environment is dynamic, so the mobile robot continuously acquires new environment and target information, estimates the target position, and plans a trajectory to realize real-time search and tracking of the dynamic target, ensuring the accuracy of target positioning.

[0034] Compared with the prior art, the adaptive particle filter tree technology proposed in the application is a planning method based on Monte Carlo tree search, which can balance decision exploration and utilization, and selectively search for feasible paths in the future. At the same time, its adaptive nature can dynamically adjust the planning time domain according to the current reward information, saving computing resources. Combined with the action space composed of motion primitives, the application can efficiently generate a long-time smooth path that considers the influence of future observations and conforms to the kinematic model, thereby efficiently completing the target search and tracking problem. BRIEF DESCRIPTION OF DRAWINGS

[0035] Figure 1 is an overview of the adaptive particle filter tree.

[0036] Figure 2 is a construction process diagram of the adaptive particle filter tree.

[0037] Figure 3 is a performance index comparison diagram of different methods in the target search and tracking task.

[0038] Figure 4 is a trajectory comparison diagram of different methods in the target search and tracking task. DETAILED DESCRIPTION

[0039] The application will be described in detail below in combination with the drawings and examples.

[0040] The technical scheme adopted by the method is: a robot dynamic target search and tracking method based on an adaptive particle filter tree. The method is a path planning method considering model uncertainty and limited observation field with information quantity, and is used to solve dynamic target search and tracking. As shown in Figure 1As shown, the method mainly includes two aspects of progress: first, the mutual information estimation method based on sigma points is proposed, which can quickly and accurately calculate the mutual information in the case of non-Gaussian state distribution in continuous state space and observation space, providing the target function basis for the real-time stable operation of the planning part. Second, based on the Monte Carlo tree search method, the adaptive particle filter tree technology is proposed, which can adaptively adjust the planning time domain according to the current known information, and can efficiently generate the robot motion path that meets the kinematic constraint, actively avoids obstacles and has information amount, so as to better complete the dynamic target search and tracking task.

[0041] The scheme includes the following steps:

[0042] Step 1: The mobile robot obtains the surrounding environment information at the current location by using a sensor with a limited field of view, such as a camera, a radar, etc. The observation result of the target by the mobile robot through various sensors is defined as z. If the target is within the observation range of the mobile robot at this time, the mobile robot will obtain the observation information of the target, that is, the distance and angle information of the target relative to the robot; otherwise, the observation result of the target by the mobile robot is empty. Under the influence of inevitable sensor noise, the mobile robot uses the particle filter technology to filter the inaccurate observation result z to obtain the estimation result of the target position.

[0043] Step 2: Based on the current particle-based target position estimation result and the state of the mobile robot itself, the tree is built based on the Monte Carlo tree search method, and the four steps of selection-expansion-simulation-backtracking are iteratively performed, and the adaptive particle filter tree is incrementally built. Nodes are added to the adaptive particle filter tree and the information of the updated nodes is maintained for subsequent mobile robot decision-making. The nodes in the adaptive particle filter tree are used to store the information of the mobile robot in the planning time domain, including the position that the mobile robot may reach in the planning time domain and the corresponding reward, so the adaptive particle filter tree is used to explore the possible trajectory of the mobile robot in the future, and provide the optimal motion decision scheme for the mobile robot. The adaptive particle filter tree includes two types of nodes: action nodes and state nodes, which are alternately generated according to the action-observation sequence to simulate the actions taken by the planning mobile robot and the observation results obtained during the movement in the planning time domain. As shown in the figure, the tree building process includes four steps of selection-expansion-simulation-backtracking: Figure 2

[0044] Step 2.1 (selection): Based on the current strategy tree information, the mobile robot selects the most valuable planning path in the current tree according to the UCB (upper confidence bound) criterion, that is, a series of action-state nodes. The UCB criterion is used to select the optimal child node n * ​, the formula is as follows:

[0045]

[0046] where Q is the cumulative reward of the node, N(n) is the number of times that the node n is visited, N(n') is the number of times that the node n' is visited, C is the set of the child nodes of the node n, and c is the parameter for balancing exploration and exploitation. Starting from the root node, the mobile robot constantly selects the optimal child node through the criterion to find the current optimal planning path. Further, according to the selected series of action-state nodes, the action-observation sequence and the particle filtering technology are used to predict and update the particle estimation results.

[0047] Step 2.2 (extension): After the mobile robot finds the end point of the selected planning path, the action-observation sequence is sampled from the action set of the end point corresponding state node and the environment information, the reward brought by the action is calculated using the mutual information estimation method based on sigma points, and the particle estimation results are updated using particle filtering to extend new action nodes and state nodes. The calculation process of the mutual information calculation method based on sigma points is as follows:

[0048] In the calculation process of mutual information, the calculation of the entropy H(z) of the observation result z of the target by the mobile robot is involved, that is, the calculation of

[0049] H(z) = -∫P(z)logP(z)dz

[0050] Under the assumption that the observation noise is Gaussian distribution, the probability distribution P(z) of the observation result z of the target by the mobile robot satisfies a Gaussian mixture distribution where μ i and V i are the mean and variance of the i-th Gaussian component, respectively. For the i-th Gaussian distribution The sigma points and corresponding weights are defined as j = 0, …, 2m, where the dimension m of the observation result is defined. The specific calculation formula is

[0051]

[0052] where λ is used to measure the dispersion degree of the sigma points. According to the obtained sigma points and weights, each Gaussian component is approximated, that is,

[0053]

[0054] Thus, the formula for approximating the entropy of the Gaussian mixture model is obtained

[0055]

[0056] Using the formula, the mutual information result can be calculated quickly and explicitly, which is used to measure the reward of the node in the planning path and lays the foundation for subsequent mobile robot decision-making.

[0057] Further, the state space is regularly divided into a plurality of small squares, and a weighted average of particles in each square is obtained to obtain new particles, which can reduce the number of particles required for calculation, thereby improving the calculation efficiency of the estimation method and realizing efficient and accurate estimation of mutual information. The sigma point mutual information calculation method proposed in the application uses a small number of fixed position samples to approximate the calculation of mutual information. By using the properties of sigma points to keep the first moment and second moment information unchanged, the estimation accuracy of the calculation result is ensured, and the small number of sampling quantities avoids high method calculation complexity, thereby improving the algorithm operation efficiency.

[0058] Step 2.3 (simulation): for the newly expanded state node in the expansion process, an adaptive rollout process is performed, that is, the mobile robot of the newly expanded node is simulated according to the pre-set action strategy, the cumulative discounted reward of the process is calculated, and the process is terminated until the termination state or the preset time budget is reached, which is used to estimate the value of the new node. If the information gain obtained by the process before reaching the termination state or the preset time budget is greater than the pre-set threshold, the rollout process is terminated in advance, which is used to adaptively adjust the planning time domain.

[0059] Step 2.4 (backtracking): according to the reward information obtained by the adaptive rollout process in the simulation step, the visited action nodes are backtracked, and the node visit times and reward information are updated. The specific process is to take the newly expanded node as the current node n', find its parent node n, and update the parent node n. The formula is as follows:

[0060] N(n)=N(n)+1

[0061]

[0062] where r is the immediate reward, V is the total discounted reward in the planning time domain, and gamma is the discount factor. After updating the information of node n, its parent node is selected as the current node and the above steps are iteratively performed until the root node is backtracked. By repeatedly selecting, expanding, simulating and backtracking the four steps, an updated strategy tree is iteratively established, which is used for selecting the optimal strategy of the mobile robot.

[0063] Step 3: based on the established adaptive particle filtering tree, the mobile robot selects the node with the maximum reward in the root node, that is, the optimal decision of the current position of the mobile robot. Subsequently, the mobile robot performs the motion control corresponding to the node with the maximum reward to generate a motion trajectory.

[0064] Step 4: After the mobile robot completes the motion trajectory and reaches the new position, steps 1-3 are repeated, that is, the mobile robot re-plans according to the newly obtained observation results until the target search and tracking task is completed. It is noted that the target object in the environment is also dynamic during the movement of the mobile robot. In this scheme, the mobile robot can continuously obtain new environment and target information, estimate the target position, and plan a trajectory to realize real-time search and tracking of the dynamic target, thereby ensuring the accuracy of target positioning.

[0065] The adaptive particle filtering tree proposed in the present application can adaptively adjust the planning time domain according to the current mode and information to adapt to different task stages, thereby ensuring the effect of the planned trajectory and speeding up the efficiency of the algorithm operation, achieving real-time and stable path planning to complete the dynamic target search and tracking task.

[0066] The present application will show the simulation verification results of mutual information estimation and target search and tracking. First, the results of the sigma point-based estimation method and the Monte Carlo integral, the Taylor expansion-based estimation method in mutual information estimation are compared, as shown in Table 1:

[0067] Table 1: Comparison of results of different mutual information estimation methods

[0068] Method Absolute error Relative error (%) Calculation time (s) Monte Carlo integration - - 0.0614 Zeroth order Taylor expansion estimate 0.4220 38.9 0.0070 Second order Taylor expansion estimate 0.0993 8.66 0.3502 Sigma point estimate 0.0395 3.42 0.0302 Sigma point estimate + particle simplification 0.0533 4.69 0.0017

[0069] The present application takes the result of the Monte Carlo integral as the true value, and calculates the absolute error, relative error and calculation time of the calculation results of each method. The sigma point-based estimation method proposed in the present application has the smallest calculation error, and after combining the particle simplification technology, the calculation speed is significantly improved on the basis of still maintaining high estimation accuracy, and the accuracy and efficiency are verified. In addition, the present application method and the greedy method (NBV), the sampling-based method (IIG) are compared in the dynamic target search and tracking task, the comparison indexes include the time difference of finding the target, the target loss probability during tracking and the target estimation error during tracking, and the single-peak and multi-peak initial target distribution is considered, and the quantitative results are shown in Table 2. Figure 3 It can be seen that, whether in the single-peak or multi-peak initial target distribution, the present application has shorter search time, lower target loss probability and estimation error, which shows that the method has higher search efficiency and more stable tracking performance, and can provide more efficient and more reliable performance for the dynamic target search and tracking task. The trajectory comparison results are shown in Figure 2. Figure 4 It can be seen that the present application can find the target faster with a smooth trajectory, and stably track the target under uncertain conditions, which embodies the efficiency, robustness and real-time performance of the present application.

[0070] References

[0071] [1] Charrow B, Kumar V, Michael N. Approximate representations for multi-robot control policies that maximize mutual information [J]. Autonomous Robots, 2014, 37: 383-400.

[0072] [2] Ghaffari Jadidi M, Valls Miro J, Dissanayake G. Sampling-based incremental information gathering with applications to robotic exploration and environmental monitoring [J]. The International Journal of Robotics Research, 2019, 38(6): 658-685.

[0073] [3] Hoffmann G M, Tomlin C J. Mobile sensor network control using mutual information methods and particle filters [J]. IEEE Transactions on Automatic Control, 2009, 55(1): 32-47.

Claims

1. A robot dynamic target search and tracking method based on adaptive particle filter tree, characterized in that, Comprising the following steps: Step 1: The mobile robot uses a sensor with a limited field of view, including a camera, radar, to obtain the surrounding environment information at the current location; define the observation result of the target by the mobile robot through multiple sensors as z; if the target is within the observation range of the mobile robot, the mobile robot obtains the observation information of the target, that is, the distance and angle information of the target relative to the robot; otherwise, the observation result of the target by the mobile robot is empty; the mobile robot uses particle filtering technology to filter the observation result z to obtain the estimation result of the target position; Step 2: Based on the current particle-based target position estimation result and the state of the mobile robot itself, a tree is built based on the Monte Carlo tree search method, and the four steps of selection-expansion-simulation-backtracking are iteratively performed to incrementally build an adaptive particle filtering tree, and nodes are continuously added to the adaptive particle filtering tree and the information of the updated nodes is maintained for subsequent mobile robot decision-making; the nodes in the adaptive particle filtering tree are used to store the information of the mobile robot in the planning time domain, including the positions that the mobile robot may reach in the planning time domain and the corresponding rewards, and the adaptive particle filtering tree is used to explore the possible trajectories of the mobile robot in the future and provide the optimal motion decision scheme for the mobile robot; the adaptive particle filtering tree contains two types of nodes: action nodes and state nodes, which are alternately generated according to the action-observation sequence to simulate the actions taken by the planning mobile robot and the observation results obtained during the movement in the planning time domain; Step 3: Based on the established adaptive particle filtering tree, the mobile robot selects the child node with the maximum reward in the root node, that is, the optimal decision for the current location of the mobile robot; then, the mobile robot executes the motion control corresponding to the child node with the maximum reward to generate a motion trajectory; Step 4: After the mobile robot executes the motion trajectory and reaches a new position, steps 1-3 are repeated, that is, the mobile robot re-plans according to the newly obtained observation results until the target search and tracking task is completed; during the movement of the mobile robot, the target object in the environment is dynamic, so the mobile robot continuously obtains new environment and target information, estimates the target position, and plans a trajectory to realize real-time search and tracking of the dynamic target, ensuring the accuracy of target positioning; The tree building process includes four steps of selection-expansion-simulation-backtracking: Step 2.1, Selection: Based on the current policy tree information, the mobile robot selects the most valuable planning path in the current tree, i.e., a series of action-state nodes, according to the UCB criterion, which balances action exploration and exploitation; the UCB criterion is used to select the optimal child node n in node n * The specific selection formula is as follows: Where Q is the cumulative reward of the node, N(n) is the number of times that node n has been visited, N(n') is the number of times that node n' has been visited, C is a set of child nodes of node n, and c is a parameter for balancing exploration and utilization; starting from the root node, the mobile robot continuously selects the optimal child node through the criterion to find the current optimal planning path; based on the selected series of action-state nodes, the action-observation sequence and particle filtering technology are used to predict and update the particle estimation result; Step 2.2, expansion: after the mobile robot finds the end point of the selected planning path, the mobile robot samples the action-observation sequence from the action set and environment information of the state node corresponding to the end point, calculates the reward brought by the action-observation sequence using the mutual information estimation method based on sigma points, and updates the particle estimation result using particle filtering to expand a new action node and a state node; Step 2.3, simulation: for the state node newly expanded in the expansion process, an adaptive rollout process is performed, that is, the mobile robot of the newly expanded node is simulated according to the pre-set action strategy, and the cumulative discounted reward of the process is calculated until the termination state is reached or the preset time budget is reached, which is used to estimate the value of the new node; If the information gain obtained before reaching the termination state or the preset time budget is greater than the pre-set threshold, the rollout process is terminated in advance, which is used to adaptively adjust the planning horizon; Step 2.4, backtracking: according to the reward information obtained in the simulation step, the visited action nodes are backtracked, and the node visit times and reward information are updated; the specific process is to take the newly expanded node as the current node n', find the parent node n of the current node n', and update the parent node n, the formula is as follows: N(n) = N(n) + 1 where r is the immediate reward, V is the total discounted reward in the planning horizon, and γ is the discount factor; after updating the information of the node n, the parent node is selected as the current node and the above steps are iteratively performed until the root node is reached; by repeatedly selecting, expanding, simulating and backtracking, an updated strategy tree is iteratively established for the selection of the optimal strategy of the mobile robot.

2. The adaptive particle filter tree-based robot dynamic target search and tracking method according to claim 1, wherein, The calculation process of the mutual information calculation method based on sigma points is as follows: In the calculation process of mutual information, the entropy H(z) of the observation result z of the mobile robot on the target is involved, that is, the calculation of H(z) = -∫P(z)log P(z)dz In the case of Gaussian observation noise, the probability distribution P(z) of the observation z of the target by the mobile robot satisfies a Gaussian mixture distribution where μ i and V i are the mean and variance of the i-th Gaussian component, respectively; for the i-th Gaussian distribution The sigma points and the corresponding weights are defined as where m is defined as the dimension of the observation. The specific calculation formula is Where λ is used to measure the dispersion degree of the sigma point; according to the obtained sigma point and weight, each Gaussian component is approximated, that is, Thus, the formula for approximating the entropy of the Gaussian mixture model is obtained The mutual information result is quickly calculated using the formula for the entropy of the Gaussian mixture model, which is used to measure the reward of the node in the planning path and lay the foundation for the subsequent decision of the mobile robot.

3. The adaptive particle filter tree-based robot dynamic target search and tracking method according to claim 1, wherein, The state space is regularly divided into a plurality of small squares, and the particles in each square are weighted and averaged to obtain new particles, which reduces the number of particles required for calculation, improves the calculation efficiency of the estimation method, and realizes efficient and accurate estimation of mutual information.

Images