An online monte carlo tree search method with robust properties

Abstract

The application relates to the field of autonomous motion machine intelligence search, and provides an online Monte Carlo tree search method with robust characteristics. A belief state of a target robot at a current moment is acquired and uncertainty quantification is carried out, the uncertainty quantification acquires a dynamic uncertainty radius based on a truncated mapping function; the target robot enters a planning stage, simulated sampling is carried out according to a Monte Carlo tree search framework to acquire a reward sample, the Monte Carlo tree search framework is based on a reward function and risk perception; then, robust value evaluation is carried out according to the reward sample, the robust value evaluation is based on a strong duality theorem of a total variation distance; the planning stage is ended and a decision stage is entered, current period action output is acquired through screening, after an action is executed according to the current period action output, the belief state is updated and the next period is started. The application improves the decision robustness, calculation timeliness and action sequence geometric safety of the autonomous motion robot.

Description

Technical Field

[0001] This invention relates to the field of intelligent search for autonomous motion machines, and in particular to an online Monte Carlo tree search method with robust properties. Background Technology

[0002] In extreme environments such as nuclear power plant decommissioning, radioactive waste disposal, and deep-sea exploration, the autonomous planning of mobile robots faces stringent challenges. These environments are characterized by "perception-geometric dual constraints": on the one hand, strong ionizing radiation can interfere with CMOS image sensors or LiDAR electronic components, generating a high proportion of impulse noise (snow noise in images), causing severe non-Gaussian shifts in positioning signals; on the other hand, maintenance channels often contain narrow gates or dense obstacles, requiring robots to have centimeter-level motion accuracy.

[0003] Traditional robust planning algorithms (such as Robust MPC) often require solving complex quadratic or semidefinite programming problems to ensure safety, which exceeds the 100ms real-time response limit of embedded systems. In the field of autonomous planning, active estimation and planning algorithms based on POMCP (Partially Observable Monte Carlo Programming) are typically represented by the MCTS framework and its variants. However, existing MCTS lacks mathematical mechanisms to cope with extreme cognitive uncertainty. Furthermore, under the influence of non-stationary, non-Gaussian observation noise and stringent real-time computing resource constraints, traditional online planning algorithms, due to their over-reliance on mean expectation evaluation, easily overlook low-probability catastrophic events when dealing with heavy-tailed distributions, leading to a lack of decision robustness. Simultaneously, existing sub-Brutal optimization schemes suffer from excessively high computational complexity due to high dimensions, making it difficult to meet the "always-on" response requirements within a 100ms control cycle for robots.

[0004] Therefore, designing a self-planning search method that can balance decision robustness, computational efficiency, and geometric safety of action sequences under extreme cognitive uncertainty has become an urgent problem to be solved. Summary of the Invention

[0005] Based on this, the present invention proposes a robust online Monte Carlo tree search method. This method obtains the current belief state of the target robot and quantifies its uncertainty, using a truncated mapping function to obtain the dynamic uncertainty radius. The target robot then enters a planning phase, performing simulated sampling based on a Monte Carlo tree search framework to obtain reward samples. This framework is based on a reward function and risk perception. Robust value assessment is then performed based on these reward samples, using the strong duality theorem of total variational distance. The planning phase ends, and the robot enters a decision-making phase, where the current cycle's action output is selected and executed. After executing the action based on the current cycle's action output, the belief state is updated, and the next cycle begins. This invention improves the decision robustness, computational efficiency, and geometrical safety of action sequences for autonomous robots.

[0006] This invention proposes a robust online Monte Carlo tree search method, comprising: The current belief state of the target robot is obtained and its uncertainty is quantified. The belief state is defined as the posterior probability distribution of the state after all historical observations and actions. The uncertainty quantification is based on a truncated mapping function to obtain the dynamic uncertainty radius, which is used for robust optimization. The target robot enters the planning phase and performs simulated sampling based on the Monte Carlo tree search framework to obtain reward samples. The Monte Carlo tree search framework is based on the reward function and risk perception. Then, a robust value assessment is performed based on the returned samples, which is based on the strong duality theorem of total variational distance; The planning phase ends and the decision-making phase begins. The current cycle action output is selected and obtained. After the action is executed according to the current cycle action output, the belief state is updated and the next cycle begins.

[0007] In summary, based on the robust online Monte Carlo tree search method described above, the current belief state of the target robot is obtained and its uncertainty is quantified. The belief state is defined as the posterior probability distribution of the state given all historical observations and actions. The uncertainty quantification is based on a truncated mapping function to obtain the dynamic uncertainty radius, which is used for robust optimization. The target robot enters the planning phase, performing simulated sampling according to the Monte Carlo tree search framework to obtain reward samples. This framework is based on a reward function and risk perception, ensuring that the search tree grows only within a probabilistically safe region, significantly improving search efficiency and safety, and achieving rapid pruning for front-end safety. Robust value evaluation is then performed based on the reward samples, using the strong duality theorem of total variational distance. This rapid solution satisfies the stringent real-time requirements of highly dynamic tasks, achieving fast and accurate robust evaluation at the back end. The planning phase ends, and the decision phase begins. The current cycle action output is selected and executed according to the current cycle action output. The belief state is then updated, and the next cycle begins. This forms a closed-loop rolling optimization of "planning-execution-update". This framework enables adaptive robust behavior to emerge without explicit programming: as the robot approaches an obstacle, the reduced distance makes safe pruning easier to trigger, eliminating aggressive actions; simultaneously, increased uncertainty leads to a more conservative robustness assessment, and the system automatically outputs instructions for deceleration and smoothing the trajectory, thereby significantly improving survivability in environments with degraded perception. This invention improves the decision robustness, computational efficiency, and geometric safety of action sequences in autonomous motion robots.

[0008] Furthermore, the step of obtaining the current belief state of the target robot and quantifying its uncertainty specifically includes: The belief state of the target robot at the current moment is obtained, and the specific algorithm for obtaining the belief state is as follows: , in, Representing state The state of belief at that time Represents a probability distribution. This represents the state estimate of the nominal model. Indicates the state of the target robot. Indicates historical observation, Indicates the action taken at the previous moment; The belief state is updated online based on a recursive Bayesian filter, and the online update includes a prediction step and an update step. The prediction step specifically involves obtaining the prior distribution based on the belief state and the action performed in the previous time step using a nominal model. The specific algorithm for the nominal model is as follows: , in, This represents the state estimate of the nominal model. The function represents the state-action relationship. This indicates the state at the previous moment. Indicates the action taken at the previous moment; The update step specifically involves, after obtaining the noisy current observation, correcting the prior based on the observation model to obtain the posterior belief. The specific algorithm for the observation model is as follows: , in, This represents the observation results of the observation model. This represents an ideal observation model. This represents non-Gaussian distributed perturbation noise; Obtain the covariance matrix of the state estimation mean and the state estimation error. Then, transform the trace of the covariance matrix using a truncation-based mapping function to obtain the dynamic uncertainty radius. The specific algorithm for obtaining the dynamic uncertainty radius is as follows: , in, Represents the radius of dynamic uncertainty. Indicates truncation. Represents a single dual variable. The trace of the covariance matrix. and These represent the minimum radius and the maximum radius, respectively.

[0009] Furthermore, the target robot enters the planning phase, which involves simulated sampling based on the Monte Carlo tree search framework to obtain return samples. This specifically includes: The reward for the state-action pair is calculated, based on navigation and safety rewards, and the specific algorithm for the reward is as follows: , , in, Indicates a reward. Indicates the state of the target robot. Indicates the action of the target robot. and These represent the task efficiency weight and the task safety weight, respectively. and These represent navigation rewards and safety rewards, respectively. The target location status is indicated. The navigation reward is based on the relative distance between the current location and the target location. The safety reward is based on the simulated collision results. When a collision with an obstacle is detected in the simulation, the safety reward is set to a minimum value, which is negative. When no collision with an obstacle is detected in the simulation, the safety reward is set to 0. The target robot proceeds to the next node expansion, which is based on the upper bound of the collision probability calculated using Chebyshev's inequality. The specific algorithm for the upper bound of the collision probability is as follows: , in, This represents the upper bound of the collision probability. The trace of the covariance matrix representing the error in the state estimation at the next time step. Represents Euclidean distance. This represents the mean of the predicted state at the next moment. If the upper bound of the collision probability exceeds the preset safety risk threshold, the current action is determined to be a high-risk branch, the expansion of the target node is immediately abandoned, and the robustness value of the target node is marked as negative infinity.

[0010] Furthermore, the step of conducting a robust value assessment based on the return sample specifically includes: An empirical return distribution is formed based on the return samples, and a robust value assessment is performed. The specific algorithm for the robust value assessment is as follows: , , in, Indicates robust value, Indicates the distribution of beliefs. Indicates the action of the target robot. Represents a single dual variable. Represents the total number of samples. Indicates the sample ordinal number. Indicates the return sample, Represents the radius of dynamic uncertainty. This represents the sample mean.

[0011] Furthermore, the steps of ending the planning phase and entering the decision-making phase, filtering and obtaining the current cycle action output, executing the action according to the current cycle action output, updating the belief state, and starting the next cycle specifically include: The specific algorithm for filtering the output of the current cycle action is as follows: , in, Indicates the output of the current cycle action. This represents all available sub-actions of the current cycle root node. It represents the robustness value of the current cycle action.

[0012] This invention proposes a robust online Monte Carlo tree search system, comprising: The update module is used to obtain the belief state of the target robot at the current moment and perform uncertainty quantification. The belief state is defined as the posterior probability distribution of the state after all historical observations and actions. The uncertainty quantification is based on a truncated mapping function to obtain the dynamic uncertainty radius, which is used for robust optimization. The simulation module is used to simulate sampling based on the Monte Carlo tree search framework to obtain reward samples when the target robot enters the planning stage. The Monte Carlo tree search framework is based on the reward function and risk perception. An evaluation module is used to perform a robust value assessment based on the return samples, the robust value assessment being based on the strong duality theorem of total variational distance; The decision-making module is used to end the planning phase and enter the decision-making phase. It filters and obtains the current cycle action output, executes the action according to the current cycle action output, updates the belief state, and starts the next cycle.

[0013] The present invention also provides a storage medium that stores one or more programs, which, when executed by a processor, implement the robust online Monte Carlo tree search method described above.

[0014] The present invention also provides a computer device, the computer device including a memory and a processor, wherein: The memory is used to store computer programs; When the processor executes the computer program stored in the memory, it implements the robust online Monte Carlo tree search method described above. Attached Figure Description

[0015] Figure 1 The flowchart shows the robust online Monte Carlo tree search method proposed in the first embodiment of the present invention. Figure 2 This is a schematic diagram of the robust online Monte Carlo tree search system proposed in the second embodiment of the present invention. Figure 3 This is a logical framework diagram of the robust online Monte Carlo tree search method proposed in the first embodiment of the present invention; Figure 4a The table shows the comparison of the success rate and reward value between the robust online Monte Carlo Tree Search (MCTS) method proposed in the first embodiment of this invention and the normal non-robust method MCTS in the noise range of 0.06-0.36. Figure 4b The graph shows a comparison of the success rates of the robust online Monte Carlo Tree Search (MCTS) method proposed in the first embodiment of this invention with those of the normal, non-robust method, within the noise range of 0.06-0.36. Figure 4c The graph shows a comparison of the MCTS reward values ​​between the robust online Monte Carlo Tree Search method proposed in the first embodiment of this invention and the normal non-robust method in the noise range of 0.06-0.36. Figure 4d The figure shows the simulation results of 50 runs of the robust online Monte Carlo tree search method proposed in the first embodiment of the present invention under 36% noise. Figure 4e The figure shows the results of 50 simulation tests of the normal non-robust method MCTS under 36% noise. The following detailed description, in conjunction with the accompanying drawings, will further illustrate the present invention. Detailed Implementation

[0016] To facilitate understanding of the present invention, a more complete description will be given below with reference to the accompanying drawings. Several embodiments of the invention are illustrated in the drawings. However, the invention can be implemented in many different forms and is not limited to the embodiments described herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete.

[0017] It should be noted that when a component is said to be "fixed to" another component, it can be directly on the other component or there may be an intervening component. When a component is said to be "connected to" another component, it can be directly connected to the other component or there may be an intervening component. The terms "vertical," "horizontal," "left," "right," and similar expressions used in this document are for illustrative purposes only.

[0018] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains. The terminology used herein in the description of the invention is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention. The term "and / or" as used herein includes any and all combinations of one or more of the associated listed items.

[0019] Please see Figure 1 The diagram shows a flowchart of the robust online Monte Carlo tree search method proposed in the first embodiment of the present invention. Including steps S01 to S04, wherein: Step S01: Obtain the current belief state of the target robot and quantify its uncertainty; It should be noted that in this embodiment, the belief state is defined as the posterior probability distribution of the state given all historical observations and actions. The uncertainty quantization is based on a truncated mapping function to obtain the dynamic uncertainty radius, which is used for robust optimization to obtain the belief state of the target robot at the current moment. The specific algorithm for the belief state is as follows: , in, Representing state The state of belief at that time Represents a probability distribution. This represents the state estimate of the nominal model. Indicates the state of the target robot. Indicates historical observation, Indicates the action taken at the previous moment; The belief state is updated online based on a recursive Bayesian filter, and the online update includes a prediction step and an update step. The prediction step specifically involves obtaining the prior distribution based on the belief state and the action performed in the previous time step using a nominal model. The specific algorithm for the nominal model is as follows: , in, This represents the state estimate of the nominal model. The function represents the state-action relationship. This indicates the state at the previous moment. Indicates the action taken at the previous moment; The update step specifically involves, after obtaining the noisy current observation, correcting the prior based on the observation model to obtain the posterior belief. The specific algorithm for the observation model is as follows: , in, This represents the observation results of the observation model. This represents an ideal observation model. This represents non-Gaussian distributed perturbation noise; Obtain the covariance matrix of the state estimation mean and the state estimation error. Then, transform the trace of the covariance matrix using a truncation-based mapping function to obtain the dynamic uncertainty radius. The specific algorithm for obtaining the dynamic uncertainty radius is as follows: , in, Represents the radius of dynamic uncertainty. Indicates truncation. Represents a single dual variable. The trace of the covariance matrix. and These represent the minimum radius and the maximum radius, respectively.

[0020] Step S02: The target robot enters the planning phase and performs simulated sampling based on the Monte Carlo tree search framework to obtain return samples; It should be noted that in this embodiment, the Monte Carlo tree search framework calculates the reward for state-action pairs based on a reward function and risk perception. The reward is based on navigation reward and safety reward, and the specific algorithm for the reward is as follows: , , in, Indicates a reward. Indicates the state of the target robot. Indicates the action of the target robot. and These represent the task efficiency weight and the task safety weight, respectively. and These represent navigation rewards and safety rewards, respectively. The target location status is indicated. The navigation reward is based on the relative distance between the current location and the target location. The safety reward is based on the simulated collision results. When a collision with an obstacle is detected in the simulation, the safety reward is set to a minimum value, which is negative. When no collision with an obstacle is detected in the simulation, the safety reward is set to 0. The target robot proceeds to the next node expansion, which is based on the upper bound of the collision probability calculated using Chebyshev's inequality. The specific algorithm for the upper bound of the collision probability is as follows: , in, This represents the upper bound of the collision probability. The trace of the covariance matrix representing the error in the state estimation at the next time step. Represents Euclidean distance. This represents the mean of the predicted state at the next moment. If the upper bound of the collision probability exceeds the preset safety risk threshold, the current action is determined to be a high-risk branch, the expansion of the target node is immediately abandoned, and the robustness value of the target node is marked as negative infinity.

[0021] Step S03: Perform a robust value assessment based on the return sample; It should be noted that in this embodiment, the robust value assessment is based on the strong duality theorem of total variational distance, forming an empirical reward distribution based on the reward samples, and then performing robust value assessment. The specific algorithm for robust value assessment is as follows: , , in, Indicates robust value, Indicates the distribution of beliefs. Indicates the action of the target robot. Represents a single dual variable. Represents the total number of samples. Indicates the sample ordinal number. Indicates the return sample, Represents the radius of dynamic uncertainty. This represents the sample mean.

[0022] Step S04: End the planning phase and enter the decision-making phase. Filter and obtain the action output of the current cycle. After executing the action according to the action output of the current cycle, update the belief state and start the next cycle. It should be noted that, in this embodiment, the steps of ending the planning phase and entering the decision-making phase, filtering and obtaining the current cycle action output, executing the action according to the current cycle action output, updating the belief state, and starting the next cycle specifically include: The specific algorithm for filtering the output of the current cycle action is as follows: , in, Indicates the output of the current cycle action. This represents all available sub-actions of the current cycle root node. It represents the robustness value of the current cycle action.

[0023] For the simulation, please refer to the specific logic framework. Figure 3For an autonomous motion chassis system, at each discrete moment, the system first uses the belief state of the previous moment as the current cognitive state. The belief state is not a definite location point, but a joint probabilistic description of the uncertainty of the vehicle's true state and the observation mode. Subsequently, based on the currently executed control input, the algorithm uses a nominal kinematic model to make a priori prediction of the position the vehicle may reach in the next moment. This step reflects 'where it is likely to go if the system evolves according to nominal dynamics'. After obtaining the prediction result, the system receives the current sensor observation. However, since the observation may contain ordinary Gaussian noise, degradation noise, or even anomalous jumps, the algorithm does not directly treat the observation value as the true position. Instead, it uses an edge-filter to calculate the probability of the observation under the three observation modes of normal, degradation, and anomalous, and combines the mode transition prior to perform a Bayesian update on the probabilities of the three modes, thereby obtaining the posterior belief state of the current position. Next, the planner uses this posterior belief state as the basis for decision-making. Within a limited time budget, it enumerates a set of candidate actions. First, it uses probabilistic safety constraints to eliminate actions that might lead to excessive collision risks. Then, it calculates the immediate value of the remaining actions, which consists of task progress rewards and information gains. To further consider the long-term effects of actions over several steps, the algorithm performs short-term look-ahead simulations of candidate actions based on Monte Carlo tree search to estimate their future cumulative returns. Simultaneously, considering the potential mismatch between the actual observation distribution and the nominal model used internally by the algorithm, the system introduces an uncertainty radius to characterize this distribution bias and conservatively corrects the obtained nominal value through robust dual optimization, thus preventing the planner from being overly optimistic about future returns. Finally, the algorithm selects the action with the highest robust value from all candidate actions and applies it to the environment. The car moves one step accordingly, the environment returns to a new real state and noisy observations, and the system updates the belief state based on the new information, entering the next round of the closed-loop online planning process of 'prediction, update, evaluation, and decision'. It is through this cyclical mechanism of continuously performing state estimation, risk assessment, robust decision-making, and closed-loop correction within the belief state space that the algorithm can gradually guide the vehicle to stably reach the target position under obstacle avoidance constraints, even under conditions of partial observability, contaminated observations, and limited real-time performance. Please refer to the simulation results. Figure 4a , Figure 4b , Figure 4c , Figure 4d , Figure 4e The simulation results show that the success rate of the car reaching the target position by passing through the noise region (applied from 0.06 to 0.36 with a noise step size of 0.03) under 50 simulation environments is higher. The simulation results show that the proposed algorithm is more robust and less affected by the uncertainty caused by noise compared with the traditional MCTS method.

[0024] Please see Figure 2The diagram shows a robust online Monte Carlo tree search system proposed in the second embodiment of the present invention. The system includes: Update module 10 is used to obtain the belief state of the target robot at the current moment and perform uncertainty quantification. The belief state is defined as the posterior probability distribution of the state after all historical observations and actions. The uncertainty quantification is based on a truncated mapping function to obtain the dynamic uncertainty radius, which is used for robust optimization. Simulation module 20 is used for the target robot to enter the planning stage and perform simulated sampling according to the Monte Carlo tree search framework to obtain reward samples. The Monte Carlo tree search framework is based on the reward function and risk perception. Evaluation module 30 is used to perform robust value evaluation based on the return sample, the robust value evaluation being based on the strong duality theorem of total variation distance; The decision module 40 is used to end the planning phase and enter the decision phase, filter and obtain the current cycle action output, execute the action according to the current cycle action output, update the belief state and start the next cycle.

[0025] The present invention also proposes a computer storage medium storing one or more programs that, when executed by a processor, implement the robust online Monte Carlo tree search method described above.

[0026] The present invention also proposes a computer device, including a memory and a processor, wherein the memory is used to store a computer program, and the processor is used to execute the computer program stored in the memory to implement the above-described robust online Monte Carlo tree search method.

[0027] Those skilled in the art will understand that the logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as a sequenced list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus, or device (such as a computer-based system, a processor-included system, or other system that can fetch and execute instructions from, an instruction execution system, apparatus, or device). For the purposes of this specification, "computer-readable medium" can mean any means that can contain stored, communicated, propagated, or transmitted programs for use by, or in conjunction with, an instruction execution system, apparatus, or device.

[0028] More specific examples of computer-readable media (a non-exhaustive list) include: electrical connections (electronic devices) having one or more wires, portable computer disk drives (magnetic devices), random access memory (RAM), read-only memory (ROM), erasable and editable read-only memory (EPROM or flash memory), fiber optic devices, and portable optical disc read-only memory (CDROM). Furthermore, computer-readable media can even be paper or other suitable media on which the program can be printed, because the program can be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, interpreting, or otherwise processing as necessary, and then stored in computer memory.

[0029] It should be understood that various parts of the present invention can be implemented in hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented in software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, it can be implemented using any one or a combination of the following techniques known in the art: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.

[0030] In the description of this specification, references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0031] The embodiments described above are merely illustrative of several implementations of the present invention, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of the present invention. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the present invention, and these modifications and improvements all fall within the scope of protection of the present invention. Therefore, the scope of protection of this patent should be determined by the appended claims.

Claims

1. A robust online Monte Carlo tree search method, characterized in that, include: The current belief state of the target robot is obtained and its uncertainty is quantified. The belief state is defined as the posterior probability distribution of the state after all historical observations and actions. The uncertainty quantification is based on a truncated mapping function to obtain the dynamic uncertainty radius, which is used for robust optimization. Once the target robot enters the planning phase, it performs simulated sampling based on the Monte Carlo tree search framework to obtain reward samples. The Monte Carlo tree search framework is based on the reward function and risk perception. Then, a robust value assessment is performed based on the returned samples, which is based on the strong duality theorem of total variational distance; The step of conducting a robust value assessment based on the return sample specifically includes: An empirical return distribution is formed based on the return samples, and a robust value assessment is performed. The specific algorithm for the robust value assessment is as follows: ， ， in, Indicates robust value, Indicates the distribution of beliefs. Indicates the action of the target robot. Represents a single dual variable. Represents the total number of samples. Indicates the sample ordinal number. Indicates the return sample, Represents the radius of dynamic uncertainty. This represents the sample mean; The planning phase ends and the decision-making phase begins. The current cycle action output is selected and obtained. After the action is executed according to the current cycle action output, the belief state is updated and the next cycle begins.

2. The robust online Monte Carlo tree search method according to claim 1, characterized in that, The step of obtaining the current belief state of the target robot and quantifying its uncertainty specifically includes: The belief state of the target robot at the current moment is obtained, and the specific algorithm for obtaining the belief state is as follows: ， in, Representing state The state of belief at that time Represents a probability distribution. This represents the state estimate of the nominal model. Indicates the state of the target robot. Indicates historical observation, Indicates the action taken at the previous moment; The belief state is updated online based on a recursive Bayesian filter, and the online update includes a prediction step and an update step. The prediction step specifically involves obtaining the prior distribution based on the belief state and the action performed in the previous time step using a nominal model. The specific algorithm for the nominal model is as follows: ， in, This represents the state estimate of the nominal model. The function represents the state-action relationship. This indicates the state at the previous moment. Indicates the action taken at the previous moment; The update step specifically involves, after obtaining the noisy current observation, correcting the prior based on the observation model to obtain the posterior belief. The specific algorithm for the observation model is as follows: ， in, This represents the observation results of the observation model. This represents an ideal observation model. This represents non-Gaussian distributed perturbation noise; Obtain the covariance matrix of the state estimation mean and the state estimation error. Then, transform the trace of the covariance matrix using a truncation-based mapping function to obtain the dynamic uncertainty radius. The specific algorithm for obtaining the dynamic uncertainty radius is as follows: ， in, Represents the radius of dynamic uncertainty. Indicates truncation. Represents a single dual variable. The trace of the covariance matrix. and These represent the minimum radius and the maximum radius, respectively.

3. The robust online Monte Carlo tree search method according to claim 1, characterized in that, The step of the target robot entering the planning phase, which involves simulated sampling based on the Monte Carlo tree search framework to obtain return samples, specifically includes: The reward for the state-action pair is calculated, based on navigation and safety rewards, and the specific algorithm for the reward is as follows: ， ， in, Indicates a reward. Indicates the state of the target robot. Indicates the action of the target robot. and These represent the task efficiency weight and the task safety weight, respectively. and These represent navigation rewards and safety rewards, respectively. The target location status is indicated. The navigation reward is based on the relative distance between the current location and the target location. The safety reward is based on the simulated collision results. When a collision with an obstacle is detected in the simulation, the safety reward is set to a minimum value, which is negative. When no collision with an obstacle is detected in the simulation, the safety reward is set to 0. The target robot proceeds to the next node expansion, which is based on the upper bound of the collision probability calculated using Chebyshev's inequality. The specific algorithm for the upper bound of the collision probability is as follows: ， in, This represents the upper bound of the collision probability. The trace of the covariance matrix representing the error in the state estimation at the next time step. Represents Euclidean distance. This represents the mean of the predicted state at the next moment. If the upper bound of the collision probability exceeds the preset safety risk threshold, the current action is determined to be a high-risk branch, the expansion of the target node is immediately abandoned, and the robustness value of the target node is marked as negative infinity.

4. The robust online Monte Carlo tree search method according to claim 1, characterized in that, The steps of ending the planning phase and entering the decision-making phase, filtering and obtaining the current cycle action output, executing the action according to the current cycle action output, updating the belief state, and starting the next cycle specifically include: The specific algorithm for filtering the output of the current cycle action is as follows: ， in, Indicates the output of the current cycle action. This represents all available sub-actions of the current cycle root node. It represents the robustness value of the current cycle action.

5. A robust online Monte Carlo tree search system, characterized in that, include: The update module is used to obtain the belief state of the target robot at the current moment and perform uncertainty quantification. The belief state is defined as the posterior probability distribution of the state after all historical observations and actions. The uncertainty quantification is based on a truncated mapping function to obtain the dynamic uncertainty radius, which is used for robust optimization. The simulation module is used to simulate sampling based on the Monte Carlo tree search framework to obtain reward samples when the target robot enters the planning stage. The Monte Carlo tree search framework is based on the reward function and risk perception. An evaluation module is used to perform a robust value assessment based on the return samples, the robust value assessment being based on the strong duality theorem of total variational distance; The step of conducting a robust value assessment based on the return sample specifically includes: An empirical return distribution is formed based on the return samples, and a robust value assessment is performed. The specific algorithm for the robust value assessment is as follows: ， ， in, Indicates robust value, Indicates the distribution of beliefs. Indicates the action of the target robot. Represents a single dual variable. Represents the total number of samples. Indicates the sample ordinal number. Indicates the return sample, Represents the radius of dynamic uncertainty. This represents the sample mean; The decision-making module is used to end the planning phase and enter the decision-making phase. It filters and obtains the current cycle action output, executes the action according to the current cycle action output, updates the belief state, and starts the next cycle.

6. A storage medium, characterized in that, The storage medium stores one or more programs that, when executed by a processor, implement the robust online Monte Carlo tree search method as described in any one of claims 1-4.

7. A computer device, characterized in that, The computer device includes a memory and a processor, wherein: The memory is used to store computer programs; When the processor executes the computer program stored in the memory, it implements the robust online Monte Carlo tree search method according to any one of claims 1-4.

Images