A human-machine collaborative confrontation control optimization system for a complex dynamic environment

Abstract

The present application relates to the technical field of mobile robot autonomous navigation and man-machine shared control, in particular to a man-machine cooperative confrontation control optimization system for complex dynamic environment, the system comprises an environment perception module, an autonomous decision module with MC-Dropout, a man-machine confrontation fusion module based on CVaR, a tubular robust MPC safety layer module and an execution module, the present application quantifies the tail risk of human instruction through a distributed value evaluation network and conditional value at risk, constructs a confrontation divergence to identify dangerous intention, realizes adaptive smooth deprivation of control right through cost optimization, at the same time, constructs a forward reachable tube of obstacles through Minkowski sum, converts the minimax safety problem into geometric boundary detection, realizes physical safety hard constraint in the confrontation environment, solves the double problems of malicious interception of external dynamic obstacles and confrontation conflict of internal man-machine intention, and significantly improves the safety and task efficiency of the robot in complex unstructured environment.

Description

Technical Field

[0001] This invention relates to the field of autonomous navigation and human-machine shared control technology for mobile robots, specifically a human-machine collaborative adversarial control optimization system for complex dynamic environments. Background Technology

[0002] With the rapid development of robotics technology, intelligent navigation and autonomous operation of mobile robots in unknown unstructured environments (post-disaster search and rescue, complex warehousing, field exploration and security patrol) has become a research hotspot. In these scenarios, robots not only need to have efficient path planning capabilities, but also need to be able to work collaboratively with human operators, combining human intuition and experience with the machine's precise execution capabilities to complete complex tasks.

[0003] Currently, methods for solving autonomous navigation in mobile robots are mainly divided into classical theory-based methods and data-driven methods based on deep reinforcement learning. However, existing technologies have the following significant drawbacks in practical applications: 1. Ignoring the dynamic obstacle adversarial properties in the environment: Traditional MPC relies heavily on accurate system and environment models and is extremely sensitive to perceived noise and nondeterministic behavior. In complex real-world scenarios involving dynamic obstacles with malicious interception, racing games, or electronic interference, obstacles often have subjective game and adversarial properties. Existing control models usually assume that obstacles are static or move at a constant speed, lacking the expectation of worst-case adversarial disturbances, which can easily lead to local failures or collision risks. 2. Unresolved human-machine adversarial conflicts in human-machine shared control: In the field of human-machine shared control, the core challenge lies in assessing the reliability of autonomous strategies to determine when humans should intervene. Existing technologies mostly assume that human instructions are absolutely correct. However, in adversarial scenarios with high pressure, high cognitive load, or communication hijacking, human operators are very likely to issue adversarial instructions that completely violate physical security or are diametrically opposed to machine strategies. This makes it impossible for existing fixed weight fusion or hard switching mechanisms to quantify such human-machine intention discrepancies in real time. When dealing with internal adversarial conflicts, it can lead to struggles for system control and violent fluctuations, or even directly execute fatal human errors. 3. Pure DRL methods have black-box properties and uncertainty defects: When faced with new scenarios outside the distribution, pure DRL methods may generate unstable or unconstrained control inputs, which may lead to collisions or system failures. Summary of the Invention

[0004] The purpose of this invention is to provide a human-machine collaborative adversarial control optimization system for complex dynamic environments. It aims to solve the control failure and collision risk problems of mobile robots in complex game-theoretic environments caused by malicious interception by external dynamic obstacles and internal human-machine intention conflicts by introducing a distributed value assessment network and tubular robust model predictive control. This system utilizes conditional risk value to quantify the tail risk of human commands in real time and constructs adversarial divergence. When high-risk or erroneous commands are detected, adaptive and smooth deprivation of control is achieved. Simultaneously, forward reachability tubes for obstacles are constructed to cover adversarial disturbances in the worst-case scenario. Physical hard constraints under minimax optimization are executed in the prediction time domain, thereby improving the robot's task achievement rate and human-machine collaboration efficiency in unstructured dynamic environments while ensuring safety.

[0005] This invention solves the above-mentioned technical problems through the following technical solution: a human-machine collaborative adversarial control optimization system for complex dynamic environments, comprising: The environmental state perception module is used to acquire the motion state of the mobile robot and the state of environmental obstacles and generate an environmental state vector. The autonomous decision-making module is used to generate autonomous control commands based on the environmental state vector; The manual instruction acquisition module is used to acquire manual control instructions input by the operator. The machine confidence assessment module is used to determine machine confidence based on the cognitive uncertainty of autonomous decision-making and the feasibility of safe actions in the environment; The human-machine adversarial intent quantification and fusion module is used to construct a return probability distribution through a distributed value assessment network, calculate the conditional risk value (CVaR) of human control instructions, determine the adversarial divergence based on the conditional risk value and autonomous control instructions, determine the basic weight of human control instructions through a cost function that includes the cost of intervention and the cost of control risk, couple the basic weight with the machine confidence to obtain the fusion weight, and fuse human control instructions and autonomous control instructions to form a fused control instruction. The robust model predictive control safety verification module is used to establish a dynamic obstacle prediction model that includes nominal motion state and bounded disturbance, and to construct a forward reachable tube for the dynamic obstacle in the prediction time domain. Based on the forward reachable tube, tubular avoidance constraints are applied to the robot's predicted trajectory to form a set of safe actions. When the control command to be executed does not meet the tubular avoidance constraints, it is projected onto the set of safe actions to obtain a safe control command. The action execution and status update module is used to execute the safety control commands and update the environmental status to form a closed-loop control.

[0006] Specifically, the environmental state perception module includes a lidar, an inertial measurement unit, and a wheeled odometer. The environmental state vector includes at least the spatial distribution information of obstacles relative to the mobile robot, the current linear velocity of the mobile robot, and the current angular velocity of the mobile robot. The autonomous decision-making module uses the Twin Delayed Deep Deterministic Policy Gradient algorithm to construct a deep reinforcement learning policy network and generates the autonomous control instructions based on the environmental state vector. The autonomous decision-making module and the manual instruction acquisition module operate in parallel to obtain the autonomous control instruction and the manual control instruction, respectively.

[0007] Specifically, the machine confidence assessment module is configured as follows: During the inference process of the deep reinforcement learning policy network, the random deactivation mechanism is kept on, and forward propagation with different random masks is performed multiple times on the same environment state vector to obtain multiple action samples. Calculate the sample variance of the multiple action samples in each dimension of the action space, and determine the cognitive uncertainty index based on the sample variance of each dimension; The cognitive uncertainty index is converted into Bayesian confidence level using a nonlinear mapping function; Within the preset prediction time domain, trajectory prediction is performed on multiple candidate actions, and the number of safe actions that meet the collision avoidance conditions is counted. The machine confidence is determined based on the environmental geometric feasibility represented by the number of safe actions and the Bayesian confidence, so that the machine confidence simultaneously represents the reliability of the deep reinforcement learning policy network's cognition of the current environment and the safe operating space in the current environment.

[0008] Specifically, the distributed value assessment network is used to divide the return probability space into multiple quantile intervals and output multiple return prediction values ​​corresponding to different quantile positions, and construct a discrete return probability distribution corresponding to the state and action pairs based on the multiple return prediction values. The human-machine adversarial intent quantification and fusion module is configured to calculate the conditional value of risk (CVaR) based on the reward probability distribution corresponding to the human control command at a preset risk confidence level, so as to characterize the tail risk of the human control command in the case of low reward, and construct the adversarial divergence based on the autonomous control command and the conditional value of risk (CVaR), so that the adversarial divergence increases as the tail risk of the human control command increases.

[0009] Specifically, the cost function includes a control intervention cost that characterizes the degree of intervention caused by reducing human control, and a control risk cost that characterizes the degree of risk caused by executing human control instructions. The human-machine adversarial intent quantification and fusion module obtains the basic weights corresponding to the human control commands by minimizing the cost function, and dynamically adjusts the basic weights according to the machine confidence to obtain the fused weights of the human control commands. The higher the machine confidence and / or the greater the adversarial divergence, the lower the fusion weight of the manual control command, and the manual control command and the autonomous control command are weighted and fused based on the fusion weight.

[0010] Specifically, the robust model predictive control safety verification module is configured as follows: Based on the distance information obtained by the lidar and the corresponding scanning angle, an obstacle point set in the local coordinate system of the mobile robot is constructed. The collision detection threshold is determined based on the robot radius of the mobile robot and the preset safety buffer distance; A differential-driven discrete kinematics model is used to perform forward integration of candidate control commands in the prediction time domain according to a preset time step to generate the corresponding robot prediction trajectory. Calculate the minimum distance between the robot's predicted trajectory and the set of obstacle points, and determine whether the corresponding candidate control command is safe based on the minimum distance and the collision determination threshold; The candidate control commands are obtained by sampling in the online velocity action space and angular velocity action space, and a set of safe actions is formed based on the safety judgment results of the candidate control commands.

[0011] Specifically, the dynamic obstacle prediction model represents the motion state of a dynamic obstacle as a combination of a nominal motion state and a bounded perturbation, wherein the nominal motion state is predicted based on the currently observed motion state of the dynamic obstacle, and the bounded perturbation is used to characterize the unmodeled dynamic and / or the adversarial motion of the dynamic obstacle relative to the mobile robot. The robust model predictive control safety verification module obtains the forward reachability tube by means of Minkowski algorithm and recursion over multiple consecutive prediction steps, based on the basic geometric envelope of the dynamic obstacle, the nominal motion state, and the bounded perturbation, so that the forward reachability tube covers the possible space occupied by the dynamic obstacle within the corresponding prediction step.

[0012] Specifically, the robust model predictive control safety verification module transforms the minimax safety constraint for the bounded disturbance into a geometric space avoidance constraint for the forward reachable tube through constraint tightening. The geometric space avoidance constraint is used to limit the mobile robot to maintain a safe distance between the predicted trajectory of the mobile robot in the prediction time domain and the forward reachable tube. The safe distance is determined based on the maximum permissible disturbance of the dynamic obstacle and the preset adversarial robust amplification factor. The candidate control command is added to the set of safe actions only if the robot's predicted trajectory corresponding to the candidate control command satisfies the geometric space avoidance constraint in the prediction time domain.

[0013] Specifically, the robust model predictive control safety verification module performs safety verification before and after control command fusion. When the control instruction to be executed belongs to the set of safety actions, the control instruction to be executed is directly determined as a safety control instruction; When the control instruction to be executed does not belong to the set of safety actions, the candidate control instruction with the smallest weighted deviation from the control instruction to be executed is selected from the set of safety actions according to the principle of minimum intervention. The cost of correcting linear velocity and angular velocity is limited by weight matrix. When the set of safety actions is empty, emergency obstacle avoidance control, including deceleration, steering, and / or manual prompts, is triggered.

[0014] Specifically, the action execution and status update module is used to convert the safety control command into motor control signals corresponding to the underlying drive mechanism of the mobile robot; After the mobile robot executes the safety control command, the robot's pose state and the obstacle state in the environment are reacquired, and the environmental state vector is updated based on the reacquired pose state and obstacle state. The environmental state vector is then executed cyclically according to the control cycle, including environmental state perception, autonomous control command generation, machine confidence assessment, human-machine adversarial intent quantification, control command fusion, robust safety verification, action execution, and state update.

[0015] The beneficial effects of this invention are: 1. This invention introduces a distributed value assessment network, which uses Conditional Value at Risk (CVaR) to specifically characterize the tail risk of human commands in adversarial environments. It constructs a human-machine adversarial divergence to achieve highly sensitive identification of dangerous intentions, transforming the allocation of control weights into a strict cost function optimization problem. When a human command is detected to have a huge adversarial risk, the system automatically and completely strips the human control, completely eliminating the risk of control struggle and system oscillation, effectively filtering malicious or erroneous commands, and breaking through the inherent assumption of the absolute correctness of human commands in existing technologies. 2. This invention innovatively introduces a tubular robust MPC safety verification layer. By using Minkowski and online construction of forward reachable tubes for dynamic obstacles, it incorporates unmodeled dynamics and active adversarial disturbances into safety considerations, accurately covering the worst adversarial trajectory of obstacles. Through a constraint tightening strategy, it transforms the incalculable infinite-dimensional minimax optimization into a geometric space boundary detection problem. While ensuring real-time performance, it constructs a physical safety barrier for the robot in adversarial environments, solving the control failure problem caused by the assumption of static or uniform speed of obstacles in traditional methods. 3. This system uses the MC-Dropout mechanism to quantify the cognitive uncertainty of the autonomous agent in real time, and couples it with environmental geometric feasibility information to generate a composite confidence level. This enables the robot to adaptively increase the weight of the need for human intervention when entering an unfamiliar environment, achieving a smooth and safe transition of human-robot collaboration and making up for the black-box instability of the pure DRL method. 4. This invention uses TD3 deep reinforcement learning as the master decision-maker to ensure efficient planning in unstructured environments. At the same time, it adopts a uniform sampling method to use MPC as a lightweight online safety layer. Dangerous actions are optimized by constraint projection according to the principle of minimum intervention, which reduces the computational power consumption of traditional MPC and makes up for the lack of safety constraints in pure DRL methods, thus achieving a balance between efficiency and safety. Attached Figure Description

[0016] Figure 1 This is a diagram illustrating the overall framework of the human-machine collaborative adversarial control optimization system for complex dynamic environments according to the present invention.

[0017] Figure 2 This is a diagram of the Bayesian neural network structure of MC-Dropout in this invention.

[0018] Figure 3 This is the robust MPC security layer management layer based on a tubular structure in this invention.

[0019] Figure 4 This is a schematic diagram of the experimental environment for the mobile robot of this invention. Detailed Implementation

[0020] The present invention will be further illustrated by way of embodiments below, but the present invention is not limited to the scope of the embodiments.

[0021] Example 1:

[0022] like Figure 1As shown, this invention proposes a human-robot collaborative adversarial control optimization system for complex dynamic environments. The robot acquires observations S through sensors. The DRL policy generates actions and can accept operator commands. Both are first verified by an adversarial robust MPC safety verification layer based on Minimax optimization: if it fails, the MPC will fully evaluate the worst-case adversarial trajectory that the dynamic obstacle might take, correct the actions through online constraint optimization, and re-verify; if it passes, it enters the human-robot collaboration and adversarial module. In this module, the system innovatively introduces a human-robot adversarial intent recognition mechanism, which calculates the human-robot adversarial divergence. To quantify internal command conflicts, and combined with the system composite confidence C output by MC-Dropout, the manual input is exponentially penalized and dynamically fused to generate fused control commands. After fusion, a second MPC verification is performed; if still unsafe, the process is repeated until the physical constraints under extreme conditions are met. Finally, the safe command is executed, and the execution result is fed back to the perception module, forming a closed loop. This system includes: The environmental state perception module is used to acquire the motion state of the mobile robot and the state of environmental obstacles and generate an environmental state vector. The autonomous decision-making module is used to generate autonomous control commands based on the environmental state vector; The manual instruction acquisition module is used to acquire manual control instructions input by the operator. The machine confidence assessment module is used to determine machine confidence based on the cognitive uncertainty of autonomous decision-making and the feasibility of safe actions in the environment; The human-machine adversarial intent quantification and fusion module is used to construct a return probability distribution through a distributed value assessment network, calculate the conditional risk value (CVaR) of human control instructions, determine the adversarial divergence based on the conditional risk value and autonomous control instructions, determine the basic weight of human control instructions through a cost function that includes the cost of intervention and the cost of control risk, couple the basic weight with the machine confidence to obtain the fusion weight, and fuse human control instructions and autonomous control instructions to form a fused control instruction. The robust model predictive control safety verification module is used to establish a dynamic obstacle prediction model that includes nominal motion state and bounded disturbance, and to construct a forward reachable tube for the dynamic obstacle in the prediction time domain. Based on the forward reachable tube, tubular avoidance constraints are applied to the robot's predicted trajectory to form a set of safe actions. When the control command to be executed does not meet the tubular avoidance constraints, it is projected onto the set of safe actions to obtain a safe control command. The action execution and status update module is used to execute the safety control commands and update the environmental status to form a closed-loop control.

[0023] The specific work includes the following steps: Step 1: Synchronization initialization of environmental state perception and control sources The system first acquires raw observation data of the robot's environment in real time using an airborne LiDAR, inertial measurement unit (IMU), and wheeled odometry. This raw observation data is then converted into a state vector. This includes the polar coordinate distribution of obstacles relative to the robot and the robot's current linear velocity. With angular velocity In obtaining Subsequently, the system launches two control branches in parallel: the autonomous decision-making branch utilizes the policy network of the deep reinforcement learning algorithm TD3 (Twin Delayed Deep Deterministic Policy Gradient). according to Derive autonomous control quantities The manual control branch captures the original instructions input by the operator through the interactive interface in real time. These two commands together serve as the input reference for the subsequent shared control system.

[0024] Step 2: Machine confidence assessment based on Monte Carlo sampling (MC-Dropout) like Figure 2 As shown, to evaluate the reliability of the autonomous agent's decisions in real time when facing untrained scenarios (i.e., out-of-distribution data), the system introduces a random dropout mechanism in the inference phase of the TD3 policy network. This mechanism works by testing the same observation value... Perform M forward propagations with random masks to obtain a set of action samples. The system calculates the sample variance for each dimension of the actions in the set. And define the cognitive uncertainty index U as the average of the variances of each dimension: ; Where d is the dimension of the action space. To convert this into mathematical confidence, the system uses a parameterized... and A controlled sigmoid function maps uncertainty to Bayesian confidence levels. : ; The physical significance of this step lies in the fact that when a robot enters a completely unfamiliar and complex environment, the action deviation between different samples will increase, leading to... This reduces the system's need for human intervention.

[0025] In addition to considering the cognitive confidence level within the algorithm, the system also needs to incorporate feedback from environmental geometric constraints. This invention utilizes the MPC safety layer to perform parallel trajectory evolution prediction for N preset candidate actions within the prediction time domain T, based on a simplified robot kinematics model. The system uses a collision detection algorithm to count the number of safe actions that will not cause a collision under the current obstacle distribution. Compare this value with the safety threshold. Compare and combine with the aforementioned Calculate the final machine confidence score C: ; in This is the lower confidence level factor. This composite index C can simultaneously reflect whether the agent is familiar with the environment and whether the current environment has sufficient safe operating margin.

[0026] Step 3: Quantification and Adaptive Fusion of Human-Machine Adversarial Intent Based on Bloom Bar Value Assessment and CVaR After obtaining the machine confidence score C, traditional methods typically assign weights directly or use simple linear switching. However, in extreme adversarial environments, human operators may issue potentially fatal instructions due to cognitive overload or interference. To more accurately and cutting-edgely quantify this internal human-machine adversarial process, this step introduces a distributed criterion network and combines Conditional Value at Risk (CVaR) with optimization theory to derive and generate an adaptive human-machine fusion strategy. The specific implementation process is as follows: Instead of outputting a single expected reward, the distributed Critic network divides a continuous probability space into N equal parts. The final layer of the network uses a linear output to generate a prediction vector containing N quantile locations. The system treats these N equally probable discrete predictions as Dirac deltaic values. The mixture of functions (Dirac delta functions) formally defines and reconstructs state-action pairs mathematically. Complete return probability distribution : ; This reconstructed distribution enables the system not only to grasp the expected value of the return but also to accurately characterize the variance and long-tail features of the return. Simultaneously, to ensure optimization efficiency in a normal, non-adversarial environment, the system constrains the arithmetic expectation of this distribution to equal the traditional Q-value. ; For real-time captured operator raw instructions The system not only evaluates its expected returns but also focuses on its tail risk in adversarial environments, i.e., its performance in the worst-case scenario. Confidence level parameters are set. (For example (Representing the worst 5% scenario), calculating the conditional value of risk for human instructions. : ; in, It is a reward distribution -Quantities. The lower the value, the greater the potential risk that human instructions will cause catastrophic consequences when faced with sudden changes in obstacles.

[0027] Autonomous control quantity derived from autonomous decision-making branch Constructing adversarial divergence based on CVaR risk characteristics of human commands. : ; In the formula, This is the risk aversion adjustment coefficient. The physical meaning of this formula is: even if the expected return of a human instruction is acceptable, as long as its tail risk is extremely high (i.e., ...), the risk aversion will be mitigated. (extremely small) It will also increase dramatically, thus becoming extremely sensitive to detecting dangerous human intentions.

[0028] This system will ultimately determine the human weight. The allocation is transformed into a rigorous optimization problem. The total cost function of fusion control is defined. It consists of two parts: the intervention cost of depriving humans of control and the risk cost of executing dangerous instructions. ; in, To counteract the sensitivity coefficient. To obtain the optimal human-machine weight allocation. ,right about Differentiate and set it equal to 0: ; The fundamental analytical solution for obtaining the optimal human intervention weights is: ; Finally, the aforementioned basic weights based on extreme value risk and cost optimization will be applied. With machine confidence level C Dynamic coupling yields the weight proportions in the final generated fusion control command: ; This theoretical derivation not only achieves a smooth allocation of control, but also provides a mathematically solid guarantee: when the agent has extreme confidence in the environment ( ) or human commands pose a significant risk of confrontation ( When this happens, the system can theoretically and automatically achieve a smooth and complete stripping of human control from the ground up. This effectively filters out dangerous confrontation commands from humans.

[0029] Finally, fusion control commands are generated. : ; This step not only achieves smooth allocation but also effectively filters out dangerous and confrontational commands from humans.

[0030] Step 4: Constraint Projection Optimization Based on MPC Security Layer In this invention, the MPC is not used as a replacement for the DRL or human operation as a main controller, but rather as an online safety layer: its primary responsibility is to predict the future trajectory and perform collision detection at each control step based on current radar data. This invention performs model prediction in a local robot coordinate system, based on radar distance readings. Corresponding scanning angle Mapping the local obstacle point set and with robot radius With safety buffer The sum of the values ​​is used as the threshold for judgment. .

[0031] Specifically, the prediction model uses difference-driven discretization, assuming the robot's initial pose in the local coordinate system is... For a certain constant control Within the prediction time domain T, with time steps Forward Integral Step, the trajectory update formula is: ; Based on this discrete predicted trajectory, the present invention defines control. The corresponding minimum obstacle distance is: ; like Then it is believed If there is no collision risk in the prediction time domain, otherwise it is considered... These are unsafe actions. However, they also constitute a set of safe actions: ;

[0032] Candidate set It is obtained by uniform sampling over the action space, i.e. ; and For sampling space.

[0033] Step 5: Dynamic Adversarial MPC Security Layer Based on Tube-based Robustness like Figure 3 As shown, considering that the dynamic obstacles in the environment not only have motion uncertainties, but may even have adversarial properties of active interception, the system introduces Tube-based Robust MPC to specifically solve the Minimax safety verification problem.

[0034] First, define the prediction state equation for dynamic obstacles. Assume the center position of the obstacle in the local coordinate system is... Its true motion model consists of a nominal model and adversarial perturbations: ; in, Based on the current observation speed The nominal kinematic forward integral, This represents an unmodeled dynamic or subjective adversarial perturbation in the environment. It is assumed that the perturbation is bounded, i.e. ,in .

[0035] To cover all possible worst-case adversarial trajectories within the prediction time domain T, the system constructs a "Forward Reachable Tube" for obstacles online. For the prediction step size... The absolute space occupied by obstacles Through Minkowski and (Minkowski Sum, symbolized as) Recursive generation: ; in, This represents the basic geometric envelope of the obstacle. With the prediction step size... As the uncertainty increases, the boundary of uncertainty will spread in a funnel shape, which precisely quantifies the spatial distribution of the worst-case adversarial situation.

[0036] In previous collision detection methods, the minimum-maximum projection condition was: the robot's predicted trajectory under all possible adversarial perturbations. A minimum distance requirement must be met. This invention transforms the Minimax problem into a deterministic online solution by constraint tightening. The tightened safety distance determination function is defined as follows: ; Safety Actions Collection The admission criteria have been updated to strict tubular avoidance constraints: ; In the formula, To counteract the robust amplification factor, The maximum adversarial disturbance that the obstacle can generate per second. The candidate action is defined if and only if... An action is considered absolutely safe and is only incorporated when it allows the robot's trajectory to completely avoid the aforementioned expanded uncertainty channel. middle.

[0037] In actual operation, when the system acquires a target action (This could come from a DRL policy mapped to the physical action space, or it could come from manual input.) First, the safety of the action is checked: if... If the action is safe, then execute it directly; otherwise, it must be performed from the beginning. Choose an alternative action. The selection criterion follows the principle of minimum perturbation: ; ; The weight matrix This step is used to adjust the tolerance for linear and angular velocity corrections. It ensures that the robot will never experience a physical collision, even in the event of human error or AI decision-making bias. When the safety set is empty, the system triggers emergency obstacle avoidance strategies (such as deceleration, rapid steering, or manual prompting) to avoid unacceptable collision risks.

[0038] This invention transforms the previously incalculable infinite-dimensional Minimax problem into a boundary detection problem in geometric space, significantly improving the real-time performance of the algorithm.

[0039] Step Six: Action Execution and Timing State Update Final optimized security instructions The signal is converted into a PWM control signal for the underlying motor and executed. After the action is executed, the robot reaches a new pose, and the environmental state is updated. Experimental environment such as Figure 4 As shown, the system returns to step one for high-frequency iteration. Through a closed-loop mechanism, this invention achieves deep coupling of TD3's efficient planning capabilities, human intuition and experience, and MPC's strict security boundaries in unknown adversarial environments, thereby maximizing task execution efficiency and human control experience while ensuring safety.

[0040] Example 2:

[0041] Based on Example 1, this embodiment further explains the process of determining machine confidence, human-machine adversarial risk, and human control weights after a mobile robot enters a dynamic scenario where it has not participated in the training of the autonomous decision-making network.

[0042] This embodiment employs a differential-driven mobile robot with a radius of 0.28 meters, an allowable linear velocity of 0 to 1.2 meters per second, an allowable angular velocity of -1.0 to 1.0 radians per second, and a control cycle of 0.1 seconds. A lidar continuously collects distance and orientation information of obstacles around the robot, an inertial measurement unit collects the robot's current angular velocity, and a wheeled odometry system collects the robot's current linear velocity and pose change. The controller arranges the sensor data into an environmental state vector according to a predetermined data arrangement order and normalizes it according to the maximum effective measurement range corresponding to each sensor quantity, ensuring that different physical quantities input to the autonomous decision-making network maintain similar numerical scales.

[0043] The autonomous decision-making network employs a dual-delay deep deterministic policy gradient network. Each control cycle, the network outputs two normalized control variables: the first corresponds to the robot's linear velocity, and the second to its angular velocity. Adding the first normalized control variable to 1 and multiplying by 0.6 yields the robot's actual linear velocity; converting the second normalized control variable to a ratio of 1 radian per second gives the robot's actual angular velocity. This process transforms the autonomous decision-making network output into motion control variables that the robot chassis can execute.

[0044] To determine whether the current environmental state significantly deviates from the environmental state covered during the training phase of the autonomous decision-making network, the random deactivation layer in the autonomous decision-making network is kept in the enabled state during each control cycle, and 20 forward computations are performed continuously for the same environmental state. Each forward computation uses a different combination of random deactivation nodes, thereby obtaining 20 sets of autonomous control action samples.

[0045] For the linear velocity dimension, firstly, the 20 linear velocity samples are summed and divided by 20 to obtain the average linear velocity sample. Then, the average linear velocity sample is subtracted from each linear velocity sample, and the 20 differences are squared. The sum of these 20 squares is then divided by 19 to obtain the linear velocity sample variance. The angular velocity dimension is calculated using the same process to obtain the angular velocity sample variance. The linear velocity sample variance and the angular velocity sample variance are added together and divided by 2. The result is taken as the cognitive uncertainty corresponding to the current environmental state.

[0046] The criterion for judging cognitive uncertainty is determined by independent validation data after the autonomous decision-making network has been trained.

[0047] Specifically, all verification states recorded by the mobile robot during collision-free navigation in a known environment are input into the autonomous decision-making network one by one. For each verification state, 20 random deactivation forward computations are performed, and the cognitive uncertainty corresponding to each verification state is calculated according to the aforementioned process. All cognitive uncertainties are arranged in ascending order of value, and a statistical value that ensures approximately 95% of the cognitive uncertainties in all verification states are not higher than this value is selected as the uncertainty center threshold. Furthermore, a statistical value that ensures approximately 99% of the cognitive uncertainties in all verification states are not higher than this value is selected as the high uncertainty reference value.

[0048] In this embodiment, the uncertainty center threshold obtained from the verification data is 0.018, and the high uncertainty reference value is 0.032. Thus, 0.018 corresponds to the cognitive uncertainty range of most environmental states under normal navigation conditions, while 0.032 corresponds to the higher cognitive uncertainty level that can be achieved by only a small number of environmental states in the normal verification data.

[0049] After obtaining the current cognitive uncertainty, it is converted into cognitive confidence. Specifically, the calculation involves first subtracting 0.018 from the current cognitive uncertainty, multiplying the difference by 156.9, calculating the natural index value corresponding to this product, adding the natural index value to 1, and finally dividing the result by 1 to obtain the cognitive confidence. The value of 156.9 is derived in reverse from the constraint that the confidence level drops to 0.1 corresponding to a high uncertainty reference value. Specifically, the natural logarithm of 9 is calculated first, and then the result is divided by the difference between 0.032 and 0.018 to obtain 156.9. Using this conversion relationship, when the cognitive uncertainty is 0.018, the cognitive confidence is 0.5; as the cognitive uncertainty exceeds 0.018, the cognitive confidence continuously decreases, thus avoiding sudden changes in control caused by hard switching using a single threshold.

[0050] After obtaining cognitive confidence, the operable space of the robot in the current environment is evaluated. In this embodiment, 11 linear velocities are selected at equal intervals within the allowable linear speed range, and 21 angular velocities are selected at equal intervals within the allowable angular velocity range. The combination of the two forms 231 candidate control actions.

[0051] For each set of candidate control actions, using the robot's current pose as the trajectory prediction starting point, 20 motion state recursions are continuously executed at 0.1-second intervals within a 2-second prediction time domain. The robot radius is 0.28 meters, and the safety buffer distance is set at 0.15 meters, which together equal 0.43 meters. This 0.43-meter distance is used as the base collision detection distance between the robot and static obstacles. When the distance between all 20 predicted positions corresponding to a candidate control action and the nearest obstacle is not less than 0.43 meters, the candidate control action is counted in the number of safe actions.

[0052] The baseline for the number of safe actions is determined using a collision-free navigation verification trajectory. Specifically, for each environmental state in the collision-free navigation verification trajectory, trajectory prediction for the aforementioned 231 sets of candidate control actions is performed. The number of safe actions that meet the collision avoidance requirements in each environmental state is recorded, and all safe actions are arranged in ascending order. A statistical value that ensures approximately 90% of the verification states have no fewer than this number of safe actions is selected as the baseline for the number of safe actions. In this embodiment, the baseline for the number of safe actions obtained from the verification data is 58.

[0053] When the number of safe actions within the current control cycle is not less than 58, the environmental geometrical feasibility is set to 1. When the number of safe actions is less than 58, the current number of safe actions is divided by 58, and the resulting ratio is used as the environmental geometrical feasibility. Subsequently, the environmental geometrical feasibility is multiplied by 0.8, the result is added to 0.2, and then multiplied by the aforementioned cognitive confidence to obtain the machine confidence. After this processing, when the number of safe actions available to the robot in the current environment decreases, the machine confidence decreases accordingly. Simultaneously, the cognitive confidence still participates in the final machine confidence calculation, allowing the autonomous decision-making network's cognitive uncertainty regarding the unfamiliar environment to directly influence the allocation of control.

[0054] After the operator inputs a manual control command, the distributed value assessment network evaluates the value of both the manual and autonomous control commands. In this embodiment, the complete return distribution is divided into 51 equally probable return locations; therefore, each manual and autonomous control command receives one of 51 predicted return values.

[0055] When determining the conditional value of risk (VLT) of a manually controlled instruction, the 51 projected returns corresponding to the instruction are arranged in ascending order, and the lowest 10% of returns are calculated. 51 multiplied by 10% equals 5.1. To ensure that all projected returns within the lowest 10% are included in the risk assessment, the number is rounded up to 6. Therefore, the lowest 6 projected returns are selected. The conditional VLT of the manually controlled instruction is obtained by adding these 6 projected returns together and dividing by 6.

[0056] For example, within one control cycle of this embodiment, the six lowest predicted return values ​​corresponding to the manual control instruction are -0.92, -0.87, -0.81, -0.75, -0.69, and -0.62, respectively. Adding these six predicted return values ​​together yields -4.66. Dividing -4.66 by 6 gives a conditional value of risk of approximately -0.777 for the manual control instruction.

[0057] The return level of autonomous control commands is determined by averaging its 51 predicted return values. Specifically, the 51 predicted autonomous control return values ​​are summed and then divided by 51. The average return of autonomous control commands obtained in this control period is 0.34.

[0058] The degree of antagonism between manual control commands and autonomous control commands is then determined. This degree of antagonism considers both the degree of deviation from the action direction and the tail risk of the manual control commands.

[0059] The calculation process for the degree of motion deviation is as follows: First, subtract the normalized linear velocity of the autonomous control command from the normalized linear velocity of the manual control command, and square the difference; then, subtract the normalized angular velocity of the autonomous control command from the normalized angular velocity of the manual control command, and square the difference; add the two squared results together and divide by 2 to obtain the degree of motion deviation.

[0060] During this control cycle, the normalized linear velocity and normalized angular velocity of the autonomous control commands were 0.42 and 0.18, respectively, while those of the manual control commands were 0.85 and -0.65, respectively. The difference between the manual and autonomous linear velocities was 0.43, which squared to obtain 0.1849; the difference between the manual and autonomous angular velocities was -0.83, which squared to obtain 0.6889. Adding 0.1849 and 0.6889 and dividing by 2 yields a motion deviation of approximately 0.437.

[0061] The tail risk differential for manually controlled instructions is obtained by subtracting the average return of autonomously controlled instructions from the conditional value of risk of manually controlled instructions. Subtracting the conditional value of risk of manually controlled instructions (-0.777) from the average return of autonomously controlled instructions (0.34) yields 1.117. When this difference is less than 0, the tail risk differential is set to 0, thus increasing the risk penalty only when the low-return risk of manually controlled instructions is higher than that of autonomously controlled instructions.

[0062] To ensure that the degree of action deviation and the tail risk difference are on a comparable numerical scale, a risk scaling coefficient is added to the tail risk difference. The risk scaling coefficient is determined using independent validation data. Specifically, all degree of action deviation and all tail risk differences in the validation data are statistically analyzed, and representative values ​​at the middle position of the sorted data are obtained for each. The representative value of the degree of action deviation is then divided by the representative value of the tail risk difference, and the resulting ratio is used as the risk scaling coefficient. In this embodiment, the risk scaling coefficient obtained from the validation data is 0.8. Therefore, multiplying the tail risk difference of 1.117 by 0.8 yields approximately 0.894. Adding 0.894 to the degree of action deviation of 0.437 gives an adversarial divergence of approximately 1.331 between the current manual control command and the autonomous control command.

[0063] After obtaining the adversarial divergence, the basic weight of manual control is determined by minimizing the cost of intervention in control and the cost of executing dangerous instructions. The manual control weight is limited to between 0 and 1. For any manual control weight to be evaluated, first subtract the manual control weight from 1, and square the difference as the cost of intervention in control caused by reducing manual control; then square the manual control weight again and multiply it by the current adversarial divergence and adversarial sensitivity coefficient in turn, and the result is the risk cost of continuing to execute manual control instructions. The two costs are added together to obtain the total cost corresponding to the manual control weight.

[0064] When the total cost reaches its minimum value, the corresponding artificial control weight is used as the basic artificial control weight. After adjusting for a first-order rate of change of 0 for the total cost, the specific calculation process is as follows: first, multiply the adversarial sensitivity coefficient by the adversarial divergence; then, add the product to 1; finally, divide 1 by the sum to obtain the basic artificial control weight. The adversarial sensitivity coefficient is determined according to the control requirement that the basic artificial control weight should not exceed 0.25 when the adversarial divergence reaches 1. Substituting the basic artificial control weight of 0.25 into the aforementioned calculation relationship for reverse calculation, we obtain an adversarial sensitivity coefficient of not less than 3; this embodiment uses 3.

[0065] During this control period, the adversarial divergence is 1.331. First, multiply 1.331 by 3 to get 3.993; then add 3.993 to 1 to get 4.993; finally, divide 1 by 4.993 to get the artificial control base weight of approximately 0.200.

[0066] During this control cycle, the cognitive uncertainty obtained through 20 random deactivation forward calculations is 0.026. Following the aforementioned cognitive confidence conversion process, the cognitive confidence is approximately 0.222. Of the current 231 candidate control actions, 35 meet the collision avoidance requirements; therefore, dividing 35 by 58 yields an environmental geometric feasibility of approximately 0.603. Multiplying 0.603 by 0.8 gives approximately 0.482, adding this to 0.2 gives approximately 0.682, and then multiplying this by the cognitive confidence of 0.222 yields a machine confidence of approximately 0.151.

[0067] The final manual control weight is determined by both the basic manual control weight and the machine confidence level. Specifically, the machine confidence level of 0.151 is subtracted from 1 to obtain 0.849. Then, 0.849 is multiplied by the basic manual control weight of 0.200 to obtain a final manual control weight of approximately 0.170. Correspondingly, the autonomous control weight is 1 minus 0.170, resulting in 0.830. When generating fused control commands, the normalized linear velocity of the manual control command is multiplied by 0.170, and the normalized linear velocity of the autonomous control command is multiplied by 0.830. The two terms are then added together to obtain a normalized fused linear velocity of approximately 0.493. The manual angular velocity and autonomous angular velocity are processed in the same way to obtain a normalized fused angular velocity of approximately 0.039. Adding the normalized fusion linear velocity of 0.493 to 1 gives 1.493, which is then multiplied by 0.6, resulting in an actual fusion linear velocity of approximately 0.896 meters per second and a fusion angular velocity of 0.039 radians per second.

[0068] Therefore, when the robot enters an environment outside the coverage of the autonomous decision-making network's training data, the cognitive uncertainty obtained from random deactivation calculations increases, leading to a decrease in machine confidence and preserving control space for human intervention. When human control commands deviate significantly from autonomous control commands and the low-reward risk of human control commands increases, the adversarial divergence increases accordingly, and the basic weight of human control decreases, thereby suppressing high-risk human control commands. The resulting fused control commands are then input into the robust model predictive control safety verification module, and after meeting the collision avoidance conditions in the prediction time domain, they are sent to the robot's underlying actuators.

[0069] Example 3:

[0070] Within the control framework of Embodiment 1, this embodiment further illustrates the specific formation process of the dynamic obstacle disturbance boundary, forward reachability tube, safety action set, and control action correction when a dynamic obstacle has a lateral approach and interception tendency.

[0071] This embodiment employs a differential-driven mobile robot with a radius of 0.30 meters, a basic safety buffer distance of 0.15 meters, a control cycle of 0.1 seconds, and a prediction time domain of 2.5 seconds for model predictive control. Therefore, for each safety verification, 25 consecutive predicted positions are calculated for both the robot and the dynamic obstacle.

[0072] The lidar continuously acquires the contour points of dynamic obstacles in the environment. Contour points belonging to the same dynamic obstacle are spatially clustered, and the obstacle's center position is determined based on the spatial positions of all contour points in that cluster. Between adjacent control cycles, data association for the same dynamic obstacle is completed based on changes in distance to the obstacle's center position and the continuity of movement.

[0073] The controller saves the center position of the same dynamic obstacle for the last 5 control cycles and determines the current lateral and longitudinal velocities of the obstacle in the robot's local coordinate system based on these 5 center positions.

[0074] Specifically, the sampling times corresponding to the five most recent center positions are set to 0 seconds, 0.1 seconds, 0.2 seconds, 0.3 seconds, and 0.4 seconds, respectively. A straight line is fitted to each of the five sampling times and its corresponding lateral position, minimizing the sum of the squared errors between the fitted line and the five actual lateral positions. The slope of the fitted line is taken as the obstacle's current lateral velocity. The longitudinal velocity is calculated in the same way, i.e., a straight line is fitted to the same five sampling times and their corresponding longitudinal positions, and the slope of the fitted line is taken as the obstacle's current longitudinal velocity.

[0075] Determining the movement speed by using five consecutive obstacle locations can reduce the impact of position detection fluctuations in a single lidar scan cycle on the obstacle speed estimation results.

[0076] After obtaining the current lateral and longitudinal velocities of the dynamic obstacle, the current center position of the obstacle is used as the starting point for nominal trajectory prediction. The calculation process for the first predicted position is as follows: multiply the obstacle's lateral velocity by 0.1 seconds to obtain the lateral displacement, and add this lateral displacement to the obstacle's current lateral coordinate; simultaneously, multiply the obstacle's longitudinal velocity by 0.1 seconds to obtain the longitudinal displacement, and add this longitudinal displacement to the obstacle's current longitudinal coordinate, thus obtaining the obstacle's first nominal predicted position. The second predicted position is based on the first nominal predicted position by adding the lateral and longitudinal displacements corresponding to a control cycle of 0.1 seconds. This process is repeated 25 times to obtain the nominal motion trajectory of the dynamic obstacle within the next 2.5 seconds.

[0077] Considering that dynamic obstacles may change the original speed of movement or take actions to actively approach the robot's movement trajectory, this embodiment determines the single-step position disturbance boundary based on the historical dynamic obstacle trajectory.

[0078] During the dynamic obstacle trajectory acquisition phase, the recorded continuous motion trajectory is fitted with the same five positions and velocities as the online prediction. In each control cycle, the predicted position for the next control cycle is calculated based on the obstacle position and fitted velocity from the previous control cycle. Then, the straight-line distance between the predicted position and the actual detected obstacle center position in the next control cycle is calculated, and the resulting distance is recorded as a single-step prediction residual.

[0079] After obtaining all single-step prediction residuals, the residuals are arranged in ascending order of value, and an upper limit for the residuals is determined that ensures approximately 99% of the historical single-step prediction residuals do not exceed this value. In this embodiment, the upper limit for the single-step prediction residuals obtained from historical dynamic obstacle trajectories is 0.035 meters.

[0080] The position detection error of the LiDAR is further factored in. In this embodiment, repeated measurements are performed on a fixed reference target within the robot's actual working distance. By comparing the cluster center position calculated by the LiDAR with the actual position of the reference target, the maximum position error of the obstacle cluster center is found to be 0.015 meters. The upper bound of the single-step prediction residual of 0.035 meters is added to the position detection error of 0.015 meters to obtain 0.05 meters, which is then determined as the single-step position disturbance radius of the dynamic obstacle within a 0.1-second control cycle. The basic geometric envelope of the dynamic obstacle is a circular region with a radius of 0.35 meters centered on the nominal prediction center of the obstacle. At the first prediction moment, with the first nominal prediction position as the center, a Minkowski sum operation is performed on the basic geometric envelope with a radius of 0.35 meters and the single-step disturbance region with a radius of 0.05 meters. Since both regions are circular, the radius of the region obtained after the Minkowski sum operation is equal to the sum of 0.35 meters and 0.05 meters.

[0081] At the second prediction time, considering the potential continuous positional disturbances that may occur within two control cycles, two single-step disturbance radii are accumulated based on the 0.35-meter basic geometric envelope. Subsequent prediction times accumulate the corresponding single-step disturbance range according to the number of prediction steps. Therefore, the space potentially occupied by obstacles gradually expands as the prediction time increases.

[0082] To cover trajectories where the error accumulation rate during continuous multi-step prediction exceeds the linear accumulation rate of a single step, a robust amplification factor is further added to the accumulated disturbance. This robust amplification factor is determined using historical multi-step prediction trajectories.

[0083] Specifically, for historical dynamic obstacle trajectories, nominal position predictions are performed from prediction step 1 to prediction step 25. In each prediction step, the straight-line distance between the actual obstacle position and the corresponding nominal predicted position is calculated; then, this straight-line distance is divided by the product of the current prediction step number and 0.05 meters to obtain the error amplification ratio corresponding to the historical predicted position.

[0084] All historical error amplification ratios are arranged in ascending order of value, and the amplification ratio used to construct the forward reachable area is gradually increased until at least 99.5% of the actual obstacle locations in the historical prediction data are within the correspondingly expanded prediction area. The minimum amplification ratio that meets the above coverage requirement is used as the basis for determining the robust amplification coefficient. In this embodiment, the minimum amplification ratio that meets the above coverage requirement is calculated to be 1.18. To avoid parameter truncation reducing the coverage of the forward reachable area, this value is adjusted upward to 1.2, and 1.2 is used as the robust amplification coefficient in the online control process.

[0085] In actual control, the disturbance propagation distance corresponding to a given prediction step is obtained by multiplying the prediction step number, the single-step position disturbance radius of 0.05 meters, and the robust amplification factor of 1.2 in sequence. For example, the disturbance propagation distance for the 5th prediction step is obtained by multiplying 5 by 0.05 meters and then by 1.2; the distance for the 10th prediction step is obtained by multiplying 10 by 0.05 meters and then by 1.2.

[0086] Subsequently, the perturbation expansion distance corresponding to each prediction step is added to the basic geometric envelope radius of the dynamic obstacle (0.35 meters) to obtain the forward reachable region of the dynamic obstacle at the corresponding prediction time. This process is repeated continuously from the first prediction step to the 25th prediction step, thus obtaining the forward reachable tube of the dynamic obstacle covering the entire 2.5-second prediction time domain. After forming the forward reachable tube of the dynamic obstacle, the robot's candidate control actions are safely verified. In this embodiment, 11 linear velocities are selected at intervals of 0.1 meters per second within the linear velocity range of 0 to 1.0 meters per second, and 13 angular velocities are selected at intervals of 0.2 radians per second within the angular velocity range of -1.2 to 1.2 radians per second, forming a total of 143 sets of candidate control actions.

[0087] For each set of candidate control actions, the robot's current local position and current heading are used as the prediction starting point, and the motion state is continuously recursively calculated 25 times. In each recursion, the cosine value of the current robot heading angle is first calculated, and then the cosine value is multiplied by the candidate linear velocity and the 0.1-second control cycle in sequence. The result is used as the longitudinal displacement generated by the robot in the current prediction step, and the longitudinal displacement is added to the longitudinal coordinate of the previous prediction step.

[0088] The sine of the current robot heading angle is then calculated. This sine is multiplied sequentially by the candidate linear velocity and the 0.1-second control cycle. The result is used as the lateral displacement of the robot in the current prediction step, and this lateral displacement is added to the lateral coordinate of the previous prediction step. Simultaneously, the candidate angular velocity is multiplied by 0.1 seconds, and the result is used as the heading angle change in the current prediction step, which is added to the heading angle of the previous prediction step. After executing this process 25 times, the complete robot prediction trajectory corresponding to the candidate control action within the next 2.5 seconds is obtained.

[0089] For the robot's first predicted position, the straight-line distance between this position and the first nominal predicted center of the dynamic obstacle is calculated, along with the minimum permissible center distance corresponding to this prediction moment. The minimum permissible center distance is obtained by adding the robot radius of 0.30 meters, the basic safety buffer distance of 0.15 meters, the basic geometric envelope radius of the dynamic obstacle of 0.35 meters, and the disturbance expansion distance corresponding to the first prediction step. The same process is used for the robot's second to 25th predicted positions, calculating the distance to the nominal predicted center of the dynamic obstacle at the corresponding moment, and comparing the obtained actual center distance with the minimum permissible center distance corresponding to that prediction step.

[0090] A candidate control action is added to the safe action set only if all 25 predicted robot positions corresponding to that action satisfy the condition that the actual center distance is not less than the corresponding minimum allowable center distance. If any one of the predicted steps fails to satisfy the corresponding minimum allowable center distance, the candidate control action is excluded from the safe action set. Taking the 8th predicted step as an example, first, multiply the predicted step number 8 by the single-step position disturbance radius of 0.05 meters to obtain 0.40 meters; then multiply 0.40 meters by the robust amplification factor of 1.2 to obtain 0.48 meters, which is the disturbance expansion distance corresponding to the 8th predicted step. The robot radius of 0.30 meters, the basic safety buffer distance of 0.15 meters, the obstacle's basic geometric envelope radius of 0.35 meters, and the disturbance expansion distance of 0.48 meters are added sequentially to obtain the minimum allowable center distance of 1.28 meters for the 8th predicted step. Therefore, the actual distance between the robot's 8th predicted position and the 8th nominal predicted center of the dynamic obstacle must be no less than 1.28 meters. After the human-machine adversarial intent quantification and fusion module generates the fused control action, the fused control action is first subjected to a complete safety verification according to the above 25-step trajectory prediction process. When all 25 robot prediction positions corresponding to the fused control action meet the corresponding minimum allowable center distance, the fused control action remains unchanged.

[0091] When any predicted position corresponding to a fused control action enters the forward reachable area of ​​a dynamic obstacle, the fused control action is determined to be an unsafe action, and a new control action is selected from the current set of safe actions. The principle of minimum correction is used when reselecting a control action. For each candidate control action in the set of safe actions, first, the linear velocity of the candidate control action is subtracted from the linear velocity of the original fused control action, and then the difference is squared. Subsequently, the angular velocity of the candidate control action is subtracted from the angular velocity of the original fused control action, and the difference is squared.

[0092] In this embodiment, the total width of the allowable range for linear velocity is 1.0 m / s, therefore the squared difference of linear velocity is directly used as the linear velocity correction cost. The total width of the allowable range for angular velocity is 2.4 radians / s. To eliminate the projection bias caused by the difference in range between linear velocity and angular velocity, 2.4 is squared to get 5.76, and then 1 is divided by 5.76 to get approximately 0.174. Multiplying 0.174 by the squared difference of angular velocity yields the normalized angular velocity correction cost.

[0093] The linear velocity correction cost is added to the normalized angular velocity correction cost to obtain the total correction cost of the candidate control action relative to the original fused control action. The controller sequentially calculates the total correction cost of all candidate control actions within the safety action set and selects the candidate control action with the minimum total correction cost as the safety control command.

[0094] In one control cycle of this embodiment, the target linear velocity output by the human-machine fusion module is 0.8 meters per second, and the target angular velocity is -0.1 radians per second. After predicting the trajectory of the target control action in 25 steps, the robot's predicted position enters the forward reachable area of ​​the dynamic obstacle in the 8th prediction step. Therefore, it is determined that the target control action does not meet the tubular avoidance requirements.

[0095] Of the 143 candidate control actions, 17 groups have all 25 predicted positions that satisfy the corresponding minimum allowable center distance, thus forming a set of 17 safe actions. The controller calculates the linear velocity correction cost and angular velocity correction cost of each of the 17 safe candidate control actions relative to the original target control action, adds the two correction costs, and selects the candidate control action with the minimum total correction cost as the safe control command. After the safe control command is selected, it is not directly sent to the robot actuator, but is re-verified according to the same 25-step trajectory prediction process. Only when all 25 predicted positions satisfy the corresponding tubular avoidance requirements during the re-verification process is the safe control command converted into a robot-level drive control signal.

[0096] When none of the 143 candidate control actions meet all the requirements for tubular obstacle avoidance, the safe action set is empty, and the controller enters the emergency deceleration process. In this embodiment, a stable braking deceleration of 1.2 meters per second squared was determined through robot chassis braking experiments. Since the control cycle is 0.1 seconds, the current linear velocity command is reduced by 0.12 meters per second after each control cycle, and the robot's linear velocity is continuously reduced at the same deceleration rate until it reaches 0. During the emergency deceleration process, the lidar continues to update the dynamic obstacle position according to the control cycle, and the controller synchronously recalculates the dynamic obstacle's movement speed, single-step disturbance range, and forward reachable tubes, and regenerates the safe action set.

[0097] When the robot needs to change its heading during deceleration, candidate turning trajectories to the left and right of the robot's current heading are predicted, and the minimum distance between the candidate trajectories on both sides and the forward reachable area of ​​the dynamic obstacle is calculated. The minimum distance on the left and the minimum distance on the right are compared, and the side with the larger minimum distance is selected as the robot's turning direction. In the next control cycle, the safety action selection continues according to the updated environmental state.

[0098] Through the above implementation process, the single-step position disturbance range of dynamic obstacles is determined jointly by the historical prediction residual and the lidar position detection error. The robust amplification coefficient is determined by the actual coverage result of the historical multi-step predicted trajectory. The forward reachable area of ​​each predicted step is formed by the obstacle's basic geometric envelope, cumulative position disturbance, and robust amplification coefficient. The robot's safe distance further incorporates the robot's own radius and basic safety buffer distance, ensuring that the collision determination for each predicted step has a clear calculation basis.

[0099] Meanwhile, when the human-machine fusion control action does not meet the forward reachability tube avoidance requirements, the linear velocity correction cost and angular velocity correction cost are calculated one by one for the candidate control actions in the set of safety actions. The control action with the smallest deviation from the original fusion control action and that satisfies all predicted safety constraints is obtained, thereby completing the safety constraint correction while maintaining the original human-machine control intention.

[0100] This invention is not limited to the embodiments described above. Any changes in shape or structure shall fall within the protection scope of this invention. The protection scope of this invention is defined by the appended claims. Those skilled in the art may make various changes or modifications to these embodiments without departing from the principles and essence of this invention, but all such changes and modifications shall fall within the protection scope of this invention.

Claims

1. A human-machine collaborative adversarial control optimization system for complex dynamic environments, characterized in that, include: The environmental state perception module is used to acquire the motion state of the mobile robot and the state of environmental obstacles and generate an environmental state vector. The autonomous decision-making module is used to generate autonomous control commands based on the environmental state vector; The manual instruction acquisition module is used to acquire manual control instructions input by the operator. The machine confidence assessment module is used to determine machine confidence based on the cognitive uncertainty of autonomous decision-making and the feasibility of safe actions in the environment; The human-machine adversarial intent quantification and fusion module is used to construct a return probability distribution through a distributed value assessment network, calculate the conditional risk value (CVaR) of human control instructions, determine the adversarial divergence based on the conditional risk value and autonomous control instructions, determine the basic weight of human control instructions through a cost function that includes the cost of intervention and the cost of control risk, couple the basic weight with the machine confidence to obtain the fusion weight, and fuse human control instructions and autonomous control instructions to form a fused control instruction. The robust model predictive control safety verification module is used to establish a dynamic obstacle prediction model that includes nominal motion state and bounded disturbance, and to construct a forward reachable tube for the dynamic obstacle in the prediction time domain. Based on the forward reachable tube, tubular avoidance constraints are applied to the robot's predicted trajectory to form a set of safe actions. When the control command to be executed does not meet the tubular avoidance constraints, it is projected onto the set of safe actions to obtain a safe control command. The action execution and status update module is used to execute the safety control commands and update the environmental status to form a closed-loop control.

2. The human-machine collaborative adversarial control optimization system for complex dynamic environments according to claim 1, characterized in that, The environmental state perception module includes a lidar, an inertial measurement unit, and a wheeled odometer. The environmental state vector includes at least the spatial distribution information of obstacles relative to the mobile robot, the current linear velocity of the mobile robot, and the current angular velocity of the mobile robot. The autonomous decision-making module uses the Twin Delayed Deep Deterministic Policy Gradient algorithm to construct a deep reinforcement learning policy network and generates the autonomous control instructions based on the environmental state vector. The autonomous decision-making module and the manual instruction acquisition module operate in parallel to obtain the autonomous control instruction and the manual control instruction, respectively.

3. The human-machine collaborative adversarial control optimization system for complex dynamic environments according to claim 1, characterized in that, The machine confidence assessment module is configured as follows: During the inference process of the deep reinforcement learning policy network, the random deactivation mechanism is kept on, and forward propagation with different random masks is performed multiple times on the same environment state vector to obtain multiple action samples. Calculate the sample variance of the multiple action samples in each dimension of the action space, and determine the cognitive uncertainty index based on the sample variance of each dimension; The cognitive uncertainty index is converted into Bayesian confidence level using a nonlinear mapping function; Within the preset prediction time domain, trajectory prediction is performed on multiple candidate actions, and the number of safe actions that meet the collision avoidance conditions is counted. The machine confidence is determined based on the environmental geometric feasibility represented by the number of safety actions and the Bayesian confidence, so that the machine confidence simultaneously represents the reliability of the deep reinforcement learning policy network's cognition of the current environment and the safe operating space in the current environment.

4. The human-machine collaborative adversarial control optimization system for complex dynamic environments according to claim 1, characterized in that, The distributed value assessment network is used to divide the return probability space into multiple quantile intervals and output multiple return prediction values ​​corresponding to different quantile positions. Based on the multiple return prediction values, a discrete return probability distribution corresponding to the state and action pair is constructed. The human-machine adversarial intent quantification and fusion module is configured to calculate the conditional value of risk (CVaR) based on the reward probability distribution corresponding to the human control command at a preset risk confidence level, so as to characterize the tail risk of the human control command in the case of low reward, and construct the adversarial divergence based on the autonomous control command and the conditional value of risk (CVaR), so that the adversarial divergence increases as the tail risk of the human control command increases.

5. The human-machine collaborative adversarial control optimization system for complex dynamic environments according to claim 4, characterized in that, The cost function includes a control intervention cost that characterizes the degree of intervention caused by reducing human control, and a control risk cost that characterizes the degree of risk caused by executing human control instructions. The human-machine adversarial intent quantification and fusion module obtains the basic weights corresponding to the human control commands by minimizing the cost function, and dynamically adjusts the basic weights according to the machine confidence to obtain the fused weights of the human control commands. The higher the machine confidence and / or the greater the adversarial divergence, the lower the fusion weight of the manual control command, and the manual control command and the autonomous control command are weighted and fused based on the fusion weight.

6. The human-machine collaborative adversarial control optimization system for complex dynamic environments according to claim 1, characterized in that, The robust model predictive control safety verification module is configured as follows: Based on the distance information obtained by the lidar and the corresponding scanning angle, a set of obstacle points in the local coordinate system of the mobile robot is constructed. The collision detection threshold is determined based on the robot radius of the mobile robot and the preset safety buffer distance; A differential-driven discrete kinematics model is used to perform forward integration of candidate control commands in the prediction time domain according to a preset time step to generate the corresponding robot prediction trajectory. Calculate the minimum distance between the robot's predicted trajectory and the set of obstacle points, and determine whether the corresponding candidate control command is safe based on the minimum distance and the collision determination threshold; The candidate control commands are obtained by sampling in the online velocity action space and angular velocity action space, and a set of safe actions is formed based on the safety judgment results of the candidate control commands.

7. The human-machine collaborative adversarial control optimization system for complex dynamic environments according to claim 1, characterized in that, The dynamic obstacle prediction model represents the motion state of a dynamic obstacle as a combination of a nominal motion state and a bounded perturbation. The nominal motion state is predicted based on the currently observed motion state of the dynamic obstacle, and the bounded perturbation is used to characterize the adversarial motion of unmodeled dynamics and / or dynamic obstacles relative to the mobile robot. The robust model predictive control safety verification module obtains the forward reachability tube by means of Minkowski algorithm and recursion over multiple consecutive prediction steps, based on the basic geometric envelope of the dynamic obstacle, the nominal motion state, and the bounded perturbation, so that the forward reachability tube covers the possible space occupied by the dynamic obstacle within the corresponding prediction step.

8. The human-machine collaborative adversarial control optimization system for complex dynamic environments according to claim 7, characterized in that, The robust model predictive control safety verification module transforms the minimax safety constraint for the bounded perturbation into a geometric space avoidance constraint for the forward reachable tube through constraint tightening. The geometric space avoidance constraint is used to limit the mobile robot to maintain a safe distance between the predicted trajectory of the mobile robot in the prediction time domain and the forward reachable tube. The safe distance is determined based on the maximum permissible disturbance of the dynamic obstacle and the preset adversarial robust amplification factor. The candidate control command is added to the set of safe actions only if the robot's predicted trajectory corresponding to the candidate control command satisfies the geometric space avoidance constraint in the prediction time domain.

9. The human-machine collaborative adversarial control optimization system for complex dynamic environments according to any one of claims 1 or 6 to 8, characterized in that, The robust model predictive control safety verification module performs safety verification before and after control command fusion. When the control instruction to be executed belongs to the set of safety actions, the control instruction to be executed is directly determined as a safety control instruction; When the control instruction to be executed does not belong to the set of safety actions, the candidate control instruction with the smallest weighted deviation from the control instruction to be executed is selected from the set of safety actions according to the principle of minimum intervention. The cost of correcting linear velocity and angular velocity is limited by weight matrix. When the set of safety actions is empty, emergency obstacle avoidance control, including deceleration, steering, and / or manual prompts, is triggered.

10. The human-machine collaborative adversarial control optimization system for complex dynamic environments according to claim 1, characterized in that, The action execution and status update module is used to convert the safety control command into motor control signals corresponding to the underlying drive mechanism of the mobile robot; After the mobile robot executes the safety control command, the robot's pose state and the obstacle state in the environment are reacquired, and the environmental state vector is updated based on the reacquired pose state and obstacle state. The environmental state vector is then executed cyclically according to the control cycle, including environmental state perception, autonomous control command generation, machine confidence assessment, human-machine adversarial intent quantification, control command fusion, robust safety verification, action execution, and state update.

Images