Risk-adaptive robot hierarchical social navigation method and system

Abstract

The invention provides a risk-adaptive robot hierarchical social navigation method and system, and relates to the technical field of strategy optimization, and the method comprises the steps: determining a speed fluctuation index and an acceleration fluctuation index based on the speed vector data of a pedestrian sample; performing weighted fusion on the speed fluctuation index and the acceleration fluctuation index to obtain an original uncertainty score, performing normalization on the original uncertainty score, and performing linear mapping to a preset unpredictability interval to obtain an unpredictability score; taking the robot state data sample, the pedestrian state data sample and the unpredictability score as input, taking a preset navigation point of the robot, an expected cruising speed of the robot and a prudent coefficient as output, and performing iterative training and optimization on the initial prediction network model in combination with near-end strategy optimization to obtain a target prediction network model; and inputting the current robot state data, the pedestrian state data and the unpredictability score into the target prediction network model, and determining a current navigation strategy.

Description

Technical Field

[0001] This invention relates to the field of strategy optimization technology, and in particular to a risk-adaptive hierarchical social navigation method and system for robots. Background Technology

[0002] With the increasing use of mobile robots in public places such as shopping malls, hospitals, and airports, robots need to navigate autonomously in dynamic crowds. Existing navigation methods can be mainly divided into three categories: reaction-based methods, deep reinforcement learning-based methods, and optimization-based methods. Reaction-based methods, such as ORCA (Optimal Reciprocal Collision Avoidance) and the Social Force model, have the advantage of high computational efficiency, but lack understanding of pedestrian intentions and are prone to freezing in high-density, complex interactions. Deep reinforcement learning-based methods, such as CADRL (Collision Avoidance with Deep Reinforcement Learning), learn end-to-end navigation strategies through simulation and can handle complex interactions. However, these methods are usually black-box models, and their internal decision-making logic is difficult to interpret. Some recent works have attempted to improve robustness by constructing high-risk scenarios and introducing uncertainty-related rewards or constraints, but inherent drawbacks remain: their risk response mechanisms are still implicit within the neural network, lacking interpretability, verifiability, and controllability, making them unsuitable for robot applications with high safety requirements. Optimization-based methods, such as Learning-MPC (Multi-Learning Dynamics Residual or Cost Correction) or high-level output trajectories / weights followed by MPC, cannot directly map risk information to the MPC's safety radius or obstacle cost weights, lacking a structured "hierarchical risk adjustment mechanism." These shortcomings of existing navigation methods result in insufficient safety and unclear risk adjustment mechanisms for robots navigating dynamic crowds. Summary of the Invention

[0003] In view of this, the present invention proposes a risk-adaptive hierarchical social navigation method and system for robots.

[0004] The technical solution of this invention is implemented as follows: The first aspect of this invention provides a risk-adaptive hierarchical social navigation method for robots, comprising: Obtain velocity vector data of pedestrian samples, and determine the velocity fluctuation index and acceleration fluctuation index of the pedestrian samples based on the velocity vector data; The velocity fluctuation index and the acceleration fluctuation index are weighted and fused to obtain the original uncertainty score. The original uncertainty score is normalized and linearly mapped to a preset unpredictability range to obtain an unpredictability score. The robot state data sample, pedestrian state data sample, and the unpredictability score are used as input data. The robot's preset navigation point, the robot's expected cruising speed, and the caution coefficient are used as output data. The initial prediction network model is iteratively trained and optimized by combining near-end policy optimization to obtain the target prediction network model. The caution coefficient represents the overall conservatism of the robot in dealing with the current scenario. The initial prediction network model includes a policy network, a prediction network, and a value network. The current robot state data, pedestrian state data, and unpredictability score are input into the target prediction network model to determine the current navigation strategy.

[0005] Based on the above technical solution, preferably, the velocity vector data includes a velocity mode sequence and an acceleration mode sequence; the step of determining the velocity fluctuation index and acceleration fluctuation index of the pedestrian sample based on the velocity vector data includes: Obtain the first standard deviation of the velocity vector and the second standard deviation of the acceleration vector, and determine the first standard deviation and the second standard deviation as the velocity fluctuation index and the acceleration fluctuation index, respectively.

[0006] Based on the above technical solutions, preferably, the policy network includes an Actor-Critic network with a shared encoder; the process of taking robot state data samples, pedestrian state data samples, and the unpredictability score as input data, and the robot's preset navigation point, the robot's expected cruising speed, and caution coefficient as output data, and iteratively training and optimizing the initial prediction network model in conjunction with near-end policy optimization to obtain the target prediction network model includes: The pedestrian state data samples and the unpredictability score are encoded to obtain a normalized feature vector; The normalized feature vector is input into the Actor-Critic network to obtain Gaussian distribution features, and the original three-dimensional motion is sampled from the Gaussian features; The features of each dimension of the original three-dimensional motion are processed by nonlinear compression and interval mapping to obtain the sub-target direction, velocity preference and caution coefficient. The sub-target direction, velocity preference and caution coefficient are then passed as parameterized interfaces to the prediction network for solving, thereby determining the network parameters of the prediction network.

[0007] Based on the above technical solutions, preferably, after determining the network parameters of the prediction network, the method further includes: Obtain the corresponding reward parameters and termination flag from the sample environment, and acquire the logarithmic probability and value estimate of the original three-dimensional action; The reward parameters, the termination flag, the log probability, the value estimate, and the normalized feature vector are written into the trajectory buffer. When a trajectory ends or the buffer is full, the network parameters of the prediction network are optimized by combining the generalized advantage estimation algorithm.

[0008] Based on the above technical solutions, preferably, the iterative training and optimization of the initial prediction network model combined with near-end policy optimization includes: The input data and the output data are divided into multiple data subsets. In each iteration, the network parameters of the prediction network are optimized using different data subsets.

[0009] Based on the above technical solution, preferably, the step of optimizing the network parameters of the prediction network using different data subsets in each iteration further includes: In each iteration, the network parameters of the prediction network are optimized using different subsets of data. This is combined with minimizing the value function loss and adding an entropy regularization term to adjust the network parameters of the prediction network until the KL divergence exceeds a first preset threshold, at which point the iteration stops.

[0010] Based on the above technical solutions, preferably, the iterative training and optimization of the initial prediction network model combined with near-end policy optimization further includes: If the unpredictability score is less than the second preset threshold, the obstacle avoidance cost function is weighted using the first caution coefficient to adjust the network parameters of the prediction network; the obstacle avoidance cost function monotonically increases with risk. If the unpredictability score is not less than the second preset threshold, the obstacle avoidance cost function is weighted using a second caution coefficient to adjust the network parameters of the prediction network; the second caution coefficient is greater than the first caution coefficient.

[0011] Furthermore, a second aspect of the present invention provides a risk-adaptive hierarchical social navigation system for robots, comprising: a data acquisition module, a scoring acquisition module, a training and optimization module, and a strategy determination module; wherein, The data acquisition module is configured to acquire velocity vector data of pedestrian samples and determine the velocity fluctuation index and acceleration fluctuation index of the pedestrian samples based on the velocity vector data. The scoring acquisition module is configured to perform weighted fusion of the velocity fluctuation index and the acceleration fluctuation index to obtain an original uncertainty score, normalize the original uncertainty score, and linearly map it to a preset unpredictability range to obtain an unpredictability score. The training optimization module is configured to take robot state data samples, pedestrian state data samples, and the unpredictability score as input data, and the robot's preset navigation point, the robot's expected cruising speed, and caution coefficient as output data. It iteratively trains and optimizes the initial prediction network model in conjunction with near-end policy optimization to obtain the target prediction network model. The caution coefficient represents the overall conservative degree of the robot in dealing with the current scenario. The initial prediction network model includes a policy network, a prediction network, and a value network. The strategy determination module is configured to input the current robot state data, pedestrian state data, and unpredictability score into the target prediction network model to determine the current navigation strategy.

[0012] More preferably, a third aspect of the present invention provides an electronic device including a processor and a memory; the memory has a computer program stored therein, wherein the computer program, when executed by the processor, implements the risk-adaptive robot hierarchical social navigation method described in the first aspect.

[0013] More preferably, a fourth aspect of the present invention provides a non-transitory computer storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the risk-adaptive robot hierarchical social navigation method described in the first aspect.

[0014] The risk-adaptive hierarchical social navigation method and system for robots of the present invention have the following advantages over the prior art: 1. By extracting the temporal features of pedestrian velocity vectors, the randomness and unpredictability of pedestrian motion are directly quantified. Based on this, velocity fluctuation indicators and acceleration fluctuation indicators are weighted, fused, and normalized, then mapped to a preset interval to form a continuous and interpretable risk indicator that directly reflects the complexity of pedestrian behavior. Furthermore, robot state, pedestrian state, and unpredictability score are input into the model, covering three risk sources: static environment, dynamic obstacles, and behavioral uncertainty. Combined with dynamically adjusted caution coefficients and reinforcement learning optimization strategies, quantitative modeling and dynamic risk perception of pedestrian behavior uncertainty are achieved, significantly improving the robot's safety and adaptability in complex dynamic scenarios.

[0015] 2. By encoding pedestrian states and unpredictability scores into normalized feature vectors, the dimensionality of data with different scales and distributions is unified. The encoding process can automatically learn the nonlinear correlation between pedestrian states and unpredictability, extracting high-order features. The Actor network outputs the mean and variance of a Gaussian distribution, generating 3D raw actions through sampling, avoiding the policy from getting trapped in local optima, and allowing for the discovery of better actions during the training phase. In addition, nonlinear compression and interval mapping restrict actions to a safe range, ensuring navigation safety.

[0016] 3. Reward parameters, termination flags, log probabilities, value estimates, and normalized feature vectors are written into the trajectory buffer. Network parameters are updated in batches when the trajectory ends or the buffer is full. Batch updates utilize more data to calculate gradients, improving the stability of model parameter updates. GAE combined with Monte Carlo estimation generates a more accurate advantage function estimate, measuring the superiority of actions relative to the average policy, guiding policy updates, reducing gradient estimation errors, and making policy updates more stable. Combining minimizing the value function loss and entropy regularization to enhance exploration improves policy evaluation accuracy and training stability, preventing the policy from prematurely getting trapped in local optima. Attached Figure Description

[0017] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0018] Figure 1 A flowchart illustrating a risk-adaptive layered social navigation method for robots provided in an embodiment of the present invention; Figure 2 This is a schematic diagram of the structure of a risk-adaptive robot hierarchical social navigation system provided in an embodiment of the present invention; Figure 3 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention. Detailed Implementation

[0019] The technical solutions of the present invention will be clearly and completely described below with reference to the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present invention.

[0020] In some embodiments, such as Figure 1 As shown,Figure 1 This is a flowchart illustrating a risk-adaptive layered social navigation method for robots provided in an embodiment of the present invention; the risk-adaptive layered social navigation method for robots provided by the present invention includes: S110: Obtain the velocity vector data of the pedestrian samples, and determine the velocity fluctuation index and acceleration fluctuation index of the pedestrian samples based on the velocity vector data.

[0021] Here, pedestrians can be detected using devices such as LiDAR, visual sensors, and inertial measurement units to obtain velocity vector data of pedestrian samples. Velocity vector data includes velocity, acceleration, velocity direction, and acceleration direction. Velocity fluctuation indicators characterize the degree of change in pedestrian speed, while acceleration fluctuation indicators characterize the severity of changes in direction or the start / stop of pedestrian movement.

[0022] In some embodiments, the velocity vector data includes a velocity magnitude sequence and an acceleration magnitude sequence; determining the velocity fluctuation index and acceleration fluctuation index of the pedestrian sample based on the velocity vector data includes: Obtain the first standard deviation of the velocity vector and the second standard deviation of the acceleration vector, and determine the first standard deviation and the second standard deviation as the velocity fluctuation index and the acceleration fluctuation index, respectively.

[0023] For example, obtain the first in the scenario A pedestrian recently A sequence of velocity vectors at each sampling time point. Let be... For the current moment, the set of historical velocity vectors can be represented as:

[0024] ; Each velocity vector can be represented as: ; Based on the velocity vector sequence, the velocity mode sequence and acceleration mode sequence of the pedestrian are calculated respectively to characterize the intensity of the motion. The velocity modulus sequence can be represented as the Euclidean norm of the velocity vector at each time step: ; The acceleration modulus sequence can be represented as the Euclidean norm of the velocity vector difference between adjacent time steps: ; Speed ​​fluctuation index For velocity mode sequence Standard deviation, acceleration fluctuation index For acceleration mode sequence The standard deviation.

[0025] S120 performs weighted fusion of velocity fluctuation index and acceleration fluctuation index to obtain the original uncertainty score, normalizes the original uncertainty score, and linearly maps it to the preset unpredictability interval to obtain the unpredictability score.

[0026] In this embodiment, a weighted summation method is used to fuse the velocity fluctuation index and the acceleration fluctuation index to obtain the original uncertainty score: ; in, This is the acceleration fluctuation weighting coefficient.

[0027] Limit the original uncertainty score to Within the range, the normalized intermediate value is obtained. : ; Linearly map the median value to a preset unpredictability range: ; Here, the lower limit of the pre-defined unpredictability range is... Preset upper limit The final output Used to adjust the security constraints of downstream model prediction networks.

[0028] S130: The robot state data sample, pedestrian state data sample, and unpredictability score are used as input data. The robot's preset navigation point, the robot's expected cruising speed, and caution coefficient are used as output data. The initial prediction network model is iteratively trained and optimized by combining near-end policy optimization to obtain the target prediction network model. The caution coefficient represents the overall conservatism of the robot in dealing with the current scenario. The initial prediction network model includes a policy network, a prediction network, and a value network.

[0029] In some embodiments, the policy network includes an Actor-Critic network with a shared encoder; taking robot state data samples, pedestrian state data samples, and unpredictability scores as input data, and the robot's preset navigation point, the robot's expected cruising speed, and caution coefficient as output data, the initial prediction network model is iteratively trained and optimized using near-end policy optimization to obtain a target prediction network model, including: The pedestrian state data samples and unpredictability scores are encoded to obtain normalized feature vectors; The normalized feature vector is input into the Actor-Critic network to obtain Gaussian distribution features, and the original 3D motion is sampled from the Gaussian features. The features of each dimension of the original 3D motion are processed by nonlinear compression and interval mapping to obtain the sub-target direction, velocity preference and caution coefficient. The sub-target direction, velocity preference and caution coefficient are then passed as parameterized interfaces to the prediction network for solving, thereby determining the network parameters of the prediction network.

[0030] In this embodiment, the caution factor It can be based on normalized scene feature vectors by a policy network. The calculation shows that its operational logic satisfies: ; in, A pre-defined deep neural network encoder is used to extract latent space features of the scene; This is the normalized mapping function; and These are the learning parameters for the output layer of the policy network; and The preset lower and upper limits of the dynamic adjustment range of the caution coefficient.

[0031] Constructing an Actor-Critic Network for a Shared Encoder: Encoder The output three-dimensional motion mean is , The output state value is and maintain learnable log-variance. To form a Gaussian policy distribution; using As an optimizer.

[0032] By encoding pedestrian state data samples and unpredictability scores, a normalized scene feature vector is obtained. , and thus enter Gaussian distribution is obtained During the training phase, 3D raw motion is sampled from the distribution. Simultaneously record the logarithmic probability of this action. With value estimation .Will Each dimension is nonlinearly compressed and mapped to obtain the sub-target direction, velocity preference, and caution coefficient. These parameters are then passed as a parameterized interface to the prediction network to determine its network parameters. During training, to ensure that each step of the higher-level layer generates a new decision, the higher-level layer is forcibly refreshed to maintain the timer.

[0033] In some embodiments, after determining the network parameters of the prediction network, the method further includes: Obtain the corresponding reward parameters and termination flags from the sample environment, and acquire the logarithmic probability and value estimate of the original three-dimensional action; The reward parameters, termination flag, log probability, value estimate, and normalized feature vector are written into the trajectory buffer. When a trajectory ends or the buffer is full, the network parameters of the prediction network are optimized by combining the generalized advantage estimation algorithm.

[0034] In some embodiments, iterative training and optimization of the initial prediction network model in conjunction with proximal policy optimization includes: The input and output data are divided into multiple data subsets, and the network parameters of the prediction network are optimized using different data subsets in each iteration.

[0035] In some embodiments, the network parameters of the prediction network are optimized using different subsets of data in each iteration, further comprising: In each iteration, the network parameters of the prediction network are optimized using different subsets of data. This is combined with minimizing the loss of the value function and adding an entropy regularization term to adjust the network parameters of the prediction network until the KL divergence exceeds a first preset threshold, at which point the iteration stops.

[0036] In some embodiments, iterative training and optimization of the initial prediction network model in conjunction with proximal policy optimization further includes: When the unpredictability score is less than the second preset threshold, the obstacle avoidance cost function is weighted using the first caution coefficient to adjust the network parameters of the prediction network; the obstacle avoidance cost function monotonically increases with risk. If the unpredictability score is not less than the second preset threshold, the obstacle avoidance cost function is weighted using the second caution coefficient to adjust the network parameters of the prediction network; the second caution coefficient is greater than the first caution coefficient.

[0037] In this embodiment, a nonlinear model is constructed to solve for the optimal control sequence within a finite time domain. Within each prediction time domain, the optimization objective of the prediction network consists of a weighted sum of the following components: target deviation cost, velocity deviation cost, and risk-adjusted obstacle avoidance cost. The target deviation cost is calculated based on the squared deviation between the current predicted position and a given sub-target at a higher level, multiplied by the target weight, and used to drive the robot trajectory towards the predetermined sub-target. Here, the robot radius, pedestrian radius, and baseline safety distance can be considered to select an appropriate spatial margin. The velocity deviation cost is calculated based on the squared deviation between the robot's actual speed and the desired cruising speed, multiplied by the speed weight, and used to balance safety and traffic efficiency. The risk-adjusted obstacle avoidance cost is constructed for each pedestrian or obstacle within the prediction domain, based on the relative distance between the robot and the obstacle and the safety margin after uncertainty inflation. A caution coefficient output by the higher-level policy is introduced to weight this obstacle avoidance cost, monotonically enhancing the conservatism of the robot's motion when environmental risk increases.

[0038] S140: Input the current robot state data, pedestrian state data, and unpredictability score into the target prediction network model to determine the current navigation strategy.

[0039] Here, the current navigation strategy includes sub-target directions, such as detour waypoints, and speed preferences, such as walking speed and caution coefficient, forming executable navigation instructions.

[0040] In some embodiments, please refer to Figure 2 , Figure 2 This is a schematic diagram of a risk-adaptive hierarchical social navigation system for robots provided in an embodiment of the present invention. The present invention provides a risk-adaptive hierarchical social navigation system 200 for robots, comprising: a data acquisition module 210, a scoring acquisition module 220, a training and optimization module 230, and a strategy determination module 240; wherein,

[0041] The data acquisition module 210 is configured to acquire the velocity vector data of pedestrian samples and determine the velocity fluctuation index and acceleration fluctuation index of pedestrian samples based on the velocity vector data. The scoring acquisition module 220 is configured to perform weighted fusion of velocity fluctuation index and acceleration fluctuation index to obtain the original uncertainty score, normalize the original uncertainty score, and linearly map it to a preset unpredictability range to obtain the unpredictability score. The training and optimization module 230 is configured to take robot state data samples, pedestrian state data samples, and unpredictability scores as input data, and the robot's preset navigation point, the robot's expected cruising speed, and caution coefficient as output data. It iteratively trains and optimizes the initial prediction network model in conjunction with near-end policy optimization to obtain the target prediction network model. The caution coefficient represents the overall conservatism of the robot in dealing with the current scenario. The initial prediction network model includes a policy network, a prediction network, and a value network. The strategy determination module 240 is configured to input the current robot state data, pedestrian state data and unpredictability score into the target prediction network model to determine the current navigation strategy.

[0042] In some embodiments, the velocity vector data includes a velocity magnitude sequence and an acceleration magnitude sequence; the data acquisition module 210 is specifically configured as follows: Obtain the first standard deviation of the velocity vector and the second standard deviation of the acceleration vector, and determine the first standard deviation and the second standard deviation as the velocity fluctuation index and the acceleration fluctuation index, respectively.

[0043] In some embodiments, the policy network includes an Actor-Critic network with a shared encoder; the training optimization module 230 is specifically configured as follows: The pedestrian state data samples and unpredictability scores are encoded to obtain normalized feature vectors; The normalized feature vector is input into the Actor-Critic network to obtain Gaussian distribution features, and the original 3D motion is sampled from the Gaussian features. The features of each dimension of the original 3D motion are processed by nonlinear compression and interval mapping to obtain the sub-target direction, velocity preference and caution coefficient. The sub-target direction, velocity preference and caution coefficient are then passed as parameterized interfaces to the prediction network for solving, thereby determining the network parameters of the prediction network.

[0044] In some embodiments, the training optimization module 230 is further configured as follows: Obtain the corresponding reward parameters and termination flags from the sample environment, and acquire the logarithmic probability and value estimate of the original three-dimensional action; The reward parameters, termination flag, log probability, value estimate, and normalized feature vector are written into the trajectory buffer. When a trajectory ends or the buffer is full, the network parameters of the prediction network are optimized by combining the generalized advantage estimation algorithm.

[0045] In some embodiments, the training optimization module 230 is further configured as follows: The input and output data are divided into multiple data subsets, and the network parameters of the prediction network are optimized using different data subsets in each iteration.

[0046] In some embodiments, the training optimization module 230 is further configured as follows: In each iteration, the network parameters of the prediction network are optimized using different subsets of data. This is combined with minimizing the loss of the value function and adding an entropy regularization term to adjust the network parameters of the prediction network until the KL divergence exceeds a first preset threshold, at which point the iteration stops.

[0047] In some embodiments, the training optimization module 230 is further configured as follows: When the unpredictability score is less than the second preset threshold, the obstacle avoidance cost function is weighted using the first caution coefficient to adjust the network parameters of the prediction network; the obstacle avoidance cost function monotonically increases with risk. If the unpredictability score is not less than the second preset threshold, the obstacle avoidance cost function is weighted using the second caution coefficient to adjust the network parameters of the prediction network; the second caution coefficient is greater than the first caution coefficient.

[0048] It should be noted that the risk-adaptive robot hierarchical social navigation system provided in this application embodiment and the risk-adaptive robot hierarchical social navigation method provided in this application embodiment are based on the same application concept. Therefore, the specific implementation of this embodiment can refer to the implementation of the aforementioned risk-adaptive robot hierarchical social navigation method, and the repeated parts will not be described again.

[0049] In some embodiments, please refer to Figure 3 , Figure 3 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. The electronic device 300 provided in this embodiment includes a processor 310 and a memory 320; the memory 320 stores a computer program, wherein the computer program, when executed by the processor, implements the aforementioned risk-adaptive robot hierarchical social navigation method.

[0050] Specifically, processor 310 may include, for example, a general-purpose microprocessor, an instruction set processor and / or an associated chipset and / or a special-purpose microprocessor (e.g., an application-specific integrated circuit (ASIC)), etc. Processor 310 may also include onboard memory for caching purposes. Processor 310 may be a single processing unit or multiple processing units for performing different actions of the method flow according to embodiments of this application.

[0051] The memory 320 may be any medium capable of containing, storing, transmitting, propagating, or transmitting instructions. For example, the memory 320 may include, but is not limited to, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices, apparatuses, or propagation media. Specific examples of the memory 320 include: magnetic storage devices such as magnetic tape or hard disk drives (HDDs); optical storage devices such as optical discs (CD-ROMs); and may also be random access memory (RAM) or flash memory; and / or wired / wireless communication links.

[0052] This application also provides a non-transitory computer-readable medium storing a computer program that, when executed by a processor, implements the aforementioned risk-adaptive robot hierarchical social navigation method. This computer-readable medium may be included in the device / apparatus / system described in the above embodiments; or it may exist independently and not assembled into that device / apparatus / system. The aforementioned computer-readable medium carries one or more programs, which, when executed, implement the method according to the embodiments of this application.

[0053] According to embodiments of this application, a computer-readable medium may be a computer-readable signal medium or a computer-readable storage medium, or any combination thereof. A computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this application, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. In this application, a computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. Computer-readable signal media can also be any computer-readable medium other than computer-readable storage media, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to: wireless, wired, optical fiber, radio frequency signals, etc., or any suitable combination thereof.

[0054] Those skilled in the art will understand that the features described in the various embodiments and / or claims of this application can be combined and / or combined in various ways, even if such combinations or combinations are not explicitly described in this application. In particular, the features described in the various embodiments and / or claims of this application can be combined and / or combined in various ways without departing from the spirit and teachings of this application. All such combinations and / or combinations fall within the scope of this application. Therefore, the scope of this application should not be limited to the above embodiments, but should be defined not only by the appended claims, but also by their equivalents. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this invention should be included within the protection scope of this invention.

Claims

1. A risk-adaptive hierarchical social navigation method for robots, characterized in that, include: Obtain velocity vector data of pedestrian samples, and determine the velocity fluctuation index and acceleration fluctuation index of the pedestrian samples based on the velocity vector data; The velocity fluctuation index and the acceleration fluctuation index are weighted and fused to obtain the original uncertainty score. The original uncertainty score is normalized and linearly mapped to a preset unpredictability range to obtain an unpredictability score. The robot state data sample, pedestrian state data sample, and the unpredictability score are used as input data. The robot's preset navigation point, the robot's expected cruising speed, and the caution coefficient are used as output data. The initial prediction network model is iteratively trained and optimized by combining near-end policy optimization to obtain the target prediction network model. The caution coefficient represents the overall conservatism of the robot in dealing with the current scenario. The initial prediction network model includes a policy network, a prediction network, and a value network. The current robot state data, pedestrian state data, and unpredictability score are input into the target prediction network model to determine the current navigation strategy.

2. The risk-adaptive hierarchical social navigation method for robots as described in claim 1, characterized in that, The velocity vector data includes a velocity modulus sequence and an acceleration modulus sequence; The step of determining the velocity fluctuation index and acceleration fluctuation index of the pedestrian sample based on the velocity vector data includes: Obtain the first standard deviation of the velocity vector and the second standard deviation of the acceleration vector, and determine the first standard deviation and the second standard deviation as the velocity fluctuation index and the acceleration fluctuation index, respectively.

3. The risk-adaptive hierarchical social navigation method for robots as described in claim 1, characterized in that, The policy network includes an Actor-Critic network with a shared encoder; the robot state data samples, pedestrian state data samples, and the unpredictability score are used as input data, and the robot's preset navigation point, the robot's expected cruising speed, and caution coefficient are used as output data. The initial prediction network model is iteratively trained and optimized using near-end policy optimization to obtain the target prediction network model, which includes: The pedestrian state data samples and the unpredictability score are encoded to obtain a normalized feature vector; The normalized feature vector is input into the Actor-Critic network to obtain Gaussian distribution features, and the original three-dimensional motion is sampled from the Gaussian features; The features of each dimension of the original three-dimensional motion are processed by nonlinear compression and interval mapping to obtain the sub-target direction, velocity preference and caution coefficient. The sub-target direction, velocity preference and caution coefficient are then passed as parameterized interfaces to the prediction network for solving, thereby determining the network parameters of the prediction network.

4. The risk-adaptive hierarchical social navigation method for robots as described in claim 3, characterized in that, After determining the network parameters of the prediction network, the process also includes: Obtain the corresponding reward parameters and termination flag from the sample environment, and acquire the logarithmic probability and value estimate of the original three-dimensional action; The reward parameters, the termination flag, the log probability, the value estimate, and the normalized feature vector are written into the trajectory buffer. When a trajectory ends or the buffer is full, the network parameters of the prediction network are optimized by combining the generalized advantage estimation algorithm.

5. The risk-adaptive hierarchical social navigation method for robots as described in claim 4, characterized in that, The iterative training and optimization of the initial prediction network model by combining proximal policy optimization includes: The input data and the output data are divided into multiple data subsets. In each iteration, the network parameters of the prediction network are optimized using different data subsets.

6. The risk-adaptive hierarchical social navigation method for robots as described in claim 5, characterized in that, The step of optimizing the network parameters of the prediction network using different subsets of data in each iteration also includes: In each iteration, the network parameters of the prediction network are optimized using different subsets of data. This is combined with minimizing the value function loss and adding an entropy regularization term to adjust the network parameters of the prediction network until the KL divergence exceeds a first preset threshold, at which point the iteration stops.

7. The risk-adaptive hierarchical social navigation method for robots as described in claim 1, characterized in that, The iterative training and optimization of the initial prediction network model by combining proximal policy optimization also includes: If the unpredictability score is less than the second preset threshold, the obstacle avoidance cost function is weighted using the first caution coefficient to adjust the network parameters of the prediction network; the obstacle avoidance cost function monotonically increases with risk. If the unpredictability score is not less than the second preset threshold, the obstacle avoidance cost function is weighted using a second caution coefficient to adjust the network parameters of the prediction network; the second caution coefficient is greater than the first caution coefficient.

8. A risk-adaptive hierarchical social navigation system for robots, characterized in that, include: The module comprises a data acquisition module, a scoring acquisition module, a training and optimization module, and a policy determination module; among which, The data acquisition module is configured to acquire velocity vector data of pedestrian samples and determine the velocity fluctuation index and acceleration fluctuation index of the pedestrian samples based on the velocity vector data. The scoring acquisition module is configured to perform weighted fusion of the velocity fluctuation index and the acceleration fluctuation index to obtain an original uncertainty score, normalize the original uncertainty score, and linearly map it to a preset unpredictability range to obtain an unpredictability score. The training optimization module is configured to take robot state data samples, pedestrian state data samples, and the unpredictability score as input data, and the robot's preset navigation point, the robot's expected cruising speed, and caution coefficient as output data. It iteratively trains and optimizes the initial prediction network model in conjunction with near-end policy optimization to obtain the target prediction network model. The caution coefficient represents the overall conservative degree of the robot in dealing with the current scenario. The initial prediction network model includes a policy network, a prediction network, and a value network. The strategy determination module is configured to input the current robot state data, pedestrian state data, and unpredictability score into the target prediction network model to determine the current navigation strategy.

9. An electronic device comprising a processor and a memory; said memory storing a computer program, wherein, When executed by the processor, the computer program implements the risk-adaptive robot hierarchical social navigation method according to any one of claims 1 to 7.

10. A non-transitory computer storage medium, characterized in that, It stores a computer program, wherein the computer program, when executed by a processor, implements the risk-adaptive robot hierarchical social navigation method as described in any one of claims 1 to 7.

Images