Behavior decision method and device, electronic equipment and storage medium

Abstract

The application provides a behavior decision method and device, electronic equipment and a storage medium, wherein the method comprises: determining a current map, and pedestrian trajectories and robot trajectories at future preset continuous time points under each behavior strategy simulated by a forward trajectory; determining trajectory feature vectors under each behavior strategy based on the pedestrian trajectories and robot trajectories under each behavior strategy and the current map; and evaluating each behavior strategy based on an evaluation cost function, a weight vector and the trajectory feature vectors under each behavior strategy to obtain an optimal strategy. The weight vector is obtained by training based on samples composed of sample pedestrian trajectories and sample robot example trajectories, which realizes strategy evaluation of the evaluation cost function according to the trajectory feature vectors and the weight vector, avoids manual adjustment of feature weights of the cost function, and thus improves the adaptability to complex scenes.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of artificial intelligence, and in particular to a behavior decision method and device, electronic equipment and a storage medium. BACKGROUND

[0002] With the development of robot technology, more and more robots work in a scenario coexisting with humans. When the robot moves in the human's life and work place, it needs to adapt to a complex and dynamic scenario and avoid the human's activities. In recent years, some research has proposed a multi-strategy decision model for robots, which simulates the future scenario state forward and designs a cost function to evaluate and select the strategy, having good real-time performance.

[0003] However, the current robot behavior decision method can automatically adjust the action strategy according to the change of the scenario, but the cost function used to evaluate the scenario is relatively simple and does not consider the comfort factors such as the obstacle avoidance distance between the robot and the pedestrian in the environment. In addition, the existing method mainly relies on manual adjustment of the feature weights in the cost function, and has insufficient adaptability to complex scenarios. SUMMARY

[0004] The present application provides a behavior decision method, device, electronic equipment and storage medium to solve the defects of the prior art that the cost function for evaluating the scenario is simple and the weight needs to be manually adjusted, resulting in insufficient adaptability to complex scenarios.

[0005] The present application provides a behavior decision method, comprising:

[0006] determining a current map, and pedestrian trajectories and robot trajectories at future preset continuous multiple time points under each behavior strategy obtained by forward trajectory simulation;

[0007] determining trajectory feature vectors under each behavior strategy based on the pedestrian trajectories and robot trajectories under each behavior strategy and the current map;

[0008] evaluating each behavior strategy based on an evaluation cost function, applying a weight vector and the trajectory feature vectors under each behavior strategy, and obtaining an optimal strategy; the weight vector is obtained by training based on a sample composed of sample pedestrian trajectories and sample robot example trajectories.

[0009] According to the behavior decision method provided by the present application, the trajectory feature vectors under each behavior strategy are determined based on the pedestrian trajectories and robot trajectories under each behavior strategy, comprising:

[0010] determining the minimum distance between the robot and the obstacle within the future preset continuous multiple time points under each behavior strategy based on the robot trajectories under each behavior strategy and the current map;

[0011] Based on the pedestrian trajectory and the robot trajectory under each behavior strategy, determining the minimum distance between the robot and the pedestrian within a plurality of preset consecutive moments in the future under each behavior strategy;

[0012] Determine the distance between the robot and the preset target point under each behavior strategy based on the latest trajectory position and the preset target point position in the robot trajectory under each behavior strategy;

[0013] Based on the robot trajectory and the preset global path under each behavior strategy, determining the maximum deviation distance between the robot and the preset global path under each behavior strategy;

[0014] Based on the minimum distance to obstacles under each behavior strategy, the minimum distance to pedestrians under each behavior strategy, the distance to the preset target point under each behavior strategy, and the maximum deviation distance from the preset global path under each behavior strategy, the trajectory feature vector under each behavior strategy is determined.

[0015] According to a behavior decision-making method provided by the present invention, the training steps of the weight vector are as follows:

[0016] Determining a current weight vector and the sample pair;

[0017] Based on the starting position and the ending position of the sample robot example trajectory in the sample pair, the sample pedestrian trajectory in the sample pair, and the current weight vector, the evaluation cost function is applied to determine the robot planning trajectory;

[0018] Determining a first trajectory feature vector based on the sample robot example trajectory in the sample pair and the sample pedestrian trajectory in the sample pair; and determining a second trajectory feature vector based on the robot planned trajectory and the sample pedestrian trajectory in the sample pair;

[0019] Based on the first trajectory feature vector and the second trajectory feature vector, a gradient of the current weight vector is determined; and based on the gradient, the current weight vector is iteratively adjusted until the current weight vector converges to obtain the weight vector.

[0020] According to a behavior decision method provided by the present invention, the steps of determining pedestrian trajectories under each behavior strategy include:

[0021] Determine the pedestrian trajectory at the current prediction time and under any of the aforementioned behavioral strategies;

[0022] Based on the pedestrian position under any of the strategies at the moment before the current prediction moment, the robot position under any of the strategies at the moment before the current prediction moment, and the current map, the pedestrian trajectory prediction model is applied to perform prediction to obtain the pedestrian position at the current prediction moment under any of the strategies;

[0023] The pedestrian position is added to the pedestrian trajectory under any of the strategies, and the next moment of the current prediction moment is iterated as the current prediction moment until the current prediction moment is the end moment of the preset multiple consecutive moments in the future, thereby obtaining the pedestrian trajectory under any of the strategies.

[0024] According to a behavior decision method provided by the present invention, the steps of determining the robot trajectory under each behavior strategy include:

[0025] Determine the robot trajectory at the current planning moment and under any of the aforementioned behavioral strategies;

[0026] Based on the robot position under any one of the strategies at a moment before the current planning moment and the any one of the strategies, the robot trajectory planning model is applied to perform planning to obtain the robot position at the current planning moment under the any one of the strategies;

[0027] The robot position is added to the robot trajectory under any of the strategies, and the next moment of the current planning moment under any of the strategies is used as the current planning moment under any of the strategies for iteration until the current planning moment is the end moment of the preset multiple consecutive moments in the future, thereby obtaining the robot trajectory under any of the strategies.

[0028] According to a behavior decision-making method provided by the present invention, the example trajectory of the sample robot in the sample pair is based on the motion trajectory recorded by a human-operated robot from a starting position to an end position in a simulation environment; the simulation environment includes the starting position, the end position and the trajectories of multiple autonomously moving pedestrians.

[0029] According to a behavior decision-making method provided by the present invention, the behavior strategies include: obstacle avoidance, pedestrian following, and stopping.

[0030] The present invention also provides a behavior decision-making device, comprising:

[0031] The determination module is used to determine the current map and the pedestrian and robot trajectories at multiple preset consecutive moments in the future under each behavior strategy obtained by forward trajectory simulation;

[0032] A feature module, configured to determine a trajectory feature vector under each behavior strategy based on the pedestrian trajectory and the robot trajectory under each behavior strategy and the current map;

[0033] An evaluation module is configured to evaluate each behavior strategy based on an evaluation cost function, a weight vector and a trajectory feature vector under each behavior strategy, to obtain an optimal strategy, wherein the weight vector is obtained by training a sample pair composed of a sample pedestrian trajectory and a sample robot trajectory.

[0034] The application further provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the behavior decision method according to any one of the above when executing the program.

[0035] The application further provides a non-transitory computer readable storage medium, which stores a computer program, wherein the computer program is executable on a processor to implement the behavior decision method according to any one of the above.

[0036] The application further provides a computer program product, which includes a computer program, wherein the computer program is executable on a processor to implement the behavior decision method according to any one of the above.

[0037] The application provides a behavior decision method, device, electronic device and storage medium, which adds a trajectory feature vector under each behavior strategy in the evaluation cost function, and obtains a weight vector through reverse training, so as to realize the strategy evaluation of the evaluation cost function according to the trajectory feature vector and the weight vector, avoid manual adjustment of the feature weight of the cost function, and improve the adaptability to complex scenes. BRIEF DESCRIPTION OF DRAWINGS

[0038] In order to more clearly illustrate the technical solutions of the present application or prior art, the following will briefly introduce the drawings needed in the embodiments or prior art description. Obviously, the drawings in the following description are some embodiments of the present application, and for those skilled in the art, other drawings can also be obtained without creative labor.

[0039] Figure 1 is a flowchart of the behavior decision method provided by the application;

[0040] Figure 2 is a flowchart of the trajectory feature vector acquisition method provided by the application;

[0041] Figure 3 is a flowchart of the weight vector training method provided by the application;

[0042] Figure 4 is a structural diagram of the behavior decision device provided by the application;

[0043] Figure 5 is a structural diagram of the electronic device provided by the application. DETAILED DESCRIPTION

[0044] To make the objectives, technical solutions, and advantages of the present invention more clear, the technical solutions of the present invention will be clearly and completely described below in conjunction with the accompanying drawings. Obviously, the embodiments described are only some of the embodiments of the present invention, not all of them. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts shall fall within the scope of protection of the present invention.

[0045] Early robot behavior decision-making methods set rules for different scenarios, using finite state machines or hierarchical state machines to evaluate the current scene state and select a corresponding policy. For example, robot motion rules are designed based on information such as obstacle speed and the available space in the environment, and behavioral decisions are generated in real time according to the pre-set rules. However, these methods often assume uniform motion of moving obstacles and do not consider the obstacles' reactions to the autonomous vehicle's motion. Therefore, the decision results of these methods may pose potential risks. To account for future changes in the scene state, some methods incorporate predictions of the agent's motion state into the policy decision-making process, making the policy selection more forward-looking. For example, Bayesian classifiers are used to predict the movement trends of obstacles and tree-based mechanisms are designed to search for the optimal policy. Due to the strong uncertainty of policy decisions, some methods model the policy decision problem as a partially observable Markov decision process (POMDP) ​​for policy solution. However, online POMDP solvers typically require a large amount of computing resources and are computationally inefficient, limiting their application in complex real-world scenarios. In recent years, some studies have proposed multi-strategy decision-making models for robots, which have good real-time performance by forward simulating future scene states and designing cost functions to evaluate and select strategies.

[0046] However, although the current robot behavior decision-making method can automatically adjust the action strategy according to the scene changes, the cost function used to evaluate the scene is relatively simple and mainly relies on manual adjustment of the feature weights in the cost function, and its adaptability to complex scenes is insufficient.

[0047] Therefore, how to improve the adaptability of the cost function is a technical problem that needs to be solved urgently by those skilled in the art.

[0048] In response to the above technical problems, an embodiment of the present invention provides a behavior decision-making method. Figure 1 This is a flow chart of the behavior decision-making method provided by the present invention. Figure 1 As shown, the method includes:

[0049] Step 110, determining a current map and pedestrian trajectories and robot trajectories at future preset continuous time points under each behavior strategy through forward trajectory simulation;

[0050] It should be noted that the forward trajectory simulation of the pedestrian trajectory under each behavior strategy can be predicting the pedestrian position at future continuous time points according to the pedestrian historical trajectory and the current map to obtain the pedestrian trajectory, or predicting the pedestrian position at future continuous time points according to the pedestrian historical trajectory, the robot trajectory and the current map to obtain the pedestrian trajectory, and the present application does not make any limitation. The forward trajectory simulation of the robot trajectory under each behavior strategy can be iteratively planning the robot position at future continuous time points according to the trajectory position of the robot, applying each behavior strategy to obtain the robot trajectory.

[0051] Step 120, determining the trajectory feature vector under each behavior strategy based on the pedestrian trajectory and the robot trajectory under each behavior strategy and the current map;

[0052] Step 130, evaluating each behavior strategy based on the evaluation cost function, the weight vector and the trajectory feature vector under each behavior strategy to obtain the optimal strategy; the weight vector is obtained through inverse training by a sample pair composed of a sample pedestrian trajectory and a sample robot example trajectory.

[0053] Considering that the robot needs to be safe and fast to reach the target in a human-robot mixed environment, and needs to maintain a suitable distance from factors in the environment, such as obstacles and pedestrians, therefore, the trajectory feature vector is introduced when the evaluation cost function is used for decision-making in the embodiment of the present application, and in order to avoid relying on manual adjustment of the weight in the cost function, therefore, the weight vector is obtained through inverse training.

[0054] Specifically, after obtaining the weight vector through inverse reinforcement learning by a sample pair composed of a sample pedestrian trajectory and a sample robot example trajectory, the distance from each environmental factor is calculated according to the pedestrian trajectory and the robot trajectory under each behavior strategy, and each environmental factor distance is taken as a dimension in the vector according to the predetermined dimension order to form the trajectory feature vector. Wherein, the environmental factors can include the minimum distance between the robot and the obstacle, the minimum distance between the robot and the pedestrian, and the nearest distance from the robot to the target point, etc., and the embodiment of the present application does not make any limitation. After obtaining the trajectory feature vector under each behavior strategy, the minimum expectation of each behavior strategy is obtained by combining the weight vector to obtain the optimal strategy.

[0055] It should be noted that the sample robot example trajectory can be obtained by recording the trajectory points when a human-controlled robot moves from a starting position to an ending position in a simulation environment containing multiple autonomous moving pedestrians. The dimension of the weight vector is the same as the dimension of the trajectory feature vector.

[0056] The formula for determining the optimal strategy is as follows:

[0057]

[0058] Where, π * represents the optimal strategy, π represents the behavior strategy, C represents the evaluation cost function, represents the pedestrian trajectory from pedestrian 1 to pedestrian N under the behavior strategy π, represents the robot trajectory under policy π.

[0059] The evaluation cost function C is a linear combination of a series of features, and the formula is as follows:

[0060]

[0061] Where w=(w1,w2,...,w k ) represents the weight vector, f=(f1,f2,...,f k ) represents the trajectory feature vector, f1,f2,...,f k Represents the comfort distance of each environmental factor, and k represents k environmental factors.

[0062] The behavior decision-making method provided by an embodiment of the present invention adds the trajectory feature vector under each behavior strategy to the evaluation cost function and obtains the weight vector through inverse training. This enables the evaluation cost function to perform strategy evaluation based on the trajectory feature vector and the weight vector while avoiding manual adjustment of the feature weights of the cost function, thereby improving the adaptability to complex scenarios.

[0063] Based on the above embodiments, Figure 2 It is a flow chart of the method for obtaining trajectory feature vector provided by the present invention, such as Figure 2 As shown, step 120 includes:

[0064] Step 121, based on the robot trajectory under each behavior strategy and the current map, determining the minimum distance between the robot and the obstacle within a predetermined number of consecutive moments in the future under each behavior strategy;

[0065] Step 122, based on the pedestrian trajectory and the robot trajectory under each behavior strategy, determining the minimum distance between the robot and the pedestrian within a predetermined number of consecutive moments in the future under each behavior strategy;

[0066] Step 123, based on the latest trajectory position of the robot in each behavior strategy and the position of the preset target point, determining the distance between the robot and the preset target point under each behavior strategy;

[0067] Step 124, based on the robot trajectory under each behavior strategy and the preset global path, determining the maximum deviation distance between the robot and the preset global path under each behavior strategy;

[0068] Step 125 , based on the minimum distance between the robot and obstacles at multiple preset consecutive moments in the future under each behavioral strategy, the minimum distance between the robot and pedestrians at multiple preset consecutive moments in the future under each behavioral strategy, the distance between the robot and the preset target point under each behavioral strategy, and the maximum deviation distance between the robot and the preset global path under each behavioral strategy, determine the trajectory feature vector under each behavioral strategy.

[0069] Taking into account that the environmental factors that affect the robot's safe and rapid movement to the target point during its movement toward the target point are mainly the distance between the robot and obstacles, the distance between the robot and pedestrians, the distance between the robot and a preset target point, and the maximum deviation distance between the robot and a preset global path, the embodiment of the present invention determines the trajectory feature vector based on the minimum distance between the robot and obstacles, the minimum distance between the robot and pedestrians, the distance between the robot and a preset target point, and the maximum deviation distance between the robot and a preset global path.

[0070] Specifically, a distance calculation is performed between the obstacle position information in the current map and the position in the robot trajectory under each behavioral strategy to obtain the minimum distance between the robot and the obstacle within a preset multiple consecutive moments in the future under each behavioral strategy. A distance calculation is performed between the position in the pedestrian trajectory and the position in the robot trajectory under each behavioral strategy to obtain the minimum distance between the robot and the pedestrian within a preset multiple consecutive moments in the future under each behavioral strategy. A distance calculation is performed between the latest trajectory position and the preset target point position in the robot trajectory under each behavioral strategy to obtain the distance between the robot and the preset target point under each behavioral strategy. A distance calculation is performed one by one between the position in the robot trajectory under each behavioral strategy and the position in the preset global path to obtain the maximum deviation distance between the robot and the preset global path under each behavioral strategy. Then, the minimum distance between the robot and the obstacle within a preset multiple consecutive moments in the future under each behavioral strategy, the minimum distance between the robot and the pedestrian within a preset multiple consecutive moments in the future under each behavioral strategy, the distance between the robot and the preset target point under each behavioral strategy, and the maximum deviation distance between the robot and the preset global path under each behavioral strategy are all used as a dimension in the vector, and the trajectory feature vector under each behavioral strategy is obtained according to the preset dimension mapping relationship.

[0071] It should be noted that the latest trajectory position is the robot's position at the end of the robot's trajectory. The dimensional mapping relationship represents the dimension of the vector that records the minimum distance between the robot and an obstacle, the minimum distance between the robot and a pedestrian, the distance between the robot and the preset target point, and the maximum deviation between the robot and the preset global path over multiple preset consecutive moments in the future. For example, if the minimum distance between the robot and an obstacle over multiple preset consecutive moments in the future is 5, the dimension of the vector is 1; the minimum distance between the robot and a pedestrian over multiple preset consecutive moments in the future is 4, the dimension of the vector is 2; the distance between the robot and the preset target point is 20, the dimension of the vector is 3; the maximum deviation between the robot and the preset global path is 7, the dimension of the vector is 4, then the trajectory feature vector is [5, 4, 20, 7].

[0072] The minimum distance f1 between the robot and the obstacle within a plurality of consecutive future moments is preset, the minimum distance f2 between the robot and the pedestrian within a plurality of consecutive future moments is preset, the distance f3 between the robot and the preset target point is preset, and the maximum deviation distance f4 between the robot and the preset global path is preset.

[0073] Wherein: suppose there are T moments in the preset multiple consecutive moments in the future.

[0074] f1=Dist(ξ r ,M) represents the shortest distance between the robot and the environmental obstacles in the predicted future T time; r is the robot trajectory within time T, and M is the current map.

[0075] f2=Dist(ξ r ,ξ1,ξ2,...,ξ N ) represents the shortest distance between the robot and other pedestrians in the predicted future time T, ξ1~ξ N represents the pedestrian trajectory from pedestrian 1 to pedestrian N at time T; ξ r is the robot trajectory at time T.

[0076] f3=Dist(ξ r ,G) represents the predicted distance between the robot and the target point G at the Tth moment in the future; ξ r is the robot trajectory at time T.

[0077] f2=Dist(ξ r ,P) represents the maximum distance that the robot position deviates from the preset global path P within the predicted future time T.

[0078] Based on the above embodiments, Figure 3 Schematic diagram of the weight vector training method provided by the present invention. Figure 3 As shown, the training steps of the weight vector are as follows:

[0079] Step 310, determining the current weight vector and the sample pair;

[0080] It should be noted that the current weight vector represents the weight vector for each iteration when applying sample pairs for iterative inverse reinforcement learning. Prior to inverse reinforcement learning, the current weight vector is assigned an initial value. The sample pair consists of a sample robot trajectory and a sample pedestrian trajectory within the sample pair. Each sample in the sample pedestrian trajectory contains one or more pedestrian trajectories. The sample robot trajectory is acquired by a human-operated robot in a simulation environment. When acquiring the sample robot trajectory, the surrounding sample pedestrian trajectories that generated the sample trajectory are also acquired. The simulation environment is constructed using the robot's starting and ending positions, as well as the starting and ending positions of each of the multiple autonomous pedestrians.

[0081] Step 320 , determining a planned trajectory of the robot based on the starting position and the ending position of the sample robot trajectory in the sample pair, the sample pedestrian trajectory in the sample pair, and the current weight vector;

[0082] Specifically, according to the starting position and ending position of the sample robot's example trajectory in the sample pair, starting from the starting position, the evaluation cost function is applied to perform strategy evaluation according to the historical trajectory of the sample pedestrian trajectory in the sample pair at the current planning moment and the current position of the robot, as well as the current weight vector, to obtain the current strategy, and the robot's action is controlled according to the decision until it reaches the end position, and the robot's planned trajectory is obtained.

[0083] It should be noted that the moments in the sample robot's example trajectory in the sample pair and the moments in the sample pedestrian's trajectory in the sample pair are in one-to-one correspondence.

[0084] Step 330: Determine a first trajectory feature vector based on the sample robot trajectory and the sample pedestrian trajectory; and determine a second trajectory feature vector based on the robot planned trajectory and the sample pedestrian trajectory.

[0085] Step 340 : Determine the gradient of the current weight vector based on the first trajectory feature vector and the second trajectory feature vector; and iteratively adjust the current weight vector based on the gradient until the current weight vector converges to obtain a weight vector.

[0086] Specifically, a first trajectory feature vector is calculated using sample robot trajectories and sample pedestrian trajectories through a comfortable distance calculation. Simultaneously, a second trajectory feature vector is calculated using the robot's planned trajectory and sample pedestrian trajectories through a comfortable distance calculation. After obtaining the first and second trajectory feature vectors, the gradient of the current weight vector is calculated based on the first and second trajectory feature vectors. Gradient descent is then performed using the gradient and the current weight vector is iteratively adjusted until convergence. The current weight vector is then used as the trained weight vector.

[0087] It should be noted that the current weight vector w is trained using the sample pair D. Through repeated iterative optimization, the current weight vector w is optimized by continuously using the sample pair D. The steps are as follows:

[0088] (1) Set a random initial weight vector w0 and step size δ, and use the initial weight vector as the current weight vector w;

[0089] (2) For each sample robot trajectory in D In the same simulation scenario, the robot planning trajectory is obtained using the current weight vector w

[0090] (3) Calculate the sample robot trajectory and robot planning trajectory The eigenvector of and

[0091] (4) Traverse all sample data in D once and calculate the gradient of weight w

[0092] (5) Perform gradient descent w←w-δΔw.

[0093] (6) Repeat steps (2) to (5) until the current weight vector w converges.

[0094] Based on the above embodiment, the steps for determining pedestrian trajectories under each behavior strategy include:

[0095] Step S11, determining the pedestrian trajectory at the current prediction moment and under any of the behavior strategies;

[0096] It should be noted that the initial pedestrian trajectory under any strategy in each behavior strategy is empty. Thereafter, each time the pedestrian position at a moment is predicted, the pedestrian position is added to the pedestrian trajectory under the strategy.

[0097] Step S12, based on the pedestrian position under the strategy at the moment before the current prediction moment, the robot position under the strategy at the moment before the current prediction moment, and the current map, a pedestrian trajectory prediction model is applied to perform prediction to obtain the pedestrian position at the current prediction moment under the strategy;

[0098] Step S13, adding the pedestrian position to the pedestrian trajectory under the strategy, and iterating the next moment of the current predicted moment under the strategy as the current pedestrian moment under the strategy until the current predicted moment is the end moment of multiple preset consecutive moments in the future, and obtaining the pedestrian trajectory under the strategy.

[0099] Specifically, the pedestrian position under the strategy at the moment before the current prediction moment, the robot position under the strategy at the moment before the current prediction moment, and the current map are input into the pedestrian trajectory prediction model for prediction, and the pedestrian position at the current prediction moment under the strategy is output by the pedestrian trajectory prediction model. Then, the pedestrian position at the current prediction moment under the strategy is added to the pedestrian trajectory under the strategy, and the next moment after the current prediction moment under the strategy is used as the current pedestrian moment under the strategy. This is iterated until the current prediction moment under the strategy is the end moment of multiple preset consecutive moments in the future, at which time the pedestrian trajectory under the strategy is obtained.

[0100] It should be noted that the pedestrian position of pedestrian i at time t is defined as The position of the robot at time t is Based on the robot position, pedestrian position, and the current map, a forward simulation of the pedestrian trajectory is performed as follows:

[0101]

[0102] Where, represents the position of the i-th pedestrian at the next moment of time t, represents the positions of the first to the Nth pedestrian at time t, Indicates the position of the robot at time t, M represents the current map. In actual prediction, t is the current moment, f p represents the pedestrian trajectory prediction model.

[0103] Based on the above embodiment, the steps for determining the robot trajectory under each behavior strategy include:

[0104] Step S21, determining the robot trajectory at the current planning moment and under any of the behavior strategies;

[0105] It should be noted that the pedestrian trajectory under this strategy is initially empty. Every time the robot position at a moment is planned, the robot position is added to the pedestrian trajectory under this strategy.

[0106] Step S22, based on the robot position under the strategy at the moment before the current planning moment and the strategy, the robot trajectory planning model is applied to perform planning to obtain the robot position at the current planning moment under the strategy;

[0107] Step S23, adding the robot position to the robot trajectory under the strategy, and iterating the next moment of the current planning moment under the strategy as the current planning moment under the strategy until the current planning moment is the end moment of multiple preset consecutive moments in the future, to obtain the robot trajectory under the strategy.

[0108] Specifically, the strategy and the current planned position under the strategy are input into the robot trajectory planning model for planning, and the robot position at the current planning moment under the strategy is obtained. Then, the robot position is added to the robot trajectory under the strategy, and the next moment of the current planning moment under the strategy is used as the current planning moment under the strategy. This is iterated until the current predicted moment under the strategy is the end moment of multiple preset consecutive moments in the future. At this time, the robot trajectory under the strategy is obtained.

[0109] It should be noted that the pedestrian position of pedestrian i at time t is defined as The position of the robot at time t is Based on the robot position, pedestrian position, and current map, the forward simulation of the robot trajectory is performed as follows:

[0110]

[0111] Where, represents the robot position at the next moment at time t, Represents the position of the robot at time t, M represents the current map, π represents the behavior strategy, and in actual prediction, t is the current moment.

[0112] In addition, after obtaining the pedestrian position and robot position at time t+1, the pedestrian position and robot position at time t+2 are obtained according to the forward simulation of the pedestrian trajectory and the forward simulation of the robot trajectory. By iterating the preset threshold T times, the pedestrian trajectory of pedestrian i in the future T time can be obtained. The robot's trajectory in the future T time

[0113] Based on the above embodiment, the example trajectory of the sample robot in the sample pair is based on the motion trajectory recorded by a human-operated robot from a starting position to an ending position in a simulation environment; the simulation environment includes a starting position, an ending position and the trajectories of multiple autonomously moving pedestrians.

[0114] It should be noted that the starting position and ending position of the robot's running trajectory are randomly set in the simulation environment, and the starting position and ending position of multiple autonomous pedestrians are set. The autonomous pedestrians can move back and forth between their corresponding starting position and ending position. The person operates the robot from the starting position and controls the robot according to the positions of the multiple autonomous pedestrians until the robot is controlled to move to the end position. The movement of the robot starts from the starting position and ends at the moment when the robot stops moving at the end position. The movement trajectory of the robot between the two moments is recorded to form a sample robot example trajectory in the sample pair. At the same time, the movement trajectory of each autonomous pedestrian between the two moments is recorded as the sample pedestrian trajectory in the sample pair.

[0115] Based on the above embodiments, the present invention provides a preferred embodiment, in which the behavior strategies include: obstacle avoidance, pedestrian following, and stopping.

[0116] It should be noted that the robot's navigation behavior strategy Π = {avoid obstacles, follow pedestrian i, stop} is defined as:

[0117] Obstacle avoidance strategy: When there is a low or medium density of human activity around the robot's global path and a traversable path exists, the robot will select an obstacle avoidance strategy. Specifically, the robot sets a local target point based on the preset global path, avoids pedestrians in the environment, and performs local motion planning to avoid obstacles.

[0118] In the case of a crowded and slow-moving global path, the robot selects pedestrian i, whose movement trend is closest to the pre-set global path, as the leader and executes the following strategy. The specific implementation method is to set a local target point based on the position of pedestrian i and perform local obstacle avoidance motion planning.

[0119] Stop strategy: In an emergency or when the global path is completely blocked, the stop strategy is selected. The specific execution method is that the robot slows down and stops to wait for the surrounding situation to change.

[0120] The behavior decision-making device provided by the present invention is described below. The behavior decision-making device described below and the behavior decision-making method described above can be referenced to each other.

[0121] Figure 4 Schematic diagram of the behavior decision-making device provided by the present invention. Figure 4 As shown, the behavior decision-making device provided by the embodiment of the present invention includes: a determination module 410, a feature module 420 and an evaluation module 430.

[0122] in,

[0123] The determining module 410 is configured to determine a current map and pedestrian trajectories and robot trajectories at a plurality of preset future continuous time points under each behavior strategy obtained through forward trajectory simulation.

[0124] The feature module 420 is configured to determine a trajectory feature vector under each behavior strategy based on the pedestrian trajectories and the robot trajectories under each behavior strategy and the current map.

[0125] The evaluation module 430 is configured to evaluate each behavior strategy based on an evaluation cost function, apply a weight vector and the trajectory feature vector under each behavior strategy, and obtain an optimal strategy.

[0126] The behavior decision device provided in the embodiment of the present application can determine a current map and pedestrian trajectories and robot trajectories at a plurality of preset future continuous time points under each behavior strategy through the determining module, determine a trajectory feature vector under each behavior strategy based on the pedestrian trajectories and the robot trajectories under each behavior strategy and the current map through the feature module, and evaluate each behavior strategy based on an evaluation cost function, apply a weight vector and the trajectory feature vector under each behavior strategy, and obtain an optimal strategy through the evaluation module.

[0127] According to any one of the above embodiments, the feature module 410 comprises:

[0128] The obstacle distance sub-module is configured to determine a minimum distance between the robot and an obstacle within the plurality of preset future continuous time points under each behavior strategy based on the robot trajectories under each behavior strategy and the current map.

[0129] The pedestrian distance sub-module is configured to determine a minimum distance between the robot and a pedestrian within the plurality of preset future continuous time points under each behavior strategy based on the pedestrian trajectories and the robot trajectories under each behavior strategy.

[0130] The target point distance sub-module is configured to determine a distance between the robot and a preset target point under each behavior strategy based on a latest trajectory position in the robot trajectories under each behavior strategy and a preset target point position.

[0131] The deviation distance sub-module is configured to determine a maximum deviation distance between the robot and a preset global path under each behavior strategy based on the robot trajectories under each behavior strategy and the preset global path.

[0132] The vector submodule is used to determine the trajectory feature vector under each behavioral strategy based on the minimum distance between the robot and obstacles at multiple consecutive moments in the future, the minimum distance between the robot and pedestrians at multiple consecutive moments in the future, the distance between the robot and the preset target point under each behavioral strategy, and the maximum deviation distance between the robot and the preset global path under each behavioral strategy.

[0133] Based on any of the above embodiments, the behavior decision-making device provided by the embodiment of the present invention further includes: a reverse training module, which includes:

[0134] An initialization submodule, used to determine the current weight vector and the sample pair;

[0135] a planning submodule for determining a planned trajectory of the robot based on the starting and ending positions of the sample robot example trajectories in the sample pair, the sample pedestrian trajectories in the sample pair, and the current weight vector;

[0136] A vector submodule is configured to determine a first trajectory feature vector based on a sample robot trajectory and a sample pedestrian trajectory; and to determine a second trajectory feature vector based on a planned robot trajectory and a sample pedestrian trajectory;

[0137] The iterative training submodule is used to determine the gradient of the current weight vector based on the first trajectory feature vector and the second trajectory feature vector; and iteratively adjust the current weight vector based on the gradient until the current weight vector converges to obtain the weight vector.

[0138] Based on any of the above embodiments, the behavior decision-making device provided in the embodiment of the present invention further includes: a pedestrian forward simulation module, which includes:

[0139] The prediction condition submodule is used to determine the pedestrian trajectory at the current prediction moment and under any of the behavior strategies;

[0140] The pedestrian position prediction submodule is used to apply the pedestrian trajectory prediction model to predict the pedestrian position at the current prediction moment under the strategy based on the pedestrian position under the strategy at the previous moment, the robot position under the strategy at the previous moment, and the current map.

[0141] The pedestrian trajectory determination submodule is used to add the pedestrian position to the pedestrian trajectory under the strategy, and iterate the next moment of the current prediction moment as the current prediction moment until the current prediction moment is the end moment of multiple preset consecutive moments in the future, thereby obtaining the pedestrian trajectory under the strategy.

[0142] Based on any of the above embodiments, the behavior decision-making device provided by the embodiment of the present invention further includes: a robot forward simulation module, which includes:

[0143] The planning condition submodule is used to determine the robot trajectory at the current planning moment and under any of the behavior strategies;

[0144] The planning position submodule is used to plan based on the robot position under the strategy at the previous moment and the strategy, and obtain the robot position at the current planning moment under the strategy;

[0145] The robot trajectory determination submodule is used to add the robot position to the robot trajectory under the strategy, and iterate the next moment of the current planning moment under the strategy as the current planning moment under the strategy until the current planning moment is the end moment of multiple preset consecutive moments in the future, thereby obtaining the robot trajectory under the strategy.

[0146] Figure 5 An example of a physical structure diagram of an electronic device is shown below. Figure 5 As shown, the electronic device may include: a processor 510, a communication interface 520, a memory 530, and a communication bus 540, wherein the processor 510, the communication interface 520, and the memory 530 communicate with each other via the communication bus 540. The processor 510 may call the logic instructions in the memory 530 to execute the behavior decision method, which includes: determining the current map, and the pedestrian trajectories and robot trajectories at multiple preset consecutive moments in the future under each behavior strategy obtained by forward trajectory simulation; determining the trajectory feature vector under each behavior strategy based on the pedestrian trajectories and robot trajectories under each behavior strategy and the current map; based on the evaluation cost function, applying the weight vector and the trajectory feature vector under each behavior strategy to evaluate each behavior strategy and obtain the optimal strategy; the weight vector is obtained by training a sample pair consisting of a sample pedestrian trajectory and a sample robot example trajectory.

[0147] In addition, the logic instructions in the above-mentioned memory 530 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when sold or used as an independent product. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0148] On the other hand, the present invention also provides a computer program product, which includes a computer program, which can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the behavior decision-making method provided by the above methods, which includes: determining the current map, and the pedestrian trajectories and robot trajectories at multiple preset consecutive moments in the future under each behavior strategy obtained by forward trajectory simulation; determining the trajectory feature vector under each behavior strategy based on the pedestrian trajectories and robot trajectories under each behavior strategy, and the current map; based on the evaluation cost function, applying the weight vector and the trajectory feature vector under each behavior strategy, evaluating each behavior strategy to obtain the optimal strategy; the weight vector is obtained by training a sample pair consisting of a sample pedestrian trajectory and a sample robot example trajectory.

[0149] On the other hand, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, is implemented to execute the behavior decision-making method provided by the above-mentioned methods, the method comprising: determining the current map, and the pedestrian trajectories and robot trajectories at preset multiple consecutive moments in the future under each behavior strategy obtained by forward trajectory simulation; determining the trajectory feature vector under each behavior strategy based on the pedestrian trajectories and robot trajectories under each behavior strategy, and the current map; based on the evaluation cost function, applying the weight vector and the trajectory feature vector under each behavior strategy, evaluating each behavior strategy to obtain the optimal strategy; the weight vector is obtained by training a sample pair consisting of a sample pedestrian trajectory and a sample robot example trajectory.

[0150] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, i.e., they may be located in one location or distributed across multiple network units. Some or all of the modules may be selected based on actual needs to achieve the objectives of the present embodiment. Persons of ordinary skill in the art will be able to understand and implement the present invention without inventive effort.

[0151] Through the description of the above embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus a necessary general hardware platform, or of course, by hardware. Based on this understanding, the essence of the above technical solution or the part that contributes to the existing technology can be embodied in the form of a software product. The computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a magnetic disk, an optical disk, etc., and includes a number of instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or certain parts of the embodiments.

[0152] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the various embodiments of the present invention.

Claims

1. A behavioral decision-making method, characterized in that: include: Determine the current map and the pedestrian and robot trajectories for multiple consecutive moments in the future under each behavioral strategy obtained from the forward trajectory simulation; Determining trajectory feature vectors under each behavior strategy based on the pedestrian trajectory and the robot trajectory under each behavior strategy and the current map; Based on the evaluation cost function, the weight vector and the trajectory feature vector under each behavior strategy are applied to evaluate each behavior strategy to obtain the optimal strategy; The weight vector is obtained based on training of sample pairs consisting of sample pedestrian trajectories and sample robot example trajectories; The determining of the trajectory feature vectors under each behavior strategy based on the pedestrian trajectory and the robot trajectory under each behavior strategy and the current map includes: Based on the robot trajectory under each behavior strategy and the current map, determining the minimum distance between the robot and the obstacle within a plurality of preset consecutive moments in the future under each behavior strategy; Based on the pedestrian trajectory and the robot trajectory under each behavior strategy, determining the minimum distance between the robot and the pedestrian within a plurality of preset consecutive moments in the future under each behavior strategy; Determine the distance between the robot and the preset target point under each behavior strategy based on the latest trajectory position and the preset target point position in the robot trajectory under each behavior strategy; Based on the robot trajectory and the preset global path under each behavior strategy, determining the maximum deviation distance between the robot and the preset global path under each behavior strategy; Determine the trajectory feature vector under each behavioral strategy based on the minimum distance between the robot and the obstacle at multiple consecutive future moments under each behavioral strategy, the minimum distance between the robot and the pedestrian at multiple consecutive future moments under each behavioral strategy, the distance between the robot and the preset target point under each behavioral strategy, and the maximum deviation distance between the robot and the preset global path under each behavioral strategy; The training steps of the weight vector are as follows: Determining a current weight vector and the sample pair; Based on the starting position and the ending position of the sample robot example trajectory in the sample pair, the sample pedestrian trajectory in the sample pair, and the current weight vector, the evaluation cost function is applied to determine the robot planning trajectory; Determining a first trajectory feature vector based on the sample robot example trajectory in the sample pair and the sample pedestrian trajectory in the sample pair; and determining a second trajectory feature vector based on the robot planned trajectory and the sample pedestrian trajectory in the sample pair; Based on the first trajectory feature vector and the second trajectory feature vector, a gradient of the current weight vector is determined; and based on the gradient, the current weight vector is iteratively adjusted until the current weight vector converges to obtain the weight vector.

2. The behavior decision-making method according to claim 1, characterized in that: The steps of determining pedestrian trajectories under each behavior strategy include: Determine the pedestrian trajectory at the current prediction time and under any of the aforementioned behavioral strategies; Based on the pedestrian position under any of the strategies at the moment before the current prediction moment, the robot position under any of the strategies at the moment before the current prediction moment, and the current map, the pedestrian trajectory prediction model is applied to perform prediction to obtain the pedestrian position at the current prediction moment under any of the strategies; The pedestrian position is added to the pedestrian trajectory under any of the strategies, and the next moment of the current prediction moment is iterated as the current prediction moment until the current prediction moment is the end moment of the preset multiple consecutive moments in the future, thereby obtaining the pedestrian trajectory under any of the strategies.

3. The behavior decision-making method according to claim 1, characterized in that: The steps of the robot trajectory under each behavior strategy include: Determine the robot trajectory at the current planning moment and under any of the aforementioned behavioral strategies; Based on the robot position under any one of the strategies at a moment before the current planning moment and the any one of the strategies, the robot trajectory planning model is applied to perform planning to obtain the robot position at the current planning moment under the any one of the strategies; The robot position is added to the robot trajectory under any of the strategies, and the next moment of the current planning moment under any of the strategies is used as the current planning moment under any of the strategies for iteration until the current planning moment is the end moment of the preset multiple consecutive moments in the future, thereby obtaining the robot trajectory under any of the strategies.

4. The behavior decision-making method according to claim 1, characterized in that: The sample robot example trajectory in the sample pair is based on the motion trajectory recorded by a human-operated robot from a starting position to an ending position in a simulation environment; the simulation environment includes the starting position, the ending position and the trajectories of multiple autonomously moving pedestrians.

5. A behavior decision-making device, characterized in that: include: The determination module is used to determine the current map and the pedestrian and robot trajectories at multiple preset consecutive moments in the future under each behavior strategy obtained by forward trajectory simulation; A feature module, configured to determine a trajectory feature vector under each behavior strategy based on the pedestrian trajectory and the robot trajectory under each behavior strategy and the current map; An evaluation module is configured to evaluate each behavioral strategy based on an evaluation cost function and apply a weight vector and a trajectory feature vector under each behavioral strategy to obtain an optimal strategy; the weight vector is obtained by training a sample pair consisting of a sample pedestrian trajectory and a sample robot trajectory; The determining of the trajectory feature vectors under each behavior strategy based on the pedestrian trajectory and the robot trajectory under each behavior strategy and the current map includes: Based on the robot trajectory under each behavior strategy and the current map, determining the minimum distance between the robot and the obstacle within a plurality of preset consecutive moments in the future under each behavior strategy; Based on the pedestrian trajectory and the robot trajectory under each behavior strategy, determining the minimum distance between the robot and the pedestrian within a plurality of preset consecutive moments in the future under each behavior strategy; Determine the distance between the robot and the preset target point under each behavior strategy based on the latest trajectory position and the preset target point position in the robot trajectory under each behavior strategy; Based on the robot trajectory and the preset global path under each behavior strategy, determining the maximum deviation distance between the robot and the preset global path under each behavior strategy; Determine the trajectory feature vector under each behavioral strategy based on the minimum distance between the robot and the obstacle at multiple consecutive future moments under each behavioral strategy, the minimum distance between the robot and the pedestrian at multiple consecutive future moments under each behavioral strategy, the distance between the robot and the preset target point under each behavioral strategy, and the maximum deviation distance between the robot and the preset global path under each behavioral strategy; The training steps of the weight vector are as follows: Determining a current weight vector and the sample pair; Based on the starting position and the ending position of the sample robot example trajectory in the sample pair, the sample pedestrian trajectory in the sample pair, and the current weight vector, the evaluation cost function is applied to determine the robot planning trajectory; Determining a first trajectory feature vector based on the sample robot example trajectory in the sample pair and the sample pedestrian trajectory in the sample pair; and determining a second trajectory feature vector based on the robot planned trajectory and the sample pedestrian trajectory in the sample pair; Based on the first trajectory feature vector and the second trajectory feature vector, a gradient of the current weight vector is determined; and based on the gradient, the current weight vector is iteratively adjusted until the current weight vector converges to obtain the weight vector.

6. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the program, the behavior decision method according to any one of claims 1 to 4 is implemented.

7. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the behavior decision method according to any one of claims 1 to 4 is implemented.

8. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the behavior decision method according to any one of claims 1 to 4 is implemented.

Images