Safe and efficient service robot navigation method and system in dense crowd environment

Abstract

The invention belongs to the field of robot navigation, and provides a safe and efficient service robot navigation method and system in a dense crowd environment. The safe and efficient service robot navigation method in the dense crowd environment comprises the following steps: acquiring crowd state information, and performing decision navigation on a robot according to the crowd state information and the state information of the robot in the dense crowd environment; extracting social interaction features and spatio-temporal features from the directed graph by using a spatio-temporal diagram attention-based value network, and obtaining crowd features in combination with a multi-head self-attention mechanism; obtaining a robot state estimation value based on a mapping relation between the crowd characteristics and the robot state and the robot state estimation value; and based on the robot state estimation value, the value network and a preset reward function, determining an optimal navigation strategy of the robot by utilizing a reinforcement learning framework so as to realize safe and efficient navigation of the robot in a dense crowd environment.

Description

Technical Field

[0001] The present invention belongs to the field of robot navigation, and particularly relates to a safe and efficient service robot navigation method and system in a dense crowd environment. Background Art

[0002] The statements in this part only provide background technical information related to the present invention, and do not necessarily constitute prior art.

[0003] The navigation of existing service robots in complex social scenarios with dense crowds still faces the following challenges: there are a large number of dynamic pedestrians in the scenario, and pedestrians have the ability of autonomous decision-making. They will adjust their own movements in real time according to their own states, other scenario information, and interaction behaviors with surrounding pedestrians. It is difficult for robots to understand the complex behaviors of pedestrians and respond in a timely manner.

[0004] In deep reinforcement learning, the prior art extracts high-level abstract features through a value network. For example, using social interaction information between humans, using a heuristic function to approximate, or using a graph convolutional neural network to extract social interaction features between humans. However, these methods cannot fully consider complex social interactions. The prior art uses a reward function to guide the behavior of the robot. However, the current reward functions generally set the comfortable distance that should be maintained between pedestrians and the robot as a fixed value. In this way, when the crowd density increases, the robot cannot adaptively adjust the distance from pedestrians, resulting in a reduction or even disappearance of the feasible area, and generally only considering the safety of navigation, it is difficult to ensure the time efficiency of navigation. Summary of the Invention

[0005] In order to solve the technical problems existing in the above background art, the present invention provides a safe and efficient service robot navigation method and system in a dense crowd environment. Firstly, a novel value network based on spatio-temporal graph attention is proposed to improve the robot's ability to understand human social interactions. Secondly, a risk area of the robot is designed based on pedestrian speed and crowd density. Thirdly, a reward function is designed to achieve safe and socially compliant navigation by guiding the robot to avoid the above-mentioned risk areas.

[0006] In order to achieve the above object, the present invention adopts the following technical solutions:

[0007] The first aspect of the present invention provides a safe and efficient service robot navigation method in a dense crowd environment.

[0008] A safe and efficient service robot navigation method in a dense crowd environment, which includes:

[0009] Modeling the dense crowd and the robot as a directed graph according to the relative spatial position relationship between the robot and pedestrians in the dense crowd environment;

[0010] Extract social interaction features and spatio-temporal features from the directed graph using a value network based on spatio-temporal graph attention, and then combine with the multi-head self-attention mechanism to obtain crowd features;

[0011] Based on the mapping relationship between the crowd features and the robot state and the estimated value of the robot state, obtain the estimated value of the robot state;

[0012] Based on the estimated value of the robot state, the value network and a preset reward function, use the reinforcement learning framework to determine the optimal navigation strategy of the robot to achieve safe and efficient navigation of the robot in a dense crowd environment;

[0013] Among them, on the basis of setting a risk area, the reward function enables the robot to reach the target position while ensuring the safety, time efficiency and path smoothness of the robot navigation; the risk area is determined by the pedestrian speed and the crowd density.

[0014] As an implementation, the objective function corresponding to the optimal navigation strategy and the optimal speed of the robot determined by the reinforcement learning framework is:

[0015]

[0016] Among them, represents the reward obtained at time t; A represents the action space; a t represents the action of the robot at time t; γ ∈ (0, 1) is a discount factor, V * is the optimal value function, is the state transition probability from time t to time t Δ = t + Δt; Δt is the time interval; v pref is the preferred speed; is the joint state of the robot at time t; is the joint state of the robot at t Δ moment; is the optimal navigation strategy; T is the final time step; t * is the future time step starting from the current time step t.

[0017] As an implementation, the expression of the reward function is:

[0018]

[0019]

[0020]

[0021]

[0022] rsmooth = -0.01(|ω t - ω t-1 | + |v t - v t-1 |)

[0023] where r goal is the reward for the robot to reach the target point; r safe is the penalty for the robot's collision behavior or entering the robot's risk area; r time in which t limit is a constant representing the specified upper limit of the navigation time; r smooth is the penalty for the changes in the linear velocity v and the angular velocity ω; the subscript t of the linear velocity v and the angular velocity ω represents the t-th moment; d g represents the distance between the robot and the target point, represents the reward obtained at the t-th moment; represents the joint state of the entire system at the t-th moment; a t represents the action at the t-th moment; d r represents the minimum separation distance between the robot and the pedestrian within the time interval Δt; r' represents the radius of the fan-shaped area in the robot's final risk area; r h represents the radius of the pedestrian itself.

[0024] As an implementation, the value network based on spatio-temporal graph attention includes a person-person attention layer and a robot-person attention layer; the person-person attention layer is used to extract person-person interaction features, and the robot-person attention layer is used to extract robot-person interaction features, and fuses the person-person interaction features and the robot-person interaction features into social interaction features; based on the social interaction features and the multi-head self-attention mechanism, corresponding time features are extracted.

[0025] The second aspect of the present invention provides a safe and efficient service robot navigation system in a dense crowd environment.

[0026] A safe and efficient service robot navigation system in a dense crowd environment, comprising:

[0027] A crowd state acquisition module, which is used to acquire crowd state information;

[0028] A decision-making navigation module, which is used to make decision-making navigation for the robot according to the crowd state information and the state information of the robot in the dense crowd environment;

[0029] The decision-making navigation module further includes:

[0030] A directed graph construction sub-module, which is used to determine the relative spatial position relationship between a robot and pedestrians in a dense crowd environment according to the crowd status information, and model the dense crowd and the robot as a directed graph;

[0031] A feature extraction sub-module, which uses a value network based on spatio-temporal graph attention to extract social interaction features and spatio-temporal features from the directed graph, and then combines a multi-head self-attention mechanism to obtain crowd features;

[0032] A feature extraction sub-module, which determines the optimal navigation strategy of the robot based on the robot state estimate, the value network, and a preset reward function, and uses a reinforcement learning framework to achieve safe and efficient navigation of the robot in a dense crowd environment;

[0033] Among them, on the basis of setting a risk area, the reward function enables the robot to reach the target position while ensuring the safety, time efficiency, and path smoothness of the robot navigation; the risk area is determined by the pedestrian speed and the crowd density.

[0034] The third aspect of the present invention provides a computer-readable storage medium.

[0035] A computer-readable storage medium, on which a computer program is stored, and when the program is executed by a processor, it implements the steps in the safe and efficient service robot navigation method in a dense crowd environment as described above.

[0036] The fourth aspect of the present invention provides a computer program product.

[0037] A computer program product, including a computer program / instructions, and when the computer program / instructions are executed by a processor, it implements the steps in the safe and efficient service robot navigation method in a dense crowd environment as described above.

[0038] The fifth aspect of the present invention provides a service robot.

[0039] A service robot, including a robot body, a memory, a processor, and a computer program stored on the memory and executable on the processor, and when the processor executes the program, it implements the steps in the safe and efficient service robot navigation method in a dense crowd environment as described above.

[0040] Compared with the prior art, the beneficial effects of the present invention are:

[0041] The present invention proposes a navigation method for service robots in crowded scenarios. Based on a value network with spatio-temporal graph attention, it enhances the robot's ability to understand human social interactions. A risk area for the robot is designed based on pedestrian speed and crowd density. A reward function is designed, with the comfortable distance to be maintained between pedestrians and the robot set as a variable. In this way, when the crowd density increases, the robot can adaptively adjust the distance from pedestrians, and achieve safe and socially compliant navigation by guiding the robot to avoid the above-mentioned risk areas, realizing safe and efficient navigation of the robot in a crowded environment.

[0042] Advantages of additional aspects of the present invention will be given in part in the following description, become apparent in part from the following description, or be understood through the practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] The accompanying drawings forming a part of this specification are used to provide a further understanding of the present invention. The schematic embodiments and descriptions thereof of the present invention are used to explain the present invention and do not constitute an improper limitation to the present invention.

[0044] Figure 1 is the architecture diagram of the value network based on spatio-temporal graph attention according to an embodiment of the present invention;

[0045] FIG. 2(a) is the pedestrian safety area according to an embodiment of the present invention;

[0046] FIG. 2(b) is the final geometric shape of the risk area according to an embodiment of the present invention;

[0047] Figure 3 is the schematic diagram for judging whether the robot is located in the risk area according to an embodiment of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0048] The present invention will be further described below in conjunction with the accompanying drawings and embodiments.

[0049] It should be noted that the following detailed description is illustrative and is intended to provide further explanation of the present invention. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by those of ordinary skill in the technical field to which the present invention belongs.

[0050] It should be noted that the terms used herein are only for describing specific embodiments and are not intended to limit the exemplary embodiments according to the present invention. As used herein, unless the context clearly indicates otherwise, the singular form is also intended to include the plural form. In addition, it should be understood that when the terms "comprise" and / or "include" are used in this specification, they indicate the presence of features, steps, operations, devices, components, and / or combinations thereof.

[0051] Embodiment 1

[0052] In one or more embodiments, a safe and efficient navigation method for a service robot in a dense crowd environment is provided, which includes:

[0053] Step S101: Obtain crowd state information.

[0054] Step S102: Make a decision on the navigation of the robot according to the crowd state information and the state information of the robot in the dense crowd environment;

[0055] The step S102 specifically further includes the following steps:

[0056] Step S1021: Model the dense crowd and the robot as a directed graph according to the relative spatial position relationship between the robot and the pedestrians in the dense crowd environment.

[0057] The present invention models the crowd and the robot as a directed graph G t represents the relative spatial position relationship between the robot and the pedestrians at time t, which is defined as G t =(V t , E t ), where represents the node information of the graph, represents the i-th node in the graph. That is, the edge between nodes in the graph represents the importance of agent i to agent j or agent j to agent i at time t. N represents the number of pedestrians.

[0058] Step S1022: Use a value network based on spatio-temporal graph attention to extract social interaction features and spatio-temporal features from the directed graph, and then combine a multi-head self-attention mechanism to obtain crowd features.

[0059] When humans walk in a crowd, they will predict the movement trends of the surrounding pedestrians through their behaviors to generate social interactions with others. Understanding this interaction is beneficial for the robot to navigate. The graph structure can well represent the spatial relationship between pedestrians. Although the graph convolutional network can capture social interaction features to a certain extent, it is difficult to construct a reasonable adjacency matrix to characterize the relationship between pedestrians, and it does not consider that the influence degrees of different pedestrians on the robot are different. And the current methods generally only consider the spatial relationship between pedestrians, but do not consider the temporal relationship of pedestrian movement. Therefore, as Figure 1 shown, the present invention designs a value network based on spatio-temporal graph attention to extract complex social interaction features and spatio-temporal features, infer the movement trends of pedestrians, and output state values to determine the actions of the robot.

[0060] Since the states of the robot and the pedestrians are in different dimensions, two multi-layer perceptrons φ with ReLU activation functions are usedr ( ) and φ h ( ) maps the vectors and into the embedding vectors. The specific calculation process is as follows:

[0061]

[0062]

[0063]

[0064] where W r and W h are the network weights of φ r ( ) and φ h ( ) respectively, is the embedding vector of the robot, is the embedding vector of the pedestrian, and H 0 is the input of the subsequent network.

[0065] The position p = [p x , p y , velocity v = [v x , v y and radius r of the agent constitute the observable state. The target position p g , preferred velocity v pref and heading angle θ of the agent constitute the unobservable state. To make the state representation more general, it is transformed into a coordinate system with the robot's position as the coordinate origin and the direction from the robot to the target position as the x-axis. The robot state and the observable state of the i-th pedestrian are represented as follows:

[0066]

[0067]

[0068] where d g = ||p - p g ||2 represents the distance between the robot and the target point, and d i = ||p - p i ||2 represents the distance between the robot and the i-th pedestrian. are the components of the position of the i-th pedestrian in the x and y directions respectively; are the components of the velocity of the i-th pedestrian in the x and y directions respectively; is the radius of the i-th pedestrian.

[0069] The interaction between humans and robots directly affects the robot's decision-making, while the interaction between humans indirectly affects the robot's decision-making. Therefore, in the present invention, a human-human attention layer and a robot-human attention layer are designed to extract human-human interaction and robot-human interaction respectively. The human-human attention layer only calculates the interaction between humans and masks the robot nodes through a masking mechanism. The specific calculation process is as follows:

[0070] e ij =LeakyReLU(φ ah (Q HH ||K HH ,W ah ))

[0071]

[0072]

[0073] Among them, the pedestrian state is mapped into Q through two fully connected layers with ReLU activation functions HH and K HH , that is || represents the concatenation operation. The state is mapped into attention weights α through the fully connected layer φ ah (·). N ij is the normalized attention coefficient, and N i represents all neighbor nodes of the i-th node in the graph. In the robot-human attention layer, the robot node and the pedestrian node aggregated in the first layer are used as inputs for graph attention calculation. The specific calculation process is as follows:

[0074]

[0075]

[0076]

[0077] In the formula, the state of the robot and the vector output by the human-human attention layer are mapped into Q RH and K RH , that is is a fully connected layer with a ReLU activation function. The output after passing through the human-human attention layer and the robot-human attention layer is where n represents the number of pedestrians.

[0078] Since there are various types of interaction features among pedestrians, the present invention uses multiple independent channels to extract the interaction features and then combines the outputs of each independent channel:

[0079]

[0080] Among them, M represents the number of channels.

[0081] The pedestrian movement at the current moment is affected by the previous moment and will affect the future moment. Fully extracting this temporal feature is beneficial to the robot's decision-making. Therefore, in this chapter, the multi-head self-attention mechanism is used to extract the temporal dependence of pedestrian movement between adjacent four time frames. Use three different linear layers to map H all into q, k, v:

[0082] q = W q H all

[0083] k = W k H all

[0084] v = W v H all

[0085] After aggregating the temporal information through the multi-head attention mechanism with residual connections, the matrix o t is obtained:

[0086]

[0087] o t = contact(head1,...,head h )W o + H 1

[0088] Among them, represents the dimension of k, h represents the number of attention heads, and W o is a linear layer. Then, layer normalization is performed and the residual connection is used to obtain the crowd feature Θ t :

[0089] Θ t = φ t (LN(o t ),W t )+ o t

[0090] Among them, φ t () is a multi-layer perceptron with a ReLU activation function, and W t is the weight of φ t ().

[0091] Step S1023: Based on the mapping relationship between the crowd feature and the robot state and the robot state estimate value, obtain the robot state estimate value;

[0092] Based on the crowd feature Θt and the robot state, with the corresponding state value ζ t can be estimated as:

[0093]

[0094] Using the state value ζ t replace in the objective function

[0095] where, Ψ ζ (·) is a fully connected layer containing the ReLU activation function, and W ζ is the network weight.

[0096] Step S1024: Based on the robot state estimate, the value network, and the preset reward function, use the reinforcement learning framework to determine the optimal navigation strategy for the robot to achieve safe and efficient navigation of the robot in a dense crowd environment.

[0097] The task can be described as a robot moving in an environment with n dynamic pedestrians and reaching the target without any collisions. In the reinforcement learning framework, this task can be formulated as a partially observable Markov decision problem. For each agent (pedestrian or robot), the state can be divided into observable and unobservable parts.

[0098] At each time step, each agent can observe its own complete state (observable state and unobservable state) and other agents. Therefore, the joint state of the entire system at time t can be defined as Assume that the robot can determine the action command and adjust the speed in a timely manner according to the navigation strategy π at each time step. The goal is to find an optimal strategy π that maximizes the expected reward to maximize the expected return. The objective function is constructed as follows:

[0099]

[0100]

[0101] In the formula represents the reward obtained at time t, a t represents the action at time t; A represents the action space, γ ∈ (0, 1) is a discount factor, and V * is the optimal value function, is the state transition probability from time t to time t Δ = t + Δt. v pref is the preferred speed. T is the final time step; t * is the future time step starting from the current time step t.

[0102] Among them, based on the set risk area, the reward function enables the robot to reach the target position while ensuring the safety, time efficiency, and path smoothness of the robot navigation; the risk area is determined by the pedestrian speed and crowd density.

[0103] By considering the actions that pedestrians may perform in the current state to form an action space as the risk area of the robot, and by designing a reward function to guide the robot to avoid this area, the understanding of human behavior by the robot can be effectively improved. When the pedestrian's moving speed is low, the pedestrian can move in any direction, while when the speed is high, the pedestrian will move more forward rather than laterally. In addition, as the crowd density increases, pedestrians are more accepting of the approach of other people and robots. Therefore, the action space of pedestrians should be related to the pedestrian speed and the surrounding crowd density. The present invention designs the risk area of the robot based on the pedestrian speed and crowd density, and this risk area is composed of a pedestrian safety area and a comfort area.

[0104] As shown in Fig. 2(a), the black dotted circle in the area BCDE represents a pedestrian, and point A is the center of the pedestrian. All angles are defined with reference to the positive x-axis of the coordinate system, with counterclockwise angles set as positive and clockwise angles set as negative. The fan-shaped area ABC shown in Fig. 2(a) is designed based on the human speed, and the relationship between its fan angle and radius and the human speed is as follows:

[0105] r = m v v + d s

[0106]

[0107] Among them, m v is the speed weight, v h is the pedestrian speed, d s = r h + d m is the fan radius when the pedestrian is stationary, r h is the radius of the pedestrian itself, d m is the minimum safety distance required by the pedestrian. As the pedestrian speed v h increases, the fan angle of the fan-shaped area decreases exponentially while the radius increases linearly, and vice versa. The circular ring area formed by DE and the pedestrian in Fig. 2(a) ensures the overall safety of the pedestrian during walking, and the width is d m , and the fixed value is taken as 0.1 m.

[0108] As shown in regions B'BCC' and E'EDD' in Figure 2(b), that is, the blue-filled regions, a comfortable area for pedestrians is designed on the periphery of the safety area, which ensures the comfort of pedestrians. Given that in a densely populated environment, people are generally more accepting of the approach of others, in this chapter, the social distance d social is regarded as a function of the density of the surrounding population rather than a fixed quantity:

[0109]

[0110] where ρ den represents the number of pedestrians within a circle with a radius of 2m centered on the current pedestrian.

[0111] The radius of the sector region in the final risk area of the robot is:

[0112] r′ = m v v + d s + d social

[0113] As shown in Figure 2(b), the calculation formulas for the boundary points B' and C' of the sector region are as follows:

[0114]

[0115] In the formula, (x a , y a ) is the coordinate of the center point A of the pedestrian, and β is the direction of the pedestrian's speed relative to the reference axis. A comfortable area is set on the periphery of the safety area around the pedestrian, and the width of its circular ring is d social . As the population density ρ den increases, the radius of the sector region and the width of the circular ring decrease linearly, and vice versa.

[0116] To determine whether the robot enters the risk area, the risk area is divided into a sector area and a circular ring area, corresponding to Figure 3 the red and yellow filled parts in. To determine whether the robot enters the sector area, two conditions need to be met. One is the distance condition, that is, d r ≤ r′ - r h , where d r is the minimum separation distance between the robot and the pedestrian within the time interval Δt. The other is the angle condition, that is, β - α / 2 ≤ γ ≤ β + α / 2, where γ is the angle of the center position of the robot relative to the reference axis. Obviously, the robot in the figure meets the angle condition but does not meet the distance condition, so it does not enter the sector area. In addition, to determine whether the robot enters the circular ring area, one condition needs to be met, that is, d r ≤ r h + d m + d cObviously, the robot in the figure does not meet this condition, so it does not enter the circular area of the risk zone either.

[0117] Robot navigation is a multi-objective optimization problem, and each objective corresponds to a different navigation subtask. The present invention decomposes it into the following subtasks. First is the target arrival subtask, which aims to ensure that the robot can accurately reach the specified target position. Second is the collision avoidance subtask, which aims to ensure that the robot does not collide during navigation. Third is the time optimization subtask, which aims to ensure that the robot reduces the time to complete navigation. Fourth is the trajectory smoothing subtask, which aims to ensure that the robot completes navigation with a smooth trajectory. Based on the above risk zone, the present invention designs a reward function to enable the robot to reach the target while ensuring the safety, time efficiency, and path smoothness of robot navigation, and its specific expression is as follows:

[0118]

[0119]

[0120]

[0121]

[0122] r smooth = -0.01(|ω t -ω t-1 | + |v t -v t-1 |)

[0123] where r goal , r safe , r time , r smooth are the rewards corresponding to the above four subtasks. r goal rewards the robot for reaching the target point to ensure that the robot basically completes the navigation task. r safe penalizes the robot for colliding or entering the robot risk zone to ensure navigation safety and pedestrian comfort. r time in which t limit is a constant representing the specified upper limit of navigation time. The closer the navigation time to the target point is to the upper limit, the greater the penalty, ensuring the time efficiency of navigation. r smooth penalizes the changes in the linear velocity v and angular velocity ω. The greater the degree of change, the greater the penalty, ensuring that the robot's trajectory is smoother.

[0124] Embodiment 2

[0125] In one or more embodiments, a safe and efficient service robot navigation system in a dense crowd environment is adopted, which includes:

[0126] A crowd status acquisition module for acquiring crowd status information;

[0127] A decision-making navigation module for making decision-making navigation for the robot according to the crowd status information and the status information of the robot in a dense crowd environment;

[0128] The decision-making navigation module further includes:

[0129] A directed graph construction sub-module for determining the relative spatial position relationship between the robot and pedestrians in a dense crowd environment according to the crowd status information, and modeling the dense crowd and the robot as a directed graph;

[0130] A feature extraction sub-module for extracting social interaction features and spatio-temporal features from the directed graph by using a value network based on spatio-temporal graph attention, and then combining a multi-head self-attention mechanism to obtain crowd features;

[0131] A feature extraction sub-module for determining the optimal navigation strategy of the robot by using a reinforcement learning framework based on the robot state estimation value, the value network and a preset reward function, so as to realize the safe and efficient navigation of the robot in a dense crowd environment;

[0132] Wherein, on the basis of setting a risk area, the reward function enables the robot to reach the target position while ensuring the navigation safety, time efficiency and path smoothness of the robot; the risk area is determined by the pedestrian speed and the crowd density.

[0133] Specifically, in the feature extraction sub-module, the objective function corresponding to determining the optimal navigation strategy and the optimal speed of the robot by using a reinforcement learning framework is:

[0134]

[0135] Wherein, represents the reward obtained at time t; A represents the action space; a t represents the action of the robot at time t; γ∈(0,1) is a discount factor, V * is the optimal value function, is the state transition probability from time t to time t Δ =t+Δt; Δt is the time interval; v pref is the preferred speed; is the joint state of the robot at time t; is the joint state of the robot at t Δ moment; is the optimal navigation strategy; T is the final time step; t * is the future time step starting from the current time step t.

[0136] In the feature extraction sub-module, the expression of the reward function is as follows:

[0137]

[0138]

[0139]

[0140]

[0141] r smooth = -0.01(|ω t - ω t-1 | + |v t - v t-1 |)

[0142] where r goal is the reward for the robot to reach the target point; r safe is the penalty for the robot to have a collision behavior or enter the risk area of the robot; in r time t limit is a constant representing the upper limit of the specified navigation time; r smooth is the penalty for the change of the linear velocity v and the angular velocity ω; the subscript t of the linear velocity v and the angular velocity ω represents the t-th moment; d g represents the distance between the robot and the target point, represents the reward obtained at the t-th moment; represents the joint state of the entire system at the t-th moment; a t represents the action at the t-th moment; d r represents the minimum separation distance between the robot and the pedestrian within the time interval Δt; r′ represents the radius of the sector area in the final risk area of the robot; r h represents the radius of the pedestrian itself.

[0143] It should be noted here that each module in the embodiments of the present invention corresponds to each step in the above embodiments, and the specific implementation process is the same, which will not be elaborated here.

[0144] Embodiment Three

[0145] This embodiment provides a computer-readable storage medium, on which a computer program is stored, and when the program is executed by a processor, it implements the steps in the method for safe and efficient service robot navigation in a dense crowd environment as described above.

[0146] Embodiment Four

[0147] A computer program product includes a computer program / instructions. When the computer program / instructions are executed by a processor, the steps in the safe and efficient service robot navigation method in a dense crowd environment as described above are implemented.

[0148] Embodiment 5

[0149] This embodiment provides a service robot, including a robot body, a memory, a processor, and a computer program stored on the memory and executable on the processor. When the processor executes the program, the steps in the safe and efficient service robot navigation method in a dense crowd environment as described above are implemented.

[0150] The present invention is described with reference to the flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each flow and / or block in the flowcharts and / or block diagrams, and the combination of flows and / or blocks in the flowcharts and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to the processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing devices to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing devices generate a device for implementing the specified functions in one Figure 1 one flow or multiple flows and / or blocks Figure 1 one block or multiple blocks.

[0151] The above are only the preferred embodiments of the present invention and are not used to limit the present invention. For those skilled in the art, the present invention can have various changes and modifications. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A safe and efficient navigation method for service robots in a dense crowd environment, characterized in that Including: Obtain crowd status information; Based on the crowd status information and the status information of the robot in a dense crowd environment, make decision-making navigation for the robot; The specific process is as follows: Based on the crowd status information, determine the relative spatial position relationship between the robot and pedestrians in a dense crowd environment, and model the dense crowd and the robot as a directed graph; Use the value network based on spatio-temporal graph attention to extract social interaction features and spatio-temporal features from the directed graph, and then combine the multi-head self-attention mechanism to obtain crowd features; Based on the mapping relationship between the crowd features and the robot status and the robot status estimate value, obtain the robot status estimate value; Based on the robot status estimate value, the value network and a preset reward function, use the reinforcement learning framework to determine the optimal navigation strategy of the robot to achieve safe and efficient navigation of the robot in a dense crowd environment; Among them, on the basis of setting a risk area, the reward function enables the robot to reach the target position while ensuring the safety, time efficiency and path smoothness of the robot navigation; the risk area is determined by the pedestrian speed and crowd density.

2. The safe and efficient service robot navigation method in a dense crowd environment according to claim 1, characterized in that, The objective function corresponding to the optimal navigation strategy and optimal speed of the robot determined by using the reinforcement learning framework is: Among them, represents the reward obtained at time t; A represents the action space; a t represents the action of the robot at time t; γ ∈ (0, 1) is a discount factor, V * is the optimal value function, is the state transition probability from time t to time t Δ = t + Δt; Δt is the time interval; v pref is the preferred speed; is the joint state of the robot at time t; is the joint state of the robot at t Δ time step; is the optimal navigation strategy; T is the final time step; t * is the future time step starting from the current time step t.

3. The safe and efficient service robot navigation method in a dense crowd environment according to claim 1, wherein The expression of the reward function is: r smooth = -0.01(|ω t - ω t-1 | + |v t - v t-1 |) where r goal is the reward for the robot to reach the target point; r safe is the penalty for the robot to have a collision behavior or enter the robot's risk area; r time in which t limit is a constant representing the specified upper limit of the navigation time; r smooth is the penalty for the changes in the linear velocity v and the angular velocity ω; the subscript t of the linear velocity v and the angular velocity ω represents the t-th moment; d g represents the distance between the robot and the target point, represents the reward obtained at the t-th moment; represents the joint state of the entire system at the t-th moment; a t represents the action at the t-th moment; d r represents the minimum separation distance between the robot and the pedestrian within the time interval Δt; r′ represents the radius of the sector area in the robot's final risk area; r h represents the radius of the pedestrian itself.

4. The safe and efficient service robot navigation method in a dense crowd environment according to claim 1, wherein, The value network based on spatio-temporal graph attention includes a person-person attention layer and a robot-person attention layer; the person-person attention layer is used to extract person-person interaction features, and the robot-person attention layer is used to extract robot-person interaction features, and fuse the person-person interaction features and the robot-person interaction features into social interaction features; based on the social interaction features and the multi-head self-attention mechanism, extract the corresponding time features.

5. A safe and efficient navigation method for service robots in a dense crowd environment, characterized in that, Including: A crowd status acquisition module, which is used to obtain crowd status information; A decision-making navigation module, which is used to make decision-making navigation for the robot according to the crowd status information and the status information of the robot in a dense crowd environment; The decision-making navigation module further includes: A directed graph construction sub-module, which is used to determine the relative spatial position relationship between the robot and pedestrians in a dense crowd environment according to the crowd status information, and model the dense crowd and the robot as a directed graph; A feature extraction sub-module, which uses the value network based on spatio-temporal graph attention to extract social interaction features and spatio-temporal features from the directed graph, and then combines the multi-head self-attention mechanism to obtain crowd features; A feature extraction sub-module, which based on the robot status estimate value, the value network and a preset reward function, uses the reinforcement learning framework to determine the optimal navigation strategy of the robot to achieve safe and efficient navigation of the robot in a dense crowd environment; Among them, on the basis of setting a risk area, the reward function enables the robot to reach the target position while ensuring the safety, time efficiency and path smoothness of the robot navigation; the risk area is determined by the pedestrian speed and crowd density.

6. The safe and efficient service robot navigation system in a dense crowd environment according to claim 5, characterized in that, In the feature extraction sub-module, the objective function corresponding to the optimal navigation strategy and optimal speed of the robot determined by using the reinforcement learning framework is: where, represents the reward obtained at time t; A represents the action space; a t represents the action of the robot at time t; γ ∈ (0, 1) is a discount factor, V * is the optimal value function, is the state transition probability from time t to time t Δ = t + Δt; Δt is the time interval; v pref is the preferred speed; is the joint state of the robot at time t; is the joint state of the robot at t Δ time; is the optimal navigation strategy; T is the final time step; t * is the future time step starting from the current time step t.

7. The safe and efficient service robot navigation system in a dense crowd environment according to claim 5, characterized in that In the feature extraction sub-module, the expression of the reward function is: r smooth = -0.01(|ω t - ω t-1 | + |v t - v t-1 |) Among them, r goal is the reward for the robot to reach the target point; r safe is the penalty for the robot to have a collision behavior or enter the robot risk area; r time In which, t limit is a constant representing the upper limit of the specified navigation time; r smooth is the penalty for the changes in the linear velocity v and the angular velocity ω; the subscript t of the linear velocity v and the angular velocity ω represents the t-th moment; d g represents the distance between the robot and the target point, represents the reward obtained at the t-th moment; represents the joint state of the entire system at the t-th moment; a t represents the action at the t-th moment; d r represents the minimum separation distance between the robot and the pedestrian within the time interval Δt; r′ represents the radius of the fan-shaped area in the final risk area of the robot; r h represents the radius of the pedestrian himself.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by a processor, it implements the steps in the safe and efficient service robot navigation method in a dense crowd environment as described in any one of claims 1-4.

9. A computer program product, comprising a computer program / instructions, characterized in that, When the computer program / instructions are executed by a processor, it implements the steps in the safe and efficient service robot navigation method in a dense crowd environment as described in any one of claims 1-4.

10. A service robot, comprising a robot body, a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the steps in the safe and efficient service robot navigation method in a dense crowd environment as described in any one of claims 1-4.

Images