Robot safety control method and system oriented to man-machine joint operation environment

Abstract

The invention discloses a robot safety control method and system oriented to a man-machine co-operation environment, and the system considers random human factor risks in the environment in a planning level, and plans an optimal safety path, thereby avoiding man-machine conflicts in advance in the planning level, and guaranteeing the safety of man-machine co-operation. The method comprises the following steps: generating a multi-path initial solution set based on an optimal priority and backtracking mechanism by utilizing a reinforcement learning exploration environment; a redundant path screening mechanism is adopted, redundant similar paths are deleted, and the diversity of a multi-path initial solution set is reserved; and finally, in combination with a random human factor risk in an environment, calculating a path risk from multiple dimensions, and generating an optimal security path. According to the method, the problem that a traditional optimal path planning and Markov decision process cannot be suitable for a dynamic environment can be solved, risks are avoided in advance on the planning level, and an optimal safety control strategy and an optimal operation path are provided for operation of the mobile robot.

Description

Technical Field

[0001] This invention relates to the field of robot control technology, and in particular to a robot safety control method and system for human-robot collaborative working environments. Background Technology

[0002] With the advancement of artificial intelligence technology, the demand for robots to perform tasks is increasing, and in some special scenarios, robots need to work collaboratively with humans. How to ensure that robots avoid collisions with humans (or minimize collisions) while completing tasks has become an urgent problem to solve. However, the control technology for robots performing tasks in such scenarios still needs improvement, especially in environments where humans are dynamic factors. The uncertainty of human movement brings many risks to robot task performance, thus placing higher demands on robot task control and safe path planning. One typical application scenario is human-robot collaboration in smart warehousing, where collaborative work effectively improves work efficiency. Currently, in smart warehousing environments, robots need to reach designated locations according to planned paths and then complete tasks such as moving goods. In manned warehouses, robots need to plan a safe path to minimize collisions with humans. Due to the uncertainty of human movement in warehouses, the probability of robot-human collisions is greatly increased. Existing robot control systems do not consider the dynamic characteristics of the warehouse environment at the planning level, thus failing to provide safe path planning and control strategies to ensure the safety of both humans and robots.

[0003] Existing research shows that simple optimal path planning (e.g., A*) can enable robots to move from their starting position to their target position with optimal efficiency. However, it does not consider the uncertainties and risks in the environment, meaning it cannot plan safe paths for robots in dynamic environments. While reinforcement learning algorithms can effectively handle path planning in dynamic environments, their training process is unstable in multi-person environments, often leading to inconsistent results, and therefore cannot significantly improve safety. Summary of the Invention

[0004] Purpose of the invention: The purpose of this invention is to provide a robot safety control method and system for human-robot collaborative working environments. By considering the risks brought about by uncertain factors in the environment, the method plans the optimal working path for the robot to avoid risks at the planning level, thereby improving the safety of the robot.

[0005] Technical solution: A robot safety control system for human-robot collaborative working environments, which considers random human-caused risks in the environment and plans the optimal safe path; including a model definition module, a risk assessment module, and a path planning module; wherein humans are considered as dynamic environmental factors;

[0006] The model definition module uses the MDP method to model the environment, transforming abstract problems into mathematical models to solve the robot's path planning problem.

[0007] The risk assessment module predicts and assesses the collision risk caused by random human movement in the environment, including a random simulation module, a risk simulation module, and a risk calculation module.

[0008] The stochastic simulation module simulates the movement rules of uncertain factors by defining a stochastic simulation model;

[0009] The risk simulation module, based on a motion model, simulates human movement trajectories multiple times.

[0010] The risk calculation module is used to calculate risk information present at specific locations in the environment;

[0011] The path planning module uses a combination of reinforcement learning and path planning algorithms to plan safe paths, including a multi-path generator, a redundant path filter, and a decision unit.

[0012] The multi-path generator, based on the Q-table obtained through reinforcement learning, employs a best-first and backtracking mechanism to acquire multiple optimal / suboptimal paths; the resulting path set is used as the initial path set.

[0013] The redundant path filter is used to select from the initial path set. Filter out highly similar paths;

[0014] The decision-making unit selects the safest path by taking into account human risks.

[0015] Furthermore, by employing the robot safety control system as described in claim 1, the optimal safe path for the robot to perform its tasks is generated under uncertain conditions; this includes the following steps:

[0016] S1. The MDP method is used to establish a mobile model of the robot and to evaluate the collision risk value caused by random human movement in the environment.

[0017] S2, based on a multi-path generator, generates a set of initial paths from the starting position to the target position.

[0018] S3 uses a redundant path filter to filter the initial path set and obtain a multi-path set.

[0019] S4, based on the risk value, the decision-making unit selects the safest path from the set of multiple paths.

[0020] Further, through the risk assessment module, the collision risk brought by the random movement of people in the environment is predicted and evaluated; the steps are as follows:

[0021] S211. Assume that humans select the corresponding action a from the action set A = {forward, backward, left, right, wait} with probability p; and set the following principles: no conflict occurs between any two human individuals; humans tend to move in the target direction; s , ,

[0032] , ,

[0031] ,

[0028] , , , , , , v ,

[0029] , , , ,

[0027] , ,

[0030] From the action set A = {forward, backward, left, right, wait}, select the corresponding action a; and set the following principles: no conflict occurs between any two human individuals; humans tend to move in the target direction;

[0022] S212. Use the random simulation method to construct a human motion model;

[0023] Assume that a person k is at position at time step t After taking action a t Moves to the position at time step t + 1 Denoted as For all feasible actions The optimal action a * Is defined as:

[0024]

[0025] Where, the Manh(·) function calculates the Manhattan distance from To the target position Of;

[0026] Assume that C represents the set of priority actions, Including all optimal actions a * And the conditional probability distribution of action a t Is defined as:

[0027]

[0028] Where, |·| represents the number of actions in the corresponding action set, ζ ∈ [0, 0.2),

[0029] For a person k, assume Is the state observed at time step t = 0. Starting from the observation state The conditional probability that vertex v is occupied by person k at time step t is:

[0030]

[0031] Where, N s Represents the number of simulations, and N v Records the number of times vertex v is visited at time step t; 0 < t ≤ T, and T is the termination time;

[0032] S213. The risk value R(v, t) generated by K people at vertex v is calculated by the risk calculation module as:

[0033]

[0034] Finally, we obtain the human risk set ξ={ξ1,ξ2,…,ξ} for the time series t∈(0,T]. T}, from this point on ξ t Obtain the risk value of vertex v at any time step t.

[0035] Furthermore, the search process of the multi-path generator follows the principle that for any state, there are multiple possible actions and the corresponding Q-values ​​satisfy Q. max Q min Given the conditions, actions with larger Q values ​​are prioritized, followed by actions with smaller Q values; obtaining the optimal path includes the following steps:

[0036] SA1 constructs an optimal path based on the action value function in MDP;

[0037] SA2, backtracking sequentially to a state with multiple actions, and searching for other optimal paths;

[0038] SA3: Repeat step SA2 to complete the search for all actions, and use the set of all optimal paths obtained as the initial path set.

[0039] Furthermore, from the initial path set The process of filtering out highly similar paths includes the following steps:

[0040] S31, from the initial path set The baseline path is obtained;

[0041] S32, based on these benchmark paths, using a diversity metric, an iterative method is used to obtain a multi-path set X; this iterative method uses the benchmark path X... b Based on this, similar paths are eliminated using diversity metrics, and then the least similar path is selected from the remaining paths as the base path for the next iteration.

[0042] For any two paths x i and x j The distance d at time step t t Defined as:

[0043]

[0044] Among them, (D) x D y ) represents the size of the map, and Manh(·) represents the state. and Manhattan distance between them;

[0045] Cosine similarity c at time step t t Defined as:

[0046]

[0047] Where, vector vector The state at time t0;

[0048] Define the diversity metric function Div(·) for any two paths as follows:

[0049]

[0050] By setting a lower bound function LB(x) i Ensure that the selected path differs significantly from all baseline paths; lower bound function LB(x) i The expression for ) is:

[0051] LB(x i ) = min xj∈X Div(x i ,x j )

[0052] in, Representing path x i and x j The termination time step; min(·) and max(·) are the minimum value function and the maximum value function, respectively; Representing path x i and x j The length.

[0053] Furthermore, the decision-making unit selects the safest path from the set of multiple paths, including the following steps:

[0054] S41, for a given path x i The formula for calculating total risk is:

[0055]

[0056] Among them, V r (v) represents the vertex collision risk at position v, E r (v) represents the edge conflict risk from position v at time step t to position v′ at time step t+1;

[0057] Introducing the human risk set ξ at time step t t Then, there is:

[0058]

[0059] S42, Find the path x with the minimum risk in the multi-path set X. * for:

[0060]

[0061] in, The function obtains Risk(x) i The path corresponding to the minimum value.

[0062] Compared with the prior art, the significant advantages of this invention are as follows:

[0063] 1. The control method of this invention employs a phased path planning algorithm, from generating an initial multi-path set to filtering out redundant paths, and finally combining risk information to determine the optimal safe path. This solves the problem of safe path planning for robots in manned environments.

[0064] 2. The control method of the present invention takes into account the risks brought about by uncertain factors in the environment, and plans the best operation path for the robot to avoid risks at the planning level, thereby improving the safety of the robot.

[0065] 3. This invention proposes an optimal safe path planning algorithm based on reinforcement learning in manned environments. Compared with traditional optimal path planning algorithms and MDP algorithms, the optimal safe path planning algorithm architecture of this invention is more effective under simulated experimental conditions and can effectively reduce collisions between robots and humans. Attached Figure Description

[0066] Figure 1 This is a schematic diagram of the architecture of the control system of the present invention;

[0067] Figure 2 This is a schematic diagram of the risk assessment module;

[0068] Figure 3 This is a schematic diagram of the path planning module;

[0069] Figure 4 The diagram illustrates the multi-path generation process, where (a) constructs an optimal path based on the action value function in the MDP, (b) backtracks in an orderly manner to states with multiple actions based on the path constructed in (a) to search for other optimal paths, (c) obtains the optimal path when backtracking to a higher-level state, and (d) obtains a suboptimal path based on other optimal paths in (b).

[0070] Figure 5 This is a flowchart of the control method of the present invention;

[0071] Figure 6 This is a schematic diagram simulating a warehouse environment.

[0072] Figure 7 The diagram shows a comparison of job paths obtained using different algorithms. (a) represents the job path obtained using the control method of this invention, (b) represents the job path obtained using the MDP algorithm, and (c) represents the job path obtained using the A* algorithm. Detailed Implementation

[0073] The present invention will now be described in further detail with reference to the accompanying drawings and specific embodiments.

[0074] This invention provides a robot safety control system for human-robot collaborative working environments, the system architecture of which is shown in the attached figure. Figure 1 As shown, the control system consists of the following three parts: model definition module, risk assessment module, and path planning module; among them, people are a dynamic factor in the environment, and the randomness of people's movement in the environment is the cause of risk.

[0075] The model definition module is primarily responsible for problem modeling, transforming abstract problems into mathematical models that allow for the solution of planning problems using a solveable approach. In this embodiment, the Markov Decision Process (MDP) method is mainly used to model the environment and then solve the control problem of the mobile robot.

[0076] The risk assessment module primarily predicts and evaluates the collision risk caused by random human movement in the environment. In this embodiment, a stochastic simulation method is used to calculate the human risk assessment within a two-dimensional grid-based experimental environment. Figure 2 As shown, the risk assessment module mainly consists of a stochastic simulation module, a risk simulation module, and a risk calculation module:

[0077] The stochastic simulation module simulates the movement rules of uncertain factors by defining a stochastic simulation model;

[0078] The risk simulation module, based on a motion model, simulates human movement trajectories multiple times.

[0079] The risk calculation module is used to calculate risk information at specific locations in the environment.

[0080] The path planning module uses a combination of reinforcement learning and path planning algorithms to plan safe paths. For example... Figure 3 As shown, the path planning module mainly consists of a multi-path generator, a redundant path filter, and a decision unit:

[0081] The multi-path generator, based on a learned Q-table, employs a best-first and backtracking mechanism to obtain multiple optimal / suboptimal paths; the resulting path set is used as the initial path set.

[0082] Redundant path filter, used to select from the initial set of paths By filtering out highly similar paths, the diversity of paths is still maintained even after reducing the number of paths in the set.

[0083] Decision-making unit: mainly selects the safest path, adopts a method that takes into account human risk, and considers the impact of the person's location on the path; the total risk value of the path depends not only on the position v at time step t, but also on potential future risks.

[0084] To address the problems of existing technologies, this invention proposes a robot safety control method that considers the risks of movement due to uncertainties. This method employs a multi-stage path planning algorithm based on Q-learning, consisting of three stages: The first stage generates multiple paths from the starting position to the target position; the second stage eliminates redundant paths from the existing path set; and the third stage, combining risk information of uncertainties, determines the safest path from the path set after eliminating redundant paths. Because this invention considers uncertainties in the environment, it mitigates risks in advance at the planning level, providing a safe path for the mobile robot to complete tasks in uncertain environments. The flowchart of the robot safety control method is as follows: Figure 5 As shown, the steps are as follows:

[0085] Step 1: Build a mobile model for the robot and assess the collision risk caused by random human movement in the environment;

[0086] Step 11: Establish the robot's motion model using the model definition module;

[0087] In this embodiment, the robot needs to complete a specific task at a designated location, and the robot acts as an intelligent agent. Consider an intelligent agent operating in an environment where humans move randomly, where the agent shares the same environment with K individuals, and each agent has a corresponding target location. Humans move in the environment using random strategies and without control, which leads to uncertainty for the agent when accessing its target location. Therefore, at any discrete time step t, the agent may conflict with some person k (k∈{1,…,K}).

[0088] The robot works on a two-dimensional grid (D) x ×D y The world can be represented as G =<V,E> Among them, (D) x D y Let V be the size of the map, V represent the set of vertices, and E represent the set of edges. Using MDP to model the robot's movement, the robot's movement model is represented as:

[0089]

[0090] Where S is the set of states of the agent. Where v x and v y Represents the coordinates of vertex v in the grid world; s0∈S is the initial state of the agent, s g ∈S is the target state of the agent;

[0091] A is the finite set of actions that the agent can take;

[0092] P is the state transition probability of the environment: S×A×S→[0,1];

[0093] R is the reward function: Where s, s′∈S, s′ is the state at the next time step, a t ∈A;

[0094] γ is the discount factor: γ∈[0,1].

[0095] An agent takes an action a at any time step t. t ∈A(s t After that, it will be with probability p(s) t+1 |s t ,a t From state s t Transition to state s t+1 Among them, satisfying At this point, the agent will receive an immediate reward R(s). t ,a t ,s t+1 The discount factor γ is used to balance the weights of immediate and future rewards. The optimal policy is solved using the Q-learning algorithm via the Bellman equation, and the action-value function Q is stored in a Q-table, where each Q-value corresponds to a specific state-action pair. Q(s,a) is updated according to the following formula (1):

[0096] Q(s,a)←Q(,a)+α(R(s,a,s′)+γmax a∈A(s′) Q(s,a′)-Q(s,a)) (2)

[0097] Where α represents the learning rate, and a′ is the action chosen by the state s′ at the next time step.

[0098] Unlike the classic MDP learning strategy, the MDP learner in this invention enhances the agent's ability to explore the environment by constructing a fully covered Q-table (i.e., covering the entire workspace as much as possible). To this end, when employing the ε-greedy strategy, the algorithm of this invention sets the greedy parameter ε to a larger value than that of the classic MDP to strengthen environment exploration.

[0099] Step 22: Through the risk assessment module, predict and evaluate the collision risk brought by the random movement of people in the environment;

[0100] Assume that humans select the corresponding action a from the action set A = {forward, backward, left, right, wait} with probability p. Since people move autonomously without being controlled by the system, this probability p a is unknown. To more realistically simulate human natural movement behavior, the following principles are set: a has unknownness. To more realistically simulate human natural movement behavior, the following principles are set:

[0101] ① There is no conflict between any two human individuals;

[0102] ② Humans tend to move in the target direction;

[0103] Based on the above rules, a human motion model is constructed using the random simulation method as follows:

[0104] The position of a person k at time step t takes the action a t and then moves to the position at time step t + 1 denoted as For all feasible actions the optimal action a * is defined as:

[0105]

[0106] where the Manh(·) function calculates the Manhattan distance from to the target position The Manhattan distance.

[0107] Let C denote the set of preferred actions, which contains all actions that satisfy Equation (3). Preferred actions are given a higher conditional probability to encourage human movement to be goal-oriented. All feasible actions a t The conditional probability distribution is defined as:

[0108]

[0109] where |·| represents the number of actions in the corresponding action set, ζ ∈ [0, 0.2),

[0110] For a person k, let be defined as the state observed at time step t = 0 ([[]] represents the vertex, that is, person k occupies vertex v). Starting from the observation state [[[]] until the termination time T, the human motion model is run in the random simulation module to predict the human movement path. The conditional probability that vertex v is occupied by person k at time step t (0 < t ≤ T) is:

[0111]

[0112] Where, N s N represents the number of simulations. v Record the number of times vertex v is visited at time step t.

[0113] The risk calculation module calculates the risk value R(v,t) of K individuals at vertex v as follows:

[0114]

[0115] Finally, we obtain the human risk set ξ={ξ1,ξ2,…,ξ} for the time series t∈(0,T]. T}, from which we can directly obtain ξ t Obtain the risk value of vertex v at any time step t.

[0116] Step 2: Based on the multi-path generator, generate a set of initial paths from the starting position to the target position;

[0117] The search process of the multipath generator follows these principles: for example Figure 4 In the middle state, there are multiple possible actions and the corresponding Q values ​​satisfy Q. max Q min If the conditions are met, the action with the larger Q value is prioritized, followed by the action with the smaller Q value. If two or more actions have the same Q value, one of them is randomly selected.

[0118] The multi-path generator employs a best-first and backtracking mechanism, performing the following steps:

[0119] Step A1: Construct an optimal path based on the action value function in the MDP, such as... Figure 4 As shown in (a) in the figure.

[0120] Step A2 involves sequentially backtracking to states with multiple actions to search for other optimal paths, such as based on... Figure 4 The optimal path generated in (a) is as follows: Figure 4 As shown in (b).

[0121] Step A3: Repeat step A2 to complete the search for all actions, and use the resulting path set as the initial path set.

[0122] like Figure 4 (b) in s t All previous states and Figure 4 The same applies to (a) in the previous example. Similarly, when backtracking to a higher-level state, the optimal path can be obtained as follows: Figure 4As shown in (c) of the diagram. Wherein, based on... Figure 4 (b) generates a suboptimal path, such as Figure 4 As shown in (d) in the diagram. The initial set of paths generated by the multipath generator. Both methods allow for setting reasonable time budgets for different map sizes. This method prioritizes constructing the optimal path and only generates the suboptimal path after all optimal paths have been completed. This strategy significantly improves path generation efficiency and reduces memory consumption.

[0123] Step 3: Use a redundant path filter to filter paths;

[0124] The redundant path filter starts from the initial path set. By filtering out highly similar paths, the diversity of paths is maintained even after reducing the number of paths in the set. This operation involves two steps:

[0125] Step 31, from the initial path set The baseline path is obtained;

[0126] Step 32: Based on these baseline paths, a multi-path set X is obtained through an iterative method using a diversity metric. This iterative method uses the baseline path X... b Based on this, similar paths are eliminated using diversity metrics, and then the least similar path is selected from the remaining paths as the base path for the next iteration.

[0127] like Figure 6 As shown, based on the starting point s0 and the target point s g Two edge tracing paths with the greatest difference are selected as baseline paths. The pseudocode for the complete path selection process is shown in Table 1.

[0128] Table 1. Pseudocode of the path filtering process

[0129]

[0130]

[0131] In this invention, similar paths are filtered out using a diversity metric. This diversity metric comprehensively evaluates the distance metric and cosine similarity between paths. For any two paths x i and x j The distance d at time step t t Defined as:

[0132]

[0133] Among them, (D) x D y ) represents the size of the map, and Manh(·) represents the state. and The Manhattan distance between them.

[0134] Its cosine similarity c at time step t t Defined as:

[0135]

[0136] Where, vector vector This represents the state at time t0.

[0137] d t ∈[0,2] is used to quantify spatial position deviation, while c t ∈[0,2] is used to evaluate the diversity of motion directions. t The smaller, the more it indicates and The closer their spatial positions are; t The larger the value, the more it means the vector... and The larger the angle between them.

[0138] Based on equations (7) and (8), the diversity measurement index function Div(·) for any two paths is defined as:

[0139]

[0140] Among them, T x This represents the final time step of path x, where min(·) and max(·) are the minimum and maximum value functions, respectively; L x This represents the length of path x.

[0141] Therefore, Div(x) i ,x j The larger the value, the better the path x. i and x j The higher the difference between them, the lower the similarity; conversely, the smaller the value, the lower the difference, and the higher the similarity.

[0142] Furthermore, the lower bound function LB(x) i ) is defined as:

[0143]

[0144] In Equation (10), the use of the min function ensures that there are significant differences between the selected path and all benchmark paths.

[0145] Step 4: Based on the risk value, the decision-making unit selects the safest path from the set of multiple paths;

[0146] Based on the consideration of the impact of a person's location on the path, the total risk value of the path depends not only on the position v at time step t, but also on potential future risks.

[0147] For a given path x i The formula for calculating total risk is:

[0148]

[0149] Among them, V r (v) represents the vertex collision risk at position v, E r (v) represents the edge conflict risk from position v at time step t to position v′ at time step t+1.

[0150] Introducing the human risk set ξ at time step t t Then, equation (11) can be rewritten as:

[0151]

[0152] Therefore, the optimal safe path problem is transformed into finding the path x with the minimum risk from the path set X. * for:

[0153]

[0154] in, The function obtains Risk(x) i The path corresponding to the minimum value.

[0155] Select as Figure 6 The simulated environment shown has a 10x10 map, with black squares representing obstacles. The goal of the moving robot is to move from the starting position s0 to the target position s0. g Complete in s g The task at hand; the goal is to move from the starting position h0 to the target position h. g .

[0156] In problem modeling, the parameters are set as follows:

[0157] Learning rate α = 0.7

[0158] Discount factor γ = 0.9

[0159] Exploration rate ∈ = 0.7

[0160] Threshold (Algorithm 1) β = 0.2

[0161] The specific definition of the reward function in this embodiment is shown in Table 2.

[0162] Table 2 Definition of Reward Function

[0163] remuneration Definition method r＝-0.1 Basic step size cost r＝+1.0 Rewards for reaching the goal r＝-1.0 Collision with people

[0164] In this embodiment, the algorithm source code is written in Python, and the job path generated by the algorithm is as follows: Figure 7 Figure (a) in the figure.

[0165] Figure 7 In the examples, (b) and (c) are schematic diagrams of paths generated using the MDP algorithm and the A* algorithm, respectively. In the embodiments, multiple simulations of safe path planning were performed to verify the effectiveness of the algorithm of this invention from multiple dimensions, as detailed in Table 3.

[0166] Table 3 Multidimensional Assessment

[0167] method Average Conflict Rate Success rate A* 0.19±0.48 0.85±0.36 MDP 0.11±0.23 0.90±0.21 The path planning algorithm of this invention 0.03±0.17 0.97±0.17

[0168] in,

[0169] Average collision rate: the average number of collisions between a human and a robot; Success rate: the robot's success rate from s0 to s g The number of times, and during the movement, there was no collision with anyone.

[0170] Table 3 shows that, Figure 6 In the 10*10 grid environment shown, with the parameter settings in Table 2 and a termination time T=20, 100 path planning simulations were performed. In safe path planning, two metrics—average conflict rate and planning success rate—show that, compared to A... * The MDP algorithm in the path planning of this invention has advantages.

Claims

1. A robot safety control system for human-robot collaborative working environments, characterized in that, Considering random human-caused risks in the environment, the system plans the optimal safe path; it includes a model definition module, a risk assessment module, and a path planning module. Humans are a dynamic factor in the environment; The model definition module uses the MDP method to model the environment, transforming abstract problems into mathematical models to solve the robot's path planning problem. The risk assessment module predicts and assesses the collision risk caused by random human movement in the environment, including a random simulation module, a risk simulation module, and a risk calculation module. The stochastic simulation module simulates the movement rules of uncertain factors by defining a stochastic simulation model; The risk simulation module, based on a motion model, simulates human movement trajectories multiple times. The risk calculation module is used to calculate risk information present at specific locations in the environment; The path planning module uses a combination of reinforcement learning and path planning algorithms to plan safe paths, including a multi-path generator, a redundant path filter, and a decision unit. The multi-path generator, based on the Q-table obtained through reinforcement learning, employs a best-first and backtracking mechanism to acquire multiple optimal / suboptimal paths; the resulting path set is used as the initial path set. The redundant path filter is used to select from the initial path set. Filter out highly similar paths; The decision-making unit selects the safest path by taking into account human risks.

2. A robot safety control method for human-robot collaborative working environments, characterized in that, By employing the robot safety control system as described in claim 1, the optimal safe path for the robot to perform its tasks can be generated in an uncertain environment; The steps include the following: S1. The MDP method is used to establish a mobile model of the robot and to evaluate the collision risk value caused by random human movement in the environment. S2, based on a multi-path generator, generates a set of initial paths from the starting position to the target position. S3 uses a redundant path filter to filter the initial path set and obtain a multi-path set. S4, based on the risk value, the decision-making unit selects the safest path from the set of multiple paths.

3. The robot safety control method for human-machine collaborative working environments according to claim 2, characterized in that, The risk assessment module is used to predict and assess the collision risks caused by random human movement in the environment. The steps include the following: S211, Suppose that humans have a probability p a Choose the appropriate action 'a' from the action set A = {forward, backward, left, right, wait}; and set the following principles: no conflict occurs between any two human individuals; humans tend to move in the direction of the target. S212, a human motion model is constructed using a stochastic simulation method; Suppose a person k is at position t in time step t. Take action a t Then, move to time step t+1. Recorded as For all feasible actions Optimal action a * Defined as: The Manh(·) function calculates from To the target location Manhattan distance; Let C represent the set of priority actions. Includes all optimal actions a * Action a t The conditional probability distribution is defined as: Where |·| represents the number of actions in the corresponding action set, ξ∈[0,0.2). For a person k, let... For the state observed at time step t = 0, from the observed state Initially, the conditional probability that vertex v is occupied by person k at time step t is: Among them, N s represents the number of simulation times, and N v records the number of times vertex v is visited at time step t; 0 < t ≤ T, where T is the termination time; S213, the risk value R(v,t) generated by K individuals at vertex v is calculated by the risk calculation module as follows: Finally, we obtain the human risk set ξ={ξ1,ξ2,…,ξ} for the time series t∈(0,T]. T }, from this point on ξ t Obtain the risk value of vertex v at any time step t.

4. The robot safety control method for human-machine collaborative working environments according to claim 2, characterized in that, The search process of the multipath generator follows these principles: for any state, there are multiple possible actions and their corresponding Q values ​​satisfy Q... max Q min Given the conditions, actions with larger Q values ​​are prioritized, followed by actions with smaller Q values; obtaining the optimal path includes the following steps: SA1 constructs an optimal path based on the action value function in MDP; SA2, backtracking sequentially to a state with multiple actions, and searching for other optimal paths; SA3: Repeat step SA2 to complete the search for all actions, and use the set of all optimal paths obtained as the initial path set.

5. The robot safety control method for human-robot collaborative working environments according to claim 2, characterized in that, From the initial path set The process of filtering out highly similar paths includes the following steps: S31, from the initial path set The baseline path is obtained; S32, based on these benchmark paths, using a diversity metric, an iterative method is used to obtain a multi-path set X; this iterative method uses the benchmark path X... b Based on this, similar paths are eliminated using diversity metrics, and then the least similar path is selected from the remaining paths as the base path for the next iteration. For any two paths x i and x j The distance d at time step t t Defined as: Among them, (D) x D y ) represents the size of the map, and Manh(·) represents the state. and Manhattan distance between them; Cosine similarity c at time step t t Defined as: Where, vector vector The state at time t0; Define the diversity metric function Div(·) for any two paths as follows: By setting a lower bound function LB(x) i Ensure that the selected path differs significantly from all baseline paths; lower bound function LB(x) i The expression for ) is: in, Representing path x i and x j The termination time step; min(·) and max(·) are the minimum value function and the maximum value function, respectively; Representing path x i and x j The length.

6. The robot safety control method for human-machine collaborative working environments according to claim 2, characterized in that, The decision-making unit selects the safest path from the set of multiple paths, including the following steps: S41, for a given path x i The formula for calculating total risk is: Among them, V r (v) represents the vertex collision risk at position v, E r (v) represents the position from time step t to time step t+1. ′ The risk of border conflict; Introducing the human risk set ξ at time step t t Then, there is: S42, Find the path x with the minimum risk in the multi-path set X. * for: in, The function obtains Risk(x) i The path corresponding to the minimum value.

Images