A multi-robot task search method applied to a rescue environment

Abstract

The application discloses a kind of multi-robot task search methods applied to rescue environment, can realize multi-robot cooperation under the condition of unknown task position, sensor noise uncertainty, complete efficient, complete task search.The method comprises the following steps: discretization rescue environment and according to rescue environment and the information of robot participating in search, construct object-oriented multi-machine part observable Markov decision model to describe task search problem.Each robot judges whether it is trapped by judging its current state, if trapped, then adopt the guiding decision of belief clustering and path planning based on adaptive threshold, otherwise, according to the maximum reward of generated tree, make a decision.Each robot executes current execution action, obtains environment observation and reward value, updates the belief of each object, and determines the most likely state of each object at each location according to the belief of all robots, to guide the autonomous decision of robot next time.

Description

Technical Field

[0001] This invention relates to the field of task search, and in particular to a multi-robot task search method applicable to emergency rescue environments. Background Technology

[0002] The advantage of intelligent robots is their ability to reason about multiple objects in uncertain environments, replacing humans. For example, after a nuclear accident, robots can replace humans in tasks such as search and rescue operations and locating leak sources in high-radiation environments. After an accident, the locations of personnel and hazards are often unknown; therefore, determining the tasks to be completed in the environment is a crucial step in the rescue process—a multi-machine, multi-target search (MRMOS) process. However, multi-machine operations in unknown environments such as hazard identification and disaster relief are characterized by environmental uncertainty and numerous obstacles, increasing the difficulty. For example, the location of targets to be searched or contaminants to be cleaned is uncertain; collapses cause complex and dispersed obstacle distributions; and radiation particles can cause sensor data to shift. Search techniques using only deterministic models will lead to poor search results. To address these characteristics, partially observable decision processes (MIDMs) are a suitable model, using beliefs to express environmental uncertainty.

[0003] Considering that robots typically need to perform multi-objective searches in large-scale environments, the belief space, being equal to the combination of all possible positional states of an object, inevitably grows exponentially with the number of objects, causing the dimensionality curse and severely impacting search efficiency. Object-oriented partially observable Markov decision processes (OO-POMDPs) are an innovative framework that can decompose beliefs into each object to achieve dimensionality reduction. In rescue environments, where the environment is often large and there are many tasks, a single agent cannot complete the search for all tasks within limited time and energy constraints. Multi-agent systems, with their parallel and cooperative characteristics, can effectively improve search efficiency. While OO-POMDPs perform well with a single agent, their lack of defined interaction methods makes them unsuitable for multi-machine cooperative searches.

[0004] Therefore, how to fully leverage the advantages of intelligent robots to achieve accurate and complete task searching in situations with large search spaces, unknown target locations, complex distributions, and sensor errors, enabling multiple robots to express and share environmental states and make autonomous decisions based on the current state, ultimately achieving the goal of rapid searching for all tasks in the environment, has become an urgent problem to be solved. Summary of the Invention

[0005] In view of this, the present invention provides a multi-robot task search method in emergency rescue environments, enabling robots to autonomously achieve accurate and complete searches of various environmental scales under conditions of target uncertainty, complex distribution, and sensor observation errors. For information sharing, a maximum-minimum method is used, calculating and sharing the belief of each task at each location based on the information of each robot. For decision-making, an online planning algorithm guided by distress detection is used, where each robot selects the action with the greatest benefit based on its current state.

[0006] To achieve the above objectives, the technical solution of the present invention is as follows: the robot hardware comprises a lidar, a robot vehicle-mounted unit, and a mobile robot body; the method includes the following steps:

[0007] Step 1: Discretize the rescue environment. Based on the rescue environment and the information of the robots participating in the search, construct an object-oriented multi-machine partially observable Markov decision model to describe the task search problem.

[0008] Step 2: Each robot determines whether it is trapped based on its current state. If the robot is trapped, a threshold is first determined using an adaptive method, then the points with belief values ​​in the adaptive range are clustered, and then the A* method is used to guide the decision and determine the current action to be executed. If the robot is not trapped, the action with the highest reward is selected as the current action to be executed.

[0009] Step 3: Each robot performs its current action, obtains environmental observations and reward values, and updates the belief of each object based on the original belief and the observations.

[0010] Step 4: Synchronize the beliefs of all robots using the minimax method, determine the most likely state of each object at each location, and guide the robot's autonomous decision-making in the next moment.

[0011] Repeat steps two through four until all tasks within the environment have been found.

[0012] Furthermore, the entire rescue environment is discretized into L×L grids, where L is the environment scale, defined according to the actual situation.

[0013] Furthermore, based on information from the rescue environment and the robots participating in the search, an object-oriented, multi-machine, partially observable Markov decision model is constructed to describe the task search problem, specifically:

[0014] Information about the rescue environment and the robots involved in the search includes: prior environmental information and status. action Observations with sensor noise State transition model and reward function Constructing a quintuple

[0015] The state It consists of the state of each robot and each task.

[0016] action It consists of a set of {movement in four directions, observation, and search}.

[0017] Observations with sensor noise Defined by an event-based Gaussian model with errors (∈,σ); ∈ and σ are the expectation and variance of the Gaussian model, respectively.

[0018] State transition model and reward function Defined according to the actual situation.

[0019] This leads to an object-oriented, multi-machine, partially observable Markov decision model that describes the task search problem.

[0020] Furthermore, each robot determines whether it is trapped based on its current state. Specifically, each robot constructs a Monte Carlo tree based on the simulation. If a robot determines that all the rewards in the Monte Carlo tree are less than zero, then the robot is trapped; otherwise, the robot is not trapped.

[0021] Furthermore, if the robot becomes trapped, a threshold is first determined using an adaptive method. Then, points with belief values ​​within the adaptive interval are clustered, and the A* method is used for guidance decision-making to determine the current action to be executed. Specifically:

[0022] First, the left boundary of the adaptive threshold is determined using the following relationship:

[0023]

[0024] Where T L N is the left boundary. f Let N be the number of tasks found, and B be the total number of tasks. j Let P be the maximum value of the belief of robot j, O be the set of all locations, L be the scale of the environment, and the right boundary be set to a set value.

[0025] Then the robot clusters all positions of the belief value in the adaptive interval into m classes. The robot selects the point closest to itself as the guide point, uses the A* algorithm to plan the path, and takes the first planned action as its current action.

[0026] Conversely, it uses the action with the highest reward in the Monte Carlo tree as its own behavior.

[0027] Furthermore, the robot updates the beliefs of each object based on the original beliefs and observations, specifically as follows:

[0028] Since the rescue mission is independent, the belief can be decomposed into an object-oriented manner according to the mission, reducing the exponential belief space to a linear space.

[0029] After each robot performs its current action, the current observations and initial beliefs are used to update the beliefs of each object separately using an object-oriented belief update method:

[0030] b′ i (s i )=ηp(z i |s i )b i (s i )

[0031] Where b′ i (s i ) represents the updated belief, η is the normalization factor, and p(z) is the normalization factor. i |s i ) is in state s i z was observed under the condition i The probability, z i It is each agent's observation of object i, b i (s i ) represents a state of s i The belief at the time; s i This represents the state of the i-th object.

[0032] Furthermore, based on the belief of synchronizing all robots using the max-min method, the synchronization process is as follows:

[0033] ① Let S be the set of all raster numbers, L represents the environmental scale. A set of non-negative integers; create and initialize the synchronization process belief variable b. c This includes the beliefs of N tasks; the belief of a specific task i during synchronization is represented by variable b. ci This indicates that during the synchronization process, the robots unanimously agree that the probability of task i being located at position (x, y) is b. cixy For any x, y, and all data b within the range cixy Composition b ci ;

[0034] ② Select a task i in sequence. If the task number i is less than or equal to N, proceed to step ③; otherwise, proceed to step ⑦.

[0035] ③ Select a position (x,y)∈S sequentially and proceed to step ④; if all positions have been traversed, proceed to step ⑥;

[0036] ④ Calculate the maximum value M and minimum value m of the belief of task i at position (x,y) among all robots;

[0037] ⑤ If M > 1 / L 2 Then let b cixy =M, if m≤1 / L 2 Let b cixy =m, and in other cases let b cixy =1 / L 2 Return to step ③;

[0038] ⑥ After traversing all locations, obtain the belief variable b. ci , will b ci Store in b after normalization c Return to step ②;

[0039] ⑦ Let b j =b c , where b j The synchronization process ends when the belief of the robot numbered j, j = 1, 2, ..., m is established.

[0040] Beneficial effects:

[0041] This invention provides a multi-robot task search method for emergency rescue environments, offering the following advantages: First, compared to deterministic decision-making models where the task is known and sensitive to observation noise, this invention, by introducing a partially observable model and defining beliefs to represent states, can handle situations where the task is unknown, and sensor noise has almost no impact on the search algorithm's performance. Second, compared to the limitations of single-robot decision-making techniques such as robot capabilities and endurance, this invention leverages the advantages of multi-robot collaboration, utilizing the belief synchronization method proposed in step four to achieve parallel multi-robot search and information sharing, improving overall search efficiency, relaxing constraints on individual robot working time and energy, and making it applicable to larger-scale search problems. Third, compared to the limited depth of the search tree constructed by classical partially observable decision-making, which easily leads to trapped problems where simulated gains are all negative in large-scale searches, this invention can identify deadlocked states and provide guidance schemes, ensuring the completeness of the search. The search method of this invention can search for all tasks in a short time, saving valuable time for rescue operations. Attached Figure Description

[0042] Figure 1 —A flowchart of a multi-robot task search method in an emergency rescue environment provided by the present invention;

[0043] Figure 2 —Illustration of the robot being trapped;

[0044] Figure 3—A diagram illustrating decision-making methods;

[0045] Figure 4 —A diagram illustrating the dimensionality reduction of object-oriented beliefs. Detailed Implementation

[0046] The present invention will now be described in detail with reference to the accompanying drawings and embodiments.

[0047] This invention provides a multi-robot task search method in an emergency rescue environment, the process of which is as follows: Figure 1 As shown, the steps of the present invention are as follows:

[0048] Step 1: The robot builds a multi-machine, multi-task search problem model based on the requirements.

[0049] The robot's task is to search for n unknown static target points of the same type within a search space containing multiple static obstacles. The search in the two-dimensional environment can be represented by five parameters: state, action, observation, transition, and reward. Description. State can be decomposed into an object-oriented form, represented as... Where s r Indicates the robot's state, s t This represents the task's state. Actions can be defined by the set {East, South, West, North, Look, Find}. The robot's step size for movement in each of the four directions is one grid cell. Look(d) represents the robot monitoring a 360° area of ​​depth d using LiDAR and returning whether a task exists within that area. For observations, in a rescue environment, this can be represented as... Define events E1, E2, and E3, and the robot's observation region F. When E1 occurs, task i ∈ F and the sensor's observation originates from task i. When E2 occurs, task i ∈ F, but the sensor's observation originates from another task. E3 represents... therefore Equation (1) can be used for calculation, where Equation (2) can be used for calculation. (2) As a normalization function, this invention uses a function with expectation ∈ and covariance matrix Σ. i =σ 2 I 2×2 p(E) m |s) can be represented by equation (3).

[0050]

[0051]

[0052]

[0053] For the state transition model, this invention expresses the state transition probabilities of the target and obstacles as Pr(s) t′ |s t ,a)=1(s t′ =s t After a robot performs an action, its state will also undergo a deterministic transition, which can be represented as Pr(s). r′ |s r a) = 1. The reward function of this invention is set as follows: the robot receives a reward of +1000 for discovering a task, and -1000 if the task position is incorrect. A movement action yields a reward of -2, and a Look action yields a reward of -1. The reward for performing a Find action is 0.

[0054] Step 2: The robot makes autonomous decisions based on the current state.

[0055] When robots make autonomous decisions, they encounter the following problems: Due to the lack of designated rescue sub-regions, robots typically need to complete target search tasks across the entire environment. This leads to difficulties in reaching certain areas, such as corners or narrow corridors, in the later stages of the search phase, even with a limited number of simulations. At this point, the Monte Carlo tree constructed by the robot always provides a reward r < 0 for any action a, such as... Figure 2 As shown. At this point, the robot will choose an action with a higher reward. Since the simulation is random, this process is equivalent to random selection and is meaningless for target search. At this time, the robot enters a deadlock situation. If the simulation depth is blindly increased, the number of simulations within a limited time is too small to obtain the optimal result. Therefore, in step two, the robot first needs to determine whether it is currently trapped in a deadlock. If it is not trapped, it adopts the action corresponding to the highest reward. If it is trapped, for the limited tree depth, a guidance method based on clustering and path planning proposed in this invention is used. The purpose of this method is to find points in the environment with a high density of large belief values ​​equal to the number of robots. Then, the A* path planning algorithm is used to guide the trapped robot to each area where there may be a task. This method can find all targets in a tree with a limited depth. Figure 3 As shown. The guidance method for the trapped robot j is as follows:

[0056] ① Determine if the robot is in the guidance process. If there is already a guidance point, jump to step ④.

[0057] ② Obtain all points whose belief values ​​fall within the adaptive threshold range, denoted as P0.

[0058] ③ Cluster P0 into N classes using the K-means clustering method, and select the point closest to the robot as the target. If the point is an obstacle, select the point closest to that point as the target point.

[0059] ④ Run the A* algorithm to obtain a path from the robot's current position to the target point. Use the first element of this path as the robot's next action.

[0060] The adaptive threshold can be defined according to the standard of equation (4): the left boundary is determined by the number of tasks N found. f The ratio of all tasks N to the total number of tasks is determined. A lower ratio in the early stages of the search indicates many tasks have not been searched; therefore, the left boundary should be fixed at a lower value to ensure no tasks are missed. In the later stages of the search, the value can be increased to improve efficiency. Here, P is the set of all locations, O is the set of all tasks, and L is the environment size. The right interval is set to 0.95.

[0061]

[0062] Step 3: The robot performs actions and updates the individual's beliefs about the environmental state based on observations.

[0063] When all robots have completed their selection of the next action, they execute the actions in parallel. After execution, the rescue environment switches to s′. Simultaneously, each robot receives environmental observations (values ​​obtained when performing the Look operation) and a reward. Because robots use beliefs to represent the probability distribution of the current environmental state b∈[0,1] in uncertain environments, they need to update their previous beliefs upon receiving observations. In the rescue scenario considered in this invention, the tasks are independent of each other, and their probability distributions are independent. Therefore, the large-space target belief can be decomposed into the robot's belief about the object. Specifically, belief b is decomposed into a specific object obj. i The state space contains n points to achieve the effect of belief dimensionality reduction, such as Figure 4 As shown. This belief suggests that it can solve the problem of the large domain dimensionality curse faced by task search in emergency environments. Based on the belief update method of partially observable Markov decision processes, under the constraint of deterministic behavior, each agent can update the belief for each object i according to formula (5). Where η is the normalization factor z. i It represents each agent's observation of object i.

[0064] b′ i (s i )=ηp(z i |s i )b i (s i (5)

[0065] Step 4: Synchronize the robots' beliefs to achieve a unified understanding of the current environment.

[0066] This step introduces an effective information exchange method among multiple robots, avoiding repeated searches in a certain area and improving overall collaborative performance. In step three, robots use beliefs to represent the environmental state; therefore, the interactive search situation involves synchronizing the beliefs of all robots towards each object. Therefore, step four proposes a belief synchronization method based on maximum-minimum values ​​for objects. The synchronization process is as follows:

[0067] ① Let S be the set of all raster numbers, L represents the environmental scale. A set of non-negative integers; create and initialize the synchronization process belief variable b. c This includes the beliefs of N tasks; the belief of a specific task i during synchronization is represented by variable b. ci This indicates that during the synchronization process, the robots unanimously agree that the probability of task i being located at position (x, y) is b. cixy For any x, y, and all data b within the range cixy Composition b ci ;

[0068] ② Select a task i in sequence. If the task number i is less than or equal to N, proceed to step ③; otherwise, proceed to step ⑦.

[0069] ③ Select a position (x,y)∈S sequentially and proceed to step ④; if all positions have been traversed, proceed to step ⑥;

[0070] ④ Calculate the maximum value M and minimum value m of the belief of task i at position (x,y) among all robots;

[0071] ⑤ If M > 1 / L 2 Then let b cixy =M, if m≤1 / L 2 Let b cixy =m, and in other cases let b cixy =1 / L 2 Return to step ③;

[0072] ⑥ After traversing all locations, obtain the belief variable b. ci , will b ci Store in b after normalization c Return to step ②;

[0073] ⑦ Let b j =b c , where b j The synchronization process ends when the belief of the robot numbered j, j = 1, 2, ..., m is established.

[0074] Using the synchronization method proposed in the above steps, each robot can interact with other robots, thereby obtaining the most likely state of each object at each location, thus improving search efficiency. The synchronization effect is illustrated in the diagram. Figure 4 As shown.

[0075] Repeat steps two through four until all tasks within the environment have been found.

[0076] This invention tested the algorithm in a randomized simulation environment. A randomly generated environment of size 25x25 contained 20 tasks, with obstacles accounting for 20% of the total area. Three robots collaborated in the search. The algorithm is described using an object-oriented partially observable Markov decision process. The robots choose between Monte Carlo or guided decisions based on their current state, and update and synchronize their beliefs.

[0077] In summary, the above are merely preferred embodiments of the present invention and are not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A multi-robot task search method applied in rescue environments, characterized in that, The robot hardware comprises a lidar, a robot-mounted unit, and a mobile robot body; the method includes the following steps: Step 1: Discretize the rescue environment. Based on the rescue environment and the information of the robots participating in the search, construct an object-oriented multi-machine partially observable Markov decision model to describe the task search problem. Step 2: Each robot determines whether it is trapped based on its current state. If the robot is trapped, a threshold is first determined using an adaptive method, then the points with belief values ​​in the adaptive range are clustered, and then the A* method is used to guide the decision and determine the current action to be executed. If the robot is not trapped, the action with the highest reward is selected as the current action to be executed. Step 3: Each robot executes its current action, obtains environmental observations and reward values, and updates the belief of each object based on its original belief and observations. Step 4: Synchronize the beliefs of all robots using the minimax method to determine the most likely state of each object at each location, guiding the robot's autonomous decision-making in the next moment; Repeat steps two through four until all tasks within the environment have been found.

2. The multi-robot task search method for rescue environments according to claim 1, characterized in that, The task search problem is described by constructing an object-oriented, multi-machine, partially observable Markov decision model based on information from the rescue environment and the robots participating in the search. Specifically: The entire rescue environment is discretized into L×L grids, where L is the environment size, defined according to the actual situation. Information about the rescue environment and the robots involved in the search includes: prior environmental information and status. action Observations with sensor noise State transition model and reward function Constructing a quintuple The state It consists of the state of each robot and each task; action It consists of a set of {movement in four directions, observation, and search}; where the step size of the movement in each of the four directions is 1 grid cell. Observations with sensor noise Defined by an event-based Gaussian model with error (∈,σ); ∈ and σ are the expectation and variance of the Gaussian model, respectively; State transition model and reward function Defined according to actual circumstances; This leads to an object-oriented, multi-machine, partially observable Markov decision model that describes the task search problem.

3. A multi-robot task search method for rescue environments according to claim 1 or 2, characterized in that, Each robot determines whether it is trapped based on its current state. Specifically, each robot constructs a Monte Carlo tree based on the simulation. If a robot determines that all the rewards in the Monte Carlo tree are less than zero, then the robot is trapped; otherwise, the robot is not trapped.

4. A multi-robot task search method for rescue environments according to claim 1 or 2, characterized in that, If the robot becomes trapped, a threshold is first determined using an adaptive method. Then, points with belief values ​​within the adaptive interval are clustered. Finally, the A* method is used for guidance decision-making to determine the current action to be executed. Specifically: First, the left boundary of the adaptive threshold is determined using the following relationship: Where T L N is the left boundary. f Let N be the number of tasks found, and B be the total number of tasks. j Let P be the maximum value of the belief of robot j, O be the set of all locations, L be the scale of the environment, and the right boundary be set to a set value. Then the robot clusters all positions of the belief value in the adaptive interval into m classes. The robot selects the point closest to itself as the guide point, uses the A* algorithm to plan the path, and takes the first planned action as its current action. Conversely, it uses the action with the highest reward in the Monte Carlo tree as its own behavior.

5. The multi-robot task search method for rescue environments according to claim 4, characterized in that, The robot updates the beliefs of each object based on the original beliefs and observations, specifically as follows: Since the rescue mission is independent, the belief can be decomposed into an object-oriented manner according to the mission, reducing the exponential belief space to a linear space. After each robot performs its current action, the current observations and initial beliefs are used to update the beliefs of each object separately using an object-oriented belief update method: b′ i (s i )=ηp(z i |s i )b i (s i ) Where b′ i (s i ) represents the updated belief, η is the normalization factor, and p(z) is the normalization factor. i |s i ) is in state s i z was observed under the condition i The probability, z i It is each agent's observation of object i, b i (s i ) represents a state of s i The belief at the time; s i This represents the state of the i-th object.

6. The multi-robot task search method for rescue environments according to claim 5, characterized in that, Based on the belief of synchronizing all robots using the max-min method, the synchronization process is as follows: ① Let S be the set of all raster numbers, L represents the environmental scale. A set of non-negative integers; create and initialize the synchronization process belief variable b. c This includes the beliefs of N tasks; the belief of a specific task i during synchronization is represented by variable b. ci This indicates that during the synchronization process, the robots unanimously agree that the probability of task i being located at position (x, y) is b. cixy For any x, y, and all data b within the range cixy Composition b ci ; ② Select a task i in sequence. If the task number i is less than or equal to N, proceed to step ③; otherwise, proceed to step ⑦. ③ Select a position (x,y)∈S sequentially and proceed to step ④; if all positions have been traversed, proceed to step ⑥; ④ Calculate the maximum value M and minimum value m of the belief of task i at position (x,y) among all robots; ⑤ If M > 1 / L 2 Then let b cixy =M, if m≤1 / L 2 Let b cixy =m, and in other cases let b cixy =1 / L 2 Return to step ③; ⑥ After traversing all locations, obtain the belief variable b. ci , will b ci Store in b after normalization c Return to step ②; ⑦ Let b j =b c , where b j The synchronization process ends when the belief of the robot numbered j, j = 1, 2, ..., m is established.

Images