Multi-robot task conflict resolution and dynamic scheduling system and method

Abstract

The application discloses a multi-robot task conflict resolution and dynamic scheduling system and method, and belongs to the technical field of robot control. The system comprises: a perception detection layer, which is used for acquiring operation information of a plurality of robots and a space-time semantic map of tasks to be executed by the robots, and generating conflict information when at least two target robots are detected to have a conflict; a decision scheduling layer, which is used for determining a task execution priority of the tasks to be executed according to the conflict information and a priority factor and a value function of the tasks to be executed corresponding to each target robot, rearranging the tasks to be executed corresponding to the target robots, and generating a task sequence; and an execution control layer, which is used for generating a dynamic scheduling instruction according to the task sequence, the space-time semantic map and the operation information of the target robots, and delivering the dynamic scheduling instruction to the corresponding target robots. The system can flexibly cope with a dynamic scheduling scene of the plurality of robots in real time.

Description

TECHNICAL FIELD

[0001] The present application belongs to the technical field of robot control, and particularly relates to a multi-robot task conflict resolution and dynamic scheduling system and method. BACKGROUND

[0002] With the rapid development of automation technology, multi-robot systems (MRS) have been widely applied in warehouse logistics, intelligent inspection, collaborative manufacturing and other fields. Through the collaborative work of robots, the efficiency and coverage of task execution are improved. However, when the multi-robot system operates in a complex dynamic environment, it often faces core challenges such as task allocation, path planning and conflict resolution.

[0003] Currently, in order to solve the problems of task allocation, path planning and conflict resolution of multi-robot, genetic algorithm is used to re-encode and group tasks, simplify large-scale integer programming problems, and design specific solutions for navigation conflicts and task quantity changes.

[0004] However, the above technical solution needs iterative optimization and global search, and the reaction speed is insufficient. When dealing with multi-robot dynamic scheduling scenarios, there is a defect of insufficient support for real-time and flexibility. SUMMARY

[0005] The present application aims to at least solve one of the technical problems existing in the prior art. To this end, the present application provides a multi-robot task conflict resolution and dynamic scheduling system and method, which can flexibly cope with the dynamic scheduling scenario of multi-robot in real time.

[0006] In a first aspect, the present application provides a multi-robot task conflict resolution and dynamic scheduling system, which comprises:

[0007] comprising a perception detection layer, a decision scheduling layer and an execution control layer connected in sequence;

[0008] The perception detection layer is configured to obtain the running information of a plurality of robots and the spatiotemporal semantic map of the robots executing a to-be-executed task, and generate conflict information when at least two target robots are detected to have a conflict;

[0009] The decision scheduling layer is configured to determine the task execution priority of the to-be-executed task according to the conflict information, and a priority factor and a value function of the to-be-executed task corresponding to each target robot, to rearrange the to-be-executed task corresponding to the target robot, generate a task sequence, and the priority factor includes task urgency, task importance and execution ability of the target robot, and the value function is used to quantify the influence weight of each priority factor on the task execution priority;

[0010] The execution control layer is configured to generate dynamic scheduling instructions according to the task sequence, the spatiotemporal semantic map, and the running information of the target robot, and send the dynamic scheduling instructions to the corresponding target robot.

[0011] According to an embodiment of the present application, the perception detection layer comprises interconnected detection modules and judgment modules.

[0012] The detection module is configured to model future trajectories of each robot according to the running information and the spatiotemporal semantic map, to obtain a spatiotemporal state corridor.

[0013] The conflict prediction list between the relevant robots is generated according to the spatiotemporal state corridor and the spatiotemporal semantic map.

[0014] The judgment module is configured to determine each predicted conflict in the conflict prediction list, and generate the conflict information corresponding to the target robot determined as a true conflict.

[0015] According to an embodiment of the present application, the detection module is further configured to:

[0016] detect the spatiotemporal state corridor, and determine an intersection region and at least two relevant robots corresponding to the intersection region in a case where there is a volume intersection between the spatiotemporal state corridors.

[0017] determine a region attribute of the intersection region and a spatiotemporal semantic corridor label of the relevant robots in the intersection region based on the spatiotemporal semantic map.

[0018] generate the conflict prediction list in a case where it is determined that a conflict occurs between the at least two relevant robots according to the region attribute and the spatiotemporal semantic corridor label.

[0019] According to an embodiment of the present application, the conflict information comprises a conflict type and a conflict severity, and the judgment module comprises a screening submodule, an identification submodule, and a grading submodule connected in sequence.

[0020] The screening submodule is configured to remove predicted conflicts determined as false positive conflicts in the conflict prediction list based on traffic rule semantics, behavior intention semantics, and high-order task constraints in the spatiotemporal semantic map, and screen at least one true conflict, wherein the high-order task constraints comprise dependency relationship restrictions and cooperation requirement restrictions between to-be-executed tasks.

[0021] The identification submodule is configured to identify a conflict type of the true conflict according to resource distribution, path planning information in the spatiotemporal semantic map, and a task priority of the to-be-executed task.

[0022] The hierarchical submodule is configured to determine a conflict severity of the real conflict according to the conflict type.

[0023] According to an embodiment of the present application, the conflict type includes a path conflict, a resource conflict, and a priority conflict, and the hierarchical submodule is further configured to:

[0024] In a case where the conflict type of the real conflict is determined to be the path conflict, the conflict severity is determined according to a relative speed of a target robot, a collision angle, and a region feature of an intersection region corresponding to the real conflict;

[0025] In a case where the conflict type of the real conflict is determined to be the resource conflict, the conflict severity is determined according to a target robot resource application order, a resource use time length, and a resource demand urgency corresponding to the real conflict;

[0026] In a case where the conflict type of the real conflict is determined to be the priority conflict, the conflict severity is determined according to a task value density of the target robot, a task urgency, a rescheduling cost, and a task priority difference corresponding to the real conflict;

[0027] The task value density is used to represent a ratio of a benefit to an execution cost of a task performed by the target robot, the task urgency is used to represent a time constraint degree of a task to be executed, and the rescheduling cost is used to represent an additional cost generated due to task rearrangement.

[0028] According to an embodiment of the present application, the decision scheduling layer includes a calculation module, a decision module, and a scheduling module connected in sequence:

[0029] The calculation module is configured to calculate an adaptation degree of the target robot in executing the task to be executed according to a heterogeneous capability vector of the target robot and a task demand vector of the task to be executed based on the space-time semantic map;

[0030] The decision module is configured to calculate a task execution priority of each target robot through the value function according to a conflict type and a conflict severity in the conflict information, a task urgency, a task importance of the task to be executed, and the adaptation degree.

[0031] The decision module is configured to perform dynamic priority adjustment on the task to be executed corresponding to the target robot according to the task execution priority of each target robot, and generate an adjusted task priority list.

[0032] The scheduling module is configured to rearrange the task to be executed corresponding to the target robot according to the adjusted task priority list, and generate the task sequence including a task order and an execution time.

[0033] The isomerism capability vector is determined according to differences in execution capability of the target robot on different task types, the task demand vector is used to represent demand characteristics of the robot capability of the to-be-executed task, and the adaptation degree is used to represent a matching degree between the target robot and the to-be-executed task.

[0034] According to an embodiment of the present application, the execution control layer comprises an execution module and a scheduling module connected to each other.

[0035] The execution module is configured to determine a motion trajectory and action parameters required by each target robot to execute a corresponding to-be-executed task according to the task sequence and running information of the target robot.

[0036] The scheduling module is configured to generate the dynamic scheduling instruction according to the motion trajectory and action parameters and the space-time semantic map, wherein the dynamic scheduling instruction comprises the to-be-executed task, the motion trajectory, the action parameters and an execution time of the target robot.

[0037] According to an embodiment of the present application, the execution control layer further comprises a control module connected to the scheduling module.

[0038] The control module is configured to, in a case where feedback information of the target robot responding to the dynamic scheduling instruction and executing the to-be-executed task is received, parse the feedback information to obtain an actual motion trajectory and action execution of the target robot.

[0039] According to the actual motion trajectory and action execution, an update instruction is generated, and the update instruction is used to trigger updating of the space-time semantic map.

[0040] According to an embodiment of the present application, the control module is further configured to:

[0041] In a case where the feedback information is received, the feedback information is segmented by an upper-layer sliding window in a time dimension to obtain a plurality of segmented information.

[0042] Each segmented information is clustered by a lower-layer sliding window in a space dimension to obtain a plurality of space-time clustering results.

[0043] According to the space-time clustering results, a trajectory deviation parameter and an action deviation parameter of the target robot are determined.

[0044] Based on the trajectory deviation parameter and the action deviation parameter, fluctuation characteristics of the motion trajectory and action execution of the target robot are analyzed.

[0045] According to the fluctuation characteristics, the window size of the upper layer sliding window and the sliding step of the lower layer sliding window are dynamically adjusted, the trajectory deviation parameter and the action deviation parameter are recalculated, and in a case where the trajectory deviation parameter and the action deviation parameter satisfy a preset threshold condition, the update instruction is generated.

[0046] In a second aspect, the application provides a multi-robot task conflict resolution and dynamic scheduling method, applied to the multi-robot task conflict resolution and dynamic scheduling system as in the first aspect, and the method comprises:

[0047] Through the perception detection layer, the running information of the plurality of robots and the spatiotemporal semantic map of the robots performing the to-be-executed tasks are acquired, and in a case where at least two target robots are detected to have a conflict, conflict information is generated;

[0048] Through the decision scheduling layer, according to the conflict information and a priority factor and a value function of each target robot corresponding to a to-be-executed task, a task execution priority of the to-be-executed task is determined, the to-be-executed task corresponding to the target robot is rearranged, a task sequence is generated, the priority factor comprises a task urgency, a task importance and an execution capability of the target robot, and the value function is used to quantify an influence weight of each priority factor on the task execution priority;

[0049] Through the execution control layer, according to the task sequence, the spatiotemporal semantic map and the running information of the target robots, a dynamic scheduling instruction is generated and delivered to the corresponding target robots.

[0050] Additional aspects and advantages of the application will be in part apparent and in part pointed out hereinafter.

[0051] The multi-robot task conflict resolution and dynamic scheduling system and method provided by the application have the following beneficial effects compared with the prior art:

[0052] (1) Through the perception detection layer in combination with the running information and the spatiotemporal semantic map, the conflict is actively and in real time predicted, the priority factor and the value function are introduced in the decision scheduling layer, the multi-dimensional information such as the task urgency, the importance and the robot capability can be quantitatively evaluated and the task is rearranged, the intelligent decision is realized, the key task is ensured to be executed in priority, the intelligence and the rationality of the decision are improved, the dynamic scheduling instruction is delivered through the execution control layer, the running efficiency, the automation level and the running efficiency of the multi-robot system are improved, the conflict is resolved in real time during the task execution, the robot is prevented from being stalled or the task is prevented from failing, the stability and the execution efficiency of the multi-robot cooperation system are improved, and the dynamic scheduling scene of the multi-robot can be flexibly coped with in real time.

[0053] (2) By modeling the future trajectory and the spatio-temporal state corridor, the conflict detection is promoted from the simple current spatial overlap to the prediction in the future spatio-temporal field, so that the potential conflict can be found earlier and more accurately, and the false positive conflict can be filtered out through the secondary verification of the conflict prediction list, so as to avoid unnecessary scheduling of the system on the non-existing problem, save the computing resources, and improve the system efficiency. BRIEF DESCRIPTION OF DRAWINGS

[0054] The above and / or additional aspects and advantages of the present application will become apparent and be readily appreciated from the description of the embodiments, taken in conjunction with the following drawings in which:

[0055] Figure 1 is a structural schematic diagram of a multi-robot task conflict resolution and dynamic scheduling system provided by an embodiment of the present application;

[0056] Figure 2 is a flow schematic diagram of a multi-robot task conflict resolution and dynamic scheduling method provided by an embodiment of the present application. DETAILED DESCRIPTION

[0057] The technical solutions in the embodiments of the present application will be described clearly below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only some of the embodiments of the present application, not all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art belong to the scope of protection of the present application.

[0058] The terms "first", "second", and the like in the specification and claims of the present application are used to distinguish similar objects, and are not used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged under appropriate circumstances, so that the embodiments of the present application can be implemented in an order other than those illustrated or described herein, and the objects distinguished by "first", "second", etc. are usually a class, not limited to the number of objects, for example, the first object can be one or more. In addition, "and / or" in the specification and claims means at least one of the connected objects, and the character " / ", generally represents a "or" relationship between the front and rear associated objects.

[0059] The multi-robot task conflict resolution and dynamic scheduling system and the multi-robot task conflict resolution and dynamic scheduling method provided by the embodiments of the present application will be described in detail below in conjunction with the drawings, through specific embodiments and their application scenarios.

[0060] As shown in the figure, the multi-robot task conflict resolution and dynamic scheduling system includes a perception detection layer 110, a decision scheduling layer 120 and an execution control layer 130 connected in sequence; Figure 1

[0061] ​The perception detection layer 110 is configured to acquire operation information of multiple robots and a spatio-temporal semantic map of tasks to be executed by the robots, and generate conflict information when at least two target robots are detected to have a conflict.

[0062] The decision scheduling layer 120 is configured to determine a task execution priority of the tasks to be executed according to the conflict information and a priority factor and a value function of the tasks to be executed by each target robot, rearrange the tasks to be executed by the target robots, and generate a task sequence, wherein the priority factor includes a task urgency, a task importance and an execution capability of the target robot, and the value function is used to quantify an influence weight of each priority factor on the task execution priority.

[0063] The execution control layer 130 is configured to generate a dynamic scheduling instruction according to the task sequence, the spatio-temporal semantic map and the operation information of the target robots, and send the dynamic scheduling instruction to the corresponding target robots.

[0064] It can be understood that the operation information is multiple-dimensional state data generated by the robot in the working process, including an operation state of the robot and environment information, wherein the operation state is a specific situation of the robot in executing a task, for example, the operation information includes a position, a motion state, a task progress, a fault state and a power constraint of the robot, the power constraint is a physical motion capability limitation of the robot, including a maximum speed, an acceleration, a minimum turning radius, a bearing capacity and the like, and a cruising capability, a load capacity and a motion speed limitation, which can affect a possible motion trajectory of the robot, and the environment information is a current operation environment of all robots.

[0065] The task to be executed is a work instruction pre-assigned to the robot by the system, including information such as a task type, a task target position, a task start and end time and the like.

[0066] The spatio-temporal semantic map is a map that fuses spatial structure, time dynamics and semantic information, including three-dimensional spatio-temporal information of obstacles, roads and the like spatial information on the map, a robot motion trajectory, a task execution area and a resource occupation state, time information such as whether a certain area is available at a certain time, and semantic information such as a charging station, a high-speed channel or a loading and unloading area, which not only records a current position and a task area of the robot, but also includes a potential conflict area, a resource occupation situation and state changes of each area at different time points, thereby providing the system with comprehensive environment perception capability.

[0067] The target robot is a robot identified by the system as being involved in a conflict that needs to be handled and solved.

[0068] The conflict information is alarm information generated when the paths, resource usage or task plans of multiple robots are predicted to be in conflict, and includes the conflicting parties, the conflict location, the predicted occurrence time, the conflict type, and the conflict degree. The conflict type includes path intersection conflict, resource contention conflict and time sequence dependent conflict, and the conflict degree is quantitatively graded according to the influence degree of the conflict on task execution.

[0069] The priority factor is a set of indexes for evaluating the urgency and importance of the to-be-executed task. In the priority factor, the task urgency is determined according to the proximity of the deadline, and the task importance is used to evaluate the value or consequence severity of the to-be-executed task, including safety criticality and production impact degree. The execution capability of the target robot includes the efficiency and adaptability of the target robot in completing the to-be-executed task, the adaptability to the task environment, the precision of action completion, and the utilization capability of energy, including residual power, load state and historical efficiency, etc.

[0070] The value function is used to convert the priority factor into a unified benefit index.

[0071] The task execution priority is the task execution sequence weight obtained by the comprehensive evaluation of the value function on the priority factor, and is used to sort all to-be-executed tasks.

[0072] The task sequence includes a task plan table of task order, start and end time, and resource allocation.

[0073] The dynamic scheduling instruction is a real-time adjustable control command issued to each target robot, including the specific content of the target point, path, speed, action type, etc. of the to-be-executed task, as well as the execution sequence and execution time, to ensure that the robot can accurately and efficiently execute the corresponding action.

[0074] The execution control layer triggers to update the space-time semantic map in the case of receiving feedback information that the target robot executes the to-be-executed task in response to the dynamic scheduling instruction, to continuously optimize based on the actual execution of the robot and the change of the environment. The feedback information is the result state report returned by the robot after executing the dynamic scheduling instruction, which is used to form a closed-loop control. This process enhances the robustness of the system in a complex and changing environment, and guarantees the dynamic adaptability of the system.

[0075] The perception detection layer can be deployed in an edge computing unit, and through real-time collection and analysis of running information, it can detect the conflict situation that may occur between at least two target robots and generate conflict information.

[0076] The decision scheduling layer can be deployed on an industrial server. The received conflict information is combined with the priority factor of each target robot and the value function of the to-be-executed task to determine the task execution priority of each to-be-executed task, rearrange the to-be-executed tasks corresponding to the target robots, and generate a task sequence.

[0077] The execution control layer can be deployed on the processor of each robot body. According to the generated task sequence, the spatiotemporal semantic map, and the running information of the target robot, a targeted dynamic scheduling instruction is generated and accurately issued to the controller of the corresponding target robot.

[0078] In actual execution, the system can be deployed in an intelligent warehouse logistics center. In the case of needing to carry out material handling, n robots perform m to-be-executed tasks such as cargo handling, sorting, shelving, and the like.

[0079] The global state vector of the system at time t is:

[0080]

[0081] wherein, is the spatiotemporal semantic map at time t, is the set of running states of the n robots, is the running state of the i th robot, is the position of the i th robot, is the speed of the i th robot, is the remaining power of the i th robot, is the execution capability of the i th robot; is the state set of the m to-be-executed tasks, is the state of the j th to-be-executed task, is the progress of the j th to-be-executed task, is the priority factor of the j th to-be-executed task, is the position of the j th to-be-executed task, is the deadline of the j th to-be-executed task; is the conflict information corresponding to the set of predicted conflicts, including the information of K predicted conflicts, and each predicted conflict information includes the related robot pair involved in the conflict, the conflict type, the conflict position, and the like.

[0082] In the perception detection layer, the spatiotemporal semantic map at time t is :

[0083]

[0084] wherein, is a perception fusion function, which is constructed based on Simultaneous Localization And Mapping (SLAM) technology, is the environment information monitored by the environment sensor at time t, including environment topology, obstacles, task points, and other spatio-temporal information.

[0085] Conflict detection is performed on the robots in the system:

[0086]

[0087] wherein, is a conflict detection function, which quantifies the spatio-temporal occupancy map, resource occupation matrix, and Estimated Time of Arrival (ETA) distribution into conflict indicators, and performs three-dimensional matching within a sliding time window to determine the conflict, is the scheduling state at time t.

[0088] Conflict detection function The determination rules in include path conflict rules, resource conflict rules, and priority conflict rules.

[0089] For path conflict, if two paths are too close in space, such that , the two paths are too close in space, and it is determined that there is a path conflict, is the spatio-temporal curve of the path planning of robot i, is the spatio-temporal curve of the path planning of robot i+1, is the minimum safety distance threshold for path conflict,

[0090] For resource conflict, if multiple robots are assigned the same task, , j such that , it is determined that there is a resource conflict, is the jth task to be executed is the set of robots currently assigned, is the ith robot, ;

[0091] For priority conflict, if the jth task to be executed is applied by more than two robots, the value vector of the robots to the jth task to be executed is obtained is:

[0092]

[0093] wherein, the element is the value quantification item calculated by the value function, and the value quantification items are sorted in descending order:

[0094]

[0095] The value quantification items of the two robots selected to apply for the jth task to be executed are sorted in descending order and and are compared with the priority fuzzy threshold In the case of , it is determined that there is a priority conflict.

[0096] For each predicted conflict, the conflict type, the relevant target robot, and the involved task to be executed are extracted, and the conflict severity is calculated, wherein the conflict severity is a scalar determined by combining the time urgency, resource scarcity, and safety risk indicators.

[0097] For example, in the case of the kth predicted conflict being a path conflict, the corresponding conflict severity is:

[0098]

[0099] wherein is the minimum distance between the path and the path ; is the time urgency weight coefficient; tu is the time urgency factor, which is used to represent the time urgency degree of the predicted conflict, and is obtained according to a function of the difference between the predicted occurrence time of the predicted conflict and the current time.

[0100] In the case of the kth predicted conflict being a resource conflict, the corresponding conflict severity is:

[0101]

[0102] wherein is the number of robots competing for the same resource; is the maximum number of robots that can simultaneously use the resource; is the resource scarcity weight coefficient, is the resource scarcity factor, which is used to represent the scarcity degree of the resource in the system, and can be obtained based on the historical usage frequency or demand proportion of the resource.

[0103] In the case of the kth predicted conflict being a priority conflict, the corresponding conflict severity is:

[0104]

[0105] wherein, The smaller the value is, the lower the conflict severity is.

[0106] At the decision-making scheduling layer, the priority factor of the ith robot performing the jth to-be-executed task is :

[0107]

[0108] wherein, is a progress factor, is the progress of the jth to-be-executed task, is the deadline of the jth to-be-executed task, the closer to the deadline and the slower the progress is, the more urgent it is; is a task importance factor, a high-value or critical path task is more important, which can be determined according to the inherent value and dependency weight of the task; is a robot capability factor, which is used to represent the matching degree of the robot capability and the task demand, the higher the matching degree is, the higher the score is.

[0109] For the ith robot performing the jth to-be-executed task at time t, the task execution priority is calculated by a value function :

[0110]

[0111] wherein, is a weight, satisfying .

[0112] The state space is:

[0113]

[0114] wherein, is a spatiotemporal semantic map at time t, is the running state of the ith robot, is the state of the jth to-be-executed task, is conflict information corresponding to a set of predicted conflicts, is a scheduling state at time t, which is used to represent the task allocation sequence and the historical decision record at the current time, including the task queue and the execution state of each robot.

[0115] The action space is:

[0116]

[0117] wherein, is a binary variable, which is used to represent whether the robot i performs the to-be-executed task j.

[0118] To maximize the total value of the robots and task assignment, the objective function is constructed as:

[0119]

[0120] where, represents that each task is assigned to at most one robot; represents the upper limit of the number of tasks for the i-th robot, is the number of tasks assigned to the i-th robot; is the task to be executed must be within the time window and reachable by the robot; is the order of task dependency; is the conflict penalty coefficient, is the conflict cost.

[0121] Further, dynamic programming is used to solve:

[0122]

[0123] where, is the optimal action value function, representing the expected return after performing action a from state s; is the reward function, representing the immediate utility obtained after performing action a from state s; is the discount factor, used to control the importance of future rewards; represents the expected value of state transition to s’ after performing action a from state s, is the state transition probability function, representing the probability of transitioning to state s’ after taking action a in state s, is the optimal state value function of state s’, representing the maximum expected return that can be obtained by starting from state s’ and performing according to the optimal policy; is the optimal state value function of state s, representing the maximum expected return that can be obtained by starting from state s and performing according to the optimal policy; is the optimal policy in state s, representing the optimal action that maximizes in state s.

[0124] The reward function is:

[0125]

[0126] where, is the task value; is the conflict cost, is the conflict severity, is the conflict cost function, representing whether action a triggers a conflict ; is the resource consumption, It is the resource consumption of a single task, determined based on energy consumption, time consumption, and wear and tear.

[0127] Each robot is sorted and assigned tasks according to their priority, resulting in the following task sequence:

[0128]

[0129] in, It is a function that arranges tasks in order of priority. It takes an unordered set of tasks as input and outputs an ordered sequence of tasks arranged from highest to lowest priority. In the tasks assigned to and about to be performed by robot i, all decision variables Tasks to be executed A set of.

[0130] At the execution control layer, dynamic scheduling instructions for the i-th robot are generated. for:

[0131]

[0132] in, It is a control command generator, which is obtained by concatenating a trajectory generator, a trajectory interpolator, a velocity-attitude controller, and an actuator command mapping function. In the trajectory generator, Model Predictive Control (MPC) is used in the spatiotemporal semantic map. Perform online trajectory optimization and output waypoint sequence for each time point. ,in, These are the robot's two-dimensional coordinates in the spatiotemporal semantic map. t is the angle between the robot's orientation and the positive x-axis, v is the robot's linear velocity along the path, and t is the absolute moment of reaching that pose.

[0133] In the trajectory interpolator, each waypoint sequence is organized in the form of a time series, and the state component at each time point is interpolated in the time domain. A continuous reference trajectory is obtained by uniformly sampling according to the control frequency.

[0134] In the velocity-attitude controller, the deviation between the robot's current position and the continuous reference trajectory is calculated by the linear quadratic regulator (LQR), and linear velocity and angular velocity commands are generated.

[0135] In the actuator instruction mapping function, based on the time requirements of the task sequence, if the task requires loading, taking pictures, switching lights on and off, etc., specific operation instructions such as loading and taking pictures are inserted at the corresponding trajectory key points and inserted into the trajectory key points. All instructions are packaged into a control instruction package according to the time axis and issued as a dynamic scheduling instruction.

[0136] In the velocity-attitude controller, it is necessary to adjust the continuous reference trajectory. The objective function for path planning is:

[0137]

[0138] in, It is the time consumption of the path. It is a weighting factor. This refers to the energy consumption along the path, calculated based on factors such as the robot's speed, load, and travel distance. It is the fitness coefficient. It is a path and spatiotemporal semantic map Adaptability.

[0139] The constraints of the path planning subproblem include avoiding obstacles and not conflicting with other robot paths, for example, traversing space and time. Perform path planning for multiple robots.

[0140] The multi-robot task conflict resolution and dynamic scheduling system provided in this application combines operational information and spatiotemporal semantic maps through a perception and detection layer to proactively predict conflicts in real time. Priority factors and value functions are introduced at the decision-making and scheduling layer, enabling quantitative evaluation and task rearrangement based on multi-dimensional information such as task urgency, importance, and robot capabilities. This achieves intelligent decision-making, ensuring the priority execution of critical tasks and improving the intelligence and rationality of decision-making. Dynamic scheduling instructions are issued by the execution control layer, improving the operational efficiency, automation level, and overall efficiency of the multi-robot system. By resolving conflicts in real time during task execution, robot stagnation or task failure is avoided, enhancing the stability and execution efficiency of the multi-robot collaborative system. It can flexibly and in real time respond to dynamic scheduling scenarios involving multiple robots.

[0141] In some embodiments, the perception and detection layer includes a detection module and a judgment module that are interconnected.

[0142] The detection module is used to model the future trajectory of each robot based on the operation information and the spatiotemporal semantic map to obtain a spatiotemporal state corridor.

[0143] Based on the spatiotemporal state corridor and the spatiotemporal semantic map, a conflict prediction list between relevant robots is generated;

[0144] The judgment module is configured to judge each predicted conflict in the conflict prediction list to generate the conflict information corresponding to the target robot determined as a true conflict.

[0145] The future trajectory modeling is based on the current state, target and operation information of the robot to predict the future motion path of the robot.

[0146] The space-time state corridor is a four-dimensional space region. Based on the space path, the time dimension is added to form a pipeline in the time-space four-dimensional space, which represents the space range occupied by the robot in a period of time. For example, in a two-dimensional plane, each time slice corresponds to a possible occupied area, and the time slices are stacked to form a space-time state corridor.

[0147] The related robot is a robot involved in the predicted conflict.

[0148] The predicted conflict is a potential conflict predicted. The conflict prediction list includes at least one predicted conflict, and corresponding conflict type, related robot and predicted occurrence time, etc. The true conflict is a real conflict that may occur.

[0149] In actual execution, the detection module obtains real-time operation information of each robot, including current position, speed, orientation, acceleration, task target, motion mode and current task state; loads a space-time semantic map, which contains geometric structures of the environment such as corridors and intersections, and semantic information and availability rules in the time dimension, such as priority passing area, no-passing period, dynamic obstacle high-frequency area, etc. For example, a certain channel is only allowed for a specific type of robot to pass through in a specific time period.

[0150] Based on the current state and task target of each robot, the expected motion path of the robot in a period of time is planned on the space-time semantic map, and the dynamic constraints of the robot and the semantic constraints of the environment are considered. A prediction controller is used to generate a confidence interval of the future trajectory, wherein the semantic constraints of the environment include the constraints of not crossing the wall and obeying the one-way rule. Considering the motion uncertainty of the robot and the safety margin, the predicted trajectory is expanded to an occupancy area band varying with time in the time and space dimensions to form a space-time state corridor,

[0151] In some embodiments, the detection module is further configured to:

[0152] detect the space-time state corridor, determine the intersection region and the space-time semantic corridor label of the at least two related robots in the intersection region if it is determined that there is a volume intersection between the space-time state corridors;

[0153] determine the region attribute of the intersection region and the space-time semantic corridor label of the related robots in the intersection region based on the space-time semantic map.

[0154] In the case where it is determined that a conflict occurs between the at least two related robots according to the region attribute and the spatio-temporal semantic corridor label, a conflict prediction list is generated.

[0155] The volume intersection is to expand the occupancy region of a robot from its outer contour to the space volume swept in its motion process, and the overlap of this volume of two robots in the future is the physical basis of the conflict.

[0156] The intersection region is a region where the regions occupied by any two or more spatio-temporal state corridors intersect at the same time.

[0157] The region attribute is the semantic type of the intersection region on the map, including region geometric type, traffic rule, physical limit, time constraint and historical behavior pattern, etc., wherein the geometric type is narrow channel, crossroad, T-shaped intersection, roundabout, loading and unloading area, elevator hall, etc., the traffic rule is one-way street, two-way street, prohibition of merging, priority direction of passing, the physical limit is width, height, ground friction coefficient, the time constraint is whether the intersection region is limited between the earliest time and the latest time, and the historical behavior pattern is the common conflict type, average passing time, etc.

[0158] The spatio-temporal semantic corridor label is a label of semantic information given to the spatio-temporal state corridor, for example, the semantic information includes the role label of the robot, the motion intention, and the task type, task attribute, task priority of the corresponding to-be-executed task.

[0159] In actual execution, the detection module is further used for detecting the spatio-temporal intersection of all spatio-temporal state corridors of the robots two by two, if the spatio-temporal state corridors of two robots have spatial overlap in a time period, the occupancy regions thereof have intersection at the same time, all time-space segments where the intersection occurs are recorded to extract the intersection region, determine the related robots and the earliest time and the latest time of the intersection region, map the spatial geometry of the intersection region to the spatio-temporal semantic map, and record as a potential predicted conflict, if the overlap region is located in a wide channel allowing parallel passing and the robot types are compatible, the conflict level can be reduced; if the overlap occurs in a narrow channel, a crossroad or a high-priority region, it is marked as a high-risk conflict.

[0160] All predicted conflicts are integrated into a conflict prediction list and output, including the identification (ID) of the related robots involved in each prediction list, the time interval of the predicted conflict, the spatial location of the conflict occurrence, and the conflict type.

[0161] In this embodiment, the conflict judgment is made by judging the volume intersection in combination with the space-time corridor label in the space-time semantic map, the obtained re-push information contains rich semantic scene description, a structured conflict prediction list is generated, the conflict detection process is more refined and semantic, and the quality of the output conflict information is higher.

[0162] The judgment module receives the complete conflict prediction list from the detection module, each prediction conflict record is attached with space-time state corridor and semantic context information, each potential conflict is analyzed to determine whether it is a real conflict that may actually occur, and false positives caused by prediction errors or semantic misjudgment are excluded.

[0163] In some embodiments, the conflict information includes a conflict type and a conflict severity, and the judgment module includes a screening submodule, an identification submodule and a grading submodule connected in sequence.

[0164] The screening submodule is configured to determine and remove false positive conflicts in the conflict prediction list based on traffic rule semantics, behavior intention semantics and high-level task constraints in the space-time semantic map, screen at least one real conflict, and the high-level task constraint includes a dependency relationship limit and a cooperation requirement limit between to-be-executed tasks.

[0165] The identification submodule is configured to identify the conflict type of the real conflict according to resource distribution, path planning information in the space-time semantic map, and a task priority of the to-be-executed task.

[0166] The grading submodule is configured to determine the conflict severity of the real conflict according to the conflict type.

[0167] It can be understood that the traffic rule semantics are virtual traffic rules defined in the map, and the behavior intention semantics are the behavior tendency of the robot.

[0168] The high-level task constraint is used to represent high-level rules or goals that need to be followed in the multi-robot cooperation process. It is a global constraint at the task level beyond the single robot motion.

[0169] In the high-level task constraint, the dependency relationship limit is the sequence or conditional dependency between different tasks, the execution order limit is the fixed sequence due to process flow or physical limit, and the cooperation requirement limit is the cooperation effect that multiple robots need to achieve during task execution, for example, multiple robots need to reach a designated location for cooperative work at the same time, or need to maintain a certain distance within a certain time to avoid collision, etc.

[0170] False positive conflicts are false alarms that have no actual impact and do not need to be handled; real conflicts are system contradictions that must be resolved.

[0171] In the space-time semantic map, the resource distribution includes the location, quantity and usage state of charging piles, workstations, tools and other resources in the system, the path planning information is the topological structure of the current path of the robot, and the task priority is calculated by the value function on the priority factor.

[0172] The quantitative evaluation of the conflict severity on the conflict consequences of different conflict types is obtained.

[0173] In actual execution, the screening submodule loads the space-time semantic map, extracts traffic rule semantics, behavior intention semantics and high-order task constraints, performs traffic rule compliance checking on each predicted conflict to determine whether the predicted conflict can be avoided, performs behavior intention semantic analysis to analyze whether the predicted conflict can be degraded or removed, performs high-order task constraint analysis to check whether the tasks of the two robots have dependency or sequence constraints, filters out false positive conflicts that will not actually occur due to prediction errors or environmental semantic permission if the tasks have no dependency and are both independent high-priority tasks, and the remaining conflicts that are not eliminated by the above rules are true conflicts.

[0174] The identification submodule classifies each true conflict in detail based on the resource distribution, path planning information and task priority, labels the precise conflict type for each true conflict, and forms an intermediate result with a type label.

[0175] For example, if the path planning shows that both need to pass through the center of the intersection at a similar time, the conflict occurs at a cross or T-shaped intersection, and the paths of the two robots are perpendicular or oblique, the conflict type is intersection preemption conflict; if the channel width is less than the sum of the widths of the two robots, the two robots travel in opposite directions, the path directions are opposite, and there is no lateral avoidance space, the conflict type is opposite channel blocking conflict; if one branch robot tries to merge into the main road with the main road robot at a similar time, the main road has a priority passing semantic, but the task priority of the main road robot is not high, and coordination may be needed, the conflict type is merging competition conflict; if the conflict occurs near the only resource point, for example, the only elevator port or charging position, both robots need to use the resource, and the reservation times overlap, the conflict type is resource contention conflict; if the two robots travel in the same direction, the speed of the rear vehicle is higher than that of the front vehicle, the distance is too close, the path overlap degree is high, the relative speed is large, and the braking distance is insufficient, the conflict type is rear-end risk conflict.

[0176] The hierarchical submodule quantifies the severity of the conflict according to the conflict type, task priority, environmental risk and other factors, adopts rule mapping and weighted scoring, quantifies the severity of the conflict, and divides the conflict severity according to the numerical value. In the case of high conflict severity, immediate path re-planning, forced waiting and other interventions are required; in the case of medium conflict severity, local speed adjustment can be used for relief; in the case of low conflict severity, the system can be self-coordinated without alarm, and the conflict information is output.

[0177] In this embodiment, by introducing high-order task constraints, the global consistency at the task level is improved, the system reliability is improved by filtering false positives through semantic and task constraints, and the priority of conflicts is ranked to reasonably allocate computing and scheduling resources by identifying conflict types and determining conflict severity.

[0178] By modeling future trajectories and spatio-temporal state corridors, conflict detection is elevated from simple current spatial overlap to prediction in the future spatio-temporal field, allowing for earlier and more accurate detection of potential conflicts. By performing secondary verification on the conflict prediction list, false positive conflicts can be filtered out, avoiding unnecessary scheduling by the system for non-existent problems, saving computing resources and improving system efficiency.

[0179] In some embodiments, the conflict types include path conflicts, resource conflicts, and priority conflicts, and the hierarchical sub-module is further configured to:

[0180] In a case where the conflict type of the true conflict is a path conflict, the conflict severity is determined according to the relative speed of the target robot, the collision angle, and the area characteristics of the intersection region corresponding to the true conflict.

[0181] In a case where the conflict type of the true conflict is a resource conflict, the conflict severity is determined according to the target robot resource application order, the resource usage time length, and the resource demand urgency corresponding to the true conflict.

[0182] In a case where the conflict type of the true conflict is a priority conflict, the conflict severity is determined according to the task value density of the target robot, the task urgency, the re-scheduling cost, and the task priority difference corresponding to the true conflict.

[0183] The task value density is used to represent the ratio of the benefit to the execution cost of the target robot performing the task, the task urgency is used to represent the degree of time constraint of the task to be executed, and the re-scheduling cost is used to represent the additional cost generated by task rearrangement.

[0184] Path conflicts are used to represent spatio-temporal intersections on future trajectories of robots, and there is no natural avoidance path, which may cause collisions or blockages, including intersection occupation conflicts, opposite channel blockage conflicts, convergence competition conflicts, and tail risk conflicts, as well as detour conflicts.

[0185] Resource conflicts are used to represent concurrent requests for the same limited resource by multiple robots, resulting in task delays or blockages, including resource contention conflicts, and competition for tools, equipment, and other resources.

[0186] Priority conflicts arise when multiple robots' tasks have temporal dependencies or differing priority determinations, or when high-value / urgent tasks and low-priority tasks are on the same resource or path, resulting in conflicting execution orders.

[0187] Area characteristics include area type, avoidability, and dynamic complexity. Area type includes narrow passages, intersections, blind spots, ramps, etc. Avoidability includes whether there are lateral detour paths and waiting areas. Dynamic complexity includes whether it is a high-frequency intersection area for multiple robots.

[0188] Resource request ranking is used to characterize the order in which the two robots reserve the resource. Resource usage duration is the estimated time period from the start to the end of resource usage. Resource demand urgency is determined based on the difference between the task deadline and the current time, and is assigned a task weight.

[0189] Task value density is the ratio of the benefits of a target robot performing a task to its execution cost. Task benefits include transportation value and service rating, while execution costs include energy consumption, time, and manpower. Task urgency is quantified by current time, execution time, and execution duration.

[0190] The cost of rescheduling includes the computational overhead of path replanning, communication latency, the cost of chain adjustments to other robots, as well as the delay of other tasks and the increase in total distance consumption. It can be calculated by the number of robots affected by rescheduling and the cost of a single adjustment.

[0191] Task priority differences are used to characterize the differences in urgency and importance of different tasks in achieving the overall system goal.

[0192] In actual execution, the hierarchical submodule receives a set of true conflicts. The information of each true conflict includes the conflict type, the target robot pair involved, the corresponding intersection area or resource point information, as well as task-related information such as task objectives, priorities, and planned time. It also loads the regional features and resource attributes provided by the spatiotemporal semantic map, the task value, urgency, and rescheduling cost in the task management system, and the speed, direction, and path overlap output by the path planner.

[0193] Based on the semantic differences of genuine conflicts, a differentiated assessment strategy is adopted, and multi-dimensional dynamic indicators are integrated to calculate and output the severity of the conflict.

[0194] For path conflicts, the relative speed, collision angle, and minimum safe distance of the two robots at the intersection time are extracted from the operation information. The regional features, avoidability, and dynamic complexity of the intersection area in the spatiotemporal semantic map are analyzed to construct a path conflict severity scoring model. In the scoring model, a weighted scoring method is used to calculate the comprehensive score, and the severity of the conflict is determined based on the score.

[0195] For resource conflict, the resource application order related to resource competition, resource usage time length, and resource demand urgency are obtained to build a resource conflict severity scoring model. In the scoring model, the longer the resource usage time length, the longer the resource lock, and the greater the impact. The comprehensive score is calculated, and the severity of the conflict is determined according to the numerical value of the score.

[0196] For priority conflict, task value density, task urgency, and rescheduling cost are obtained to build a priority conflict severity scoring model. In the scoring model, the greater the value density difference, the more difficult the conflict to coordinate, and the value function is mapped after difference normalization. In the urgency function, the higher the urgency, the greater the delay cost. In the rescheduling function, the higher the cost, the more it avoids rescheduling. In the priority function, the greater the priority difference, the higher priority should be given to high priority. The comprehensive score is calculated, and the severity of the conflict is determined according to the numerical value of the score.

[0197] In this embodiment, corresponding evaluation indicators are used for differential evaluation of path conflict, resource conflict, and priority conflict, improving the accuracy and reliability of conflict severity.

[0198] In some embodiments, the decision scheduling layer includes a calculation module, a decision module, and a scheduling module connected in sequence.

[0199] The calculation module is configured to calculate an adaptation degree of the target robot executing the to-be-executed task based on the spatiotemporal semantic map, the heterogeneous capability vector of the target robot, and the task demand vector of the to-be-executed task.

[0200] According to the conflict type and conflict severity in the conflict information, and the task urgency, task importance, and adaptation degree of the to-be-executed task, the task execution priority of each target robot is calculated through the value function.

[0201] The decision module is configured to dynamically adjust the task execution priority of the target robot corresponding to the to-be-executed task according to the task execution priority of each target robot, and generate an adjusted task priority list.

[0202] The scheduling module is configured to rearrange the to-be-executed task corresponding to the target robot according to the adjusted task priority list, and generate the task sequence including task order and execution time.

[0203] The heterogeneous capability vector is determined according to the difference in execution capability of the target robot on different task types, the task demand vector is used to represent the demand characteristics of the to-be-executed task on the robot capability, and the adaptation degree is used to represent the matching degree between the target robot and the to-be-executed task.

[0204] It can be understood that the heterogeneous capability vector quantifies different types of capabilities of the robot in the form of a vector, facilitating matching calculation by the computer, and the heterogeneous capability is the difference between different robots in terms of motion capability, operation precision, perception range and endurance capability.

[0205] The task demand vector quantifies the capability requirements of the task on the executor in the form of a vector.

[0206] The adaptation degree is a matching score obtained by comparing the capability vector and the demand vector.

[0207] The task priority list is a task list formed according to the task execution priority, which is used to list the new priority order of the target robot corresponding to the to-be-executed task.

[0208] In the calculation module, the heterogeneous capability vector of the target robot i is obtained, the task demand vector of the to-be-executed task j is obtained, the specific requirements of the task on the executor in these capability dimensions are clarified, the adaptation degree score is calculated for each target robot i and the task demand vector of the to-be-executed task j, and the task urgency and task importance information from the perception detection layer are combined;

[0209] The adaptation degree, task urgency, task importance and other factors are input into the value function, the value function performs weighted summation on the adaptation degree, task urgency and task importance according to the internal preset weight distribution rule, and the task execution priority is obtained.

[0210] In the decision module, the dynamic priority adjustment mechanism is started, the task completion of other robots, the real-time availability of resources and the like are comprehensively considered, the to-be-executed task corresponding to each target robot is analyzed, and it is judged whether there is a possibility to further optimize the task execution order, for example, if a low-priority task becomes executable in advance due to resource release or path unblocking, and does not cause substantial impact on high-priority tasks, then the decision module will adjust the priority of the task to make it execute in advance, and generate a task priority list.

[0211] The scheduling module plans the optimal task execution path and schedule according to the task priority list, considers the task order and execution time, avoids path conflict and resource conflict, rearranges the to-be-executed task corresponding to the target robot, and generates a task sequence containing detailed task order and execution time.

[0212] In this embodiment, the most suitable robot execution can be selected for the task by calculating the degree of adaptation between the heterogeneous capability vector and the task demand vector, the strengths of different robots are fully utilized, and the success rate and efficiency of task execution are improved. According to the type and severity of the conflict occurring in real time, the priority is dynamically calculated and adjusted in combination with the task and robot state, and dynamic and flexible priority adjustment is realized.

[0213] In some embodiments, the execution control layer includes interconnected execution modules and scheduling modules.

[0214] The execution module is configured to determine the motion trajectory and action parameter required by each target robot to execute the corresponding to-be-executed task according to the task sequence and the running information of the target robot.

[0215] The scheduling module is configured to generate the dynamic scheduling instruction according to the motion trajectory and action parameter and the space-time semantic map, wherein the dynamic scheduling instruction includes the to-be-executed task of the target robot, the motion trajectory, the action parameter, and the execution time.

[0216] It can be understood that the motion trajectory and action parameter are specific motion commands that can be understood by the underlying robot controller converted by the execution control layer from the abstract task sequence.

[0217] After receiving the task sequence, the execution module obtains the specific requirements of each task, such as the task type, target position, expected completion time, etc. At the same time, the real-time running information of the target robot is obtained, including the current position, speed, power, and possible fault state, etc. The motion trajectory required by each target robot to execute the task is calculated, as well as the corresponding action parameter on the motion trajectory. The motion trajectory considers the accessibility in space and takes into account the optimality in time to avoid conflicts with other robots or environmental obstacles. The action parameter includes but is not limited to the moving speed, acceleration, turning angle of the robot, and the force and precision when executing specific tasks (such as grabbing and carrying), etc.

[0218] After receiving the motion trajectory and action parameter provided by the execution module, the scheduling module calls the static environmental information of the space-time semantic map, such as obstacle distribution, road network, etc., as well as the real-time position, motion state, and resource occupation of the robot and other entities, to generate detailed dynamic scheduling instructions for each target robot.

[0219] The dynamic scheduling instruction contains the unique identifier of the target robot, the specific task to be executed, the predetermined motion trajectory, the action parameter, and the execution time, etc.

[0220] In this embodiment, through the close cooperation of the execution module and the scheduling module, the high-level task planning is accurately converted into bottom-level instructions executable by the robot, ensuring the efficiency and safety of task execution. The execution module considers the spatial accessibility and the time optimality by comprehensively analyzing the task requirements and the real-time state of the robot, effectively avoiding path conflicts and resource contention among robots, and accurately setting action parameters, thereby improving the accuracy and efficiency of the robot in executing tasks. The scheduling module uses the rich information of the space-time semantic map to tailor dynamic scheduling instructions for each target robot, ensuring the timeliness and accuracy of the instructions.

[0221] In some embodiments, the execution control layer further comprises a control module connected to the scheduling module.

[0222] The control module is configured to, in the case of receiving feedback information that the target robot executes the to-be-executed task in response to the dynamic scheduling instruction, parse the feedback information to obtain an actual motion trajectory and an action execution situation of the target robot.

[0223] According to the actual motion trajectory and the action execution situation, an update instruction is generated, which is used to trigger an update of the space-time semantic map.

[0224] It can be understood that the update instruction is used to trigger the update mechanism of the space-time semantic map, ensuring that the environmental information, robot state, and resource occupation data presented by the map are highly consistent with the actual scene.

[0225] In actual execution, when the target robot executes the to-be-executed task in response to the dynamic scheduling instruction, the actual motion trajectory and the action execution situation are sent to the control module in the form of feedback information. The control module parses the feedback information to accurately obtain the actual motion trajectory and the action execution situation of the target robot, such as the actual path traveled by the robot, the speed and acceleration at each position, and the specific data such as the force and precision during action execution.

[0226] The control module analyzes the differences between these data and the expected motion trajectory and action parameters. If there is a deviation between the actual execution situation and the expectation, according to the degree and nature of the deviation, an appropriate update instruction is generated to trigger the update of the space-time semantic map. The updated space-time semantic map can more accurately reflect the current environmental state, including the actual position, motion state, and resource occupation of the robot.

[0227] When the target robot responds to the dynamic scheduling instruction and executes the to-be-executed task and feeds back the information, the execution control layer triggers the update of the space-time semantic map to ensure that the entire multi-robot task conflict resolution and dynamic scheduling system can operate efficiently based on the latest and most accurate information.

[0228] The feedback mechanism triggering the update is:

[0229]

[0230] wherein, is the task progress, is the position update, is the energy consumption, is the burst condition, e.g., failure, obstacle appearance, etc.

[0231] The spatio-temporal semantic map at time t is updated as:

[0232]

[0233] wherein, is the updated spatio-temporal semantic map at time t+1; is the update function, which can update the topology of the map through a graph optimization algorithm; is the increment of the conflict set in the spatio-temporal semantic map

[0234] The update content includes but is not limited to the robot trajectory and state, task completion progress, newly detected obstacles or conflict areas, and environmental dynamic changes, e.g., the appearance of a moving obstacle.

[0235] In this embodiment, the spatio-temporal semantic map is updated by analyzing the feedback information and generating update instructions accordingly, ensuring that the entire system makes decisions based on the latest status of the real world.

[0236] In some embodiments, the control module is further configured to:

[0237] In the case of receiving the feedback information, the feedback information is segmented in the time dimension by an upper sliding window to obtain a plurality of segmented information;

[0238] Each segmented information is clustered in the space dimension by a lower sliding window to obtain a plurality of spatio-temporal clustering results;

[0239] According to the spatio-temporal clustering results, the trajectory deviation parameter and the action deviation parameter of the target robot are determined;

[0240] Based on the trajectory deviation parameter and the action deviation parameter, the fluctuation characteristics of the target robot motion trajectory and action execution are analyzed;

[0241] ​​According to the fluctuation characteristics, the window size of the upper layer sliding window and the sliding step of the lower layer sliding window are dynamically adjusted, the trajectory deviation parameter and the action deviation parameter are recalculated, and in a case where the trajectory deviation parameter and the action deviation parameter meet a preset threshold condition, the update instruction is generated.

[0242] The fluctuation characteristics are regularities exhibited by the motion deviation of the robot, such as periodic vibration, random jitter, and persistent deviation, and can be used to diagnose the state of the robot itself or environmental problems.

[0243] In actual implementation, the double-layer sliding window mechanism formed by the upper layer sliding window and the lower layer sliding window can perform fine processing on the feedback information in the time and space dimensions. The upper layer sliding window cuts continuous feedback data into multiple equal time periods along the time axis, and each time period contains the complete motion trajectory and action sequence of the target robot in the time period. The lower layer sliding window performs clustering analysis on the data in each time period in the spatial dimension according to the robot position, motion direction, and action type, and forms a cluster with time and space consistency.

[0244] The trajectory deviation parameter is obtained by calculating the Euclidean distance between the actual trajectory and the expected trajectory in each cluster, the action deviation parameter is calculated by comparing the actual value of the action parameter with the preset value, and the fluctuation characteristic model is constructed according to the trajectory deviation parameter and the action deviation parameter, so as to dynamically identify the periodic deviation and random disturbance of the robot motion.

[0245] When the root mean square error of the trajectory deviation parameter is less than 5% and the coefficient of variation of the action deviation parameter is less than 3%, the system determines that the matching degree of the current motion state and the expectation meets the standard, and the update instruction generated at this time triggers local update of the time and space semantic map. If the deviation parameter continues to exceed the standard, the adaptive adjustment mechanism of reducing the upper layer window length and increasing the lower layer window step is used to realize rapid convergence of the deviation parameter, and the accuracy and real-time performance of the map update decision are ensured.

[0246] In this embodiment, valuable fluctuation characteristics are efficiently extracted from feedback data with time and space labels through the double-layer window design, the window parameters are dynamically adjusted according to the analysis result, the analysis model can be self-optimized according to the data characteristics, the trajectory deviation and the action deviation can be more accurately captured, the accuracy of state evaluation is improved, the time and space dimensions are effectively decoupled, frequent updating of the map due to normal small jitter of the robot or sensor noise is avoided, system fluctuation is reduced, semantic information is updated only when necessary, and the stability of the system is ensured.

[0247] The embodiment of the application also provides a multi-robot task conflict resolution and dynamic scheduling method, which is applied to the multi-robot task conflict resolution and dynamic scheduling system in any of the above embodiments.

[0248] As Figure 2 shown in the figure, the multi-robot task conflict resolution and dynamic scheduling method comprises:

[0249] Step 210, acquiring, by a perception detection layer, running information of a plurality of robots and a spatiotemporal semantic map of tasks to be executed by the robots, and generating conflict information in a case where at least two target robots are detected to have a conflict;

[0250] Step 220, determining, by a decision scheduling layer, a task execution priority of the tasks to be executed according to the conflict information and a priority factor and a value function of each target robot corresponding to the tasks to be executed, to rearrange the tasks to be executed corresponding to the target robots and generate a task sequence, the priority factor comprising a task urgency, a task importance and an execution capability of the target robot, and the value function being used to quantify an influence weight of each priority factor on the task execution priority;

[0251] Step 230, generating, by an execution control layer, a dynamic scheduling instruction according to the task sequence, the spatiotemporal semantic map and the running information of the target robots and delivering the dynamic scheduling instruction to the corresponding target robots.

[0252] According to the multi-robot task conflict resolution and dynamic scheduling method provided by the embodiments of the present application, the conflict is actively and real-timely predicted by the perception detection layer in combination with the running information and the spatiotemporal semantic map, the priority factor and the value function are introduced in the decision scheduling layer, the multi-dimensional information such as the task urgency, the importance and the robot capability can be quantitatively evaluated and the tasks can be rearranged to realize intelligent decision and ensure the key tasks to be executed in priority, the intelligence and rationality of the decision are improved, the dynamic scheduling instruction is delivered by the execution control layer to improve the running efficiency, the automation level and the running efficiency of the multi-robot system, the conflict is real-timely resolved in the task execution process to avoid the robot to be stalled or the task to fail, the stability and the execution efficiency of the multi-robot cooperation system are improved, the dynamic scheduling scene of the multi-robot can be real-timely and flexibly coped with.

[0253] The embodiments of the present application are described above in combination with the drawings, but the present application is not limited to the above-mentioned specific embodiments, the above-mentioned specific embodiments are only illustrative but not limiting, and a person of ordinary skill in the art can make many forms under the inspiration of the present application without departing from the scope of the present application and the protection scope of the claims.

[0254] In the description of the application, reference has been made to descriptive terms such as "one embodiment", "some embodiments", "an embodiment", "example", "specific example" or "some examples" etc. It is emphasized that each of these terms refers to a specific feature, structure, material or characteristic described in connection with a particular embodiment or example. The descriptive terms are not necessarily referring to the same embodiment or example. Moreover, the specific features, structures, materials, or characteristics can be combined in any suitable manner in one or more embodiments or examples.

[0255] While the embodiments of the application have been shown and described, it is to be understood that the embodiments described are only by way of example and that various changes, modifications, substitutions and alterations can be made thereto without departing from the spirit and scope of the application as set forth in the claims and equivalents thereof.

Claims

1. A multi-robot task conflict resolution and dynamic scheduling system, characterized in that, The perception detection layer, the decision scheduling layer and the execution control layer are sequentially connected; The perception detection layer is configured to acquire running information of a plurality of robots and a spatiotemporal semantic map of tasks to be executed by the robots, and generate conflict information when at least two target robots are detected to have a conflict; The decision scheduling layer is configured to determine a task execution priority of the tasks to be executed according to the conflict information and a priority factor and a value function of the tasks to be executed corresponding to each of the target robots, rearrange the tasks to be executed corresponding to the target robots, and generate a task sequence, wherein the priority factor includes a task urgency, a task importance and an execution capability of the target robot, and the value function is used to quantify an influence weight of each priority factor on the task execution priority; The execution control layer is configured to generate a dynamic scheduling instruction according to the task sequence, the spatiotemporal semantic map and the running information of the target robots, and send the dynamic scheduling instruction to the corresponding target robots; The perception detection layer includes a detection module and a judgment module connected with each other; The detection module is configured to model a future trajectory of each of the robots according to the running information and the spatiotemporal semantic map, and obtain a spatiotemporal state corridor; A conflict prediction list between related robots is generated according to the spatiotemporal state corridor and the spatiotemporal semantic map; The judgment module is configured to determine each predicted conflict in the conflict prediction list, and generate the conflict information corresponding to the target robots determined to have a true conflict; The decision scheduling layer includes a calculation module, a decision module and a scheduling module connected with each other; The calculation module is configured to calculate an adaptation degree of the target robot in executing the task to be executed based on the spatiotemporal semantic map, according to a heterogeneous capability vector of the target robot and a task demand vector of the task to be executed; The value function is used to calculate a task execution priority of each of the target robots according to a conflict type and a conflict severity in the conflict information, a task urgency and a task importance of the task to be executed, and the adaptation degree; The decision module is configured to dynamically adjust a task execution priority of the target robot according to the task execution priority of each of the target robots, and generate an adjusted task priority list; The scheduling module is configured to rearrange the tasks to be executed corresponding to the target robots according to the adjusted task priority list, and generate the task sequence including a task order and an execution time; The heterogeneous capability vector is determined according to a difference in execution capability of the target robot in different task types, the task demand vector is used to represent a demand feature of a robot capability of the task to be executed, and the adaptation degree is used to represent a matching degree between the target robot and the task to be executed.

2. The multi-robot task conflict resolution and dynamic scheduling system of claim 1, wherein, The detection module is further configured to: detecting the spatiotemporal state corridor, determining an intersection region and at least two related robots corresponding to the intersection region in a case where it is determined that there is a volume intersection between the spatiotemporal state corridors; determining a region attribute of the intersection region and a spatiotemporal semantic corridor label of the related robots in the intersection region based on the spatiotemporal semantic map; generating the conflict prediction list in a case where it is determined that a conflict occurs between the at least two related robots according to the region attribute and the spatiotemporal semantic corridor label.

3. The multi-robot task conflict resolution and dynamic scheduling system of claim 1, wherein, The conflict information includes a conflict type and a conflict severity, and the judging module includes a screening submodule, an identifying submodule, and a grading submodule connected in sequence. The screening submodule is configured to determine and remove a predicted conflict that is a false positive conflict in the conflict prediction list based on traffic rule semantics, behavior intention semantics, and high-order task constraints in the spatiotemporal semantic map, and to screen at least one true conflict. The identifying submodule is configured to identify a conflict type of the true conflict according to resource distribution, path planning information, and a task priority of the to-be-executed task in the spatiotemporal semantic map. The grading submodule is configured to determine a conflict severity of the true conflict according to the conflict type.

4. The multi-robot task conflict resolution and dynamic scheduling system of claim 3, wherein, The conflict type includes a path conflict, a resource conflict, and a priority conflict, and the grading submodule is further configured to: determine the conflict severity according to a relative speed of a target robot, a collision angle, and a region feature of an intersection region corresponding to the true conflict in a case where the conflict type of the true conflict is determined to be the path conflict; determine the conflict severity according to a target robot resource application order, a resource use time length, and a resource demand urgency of the true conflict in a case where the conflict type of the true conflict is determined to be the resource conflict; determine the conflict severity according to a task value density of the target robot, a task urgency, a rescheduling cost, and a task priority difference of the true conflict in a case where the conflict type of the true conflict is determined to be the priority conflict. The task value density is used to represent a ratio of a benefit to an execution cost of a task executed by the target robot, the task urgency is used to represent a time constraint degree of the to-be-executed task, and the rescheduling cost is used to represent an additional cost generated due to task rearrangement.

5. The multi-robot task conflict resolution and dynamic scheduling system of claim 1, wherein, The execution control layer includes an execution module and a scheduling module connected to each other. The execution module is configured to determine a motion trajectory and an action parameter required for each target robot to execute a corresponding to-be-executed task according to the task sequence and running information of the target robot. The scheduling module is configured to generate the dynamic scheduling instruction according to the motion trajectory and the action parameter and the spatiotemporal semantic map, the dynamic scheduling instruction including the to-be-executed task of the target robot, the motion trajectory, the action parameter, and an execution time.

6. The multi-robot task conflict resolution and dynamic scheduling system of claim 5, wherein, The execution control layer further includes a control module connected to the scheduling module. The control module is configured to, in a case where feedback information that the target robot responds to the dynamic scheduling instruction and executes the to-be-executed task is received, parse the feedback information to obtain an actual motion trajectory and an action execution condition of the target robot; According to the actual motion trajectory and the action execution condition, an update instruction is generated, and the update instruction is used to trigger updating the space-time semantic map.

7. The multi-robot task conflict resolution and dynamic scheduling system of claim 6, wherein, The control module is further configured to: In a case where the feedback information is received, segment the feedback information in a time dimension by using an upper-layer sliding window to obtain a plurality of segmented information; Cluster each piece of segmented information in a space dimension by using a lower-layer sliding window to obtain a plurality of space-time clustering results; According to the space-time clustering results, determine a trajectory deviation parameter and an action deviation parameter of the target robot; Based on the trajectory deviation parameter and the action deviation parameter, analyze fluctuation characteristics of the motion trajectory and the action execution of the target robot; According to the fluctuation characteristics, dynamically adjust a window size of the upper-layer sliding window and a sliding step of the lower-layer sliding window, recalculate the trajectory deviation parameter and the action deviation parameter, and in a case where the trajectory deviation parameter and the action deviation parameter satisfy a preset threshold condition, generate the update instruction.

8. A multi-robot task conflict resolution and dynamic scheduling method, characterized in that, The method is applied to the multi-robot task conflict resolution and dynamic scheduling system according to any one of claims 1-7, and the method comprises: Through a perception detection layer, obtain running information of a plurality of robots and a space-time semantic map of the robots executing to-be-executed tasks, and in a case where it is detected that at least two target robots have a conflict, generate conflict information; Through a decision scheduling layer, according to the conflict information and a priority factor and a value function of each target robot corresponding to a to-be-executed task, determine a task execution priority of the to-be-executed task, rearrange the to-be-executed task corresponding to the target robot, generate a task sequence, and the priority factor comprises a task urgency, a task importance and an execution capability of the target robot, and the value function is used to quantify an influence weight of each priority factor on the task execution priority; Through an execution control layer, according to the task sequence, the space-time semantic map and the running information of the target robot, generate a dynamic scheduling instruction and send the dynamic scheduling instruction to the corresponding target robot; The perception detection layer comprises a detection module and a judgment module connected with each other; through the perception detection layer, the running information of the plurality of robots and the space-time semantic map of the robots executing the to-be-executed tasks are obtained, and in a case where it is detected that at least two target robots have a conflict, the conflict information corresponding to the target robot determined as a true conflict is generated, comprising: Through the detection module, according to the running information and the space-time semantic map, model a future trajectory of each robot to obtain a space-time state corridor; According to the space-time state corridor and the space-time semantic map, generate a conflict prediction list between related robots; Through the judgment module, determine each predicted conflict in the conflict prediction list to generate the conflict information corresponding to the target robot determined as a true conflict. The decision scheduling layer comprises a calculation module, a decision module and a scheduling module connected in sequence; the decision scheduling layer is used to determine a task execution priority of the to-be-executed task according to the conflict information and a priority factor and a value function of the to-be-executed task corresponding to each target robot, to rearrange the to-be-executed task corresponding to the target robot, and to generate a task sequence, wherein the priority factor comprises a task urgency, a task importance and an execution capability of the target robot, and the value function comprises: The calculation module is used to calculate an adaptability of the target robot to the to-be-executed task based on the spatio-temporal semantic map according to a heterogeneous capability vector of the target robot and a task demand vector of the to-be-executed task; The value function is used to calculate a task execution priority of each target robot according to a conflict type and a conflict severity in the conflict information, a task urgency, a task importance of the to-be-executed task and the adaptability; The decision module is used to dynamically adjust a task execution priority of the to-be-executed task corresponding to each target robot according to the task execution priority of each target robot, to generate an adjusted task priority list; The scheduling module is used to rearrange the to-be-executed task corresponding to the target robot according to the adjusted task priority list, to generate the task sequence comprising a task order and an execution time; The heterogeneous capability vector is determined according to a difference in execution capability of the target robot in different task types, the task demand vector is used to represent a demand feature of the to-be-executed task on the robot capability, and the adaptability is used to represent a matching degree between the target robot and the to-be-executed task.

Images