Multi-robot task conflict resolution and dynamic scheduling system and method

Abstract

The invention discloses a multi-robot task conflict resolution and dynamic scheduling system and method, and belongs to the technical field of robot control. The system comprises a perception detection layer which is used for acquiring operation information of a plurality of robots and a space-time semantic map of a to-be-executed task executed by the robots, and generating conflict information under the condition that at least two target robots are detected to conflict; the decision scheduling layer is used for determining task execution priorities of the to-be-executed tasks according to the conflict information and priority factors and value functions of the to-be-executed tasks corresponding to the target robots so as to rearrange the to-be-executed tasks corresponding to the target robots and generate a task sequence; and the execution control layer is used for generating a dynamic scheduling instruction according to the task sequence, the space-time semantic map and the operation information of the target robot and issuing the dynamic scheduling instruction to the corresponding target robot. The system can flexibly cope with a dynamic scheduling scene of multiple robots in real time.

Description

Technical Field

[0001] This application belongs to the field of robot control technology, and in particular relates to a multi-robot task conflict resolution and dynamic scheduling system and method. Background Technology

[0002] With the rapid development of automation technology, multi-robot systems (MRS) have been widely used in warehousing and logistics, intelligent inspection, and collaborative manufacturing, improving the efficiency and coverage of task execution through collaborative work among robots. However, when operating in complex and dynamic environments, MRS often face core challenges such as task allocation, path planning, and conflict resolution.

[0003] Currently, in order to solve the problems of task allocation, path planning and conflict resolution for multi-robots, genetic algorithms are used to re-encode and group tasks, simplifying large-scale integer programming problems, and specific solutions are designed for navigation conflicts and changes in task volume.

[0004] However, the above-mentioned technical solutions require iterative optimization and global search, resulting in insufficient response speed. When dealing with dynamic scheduling scenarios involving multiple robots, they suffer from insufficient support for real-time performance and flexibility. Summary of the Invention

[0005] This application aims to address at least one of the technical problems existing in the prior art. To this end, this application proposes a multi-robot task conflict resolution and dynamic scheduling system and method, which can flexibly and in real time respond to dynamic scheduling scenarios of multiple robots.

[0006] In a first aspect, this application provides a multi-robot task conflict resolution and dynamic scheduling system, the system comprising: It includes a perception and detection layer, a decision-making and scheduling layer, and an execution and control layer connected in sequence; The perception and detection layer is used to acquire the operation information of multiple robots and the spatiotemporal semantic map of the robots performing the tasks to be performed. When a conflict is detected between at least two target robots, conflict information is generated. The decision scheduling layer is used to determine the task execution priority of the task to be executed based on the conflict information and the priority factor and value function of the task to be executed corresponding to each target robot, so as to rearrange the task to be executed corresponding to the target robot and generate a task sequence. The priority factor includes the urgency of the task, the importance of the task and the execution capability of the target robot. The value function is used to quantify the influence weight of each priority factor on the task execution priority. The execution control layer is used to generate dynamic scheduling instructions and send them to the corresponding target robot based on the task sequence, the spatiotemporal semantic map, and the operation information of the target robot.

[0007] According to one embodiment of this application, the perception detection layer includes a detection module and a judgment module that are interconnected; The detection module is used to model the future trajectory of each robot based on the operation information and the spatiotemporal semantic map to obtain a spatiotemporal state corridor. Based on the spatiotemporal state corridor and the spatiotemporal semantic map, a conflict prediction list between relevant robots is generated; The judgment module is used to judge each predicted conflict in the conflict prediction list and generate the conflict information corresponding to the target robot that will be determined as a real conflict.

[0008] According to one embodiment of this application, the detection module is further configured to: The spatiotemporal state corridor is detected, and if it is determined that there is a volume intersection between the spatiotemporal state corridors, the intersection region and at least two related robots corresponding to the intersection region are determined. Based on the spatiotemporal semantic map, determine the regional attributes of the intersection region and the spatiotemporal semantic corridor labels of the relevant robots in the intersection region; If a conflict is determined between at least two related robots based on the region attributes and the spatiotemporal semantic corridor labels, the conflict prediction list is generated.

[0009] According to one embodiment of this application, the conflict information includes conflict type and conflict severity, and the judgment module includes a filtering submodule, an identification submodule, and a grading submodule connected in sequence. The filtering submodule is used to identify and remove predicted conflicts that are false positives from the conflict prediction list based on traffic rule semantics, behavioral intention semantics and higher-order task constraints in the spatiotemporal semantic map, and to filter out at least one true conflict. The higher-order task constraints include dependency restrictions and collaboration requirement restrictions between tasks to be executed. The identification submodule is used to identify the conflict type of the true conflict based on the resource distribution and path planning information in the spatiotemporal semantic map and the task priority of the task to be executed. The hierarchical submodule is used to determine the severity of the true conflict based on the conflict type.

[0010] According to one embodiment of this application, the conflict types include path conflicts, resource conflicts, and priority conflicts, and the hierarchical submodule is further configured to: If the conflict type of the true conflict is determined to be a path conflict, the severity of the conflict is determined based on the relative speed of the target robot, the collision angle, and the regional characteristics of the intersection area corresponding to the true conflict. If the conflict type of the true conflict is determined to be a resource conflict, the severity of the conflict is determined based on the target robot's resource request order, resource usage duration, and the urgency of the resource demand corresponding to the true conflict. If the conflict type of the true conflict is determined to be a priority conflict, the severity of the conflict is determined based on the target robot's task value density, task urgency, rescheduling cost, and the difference in task priority corresponding to the true conflict. The task value density is used to characterize the ratio of the benefit to the cost of the target robot performing the task, the task urgency is used to characterize the time constraint of the task to be performed, and the rescheduling cost is used to characterize the additional cost caused by task rescheduling.

[0011] According to one embodiment of this application, the decision scheduling layer includes a computing module, a decision module, and a scheduling module connected in sequence: The calculation module is used to calculate the suitability of the target robot to perform the task based on the spatiotemporal semantic map, according to the heterogeneous capability vector of the target robot and the task requirement vector of the task to be performed. Based on the conflict type and severity in the conflict information, the urgency and importance of the task to be executed, and the suitability, the task execution priority of each target robot is calculated through the value function. The decision module is used to dynamically adjust the priority of the tasks to be executed corresponding to the target robot according to the task execution priority of each target robot, and generate an adjusted task priority list. The scheduling module is used to rearrange the tasks to be executed corresponding to the target robot according to the adjusted task priority list, and generate the task sequence containing the task order and execution time. The heterogeneous capability vector is determined based on the differences in the target robot's execution capabilities across different task types. The task requirement vector is used to characterize the capability requirements of the task to be executed. The fit is used to characterize the degree of matching between the target robot and the task to be executed.

[0012] According to one embodiment of this application, the execution control layer includes an execution module and a scheduling module that are interconnected. The execution module is used to determine the motion trajectory and action parameters required for each target robot to perform the corresponding task based on the task sequence and the operation information of the target robot. The scheduling module is used to generate the dynamic scheduling instruction based on the motion trajectory and action parameters, as well as the spatiotemporal semantic map. The dynamic scheduling instruction includes the target robot's task to be executed, the motion trajectory, the action parameters, and the execution time.

[0013] According to one embodiment of this application, the execution control layer further includes a control module connected to the scheduling module; The control module is used to parse the feedback information received when the target robot responds to the dynamic scheduling instruction and executes the task to be executed, so as to obtain the actual motion trajectory and action execution status of the target robot. Based on the actual motion trajectory and action execution, an update instruction is generated, which is used to trigger the update of the spatiotemporal semantic map.

[0014] According to one embodiment of this application, the control module is further configured to: Upon receiving the feedback information, the feedback information is segmented in the time dimension through an upper sliding window to obtain multiple segmented information. By using a lower-level sliding window, each segment of information is clustered in the spatial dimension to obtain multiple spatiotemporal clustering results; Based on the spatiotemporal clustering results, the trajectory deviation parameters and motion deviation parameters of the target robot are determined; Based on the trajectory deviation parameters and the motion deviation parameters, the fluctuation characteristics of the target robot's motion trajectory and motion execution are analyzed; The window size of the upper sliding window and the sliding step size of the lower sliding window are dynamically adjusted according to the fluctuation characteristics. The trajectory deviation parameter and the motion deviation parameter are recalculated. When the trajectory deviation parameter and the motion deviation parameter meet the preset threshold conditions, the update instruction is generated.

[0015] Secondly, this application provides a method for resolving and dynamically scheduling multi-robot task conflicts, applied to the multi-robot task conflict resolution and dynamic scheduling system as described in the first aspect. The method includes: The perception and detection layer acquires the operation information of multiple robots and the spatiotemporal semantic map of the robots performing the tasks to be performed. When a conflict is detected between at least two target robots, conflict information is generated. Through the decision scheduling layer, based on the conflict information and the priority factor and value function of the task to be executed corresponding to each target robot, the task execution priority of the task to be executed is determined, so as to rearrange the task to be executed corresponding to the target robot and generate a task sequence. The priority factor includes the urgency of the task, the importance of the task, and the execution capability of the target robot. The value function is used to quantify the influence weight of each priority factor on the task execution priority. By executing the control layer, dynamic scheduling instructions are generated and sent to the corresponding target robot based on the task sequence, the spatiotemporal semantic map, and the target robot's operation information.

[0016] Additional aspects and advantages of this application will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of this application.

[0017] This application provides a multi-robot task conflict resolution and dynamic scheduling system and method, which has the following advantages over the prior art: (1) By combining the perception and detection layer with the operation information and the spatiotemporal semantic map, the conflict can be actively predicted in real time. Priority factors and value functions are introduced in the decision scheduling layer, which can perform quantitative evaluation and task rearrangement based on multi-dimensional information such as the urgency, importance and robot capabilities of the task, realize intelligent decision-making, ensure the priority execution of key tasks, and improve the intelligence and rationality of decision-making. By issuing dynamic scheduling instructions through the execution control layer, the operating efficiency, automation level and operating efficiency of the multi-robot system are improved. By resolving conflicts in real time during task execution, robot stagnation or task failure is avoided, and the stability and execution efficiency of the multi-robot collaborative system are improved. It can respond to the dynamic scheduling scenario of multi-robots in real time.

[0018] (2) By modeling future trajectories and spatiotemporal state corridors, conflict detection is upgraded from simple current spatial overlap to prediction of future spatiotemporal domains. Potential conflicts can be discovered earlier and more accurately. By performing secondary verification on the conflict prediction list, false positive conflicts can be filtered out, avoiding unnecessary scheduling of non-existent problems by the system, saving computing resources and improving system efficiency. Attached Figure Description

[0019] The above and / or additional aspects and advantages of this application will become apparent and readily understood from the description of the embodiments taken in conjunction with the following drawings, in which: Figure 1 This is a schematic diagram of the structure of the multi-robot task conflict resolution and dynamic scheduling system provided in the embodiments of this application; Figure 2This is a flowchart illustrating the multi-robot task conflict resolution and dynamic scheduling method provided in the embodiments of this application. Detailed Implementation

[0020] The technical solutions of the embodiments of this application will be clearly described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this application. All other embodiments obtained by those skilled in the art based on the embodiments of this application are within the scope of protection of this application.

[0021] The terms "first," "second," etc., used in the specification and claims of this application are used to distinguish similar objects and not to describe a specific order or sequence. It should be understood that such use of data can be interchanged where appropriate so that embodiments of this application can be implemented in orders other than those illustrated or described herein, and the objects distinguished by "first," "second," etc., are generally of the same class and the number of objects is not limited; for example, a first object can be one or more. Furthermore, in the specification and claims, "and / or" indicates at least one of the connected objects, and the character " / " generally indicates that the preceding and following objects are in an "or" relationship.

[0022] The following description, in conjunction with the accompanying drawings, details the multi-robot task conflict resolution and dynamic scheduling system and method provided in this application through specific embodiments and application scenarios.

[0023] like Figure 1 As shown, the multi-robot task conflict resolution and dynamic scheduling system includes a perception and detection layer 110, a decision scheduling layer 120 and an execution control layer 130 connected in sequence. The perception and detection layer 110 is used to acquire the operation information of multiple robots and the spatiotemporal semantic map of the robots performing tasks to be performed. When a conflict is detected between at least two target robots, conflict information is generated. The decision scheduling layer 120 is used to determine the task execution priority of the task to be executed based on the conflict information and the priority factor and value function of the task to be executed corresponding to each target robot, so as to rearrange the task to be executed corresponding to the target robot and generate a task sequence. The priority factor includes the urgency of the task, the importance of the task and the execution capability of the target robot. The value function is used to quantify the influence weight of each priority factor on the task execution priority. The execution control layer 130 is used to generate dynamic scheduling instructions and send them to the corresponding target robot based on the task sequence, the spatiotemporal semantic map, and the operation information of the target robot.

[0024] Understandably, operational information refers to the multi-dimensional state data generated by the robot during its work, including the robot's operational state and environmental information. The operational state refers to the specific situation in which the robot is performing a task. For example, operational information includes the robot's position, motion state, task progress, fault status, and dynamic constraints. Dynamic constraints are the physical movement limitations of the robot, including maximum speed, acceleration, minimum turning radius, load capacity, etc., as well as endurance, load capacity, and movement speed limitations, which can affect the robot's possible movement trajectory. Environmental information refers to the current operating environment of all robots.

[0025] The tasks to be executed are work instructions pre-assigned to the robot by the system, which include information such as task type, target location, start and end time.

[0026] A spatiotemporal semantic map is a map that integrates spatial structure, temporal dynamics, and semantic information. It includes spatial information such as obstacles and roads on the map, robot movement trajectories, task execution areas, and resource occupancy status in three-dimensional spatiotemporal information. It also includes temporal information such as whether a certain area is available at a certain time, as well as semantic information such as charging stations, highway channels, or loading and unloading areas. It not only records the robot's current position and task area, but also includes potential conflict areas, resource occupancy status, and the status changes of each area at different points in time, providing the system with comprehensive environmental perception capabilities.

[0027] The target robot is the robot that the system identifies as being involved in a conflict that needs to be handled and resolved.

[0028] Conflict information is an alarm message generated when multiple robots' paths, resource usage, or task plans are predicted to conflict. It includes the conflicting parties, conflict location, expected occurrence time, conflict type, and conflict severity. Conflict types include path intersection conflict, resource contention conflict, and time-dependent conflict. Conflict severity is quantified and graded according to the degree of impact of the conflict on task execution.

[0029] Priority factors are a set of indicators used to assess the urgency and importance of tasks to be performed. In priority factors, the urgency of a task is determined based on how close it is to the deadline, while the importance of a task is used to assess the value or severity of the consequences of the task to be performed, including safety criticality and production impact. The execution capability of the target robot includes the efficiency and adaptability of the target robot in completing the task, its adaptability to the task environment, the accuracy of its actions, its energy utilization capability, as well as remaining power, load status, and historical efficiency.

[0030] The value function is used to transform priority factors into a unified benefit index.

[0031] Task execution priority is the weight of the task execution order derived by the value function after comprehensively evaluating the priority factors, and is used to sort all tasks to be executed.

[0032] A task sequence is a task plan table that includes the task order, start and end times, and resource allocation.

[0033] Dynamic scheduling instructions are adjustable control commands issued in real time to each target robot. They include specific details such as the target point, path, speed, and action type of the task to be executed, as well as the execution order and execution time, to ensure that the robot can execute the corresponding actions accurately and efficiently.

[0034] Upon receiving feedback from the target robot in response to the dynamic scheduling command and executing the task to be performed, the execution control layer triggers an update to the spatiotemporal semantic map. This update continuously optimizes the system based on the robot's actual execution status and environmental changes. The feedback information is a result status report returned by the robot after executing the dynamic scheduling command, used to form closed-loop control. This process enhances the system's robustness in complex and changing environments, ensuring its dynamic adaptability.

[0035] The perception and detection layer can be deployed on the edge computing unit. By collecting and analyzing the operation information in real time, it can detect potential conflicts between at least two target robots and generate conflict information.

[0036] The decision scheduling layer can be deployed on an industrial-grade server. It takes the received conflict information, combines the priority factor of each target robot and the value function of the task to be executed, determines the task execution priority of each task to be executed, rearranges the tasks to be executed corresponding to the target robot, and generates a task sequence.

[0037] The execution control layer can be deployed on the processor of each robot body. Based on the generated task sequence, spatiotemporal semantic map and the target robot's operation information, it generates targeted dynamic scheduling instructions and accurately sends them to the corresponding target robot's controller.

[0038] In practice, the system can be deployed in intelligent warehousing and logistics centers. When material handling is required, n robots can perform m tasks such as handling, sorting, and shelving.

[0039] The global state vector of the system at time t is: in, It is the spatiotemporal semantic map at time t. It is a set of n running states. This is the operating state of the i-th robot. It is the position of the i-th robot. It is the speed of the i-th robot. It is the remaining battery power of the i-th robot. It represents the execution capability of the i-th robot; It is a set of states for m tasks to be executed. This is the state of the j-th task to be executed. This is the progress of the j-th task to be executed. It is the priority factor of the j-th task to be executed. It is the position of the j-th task to be executed. It is the deadline for the j-th task to be executed; It is the conflict information corresponding to the set of predicted conflicts, including information on K predicted conflicts. The information on each predicted conflict includes the relevant robot pair involved in the conflict, the conflict type, the conflict location, etc.

[0040] In the perception detection layer, the spatiotemporal semantic map at time t for: in, It is a perceptual fusion function, built based on Simultaneous Localization and Mapping (SLAM) technology. It is the environmental information monitored by the environmental sensors at time t, including spatiotemporal information such as environmental topology, obstacles, and task points.

[0041] Perform collision detection on the robots in the system: in, It is a conflict detection function that quantifies information such as spatiotemporal occupancy maps, resource occupancy matrices, and estimated time of arrival (ETA) distributions into conflict indicators, and performs three-dimensional matching within a sliding time window to determine conflicts. It is the scheduling state at time t.

[0042] Collision detection function The decision-making rules include path conflict rules, resource conflict rules, and priority conflict rules.

[0043] For path conflicts, the two paths are too close in space. Make In cases where two paths are too close in space, they are considered to be in conflict. The spatiotemporal curve for path planning of robot i. The spatiotemporal curve for path planning of robot i+1. This is the minimum safe distance threshold for path conflicts. Regarding resource conflicts, when multiple robots are assigned the same task... j makes In this case, it is determined to be a resource conflict. For the j-th task to be executed The currently assigned set of robots, For the (i+1)th robot, ; In the case of priority conflict, if the j-th task to be executed is requested by more than two robots, obtain the value vector of the robot for the j-th task to be executed. for: Among them, elements The value quantification terms are obtained through the value function, and then sorted from largest to smallest: Select the value quantification terms of the two robots applying for the j-th task to be executed. and and priority fuzzy threshold Perform numerical determination, in In such cases, it is determined to be a priority conflict.

[0044] For each predicted conflict, the conflict type, the relevant target robot, and the tasks to be performed are extracted, and the conflict severity is calculated. The conflict severity is a scalar determined by combining indicators such as time urgency, resource scarcity, and security risk.

[0045] For example, in the case where the k-th predicted conflict is a path conflict, the corresponding conflict severity is... for: in, For path and path Minimum distance between them; is the time urgency weighting coefficient; tu is the time urgency factor, used to characterize the degree of time urgency of the expected occurrence of the conflict, and is obtained as a function of the difference between the expected occurrence time of the conflict and the current time.

[0046] When the k-th predicted conflict is a resource conflict, the corresponding conflict severity is... for: in, It refers to the number of robots competing for the same resource; This is the maximum number of robots that the resource can support simultaneously. It is a resource scarcity weighting coefficient. It is a resource scarcity factor used to characterize the degree of scarcity of a resource in a system, and can be obtained based on the historical frequency of resource use or the proportion of demand.

[0047] When the k-th predicted conflict is a priority conflict, the corresponding conflict severity is... for: in, The smaller the value, the lower the severity of the conflict.

[0048] In the decision-making and scheduling layer, the priority factor for the i-th robot to execute the j-th task is... for: in, As the progress factor, This is the progress of the j-th task to be executed. It represents the deadline for the j-th task to be executed. The closer to the deadline and the slower the progress, the more urgent the task. As a task importance factor, high-value or critical path tasks are more important, and can be determined based on the inherent value of the task and the weight of dependencies. The robot capability factor is used to characterize the degree of matching between robot capabilities and task requirements. The better the robot capabilities match the task to be performed, the higher the score.

[0049] At time t, the i-th robot executes the j-th task to be executed, and the task execution priority is calculated using the value function. for: in, As weight, satisfying .

[0050] The state space is: in, It is the spatiotemporal semantic map at time t. This is the operating state of the i-th robot. This is the state of the j-th task to be executed. It is the conflict information corresponding to the set of predicted conflicts. It is the scheduling state at time t, used to characterize the task allocation sequence and historical decision records at the current time, including the task queue and execution state of each robot.

[0051] In the action space: in, It is a binary variable used to indicate whether robot i executes task j.

[0052] With the goal of maximizing the total value of robot and task allocation, the objective function is constructed as follows: in, This means that each task can be assigned to at most one robot; , represents the maximum number of tasks that the i-th robot can perform. It is the number of tasks assigned to the i-th robot; Tasks to be executed It must be within the time window and reachable by the robot; it represents the order of task dependencies. It is the conflict cost penalty coefficient. It is the cost of conflict.

[0053] Furthermore, we solve this using dynamic programming: in, It is the optimal action value function, representing the expected reward after performing action a from state s; It is the reward function, which represents the immediate utility gained after performing action a from state s; It is a discount factor used to control the importance of future rewards; This represents the expected value of the state transition to s' after performing action a from state s. It is the state transition probability function, representing the probability of transitioning to state s' after taking action a in state s. It is the optimal state value function of state s', which represents the maximum expected reward that can be obtained by following the optimal policy starting from state s'; It is the optimal state value function of state s, which represents the maximum expected reward that can be obtained by starting from state s and executing according to the optimal policy; The optimal policy in state s represents the action taken in state s that maximizes... The optimal action.

[0054] reward function for: in, It is the value of the task; It is the cost of conflict. It is the severity of the conflict. It is a conflict cost function, indicating whether action a triggers a conflict. ; It is a consumption of resources. It is the resource consumption of a single task, determined based on energy consumption, time consumption, and wear and tear.

[0055] Each robot is sorted and assigned tasks according to their priority, resulting in the following task sequence: in, It is a function that arranges tasks in order of priority. It takes an unordered set of tasks as input and outputs an ordered sequence of tasks arranged from highest to lowest priority. In the tasks assigned to and about to be performed by robot i, all decision variables Tasks to be executed A set of.

[0056] At the execution control layer, dynamic scheduling instructions for the i-th robot are generated. for: in, It is a control command generator, which is obtained by concatenating a trajectory generator, a trajectory interpolator, a velocity-attitude controller, and an actuator command mapping function. In the trajectory generator, Model Predictive Control (MPC) is used in the spatiotemporal semantic map. Perform online trajectory optimization and output waypoint sequence for each time point. ,in, These are the robot's two-dimensional coordinates in the spatiotemporal semantic map. t is the angle between the robot's orientation and the positive x-axis, v is the robot's linear velocity along the path, and t is the absolute moment of reaching that pose. In the trajectory interpolator, each waypoint sequence is organized in the form of a time series, and the state component at each time point is interpolated in the time domain. A continuous reference trajectory is obtained by uniformly sampling according to the control frequency. In the velocity-attitude controller, the deviation between the robot's current position and the continuous reference trajectory is calculated by the linear quadratic regulator (LQR), and linear velocity and angular velocity commands are generated. In the actuator instruction mapping function, based on the time requirements of the task sequence, if the task requires loading, taking pictures, switching lights on and off, etc., specific operation instructions such as loading and taking pictures are inserted at the corresponding trajectory key points and inserted into the trajectory key points. All instructions are packaged into a control instruction package according to the time axis and issued as a dynamic scheduling instruction.

[0057] In the velocity-attitude controller, it is necessary to adjust the continuous reference trajectory. The objective function for path planning is: in, It is the time consumption of the path. It is a weighting factor. This refers to the energy consumption along the path, calculated based on factors such as the robot's speed, load, and travel distance. It is the fitness coefficient. It is a path and spatiotemporal semantic map Adaptability.

[0058] The constraints of the path planning subproblem include avoiding obstacles and not conflicting with other robot paths, for example, traversing space and time. Perform path planning for multiple robots.

[0059] The multi-robot task conflict resolution and dynamic scheduling system provided in this application combines operational information and spatiotemporal semantic maps through a perception and detection layer to proactively predict conflicts in real time. Priority factors and value functions are introduced at the decision-making and scheduling layer, enabling quantitative evaluation and task rearrangement based on multi-dimensional information such as task urgency, importance, and robot capabilities. This achieves intelligent decision-making, ensuring the priority execution of critical tasks and improving the intelligence and rationality of decision-making. Dynamic scheduling instructions are issued by the execution control layer, improving the operational efficiency, automation level, and overall efficiency of the multi-robot system. By resolving conflicts in real time during task execution, robot stagnation or task failure is avoided, enhancing the stability and execution efficiency of the multi-robot collaborative system. It can flexibly and in real time respond to dynamic scheduling scenarios involving multiple robots.

[0060] In some embodiments, the perception and detection layer includes a detection module and a judgment module that are interconnected. The detection module is used to model the future trajectory of each robot based on the operation information and the spatiotemporal semantic map to obtain a spatiotemporal state corridor. Based on the spatiotemporal state corridor and the spatiotemporal semantic map, a conflict prediction list between relevant robots is generated; The judgment module is used to judge each predicted conflict in the conflict prediction list and generate the conflict information corresponding to the target robot that will be determined as a real conflict.

[0061] Among them, future trajectory modeling is based on the robot's current state, target, and operational information to predict its future movement path.

[0062] A spatiotemporal state corridor is a four-dimensional spatial region. It adds a time dimension to a spatial path, forming a conduit in a four-dimensional time-space environment, representing the spatial range a robot will occupy over a period of time. For example, on a two-dimensional plane, each time slice corresponds to a potentially occupied area, stacking over time to form a spatiotemporal state corridor.

[0063] The relevant robots are those that predict the conflict.

[0064] Predicted conflicts are potential conflicts that are foreseen. The conflict prediction list includes at least one predicted conflict, as well as the corresponding conflict type, related robots, and expected occurrence time. Real conflicts are conflicts that may actually occur.

[0065] In actual execution, the detection module acquires real-time operating information for each robot, including its current position, speed, orientation, acceleration, task objective, motion mode, and current task status; it loads a spatiotemporal semantic map, which includes geometric structures such as corridors and intersections in the environment, and marks semantic information such as priority areas, restricted periods, and high-frequency areas of dynamic obstacles, as well as availability rules in the time dimension. For example, a certain passage may be reserved for specific types of robots during a specific time period.

[0066] Based on each robot's current state and task objective, its expected motion path for a future period is planned on a spatiotemporal semantic map. Considering the robot's dynamic constraints and environmental semantic constraints, a predictive controller generates confidence intervals for the future trajectory. Environmental semantic constraints include the ability to avoid crossing walls and adherence to one-way street rules. Taking into account robot motion uncertainties and safety margins, the predicted trajectory is extended in both time and space into a time-varying occupied area band, forming a spatiotemporal state corridor. In some embodiments, the detection module is further configured to: The spatiotemporal state corridor is detected, and if it is determined that there is a volume intersection between the spatiotemporal state corridors, the intersection region and at least two related robots corresponding to the intersection region are determined. Based on the spatiotemporal semantic map, determine the regional attributes of the intersection region and the spatiotemporal semantic corridor labels of the relevant robots in the intersection region; If a conflict is determined between at least two related robots based on the region attributes and the spatiotemporal semantic corridor labels, the conflict prediction list is generated.

[0067] Volume intersection extends the area occupied by a robot from its external outline to the volume of space it sweeps through during its movement. The volumes of two robots may overlap in the future, which is the physical basis for conflict.

[0068] The intersection region is the region where the areas occupied by any two or more spatiotemporal state corridors intersect at the same time.

[0069] Regional attributes are the semantic types of intersecting regions on the map, including regional geometric types, traffic rules, physical restrictions, time constraints, and historical behavior patterns. Geometric types include narrow passages, intersections, T-junctions, roundabouts, loading and unloading areas, and elevator lobbies. Traffic rules include one-way streets, two-way streets, no merging, and priority traffic directions. Physical restrictions include width, height, and ground friction coefficient. Time constraints refer to whether the intersecting region is restricted between the earliest and latest times. Historical behavior patterns include common conflict types and average transit times of the intersecting region.

[0070] Spatiotemporal semantic corridor labels are labels that assign semantic information to spatiotemporal state corridors. For example, semantic information includes the robot's role label, motion intention, and the corresponding task type, task attributes, and task priority of the task to be executed.

[0071] In actual execution, the detection module is also used to perform pairwise spatiotemporal intersection detection on the spatiotemporal state corridors of all robots. If the spatiotemporal state corridors of two robots overlap in space within a certain time period, their occupied areas intersect at the same time. All time-space segments where the intersection occurs are recorded to extract the intersection area, identify the relevant robots, and determine the earliest and latest times of the intersection area. The spatial geometry of the intersection area is mapped to a spatiotemporal semantic map and recorded as a potential predicted conflict. If the overlapping area is located in a wide passage that allows parallel passage and the robot types are compatible, the conflict level can be reduced. If the overlap occurs in a narrow passage, intersection, or high-priority area, it is marked as a high-risk conflict.

[0072] All predicted conflicts are integrated into a conflict prediction list and output, which includes the identification (ID) of the relevant robots involved in each prediction list, the time interval of the predicted conflict, the spatial location of the conflict, and the type of conflict.

[0073] In this embodiment, by judging the intersection of volumes and combining the spatiotemporal corridor labels in the spatiotemporal semantic map to make conflict judgment, the obtained re-push information contains a rich semantic scenario description, generating a structured conflict prediction list, making the conflict detection process more refined and semantic, and the output conflict information of higher quality.

[0074] The judgment module receives a complete list of conflict predictions from the detection module. Each record of a predicted conflict is accompanied by a spatiotemporal state corridor and semantic context information. The module analyzes each potential conflict to determine whether it is a real conflict that may actually occur, and eliminates false positives caused by prediction errors or semantic misjudgments.

[0075] In some embodiments, the conflict information includes conflict type and conflict severity, and the judgment module includes a filtering submodule, an identification submodule, and a grading submodule connected in sequence. The filtering submodule is used to identify and remove predicted conflicts that are false positives from the conflict prediction list based on traffic rule semantics, behavioral intention semantics and higher-order task constraints in the spatiotemporal semantic map, and to filter out at least one true conflict. The higher-order task constraints include dependency restrictions and collaboration requirement restrictions between tasks to be executed. The identification submodule is used to identify the conflict type of the true conflict based on the resource distribution and path planning information in the spatiotemporal semantic map and the task priority of the task to be executed. The hierarchical submodule is used to determine the severity of the true conflict based on the conflict type.

[0076] It is understandable that traffic rule semantics refers to virtual traffic rules defined in the map, while behavioral intention semantics refers to the robot's behavioral tendencies.

[0077] Higher-order task constraints are used to characterize the advanced rules or objectives that must be followed during multi-robot collaboration. They are global constraints at the task level that transcend the motion of a single robot.

[0078] In higher-order task constraints, dependency constraints are the sequential or conditional dependencies between different tasks, execution order constraints are fixed sequences prescribed by process flow or physical constraints, and cooperation requirement constraints are the cooperative effects that multiple robots need to achieve during task execution. For example, multiple robots need to arrive at a designated location at the same time to perform cooperative work, or they need to maintain a certain distance within a specific time to avoid collisions.

[0079] False positives are misreports that have no real impact and do not require handling; true conflicts are systemic contradictions that must be resolved.

[0080] In the spatiotemporal semantic map, resource distribution includes the location, quantity, and usage status of resources such as charging piles, workstations, and tools in the system; path planning information is the topology of the robot's current path; and task priority is calculated by the value function on the priority factor.

[0081] The severity of the conflict is obtained through quantitative assessment of the consequences of different conflict types.

[0082] In actual execution, the filtering submodule loads the spatiotemporal semantic map, extracts traffic rule semantics, behavioral intent semantics, and higher-order task constraints, performs traffic rule compliance checks on each predicted conflict to determine whether the predicted conflict is avoidable, conducts behavioral intent semantic analysis to analyze whether the predicted conflict can be downgraded or removed, and conducts higher-order task constraint analysis to check whether there are dependencies or sequence constraints between the two robot tasks. If the tasks are independent and both are independent high-priority tasks, false positive conflicts that will not actually occur due to prediction errors or environmental semantics are filtered out. The remaining conflicts that have not been eliminated by the above rules are considered true conflicts.

[0083] The identification submodule performs fine-grained classification of each genuine conflict based on resource distribution, path planning information, and task priority, and labels each genuine conflict with a precise conflict type, forming intermediate results with type labels.

[0084] For example, if path planning shows that both robots need to pass the center of the intersection at similar times, and the conflict occurs at a crossroads or T-junction, with the two robots' paths perpendicular or oblique, then the conflict type is an intersection contention conflict. If the channel width is less than the sum of the widths of the two robots, and the two robots are traveling in opposite directions with no lateral clearance, then the conflict type is an opposing channel blockage conflict. If a robot on one side attempts to merge into the main road, and its timing is close to that of the robot on the main road, and the main road has priority, but the robot on the main road has a lower task priority and may need to coordinate, then the conflict type is a merging competition conflict. If the conflict occurs near a unique resource point, such as a unique elevator entrance or charging station, and both robots need to use this resource with overlapping scheduled times, then the conflict type is a resource contention conflict. If two robots are traveling in the same direction, the speed of the rear robot is higher than that of the front robot, the distance is too close, the path overlap is high, the relative speed is large, and the braking distance is insufficient, then the conflict type is a rear-end collision risk conflict.

[0085] The grading submodule quantifies the severity of conflicts based on factors such as conflict type, task priority, and environmental risk, using rule mapping and weighted scoring. It then categorizes the severity of conflicts according to the numerical values. In cases of high conflict severity, immediate interventions such as path replanning and forced waiting are required. In cases of medium conflict severity, local speed adjustments can be implemented to mitigate the situation. In cases of low conflict severity, the system can self-coordinate without alarms, and will output conflict information.

[0086] In this embodiment, global consistency at the task level is improved by introducing higher-order task constraints, false positives are filtered out through semantic and task constraints to enhance system reliability, and conflicts are prioritized by identifying conflict types and determining conflict severity to rationally allocate computing and scheduling resources.

[0087] By modeling future trajectories and using spatiotemporal state corridors, conflict detection is elevated from simple current spatial overlap to prediction of future spatiotemporal domains. This enables the earlier and more accurate discovery of potential conflicts. By performing secondary verification on the conflict prediction list, false positive conflicts can be filtered out, avoiding unnecessary scheduling of non-existent problems by the system, saving computational resources and improving system efficiency.

[0088] In some embodiments, the conflict types include path conflicts, resource conflicts, and priority conflicts, and the hierarchical submodule is further configured to: If the conflict type of the true conflict is determined to be a path conflict, the severity of the conflict is determined based on the relative speed of the target robot, the collision angle, and the regional characteristics of the intersection area corresponding to the true conflict. If the conflict type of the true conflict is determined to be a resource conflict, the severity of the conflict is determined based on the target robot's resource request order, resource usage duration, and the urgency of the resource demand corresponding to the true conflict. If the conflict type of the true conflict is determined to be a priority conflict, the severity of the conflict is determined based on the target robot's task value density, task urgency, rescheduling cost, and the difference in task priority corresponding to the true conflict. The task value density is used to characterize the ratio of the benefit to the cost of the target robot performing the task, the task urgency is used to characterize the time constraint of the task to be performed, and the rescheduling cost is used to characterize the additional cost caused by task rescheduling.

[0089] Path conflict is used to characterize the spatiotemporal intersection of a robot's future trajectory, where there is no natural avoidance path, which may lead to collisions or blockages. These include intersection contention, oncoming channel blocking conflict, merging competition conflict, rear-end collision risk conflict, and detour conflict.

[0090] Resource conflict is used to characterize multiple robots making concurrent requests for the same limited resource, resulting in task delays or blockages. This includes resource contention conflicts, as well as competition for resources such as tools and equipment.

[0091] Priority conflicts arise when multiple robots' tasks have temporal dependencies or differing priority determinations, or when high-value / urgent tasks and low-priority tasks are on the same resource or path, resulting in conflicting execution orders.

[0092] Area characteristics include area type, avoidability, and dynamic complexity. Area type includes narrow passages, intersections, blind spots, ramps, etc. Avoidability includes whether there are lateral detour paths and waiting areas. Dynamic complexity includes whether it is a high-frequency intersection area for multiple robots.

[0093] Resource request ranking is used to characterize the order in which the two robots reserve the resource. Resource usage duration is the estimated time period from the start to the end of resource usage. Resource demand urgency is determined based on the difference between the task deadline and the current time, and is assigned a task weight.

[0094] Task value density is the ratio of the benefits of a target robot performing a task to its execution cost. Task benefits include transportation value and service rating, while execution costs include energy consumption, time, and manpower. Task urgency is quantified by current time, execution time, and execution duration. The cost of rescheduling includes the computational overhead of path replanning, communication latency, the cost of chain adjustments to other robots, as well as the delay of other tasks and the increase in total distance consumption. It can be calculated by the number of robots affected by rescheduling and the cost of a single adjustment. Task priority differences are used to characterize the differences in urgency and importance of different tasks in achieving the overall system goal.

[0095] In actual execution, the hierarchical submodule receives a set of true conflicts. The information of each true conflict includes the conflict type, the target robot pair involved, the corresponding intersection area or resource point information, as well as task-related information such as task objectives, priorities, and planned time. It also loads the regional features and resource attributes provided by the spatiotemporal semantic map, the task value, urgency, and rescheduling cost in the task management system, and the speed, direction, and path overlap output by the path planner.

[0096] Based on the semantic differences of genuine conflicts, a differentiated assessment strategy is adopted, and multi-dimensional dynamic indicators are integrated to calculate and output the severity of the conflict.

[0097] For path conflicts, the relative speed, collision angle, and minimum safe distance of the two robots at the intersection time are extracted from the operation information. The regional features, avoidability, and dynamic complexity of the intersection area in the spatiotemporal semantic map are analyzed to construct a path conflict severity scoring model. In the scoring model, a weighted scoring method is used to calculate the comprehensive score, and the severity of the conflict is determined based on the score.

[0098] For resource conflicts, we obtain the resource application ranking, resource usage duration, and resource demand urgency related to resource competition, and construct a resource conflict severity scoring model. In the scoring model, the longer the resource usage duration and the longer the resource is locked, the greater the impact. We calculate a comprehensive score and determine the severity of the conflict based on the score.

[0099] For priority conflicts, task-related parameters such as task value density, task urgency, and rescheduling cost are obtained to construct a priority conflict severity scoring model. In the scoring model, the greater the difference in value density, the more difficult the conflict is to coordinate. The value function is mapped after difference normalization. In the urgency function, the higher the urgency, the greater the delay cost. In the rescheduling function, the higher the cost, the more rescheduling should be avoided. In the priority function, the greater the priority difference, the more the higher priority should be guaranteed. The comprehensive score is calculated, and the severity of the conflict is determined based on the score value.

[0100] In this embodiment, corresponding evaluation indicators are used to conduct differentiated evaluations for path conflicts, resource conflicts, and priority conflicts, thereby improving the accuracy and reliability of the conflict severity assessment.

[0101] In some embodiments, the decision scheduling layer includes a computing module, a decision module, and a scheduling module connected in sequence: The calculation module is used to calculate the suitability of the target robot to perform the task based on the spatiotemporal semantic map, according to the heterogeneous capability vector of the target robot and the task requirement vector of the task to be performed. Based on the conflict type and severity in the conflict information, the urgency and importance of the task to be executed, and the suitability, the task execution priority of each target robot is calculated through the value function. The decision module is used to dynamically adjust the priority of the tasks to be executed corresponding to the target robot according to the task execution priority of each target robot, and generate an adjusted task priority list. The scheduling module is used to rearrange the tasks to be executed corresponding to the target robot according to the adjusted task priority list, and generate the task sequence containing the task order and execution time. The heterogeneous capability vector is determined based on the differences in the target robot's execution capabilities across different task types. The task requirement vector is used to characterize the capability requirements of the task to be executed. The fit is used to characterize the degree of matching between the target robot and the task to be executed.

[0102] It is understandable that heterogeneous capability vectors quantify different types of robot capabilities in the form of vectors, making it easier for computers to perform matching calculations. Heterogeneous capabilities refer to the differences between different robots in terms of motion capabilities, operational precision, perception range, and endurance.

[0103] The task requirement vector quantifies the ability requirements of the task for the executor in the form of a vector.

[0104] Fit is a matching score obtained by comparing the capability vector and the demand vector.

[0105] The task priority list is a list of tasks sorted according to their execution priority, used to list the new priority order of the tasks to be executed for each target robot.

[0106] In the calculation module, the heterogeneous capability vector of the target robot i is obtained, the task requirement vector of the task to be executed j is obtained, the specific requirements of the task on the executor in these capability dimensions are clarified, and the fit score is calculated for the heterogeneous capability vector of each target robot i and the task requirement vector of the task to be executed j, combined with the task urgency and task importance information from the perception and detection layer. Factors such as suitability, task urgency, and task importance are input into the value function. The value function then performs a weighted sum of these factors according to its internally preset weight allocation rules to obtain the task execution priority.

[0107] In the decision-making module, a dynamic priority adjustment mechanism is activated. Taking into account the task completion status of other robots and the real-time availability of resources, the module analyzes the tasks to be executed for each target robot to determine whether there is a possibility to further optimize the task execution order. For example, if a low-priority task becomes executable earlier due to resource release or path unblocking without having a substantial impact on high-priority tasks, the decision-making module will adjust the priority of the task to execute it earlier and generate a task priority list.

[0108] Based on the task priority list, combined with the spatial information and temporal dynamics provided by the spatiotemporal semantic map, the scheduling module considers the task order and execution time, avoids path conflicts and resource conflicts, plans the optimal task execution path and timetable, rearranges the tasks to be executed corresponding to the target robot, and generates a task sequence containing detailed task order and execution time.

[0109] In this embodiment, by calculating the fit between the heterogeneous capability vector and the task requirement vector, the most suitable robot can be selected to execute the task, giving full play to the strengths of different robots and improving the success rate and efficiency of task execution. Based on the real-time conflict type and severity, combined with the task and robot status, the priority is dynamically calculated and adjusted, realizing dynamic and flexible priority adjustment.

[0110] In some embodiments, the execution control layer includes an execution module and a scheduling module that are interconnected. The execution module is used to determine the motion trajectory and action parameters required for each target robot to perform the corresponding task based on the task sequence and the operation information of the target robot. The scheduling module is used to generate the dynamic scheduling instruction based on the motion trajectory and action parameters, as well as the spatiotemporal semantic map. The dynamic scheduling instruction includes the target robot's task to be executed, the motion trajectory, the action parameters, and the execution time.

[0111] It is understandable that motion trajectories and motion parameters are the specific motion commands that the execution control layer uses to transform abstract task sequences into concrete motion commands that the underlying robot controller can understand.

[0112] After receiving the task sequence, the execution module obtains the specific requirements of each task, such as task type, target location, and expected completion time. At the same time, it acquires the real-time operating information of the target robot, including its current position, speed, battery level, and possible fault status. It then calculates the motion trajectory required for each target robot to perform the task, as well as the corresponding motion parameters on the trajectory. The motion trajectory takes into account spatial accessibility and temporal optimization to avoid conflicts with other robots or environmental obstacles. The motion parameters include, but are not limited to, the robot's moving speed, acceleration, turning angle, and the force and precision required to perform specific tasks (such as grasping and carrying).

[0113] After receiving the motion trajectory and action parameters provided by the execution module, the scheduling module calls upon the static environmental information of the spatiotemporal semantic map, such as obstacle distribution and road network, as well as the dynamic information reflecting the real-time position, motion status, and resource occupancy of the robot and other entities, to generate detailed dynamic scheduling instructions for each target robot.

[0114] The dynamic scheduling instruction contains key information such as the target robot's unique identifier, the specific task to be performed, the predetermined motion trajectory, motion parameters, and execution time.

[0115] In this embodiment, through close collaboration between the execution module and the scheduling module, high-level task planning is precisely translated into low-level instructions that the robot can execute, ensuring the efficiency and safety of task execution. The execution module comprehensively analyzes task requirements and the robot's real-time status, considering spatial accessibility and temporal optimization, effectively avoiding path conflicts and resource contention between robots. Precise setting of motion parameters improves the accuracy and efficiency of robot task execution. The scheduling module utilizes the rich information from the spatiotemporal semantic map to tailor dynamic scheduling instructions for each target robot, ensuring the timeliness and accuracy of the instructions.

[0116] In some embodiments, the execution control layer further includes a control module connected to the scheduling module; The control module is used to parse the feedback information received when the target robot responds to the dynamic scheduling instruction and executes the task to be executed, so as to obtain the actual motion trajectory and action execution status of the target robot. Based on the actual motion trajectory and action execution, an update instruction is generated, which is used to trigger the update of the spatiotemporal semantic map.

[0117] Understandably, the update command is used to trigger the spatiotemporal semantic map update mechanism to ensure that the environmental information, robot status, and resource usage data presented on the map are highly consistent with the actual scene.

[0118] In actual execution, when the target robot responds to the dynamic scheduling command and executes the task to be executed, it sends the actual motion trajectory and action execution status to the control module in the form of feedback information. The control module parses the feedback information to accurately obtain the actual motion trajectory and action execution status of the target robot, such as the actual path the robot has walked, the speed and acceleration at each position, and specific data such as the force and accuracy when performing the action.

[0119] The control module analyzes the differences between these data and the expected motion trajectory and action parameters. If the actual execution deviates from the expectation, it generates corresponding update instructions based on the degree and nature of the deviation to trigger an update of the spatiotemporal semantic map. The updated spatiotemporal semantic map can more accurately reflect the current environmental state, including the robot's actual position, motion state, and resource usage.

[0120] When the target robot responds to the dynamic scheduling command and performs the task to be performed, and then provides feedback, the execution control layer will trigger the updating of the spatiotemporal semantic map to ensure that the entire multi-robot task conflict resolution and dynamic scheduling system can operate efficiently based on the latest and most accurate information.

[0121] The feedback mechanism that triggers the update is as follows: in, For the progress of the mission, For location update, For energy consumption, This refers to unexpected situations, such as malfunctions or the appearance of obstacles.

[0122] Spatiotemporal semantic map at time t Update: in, This is the updated spatiotemporal semantic map at time t+1; It is an update function that can update the map's topology using graph optimization algorithms; It is a spatiotemporal semantic map The increment of the conflict set.

[0123] The updates include, but are not limited to, robot trajectory and status, task completion progress, newly detected obstacles or conflict areas, and dynamic changes in the environment, such as the appearance of moving obstacles.

[0124] In this embodiment, by parsing the feedback information and generating update instructions accordingly, the spatiotemporal semantic map is updated, ensuring that the entire system makes decisions based on the latest state of the real world.

[0125] In some embodiments, the control module is further configured to: Upon receiving the feedback information, the feedback information is segmented in the time dimension through an upper sliding window to obtain multiple segmented information. By using a lower-level sliding window, each segment of information is clustered in the spatial dimension to obtain multiple spatiotemporal clustering results; Based on the spatiotemporal clustering results, the trajectory deviation parameters and motion deviation parameters of the target robot are determined; Based on the trajectory deviation parameters and the motion deviation parameters, the fluctuation characteristics of the target robot's motion trajectory and motion execution are analyzed; The window size of the upper sliding window and the sliding step size of the lower sliding window are dynamically adjusted according to the fluctuation characteristics. The trajectory deviation parameter and the motion deviation parameter are recalculated. When the trajectory deviation parameter and the motion deviation parameter meet the preset threshold conditions, the update instruction is generated.

[0126] Fluctuation characteristics are the patterns exhibited by robot motion deviations, such as periodic vibrations, random jitters, and continuous offsets, which can be used to diagnose problems with the robot's own state or the environment.

[0127] In actual implementation, the two-layer sliding window mechanism, consisting of an upper sliding window and a lower sliding window, can perform refined spatiotemporal processing of feedback information. The upper sliding window divides the continuous feedback data into multiple equal-length time periods along the time axis, with each time period containing the complete motion trajectory and action sequence of the target robot within that time period. The lower sliding window then performs cluster analysis on the data within each time period in the spatial dimension based on the robot's position, motion direction, and action type, forming clusters with spatiotemporal consistency.

[0128] By calculating the Euclidean distance between the actual trajectory and the expected trajectory within each cluster, the trajectory deviation parameter is obtained. The actual value of the motion parameter is compared with the preset value to calculate the motion deviation parameter. Based on the trajectory deviation parameter and the motion deviation parameter, a fluctuation characteristic model is constructed to dynamically identify the periodic deviation and random disturbance of the robot's motion.

[0129] When the root mean square error of the trajectory deviation parameter is less than 5% and the coefficient of variation of the motion deviation parameter is less than 3%, the system determines that the current motion state matches the expected degree. At this time, the generated update command will trigger a local update of the spatiotemporal semantic map. If the deviation parameter continues to exceed the standard, the system will use an adaptive adjustment mechanism that reduces the duration of the upper window and increases the step size of the lower window to achieve rapid convergence of the deviation parameter, ensuring the accuracy and real-time performance of the map update decision.

[0130] In this embodiment, a dual-window design efficiently extracts valuable fluctuation characteristics from feedback data with time and space labels. The window parameters are dynamically adjusted based on the analysis results, enabling the analysis model to self-optimize according to data characteristics. This allows for more accurate capture of trajectory and motion deviations, improving the accuracy of state assessment. It also effectively decouples the spatiotemporal dimensions, avoids frequent map updates due to normal minor vibrations of the robot or sensor noise, reduces system fluctuations, and updates semantic information only when necessary, ensuring system stability.

[0131] This application also provides a method for resolving and dynamically scheduling multi-robot task conflicts, which can be applied to a multi-robot task conflict resolution and dynamic scheduling system as described in any of the above embodiments.

[0132] like Figure 2 As shown, the multi-robot task conflict resolution and dynamic scheduling method includes: Step 210: Obtain the operation information of multiple robots and the spatiotemporal semantic map of the robots performing the tasks to be performed through the perception and detection layer; generate conflict information when at least two target robots are detected to be in conflict. Step 220: Through the decision scheduling layer, based on the conflict information and the priority factor and value function of the task to be executed corresponding to each target robot, the task execution priority of the task to be executed is determined, so as to rearrange the task to be executed corresponding to the target robot and generate a task sequence. The priority factor includes the urgency of the task, the importance of the task and the execution capability of the target robot. The value function is used to quantify the influence weight of each priority factor on the task execution priority. Step 230: Through the execution control layer, based on the task sequence, the spatiotemporal semantic map, and the operation information of the target robot, generate dynamic scheduling instructions and send them to the corresponding target robot.

[0133] The multi-robot task conflict resolution and dynamic scheduling method provided in this application combines operational information and spatiotemporal semantic maps through a perception and detection layer to proactively predict conflicts in real time. Priority factors and value functions are introduced at the decision-making and scheduling layer, enabling quantitative evaluation and task rearrangement based on multi-dimensional information such as task urgency, importance, and robot capabilities. This achieves intelligent decision-making, ensuring the priority execution of critical tasks and improving the intelligence and rationality of decision-making. Dynamic scheduling instructions are issued by the execution control layer, improving the operational efficiency, automation level, and overall efficiency of the multi-robot system. By resolving conflicts in real time during task execution, robot stagnation or task failure is avoided, enhancing the stability and execution efficiency of the multi-robot collaborative system. This method can flexibly and in real time respond to dynamic scheduling scenarios involving multiple robots.

[0134] The embodiments of this application have been described above with reference to the accompanying drawings. However, this application is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make many other forms under the guidance of this application without departing from the spirit and scope of the claims, and all of these forms are within the protection scope of this application.

[0135] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "illustrative embodiment," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of this application. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0136] Although embodiments of this application have been shown and described, those skilled in the art will understand that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of this application, the scope of which is defined by the claims and their equivalents.

Claims

1. A multi-robot task conflict resolution and dynamic scheduling system, characterized in that, It includes a perception and detection layer, a decision-making and scheduling layer, and an execution and control layer connected in sequence; The perception and detection layer is used to acquire the operation information of multiple robots and the spatiotemporal semantic map of the robots performing the tasks to be performed. When a conflict is detected between at least two target robots, conflict information is generated. The decision scheduling layer is used to determine the task execution priority of the task to be executed based on the conflict information and the priority factor and value function of the task to be executed corresponding to each target robot, so as to rearrange the task to be executed corresponding to the target robot and generate a task sequence. The priority factor includes the urgency of the task, the importance of the task and the execution capability of the target robot. The value function is used to quantify the influence weight of each priority factor on the task execution priority. The execution control layer is used to generate dynamic scheduling instructions and send them to the corresponding target robot based on the task sequence, the spatiotemporal semantic map, and the operation information of the target robot.

2. The multi-robot task conflict resolution and dynamic scheduling system according to claim 1, characterized in that, The perception and detection layer includes interconnected detection modules and judgment modules; The detection module is used to model the future trajectory of each robot based on the operation information and the spatiotemporal semantic map to obtain a spatiotemporal state corridor. Based on the spatiotemporal state corridor and the spatiotemporal semantic map, a conflict prediction list between relevant robots is generated; The judgment module is used to judge each predicted conflict in the conflict prediction list and generate the conflict information corresponding to the target robot that will be determined as a real conflict.

3. The multi-robot task conflict resolution and dynamic scheduling system according to claim 2, characterized in that, The detection module is further used for: The spatiotemporal state corridor is detected, and if it is determined that there is a volume intersection between the spatiotemporal state corridors, the intersection region and at least two related robots corresponding to the intersection region are determined. Based on the spatiotemporal semantic map, determine the regional attributes of the intersection region and the spatiotemporal semantic corridor labels of the relevant robots in the intersection region; If a conflict is determined between at least two related robots based on the region attributes and the spatiotemporal semantic corridor labels, the conflict prediction list is generated.

4. The multi-robot task conflict resolution and dynamic scheduling system according to claim 2, characterized in that, The conflict information includes the conflict type and the conflict severity. The judgment module includes a filtering submodule, an identification submodule, and a grading submodule connected in sequence. The filtering submodule is used to identify and remove predicted conflicts that are false positives from the conflict prediction list based on traffic rule semantics, behavioral intention semantics and higher-order task constraints in the spatiotemporal semantic map, and to filter out at least one true conflict. The higher-order task constraints include dependency restrictions and collaboration requirement restrictions between tasks to be executed. The identification submodule is used to identify the conflict type of the true conflict based on the resource distribution and path planning information in the spatiotemporal semantic map and the task priority of the task to be executed. The hierarchical submodule is used to determine the severity of the true conflict based on the conflict type.

5. The multi-robot task conflict resolution and dynamic scheduling system according to claim 4, characterized in that, The conflict types include path conflicts, resource conflicts, and priority conflicts. The hierarchical submodule is further used for: If the conflict type of the true conflict is determined to be a path conflict, the severity of the conflict is determined based on the relative speed of the target robot, the collision angle, and the regional characteristics of the intersection area corresponding to the true conflict. If the conflict type of the true conflict is determined to be a resource conflict, the severity of the conflict is determined based on the target robot's resource request order, resource usage duration, and the urgency of the resource demand corresponding to the true conflict. If the conflict type of the true conflict is determined to be a priority conflict, the severity of the conflict is determined based on the target robot's task value density, task urgency, rescheduling cost, and the difference in task priority corresponding to the true conflict. The task value density is used to characterize the ratio of the benefit to the cost of the target robot performing the task, the task urgency is used to characterize the time constraint of the task to be performed, and the rescheduling cost is used to characterize the additional cost caused by task rescheduling.

6. The multi-robot task conflict resolution and dynamic scheduling system according to claim 1, characterized in that, The decision-making and scheduling layer includes a computing module, a decision-making module, and a scheduling module connected in sequence: The calculation module is used to calculate the suitability of the target robot to perform the task based on the spatiotemporal semantic map, according to the heterogeneous capability vector of the target robot and the task requirement vector of the task to be performed. Based on the conflict type and severity in the conflict information, the urgency and importance of the task to be executed, and the suitability, the task execution priority of each target robot is calculated through the value function. The decision module is used to dynamically adjust the priority of the tasks to be executed corresponding to the target robot according to the task execution priority of each target robot, and generate an adjusted task priority list. The scheduling module is used to rearrange the tasks to be executed corresponding to the target robot according to the adjusted task priority list, and generate the task sequence containing the task order and execution time. The heterogeneous capability vector is determined based on the differences in the target robot's execution capabilities across different task types. The task requirement vector is used to characterize the capability requirements of the task to be executed. The fit is used to characterize the degree of matching between the target robot and the task to be executed.

7. The multi-robot task conflict resolution and dynamic scheduling system according to claim 1, characterized in that, The execution control layer includes interconnected execution modules and scheduling modules; The execution module is used to determine the motion trajectory and action parameters required for each target robot to perform the corresponding task based on the task sequence and the operation information of the target robot. The scheduling module is used to generate the dynamic scheduling instruction based on the motion trajectory and action parameters, as well as the spatiotemporal semantic map. The dynamic scheduling instruction includes the target robot's task to be executed, the motion trajectory, the action parameters, and the execution time.

8. The multi-robot task conflict resolution and dynamic scheduling system according to claim 7, characterized in that, The execution control layer further includes a control module connected to the scheduling module; The control module is used to parse the feedback information received when the target robot responds to the dynamic scheduling instruction and executes the task to be executed, so as to obtain the actual motion trajectory and action execution status of the target robot. Based on the actual motion trajectory and action execution, an update instruction is generated, which is used to trigger the update of the spatiotemporal semantic map.

9. The multi-robot task conflict resolution and dynamic scheduling system according to claim 8, characterized in that, The control module is further used for: Upon receiving the feedback information, the feedback information is segmented in the time dimension through an upper sliding window to obtain multiple segmented information. By using a lower-level sliding window, each segment of information is clustered in the spatial dimension to obtain multiple spatiotemporal clustering results; Based on the spatiotemporal clustering results, the trajectory deviation parameters and motion deviation parameters of the target robot are determined; Based on the trajectory deviation parameters and the motion deviation parameters, the fluctuation characteristics of the target robot's motion trajectory and motion execution are analyzed; The window size of the upper sliding window and the sliding step size of the lower sliding window are dynamically adjusted according to the fluctuation characteristics. The trajectory deviation parameter and the motion deviation parameter are recalculated. When the trajectory deviation parameter and the motion deviation parameter meet the preset threshold conditions, the update instruction is generated.

10. A method for resolving conflicts and dynamically scheduling multi-robot tasks, characterized in that, The method, applied to the multi-robot task conflict resolution and dynamic scheduling system as described in any one of claims 1-9, comprises: The perception and detection layer acquires the operation information of multiple robots and the spatiotemporal semantic map of the robots performing the tasks to be performed. When a conflict is detected between at least two target robots, conflict information is generated. Through the decision scheduling layer, based on the conflict information and the priority factor and value function of the task to be executed corresponding to each target robot, the task execution priority of the task to be executed is determined, so as to rearrange the task to be executed corresponding to the target robot and generate a task sequence. The priority factor includes the urgency of the task, the importance of the task, and the execution capability of the target robot. The value function is used to quantify the influence weight of each priority factor on the task execution priority. By executing the control layer, dynamic scheduling instructions are generated and sent to the corresponding target robot based on the task sequence, the spatiotemporal semantic map, and the target robot's operation information.

Images