Robot scheduling method and device based on action semantics, equipment and storage medium

Abstract

The application provides a robot scheduling method and device based on action semantics, equipment and storage medium, comprising: constructing a global space-time map containing a static environment layer and an action semantics layer; obtaining an action to be executed by a mechanical arm and constructing a corresponding action semantics ontology; generating dynamic space-time constraints based on the action semantics ontology and writing them into the action semantics layer; taking the global space-time map as input, performing multi-agent path planning, and solving an initial space-time path set of the autonomous mobile robot that meets the dynamic space-time constraints; based on a preset target function, taking the action timing of the mechanical arm and the initial space-time path set of the autonomous mobile robot as optimization variables to perform bidirectional collaborative optimization, obtaining a target action timing and a target space-time path set, and controlling the composite robot system to perform scheduling work based on the target action timing and the target space-time path set. The application can improve the space resource utilization rate and system scheduling efficiency in a multi-device concurrent scene.

Description

Technical Field

[0001] This invention relates to the field of robot intelligent control and scheduling technology, and in particular to a robot scheduling method, apparatus, device and storage medium based on action semantics. Background Technology

[0002] Composite robots typically consist of a mobile chassis and a vehicle-mounted robotic arm. When performing actions such as material gripping and assembly, the links and end effectors of the robotic arm will temporarily extend beyond the outline of the mobile chassis, occupying the surrounding physical space and thus creating dynamic obstacles for other mobile entities (such as other autonomous mobile robots) in the same work area.

[0003] To address the spatial conflict issues in multi-device concurrent operation scenarios, existing technologies typically employ methods such as enlarged chassis, fixed motion point methods, or one-way avoidance methods. However, these methods have the following technical drawbacks: First, when using the enlarged chassis method, the system fixes the maximum reachable range of the robotic arm in all postures into a static virtual chassis, resulting in the long-term locking of spatial resources within this range and limiting the utilization rate of spatial resources. Second, when using the fixed motion point method, the geographical coordinates of the robotic arm's deployment are limited, which can easily lead to queuing of devices at specific coordinate points in high-concurrency multi-device scenarios. Third, when using the one-way avoidance method, obstacles are only marked using spatial coordinates and start and end times, lacking the extraction of multi-dimensional motion features; moreover, it requires the autonomous mobile robot to perform one-way avoidance, failing to utilize the adjustability of the robotic arm's motion on the time axis for bidirectional collaborative optimization, making it difficult to further improve the overall operating efficiency and throughput of the system. Summary of the Invention

[0004] This invention provides a robot scheduling method, apparatus, device, and storage medium based on action semantics to improve the utilization of space resources and system scheduling efficiency in multi-concurrency scenarios.

[0005] In a first aspect, the present invention provides a robot scheduling method based on action semantics, applied to a composite robot system composed of an autonomous mobile robot and a robotic arm. The method is executed by a computing device and includes: constructing a global spatiotemporal map comprising a static environment layer and an action semantic layer, the global spatiotemporal map including spatial and temporal dimensions; obtaining the actions to be performed by the robotic arm and constructing a corresponding action semantic ontology, wherein the action semantic ontology includes at least action type, spatiotemporal occupancy envelope, starting time window, and action elasticity attribute; generating dynamic spatiotemporal constraints based on the action semantic ontology and writing the dynamic spatiotemporal constraints into the action semantic layer; using the global spatiotemporal map as input, performing multi-agent path planning to solve for an initial spatiotemporal path set of the autonomous mobile robot that satisfies the dynamic spatiotemporal constraints; and, based on a preset objective function, performing bidirectional collaborative optimization by using the action sequence of the robotic arm and the initial spatiotemporal path set of the autonomous mobile robot as optimization variables to obtain a target action sequence and a target spatiotemporal path set, and controlling the composite robot system to perform scheduling operations based on the target action sequence and the target spatiotemporal path set.

[0006] Preferably, in constructing the corresponding action semantic ontology, the method for obtaining the spatiotemporal occupancy envelope includes: obtaining the kinematic model of the robotic arm; sampling and simulating the action to be executed at a preset time interval based on the kinematic model; and generating the spatiotemporal occupancy envelope of the action to be executed in the time and space dimensions by combining a preset safety margin.

[0007] Preferably, the action elasticity attribute includes an upper and lower bound of the action duration compression ratio, and an action interruption attribute; generating dynamic spatiotemporal constraints based on the action semantic ontology and writing the dynamic spatiotemporal constraints into the action semantic layer includes: obtaining the candidate start time and duration compression ratio of the action to be executed; determining the spatiotemporal range corresponding to the action to be executed by combining the spatiotemporal occupancy envelope in the action semantic ontology; and writing the spatiotemporal range as the dynamic spatiotemporal constraint that can be adjusted according to the candidate start time and the duration compression ratio into the action semantic layer.

[0008] Preferably, performing multi-agent path planning to solve for the initial spatiotemporal path set of the autonomous mobile robot that satisfies the dynamic spatiotemporal constraints includes: using a constraint tree search method to record conflict history through a preset constraint tree, wherein the nodes of the constraint tree correspond to accumulated agent constraints or action constraints; solving for the independent shortest spatiotemporal path for each autonomous mobile robot; expanding the nodes of the constraint tree according to a preset conflict priority order until the initial spatiotemporal path set of the autonomous mobile robot that does not overlap between the autonomous mobile robots and does not overlap with the dynamic spatiotemporal constraints is obtained.

[0009] Preferably, based on a preset objective function, the timing of the robotic arm's movements and the initial spatiotemporal path set of the autonomous mobile robot are used together as optimization variables to perform bidirectional collaborative optimization, including: determining a system-level objective function, the metric parameters of which include the waiting time and travel distance of the autonomous mobile robot, the waiting time of the robotic arm, and the task delivery deadline satisfaction status; fixing the timing of the robotic arm's movements to update the spatiotemporal path set of the autonomous mobile robot, as a first optimization sub-problem; fixing the spatiotemporal path set of the autonomous mobile robot to adjust the timing of the robotic arm's movements, as a second optimization sub-problem; and alternately executing the first optimization sub-problem and the second optimization sub-problem until the calculation result of the system-level objective function satisfies a preset convergence condition or reaches a preset iteration number threshold.

[0010] Preferably, the action semantic ontology further includes action priority, fixing the spatiotemporal path set of the autonomous mobile robot to adjust the action timing of the robotic arm, including: when the action priority is higher than a first preset threshold, locking the start time and duration of the action to be executed; when the action priority is lower than the first preset threshold and the width of the start time window is greater than a second preset threshold, shifting the start time within the range of the start time window; when the action elasticity attribute indicates that the action to be executed can be interrupted, suspending the action to be executed when the target autonomous mobile robot passes through the critical section and resuming execution after the target autonomous mobile robot passes through; when the action elasticity attribute indicates that the range of the action duration compression ratio is greater than a preset value, adjusting the action duration to reduce the spatiotemporal occupancy window.

[0011] Preferably, after controlling the composite robot system to perform scheduling operations based on the target action timing and target spatiotemporal path set, the method further includes: monitoring the execution status of the composite robot system; when a deviation in the timing of the robotic arm's actions is detected or a new task request is received, determining the affected local spatiotemporal region; maintaining the autonomous mobile robot path and robotic arm's action timing unchanged in the unaffected region, and re-executing the step of generating dynamic spatiotemporal constraints based on the action semantic ontology to obtain the target action timing and target spatiotemporal path set for entities within the local spatiotemporal region.

[0012] Secondly, the present invention provides a robot scheduling device based on action semantics, comprising: a map construction module for constructing a global spatiotemporal map including a static environment layer and an action semantic layer, wherein the global spatiotemporal map includes spatial and temporal dimensions; an ontology construction module for acquiring the actions to be executed by the robotic arm and constructing corresponding action semantic ontology, wherein the action semantic ontology includes action type, spatiotemporal occupancy envelope, starting time window, and action elasticity attribute; a constraint generation module for generating dynamic spatiotemporal constraints based on the action semantic ontology and writing the dynamic spatiotemporal constraints into the action semantic layer; a path planning module for performing multi-agent path planning with the global spatiotemporal map as input to solve for the initial spatiotemporal path set of the autonomous mobile robot that satisfies the dynamic spatiotemporal constraints; and a bidirectional collaborative optimization module for performing bidirectional collaborative optimization based on a preset objective function, using the action sequence of the robotic arm and the initial spatiotemporal path set of the autonomous mobile robot as optimization variables to obtain a target action sequence and a target spatiotemporal path set, and controlling the composite robot system to perform scheduling operations based on the target action sequence and the target spatiotemporal path set.

[0013] Thirdly, the present invention provides an apparatus comprising: a processor; and a memory for storing processor-executable instructions; wherein the processor is configured to, when executing the executable instructions, implement the method as described in any one of the preceding descriptions.

[0014] Fourthly, the present invention provides a storage medium having computer instructions stored thereon, characterized in that, when the instructions are executed by a processor, they implement the method described in any of the preceding claims.

[0015] Beneficial effects Improved utilization of space resources: Abandoned the static processing method of abstracting the maximum reachable domain as a normal chassis, and by constructing an action semantic ontology containing a spatiotemporal occupancy envelope, the computing device can generate accurate dynamic spatiotemporal constraints in the action semantic layer according to the actual occurrence time of the robotic arm's action, reducing unnecessary spatial locking range and releasing passable raster / voxel resources.

[0016] The overall system operating efficiency is improved: a two-way temporal collaborative optimization mechanism is adopted, using the spatial path of the autonomous mobile robot and the time series of the robotic arm as joint optimization variables. When the obstacle avoidance cost of the autonomous mobile robot is too high, the computing device can adjust the start time or duration of the robotic arm's movement in reverse through the objective function, overcoming the problems of excessively long paths and frequent start-stop of the equipment caused by the one-way obstacle avoidance mechanism.

[0017] The control dimension and flexibility of the scheduling system are enhanced: by constructing a semantic ontology for each action to be executed, including action elasticity attributes such as interruptibility and duration compressibility, the computing device can perform differentiated processing for actions of different priorities; when spatiotemporal conflicts occur, the scheduling algorithm is provided with decision dimensions for compression, translation or suspension on the time axis.

[0018] Improved computational efficiency of anomaly handling: When a job delay or new task insertion is detected, based on the data structure of the global spatiotemporal map, the computing device only needs to trigger incremental replanning for the affected local spatiotemporal region, while maintaining the path state of the remaining entities unchanged, thus reducing the computational resource consumption and response latency caused by replanning operations. Attached Figure Description

[0019] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the following description of the embodiments will be briefly introduced. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0020] Figure 1 The main flowchart of a robot scheduling method based on action semantics provided in the embodiments of this specification; Figure 2 A first sub-flowchart of a robot scheduling method based on action semantics provided in the embodiments of this specification; Figure 3 A second sub-flowchart of a robot scheduling method based on action semantics provided in the embodiments of this specification; Figure 4 A third sub-flowchart of a robot scheduling method based on action semantics provided in the embodiments of this specification; Figure 5 The fourth sub-flowchart of a robot scheduling method based on action semantics provided in the embodiments of this specification; Figure 6 The fifth sub-flowchart of a robot scheduling method based on action semantics provided in the embodiments of this specification; Figure 7 This is a block diagram of a robot scheduling device based on action semantics, provided as an embodiment of this specification. Detailed Implementation

[0021] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions in the embodiments of this invention will be clearly, completely, and objectively described below with reference to the accompanying drawings. It should be noted that, unless otherwise specified, the embodiments and features described in this application can be combined with each other. The described embodiments are merely some embodiments of this invention, and not all embodiments.

[0022] This invention provides a robot scheduling method based on action semantics, applied to a hybrid robot system consisting of autonomous mobile robots and robotic arms. In a specific application scenario, this hybrid robot system can be deployed in a smart warehousing center, for example, in a space with a planar dimension of 60m × 40m, arranging multiple autonomous mobile robots and multiple fixed or vehicle-mounted robotic arm workstations. The method is executed by a computing device.

[0023] Please see Figure 1 , Figure 1 This document provides a main flowchart of a robot scheduling method based on action semantics, as illustrated in an embodiment of the invention. The action semantics-based robot scheduling method specifically includes the following steps: Step S110: Construct a global spatiotemporal map that includes a static environment layer and an action semantic layer, wherein the global spatiotemporal map includes a spatial dimension and a temporal dimension; In this embodiment, to achieve spatiotemporal collision avoidance planning for each physical entity (autonomous mobile robot and robotic arm) in the composite robot system, the computing device first initializes the data structure of the global spatiotemporal map in memory. The global spatiotemporal map is essentially a four-dimensional spatiotemporal grid tensor or a continuous spatiotemporal manifold structure. Specifically, the spatial dimension can be set to two-dimensional (XY plane grid) or three-dimensional (XYZ spatial voxels), and the temporal dimension (T) is discretized into a series of time steps according to the system's control cycle. For example, the site is rasterized with a spatial resolution of 0.2m, and the temporal dimension is sliced ​​with a sampling duration of 0.1s.

[0024] The computing device loads pre-stored geometric spatial information that does not change over time, such as building structure, shelf location, static obstacles, and restricted areas, to generate the static environment layer. This static environment layer remains constant in the time dimension. Simultaneously, the computing device initializes an empty action semantic layer. The data structure of this action semantic layer uses a four-dimensional sparse tensor with action identifier, start time interval, spatiotemporal voxel list, and semantic attribute table as primary keys. This tensor is used to dynamically write the spatiotemporal occupancy and semantic attributes induced by the robotic arm's actions, which change over time, in subsequent steps.

[0025] Step S120: Obtain the action to be executed by the robotic arm and construct the corresponding action semantic ontology, wherein the action semantic ontology includes at least the action type, spatiotemporal occupancy envelope, start window and action elasticity attribute; In this embodiment, the computing device no longer simply treats the robotic arm's movements as unadjustable, fixed obstacles, but instead performs deep semantic analysis on them. For each of the actions to be performed... The computing device establishes a structured action semantic ontology in memory. In a specific mathematical expression, the action semantic ontology can be represented as a quintuple: .

[0026] In detail: (1) the type of action, namely , taken from a predefined set of action types, such as grab, place, assemble, rotate, etc. Each action type is associated with a default semantic template in the system database. (2) The spatiotemporal occupancy envelope, i.e. , indicating relative time within the action The space occupancy function for the parameters. (3) The starting time window, i.e. This defines the legal intervals within which the action to be executed is allowed to be triggered on the timeline. Determined by the preceding task nodes (such as the arrival time of materials). Determined by the mandatory constraints of the task delivery period. (4) The aforementioned action elasticity attribute, i.e. This defines the flexible characteristics of the action to be executed during the execution process.

[0027] To further explain how to obtain the key information from the above ontology, please refer to [link / reference]. Figure 2 , Figure 2 A first sub-flowchart of a robot scheduling method based on action semantics provided in the embodiments of this specification includes the following sub-steps: Sub-step S210: Obtain the kinematic model of the robotic arm; The computing device reads the standard Denavit-Hartenberg (DH) parameter table or URDF (Uniform Robot Description Format) file corresponding to the robotic arm from the device management database and establishes a forward kinematics model of the robotic arm. Using this forward kinematics model, the computing device can calculate the precise pose of the links and end effectors in the world coordinate system based on the angle variables of each joint.

[0028] Sub-step S220: Based on the kinematic model, sample and simulate the action to be executed at a preset time interval; After acquiring the task trajectory, the computing device extracts each discrete interpolation point on the trajectory according to the system's control cycle (e.g., every 100ms), and substitutes the corresponding joint angles into the kinematic model. Through simulation calculation, the set of projected coordinates of the bounding box of each link of the robotic arm in three-dimensional space is obtained for each time slice.

[0029] Sub-step S230: Combine the preset safety margin to generate the spatiotemporal occupancy envelope of the action to be executed in the time and space dimensions.

[0030] To prevent collisions caused by servo control errors or communication delays during actual execution, the computing device needs to perform an expansion operation on the simulated spatial coordinate set. During this process, the computing device can expand outwards by a preset safety margin (e.g., 0.5m) based on Minkowski algorithms. Furthermore, this safety margin can be configured to adapt to the instantaneous speed: expanding by 0.6m during high-speed execution and by 0.4m during low-speed fine-tuning. The expanded spatial geometry is then sequentially arranged in the corresponding time series to generate the accurate spatiotemporal occupancy envelope. .

[0031] Please see Figure 3 , Figure 3 The second sub-flowchart of the robot scheduling method based on action semantics provided in the embodiments of this specification further refines the constraint generation process: Sub-step S310: The action elasticity attribute includes the upper and lower bounds of the action duration compressibility ratio, and the action interruption attribute; In this step, the computing device determines the specific data structure of the motion elasticity attribute, namely... Among them, the upper and lower bounds of the action duration compressibility ratio ( This determines the permissible adjustment range of the execution speed of the action to be performed. If... This indicates that the duration of the action is not compressible (e.g., the high-precision grasping phase); action interruption attribute ( A Boolean value indicates whether the action can be safely suspended midway through execution (e.g., long-distance rotation actions can be interrupted, while laser welding processes cannot be interrupted).

[0032] Step S130: Generate dynamic spatiotemporal constraints based on the action semantic ontology, and write the dynamic spatiotemporal constraints into the action semantic layer; To transform the aforementioned continuous, semantically rich ontology into boundary conditions that can be used by the path planning algorithm, the computing device performs the following sub-steps: Sub-step S320: Obtain the candidate start time and duration compression ratio of the action to be executed; During the initialization or iterative optimization phase, the computing device allocates an initial candidate start time for the action to be executed. (Typically, the midpoint value of the starting time window is taken) and an initial duration compression ratio. (Default value is 1.0).

[0033] Sub-step S330: Combine the spatiotemporal occupancy envelope in the action semantic ontology to determine the spatiotemporal range corresponding to the action to be executed; The computing device is based on the selected candidate start time. Compression ratio The original spatiotemporal occupancy envelope is translated and scaled on the time axis to calculate the set of spatiotemporal ranges actually occupied by the action in the global coordinate system.

[0034] Sub-step S340: Write the spatiotemporal range as a dynamic spatiotemporal constraint that can be adjusted according to the candidate start time and the duration compression ratio into the action semantic layer.

[0035] The computing device generates the dynamic spatiotemporal constraint equations in mathematical form: ,in This is the nominal duration. The computing device serializes the constraint equation and injects it into the action semantic layer, marking it as a state to be scheduled. This means that as long as the subsequent algorithm is modified... or The constraint will then undergo time axis drift or time axis stretching / compression in the global spatiotemporal map, achieving true dynamic adjustability.

[0036] Step S140: Using the global spatiotemporal map as input, perform multi-agent path planning to solve for the initial spatiotemporal path set of the autonomous mobile robot that satisfies the dynamic spatiotemporal constraints; After completing the map and constraint construction, the system needs to generate a travel path for all autonomous mobile robots within the site that avoids collisions with each other and interference with the robotic arm's operations. Please refer to [link / reference]. Figure 4 The computing device performs the solution through the following sub-processes: Sub-step S410: Using a constraint tree-based search method, the conflict history is recorded through a preset constraint tree, wherein the nodes of the constraint tree correspond to the accumulated agent constraints or action constraints. The computing device builds a multi-branch state tree (i.e., a constraint tree) in memory. The root node of this constraint tree is an empty constraint set. Whenever a path conflict is detected, the computing device splits the node, generating child nodes. Each child node inherits the constraints of its parent node and adds a new constraint (e.g., robot A in time...). to Access to the spatial coordinate set is prohibited during this period. These constraints include collision avoidance constraints between autonomous mobile robots, as well as motion constraints introduced by the motion semantic layer.

[0037] Sub-step S420: Solve for the independent shortest spatiotemporal path for each of the main mobile robots; At each node of the constraint tree (i.e., low-level search), the computing device separates an independent planning subprocess. For each autonomous mobile robot in the system, the computing device searches for the lowest-cost collision-free path in the global spatiotemporal map, which is superimposed with the node's specific constraints, based on the A* (A-star) algorithm or a three-dimensional extended version of Dijkstra's algorithm (two-dimensional space + one-dimensional time), with the robot's current pose as the starting point and the target pose as the ending point.

[0038] Sub-step S430: Expand the nodes of the constraint tree according to the preset conflict priority order until the initial spatiotemporal path set of the autonomous mobile robot that does not overlap between the autonomous mobile robots and does not overlap with the dynamic spatiotemporal constraints is obtained.

[0039] The computing device (i.e., high-level search) collects all single-machine paths returned from lower levels and performs global cross-validation. If two autonomous mobile robots are detected occupying the same spatial voxel at the same time, or if a robot's path intrudes into the above formula... If the defined dynamic spatiotemporal constraint region is found to be in conflict, the computing device generates corresponding conflict stripping rules according to a preset priority order of action constraints first, followed by inter-agent constraints. This expands the constraint tree into new nodes, and the lower-level search is invoked again. This iterative process continues until all paths under a certain node pass the verification. The computing device then outputs the initial spatiotemporal path set of the autonomous mobile robot that satisfies all constraints.

[0040] Step S150: Based on the preset objective function, the action timing of the robotic arm and the initial spatiotemporal path set of the autonomous mobile robot are used as optimization variables to perform bidirectional collaborative optimization, obtain the target action timing and the target spatiotemporal path set, and control the composite robot system to perform scheduling operations based on the target action timing and the target spatiotemporal path set.

[0041] Step S140 above only achieves unidirectional obstacle avoidance by the autonomous mobile robot, and has not yet explored the system's coordination potential in the time dimension. To achieve global optimization, the computing device performs bidirectional cooperative optimization. Please refer to... Figure 5 This process encompasses the following core logic: Sub-step S510: Determine the system-level objective function, the measurement parameters of which include the waiting time and travel distance of the autonomous mobile robot, the waiting time of the robotic arm, and the task delivery period satisfaction status. The computing device constructs a unified and optimized evaluation index, the system-level objective function. Taking into account the overall operating costs of all equipment and the degree of task achievement, the specific mathematical expression can be constructed as follows: ; in, , The first The waiting time and travel distance of the autonomous mobile robot; Measure the delayed start-up time of the robotic arm; Determine the penalties for exceeding the deadline; All of these are normalized weighting coefficients configured by the system based on the importance of each indicator.

[0042] To solve this complex function containing a mixture of discrete (path) and continuous (temporal) variables, the computing device employs an alternating optimization strategy: Sub-step S520: Fix the timing of the robotic arm's movements to update the spatiotemporal path set of the autonomous mobile robot as the first optimization sub-problem; In the first optimization subproblem phase, the computing device freezes all candidate start times. Compression ratio The value of is taken. At this point, the robotic arm's movement is considered a fixed spatiotemporal obstacle. Under this boundary condition, the computing device re-invokes the aforementioned constraint tree search algorithm to find a constraint that allows the current... The combination of autonomous mobile robot paths that minimizes the value is used to refresh the spatiotemporal path set of the autonomous mobile robot.

[0043] Sub-step S530: Fix the spatiotemporal path set of the autonomous mobile robot to adjust the action sequence of the robotic arm as a second optimization sub-problem; In the second optimization sub-problem phase, the computing device freezes the travel routes and travel times of all autonomous mobile robots, and instead analyzes how these paths occupy the robotic arm's motion space. The computing device traverses all actions and performs differentiated processing based on the attributes contained in the action semantic ontology.

[0044] Specifically, as described in sub-step S610: the action semantic ontology also includes action priority, fixing the spatiotemporal path set of the autonomous mobile robot to adjust the action sequence of the robotic arm, including: Sub-step S620: When the action priority is higher than the first preset threshold, lock the start time and duration of the action to be executed; The computing device reads the contents of the main body If the action is at an absolutely critical path node (such as an extremely urgent feeding action, whose priority is set to 0.95, which is higher than the preset first threshold of 0.8), the computing device will prevent the algorithm from modifying its timing variables and force the autonomous mobile robot that conflicts with it to perform physical detour avoidance.

[0045] Sub-step S630: When the action priority is lower than the first preset threshold and the width of the starting time window is greater than the second preset threshold, shift the starting time within the range of the starting time window; When an action (e.g., with a priority set to 0.55) has moderate importance, and its With ample span, computing devices can... By shifting forward or backward on the timeline (e.g., shifting backward by 1.0 second), the system proactively makes way for an autonomous mobile robot that urgently needs to pass through the area, thereby reducing the total waiting time of the entire system.

[0046] Sub-step S640: When the action elasticity attribute indicates that the action to be executed can be interrupted, suspend the action to be executed when the target autonomous mobile robot passes through the critical section and resume execution after the target autonomous mobile robot passes through; By reading the action interruption attribute If the result is interruptible, the computing device sends a command to the underlying servo controller. When the target autonomous mobile robot is about to invade the robotic arm's action area, the robotic arm is kept stationary (suspended) in a safe posture. After it leaves the boundary, it continues to complete the remaining trajectory, thereby resolving the conflict without increasing the space avoidance distance.

[0047] Sub-step S650: When the range of the compressibility ratio of the action duration indicated by the action elasticity attribute is greater than a preset value, adjust the duration of the action to reduce the time and space occupancy window.

[0048] like This indicates that the action allows for accelerated execution. The computing device reduces the compression ratio over time. The value of the action is used to shorten the duration of the space resources occupied by the action, allowing subsequent conflicting vehicles to pass earlier.

[0049] Sub-step S540: Alternately execute the first optimization sub-problem and the second optimization sub-problem until the calculation result of the system-level objective function satisfies the preset convergence condition or reaches the preset iteration number threshold.

[0050] The computing device inputs the optimized new timing sequence back into the first optimization subproblem, and repeats this process. After each iteration, the computing device recalculates the system-level objective function. When the results of two consecutive calculations When the relative rate of change of the value is less than a preset convergence threshold (e.g., 5%), or when the execution loop reaches a preset iteration count threshold specified by the system's computing power, the computing device terminates the loop. The output data at this point is the target action timing sequence and target spatiotemporal path set after deep coupling optimization. The computing device, through a communication network (e.g., 5G / Wi-Fi), parses the above instructions into low-level navigation messages and servo drive signals, and sends them to each autonomous mobile robot and robotic arm for execution.

[0051] To ensure high availability and robustness of on-site scheduling, the method also incorporates an incremental response mechanism during actual implementation. Please refer to [link / reference]. Figure 6 It also includes the following execution monitoring phase process: Sub-step S710: Monitor the execution status of the composite robot system; The computing device synchronizes the real geographical location and percentage of action completion of all devices at fixed intervals by transmitting data back via heartbeat packets or telemetry.

[0052] Sub-step S720: When a timing deviation of the robotic arm's movements is detected or a new task request is received, determine the affected local spatiotemporal region; Due to uncertainties on-site (such as capture delays or the insertion of high-priority urgent orders by the host computer), the original plan may fail. Based on the source coordinates and time of occurrence of the deviation, the computing device delineates a local spatial boundary and a limited time look-ahead range in the global spatiotemporal map, generating a local spatiotemporal region.

[0053] Sub-step S730: Maintain the autonomous mobile robot path and robotic arm action timing in the unaffected area unchanged, and re-execute the step of generating dynamic spatiotemporal constraints based on action semantic ontology to obtain the target action timing and target spatiotemporal path set for entities in the local spatiotemporal region.

[0054] To avoid the enormous overhead of triggering a full system shutdown and recalculation, the computing device locks the states of other unrelated devices into static, immutable constraints, extracting only a few autonomous mobile robots and the robotic arm nodes that have deviated from their intended purpose within this local spatiotemporal region, treating them as local optimization variables. Subsequently, the computing device reuses the dynamic spatiotemporal constraint generation, local constraint tree search, and local bidirectional collaborative optimization processes described in S130 to S150 only for these limited entities. Incremental computation can compress the computation time from hundreds of milliseconds to tens of milliseconds (e.g., from 280ms for global recalculation to 35ms for incremental computation), ensuring agile system response.

[0055] Based on the same inventive concept, embodiments of this specification also provide a robot scheduling device based on action semantics. Please refer to... Figure 7 The device operates on the aforementioned computing equipment and specifically includes: a map construction module for constructing a global spatiotemporal map containing a static environment layer and an action semantic layer, wherein the global spatiotemporal map includes spatial and temporal dimensions; an ontology construction module for acquiring the actions to be performed by the robotic arm and constructing corresponding action semantic ontology, wherein the action semantic ontology includes action type, spatiotemporal occupancy envelope, starting time window, and action elasticity attribute; a constraint generation module for generating dynamic spatiotemporal constraints based on the action semantic ontology and writing the dynamic spatiotemporal constraints into the action semantic layer; a path planning module for performing multi-agent path planning with the global spatiotemporal map as input to solve for the initial spatiotemporal path set of the autonomous mobile robot that satisfies the dynamic spatiotemporal constraints; and a bidirectional collaborative optimization module for performing bidirectional collaborative optimization based on a preset objective function, using the action sequence of the robotic arm and the initial spatiotemporal path set of the autonomous mobile robot as optimization variables to obtain the target action sequence and the target spatiotemporal path set, and controlling the composite robot system to perform scheduling operations based on the target action sequence and the target spatiotemporal path set. (The internal logic and data flow of each of the above functional modules correspond completely to the detailed derivation in steps S110 to S150 above, and will not be repeated here.) Furthermore, this embodiment also provides a device including a processor and a memory for storing processor-executable instructions; wherein, when the processor is configured to execute the executable instructions, it implements all steps and computational logic of the robot scheduling method based on action semantics described in any of the above embodiments. This hardware device can be a standalone local scheduling server, an edge computing gateway, or a computing cluster node deployed in the cloud.

[0056] Furthermore, this embodiment also provides a storage medium storing computer instructions, which, when executed by a processor, implement all the steps of the robot scheduling method based on action semantics described above. This storage medium includes, but is not limited to, various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0057] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in the present invention, and these modifications or substitutions should all be covered within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A robot scheduling method based on action semantics, applied to a composite robot system consisting of an autonomous mobile robot and a robotic arm, characterized in that, The method, executed by a computing device, includes: constructing a global spatiotemporal map comprising a static environment layer and an action semantic layer, the global spatiotemporal map including spatial and temporal dimensions; acquiring the actions to be performed by the robotic arm and constructing corresponding action semantic ontology, wherein the action semantic ontology includes at least action type, spatiotemporal occupancy envelope, starting time window, and action elasticity attribute; generating dynamic spatiotemporal constraints based on the action semantic ontology and writing the dynamic spatiotemporal constraints into the action semantic layer; using the global spatiotemporal map as input, performing multi-agent path planning to solve for the initial spatiotemporal path set of the autonomous mobile robot that satisfies the dynamic spatiotemporal constraints; and, based on a preset objective function, performing bidirectional collaborative optimization by using the action sequence of the robotic arm and the initial spatiotemporal path set of the autonomous mobile robot as optimization variables to obtain a target action sequence and a target spatiotemporal path set, and controlling the composite robot system to perform scheduling operations based on the target action sequence and the target spatiotemporal path set.

2. The robot scheduling method based on action semantics according to claim 1, characterized in that, In constructing the corresponding action semantic ontology, the method for obtaining the spatiotemporal occupancy envelope includes: obtaining the kinematic model of the robotic arm; sampling and simulating the action to be executed at a preset time interval based on the kinematic model; and generating the spatiotemporal occupancy envelope of the action to be executed in the time and space dimensions by combining a preset safety margin.

3. The robot scheduling method based on action semantics according to claim 1, characterized in that, The motion elasticity attribute includes the upper and lower bounds of the motion duration compressibility ratio, as well as the motion interruption attribute; Based on the action semantic ontology, dynamic spatiotemporal constraints are generated and written into the action semantic layer, including: obtaining the candidate start time and duration compression ratio of the action to be executed; By combining the spatiotemporal occupancy envelope in the action semantic ontology, the spatiotemporal range corresponding to the action to be executed is determined; the spatiotemporal range is written into the action semantic layer as a dynamic spatiotemporal constraint that can be adjusted according to the candidate start time and the duration compression ratio.

4. The robot scheduling method based on action semantics according to claim 1, characterized in that, Performing multi-agent path planning to solve for the initial spatiotemporal path set of autonomous mobile robots that satisfies the dynamic spatiotemporal constraints includes: using a constraint tree search method to record conflict history through a preset constraint tree, wherein the nodes of the constraint tree correspond to accumulated agent constraints or action constraints; solving for independent shortest spatiotemporal paths for each autonomous mobile robot; expanding the nodes of the constraint tree according to a preset conflict priority order until the initial spatiotemporal path set of the autonomous mobile robots that does not overlap between the autonomous mobile robots and does not overlap with the dynamic spatiotemporal constraints is obtained.

5. The robot scheduling method based on action semantics according to claim 1, characterized in that, Based on a preset objective function, the timing of the robotic arm's movements and the initial spatiotemporal path set of the autonomous mobile robot are used together as optimization variables to perform bidirectional collaborative optimization. This includes: determining a system-level objective function, the metric parameters of which include the waiting time and travel distance of the autonomous mobile robot, the waiting time of the robotic arm, and the task delivery deadline satisfaction status; fixing the timing of the robotic arm's movements to update the spatiotemporal path set of the autonomous mobile robot, as a first optimization sub-problem; fixing the spatiotemporal path set of the autonomous mobile robot to adjust the timing of the robotic arm's movements, as a second optimization sub-problem; and alternately executing the first and second optimization sub-problems until the calculation result of the system-level objective function satisfies a preset convergence condition or reaches a preset iteration threshold.

6. The robot scheduling method based on action semantics according to claim 5, characterized in that, The action semantic ontology also includes action priority, fixing the spatiotemporal path set of the autonomous mobile robot to adjust the action timing of the robotic arm, including: locking the start time and duration of the action to be executed when the action priority is higher than a first preset threshold; shifting the start time within the range of the start time window when the action priority is lower than the first preset threshold and the width of the start time window is greater than a second preset threshold; suspending the action to be executed when the target autonomous mobile robot passes through the critical section and resuming execution after the target autonomous mobile robot passes through when the action elasticity attribute indicates that the action to be executed can be interrupted; adjusting the duration of the action to reduce the spatiotemporal occupancy window when the action elasticity attribute indicates that the range of the action duration compression ratio is greater than a preset value.

7. The robot scheduling method based on action semantics according to any one of claims 1 to 6, characterized in that, After controlling the composite robot system to perform scheduling operations based on the target action timing and target spatiotemporal path set, the method further includes: monitoring the execution status of the composite robot system; when a deviation in the timing of the robotic arm's actions is detected or a new task request is received, determining the affected local spatiotemporal region; maintaining the autonomous mobile robot path and robotic arm action timing unchanged in the unaffected region, and re-executing the step of generating dynamic spatiotemporal constraints based on the action semantic ontology to obtain the target action timing and target spatiotemporal path set for entities within the local spatiotemporal region.

8. A robot scheduling device based on action semantics, characterized in that, include: The map building module is used to build a global spatiotemporal map that includes a static environment layer and an action semantic layer. The global spatiotemporal map includes spatial and temporal dimensions. The ontology construction module is used to acquire the actions to be performed by the robotic arm and construct the corresponding action semantic ontology, wherein the action semantic ontology includes action type, spatiotemporal occupancy envelope, starting time window, and action elasticity attribute; the constraint generation module is used to generate dynamic spatiotemporal constraints based on the action semantic ontology and write the dynamic spatiotemporal constraints into the action semantic layer; the path planning module is used to perform multi-agent path planning with the global spatiotemporal map as input to solve for the initial spatiotemporal path set of the autonomous mobile robot that satisfies the dynamic spatiotemporal constraints; the bidirectional collaborative optimization module is used to perform bidirectional collaborative optimization based on a preset objective function, using the action sequence of the robotic arm and the initial spatiotemporal path set of the autonomous mobile robot as optimization variables to obtain the target action sequence and the target spatiotemporal path set, and control the composite robot system to perform scheduling operations based on the target action sequence and the target spatiotemporal path set.

9. An electronic device, characterized in that, include: processor; A memory for storing processor-executable instructions; wherein the processor is configured to implement the method as described in any one of claims 1 to 7 when executing the executable instructions.

10. A storage medium storing computer instructions thereon, characterized in that, When executed by the processor, this instruction implements the method as described in any one of claims 1 to 7.

Images