Robot control method, device, electronic equipment, medium and program product

Abstract

This application discloses a robot control method, device, electronic device, medium, and program product applied to a robot control platform. The robot control platform includes a unified protocol interface. The method includes: in response to receiving a user task objective, obtaining real-time status information of all execution entities connected to the robot control platform through the unified protocol interface; determining real-time multi-dimensional capability information corresponding to each execution entity based on the real-time status information; acquiring current environmental spatial characteristics, and decomposing the user task objective into multiple task nodes and dependencies between task nodes based on the current environmental spatial characteristics; determining the task adaptability of the execution entity to each task node based on the real-time multi-dimensional capability information of the execution entity and the task characteristics of the task nodes; using the matching relationship between task nodes and execution entities as the optimization objective, solving a preset objective function based on the task adaptability of task nodes, the priority of task nodes, and conflict terms between task nodes, and obtaining a global scheduling scheme under the condition of satisfying preset constraints including the dependencies between task nodes; controlling the corresponding execution entity through the unified protocol interface based on the global scheduling scheme, so that the execution entity executes the corresponding task node; adjusting the global scheduling scheme in response to satisfying preset risk conditions during the execution of the task node by the execution entity. This achieves efficient, stable, and real-time collaborative control for multiple heterogeneous execution entities.

Description

Technical Field

[0001] This application relates to the field of robot control technology, and in particular to a robot control method, device, electronic device, medium, and program product. Background Technology

[0002] With the rapid development of robotics technology, the collaborative work of various types of heterogeneous robots in the home environment to complete complex tasks has become an important development trend.

[0003] However, collaborative control among heterogeneous executors faces numerous challenges. Existing technologies typically rely on static capability models and preset scheduling strategies, making it difficult to adapt to real-time changes in environmental conditions and dynamic fluctuations in task requirements, resulting in insufficient timeliness and rationality of scheduling schemes. Furthermore, due to the different communication protocols used by executors from different vendors, system integration costs are high and interoperability is poor. Moreover, existing fault-tolerance mechanisms often employ global shutdown or single retry strategies; when a local node experiences an anomaly, it often leads to task interruption or service paralysis of the entire collaborative system, lacking service continuity and elastic recovery capabilities while ensuring safety. Therefore, there is an urgent need for a robot control scheme that can adapt to unstructured dynamic environments, is compatible with heterogeneous protocols, and possesses high robustness. Summary of the Invention

[0004] This application provides a robot control method, device, electronic device, medium, and program product, which realizes efficient, stable, and real-time collaborative control for multiple heterogeneous actuators.

[0005] On one hand, this application provides a robot control method applied to a robot control platform. The robot control platform includes a unified protocol interface. The method includes: in response to receiving a user task target, obtaining real-time status information of all execution entities connected to the robot control platform through the unified protocol interface; determining real-time multi-dimensional capability information corresponding to each execution entity based on the real-time status information; obtaining current environmental spatial features, and decomposing the user task target into multiple task nodes and dependencies between task nodes based on the current environmental spatial features; determining the task adaptability of the execution entity to each task node based on the real-time multi-dimensional capability information of the execution entity and the task characteristics of the task nodes; taking the matching relationship between task nodes and execution entities as the optimization objective, solving a preset objective function based on the task adaptability of task nodes, the priority of task nodes, and conflict terms between task nodes, and obtaining a global scheduling scheme under the condition of satisfying preset constraints including the dependencies between task nodes; controlling the corresponding execution entity through the unified protocol interface based on the global scheduling scheme, so that the execution entity executes the corresponding task node; and adjusting the global scheduling scheme in response to satisfying preset risk conditions during the execution of the task node by the execution entity.

[0006] On the other hand, embodiments of this application provide a robot control device, comprising: a first acquisition unit, configured to acquire real-time status information of all execution entities connected to the robot control platform through a unified protocol interface in response to receiving a user task objective; a first processing unit, configured to determine real-time multi-dimensional capability information corresponding to each execution entity based on the real-time status information; a second processing unit, configured to acquire current environmental spatial features and decompose the user task objective into multiple task nodes and dependencies between task nodes based on the current environmental spatial features; a third processing unit, configured to determine the task adaptability of the execution entity to each task node based on the real-time multi-dimensional capability information of the execution entity and the task characteristics of the task nodes; a fourth processing unit, configured to solve a preset objective function based on the task adaptability of the task nodes, the priority of the task nodes, and the conflict terms between the task nodes, with the matching relationship between the task nodes and the execution entity as the optimization objective, and obtain a global scheduling scheme under the condition of satisfying preset constraints including the dependencies between task nodes; a fifth processing unit, configured to control the corresponding execution entity through a unified protocol interface based on the global scheduling scheme, so that the execution entity executes the corresponding task node; and a sixth processing unit, configured to adjust the global scheduling scheme in response to satisfying preset risk conditions during the execution of the task node by the execution entity.

[0007] In another aspect, embodiments of this application provide an electronic device, which includes: a processor and a memory storing computer program instructions; the processor executes the computer program instructions to implement a robot control method.

[0008] In another aspect, embodiments of this application provide a computer storage medium on which computer program instructions are stored, and when the computer program instructions are executed by a processor, a robot control method is implemented.

[0009] In another aspect, embodiments of this application provide a computer program product in which instructions, when executed by the processor of an electronic device, cause the electronic device to perform a robot control method.

[0010] The robot control method, apparatus, electronic device, medium, and program product of this application obtain real-time status information of all executing entities through a unified protocol interface, breaking down information silos between cross-brand devices. Based on the real-time status information, real-time multi-dimensional capability information is determined, enabling dynamic quantitative evaluation of the capabilities of heterogeneous executing entities. Task decomposition based on environmental spatial characteristics ensures the scheduling scheme aligns with the actual operating environment. Through comprehensive optimization of task adaptability, priority, and conflict items, a global scheduling scheme that balances efficiency and safety is generated. During execution, the scheduling scheme is dynamically adjusted in response to risk conditions, ensuring the system's robustness and service continuity in complex dynamic environments. Thus, efficient, stable, and real-time collaborative control for multiple heterogeneous executing entities is achieved. Attached Figure Description

[0011] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings used in the embodiments of this application will be briefly introduced below. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0012] Figure 1 This is a flowchart illustrating a robot control method provided in an embodiment of this application; Figure 2 This is a flowchart of the overall architecture provided in the embodiments of this application; Figure 3 This is a flowchart of the protocol layer architecture provided in the embodiments of this application; Figure 4 This is a flowchart of a multi-level fault response provided in an embodiment of this application; Figure 5 This is a flowchart of the graded circuit breaker state transition provided in the embodiments of this application; Figure 6 This is a flowchart of the dynamic scheduling engine method provided in the embodiments of this application; Figure 7 This is a schematic diagram of the structure of the robot control device provided in the embodiments of this application; Figure 8 This is a schematic diagram of the structure of the electronic device provided in the embodiments of this application. Detailed Implementation

[0013] The features and exemplary embodiments of various aspects of this application will be described in detail below. To make the objectives, technical solutions, and advantages of this application clearer, the application will be further described in detail below with reference to the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are only intended to explain this application and not to limit it. For those skilled in the art, this application can be implemented without some of these specific details. The following description of the embodiments is merely to provide a better understanding of this application by illustrating examples.

[0014] It should be noted that, in this document, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising..." does not exclude the presence of additional identical elements in the process, method, article, or apparatus that includes the element.

[0015] It should be noted that the acquisition, storage, use, and processing of data in this application embodiment all comply with the relevant provisions of national laws and regulations.

[0016] It should be noted that in the embodiments of this application, certain software, components, models and other existing solutions in the industry may be mentioned. These should be regarded as exemplary and are only intended to illustrate the feasibility of implementing the technical solution of this application. However, it does not mean that the applicant has used or necessarily used the solution.

[0017] To address the problems of poor dynamic adaptability and the tendency for local anomalies to lead to global shutdown when multiple heterogeneous actuators work collaboratively in unstructured environments, the present application provides a robot control method, device, electronic device, computer storage medium, and computer program product in detail below with reference to the accompanying drawings and specific embodiments and application scenarios.

[0018] This application provides a robot control method, such as... Figure 1 As shown, this method is applied to a robot control platform, which includes a unified protocol interface. The method includes the following steps: S100: In response to receiving the user's task objective, it obtains the real-time status information of all execution entities connected to the robot control platform through the unified protocol interface; The robot control platform refers to a central decision-making system that provides unified access, intelligent task scheduling, and real-time collaborative control for various types of robot terminals. It serves as the decision-making hub of the entire collaborative control system. Specifically, it can be a software system deployed on a home gateway, edge computing device, or cloud server. For example, it could be a low-power edge computing gateway deployed in a home environment, connecting to various robots in the home via wired or wireless networks, and completing the entire chain of computation locally, from status acquisition to scheduling optimization.

[0019] The unified protocol interface refers to the standardized communication access layer provided by the robot control platform to shield the differences in proprietary communication protocols among different manufacturers, enabling unified data interaction across brands of devices. Specifically, it can be a standardized communication interface based on ROS2DDS (Data Distribution Service), a remote procedure call interface based on gRPC, or an application programming interface based on HTTP / RESTful. For example, it could be a unified topic publish / subscribe interface built on ROS2 middleware, allowing robots from different brands to interact with the platform through standardized message exchange.

[0020] The executing entity refers to various robot terminal devices that are connected to and controlled by the robot control platform, and are the actual executors of the tasks. Specifically, these can be wheeled mobile robots, quadruped robots, wheeled arm robots, cleaning robots, companion robots, etc. For example, it could be a kitchen wheeled arm robot, a mobile robot responsible for serving food, and a companion quadruped robot in the living room area that are connected to the same home environment.

[0021] Real-time status information refers to the dynamic data continuously reported by the executor during operation, reflecting its current working status. It serves as the foundational input for subsequent capability assessment and scheduling decisions. Specifically, it can include remaining battery power, current position coordinates, movement speed, load status, sensor health, joint angles, motor temperature, etc. For example, it could be a set of real-time parameters reported by a mobile robot, such as "remaining battery power 75%, currently located at kitchen entrance coordinates (3.2, 1.5), current speed 0 m / s, load 0 kg".

[0022] Specifically, as an example, when a user submits a "prepare dinner" task request to the robot control platform via a mobile app or voice command, the platform responds to the user's task objective by sending a status query request to all connected execution entities through a unified protocol interface. Each execution entity reports its current real-time status information to the platform through its own communication link. The platform utilizes a protocol adaptation layer to parse status data from different manufacturers' proprietary protocol formats into a standardized unified format, thereby obtaining the real-time status information of all execution entities. This unified data acquisition mechanism breaks down information silos between cross-brand devices, ensuring the integrity and timeliness of the data upon which upper-level scheduling decisions are based.

[0023] In practical applications, Figure 2 This is a flowchart of the overall architecture provided in the embodiments of this application, which is described in detail below: S1: Central control layer; This layer includes a task planning module, a dynamic scheduling engine, and a status monitoring and visualization module.

[0024] S2: Protocol conversion layer; This layer includes an SDK transition layer, a protocol parsing module, and a ROS2 message encapsulation module.

[0025] S3: Device execution layer; This layer contains mobile robots, quadruped robots, and wheeled robots.

[0026] S110: Based on real-time status information, determine the real-time multidimensional capability information corresponding to each execution entity; Real-time multidimensional capability information refers to comprehensive data used to quantify the current actual performance level of the executing entity across multiple capability dimensions, serving as the basis for subsequent task adaptability calculations. Specifically, it can be a capability vector composed of normalized scores from multiple capability dimensions. For example, it could be a five-dimensional vector [0.8, 0.6, 0.9, 0.7, 0.5], representing the current scores of the executing entity in the dimensions of mobility, operational precision, endurance, load capacity, and environmental adaptability, respectively.

[0027] Specifically, as an example, the control platform extracts the original metrics corresponding to each capability dimension from the real-time status information obtained in step S100, normalizes each original metric, and constructs a capability vector. This capability vector serves as the real-time multi-dimensional capability information of the executing entity. When a mobile robot's maximum range decreases due to battery aging, the system automatically lowers its score in the range capability dimension, thereby reflecting the robot's true capability status in real time. This process establishes a direct causal chain of "real-time status → capability model," effectively avoiding the problem of unreasonable task allocation caused by equipment performance degradation.

[0028] S120: Obtain the current environmental spatial characteristics, and decompose the user task objective into multiple task nodes and the dependencies between task nodes based on the current environmental spatial characteristics; Among them, the current environmental spatial characteristics refer to the perceived data reflecting the physical environmental state of the work area, which is the environmental constraint input for task decomposition and scheduling. Specifically, it can include the spatial layout of the home or workplace, the distribution of obstacles, the width of passageways, the density of personnel activity, and the functional attributes of the area. For example, it can be environmental description data such as "the kitchen area is 6 square meters, there are currently 2 people active, the corridor width is 1.2 meters, and the living room area is currently unoccupied".

[0029] In this context, a task node refers to the smallest independently schedulable task unit obtained by decomposing the user task objective according to execution logic, and it is the basic granularity of scheduling and allocation. Specifically, it can be "go to the kitchen to get food", "deliver the plate to the dining table", "clean the living room floor", etc. For example, the user task objective of "preparing dinner" can be decomposed into four task nodes: "preparing food", "getting food", "delivering food", and "cleaning up after the meal".

[0030] In this context, the dependency relationship between task nodes refers to the sequential constraints on the execution time of different task nodes, used to ensure the logical correctness of task execution. Specifically, it can be a predecessor-successor relationship, a parallel relationship, or a mutual exclusion relationship, etc. For example, the "delivering food" task node can only start after the "preparing food" task node has been completed, while "cleaning the living room" and "preparing food" can be executed in parallel.

[0031] Specifically, as an example, the control platform acquires the spatial characteristics of the current environment by fusing multi-source sensing data from LiDAR, depth cameras, and smart home sensors. The platform then decomposes the user's task objectives based on these environmental spatial characteristics. For instance, when the platform detects frequent human activity and high spatial density in the kitchen area, it breaks down the "preparing dinner" task into four task nodes: "preparing food," "waiting for kitchen space to become available," "retrieving food," and "delivering food," establishing a dependency chain where "preparing food" precedes "retrieving food," and "retrieving food" precedes "delivering food." This environmentally-aware task decomposition method allows the scheduling scheme to adapt to the dynamic changes in the actual working environment.

[0032] S130: Based on the real-time multi-dimensional capability information of the executing entity and the task characteristics of the task nodes, determine the task adaptability of the executing entity to each task node; Task characteristics refer to a set of parameters that describe the attributes and execution requirements of a task node, used to match the capabilities of the executing entity. Specifically, these can include the task's required load capacity, operational accuracy requirements, travel distance, time sensitivity, and environmental adaptability requirements. For example, the characteristics of a "food delivery" task could be "requires a 2kg load, low operational accuracy requirements, a travel distance of 8 meters, and high time sensitivity."

[0033] Task suitability refers to a quantitative score that measures the degree of matching between the real-time multidimensional capability information of the executing entity and the task characteristics of the task node. The higher the value, the more suitable the executing entity is for performing the task node. Specifically, it can be a weighted calculation between 0 and 1. For example, a mobile robot has a suitability of 0.85 for the "food delivery" task node, but only 0.32 for the "delicate operation" task node.

[0034] Specifically, as an example, the control platform compares the real-time multi-dimensional capability information of each executor with the task characteristics of each task node item by item, and calculates the suitability score of each executor for each task node through weighted calculation. For example, for the "food delivery" task node, the platform comprehensively considers the mobility (whether it can arrive quickly), load capacity (whether it can carry the tray), battery life (whether the battery is sufficient), and environmental adaptability (whether it can drive safely in a crowded environment) of each executor. Finally, it calculates the suitability score as 0.85 for the wheeled mobile robot, 0.72 for the quadruped robot, and 0.45 for the wheel-arm manipulator robot. The scheduling engine sorts the candidate executors according to the suitability score, thus providing a basis for subsequent optimal matching.

[0035] In practical applications, the formula for calculating task suitability is: ; in, The task suitability score is given to the execution entity r for executing task node t. The higher the value, the more suitable the execution entity is for executing the task node. r This is a performance indicator based on a combined score of maximum linear velocity and acceleration, with data sourced from motion parameters reported in real-time by the execution entity's SDK; A r B is an operational accuracy indicator, representing the normalized reciprocal of the end-effector repeatability accuracy. The data originates from the sensor calibration data of the actuator. r The range indicator is estimated using a battery capacity and power consumption model, with data sourced from real-time battery management system data; L r This is a load capacity indicator, calculated from the maximum load-bearing weight, with data derived from the rated parameters of the actuator; E rThe environmental adaptability index is based on a comprehensive evaluation of sensor performance and operational environment interference conditions, with data sourced from the environmental perception module. w1 to w5 are the weight coefficients for each capability dimension, all positive numbers ranging from 0 to 1. These can be fixed and preset or adaptively adjusted using historical data. It is recommended to use normalized weights during deployment to ensure the sum of all weights is 1. λ is the regularization parameter, ranging from 0.001 to 0.1, preset during model training to prevent overfitting. Dw is the model complexity penalty term, used to constrain the scale of model parameters. All the above indicators have been normalized and mapped to the [0,1] interval. Furthermore, Dtr is the Euclidean distance between the task location and the execution entity location, used to measure spatial reachability.

[0036] S140: Taking the matching relationship between task nodes and execution entities as the optimization objective, the preset objective function is solved based on the task adaptability of task nodes, the priority of task nodes, and the conflict items between task nodes. Under the premise of satisfying the preset constraints including the dependency relationship between task nodes, a global scheduling scheme is obtained. The predefined objective function refers to a mathematical model that aims to maximize the matching weights between task nodes and executors, and is used to solve for the optimal task allocation scheme under multidimensional constraints. Specifically, it can be a linear weighted objective function, an integer programming model, or a heuristic optimization model, for example, a hybrid integer programming model with the objective of weighted summation of fitness, priority, and conflict penalty terms.

[0037] The priority of a task node refers to a quantitative indicator that reflects the urgency and importance of the task node at the business level. Specifically, it can be a value between 0 and 1, calculated based on the task's urgency, user status, and energy consumption constraints. For example, the priority of an elderly person falling and rescuing a task is 1.0, and the priority of a routine cleaning task is 0.3.

[0038] Among them, the conflict term between task nodes refers to the penalty factor used to quantify the risk of interference or congestion between different task nodes in the spatiotemporal dimension. Specifically, it can be a value calculated based on factors such as task time window overlap, path intersection probability, and execution entity volume.

[0039] Among them, the preset constraints refer to the rigid restrictions that must be met when solving the objective function, which are used to ensure the physical executability and logical correctness of the scheduling scheme. Specifically, they may include task uniqueness constraints (each task is executed by only one executor), capability feasibility constraints (the executor must have the minimum capability to execute the task), time sequence constraints (the temporal connection between consecutive tasks of the same executor), and task dependency constraints, etc.

[0040] The global scheduling scheme refers to the set of optimal matching relationships between all task nodes and execution entities, along with their corresponding execution time windows, obtained by solving the objective function under all constraints. Specifically, it can be a scheduling table containing a mapping relationship between "task node - execution entity - execution time window," such as "meal preparation → wheeled arm robot → 17:00-17:20," or "meal delivery → mobile robot → 17:22-17:30," etc.

[0041] Specifically, as an example, the control platform uses a weighted sum of task suitability, task priority, and conflict penalty as the objective function, solving the problem under constraints such as task uniqueness, capability feasibility, time sequence, and task dependencies. For instance, the platform assigns the "meal preparation" task to the wheeled robot with the highest suitability and the "meal delivery" task to the mobile robot with the highest suitability. Considering the dependency between the two tasks, the platform sets the execution time window for "meal delivery" after "meal preparation" is completed, and allows a 2-minute buffer time for the transition between the two. The final output global scheduling scheme maximizes the overall matching score while ensuring that all constraints are met.

[0042] In practical applications, the formula for calculating the priority of a task node is: ; Among them, P t The overall priority score for task node t is calculated, with a value ranging from [0,1]. A larger value indicates a higher priority for the task. t This represents the urgency of the task itself, with a value ranging from [0,1]. For example, the urgency of a task involving rescuing an elderly person from a fall is 1.0, while the urgency of a routine cleaning task is 0.4. The higher the urgency level, the higher the priority of the task. t Energy cost, ranging from [0,1], includes factors such as long distance, long estimated operating time, or involvement of highly dynamic actions. A higher value indicates a lower priority. t α1 represents time sensitivity, with a value ranging from [0,1], reflecting deadline pressure. For example, tasks like cooking have high time sensitivity. α2 are weighting coefficients, ranging from (0,1), satisfying α1 + 2α2 = 1. Generally, α1 is the largest, followed by α2, and α2 determines whether tasks with closer deadlines are prioritized. This formula calculates task-level priority, not the matching priority between the executing entity and the task.

[0043] In practical applications, the preset objective function is: ; Where, x trThis is a binary decision variable, with a value of 1 indicating that task node i is assigned to execution entity r for execution, and a value of 0 indicating that it is not assigned. The value set is {0,1}. In the implementation, this parameter corresponds to a record in the task routing table issued by the scheduler; W tr Let be the matching weight between task node t and execution entity r, taking into account suitability, task priority, and conflict penalty. It is defined as the interval [0,1], where a larger weight indicates that the execution entity is more suitable to execute the task node. The objective function sums all matching schemes between task nodes and execution entities, aiming to maximize the overall matching score, thereby achieving optimal scheduling of time window allocation and task-execution entity matching.

[0044] In practical applications, the matching weight W tr The calculation formula is: ; Among them, S tr The task fit score measures the degree to which the executing entity r matches the requirements of the task node t in terms of capabilities, accuracy, endurance, load, and environmental adaptability; P t The priority score for task node t reflects business urgency, user preference, or security requirements. It is recommended to normalize it to [0,1]. It can be adjusted dynamically based on user annotation, rule inference, or time window. For example, the priority of the medicine retrieval task is 0.9, and the priority of the routine cleaning task is 0.3. Cconflict is a conflict penalty item, which represents the conflict cost or risk that may be caused by assigning the task node to the executing entity. This includes path conflict, time window overlap, kinematic conflict, etc. It can be calculated by the conflict detection module based on factors such as task time window overlap, path intersection probability, and the volume of the executing entity. It is usually normalized to [0,1]. The coefficient of this item is amplified by β2 and deducted from the total score. The higher the value, the less desirable it is to perform this match. β1 and β2 are weight coefficients with values ​​between (0,1). Their sum is 1. They can be preset by domain experts or trained through historical data and adjusted online adaptively.

[0045] In practical applications, preset constraints include the following five categories: Task uniqueness constraint: ; This constraint ensures that each task node t is executed by only one execution entity. For complex tasks that require multiple execution entities to work together, they can be broken down into multiple sub-task nodes, or the number of executions can be set according to the characteristics of the task.

[0046] Capability feasibility constraints: ; Where ytr∈{0,1} represents whether the executing entity r has the ability to execute task node t, including a comprehensive judgment of dimensions such as load capacity, operational accuracy, degree of freedom, power consumption, and environmental adaptability.

[0047] Time sequence constraints: ; The start and end times of the task nodes are determined by the scheduling scheme; `travel` is the estimated travel time for the executing entity to move from the current task node position to the next task node position; `Δ` is the safety interval to avoid congestion or collisions; `M` is a large constant used only for constraint relaxation. The above constraints only take effect when `xtτ = xtτ' = 1` (the same executing entity executes two tasks).

[0048] Task dependency constraints: ; This constraint ensures the temporal correctness of dependencies and prevents task failures due to incorrect scheduling. For example, if task node tj depends on the execution result of task node ti, then the above constraint must be satisfied.

[0049] Available time constraints for the executing entity: ; Among them, A r B r The available time window for the executing entity is defined. During unavailable time periods, such as charging cycles, maintenance windows, or specific instructions from family members, tasks should not be assigned to it.

[0050] The aforementioned objective function and constraint set together constitute a feasible collaborative scheduling mechanism for heterogeneous robots in the home. This mechanism enables automatic selection and adaptation of tasks across different robot brands, optimization of time windows in real-time environments, proactive avoidance of conflict risks, compatibility with task dependencies and changes in the availability of the executing entity, and scheduling schemes that are executable, safe, and meet the real-time requirements of the home environment.

[0051] S150: Based on a global scheduling scheme, it controls the corresponding execution entity through a unified protocol interface, enabling the execution entity to execute the corresponding task node; Specifically, as an example, the control platform, based on the matching relationships determined in the global scheduling scheme, issues corresponding control commands to each target execution entity through a unified protocol interface. For instance, the platform issues a standardized control command to the wheeled robot to "start food preparation" and a preparatory command to the mobile robot to "stand by at the kitchen entrance." After receiving the commands, each execution entity begins executing the corresponding task node according to the time window specified in the scheduling scheme.

[0052] S160: During the execution of task nodes by the executing entity, the global scheduling scheme is adjusted in response to the satisfaction of preset risk conditions.

[0053] Among them, the preset risk conditions refer to the judgment conditions of abnormal states or environmental changes that are triggered during the execution of the task and may affect the normal execution of the task. Specifically, these can be risk scores exceeding preset thresholds, abnormal deviations in sensor data, or communication link interruptions. For example, it could be the positioning drift of an executing entity exceeding 0.2 meters, or network latency exceeding 500 milliseconds.

[0054] Specifically, as an example, during the "food delivery" task, the system continuously monitors its operational status. When it detects that the robot's positioning drift exceeds 0.2 meters due to LiDAR matching failure caused by restaurant floor reflections, the system determines that a preset risk condition has been met and immediately adjusts the global scheduling plan. The adjustment could be to downgrade the robot's goal from "precise delivery to the table" to "delivery to the restaurant entrance area," while ensuring that other tasks in the global scheduling queue remain unaffected. This local adjustment mechanism achieves a balance between fault isolation and service continuity, preventing systemic turbulence caused by single-point failures.

[0055] In this embodiment, real-time status information of all execution entities is obtained through a unified protocol interface, breaking down information silos between cross-brand devices. Real-time multi-dimensional capability information is determined based on this status information, enabling dynamic quantitative evaluation of the capabilities of heterogeneous execution entities. Task decomposition is performed based on environmental spatial characteristics, ensuring the scheduling scheme aligns with the actual operating environment. A global scheduling scheme that balances efficiency and safety is generated through comprehensive optimization of task adaptability, priority, and conflict items. The scheduling scheme is dynamically adjusted in response to risk conditions during execution, ensuring the system's robustness and service continuity in complex dynamic environments. Thus, efficient, stable, and real-time collaborative control for multiple heterogeneous execution entities is achieved.

[0056] In any of the above embodiments, optionally, the unified protocol interface includes multiple protocol adapters; obtaining real-time status information of all execution entities connected to the robot control platform through the unified protocol interface, and controlling the corresponding execution entities through the unified protocol interface, includes: S300: Through the corresponding protocol adapter, it receives uplink messages in the private communication protocol format sent by each execution entity, converts the uplink messages in the private communication protocol format into platform standard messages, and obtains real-time status information; Protocol adapters refer to software conversion modules developed for specific vendors' proprietary communication protocols, used for bidirectional translation between the platform's standard message format and the vendor's proprietary communication protocol format. Specifically, they can be dynamically loaded software modules based on a plug-in architecture. Examples include an RTDE adapter developed for Universal Robots' RTDE protocol, a MiR adapter developed for MiR robots' HTTP / TCP protocol, or a DJI adapter developed for DJI drones' SDK protocol.

[0057] Among them, uplink messages in proprietary communication protocol formats refer to status reporting data sent by the executing entity to the control platform according to its proprietary data format defined by the vendor. Specifically, these can be status messages in JSON format, binary frame sequences, Modbus register data, etc. For example, it could be a JSON message reported by a mobile robot of a certain brand according to its proprietary SDK specifications, containing fields such as "battery_voltage:24.5, position_x:3200, position_y:1500".

[0058] Among them, platform standard messages refer to the standardized data format uniformly adopted within the robot control platform, independent of any specific manufacturer's hardware implementation. Specifically, they can be standard topic messages or service request / response messages based on the ROS2 message definition. For example, they can be structured messages containing standardized fields such as "battery_percentage:85, pose:{x:3.2, y:1.5, theta:0.5}, velocity:0.0".

[0059] S301: Through the corresponding protocol adapter, the platform standard message corresponding to the global scheduling scheme is converted into a downlink message in the private communication protocol format of the corresponding execution subject, and the downlink message in the private communication protocol format is sent to the corresponding execution subject.

[0060] Among them, downlink messages in proprietary communication protocol formats refer to control data after the control platform converts standardized scheduling instructions into a proprietary format that can be recognized and executed by a specific execution entity. Specifically, this can be an API request body conforming to the vendor's SDK specifications, binary control frames, etc. For example, it could be a "move_to(x:5.0, y:2.3, speed:0.5)" JSON request sent to a certain type of mobile chassis, conforming to its vendor's SDK specifications.

[0061] Specifically, as an example, when a robot control platform needs to obtain the real-time status of a certain brand of mobile robot, the robot sends an uplink message containing parameters such as battery voltage, position coordinates, and movement speed via its proprietary HTTP / TCP protocol. Upon receiving this message, the corresponding MiR protocol adapter in the platform first converts the battery voltage value into a remaining battery percentage using a discharge curve model, maps the position coordinates in the proprietary coordinate system to coordinates in the platform's unified coordinate system, and finally generates status information conforming to the platform's standard message format. Conversely, when the platform needs to issue a "move to target location" scheduling command to the robot, the MiR protocol adapter converts the target coordinates and desired speed in the platform's standard message into an API request format recognizable by the robot's proprietary protocol and sends it via the HTTP interface.

[0062] In practical applications, when a ROS2 node requests hand status, the interface encapsulation module automatically constructs a GetHandStateRequest using a private SDK model. This request describes the entire hand query without requiring additional parameters and selects the motion control endpoint MCENDPOINT based on a preset mapping. The AimRT HTTP communication module sends a POST request to / rpc / aimdk.protocol.McDataService / GetHandState, and the returned JSON / RPC data is automatically deserialized into a nested structure by the SDK. This design completely decouples the core scheduling algorithm from the underlying hardware. When the system needs to connect to a new brand or model of robot, only a new adapter plugin needs to be developed and loaded, without modifying the core code of the control platform, greatly improving the system's scalability and maintenance efficiency.

[0063] In this embodiment, by setting up multiple protocol adapters in the unified protocol interface, bidirectional conversion between private communication protocol formats and platform standard messages is realized, enabling the control platform to manage execution entities from different manufacturers in a unified manner and eliminating communication barriers between heterogeneous devices.

[0064] In any of the above embodiments, optionally, converting uplink messages in the private communication protocol format into platform standard messages, and converting platform standard messages corresponding to the global scheduling scheme into downlink messages in the private communication protocol format of the corresponding execution entity, includes: S300a: When converting uplink messages in the private communication protocol format into platform standard messages, the parameters in the uplink messages in the private communication protocol format are converted to uplink dimensions according to the dimension mapping relationship between the private communication protocol and the platform standard messages, so as to obtain the uplink message parameters after dimension conversion. S300b: Based on the data structure of the private communication protocol and the field correspondence of the platform standard message, the uplink field mapping is performed on the uplink message parameters after the unit conversion to generate the platform standard message; S301a: When converting the platform standard message corresponding to the global scheduling scheme into a downlink message in the private communication protocol format of the corresponding execution subject, the parameters in the platform standard message are converted to downlink dimensions according to the dimension mapping relationship between the platform standard message and the private communication protocol to obtain the downlink message parameters after dimension conversion. S301b: Based on the correspondence between the fields of the platform standard message and the data structure of the private communication protocol, the downlink field mapping is performed on the downlink message parameters after the unit conversion to generate downlink messages in the format of the private communication protocol.

[0065] Among them, the dimensional mapping relationship refers to the mathematical correspondence between different units of measurement or numerical ranges used by proprietary communication protocols and platform standard messages for the same physical quantity, which is the foundation for achieving data semantic interoperability. Specifically, it can be a linear scaling relationship, a non-linear mapping curve, or a lookup table, etc. For example, it can be a linear scaling relationship between the angle value range of 0 to 2000 in a certain brand of robot's proprietary protocol and the angle value range of 0 to 1000 in the platform standard message, that is, the proprietary value divided by 2 equals the standard value.

[0066] Uplink dimension conversion refers to converting physical quantity values ​​in the proprietary protocol into values ​​in the platform's standard dimensions according to a dimension mapping relationship during the data reporting process from the executing entity to the platform. Specifically, this could involve converting battery voltage values ​​into charge percentages using a discharge curve, or converting millimeter values ​​in the proprietary coordinate system into meter values ​​in the platform coordinate system. For example, converting "angle value 1500" in the proprietary protocol into "standard angle value 750" according to the mapping relationship between 0 and 2000 and 0 and 1000.

[0067] The field mapping relationship refers to the semantic mapping relationship between each field in the data structure of the private communication protocol and each field in the platform standard message, which is used to eliminate the differences in data organization methods between different protocols. Specifically, it can be a one-to-one field mapping, a many-to-one field merging, or a one-to-many field splitting, etc. For example, the "left_hand_finger1_state" field in the private protocol corresponds to the "hand_data.left.index_finger" field in the platform standard message.

[0068] Uplink field mapping refers to reorganizing the converted parameters into a data structure for the platform standard message according to the field correspondence after the unit conversion is completed. Specifically, it can be to rearrange the flat key-value pairs in the private protocol into a nested structure in the platform standard message. For example, mapping "finger1:1, finger2:0, finger3:1" in the private protocol to "hand_data:{index:1, middle:0, ring:1}" in the platform standard message.

[0069] Specifically, as an example, in the uplink direction, a certain brand of robotic arm's proprietary protocol returns a set of joint angle data, with each joint angle value ranging from 0 to 2000. The protocol adaptation module first divides each joint angle value by 2 according to the dimension mapping relationship, converting it into a value ranging from 0 to 1000 under the platform standard dimension. Subsequently, according to the field correspondence, the flat data arranged in the order of "joint_1, joint_2, ..., joint_6" in the proprietary protocol is reorganized into a nested structure of "joint_states:{positions:[q1, q2, ..., q6]}" in the platform standard message, ultimately generating a complete platform standard message. In the downlink direction, the platform issues a standardized instruction of "left-hand grasp," where "left-hand" is an enumeration value of 1 and "grab" is action code 0x05. The protocol adaptation module maps the enumeration value 1 to the register address 0x0A in the private protocol according to the field correspondence, maps the action code 0x05 to the action code 0x03 in the private protocol, and converts the target force value from the platform standard Newton unit to the PWM duty cycle value of the private protocol according to the dimension mapping relationship, and finally generates the private protocol format downlink message that the robotic arm can execute.

[0070] In practical applications, the data semantic conversion module scales based on the unit differences of the instruction / status fields. For example, it maps the 0-2000 angle values ​​of the proprietary protocol to the 0-1000 range agreed upon by ROS2, and extracts the status of each finger of the left and right hands according to the fingertip enumeration order. The conversion results are filled into the ROS2 service response or topic message, realizing unified support for synchronous query and asynchronous distribution, allowing the upper-layer controller or monitoring interface to directly consume standardized ROS2 data. Through this dual conversion mechanism, this invention ensures that every abstract instruction issued by the upper-layer scheduler can be accurately understood as a specific physical action by the underlying heterogeneous devices, while also ensuring that the status feedback of the underlying devices can be uniformly perceived by the upper layer.

[0071] In practical applications, Figure 3 This is a flowchart of the protocol layer architecture provided in an embodiment of this application, which is described in detail below: S400: ROS2 node / service call; In this step, a request to invoke a ROS2 node or service is initiated.

[0072] S410: Bridge layer interface encapsulation module; In this step, a default GetHandStateRequest is generated, and the corresponding HTTP endpoint is selected.

[0073] S420: Private protocols (such as HTTP communication modules); In this step, / rpc / aimdk.protocol.McDataService is invoked, and the corresponding HTTP endpoint is selected.

[0074] S430: Data semantic conversion module; In this step, instruction / status conversion and pointer structure mapping are performed.

[0075] S440: ROS2 message generation module; In this step, populate the HandDataGetResponse or HandDataMsg and publish it to the ROS2 service / topic.

[0076] S450: Downstream sensing and control module; This step is for use by the controller or monitoring interface.

[0077] In this embodiment, the data semantic gap between heterogeneous devices is eliminated through a dual conversion mechanism of dimensional mapping and field mapping, so that every abstract instruction issued by the upper-layer scheduler can be accurately understood by the lower-layer heterogeneous devices as a specific physical action.

[0078] In any of the above embodiments, optionally, the real-time multidimensional capability information includes multiple capability dimensions, including at least two of the following: motion capability dimension, operational precision dimension, battery life dimension, load capability dimension, and environmental adaptability dimension; based on real-time status information, determining the real-time multidimensional capability information corresponding to each execution entity includes: S1101: Extract the original metric values ​​corresponding to each capability dimension from the real-time status information; S1102: Based on the normalization parameters of the corresponding capability dimension, the original metric values ​​are normalized to obtain the normalized score of each capability dimension. S1103: Construct a capability vector for each executing entity based on the normalized score of each capability dimension, and use the capability vector as real-time multi-dimensional capability information.

[0079] Among them, the capability dimension refers to the classification index used to quantitatively evaluate the operational capabilities of the executing entity from different perspectives, and it is the basic coordinate axis for constructing the capability vector. Specifically, it can include dimensions such as motion capability, operational accuracy, endurance, load capacity, and environmental adaptability. For example, the motion capability dimension can be evaluated based on a comprehensive assessment of maximum linear velocity and acceleration; the operational accuracy dimension can be represented by the normalized reciprocal of the end-effector repeatability accuracy; the endurance dimension is estimated from the current battery capacity and real-time power consumption model; the load capacity dimension is directly taken from the maximum load-bearing weight parameter; and the environmental adaptability dimension is based on a comprehensive assessment of sensor performance and operational environmental interference conditions.

[0080] The raw metric refers to the actual physical measurement or calculated value of an executor in a certain capability dimension, which is not standardized and reflects the absolute capability level of the executor in that dimension. Specifically, it can be the maximum linear velocity (m / s), end-effector repeatability error (mm), remaining battery percentage (%), rated load (kg), etc. For example, the raw metric for the motion capability dimension of a mobile robot is a maximum linear velocity of 1.5 m / s, and the raw metric for the endurance dimension is 75% remaining battery.

[0081] The normalization parameter refers to the reference benchmark used to map raw measurement values ​​of different dimensions to a unified dimensionless interval, including upper and lower bound reference values ​​for each capability dimension. Specifically, it can be the theoretical extreme value in the equipment's factory specifications, the statistical extreme value of historical observations before deployment, or the observation extreme value dynamically updated during operation. For example, the upper bound reference value for the motion capability dimension is 2.0 m / s (the maximum speed limit of all executing entities in the system), and the lower bound reference value is 0 m / s.

[0082] The normalized score refers to the dimensionless value obtained after normalization, mapped to a unified interval (such as [0,1]), which facilitates weighted comparison of capability indicators with different dimensions. Specifically, it can be a value between 0 and 1 calculated using the minimum-maximum linear normalization formula. For positive indicators that are "the larger the better" (such as speed and load), the original value is directly subtracted from the lower bound reference value and then divided by the difference between the upper and lower bounds. For negative indicators that are "the smaller the better" (such as positioning error), the inverse transformation is performed first, followed by the normalization operation.

[0083] In practical applications, the formula for calculating the normalized score is: ; Wherein, C̄ri is the normalized score of the executor r in the i-th capability dimension. After normalization, the value is mapped to the interval [0,1]. The larger the value, the stronger the capability in that dimension or the closer it is to the upper bound. Ci is the original metric value of the executor r in the i-th capability dimension, such as the maximum linear velocity, the reciprocal of the end-point repetitive positioning error, the remaining battery percentage, the rated load, etc. This value is provided in real time by the executor SDK or the sensor. Ci^min is the lower bound reference value (minimum value) used in the system for the i-th capability dimension. It can be set by the theoretical minimum physical value, the minimum value of the equipment specification, or the historical minimum value statistically analyzed before deployment. Ci^max is the upper bound reference value (maximum value) used in the system for the i-th capability dimension. It can be set by the theoretical maximum physical value, the upper limit of the equipment specification, or the measured historical maximum value. Ci^max-Ci^min is the dimensional difference, which is used to linearly map the original metric to the range [0,1]. If Ci^max = Ci^min or an out-of-bounds condition occurs, the system will trim the normalized result or use a preset default value and trigger an adaptive reassessment process for the upper / lower bounds. To improve robustness, the original metric can be processed using a short-term moving average or exponential smoothing before normalization. The initial values ​​of the upper and lower bounds can be determined by the equipment specifications or historical observations before deployment, and are periodically updated based on the sliding window observations during runtime.

[0084] The capability vector is a multi-dimensional numerical vector composed of the normalized scores of the executing entity across all capability dimensions arranged in a fixed order. It is used to comprehensively represent the current overall operational capability of the executing entity. Specifically, it can be a multi-dimensional floating-point array. For example, the capability vector of a mobile robot is [0.75, 0.60, 0.85, 0.70, 0.55], corresponding to the current scores in the dimensions of motion capability, operational accuracy, endurance, load capacity, and environmental adaptability, respectively.

[0085] Specifically, as an example, the control platform extracts raw metrics of a mobile robot from real-time status information in five dimensions: motion capability, operational accuracy, endurance, load capacity, and environmental adaptability. These metrics are: maximum linear speed 1.5 m / s, end-effector repeatability error 2 mm, remaining battery power 75%, rated load 5 kg, and environmental adaptability score 80 points. Subsequently, the original metrics were normalized: the upper bound for the motion capability dimension was 2.0 m / s, and the lower bound was 0 m / s, with a normalized score of 1.5 / 2.0 = 0.75; the operational accuracy dimension was a reverse indicator, with an upper bound of 10 mm and a lower bound of 0 mm, with a normalized score of (10-2) / 10 = 0.80; the endurance dimension had an upper bound of 100% and a lower bound of 0%, with a normalized score of 0.75; the load capacity dimension had an upper bound of 10 kg and a lower bound of 0 kg, with a normalized score of 0.50; and the environmental adaptability dimension had an upper bound of 100 points and a lower bound of 0 points, with a normalized score of 0.80. The final capability vector for this executing entity was constructed as [0.75, 0.80, 0.75, 0.50, 0.80].

[0086] In practical applications, the capability vector of the executing entity r is defined as: ; Among them, Mr represents the motion capability component, which is based on a combined score of maximum linear velocity and acceleration, and the data comes from the motion parameters reported in real time by the execution entity's SDK; A r B represents the operational accuracy component, and the normalized reciprocal of the end-effector repeatability accuracy. The data originates from the sensor calibration data of the actuator. r The range component is estimated using a battery capacity and power consumption model, with data sourced from real-time battery management system data; L r The load capacity component is calculated from the maximum load-bearing weight, and the data comes from the rated parameters of the actuator; E r The environmental adaptability component is a comprehensive assessment based on sensor performance and operational environmental interference conditions, with data sourced from the environmental perception module. This capability vector fully quantifies the current capability level of the executing entity across five dimensions: motion, operation, endurance, load, and environmental adaptability.

[0087] In practical applications, the system acquires status parameters in real time through the robot SDK, and the scheduling center automatically updates the capability vector every 10 seconds. Update messages are published via ROS2 Topic / robot_capability_update, enabling synchronous updates of the capability status of multiple devices. This dynamic update mechanism gives the capability assessment model the ability to adapt to the evolution of the entire device lifecycle, ensuring that the capability data on which scheduling decisions are based always closely reflects the actual working conditions of the physical entity.

[0088] In this embodiment, a unified capability vector is constructed by extracting and normalizing multi-dimensional original metrics, realizing the quantitative representation and horizontal comparison of the capabilities of heterogeneous execution entities, and providing a standardized capability assessment basis for subsequent task adaptability calculation.

[0089] In any of the above embodiments, optionally, the normalization parameter includes an upper bound reference value and a lower bound reference value for each capability dimension, and the method further includes: S1102a: Based on the sliding window observations, update the upper and lower bound reference values ​​to obtain the updated upper and lower bound reference values; S1102b: Recalculate the normalized score of the capability dimension based on the updated upper and lower bound reference values; S1102c: Update the capability vector based on the normalized score of the recalculated capability dimension.

[0090] The upper and lower bound reference values ​​are two boundary values ​​in the normalization parameters used to define the numerical range of each capability dimension. The upper bound reference value represents the maximum possible value of the capability dimension within the current observation range, while the lower bound reference value represents the minimum possible value. Specifically, these can be theoretical extreme values ​​in the equipment's factory specifications, statistical extreme values ​​from historical observations before deployment, or observational extreme values ​​dynamically updated during runtime. For example, the upper bound reference value for the motion capability dimension is 2.0 m / s (the maximum speed limit for all executing entities within the system), and the lower bound reference value is 0 m / s.

[0091] Among them, the sliding window observation value refers to the statistical value of the actual operation data of the capability dimension continuously collected within a preset time window, which is used to reflect the true capability boundary of the executing entity under the current working conditions. Specifically, it can be the maximum and minimum values ​​actually observed in each capability dimension in the last 24 hours, the extreme value statistics in the last 50 task cycles, etc. For example, it can be that the maximum range of a mobile robot actually observed in the last 24 hours is 6.2 kilometers.

[0092] The recalculation of the normalized score for the capability dimension refers to re-normalizing the original metric using the updated reference values ​​after updating the upper and lower bounds, to obtain a normalized score that more closely reflects the actual capability level. Specifically, this can be achieved by using a minimum-maximum linear normalization formula: subtracting the updated lower bound reference value from the original metric and then dividing by the updated difference between the upper and lower bounds. For example, when the upper bound reference value for the range capability dimension is lowered from 10 kilometers to 6.5 kilometers, the normalized score for the current actual range of 6 kilometers increases from 0.6 to 0.92, more accurately reflecting the robot's relative level within the current available resource pool.

[0093] Specifically, as an example, the system sets a 24-hour sliding time window to continuously record the maximum and minimum values ​​actually observed by each executing entity in each capability dimension. This data is then smoothed using historical extreme values, serving as the new upper and lower bound reference values ​​for normalization at the current moment. When battery degradation of a robot is detected, leading to a decrease in its maximum endurance, the maximum endurance observed within the sliding window decreases accordingly, and the system automatically lowers the upper bound reference value. Subsequently, based on the updated upper and lower bound reference values, the normalized score for that capability dimension is recalculated. Finally, the capability vector is updated based on the recalculated normalized score. In this way, even if the robot's absolute endurance decreases, its relative score in the current capability distribution can still objectively reflect its true level within the currently available resource pool. Conversely, if a newly connected device performs better than the preset benchmark, the system automatically raises the upper bound reference value, allowing the capability score of high-quality devices to fully reflect their advantages.

[0094] In practical applications, the system acquires status parameters in real time through the robot SDK, and the scheduling center automatically updates the capability vector every 10 seconds. Update messages are published via ROS2 Topic / robot_capability_update. During the update process, the system synchronously checks whether the actual observed values ​​of each capability dimension exceed the current upper and lower bound reference values. If they do, the reference values ​​are automatically updated and the normalized score is recalculated, thereby updating the capability vector. This dynamic calibration mechanism enables the capability assessment model to adapt to the evolution of the equipment throughout its entire lifecycle, ensuring that the capability data used for scheduling decisions always closely reflects the actual operating conditions of the physical entity.

[0095] In this embodiment, the upper and lower bound reference values ​​are updated based on the sliding window observations, and the normalized score is recalculated based on the updated reference values ​​to update the capability vector. This enables the capability assessment model to adaptively track the long-term evolution of equipment performance, avoids assessment distortion caused by static benchmarks, and ensures that the capability data on which scheduling decisions are based always closely reflects the actual working conditions of the physical entity.

[0096] In any of the above embodiments, optionally, after controlling the corresponding execution entity through a unified protocol interface based on a global scheduling scheme to enable the execution entity to execute the corresponding task node, the method further includes: S1501: Based on the updated environmental spatial characteristics, adjust the time window parameters of at least one task node in the global scheduling scheme; S1502: Based on the adjusted time window parameters, the preset objective function is re-solved to obtain the updated global scheduling scheme.

[0097] The updated environmental spatial characteristics refer to the latest physical environmental state of the work area obtained by the control platform through continuous fusion of multi-source sensor data during the execution of the task nodes by the executing entity. Compared with the initial environmental spatial characteristics obtained in step S120 of Embodiment 1, the updated environmental spatial characteristics reflect the dynamic changes of the work environment during execution. Specifically, this may include real-time changes in personnel activity density, the distribution of newly added or removed obstacles, and temporary changes in the functional attributes of the area. For example, after the executing entity begins to perform the task, several family members suddenly enter the kitchen area, causing the personnel activity density in the area to increase from the initial 1 person to 4 people.

[0098] The time window parameter refers to the parameters related to the execution time range allocated to a task node in the global scheduling scheme, including the start time, end time, and length of the execution time window. Specifically, it can be the earliest start time, the latest finish time, and the allowed execution duration. For example, the time window parameters for a certain "food delivery" task node are "start time 17:22, end time 17:30, and time window length 8 minutes".

[0099] Adjusting the time window parameter of at least one task node in the global scheduling scheme refers to modifying the time window parameter of the specific task node affected based on the updated environmental spatial characteristics, rather than making indiscriminate adjustments to all task nodes. Specifically, this could involve extending or shortening the time window length, advancing or postponing the start time of the time window, etc. For example, when a sudden increase in the density of people in the kitchen area is detected, only the time window length of the "food preparation" task node associated with that area is extended, without affecting the "cleaning" task node in the living room area.

[0100] Re-solving the preset objective function refers to, after the time window parameters have been adjusted, using the adjusted time window parameters as new constraints input into the preset objective function, and re-executing the optimization process to obtain a global scheduling scheme adapted to the latest environmental state. Specifically, this can be achieved by updating only the time window boundary conditions in the constraint space while keeping task adaptability and priority weights unchanged, and then re-running the mixed-integer programming solver. For example, after extending the time window for the "meal preparation" task from 20 minutes to 25 minutes, the updated execution sequence and matching relationships of each task node can be obtained by re-solving the problem.

[0101] Specifically, as an example, during the execution of task nodes according to the global scheduling scheme, the control platform continuously updates the environmental spatial characteristics through multi-source sensing data such as LiDAR, depth cameras, and smart home sensors. When the system detects that the activity density of people in the kitchen area has increased from the initial 1 person to 4 people, it determines that the current environmental spatial characteristics have changed significantly. It then adjusts the time window parameters of the "meal preparation" task node associated with that area, extending its basic execution time window length from the original 20 minutes to 25 minutes to allow for more sufficient obstacle avoidance buffer time. Subsequently, the system re-solves the preset objective function based on the adjusted time window parameters to obtain an updated global scheduling scheme, in which the execution time window of the "meal preparation" task is extended, and the start time of the subsequent "meal delivery" task, which depends on this task, is also correspondingly delayed.

[0102] In practical applications, the acquisition frequency of updated environmental spatial features can be configured according to the system's computing power and the dynamics of the scene. For example, it can be updated every 500 milliseconds in a high-dynamic scene and every 5 seconds in a low-dynamic scene. Whenever a change in environmental spatial features exceeding a preset threshold is detected, the system automatically triggers the adjustment of the time window parameters and the re-solution of the objective function, thereby achieving closed-loop adaptation of the scheduling scheme to environmental changes.

[0103] In this embodiment, by dynamically adjusting the time window parameters based on the updated environmental spatial characteristics and resolving the preset objective function during task execution, the global scheduling scheme can respond to the dynamic changes in the working environment in real time, avoiding the failure of the scheduling scheme due to sudden environmental changes, and significantly improving the scheduling robustness of the multi-heterogeneous execution entity collaborative system in unstructured dynamic environments.

[0104] In any of the above embodiments, optionally, adjusting the time window parameter of at least one task node in the global scheduling scheme based on the updated environmental spatial characteristics includes: S1501a: When the preset work area is detected to be in a peak state of personnel activity based on the updated environmental spatial characteristics, the basic execution time window length of the task node associated with the preset work area is extended.

[0105] The preset work area refers to a specific physical space marked as requiring key monitoring in the global scheduling scheme. It serves as the spatial anchor point for updating environmental spatial characteristics and adjusting time window parameters. Specifically, it can be functional areas such as kitchens, corridors, and living rooms. For example, the kitchen area is marked as the preset work area for the "meal preparation" task node.

[0106] Peak personnel activity refers to an environmental state where the personnel density within a preset work area exceeds a preset threshold, indicating that the area is currently under non-ideal working conditions with high personnel density. Specifically, this can be detected using a dynamic occupancy grid map constructed by fusing multi-source sensing data from LiDAR, depth cameras, and smart home sensors. For example, when the personnel density in the kitchen area increases from an initial 1 person to 4 people, exceeding the preset threshold of 2 people, it is determined that the area is in a peak personnel activity state.

[0107] The basic execution time window length refers to the original execution time allocated to a task node in the global scheduling scheme. It is the optimal time window length obtained by solving a preset objective function under the initial environmental space characteristics. Specifically, it can be the number of minutes obtained through optimization calculation. For example, the basic execution time window length for the "meal preparation" task node is 20 minutes.

[0108] Extending the basic execution time window means adding extra time margin to the original basic execution time window after detecting peak personnel activity, in order to compensate for the decrease in execution efficiency and the increase in obstacle avoidance requirements caused by dense personnel. Specifically, it can be extended by a fixed percentage (such as 20%) or by a fixed duration (such as 5 minutes). For example, the basic execution time window of the "meal preparation" task node can be extended from 20 minutes to 25 minutes.

[0109] Specifically, as an example, the control platform continuously updates the environmental spatial characteristics using multi-source sensing data such as LiDAR, depth cameras, and smart home sensors. When the system detects that the activity density in the kitchen area has increased from the initial 1 person to 4 people, exceeding the preset threshold of 2 people, it determines that the kitchen area is in a peak activity state. At this time, the system extends the basic execution time window of the "meal preparation" task node associated with the kitchen area from the original 20 minutes to 25 minutes to allow more time for obstacle avoidance. Subsequently, the system re-solves the preset objective function based on the extended basic execution time window to obtain an updated global scheduling scheme, in which the execution time window of the "meal preparation" task is extended, and the start time of the subsequent "meal delivery" task that depends on this task is also postponed accordingly.

[0110] In practical applications, the extension ratio or duration can be adaptively adjusted according to the degree to which personnel activity density exceeds the threshold. For example, when personnel density exceeds the threshold by 50%, the extension is 10%; when it exceeds 100%, it is 20%; and when it exceeds 200%, it is 30%. This tiered extension mechanism ensures that the adjustment of the time window matches the severity of environmental disturbances, avoiding resource waste due to excessive extension or safety risks due to insufficient extension.

[0111] In this embodiment, by extending the basic execution time window of the associated task nodes and resolving the preset objective function when the preset work area is detected to be in a peak state of personnel activity, the scheduling scheme can adaptively compensate for the decrease in execution efficiency caused by personnel density, and maintain the continuity of task execution and service quality while ensuring work safety.

[0112] In any of the above embodiments, optionally, adjusting the time window parameter of at least one task node in the global scheduling scheme based on the updated environmental spatial characteristics includes: S1501b: When the user is detected to be in a preset special state based on the updated environmental spatial characteristics, the execution priority of the preset special task node is increased, and the start time of the execution time window of the special task node is preset in advance.

[0113] Among them, the preset special state refers to a state where the user's current physical or behavioral state deviates from the normal baseline and requires special attention from the system, as detected by environmental perception data. Specifically, it can be detected through the fusion of multi-source data such as visual sensors, wearable devices, or smart home sensors. For example, a visual sensor detects an elderly person's fall posture, a wearable device detects an abnormally high heart rate in a user, and a voice sensor detects a user issuing an emergency call for help.

[0114] Among them, the pre-defined special task nodes refer to task nodes that are pre-marked in the global scheduling scheme as being directly related to user safety or emergency needs. When a user is detected to be in a pre-defined special state, these task nodes need to be executed with priority. Specifically, these could be task nodes such as "emergency assistance", "calling emergency services", or "going to the user's location". For example, when an elderly person is detected to have fallen, the task node "going to the elderly person's location to provide assistance" is a pre-defined special task node.

[0115] Increasing execution priority refers to raising the priority weight of a pre-defined special task node in the global scheduling queue, giving it priority in resource contention. Specifically, this could involve raising the priority weight of the task node from its normal value to the highest level. For example, raising the priority weight of the "emergency rescue" task from 0.5 to 1.0 would place it at the top of all pending tasks.

[0116] In this context, advancing the start time of the execution time window refers to moving the original start time of a pre-defined special task node forward, allowing it to start earlier. Specifically, this could mean advancing the start time to the current time or the nearest available time slot. For example, advancing the start time of an "emergency rescue" task originally scheduled to start at 17:30 to the current time of 17:22, allowing it to start immediately.

[0117] Specifically, as an example, the control platform continuously updates the environmental spatial features using multi-source sensing data from visual sensors, wearable devices, and other sources. When the visual sensor detects an elderly person falling in the living room area, the system determines that the user is in a preset special state. At this time, the system immediately increases the execution priority of the "emergency rescue" task node, raising its priority weight from 0.5 to 1.0, and simultaneously advances the start time of the task's execution time window from the originally scheduled 17:30 to the current time 17:22. Subsequently, the system recalculates the preset objective function based on the adjusted execution time window start time and execution priority, obtaining an updated global scheduling scheme. In this scheme, the "emergency rescue" task is scheduled for immediate execution, the nearest quadrupedal robot with the corresponding capabilities is assigned as the execution subject, and other low-priority tasks originally scheduled to be executed during this time period are automatically postponed.

[0118] In practical applications, the detection of preset special states can be comprehensively judged by combining data from multiple sensors, and multiple confidence thresholds can be set to reduce the false alarm rate. For example, a preset special state can only be determined when both the visual sensor and the wearable device detect an anomaly simultaneously, thereby avoiding false triggers caused by sensor noise or normal user actions.

[0119] In this embodiment, by increasing the execution priority of preset special task nodes and advancing the start time of their execution time window when a user is detected to be in a preset special state, the system can respond quickly in emergency situations, ensuring that critical tasks receive the highest priority execution resources, and significantly improving the response speed and reliability of the multi-heterogeneous execution entity collaborative system in terms of user security.

[0120] In any of the above embodiments, optionally, adjusting the time window parameter of at least one task node in the global scheduling scheme based on the updated environmental spatial characteristics includes: S1501c: When multiple execution entities are detected to have spatial conflict risks in a preset area based on the updated environmental spatial characteristics, the start time of the execution time window of at least one task node with conflict risks is postponed.

[0121] Spatial conflict risk refers to the risk of collision, congestion, or mutual interference that may occur when multiple entities are dispatched to the same pre-defined area to perform tasks within the same time period, due to limited physical space. Specifically, it can be detected by predicting the degree of overlap of the movement trajectories of multiple entities in the spatiotemporal dimension. For example, when the predicted paths of two mobile robots pass through the same narrow corridor within the same time period, a spatial conflict risk is determined to exist.

[0122] The preset area refers to a specific physical space marked as requiring key monitoring of spatial conflicts in the global scheduling scheme. It is usually an area with narrow passageways and limited passage capacity. Specifically, it can be a bottleneck area such as a corridor, doorway, or elevator shaft. For example, a 1.2-meter-wide corridor connecting the kitchen and living room is marked as a preset area.

[0123] In this context, postponing the start time of the execution window refers to shifting the originally scheduled start time of a specific task node with a conflict risk to a later date, thus staggering its execution time with that of other conflicting tasks and eliminating overlap in time and space. Specifically, this could be postponing the start time until after the conflicting task is completed, or until after a preset safety interval. For example, the start time of the "food delivery" task could be postponed from 17:22 to 17:27, completely staggering its execution time with that of the "cleaning the corridor" task.

[0124] Specifically, as an example, the control platform continuously updates the environmental spatial features using LiDAR, depth camera, and robot self-positioning data. When it detects that mobile robot A is performing a "food delivery" task and is about to enter the corridor area, while mobile robot B is performing a "corridor cleaning" task and is currently inside the corridor, the system determines that there is a spatial conflict risk between the two entities in the pre-defined area of ​​the corridor. At this time, the system postpones the start time of the execution time window for mobile robot A's "food delivery" task from 17:22 to 17:27 (i.e., the time when mobile robot B is expected to complete the corridor cleaning plus a 2-minute safety buffer time). Subsequently, the system re-solves the pre-defined objective function based on the postponed start time of the execution time window to obtain an updated global scheduling scheme, in which the execution of the "food delivery" task is delayed, and the start time of the subsequent "post-meal cleaning" task that depends on this task is also delayed accordingly.

[0125] In practical applications, when multiple task nodes simultaneously involve spatial conflicts within the same preset area, the system prioritizes each task node according to its priority, ensuring that the execution time windows of high-priority tasks are not affected, while only postponing the execution time windows of low-priority tasks. This priority-based conflict resolution mechanism ensures that while eliminating spatial conflicts, it also maximizes the protection of the timeliness of critical tasks.

[0126] In this embodiment, by delaying the start time of the execution time window of the task node with the risk of conflict when multiple execution entities are detected to have a spatial conflict risk in a preset area and resolving the preset objective function, the scheduling scheme can actively avoid the spatiotemporal conflict between multiple execution entities and maintain the orderly operation of the multi-machine collaborative system while ensuring the safety of the operation.

[0127] In any of the above embodiments, optionally, during the execution of a task node by the executing entity, in response to the satisfaction of a preset risk condition, the global scheduling scheme is adjusted, including: S1601: Detect abnormal events during the execution of task nodes by the executing entity, and calculate a risk score based on the type and impact of the abnormal events; S1602: Determine the risk status corresponding to the abnormal event based on the risk score; S1603: Based on the risk status, determine the target response strategy from the preset multi-level response strategies, and adjust the global scheduling scheme based on the target response strategy.

[0128] Abnormal events refer to events detected during the execution of task nodes by the executing entity that deviate from the normal operating state, and are a prerequisite for triggering adjustments to the scheduling scheme. Specifically, these can include network communication anomalies, sensor status anomalies, mechanical execution anomalies, and environmental interaction anomalies. Examples include communication delays exceeding 500 milliseconds caused by brief fluctuations in wireless network signals, abnormal LiDAR data, and motor current exceeding rated values.

[0129] The risk score is a quantitative indicator calculated using a two-dimensional weighted model based on the type and impact of an anomaly, used to accurately distinguish the severity of anomalies. Anomaly types cover multiple categories, including network communication, sensor status, mechanical execution, and environmental interaction. Different types are assigned different basic risk weights; for example, safety anomalies involving physical contact have a significantly higher basic weight than communication anomalies that are purely data-related. The degree of impact is quantified by the deviation, duration, or scope of impact of real-time monitoring data.

[0130] The risk status is a level of anomaly severity determined by the numerical range of the risk score, and is used to map to the corresponding response strategy. Specifically, it can be a level status determined by comparing the risk score with a preset scoring threshold. For example, a risk score in the range of 0 to 30 is the first risk status, 30 to 70 is the second risk status, and 70 to 100 is the third risk status.

[0131] The pre-defined multi-level response strategy refers to a set of differentiated handling plans predefined for different risk states, with each risk state corresponding to a specific response strategy. Specifically, this can include different levels of handling measures such as partial task rearrangement, service degradation, and agency locking. For example, the first risk state corresponds to a partial task rearrangement strategy, the second risk state corresponds to a partial task rearrangement plus basic service maintenance strategy, and the third risk state corresponds to an agency locking strategy.

[0132] Among them, the target response strategy refers to the specific handling plan selected from the multi-level response strategies based on the current risk status, which is used to guide the system to adjust the global scheduling plan.

[0133] Specifically, as an example, during a mobile robot's "food delivery" task, the system detected a communication delay exceeding 500 milliseconds due to a brief fluctuation in the wireless network signal. The system calculated a risk score of 25 points based on the type of the abnormal event (network communication anomaly) and its impact (delay magnitude and duration). Based on this risk score, the system determined the risk state corresponding to the abnormal event to be the first risk state. Subsequently, the system selected a local task rescheduling strategy from its preset multi-level response strategies and adjusted the global scheduling scheme accordingly, rescheduling only the affected "food delivery" task nodes without affecting the normal execution of other task nodes.

[0134] In practical applications, risk scores can be calculated using a weighted formula: Risk Score = Type Weight × Impact Score. The type weight is preset based on the anomaly type (e.g., 0.3 for network communication anomalies and 0.8 for mechanical execution anomalies), while the impact score is dynamically calculated based on the deviation from real-time monitoring data. This dual-dimensional weighted model allows the risk score to simultaneously reflect both the inherent danger and the current severity of the anomaly.

[0135] In this embodiment, by detecting abnormal events and calculating risk scores based on their type and impact, and then determining the risk status based on the risk scores and matching the target response strategy from the multi-level response strategies, a refined hierarchical handling of abnormal events is achieved, maximizing the preservation of system service continuity and resource utilization while ensuring the bottom line of safety.

[0136] In any of the above embodiments, optionally, the risk state includes a first risk state, a second risk state, and a third risk state, wherein the degree of abnormality of the first risk state is lower than the degree of abnormality of the second risk state, and the degree of abnormality of the second risk state is lower than the degree of abnormality of the third risk state. S1603: Based on the risk status, determine the target response strategy from the preset multi-level response strategies, and adjust the global scheduling scheme based on the target response strategy, including: S1603a: When the risk status is the first risk status, perform local task rearrangement on the affected task nodes; S1603b: When the risk status is the second risk status, perform local task rearrangement on the affected task nodes and control the corresponding execution entity to maintain basic services; S1603c: When the risk status is the third risk status, lock the execution agency of the corresponding execution entity.

[0137] The three risk levels—first, second, and third—are arranged in ascending order of anomaly severity. The first risk level represents the lowest degree of anomaly, corresponding to momentary fluctuations in non-critical modules or temporary resource shortages. The second risk level represents a moderate degree of anomaly, corresponding to a significant decline in equipment performance or partial functional failure, but not yet posing a direct safety threat. The third risk level represents the highest degree of anomaly, corresponding to severe anomalies that could cause irreversible damage to personal safety, equipment, or the environment.

[0138] Local task rescheduling refers to reordering or reallocating only the task nodes directly affected by the abnormal event and their closely related downstream dependent task nodes, rather than triggering a full rescheduling. Specifically, this can involve dynamically migrating affected task nodes to alternative execution entities, adjusting execution parameters or timing, etc. For example, when a mobile robot responsible for food delivery triggers the first risk state due to abnormal LiDAR data, the system only marks the robot's "food delivery" task node as pending rescheduling, without affecting the food preparation task node that the robotic arm in the kitchen is currently performing.

[0139] Maintaining basic services refers to the capability to remove non-core functions and retain core value delivery when some equipment functions are unavailable or performance is limited, through an adaptive degradation strategy. Specifically, this could involve reducing movement speed, switching to a low-precision navigation mode, or simplifying service content. For example, the original service goal of "precise delivery to the table" could be downgraded to "delivery to the door of a designated area."

[0140] Locking the actuator refers to cutting off the actuator's power enable signal, applying the brake, or entering a hardware-level safe torque cancellation state. This ensures that even if there are logical errors in the upper-level control algorithm or sensor data drift, the actuator cannot produce any unexpected physical actions. Specifically, this can involve issuing a low-level locking command to the target actuator. For example, when the robotic arm triggers a third-risk state due to torque sensor failure, hardware-level locking can force the system to be placed in a "zero-energy" or "controlled stationary" state.

[0141] Specifically, as an example, when a mobile robot experiences a communication delay exceeding 500 milliseconds but not timeout due to a brief fluctuation in the wireless network signal, the system calculates a risk score of 25, classifying it as a first-risk state. It then performs a partial task reordering, pausing non-critical steps while preserving the task context, and automatically resuming execution once the network returns to normal. When a mobile robot triggers a second-risk state due to a positioning drift exceeding 0.2 meters, the system performs a partial task reordering for the affected task nodes, while simultaneously maintaining basic services for the robot. It automatically switches its navigation mode from high-precision semantic map matching to low-precision odometry plus visual obstacle avoidance mode, and limits its maximum speed to a safe threshold. When a collision sensor is triggered, the system calculates a risk score of 85, classifying it as a third-risk state, immediately locking all joint actuators of the robot and cutting off power output.

[0142] In practical applications, the thresholds for classifying the three risk states can be configured according to specific application scenarios. For example, in home-based elderly care scenarios, due to the involvement of the elderly's personal safety, the trigger threshold for the third risk state can be appropriately lowered to improve the sensitivity of safety protection; while in industrial warehousing scenarios, due to the relatively controllable environment, the thresholds for each level of risk state can be appropriately increased to reduce unnecessary service interruptions.

[0143] In this embodiment, by dividing the risk status into a first risk status, a second risk status, and a third risk status with increasing abnormality, and matching a differentiated target response strategy to each risk status, a refined hierarchical handling of abnormal events is achieved, maximizing the preservation of system service continuity and resource utilization while ensuring the bottom line of safety.

[0144] In any of the above embodiments, optionally, when the risk state is a first risk state, or when the risk state is a second risk state, the method further includes: S1603d: Update the risk score of the corresponding abnormal event based on the continuously acquired running status of the executing entity and environmental spatial characteristics; S1603e: When the risk score falls back to the preset recovery threshold range, cancel the currently executed target response strategy.

[0145] The "continued acquisition of the execution entity's operational status" refers to the real-time data of various performance indicators of the execution entity continuously collected by the control platform during the execution of the target response strategy. Specifically, this may include motor temperature trends, positioning accuracy stability, communication packet loss rate, sensor data quality, etc. For example, during the local task reordering execution corresponding to the first risk state, the system continuously monitors whether the communication latency returns to the normal threshold.

[0146] The continuously acquired environmental spatial features refer to the physical environmental status of the work area that the control platform continuously updates through multi-source sensors during the execution of the target response strategy. Specifically, this can include changes in personnel activity density, obstacle distribution, and area accessibility. For example, during the maintenance of basic services corresponding to the second risk state, the system continuously monitors whether the personnel density in the corridor area has decreased to a safe level.

[0147] Updating the risk score for corresponding abnormal events refers to recalculating the risk score of abnormal events based on the continuously acquired operational status of the executing entity and environmental spatial characteristics, in order to reflect the actual severity of the abnormal events at the current moment. Specifically, when the communication latency decreases from 500 milliseconds to 100 milliseconds, the risk score decreases from 25 points to 8 points; when the positioning drift returns from 0.2 meters to 0.05 meters, the risk score decreases from 55 points to 15 points.

[0148] The preset recovery threshold range refers to a pre-defined risk score interval. When the risk score falls back to this interval, it indicates that the severity of the abnormal event has been reduced to a level where normal operation can be restored. Specifically, it can be that the risk score is below a certain threshold (e.g., 10 points), or that the risk score is below a certain threshold for multiple consecutive sampling periods. For example, the preset recovery threshold range is a risk score below 10 points that lasts for more than 3 sampling periods.

[0149] Specifically, canceling the currently executing target response strategy means stopping the execution of the currently implemented response measures and restoring the original scheduling parameters of the affected task nodes. This can be done by canceling partial task rescheduling and restoring the original task allocation, or canceling basic service restrictions and restoring full functionality. For example, when communication latency returns to normal, the system cancels the partial task rescheduling strategy and restores the original execution time window and execution subject matching relationship of the affected task nodes.

[0150] Specifically, as an example, when a mobile robot triggers a first-risk state and performs a partial task rescheduling due to a brief fluctuation in the wireless network signal, the system continues to acquire the robot's communication status data. When it detects that the communication latency gradually decreases from 500 milliseconds to 80 milliseconds, and remains within 100 milliseconds for three consecutive sampling periods, the system updates the risk score of this abnormal event to 5 points, determining that the risk score has fallen back to the preset recovery threshold range. Subsequently, the system cancels the currently executed partial task rescheduling strategy, restores the original scheduling parameters of the affected task nodes, and restores the system to normal operation.

[0151] As another example, when a mobile robot triggers a second risk state due to positioning drift and performs local task rearrangement and basic service maintenance, the system continues to acquire the robot's positioning accuracy data and environmental spatial characteristics. When the positioning drift is detected to have returned from 0.2 meters to 0.03 meters, and the personnel density in the corridor area has decreased to a safe level, the system updates the risk score of this abnormal event to 8 points, determining that the risk score has fallen back to the preset recovery threshold range. Subsequently, the system cancels the currently executed target response strategy and restores the robot's full-function operation mode and original scheduling parameters.

[0152] In practical applications, Figure 4 This is a flowchart of a multi-level fault response provided in an embodiment of this application, which is described in detail below: S500: Normal operating status; S510: Resilience level (light); This state is typically triggered by temporary, self-healing minor anomalies that do not affect the normal operation of core functions. These mainly include temporary anomalies such as network fluctuations, communication delays, brief pauses in the execution entity, and occasional jumps in sensor data that do not affect the security of task execution.

[0153] S520: Functional level (medium); In this state, the anomaly level is moderate, triggered by anomalies that affect some system functions but do not endanger personnel or equipment safety. These mainly include anomalies that affect task execution quality but do not pose a safety threat, such as decreased performance of the executing entity, failure of non-critical sensors, or increased local environmental congestion.

[0154] S530: Security Level (Critical); This state is triggered by serious anomalies that may endanger the safety of personnel or equipment. These mainly include serious anomalies that could lead to safety accidents, such as collision risk, high equipment temperature, boundary crossing risk, critical component failure, and personnel entering the work area.

[0155] Figure 5 This is a flowchart of the tiered circuit breaker state transition provided in an embodiment of this application, which is described in detail below: S700: Normal operating status; In this state, fault detection can be continuously performed, and the circuit breaker process can be triggered by calculating the comprehensive risk score (RiskScore). When the comprehensive risk score reaches the trigger condition, it enters the elastic level circuit breaker state.

[0156] S710: Elastic level fuse; If the recovery decision passes in this state, the system returns to normal S700 operation. If performance continues to deteriorate, it will switch to S720; S720: Functional fuse; If the recovery decision passes in this state, the system returns to normal S700 operation. If a security risk is detected, proceed to S730; S730: Critical safety fuse. In this state, specifically after manual confirmation that the risk has been eliminated and the main sensor has completed reset calibration, the process can proceed to S740. S740: Restored to normal operating status.

[0157] In this embodiment, by continuously monitoring the operating status of the executing entity and the characteristics of the environmental space and updating the risk score when the risk status is in the first risk status or the second risk status, the target response strategy is automatically revoked when the risk score falls back to the preset recovery threshold range, thus achieving a smooth recovery from the abnormal state to the normal operating state and avoiding resource waste and service degradation caused by the continuous execution of the response strategy.

[0158] Figure 6 This is a flowchart of the dynamic scheduling engine method provided in an embodiment of this application, which is described in detail below: S800: Task input; In this step, environmental perception data and user task objectives may be received as the basic inputs for subsequent scheduling processes. Environmental perception data includes current environmental spatial characteristics (such as obstacle locations, regional congestion status, user status information, etc.), while user task objectives include the type of task to be executed, temporal dependencies, priority requirements, etc., providing complete initial conditions for subsequent task decomposition, robot matching, and scheduling optimization.

[0159] S810: Candidate robot screening; This step involves preliminary screening of robots capable of performing tasks. Specifically, it includes capability vector matching: matching the real-time multi-dimensional capability vectors of each access robot with the task characteristics (such as mobility, load capacity, environmental adaptability, etc.) of the user's task objectives to select robots whose capability dimensions meet the basic requirements of the task; and excluding devices with abnormal status: combining the robot's real-time status information to exclude abnormal devices that are faulty, offline, low in power, or performing high-priority tasks, retaining only robots with normal status and available resources as candidates to avoid ineffective scheduling.

[0160] S820: Time window optimization and allocation; This step is the core optimization step of the scheduling scheme. It performs global optimization based on a preset objective function. Specifically, it can include dynamic priority adjustment, which dynamically adjusts the priority parameters of task nodes based on task priority and current environmental state, and advances or prioritizes the execution time window of high-priority tasks. It also includes task conflict detection and balancing, which detects potential spatial and temporal conflict risks based on the time window of task nodes, robot position and path planning results, and balances conflicts by adjusting the task execution time window (postponing the start time of conflicting tasks) or replanning local paths, thereby reducing the scheduling failure rate.

[0161] S830: Command issuance; This step involves converting the scheduling scheme into executable instructions for the robot and distributing them through a unified protocol interface. Specifically, this includes ROS2 Topic publishing: for robot devices that support the ROS2 communication protocol, standardized control instructions are published through the ROS2 Topic to achieve real-time transmission of instructions; it also includes, for robot devices using proprietary protocols or specific SDKs, converting standardized control instructions into the device's proprietary protocol format through a protocol adapter, or calling the corresponding SDK interface to complete the instruction distribution.

[0162] S840: Anomaly detection and callback handling; This step involves dynamic monitoring and circuit breaker recovery during robot task execution. Specifically, it includes: handling safety violations; continuously monitoring the robot's execution and environmental status; triggering a tiered circuit breaker strategy when a safety violation event (such as collision risk, personnel intrusion, boundary crossing, etc.) is detected, and performing local task rescheduling, basic service maintenance, or safety locking operations based on the risk level; generating alternative tasks for equipment failures: when a robot equipment failure (such as power failure or communication interruption) prevents it from continuing to execute a task, an automatic alternative scheduling process is triggered, candidate robots are re-selected and alternative tasks are generated, and the global scheduling scheme is adjusted to ensure task continuity; and user interruption recovery control: when a user actively interrupts a task (such as pausing or canceling a task) or sends a control command, the system receives the user's command and adjusts the scheduling scheme, while recording the current execution status to support subsequent task recovery and achieve interactive control between the user and the system.

[0163] like Figure 7 As shown, this application embodiment provides a robot control device 200, the device 200 including: The first acquisition unit 202 is used to acquire the real-time status information of all execution entities connected to the robot control platform through a unified protocol interface in response to receiving the user's task target. The first processing unit 204 is used to determine the real-time multidimensional capability information corresponding to each execution entity based on the real-time status information. The second processing unit 206 is used to acquire the current environmental spatial features and decompose the user task objective into multiple task nodes and the dependencies between task nodes based on the current environmental spatial features. The third processing unit 208 is used to determine the task adaptability of the execution subject to each task node based on the real-time multi-dimensional capability information of the execution subject and the task characteristics of the task node. The fourth processing unit 210 is used to solve the preset objective function based on the matching relationship between task nodes and execution entities as the optimization objective, the task adaptability of task nodes, the priority of task nodes, and the conflict items between task nodes, and to obtain a global scheduling scheme under the preset constraints including the dependency relationship between task nodes. The fifth processing unit 212 is used to control the corresponding execution entity through a unified protocol interface based on a global scheduling scheme, so that the execution entity executes the corresponding task node; The sixth processing unit 214 is used to adjust the global scheduling scheme in response to the satisfaction of preset risk conditions during the execution of task nodes by the executing entity.

[0164] The robot control device 200 provided in this application embodiment obtains real-time status information of all executing entities through a unified protocol interface, breaking down information silos between cross-brand devices. Based on the real-time status information, it determines real-time multi-dimensional capability information, enabling dynamic quantitative evaluation of the capabilities of heterogeneous executing entities. Task decomposition based on environmental spatial characteristics ensures the scheduling scheme aligns with the actual operating environment. Through comprehensive optimization of task adaptability, priority, and conflict items, a global scheduling scheme that balances efficiency and safety is generated. During execution, the scheduling scheme is dynamically adjusted in response to risk conditions, ensuring the system's robustness and service continuity in complex dynamic environments. Thus, efficient, stable, and real-time collaborative control for multiple heterogeneous executing entities is achieved.

[0165] Figure 8 A schematic diagram of the hardware structure of the electronic device provided in an embodiment of this application is shown.

[0166] The electronic device may include a processor 601 and a memory 602 storing computer program instructions.

[0167] Specifically, the processor 601 may include a central processing unit (CPU), an application-specific integrated circuit (ASIC), or one or more integrated circuits that can be configured to implement the embodiments of this application.

[0168] Memory 602 may include mass storage for data or instructions. For example, and not limitingly, memory 602 may include a hard disk drive (HDD), floppy disk drive, flash memory, optical disk, magneto-optical disk, magnetic tape, or Universal Serial Bus (USB) drive, or a combination of two or more of these. Where appropriate, memory 602 may include removable or non-removable (or fixed) media. Where appropriate, memory 602 may be internal or external to the integrated gateway. In a particular embodiment, memory 602 is non-volatile solid-state memory.

[0169] Memory may include read-only memory (ROM), random access memory (RAM), disk storage media devices, optical storage media devices, flash memory devices, and electrical, optical, or other physical / tangible memory storage devices. Therefore, typically, memory includes one or more tangible (non-transitory) computer-readable storage media (e.g., memory devices) encoded with software including computer-executable instructions, and when the software is executed (e.g., by one or more processors), it is operable to perform the operations described with reference to the methods according to one aspect of this disclosure.

[0170] The processor 601 reads and executes computer program instructions stored in the memory 602 to implement any of the robot control methods in the above embodiments.

[0171] In one example, the electronic device may also include a communication interface 603 and a bus 610. For example, Figure 8 As shown, the processor 601, memory 602, and communication interface 603 are connected through bus 610 and complete communication with each other.

[0172] The communication interface 603 is mainly used to realize communication between various modules, devices, units and / or equipment in the embodiments of this application.

[0173] Bus 610 includes hardware, software, or both, that couples components of an electronic device together. For example, and not limitingly, the bus may include an Accelerated Graphics Port (AGP) or other graphics bus, an Enhanced Industry Standard Architecture (EISA) bus, a Front Side Bus (FSB), HyperTransport (HT) interconnect, an Industry Standard Architecture (ISA) bus, an Infinite Bandwidth Interconnect, a Low Pin Count (LPC) bus, a memory bus, a Microchannel Architecture (MCA) bus, a Peripheral Component Interconnect (PCI) bus, a PCI-Express (PCI-X) bus, a Serial Advanced Technology Attachment (SATA) bus, a Video Electronics Standards Association Local (VLB) bus, or other suitable buses, or combinations of two or more of these. Where appropriate, bus 610 may include one or more buses. Although specific buses are described and illustrated in embodiments of this application, this application contemplates any suitable bus or interconnect.

[0174] Furthermore, in conjunction with the robot control methods in the above embodiments, this application embodiment can provide a computer storage medium for implementation. The computer storage medium stores computer program instructions; when these computer program instructions are executed by a processor, they implement any of the robot control methods in the above embodiments.

[0175] This application also provides a computer program product, including a computer program that, when executed by a processor, implements any of the robot control methods described in the above embodiments.

[0176] It should be clarified that this application is not limited to the specific configurations and processes described above and shown in the figures. For the sake of brevity, detailed descriptions of known methods are omitted here. In the above embodiments, several specific steps are described and shown as examples. However, the method process of this application is not limited to the specific steps described and shown. Those skilled in the art can make various changes, modifications, and additions, or change the order of steps, after understanding the spirit of this application.

[0177] The functional blocks shown in the above block diagram can be implemented as hardware, software, firmware, or a combination thereof. When implemented in hardware, they can be, for example, electronic circuits, application-specific integrated circuits (ASICs), appropriate firmware, plug-ins, function cards, etc. When implemented in software, the elements of this application are programs or code segments used to perform the required tasks. Programs or code segments can be stored on a machine-readable medium or transmitted over a transmission medium or communication link via data signals carried on a carrier wave. "Machine-readable medium" can include any medium capable of storing or transmitting information. Examples of machine-readable media include electronic circuits, semiconductor memory devices, ROM, flash memory, erasable ROM (EROM), floppy disks, CD-ROMs, optical disks, hard disks, fiber optic media, radio frequency (RF) links, etc. Code segments can be downloaded via computer networks such as the Internet, intranets, etc.

[0178] It should also be noted that the exemplary embodiments mentioned in this application describe methods or systems based on a series of steps or apparatus. However, this application is not limited to the order of the above steps; that is, the steps can be performed in the order mentioned in the embodiments, or in a different order, or several steps can be performed simultaneously.

[0179] The above are merely specific embodiments of this application. Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, modules, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here. It should be understood that the protection scope of this application is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in this application, and these modifications or substitutions should all be covered within the protection scope of this application.

Claims

1. A robot control method, applied to a robot control platform, characterized in that, The robot control platform includes a unified protocol interface, and the method includes: In response to receiving the user's task objective, the system obtains the real-time status information of all execution entities connected to the robot control platform through the unified protocol interface; Based on the real-time status information, determine the real-time multidimensional capability information corresponding to each execution entity; Obtain the current environmental spatial features, and based on the current environmental spatial features, decompose the user task objective into multiple task nodes and the dependencies between the task nodes; Based on the real-time multidimensional capability information of the execution entity and the task characteristics of the task nodes, the task adaptability of the execution entity to each task node is determined. Taking the matching relationship between the task node and the execution entity as the optimization objective, the preset objective function is solved based on the task adaptability of the task node, the priority of the task node, and the conflict items between the task nodes. Under the condition of satisfying the preset constraints including the dependency relationship between the task nodes, a global scheduling scheme is obtained. Based on the global scheduling scheme, the corresponding execution entity is controlled through the unified protocol interface, enabling the execution entity to execute the corresponding task node; During the execution of the task node by the executing entity, the global scheduling scheme is adjusted in response to the satisfaction of preset risk conditions.

2. The method according to claim 1, characterized in that, The unified protocol interface includes multiple protocol adapters; The step of obtaining real-time status information of all execution entities connected to the robot control platform through the unified protocol interface includes: Through the corresponding protocol adapter, the system receives uplink messages in the private communication protocol format sent by each execution entity, converts the uplink messages in the private communication protocol format into platform standard messages, and obtains the real-time status information. The control of the corresponding execution entity through the unified protocol interface includes: Through the corresponding protocol adapter, the platform standard message corresponding to the global scheduling scheme is converted into a downlink message in the private communication protocol format of the corresponding execution subject, and the downlink message in the private communication protocol format is sent to the corresponding execution subject.

3. The method according to claim 1, characterized in that, The step of converting uplink messages in the private communication protocol format into platform standard messages, and the step of converting platform standard messages corresponding to the global scheduling scheme into downlink messages in the private communication protocol format of the corresponding execution entity, include: When converting the uplink message in the private communication protocol format into a platform standard message, the parameters in the uplink message in the private communication protocol format are converted to uplink dimensions according to the dimension mapping relationship between the private communication protocol and the platform standard message, so as to obtain the uplink message parameters after dimension conversion. Based on the correspondence between the data structure of the private communication protocol and the fields of the platform standard message, the uplink field mapping is performed on the uplink message parameters after the unit conversion to generate the platform standard message; When converting the platform standard message corresponding to the global scheduling scheme into a downlink message in the private communication protocol format of the corresponding execution entity, the parameters in the platform standard message are converted into downlink dimensions according to the dimension mapping relationship between the platform standard message and the private communication protocol to obtain the downlink message parameters after dimension conversion. Based on the correspondence between the fields of the platform standard message and the data structure of the private communication protocol, the downlink field mapping is performed on the downlink message parameters after the unit conversion to generate the downlink message in the format of the private communication protocol.

4. The method according to claim 1, characterized in that, The real-time multidimensional capability information includes multiple capability dimensions, including at least two of the following: motion capability dimension, operational precision dimension, battery life dimension, load capacity dimension, and environmental adaptability dimension. The determination of real-time multi-dimensional capability information for each execution entity based on the real-time status information includes: Extract the original metric value corresponding to each capability dimension from the real-time status information; Based on the normalization parameters of the corresponding capability dimension, the original metric values ​​are normalized to obtain the normalized score for each capability dimension. Based on the normalized score of each capability dimension, a capability vector is constructed for each executing entity, and the capability vector is used as the real-time multidimensional capability information.

5. The method according to claim 4, characterized in that, The normalization parameters include upper and lower bound reference values ​​for each capability dimension, and the method further includes: Based on the sliding window observations, the upper bound reference value and the lower bound reference value are updated to obtain the updated upper bound reference value and lower bound reference value. Based on the updated upper and lower reference values, the normalized score of the capability dimension is recalculated. The capability vector is updated based on the normalized score of the recalculated capability dimension.

6. The method according to claim 1, characterized in that, After controlling the corresponding execution entity through the unified protocol interface based on the global scheduling scheme, and causing the execution entity to execute the corresponding task node, the process further includes: Based on the updated environmental spatial characteristics, adjust the time window parameters of at least one task node in the global scheduling scheme; Based on the adjusted time window parameters, the preset objective function is re-solved to obtain the updated global scheduling scheme.

7. The method according to claim 6, characterized in that, The step of adjusting the time window parameters of at least one task node in the global scheduling scheme based on the updated environmental spatial characteristics includes: When the preset work area is detected to be in a peak period of personnel activity based on the updated environmental spatial characteristics, the basic execution time window length of the task node associated with the preset work area is extended.

8. The method according to claim 6, characterized in that, The step of adjusting the time window parameters of at least one task node in the global scheduling scheme based on the updated environmental spatial characteristics includes: When a user is detected to be in a preset special state based on the updated environmental spatial characteristics, the execution priority of the preset special task node is increased, and the start time of the execution time window of the preset special task node is advanced.

9. The method according to claim 6, characterized in that, The step of adjusting the time window parameters of at least one task node in the global scheduling scheme based on the updated environmental spatial characteristics includes: When multiple execution entities are detected to have spatial conflict risks in a preset area based on the updated environmental spatial characteristics, the start time of the execution time window of at least one task node with conflict risks is postponed.

10. The method according to claim 1, characterized in that, During the execution of the task node by the executing entity, in response to the satisfaction of preset risk conditions, the global scheduling scheme is adjusted, including: During the execution of the task node by the executing entity, abnormal events are detected, and a risk score is calculated based on the type and impact of the abnormal events. The risk status corresponding to the abnormal event is determined based on the risk score. Based on the risk status, a target response strategy is determined from the preset multi-level response strategies, and the global scheduling scheme is adjusted based on the target response strategy.

11. The method according to claim 10, characterized in that, The risk status includes a first risk status, a second risk status, and a third risk status. The degree of abnormality of the first risk status is lower than that of the second risk status, and the degree of abnormality of the second risk status is lower than that of the third risk status. The step of determining a target response strategy from a preset multi-level response strategy based on the risk status, and adjusting the global scheduling scheme based on the target response strategy, includes: When the risk status is the first risk status, a local task rearrangement is performed on the affected task nodes; When the risk status is the second risk status, perform local task rearrangement on the affected task nodes and control the corresponding execution entity to maintain basic services; When the risk status is the third risk status, the execution mechanism of the corresponding execution entity is locked.

12. The method according to claim 11, characterized in that, When the risk status is the first risk status, or when the risk status is the second risk status, the method further includes: Based on the continuously acquired operational status of the executing entity and environmental spatial characteristics, update the risk score of the corresponding abnormal event; When the risk score falls back to the preset recovery threshold range, the currently executed target response strategy is revoked.

13. A robot control device, characterized in that, The device includes: The first acquisition unit is used to acquire, in response to receiving the user's task objective, the real-time status information of all execution entities connected to the robot control platform through the unified protocol interface; The first processing unit is used to determine the real-time multidimensional capability information corresponding to each execution entity based on the real-time status information. The second processing unit is used to acquire the current environmental spatial features and decompose the user task objective into multiple task nodes and the dependencies between the task nodes based on the current environmental spatial features. The third processing unit is used to determine the task adaptability of the execution subject to each task node based on the real-time multi-dimensional capability information of the execution subject and the task characteristics of the task node. The fourth processing unit is used to optimize the matching relationship between the task node and the execution entity, and solve the preset objective function based on the task adaptability of the task node, the priority of the task node, and the conflict items between the task nodes. Under the condition of satisfying the preset constraints including the dependency relationship between the task nodes, a global scheduling scheme is obtained. The fifth processing unit is used to control the corresponding execution entity through the unified protocol interface based on the global scheduling scheme, so that the execution entity executes the corresponding task node; The sixth processing unit is used to adjust the global scheduling scheme in response to the satisfaction of preset risk conditions during the execution of the task node by the executing entity.

14. An electronic device, characterized in that, The electronic device includes: a processor and a memory storing computer program instructions; When the processor executes the computer program instructions, it implements the robot control method as described in any one of claims 1 to 12.

15. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer program instructions that, when executed by a processor, implement the robot control method as described in any one of claims 1 to 12.

16. A computer program product, characterized in that, When the instructions in the computer program product are executed by the processor of the electronic device, the electronic device causes the electronic device to perform the robot control method as described in any one of claims 1 to 12.

Images