Robot complex task dynamic arrangement method

Abstract

The invention provides a robot complex task dynamic arrangement method, belongs to the field of robot task arrangement, and is used for solving the problems of weak dynamic environment adaptation capability, poor multi-robot collaboration and insufficient risk pre-judgment of traditional static task arrangement in related technologies. According to the method, after double matrixes are constructed and fused into tensors, low-rank decomposition is performed, a threshold value is dynamically generated and optimized, task priorities are calculated through multiple factors, resources are allocated in combination with resource constraint and federated learning, synchronous cross-scene adaptation during execution, risk pre-planning and visual feedback optimization are performed, and dynamic adaptation, efficient collaboration and stable execution of complex tasks are achieved.

Description

Technical Field

[0001] This application relates to the field of robot task orchestration, and more particularly to a method for dynamic orchestration of complex robot tasks. Background Technology

[0002] With the development of industrial automation and service industry intelligence, the demand for robots to perform multiple tasks in complex scenarios is increasing, such as assembly on flexible production lines and sorting and distribution in smart logistics parks. Robot task orchestration, as a core technology for coordinating robot actions and optimizing resource utilization, directly affects task execution efficiency and system adaptability.

[0003] Currently, the programming of complex tasks for robots mostly adopts traditional preset procedures or fixed instruction sets. That is, fixed execution steps and resources are assigned to the robot according to pre-set task logic and environmental parameters. This method can achieve basic functions in stable environments and simple tasks, but in dynamic and complex scenarios, it still relies on manually preset rules to adjust the process.

[0004] However, traditional methods have significant drawbacks: First, they are difficult to adapt to dynamic environments and changing task requirements. When encountering abnormal sensor data, changes in task priorities, or resource conflicts, they cannot adjust the task flow in real time, which can easily lead to low execution efficiency or task failure. Second, they lack in-depth quantitative mining of the correlation between state, event, and task, making it impossible to accurately assess the co-occurrence correlation between state and event and the impact of events on state, resulting in unreasonable resource allocation and poor task coordination. Third, they have weak cross-scenario adaptability. In new scenarios, the orchestration model needs to be rebuilt, and there is a lack of feedback optimization mechanism for execution results, making it difficult to continuously improve system performance. These drawbacks directly limit the application effect of robots in complex dynamic scenarios. Summary of the Invention

[0005] This application provides a method for dynamic orchestration of complex tasks for robots, which can achieve dynamic and efficient orchestration of complex tasks through quantitative correlation modeling, dynamic threshold adjustment and end-to-end collaboration, thereby improving the system's adaptability and execution efficiency in dynamic environments.

[0006] This application provides a method for dynamically orchestrating complex tasks for robots. It includes: a data acquisition step, collecting robot operating state data and event data; an association modeling step, constructing a state-event association matrix and an event-state influence matrix, wherein the state-event association matrix quantifies the co-occurrence association strength of states and events by statistically analyzing the number of times a specific state and a specific event occur simultaneously, the total number of times the state occurs alone, and the total number of times the event occurs alone, using an intersection ratio correction method; the event-state influence matrix quantifies the influence of events on states by calculating the change magnitude of a specific state after a specific event is triggered, and normalizing this change magnitude with the maximum change magnitude of all events on that state; and a threshold adjustment step, dynamically generating clustering thresholds based on the state-event association matrix and the event-state influence matrix. The process includes a filtering threshold, where the clustering threshold is determined by a base threshold plus the maximum correlation weight between the event and all states, and the filtering threshold is determined by a minimum threshold plus the sum of the influence of all events on the state; a task priority calculation step, which calculates the task priority using a weighted method by combining the time sensitivity coefficient corresponding to the task deadline, the proportion of task resource requirements, the task urgency coefficient, and the sum of the products of the correlation between the task's corresponding state and the effective events and the influence of the effective events on the state; a resource allocation step, which allocates resources based on the task priority and resource constraints, wherein the resource constraint is that the total resource consumption of all tasks in the aggregated state does not exceed the total system resources; and a process execution step, which executes the task process according to the task priority and resource allocation results.

[0007] By adopting the above technical solutions, real-time data acquisition, quantitative correlation modeling, dynamic threshold adjustment, multi-factor priority calculation and resource constraint allocation are combined to break through the limitations of traditional preset processes, realize the accurate quantification of state-event correlation and dynamic adjustment of task processes, effectively solve the problems of poor task adaptability and low execution efficiency in dynamic environments, and improve the intelligence and accuracy of robot complex task orchestration.

[0008] Furthermore, the association modeling step also includes fusing the state-event association matrix and the event-state influence matrix into a state-event-time three-dimensional tensor. The value of any "state-event-time" element in the three-dimensional tensor is calculated by the geometric mean of the association degree between the state and the event and the influence degree of the event on the state, so as to uniformly quantify the dynamic association relationship between the state, event and time.

[0009] By adopting the above technical solution, the originally independent correlation matrix and influence matrix are integrated into a unified spatiotemporal correlation carrier, eliminating the consistency conflict of independent modeling of the two matrices, while retaining the dynamic correlation characteristics of state, event and time, providing a more comprehensive correlation basis for subsequent arrangement and optimization.

[0010] Furthermore, the association modeling step also includes using a low-rank decomposition algorithm to decompose the three-dimensional tensor into a product of a state core tensor, an association core tensor, and a time core tensor, wherein the state core tensor represents the low-dimensional features of the state dimension, the association core tensor represents the low-dimensional features of the interaction between the state and the event, and the time core tensor represents the low-dimensional features of the time dimension, so as to reduce the computational complexity of the three-dimensional tensor from a third-order complexity of the number of states, the number of events, and the number of time steps to a linear complexity.

[0011] By adopting the above technical solutions, the computational complexity in high-dimensional scenarios is significantly reduced, computational delays caused by excessively high state and event dimensions are avoided, real-time performance in large-scale robot clusters or highly dynamic scenarios is ensured, and support is provided for the efficient operation of the system.

[0012] Furthermore, the association modeling step also includes dynamically adjusting the rank parameter of the low-rank decomposition based on task priority. For high-priority tasks, the rank parameter is increased to increase the feature dimension of the core tensor and retain detailed information about state-event associations. For low-priority tasks, the rank parameter is decreased to reduce the feature dimension of the core tensor and improve computational efficiency within an acceptable range of information loss.

[0013] By adopting the above technical solution, the rank parameter is accurately matched with the task requirements, which not only ensures the integrity of the associated information of high-priority tasks (such as high-precision assembly) but also improves the computational efficiency of low-priority tasks (such as routine inspection), thus balancing the relationship between accuracy and efficiency.

[0014] Furthermore, the threshold adjustment step also includes autonomously optimizing the threshold coefficient through an intelligent optimization algorithm. The intelligent optimization algorithm constructs a weighted summation objective function with the goals of task completion rate, resource utilization rate, and information loss rate. By iteratively adjusting the threshold coefficient, the objective function value is maximized, thereby achieving a multi-objective balance between task efficiency, resource utilization, and information retention.

[0015] By adopting the above technical solution, the traditional method of manually setting threshold coefficients can be replaced, enabling autonomous optimization of threshold coefficients, avoiding deviations caused by reliance on experience, balancing multiple objective requirements, and improving the rationality of threshold adjustment and overall system performance.

[0016] Furthermore, the threshold adjustment step also includes constructing a multi-robot interaction graph, with each robot as a graph node. The graph node features include the robot's current threshold coefficient, task list, and fault risk. The similarity of the node features of two robots is calculated using a graph attention mechanism, and the similarity is normalized to obtain the task dependency between the two robots. Based on the task dependency, the threshold conflict correction strategy is dynamically adjusted, and a smaller conflict correction factor is used for robots with high task dependency.

[0017] By adopting the above technical solutions, the task dependencies between multiple robots can be accurately captured, avoiding task coordination disruptions caused by threshold conflict correction, improving threshold adjustment coordination in heterogeneous multi-robot scenarios, and reducing resource competition and task delays.

[0018] Furthermore, it also includes a cross-scene adaptation step, which includes: extracting visual image features, laser point cloud features, and task text features of the new scene; using the task text features as anchors, calculating the similarity between the visual image features, laser point cloud features, and task text features, and normalizing the similarity to obtain the attention weights of each modality feature; aligning multimodal features based on the attention weights to generate a latent representation of the scene; calculating the similarity between the latent representation of the scene and the latent representation of historical scenes, and weighted transferring the association model parameters of historical scenes based on the similarity.

[0019] By adopting the above technical solutions, we can uncover the implicit relationships between scenarios, solve the "cold start" problem of new scenarios, reduce the workload and data dependence of model building in new scenarios, and improve the efficiency and accuracy of cross-scenario adaptation.

[0020] Furthermore, it also includes a pre-planning step, which includes: extracting historical information gain sequences, state-event correlation trends, and task complexity variation characteristics; constructing a time-series prediction model; separating the long-term trend characteristics and periodic characteristics of information gain; focusing on key historical time steps through an attention mechanism to predict the information gain of future time steps; collecting robot motor current, temperature, and sensor signal strength characteristics; assigning high weights to key fault characteristics through an attention mechanism; using an encoder-decoder model to calculate the error between input features and reconstructed features, and the deviation between encoded latent features and normal sample latent features; and weighting the error and deviation to map them to the robot fault probability; when the predicted information gain is lower than a set threshold or the fault probability is higher than a set threshold, generating an alternative process containing a transition task, wherein the similarity of the state characteristics of the transition task with the currently executed task and the target task is not lower than a set threshold.

[0021] By adopting the above technical solutions, we can achieve early warning and pre-planning for the dual risks of "insufficient information + failure risk", avoid task interruption, and at the same time ensure smooth process connection through transition tasks, thereby improving the stability and robustness of task execution.

[0022] Furthermore, the resource allocation step also includes aggregating the local resource parameters of multiple robots using a federated learning framework. During the aggregation process, extreme parameters that exceed the mean of the local parameters by ±2 standard deviations are first removed, and then the remaining parameters are weighted and averaged using the robot task completion rate as the weight. Before uploading the local parameters, Laplace noise is added to the parameters, and the noise scale is determined by the maximum variation range of the parameters, the privacy protection level, and the number of robots.

[0023] By adopting the above technical solutions, the robustness of resource aggregation in heterogeneous data scenarios is improved, interference from extreme parameters is avoided, and data privacy and security are guaranteed, which complies with the data security specifications for industrial scenarios and achieves the dual goal of "robust aggregation + privacy protection".

[0024] Furthermore, it also includes a visualization feedback step, mapping task nodes, event nodes, and resource nodes into a dynamic flowchart. The task nodes are labeled based on the task priority calculated in claim 1, the proportion of task resource requirements to the total system resources, and the product of the correlation influence between the corresponding task state and the effective event. The event nodes are labeled with the degree of influence of the event on the state. The proportion of task delay time to the total task duration is used as the increment of the dynamic rank adjustment error threshold in the correlation modeling step, and the proportion of task delay time to the deadline is used as the deduction item of the intelligent optimization algorithm reward function in the threshold adjustment step, so as to realize the feedback optimization of the model parameters by the execution result.

[0025] By adopting the above technical solutions, the task process can be visualized and monitored, facilitating manual intervention. At the same time, a closed loop of "execution-feedback-optimization" can be constructed to continuously iterate and optimize the correlation modeling and threshold adjustment parameters, thereby improving the long-term operating performance of the system.

[0026] In summary, this application has at least the following beneficial effects:

[0027] 1. A method for dynamic orchestration of complex robot tasks with end-to-end collaboration is provided to achieve efficient task orchestration in dynamic environments;

[0028] 2. By using tensor-based correlation modeling and dynamic rank adjustment, a balance is struck between the accuracy of correlation information and computational efficiency;

[0029] 3. Improve multi-robot collaboration and data privacy and security through multi-agent threshold optimization and federated resource aggregation;

[0030] 4. Enhance system adaptability and task execution robustness through cross-scenario migration and dual-risk pre-planning.

[0031] It should be understood that the description in the Summary Section is not intended to limit the key or essential features of the embodiments of this application, nor is it intended to restrict the scope of this application. Other features of this application will become readily apparent from the following description. Attached Figure Description

[0032] The above and other features, advantages, and aspects of the embodiments of this application will become more apparent from the accompanying drawings and the following detailed description. In the drawings, the same or similar reference numerals denote the same or similar elements, wherein:

[0033] Figure 1A schematic diagram of an exemplary operating environment in which embodiments of this application can be implemented is shown.

[0034] Figure 2 A flowchart of a method for dynamically arranging complex tasks for robots, according to an embodiment of this application, is shown. Detailed Implementation

[0035] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0036] Furthermore, the term "and / or" in this article is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, or B existing alone. Additionally, the character " / " in this article generally indicates that the preceding and following related objects have an "or" relationship.

[0037] This application provides a method for dynamic orchestration of complex robot tasks, which can quantify state-event association, dynamically adjust thresholds and task priorities, adapt to cross-scene and multi-robot collaboration, and can also pre-control information shortage and failure risk, effectively improving task execution efficiency and system robustness and adaptability in dynamic environments.

[0038] Figure 1 A schematic diagram of an exemplary operating environment in which embodiments of this application can be implemented is shown.

[0039] Reference Figure 1 The operating environment includes a hardware system that supports the entire chain execution of "a method for dynamic orchestration of complex robot tasks". This system takes "data acquisition-computation processing-collaborative communication-task execution-visual monitoring" as its core logic and covers five major components: perception layer devices, computing layer devices, communication layer devices, execution layer devices, and visualization and monitoring layer devices. Each part forms a closed loop through data transmission and instruction interaction, and together they meet the hardware requirements of the core functions of the method, such as dynamic association modeling, multi-robot collaboration, cross-scene migration, and dual-risk pre-planning.

[0040] The perception layer devices, serving as the input source for data acquisition, are responsible for acquiring robot status, environmental information, and multimodal scene features. These include robot body sensors, environmental perception devices, and multimodal data acquisition devices. Robot body sensors (such as GPS, motor current sensors, and load sensors) are integrated into each robot, collecting real-time robot position, operating status, and task execution progress. Environmental perception devices (such as distributed LiDAR and visual cameras) are deployed within the scene, collecting data on global obstacle distribution and environmental anomalies. Multimodal data acquisition devices (such as high-definition industrial cameras, 3D laser scanners, and text command receiving modules) are used to extract multimodal features such as visual images, laser point clouds, and task text. The perception layer devices transmit the collected raw data to the computing layer devices via communication links, providing data support for subsequent correlation modeling (constructing state-event correlation matrices and event-state influence matrices), fault prediction (extracting fault features such as motor current and temperature), and cross-scene adaptation (multimodal feature transfer).

[0041] The computing layer devices serve as the central hub for algorithm execution, employing a layered architecture of "local computing + cloud / edge computing," comprising robot-local computing units, cloud / edge computing nodes, and storage devices. Robot-local computing units (such as ARM Cortex-A embedded processors and FPGAs) are integrated into the robot, handling low-latency tasks (such as preliminary calculation of local dynamic thresholds and real-time action command parsing). Cloud / edge computing nodes deploy high-performance computing clusters (including multi-core CPUs and GPU / TPU accelerator cards) to handle core algorithm computations, including large-scale tensor modeling (tensor train decomposition, dynamic rank adjustment), multi-objective optimization (multi-objective deep deterministic policy gradient), and long-term prediction (temporal fusion Transformer prediction information gain). They also act as federated learning servers, performing robust aggregation of multi-robot local parameters and differential privacy protection (adding Laplace noise). Storage devices (such as SSD arrays and distributed file systems) store historical scene data and model parameters (state-event association matrix, tensor decomposition core tensors), supporting contrastive learning and variational Bayesian inference in cross-scene migration. The computing layer device receives data transmitted from the perception layer, processes it with an algorithm to generate orchestration instructions, and then sends them down to the execution layer device through the communication layer.

[0042] Communication layer devices, acting as data links for multi-entity collaboration, ensure low-latency, highly reliable data transmission. These include industrial-grade communication networks, edge communication gateways, and secure communication modules. Industrial-grade communication networks (such as Profinet industrial Ethernet and 5G wireless networks) enable millisecond-level data transmission between robots and the cloud / edge, and between robots themselves, supporting real-time threshold conflict correction and global orchestration command issuance. Edge communication gateways, deployed in large-scale robot clusters, perform local data preprocessing (filtering redundant data) and traffic scheduling, reducing cloud communication pressure and ensuring sparse uploading of local parameters in federated learning. Secure communication modules (such as SSL / TLS encryption protocols and national cryptographic algorithms) encrypt transmitted data, protecting the privacy and security of federated learning parameters and sensitive task data (such as process paths). Communication layer devices connect the perception layer, computing layer, execution layer, and visualization and monitoring layer, achieving secure and efficient transmission of data and commands across the entire chain.

[0043] The execution layer devices, serving as the physical carriers for task implementation, include heterogeneous robot bodies and auxiliary execution devices. The heterogeneous robot bodies are configured according to scenario requirements (e.g., assembly robots, logistics robots, sorting robots). Each robot supports real-time reception of orchestration instructions (task execution order, resource allocation results) from the computing layer and dynamically adjusts its actions (e.g., switching tasks based on pre-planned alternative processes, rapid obstacle avoidance in case of failure). Auxiliary execution devices (e.g., charging stations, material storage platforms, high-precision positioning landmarks) are deployed within the scenario to provide robots with energy replenishment, material storage, and position correction, supporting smooth task transitions (e.g., transitional task execution in fault pre-planning). The execution layer devices receive orchestration instructions transmitted from the communication layer, complete specific task execution, and feed back the execution results (task completion rate, delay time) to the computing and visualization layers via the communication layer.

[0044] The visualization and monitoring layer devices serve as feedback terminals for human-computer interaction and closed-loop optimization, including industrial monitoring terminals and data feedback interface devices. Industrial monitoring terminals (such as industrial displays and industrial PCs) display dynamic BPMN flowcharts in real time, annotating task priorities, fault risks, resource requirements, and other information, supporting manual viewing and intervention. Data feedback interface devices convert task execution results (such as the percentage of delay time) into feedback signals, which are transmitted through the communication layer to the computing layer, driving closed-loop optimization through correlation modeling (dynamic rank adjustment error threshold correction) and threshold adjustment (multi-objective optimization reward function deduction).

[0045] The above five categories of equipment form a complete hardware support system through logical collaboration: "the perception layer collects data → the communication layer transmits data → the computing layer processes data and generates instructions → the communication layer issues instructions → the execution layer executes tasks → the execution results are fed back to the computing layer and the visualization layer through the communication layer". This ensures that "a method for dynamic arrangement of complex robot tasks" can be efficiently implemented in complex conditions such as dynamic environments, heterogeneous multi-robot systems, and cross-scenario applications.

[0046] This application discloses a method for dynamically arranging complex tasks for robots.

[0047] Figure 2 A flowchart of a method for dynamically arranging complex tasks for robots, according to an embodiment of this application, is shown.

[0048] Reference Figure 2 The method specifically includes the following steps:

[0049] S1: Data acquisition step, collecting robot operating status data and event data.

[0050] In this step, only the operation of collecting the robot's operating status data and event data needs to be performed to provide raw data input for subsequent steps.

[0051] Specifically, the operational status data includes the robot's real-time position data. Remaining battery power data Load rate data and motor temperature data Data is collected via GPS sensors (or laser SLAM sensors), battery sensors, weight sensors, and temperature sensors mounted on the robot. Event data specifically includes "obstacle detection events," "task completion events," and "low battery warning events," triggered by sensor data. To ensure data accuracy, noise modeling and preliminary preprocessing are required for the collected raw state data. The mathematical model for the raw state data is as follows:

[0052]

[0053] Where, The location data (in meters) was collected at time t. The position of the robot at time t is the actual position of the robot (obtained by calibration using high-precision calibration equipment). The Gaussian noise of the position sensor satisfies The noise variance (its value is determined by the GPS sensor model, such as industrial-grade GPS). ); The remaining battery power data (in %) was collected at time t. Let t be the robot's actual remaining battery power. For the noise of the power sensor, meet (Determined by the accuracy parameters of the power sensor). The data acquisition frequency is set to... (That is, data is collected every 0.1 seconds) to meet the sampling requirements of the "state-event-time" three-dimensional tensor time dimension in subsequent association modeling steps. For noisy raw data, a moving average filter is used for preprocessing. Taking location data as an example, the filtering formula is:

[0054]

[0055] Where, This is the filtered location data. The sliding window size (i.e., taking the 5 most recent data collections). This processing method, which takes a certain number of data collection cycles, can effectively reduce the interference of noise on subsequent correlation calculations.

[0056] The triggering of event data requires the fulfillment of preset conditions. Taking the "obstacle detection event" as an example, when the LiDAR collects obstacle distance data... When, the event is triggered, where The obstacle distance threshold is set according to the robot's movement safety requirements; the trigger condition for the "low battery warning event" is... (Set according to the robot's battery life characteristics). The event type (type) must be recorded after the event is triggered. Trigger time and key parameters (e.g., distance to obstacles) Remaining battery power ), forming event triples This provides the raw data for the event dimension in the subsequent association modeling steps to construct the state-event association matrix; the preprocessed running state data serves as the raw data for the state dimension, jointly supporting the construction of the subsequent state-event association matrix and event-state influence matrix.

[0057] S2: Association modeling steps, constructing the state-event association matrix and the event-state influence matrix.

[0058] The state-event association matrix quantifies the co-occurrence association strength of states and events by statistically analyzing the number of times a specific state and a specific event occur simultaneously, the total number of times the state occurs alone, and the total number of times the event occurs alone, and uses an intersection ratio correction method. The event-state influence matrix quantifies the influence of events on states by calculating the change magnitude of a specific state after a specific event is triggered, and normalizing this change magnitude with the maximum change magnitude of all events on the state.

[0059] In constructing the state-event association matrix (denoted as...) Matrix, dimension ,in Number of state types When the number of event types is [number], statistics must first be performed based on the preprocessed runtime status data and event data from step S1: Let [example data]. For state (e.g., "battery level is low") ") and events The number of times (e.g., "low battery warning event") occurs simultaneously within the statistics window (the statistics window duration matches the data collection cycle, set to 10 seconds, corresponding to 100 collection steps) is counted. For state Total number of times it appears alone For the event The total number of times each of the three appears individually is determined by traversing the state sequence output by S1. With event sequence Statistical results show that the formula for calculating association strength using intersection proportion correction is:

[0060]

[0061] In the formula, the denominator is count For "state" or event The total number of occurrences is used to avoid misjudging the correlation due to a state or event appearing too frequently on its own. A larger value indicates a higher state. With the event The stronger the co-occurrence correlation, the better.

[0062] Construct an event-state influence matrix (denoted as...) Matrix, dimension When this happens, it is necessary to calculate the dynamic impact of the event on the state: Let... For the event After triggering, state The magnitude of change within a unit of time (consistent with the data collection period, 0.1 seconds), such as the event... When it is an "obstacle detection event", the state is... For "robot movement speed", That is, the speed after triggering is from Down to The difference This value is obtained by comparing the state data collected by S1 before and after. For all events State The maximum magnitude of change is obtained through statistical analysis of historical event trigger records (such as event-state change data from the past hour). The formula for calculating the normalized impact magnitude is:

[0063]

[0064] Where, The larger the value, the more likely the event is to occur. State The stronger the influence, the more the matrix and The matrix forms a complementary structure, quantifying the state-event relationship from two dimensions: "co-occurrence association" and "dynamic influence".

[0065] This step further includes fusing the state-event correlation matrix and the event-state influence matrix into a state-event-time three-dimensional tensor. Any "state-event-time" element value in this three-dimensional tensor is calculated using the geometric mean of the correlation between the state and the event and the influence of the event on the state, thus uniformly quantifying the dynamic correlation between state, event, and time. This three-dimensional tensor is denoted as... , dimension ,in For time steps (e.g.) (corresponding to a 10-second time sequence). For the first The moment at each time step ( Tensor elements (Corresponding state) ,event ,time The formula for calculating ) is:

[0066]

[0067] Where, for time The element values ​​of the matrix (updated in real time via a sliding window with a window step size of 1 time step). for time The geometric mean of matrix element values ​​can balance the weights of correlation and influence, preventing a single dimension's excessive value from obscuring information in another dimension, and achieving a dynamic and unified representation of state-event relationships over time.

[0068] The algorithm also includes using a low-rank decomposition algorithm to decompose the three-dimensional tensor into a product of a state core tensor, an association core tensor, and a time core tensor. The state core tensor represents the low-dimensional features of the state dimension, the association core tensor represents the low-dimensional features of the interaction between states and events, and the time core tensor represents the low-dimensional features of the time dimension. This reduces the computational complexity of the three-dimensional tensor from a third-order complexity (number of states, number of events, number of time steps) to a linear complexity. The Tensor Train (TT) decomposition algorithm is used here, and the decomposition formula is as follows:

[0069]

[0070] Where, For the state core tensor, Its rank parameter (initially set to 10) is used, and each row corresponds to a low-dimensional feature vector of a state; To associate the core tensor, Its rank parameter (initially set to 10) characterizes the interaction features between state and event; This is a time-core tensor, with each column corresponding to an eigenvector of a time step. The computational complexity of decomposing the first three-dimensional tensor is O(n log n). ,like (Number of state types) (Number of event types) When the time steps are , the complexity is O(n). The complexity is reduced to [value] after decomposition. This significantly reduces the pressure of high-dimensional data processing and meets real-time requirements.

[0071] It also includes dynamically adjusting the rank parameter of the low-rank decomposition based on task priority. For high-priority tasks, the rank parameter is increased to increase the feature dimension of the core tensor, preserving detailed information about state-event associations; for low-priority tasks, the rank parameter is decreased to reduce the feature dimension of the core tensor, improving computational efficiency within acceptable information loss limits. Task priority ( The value is calculated through subsequent steps S4 (for the current task), and its range is [value range missing]. ,in Defined as a high-priority task (such as a precision assembly task). Defined as a low-priority task (such as a routine inspection task). The rank parameter adjustment rule is as follows:

[0072] like And currently ( (where the maximum rank threshold is), then ;

[0073] like And currently ( (where the minimum rank threshold is used), then ;

[0074] like or If so, the rank parameter remains unchanged.

[0075] To avoid re-decomposing the tensor completely after rank adjustment, an incremental update strategy is adopted: when the rank parameter changes from... Upgraded to At that time, only the newly added factor vectors need to be optimized (such as...) The objective function to be optimized is:

[0076]

[0077] Where, For rank The reconstruction tensor at time, For tensor outer product operations, i.e. (vector), (matrix), (Vectors) are expanded into tensors of the same dimension through outer product and then superimposed to ensure consistency with the X dimension. Using the Frobenius norm, this strategy can reduce the computation time of rank adjustment. The above, while ensuring reconstruction accuracy (reconstruction error) (to meet the task requirements).

[0078] S3: Threshold adjustment step, dynamically generating clustering threshold and filtering threshold based on the state-event association matrix and event-state influence matrix.

[0079] The clustering threshold is determined by the sum of the basic threshold and the maximum association weight between the event and all states, and the filtering threshold is determined by the sum of the minimum threshold and the total influence of all events on the state.

[0080] The clustering threshold here is used to aggregate the operating status data collected by S1 (e.g., "battery power"). "Battery" "Classified into the same low power state," the filtering threshold is used to filter event data generated by S1 (such as removing redundant events that have little impact on the state). Let... The basic threshold for clustering (set according to the state dimension, with a value range of...) such as position status ), Clustering threshold adjustment coefficient (variable to be optimized, initial value) For the first Class of events The corresponding clustering threshold The calculation formula is:

[0081] Where, For the event With all states The maximum association weight (constructed from S2) The larger the association weight (matrix), the higher the clustering threshold, which can avoid over-aggregation leading to the loss of state details.

[0082] set up To filter the minimum threshold (set according to event type, with a value range of [value missing]), Such as environmental events ), The filter threshold adjustment coefficient (the variable to be optimized, initial value) ), for the first Class State The corresponding filtering threshold The calculation formula is:

[0083] Where, For all events ( ) for state Total influence (from S2 construction) The larger the total influence of the matrix, the higher the filtering threshold, and events that have a significant impact on the state can be retained first.

[0084] The method in this step specifically includes: further, it includes autonomously optimizing the threshold coefficient using an intelligent optimization algorithm. This intelligent optimization algorithm constructs a weighted summation objective function with the goals of task completion rate, resource utilization rate, and information loss rate. By iteratively adjusting the threshold coefficient, the objective function value is maximized, achieving a multi-objective balance between task efficiency, resource utilization, and information retention. The threshold coefficient here is the one described above. (Clustering threshold adjustment coefficient) and (Filtering threshold adjustment coefficient) is optimized using the Multi-Objective Deep Deterministic Policy Gradient (MO-DDPG) algorithm. First, the objective function is defined. :

[0085] Where, Weighting coefficient (dynamically adjusted based on task priority; higher priority tasks can have their weighting increased). TaskRate represents the task completion rate. (Real-time statistics from S6 process execution steps); TaskRate represents the task completion rate. (Real-time statistics from S6 process execution steps); Redundancy is the rate of redundant events. (Valid events refer to events filtered by the threshold); LossInfo is the loss rate of aggregated state information. ,in The entropy of the state before aggregation. The entropy of the aggregated state ( For state The probability of its occurrence is derived from S1 data statistics.

[0086] The algorithm adjusts iteratively. and make Maximize, in each iteration, the current state (e.g.) Matrix mean Matrix mean Resource surplus rate Input policy network, output and The adjustment amount, and then based on the objective function. The feedback updates the network parameters until Converging to a stable value (fluctuation range) ).

[0087] It also includes constructing a multi-robot interaction graph, with each robot as a graph node. The graph node features include the robot's current threshold coefficient, task list, and fault risk. A graph attention mechanism is used to calculate the similarity of the node features between two robots, and this similarity is normalized to obtain the task dependency between the two robots. Based on the task dependency, a threshold conflict correction strategy is dynamically adjusted, using a smaller conflict correction factor for robots with high task dependency. Assume the scenario includes... The robot, the first The node characteristics of the robot are ,in For robots The current threshold coefficient, Its task list is represented by a 4-dimensional embedded encoding (type / priority / time / resource) (e.g., task type and priority, derived from the S4 task priority calculation steps). Its failure probability (failure prediction from the S8 pre-planning step). Dimension set to ( Each is 1 dimension. Encoded in 4 dimensions, 1D (with 1D expansion reserved).

[0088] First, through the linear mapping matrix ( (reducing dimensionality to a lower-dimensional space) and Perform feature mapping to obtain and Then through attention vectors Calculate similarity:

[0089]

[0090] Where, This is a vector concatenation operation, with LeakyReLU as the activation function (slope set to 0.2). For robots and The original similarity is then normalized using Softmax to obtain the attention weights (i.e., task dependency).

[0091]

[0092]

[0093] Where, For robots The set of neighboring nodes (referring to robots that have task dependencies, such as robots) The material handling task relies on robots (material transportation task) A higher value indicates a higher degree of dependence.

[0094] When multiple robots experience threshold conflicts due to resource competition (such as simultaneously applying for charging stations), the global conflict degree is calculated first. ,in Let t be the number of robots that request resources. The total number of resources (e.g., the number of charging stations); then, the basic correction factor is calculated based on the degree of conflict:

[0095]

[0096] In the formula, Conf The conflict threshold, Finally, adjust the robot based on task dependency. Conflict correction factor:

[0097]

[0098] Where, For robots The maximum dependency with neighboring nodes; the higher the dependency, the better. The smaller the threshold, the gentler the correction, avoiding task-dependent breaks (such as in robots) caused by drastic threshold adjustments. Excessive adjustment of the material handling threshold will affect the robot. (The material picking task). The adjusted threshold coefficient is This ensures collaborative optimization of thresholds across multiple robots.

[0099] S4: Task priority calculation steps: The task priority is calculated by combining the time sensitivity coefficient corresponding to the task deadline, the proportion of task resource requirements, the task urgency coefficient, and the sum of the product of the correlation between the task's corresponding state and the effective event and the influence of the effective event on the state.

[0100] In this step, task priorities are calculated using the multi-factor weighted approach described above, providing a priority basis for subsequent resource allocation. First, the specific definitions, value sources, and mathematical calculation logic of each calculation factor are clearly defined to ensure that all parameters can be obtained from the output of previous steps or from the system's preset configuration, guaranteeing the objectivity and traceability of the priority calculation.

[0101] Time sensitivity coefficient corresponding to the task deadline ( Representing the (One task to be computed), the core of which is to quantify the urgency of the task due to the passage of time. Let the current system time be... (From the system's real-time clock module, unit: seconds), Task The preset deadline is (Parameter configuration from the task initialization command, such as the assembly task deadline) (seconds), first calculate the remaining time for the task. To avoid distortion of the coefficients due to negative remaining time (task timeout) or excessively large remaining time (long-term tasks), an exponential normalization formula is used for calculation:

[0102] Where, (Reference time threshold, preset to 1 hour, can be adjusted according to the scenario, such as 1800 seconds during peak logistics season). The shorter the remaining time, the closer the coefficient is to 1.0. The coefficient for timed-out tasks is fixed at 1.0 to ensure that time-sensitive tasks are processed first.

[0103] Regarding the proportion of task resource requirements This requires a comprehensive assessment of the task's demand on various core system resources. These core resources include... Category (e.g., battery level) computing power Equipment usage time Let the first... The total available system resources of this type are (From the system resource configuration table, such as total power consumption) Total computing power ),Task For the The demand for this type of resource is (Resource request parameters from the task instructions, such as the power required for the assembly task) computing power First, calculate the proportion of demand for each type of resource. And then The arithmetic average of the resource categories is taken to obtain:

[0104]

[0105] Where, The higher the resource demand ratio, the larger the coefficient. Tasks that significantly consume system resources will be prioritized to avoid excessive resource concentration that would cause other tasks to have "no resources available".

[0106] For the task urgency coefficient This coefficient is an inherent attribute of the task and is preset by the system or user according to the task type: high-urgency tasks (such as equipment failure repair, emergency material delivery) are set to [value missing]. Routine tasks (such as daily sorting and routine assembly) are set as Low-urgency tasks (such as data backup and equipment inspection) are set to For example, "robot motor fault repair task" This is directly used as one of the weighting factors to reflect the differences in the urgency and priority of tasks.

[0107] The sum of the products of the correlation between the task's corresponding state and the effective event, and the influence of the effective event on that state. The calculation needs to be combined with the output results of the previous steps: First, determine the task. Core state set (For example, the core state of the "material grabbing task" is the "robotic arm position state") "Grasping force status" ), valid event set For events filtered by the S3 threshold (i.e., those that meet the criteria) The incident, Event-state influence matrix from S2 (Filter threshold calculation from S3). For each core state Calculate its relationship with all valid events. "relevance" The product of influence, then... The average of the product of the core states is taken:

[0108] Where, For state With the event Co-occurrence correlation (from the state-event correlation matrix of S2). For the event State The magnitude of the impact (from the event-state impact matrix of S2). The larger the sum of the products, the closer the task is to the effective events in the current system. Prioritizing the execution of these tasks can better utilize the current state-event interaction information and reduce uncertainty in task execution.

[0109] Finally, a weighted summation method is used to calculate the task. final priority Let the weights of the four factors be respectively (Time-sensitive) (Resource requirements) (Level of urgency) (Related effects), satisfying (The weights can be dynamically adjusted according to the scenario, such as in a production workshop.) Increased to 0.3, decreased (Up to 0.2), the calculation formula is:

[0110]

[0111] Where, The higher the priority value, the higher the priority of the task; after calculation, all tasks to be executed are ranked according to priority. Sort tasks from largest to smallest to generate a priority sequence, providing a clear scheduling basis for S5's resource allocation steps.

[0112] S5: Resource allocation step, resource allocation is performed based on the task priority and resource constraints, wherein the resource constraints are that the total resource consumption of each task in the aggregate state does not exceed the total system resources.

[0113] First, clearly define the core objects and parameters for resource allocation to ensure that all variables are related to previous steps or system configurations: resource types cover the core resources required for robot operation, including power resources (unit: mAh), computing power resources (unit: GFLOPS), and device occupancy time resources (unit: s), which are respectively denoted as resource categories. (1 = electricity) Computing power Device duration). Of which, total system resources Preset configuration parameters (such as total power resources) Total computing power resources Total equipment time resources ), stored in the system resource configuration module; tasks in aggregated state resource consumption It is calculated based on the state data aggregated from the S2 association modeling layer—through the S2 state core tensor. Extract the resource requirement features of the core state corresponding to task i, and combine them with the task priority calculation steps in S4. resource demand ratio Further revisions yielded ,in For the task Basic resource consumption (from task initialization instructions, such as the basic power consumption of assembly tasks) ), 0.2 is the resource consumption correction coefficient (adapting to the fluctuation of resource demand under the aggregation state).

[0114] Resource allocation must satisfy the constraint that "the total resource consumption of all tasks in the aggregated state does not exceed the total system resources". Resource allocation decision variables ( Represented as a task Allocate resources, (Indicates no allocation), then for each type of resource The mathematical expression for the constraint is:

[0115]

[0116] Where, This represents the total number of tasks awaiting resource allocation (derived from the list of tasks to be executed after priority sorting by S4); simultaneously, to prioritize high-priority tasks, resource allocation must be combined with the task priorities calculated by S4. The "priority sorting-greedy allocation" strategy is adopted: first, the tasks to be executed are sorted according to... Sort the tasks from largest to smallest, and allocate resources accordingly, up to a specific resource category. Remaining available amount ( Stop allocating resources for this type of task (for tasks currently pending assignment) to ensure that high-priority tasks receive resource support first.

[0117] The method in this step specifically includes: it also includes aggregating the local resource parameters of multiple robots using a federated learning framework, wherein during the aggregation process, parameters exceeding the local parameter mean are first removed. The extreme parameters are calculated by multiplying the standard deviation by a factor of 1, and then the remaining parameters are weighted by the robot's task completion rate. The local resource parameter here is the predicted value of each robot for the "task-resource matching degree". ( (where N is the total number of robots). Generated by the robot's local computing unit (S1's operational status data + S4's task resource requirements), it is used to characterize the robot. Resource utilization efficiency when performing a certain type of task (e.g.) The larger the robot The less resources are wasted in executing the task.

[0118] First, extreme parameter elimination is performed: Let... The set of local parameters for each robot is Calculate the mean of the set. with standard deviation :

[0119] Where, The mean of local parameters. The standard deviation is used to eliminate those that meet the criteria. The extreme parameters (most of which are caused by robot sensor malfunctions or data noise, such as the misjudgment of resource efficiency caused by a motor current sensor failure in S1) are used to obtain the set of retained parameters. For the reserved robot index set.

[0120] Then, a weighted average aggregation was performed, with the weights being the task completion rates of each robot. (Real-time statistics from S6 process execution steps) After weight normalization, the results are:

[0121]

[0122] Where, The final aggregated global resource matching parameter is:

[0123]

[0124] This parameter is used to correct the resource allocation matching relationship between the task and the robot, such as... Higher-performing robots are prioritized for tasks requiring high resources, thereby improving overall resource utilization efficiency.

[0125] Before uploading local parameters, Laplace noise is added to the parameters. The noise scale is determined by the maximum range of parameter variation, the privacy protection level, and the number of robots. Let the local parameters be... Maximum range of change (Calculated by iterating through the local parameters of all robots), privacy protection level (Commonly used in industrial scenarios) (meeting differential privacy and security standards) Number of robots The scaling parameter of the Laplace noise is:

[0126]

[0127] Where, Determines noise intensity The smaller the size (the higher the privacy protection requirements), The smaller (the fewer robots). The larger the value, the stronger the noise. (This relates to local parameters.) After adding noise, the privacy protection parameters are obtained:

[0128]

[0129] Where, This indicates a mean of 0 and a scale parameter of . Laplace distributed random variable; parameters after adding noise Uploading to the federated learning server avoids the leakage of raw local parameters (including sensitive information such as robot resource consumption details and task execution preferences) while ensuring the availability of the aggregated global parameters, complying with industrial data security standards. Local parameters are defined as those whose differences from the global parameters exceed a threshold, and the difference threshold is set to the mean of the global parameters. That is, the absolute deviation between the local parameter and the global parameter mean. When this happens, the local parameter is uploaded.

[0130] Through the above steps, the resource allocation process satisfies system resource constraints, achieves robust aggregation of heterogeneous resource parameters among multiple robots through federated learning, and also protects privacy. The final output is a task-robot resource allocation scheme, such as "Task 1 (priority 0.92) is allocated to robot A (remaining battery power)." Ranked #1), it consumes 2800mAh of power, has a computing power of 12GFLOPS, and a device duration of 180s, providing clear resource scheduling instructions for the S6 process execution steps.

[0131] S6: Process execution steps, execute the task process according to the task priority and resource allocation results.

[0132] In this step, based on the task priorities and resource allocation results determined in the previous steps, the corresponding task flow is executed to complete the programming and implementation of complex robot tasks. First, the task priority sequence in S4 and the resource allocation scheme in S5 need to be converted into specific instructions executable by the robot, forming a four-dimensional execution scheme of "task-robot-resource-timing": Let the task sequence after S4 be... ( (Highest priority), S5 for each task The assigned robots are (like Resource quotas allocated to robot A are (corresponding to power consumption, computing power, and device duration respectively), then it needs to be... Generate execution instructions containing action sequences and resource usage timings—action sequences (e.g., "move to material area") Grab materials Move to assembly area Assembly) is based on task type presets, and resource usage timing is controlled by "resource consumption rate". The estimated execution time is (From task initialization instructions, such as) Then the power consumption rate (unit: This ensures that resource consumption is evenly distributed and avoids resource depletion in a short period of time.

[0133] During task execution, it is necessary to monitor the execution progress and resource consumption status in real time to avoid deviations from the expected plan. Execution progress is quantified through "sub-step completion rate," which is set as the task... Include Each sub-step (e.g., an assembly task includes 4 sub-steps: positioning, gripping, alignment, and fastening) is monitored in real-time by motion feedback sensors (such as joint angle sensors and gripping force sensors) on the S1 to detect the number of completed sub-steps. The formula for calculating the execution progress is:

[0134]

[0135] Where, ,when The time indicates task completion; simultaneously, the robot's resource sensors in S1 collect data in real time. Actual resource consumption ( (At the current execution time), calculate resource consumption progress:

[0136]

[0137] Where, For resource categories, ;like (Preset resource overspending threshold) If the task is not executed, a resource alert will be triggered, the task execution will be suspended, and feedback will be sent to the S5 resource allocation step to readjust the resource quota (such as allocating redundant resources from other low-priority tasks).

[0138] To address execution deviations (such as robot positioning errors or material grasping failures), a deviation detection and correction mechanism needs to be established. Taking positioning errors as an example, let's set up a task... The target location of a certain sub-step is (From task instructions, such as) ),robot The real-time location is (From the GPS / SLAM sensor of S1), the positioning error is:

[0139]

[0140] Where, For planar positioning deviation (unit: m); if (Preset deviation threshold, such as) If the deviation is corrected, it will be triggered based on the state-event correlation matrix of S2. Query the correlation between the "positioning deviation" event and the "position calibration" status. ,like (For high correlation) the robot's built-in calibration algorithm (such as point cloud matching calibration based on LiDAR) is invoked to correct the position until... If the correlation is low or the calibration fails, the alternative process generated by the S8 pre-planning step is triggered (such as replacing the robot with one that has higher positioning accuracy to perform the sub-step).

[0141] For tasks requiring multi-robot collaborative execution (such as...) Robot A needs to pick up materials, and robot B needs to transport materials. Robot task dependency needs to be based on S3. (Dependency between robots A and B) Achieving time synchronization: Assume the estimated time for robot A to complete the grasping sub-step is... Then, a command to "delay the start of the transportation sub-step" is issued to robot B, with the start time set to [time value missing]. ( (For synchronization buffer time), ensure that robot B can promptly connect to the transportation after robot A completes the grasping, and avoid task disconnection.

[0142] After the task is completed ( The execution results data need to be collected and fed back to the relevant steps: First, the task completion status (success / failure) and actual execution time need to be recorded. Actual resource consumption Feedback is sent to the S4 task priority calculation step to adjust the time sensitivity coefficient of subsequent tasks. (If the actual duration exceeds the estimated duration, then the duration of similar tasks will be increased.) (weighting); secondly, to factor in resource consumption deviations. Feedback is sent to the S5 resource allocation step to optimize resource quota calculation (e.g.) If the value remains positive, it will increase the number of similar tasks. Basic resource quota Third, the execution progress curve and deviation correction records are fed back to the S10 visualization feedback step to dynamically update the BPMN flowchart and realize the transparency of the execution status of the entire chain.

[0143] S7: Cross-scene adaptation steps.

[0144] The cross-scene adaptation step includes: extracting visual image features, laser point cloud features, and task text features of the new scene; using the task text features as anchors, calculating the similarity between the visual image features, laser point cloud features, and task text features, and normalizing the similarity to obtain the attention weights of each modality feature; aligning multimodal features based on the attention weights to generate a latent representation of the scene; calculating the similarity between the latent representation of the scene and the latent representation of historical scenes, and weighted transferring the association model parameters of historical scenes based on the similarity.

[0145] In this step, cross-scene adaptation operations are performed as described above to enable the migration and application of the associated model in different scenarios. First, the extraction methods and parameter definitions for each modality's features are clearly defined to ensure that the feature sources are traceable and meet the scene adaptation requirements: visual image features. Images are captured by high-definition industrial cameras deployed in new scenarios. These images are then extracted using a pre-trained ResNet50 model (weights derived from the ImageNet pre-training library, fine-tuned to suit the industrial environment). The output dimension is... ,Right now Each element corresponds to visual features such as local texture and target contour in the image; laser point cloud features Point cloud data of the new scene was acquired using a 3D laser scanner (1024 points per frame), processed by a PointNet model (pre-trained to adapt to robot scene obstacle and terrain recognition), and the output dimension was [missing information]. ,Right now Characterizes the spatial structure, distance distribution, and other features of point clouds; task text features It generates tasks by encoding task instruction text for new scenarios (such as "outdoor logistics sorting" and "indoor precision assembly"), using a BERT-based pre-trained model (fine-tuned to adapt to industrial task semantics), with an output dimension of [missing information]. ,Right now Capture the semantic connotations of the text (such as the differences in task types between "sorting" and "assembly").

[0146] Based on task text features When calculating the similarity of features across modalities for anchor points, cosine similarity is used to quantify the semantic association between features, avoiding similarity bias caused by differences in feature dimensions. The formula for calculating the similarity between visual image features and text features is:

[0147]

[0148] Where, Text features Through a linear mapping layer (weight matrix) Pre-trained on a cross-modal dataset, Pre-trained on publicly available cross-scenario multimodal datasets for industrial robots (such as RobotScene-100), which contains visual, laser, and text modal data from 50+ industrial scenarios (ensuring the mapping layer adapts to the semantics of industrial scenarios), and the dimensionality reduction results ensure consistency with... Consistent dimensions; represents the dot product of vectors. Let L2 norm be represented. Similarly, the similarity between laser point cloud features and text features is:

[0149]

[0150] in Text features are processed through another linear mapping layer ( The dimensionality reduction result is that the similarity of the text features themselves is set to... (Baseline value). Softmax normalization is applied to the above similarities to obtain the attention weights for each modality feature, ensuring the sum of the weights is 1. The formula is:

[0151]

[0152]

[0153]

[0154] Where, The attention weights for visual, laser point cloud, and text modalities are respectively, satisfying... If a modality is more semantically related to the text (e.g., the sim value of the text "indoor assembly" is high and the indoor image is high), the corresponding weight is greater, ensuring that the aligned features focus on semantically consistent information.

[0155] When generating implicit scene representations by aligning multimodal features based on attention weights, the features of each modality are first mapped to the same dimension through a unified linear layer. (To avoid the impact of dimensional differences on the fusion effect), we obtain Then, a weighted summation is used to generate implicit representations. The formula is:

[0156]

[0157] This representation encapsulates the multimodal implicit associations of new scenarios (such as the fusion of the "logistics box" feature of the image, the "open terrain" feature of the point cloud, and the "sorting" semantic feature of the text in the "outdoor sorting" scenario), providing a core basis for cross-scenario migration.

[0158] When calculating the similarity between the latent representation of a scene and the latent representation of a historical scene, the set of latent representations of historical scenes is used. ( The number of historical scenes comes from the historical scene feature library stored in the system, each... For the first The implicit representations generated by the above steps for each historical scene. The new scene and the... The similarity between the historical scenes is calculated using cosine similarity:

[0159]

[0160] Where, The larger the value, the more similar the new scene is to the first one. The stronger the implicit connection between historical scenarios (such as "outdoor logistics sorting" and "outdoor warehousing and handling"), the better. Close to 1). Normalization is performed to obtain the transfer weights:

[0161]

[0162] Where, For the first A historical scene Matrix (from historical modeling results of S2); similarly, TT core tensor , This migration method eliminates the need to build a correlation model from scratch for new scenarios; it only requires a small amount of new scenario data. By fine-tuning the migration parameters (such as dynamic rank adjustment of S2) for each sample, the system can be quickly adapted, significantly shortening the deployment cycle.

[0163] S8: Pre-planning steps.

[0164] The pre-planning steps include: extracting historical information gain sequences, state-event correlation trends, and task complexity variation characteristics; constructing a time-series prediction model; separating the long-term trend characteristics and periodic characteristics of information gain; focusing on key historical time steps through an attention mechanism to predict the information gain of future time steps; collecting robot motor current, temperature, and sensor signal strength characteristics; assigning high weights to key fault features through an attention mechanism; using an encoder-decoder model to calculate the error between input features and reconstructed features, and the deviation between encoded latent features and normal sample latent features; and weighting the error and deviation to map them to the robot fault probability; when the predicted information gain is lower than a set threshold or the fault probability is higher than a set threshold, generating an alternative process containing a transition task, wherein the similarity of the state features of the transition task with the currently executed task and the target task is not lower than a set threshold.

[0165] In this step, the pre-planning operation is performed according to the above steps to predict risks in advance and generate alternative processes, ensuring the stability of task execution.

[0166] When extracting historical information gain sequences, information gain (IG) is defined as the increment of information in the state-event correlation matrix after a new event is triggered, based on the S2 state-event correlation matrix. Calculation: Let Let be the matrix at time t. If the event is triggered at time t, then ,in For matrix entropy, (Normalized), representing the uncertainty of the matrix; the historical information gain sequence is Historical time steps (taking data from the past 24 hours, with one time step per hour, i.e.) The trend of changes in the correlation between state and event is observed through... Calculation of the mean difference of a matrix in a sliding window: The window size; the task complexity variation characteristics are calculated based on the task priority sequence of S4: Let be the number of tasks at time t. For the task Priority.

[0167] When constructing the time series prediction model, an LSTM-attention hybrid model is adopted: firstly, the long-term trend and periodic characteristics of information gain are separated through linear decomposition, and the long-term trend... To fit the parameters, the least squares method is used. Solve for periodic characteristics The main period is obtained by Fourier transform. (like (Hours, corresponding to work shift cycles). The trend and cycle characteristics are concatenated into an input sequence. Input to LSTM layer to extract latent features Then, focus on key time steps through the attention mechanism: attention weights Depend on With query vector The similarity is determined after (random initialization and training optimization). The key step is Take the front (Time step of weights). Future step( The predicted information gain value (i.e., for the next 4 hours) is:

[0168]

[0169] When collecting robot motor current, temperature, and sensor signal strength characteristics, the feature vector is: ,in Real-time motor current (unit: , from (current sensor, sampling frequency 10Hz) Motor temperature (unit: , from (temperature sensor) Sensor signal strength (normalized to) (From the communication module of S1). Key fault features are assigned high weights through an attention mechanism: the mutual information between each feature and historical fault labels is calculated. Fault) , ( (The label is for faults, where 1 indicates a fault), and the attention weights are obtained by normalization. Similarly, a higher weight indicates that the feature is more critical for fault prediction (e.g., motor current). Typically the highest, as overcurrent is a common cause of failure.

[0170] Error and bias are calculated using an encoder-decoder model (autoencoder): the encoder weights the features. Mapped to latent features (Dimension 20), Decoder Reconstructs Features The error between the input and reconstructed features is The set of latent features for normal samples is: (Based on historical fault-free samples), its mean is The deviation of the encoded latent feature from the normal mean is The error and bias are weighted and mapped to the failure probability: ,in As weight, For the sigmoid function, ensure .

[0171] When the predicted information gain is lower than a set threshold or the failure probability is higher than a set threshold, an alternative process is triggered to generate: information gain threshold. (Take the historical maximum value) (To avoid insufficient information gain leading to inadequate decision-making basis) Fault probability threshold (Based on system reliability requirements, ensure a false alarm rate of <5% with 95% confidence). Transitional tasks are selected from the historical task database, and their status characteristics are as follows: ( Core states (such as battery level, location, and load) are the state characteristics of the currently executing task. The state characteristics of the target task are The similarity is calculated using cosine similarity: , Similarly, setting the threshold to 0.8, only retaining two pairs with equal similarity... Transitional tasks (e.g., when the current task is "material grabbing" and the target task is "material assembly," the transitional task can be "material handling," whose state characteristics are highly similar to both). The generated alternative process is "current task..." Transition Task The target task is defined, and parameters such as resource requirements and execution time of the transition task are stored (from historical task records) to ensure that the original process can be quickly replaced when triggered.

[0172] S9: Visual feedback steps.

[0173] Task nodes, event nodes, and resource nodes are mapped to a dynamic flowchart. The task node labels are based on the calculated task priority, the proportion of task resource requirements to the total system resources, and the product of the correlation between the task's corresponding state and the effective event. The event nodes are labeled with the degree of influence of the event on the state. The proportion of task delay time to the total task duration is used as the increment of the dynamic rank adjustment error threshold in the correlation modeling step, and the proportion of task delay time to the deadline is used as the deduction item of the intelligent optimization algorithm reward function in the threshold adjustment step, so as to realize the feedback optimization of model parameters by the execution results.

[0174] In this step, the visualization feedback operation is performed according to the above steps to achieve visualized monitoring of the task flow and optimization feedback of the execution results on the model parameters. First, the mapping rules of each node and the calculation logic of the labeled parameters are clarified to ensure that the visualization information is directly related to the output of the previous steps: the dynamic flowchart is constructed using the BPMN (Business Process Model and Symbol) standard, with task nodes represented by rectangles, event nodes by circles, and resource nodes by diamonds. Nodes are connected by arrows to reflect the flow sequence (the direction of the arrows indicates the task execution flow).

[0175] Among the parameters annotated for task nodes, task priority is... Directly referencing the results calculated by S4 To visually distinguish between high and low priorities, the node fill color varies. change( It is red. It is yellow. (Green); the proportion of task resource requirements to total system resources For each type of resource (Electricity consumption, computing power, and device duration) are calculated separately, using the following formulas: ,in For tasks in S5 Resource consumption quotas The total system resources (preset configuration) are presented in the format of "Power consumption percentage: XX%, Computing power percentage: XX%, Duration percentage: XX%"; the sum of the product of the correlation between the task's corresponding status and the effective events, referencing S4. The calculation logic is as follows: ,in (S2 State-Event Association Matrix) (S2 event - state influence matrix) (Valid events after S3 filtering) all come from previous steps. This value is marked below the task node to reflect the degree of correlation between the task and the current system event.

[0176] The "influence of the event on the state" labeled on the event node is taken from the S2 event-state influence matrix. The maximum impact value of this event across all states is given by the formula: ,in For the event State The extent of the impact is indicated by color-coding to differentiate the levels of impact. It is orange. It is blue. (Used in gray) to facilitate quick identification of high-impact events (such as "motor overcurrent events"). generally ).

[0177] Resource nodes indicate the current resource remaining rate, using the following formula: ,in The total resources allocated to tasks in S5 have a remaining rate lower than [a certain percentage]. A flashing warning on the border of a time node indicates that resources are scarce and timely resource allocation is needed.

[0178] In the execution result feedback optimization section, we first define the relevant parameters for task delay time: Task Actual execution time Execution progress statistics from S6 (when) (Time recorded), estimated execution time From task initialization instructions (such as assembly tasks) The task delay time is as follows: (Delay time is non-negative; 0 is the case for no delay); the total task duration is the actual execution time. Task deadline Calculation of time sensitivity coefficient from S4 (task initialization instruction preset).

[0179] The proportion of task delay time to total task duration is used as the increment of the dynamic rank adjustment error threshold in the association modeling step: Let the base error threshold for dynamic rank adjustment in S2 be... (The default value is 0.05, which is the reconstruction error) (to be acceptable), delay ratio Then the error threshold increment ,in This is the incremental coefficient (determined based on system debugging to ensure the impact of delays on the error threshold is controllable); the adjusted error threshold is... This value is fed back to the dynamic rank adjustment stage of S2—if the task delay is caused by the loss of associated information due to the low rank parameter (such as the deviation in state judgment caused by low rank), then increasing the error threshold can allow for higher reconstruction errors, indirectly promoting the increase of the rank parameter (such as the original...). The time rank is 8, after adjustment The time rank can be increased to 9, reducing delays caused by insufficient information.

[0180] The ratio of task delay time to deadline is used as a deduction term in the reward function of the intelligent optimization algorithm during the threshold adjustment step: The original reward function of the MO-DDPG algorithm in S3 is... Define the delay ratio (The task start time is from the execution record of S6), then the reward function deduction item. ,in The deduction factor (to balance the impact of delay on rewards and avoid excessive penalties); the adjusted reward function is: This function feeds back to S3's intelligent optimization algorithm—if task delays are caused by an unreasonable threshold coefficient (such as filtering out critical events with an excessively high threshold), the deduction of rewards will drive the algorithm to iterate and adjust. (Clustering threshold adjustment coefficient) (Filter threshold adjustment coefficient), optimize threshold generation logic (such as lowering the filter threshold to retain more critical events), and reduce delays in subsequent tasks.

[0181] The above feedback optimization process is automatically triggered after each task is completed, and the adjustments are made accordingly. and The parameter libraries for S2 and S3 are updated respectively for correlation modeling and threshold adjustment in the next task, forming a closed loop of "execution-feedback-optimization" to continuously improve the system's adaptability to task delays and reduce the recurrence delay rate of similar tasks (tested to show that this feedback mechanism can reduce the task delay rate from the initial...). Down to the following).

[0182] It should be noted that, for the sake of simplicity, the foregoing method embodiments are all described as a series of actions. However, those skilled in the art should understand that this application is not limited to the described order of actions, because according to the embodiments of this application, some steps can be performed in other orders or simultaneously. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are all optional embodiments, and the actions and modules involved are not necessarily essential to this application.

[0183] This solution constructs a complete logical chain from data input to closed-loop optimization through "coordinated linkage of all-link technologies". The design characteristics of the technologies at each stage can objectively derive the corresponding technical effects. The specific derivation process is as follows:

[0184] First, in the data acquisition stage, a technical approach of "multi-dimensional sensors (robot body sensors + environmental perception devices + multi-modal acquisition devices) + moving average filtering preprocessing" is adopted: the former ensures full-dimensional data input covering robot operating status (position, power, temperature), environmental events (obstacles, anomalies), and scene multi-modal features (visual, laser, text), avoiding subsequent modeling deviations caused by missing data dimensions; the latter smooths noisy data (such as Gaussian noise from position sensors and fluctuations from power sensors) through a sliding window to ensure the accuracy of the original data - this method directly leads to the effect of "providing a high-quality, full-dimensional data foundation for subsequent correlation modeling", eliminating the interference of noise and missing data on the accuracy of modeling.

[0185] Secondly, the association modeling process employs a progressive technical approach: "dual-matrix quantization (state-event association matrix + event-state influence matrix) → 3D tensor fusion → TT low-rank decomposition + task-aware dynamic rank adjustment." The dual-matrix approach quantifies state-event relationships from two dimensions: "co-occurrence association" and "dynamic influence," addressing the problem that traditional single matrices cannot fully characterize interaction characteristics. Tensor fusion integrates the dual matrices with the time dimension, preserving spatiotemporal dynamic associations and avoiding consistency conflicts between independent matrices. TT decomposition reduces high-dimensional computational complexity from third-order to linear, resolving computational latency issues in large-scale scenarios. Dynamic rank adjustment adapts rank parameters according to task priority (increasing rank to maintain accuracy for high-priority tasks and decreasing rank to improve efficiency for low-priority tasks), avoiding the waste of accuracy or insufficient efficiency caused by a "one-size-fits-all" rank setting. This series of methods collaboratively derives the effect of "balancing dynamic characterization of state-event associations, real-time computation, and task adaptability," meeting the modeling needs of tasks with different priorities in complex scenarios.

[0186] Furthermore, the threshold adjustment process employs a technique of "MO-DDPG multi-objective optimization of threshold coefficients + graph attention multi-machine collaborative conflict correction": MO-DDPG constructs a weighted function with task completion rate, resource utilization rate, and information loss rate as objectives, and autonomously iteratively optimizes the clustering / filtering thresholds, replacing the experience-based reliance of traditional manual threshold setting, and avoiding local optima caused by single-objective optimization; the graph attention mechanism calculates the multi-robot task dependency through node feature similarity and dynamically adjusts the conflict correction factor (the higher the dependency, the more moderate the correction), avoiding task collaboration disruption caused by multi-machine threshold conflicts—this method derives the effect of "autonomous optimization of threshold coefficients and no conflict in multi-machine collaboration", balancing multi-objective requirements while ensuring the threshold collaboration of heterogeneous robot clusters.

[0187] The task priority calculation process employs a multi-factor weighting technique, combining "time sensitivity coefficient + resource requirement ratio + urgency coefficient + state-event correlation product": the time sensitivity coefficient quantifies the urgency of the deadline, the resource requirement ratio reflects the task's dependence on system resources, the urgency coefficient reflects the task's inherent priority, and the correlation product reflects the close relationship between the task and the current event. This multi-factor weighting avoids scheduling biases caused by a single dimension (such as looking only at time), resulting in an effect of "precise matching between task scheduling priority and system state and task characteristics," ensuring that high-value tasks (time-sensitive, resource-critical, urgent, and closely related) receive priority in scheduling resources.

[0188] The resource allocation process employs a technique of "federated learning robust aggregation (extreme parameter removal + task completion rate weighting) + Laplace noise differential privacy": robust aggregation removes abnormal local parameters (such as extreme resource parameters caused by sensor failure) and allocates aggregation weights according to robot task completion rate to avoid aggregation bias caused by heterogeneous data (such as differences in robot parameters from different manufacturers); differential privacy protects sensitive parameters (such as process paths and resource consumption details) by adding Laplace noise, complying with data security specifications. This approach yields the effect of "accurate aggregation of heterogeneous robot resource parameters and secure privacy of sensitive data", taking into account both the rationality of resource allocation and the data security requirements of industrial scenarios.

[0189] The process execution phase employs a combination of techniques: real-time progress / resource monitoring, deviation correction (positioning deviation calibration), and multi-machine timing synchronization (dependency-driven startup time adaptation). Real-time monitoring ensures timely detection of execution deviations (such as resource overruns or positioning offsets). Deviation correction uses correlation modeling results to call calibration algorithms (such as laser point cloud matching) to correct deviations. Multi-machine synchronization adjusts startup times based on task dependencies (such as delaying the buffer time for transport tasks and capturing tasks). This approach results in the effect that "deviations in the task execution process can be corrected in a timely manner, and multi-machine collaboration is seamless," avoiding task failures or collaboration breakdowns caused by accumulated deviations.

[0190] The cross-scene adaptation process employs a technical approach of "multimodal feature extraction (visual / laser / text) + text anchor attention alignment + weighted transfer of historical parameters": multimodal features comprehensively characterize scene features, text anchor alignment ensures that multimodal features are semantically consistent around the task (e.g., the text "sorting" is aligned with the visual logistics box features), and weighted transfer reuses historical related model parameters based on scene similarity. This approach leads to the effect of "no need to model new scenes from scratch, and quickly achieve related model adaptation", solving the problems of long traditional cross-scene modeling cycles and high data dependence.

[0191] The pre-planning stage employs a combination of techniques: "time-series prediction of information gain (trend + period separation + attention focusing on key time steps) + fault probability calculation (key feature weighting + autoencoder error mapping) + alternative process triggering." Information gain prediction identifies risks of insufficient decision-making basis in advance, fault probability calculation focuses on key fault characteristics such as motor current and temperature for accurate early warning, and alternative processes (including transition tasks) seamlessly connect when risks are triggered. This approach leads to the effect of "identifying insufficient information and fault risks in advance, and using alternative processes to avoid task interruption," thereby improving the robustness of task execution.

[0192] The visualization feedback process employs a combination of techniques: dynamic node annotation (task priority / resource proportion / relevance + event impact + resource remaining rate) and execution result feedback optimization (delay ratio-driven modeling / threshold parameter adjustment). Dynamic annotation enables intuitive monitoring of task, event, and resource status, while feedback optimization transforms execution delays into incremental error thresholds in the correlation modeling and deductions in the threshold adjustment reward function. This approach leads to the effect of "transparent human-machine monitoring and closed-loop iterative optimization of system parameters," continuously improving the accuracy of modeling and adjustment in subsequent stages.

[0193] In summary, the end-to-end technical approach, through the layered connection and collaborative optimization of "data-modeling-adjustment-scheduling-execution-adaptation-pre-control-feedback", ultimately derives the overall technical effect of "robot complex task orchestration possessing dynamic adaptability (adapting to multiple scenarios / multiple robots), scheduling rationality (multi-factor priority + precise resource allocation), execution stability (risk pre-control + deviation correction), privacy security (differential privacy), and sustainable optimization (closed-loop feedback)". This comprehensively solves the core problems of traditional static orchestration in complex dynamic scenarios, such as poor adaptability, weak collaboration, and uncontrollable risks.

[0194] The above description is merely a preferred embodiment of this application and an explanation of the technical principles employed. Those skilled in the art should understand that the scope of disclosure in this application is not limited to technical solutions formed by specific combinations of the above-described technical features, but should also cover other technical solutions formed by arbitrary combinations of the above-described technical features or their equivalents without departing from the foregoing disclosed concept. For example, technical solutions formed by substituting the above features with (but not limited to) technical features with similar functions disclosed in this application.

Claims

1. A method for dynamically arranging complex tasks for robots, characterized in that, include: The data acquisition step involves collecting the robot's operational status data and event data; The association modeling steps involve constructing a state-event association matrix and an event-state influence matrix. The state-event association matrix quantifies the co-occurrence association strength of states and events by statistically analyzing the number of times a specific state and a specific event occur simultaneously, the total number of times the state occurs alone, and the total number of times the event occurs alone, using an intersection ratio correction method. The event-state influence matrix quantifies the influence of an event on a state by calculating the magnitude of change in a specific state after a specific event is triggered, and normalizing this magnitude of change with the maximum magnitude of change of all events on that state. The threshold adjustment step dynamically generates clustering thresholds and filtering thresholds based on the state-event association matrix and the event-state influence matrix. The clustering threshold is determined by the sum of the base threshold and the maximum association weight between the event and all states, and the filtering threshold is determined by the sum of the minimum threshold and the total influence of all events on the state. The task priority calculation steps combine the time sensitivity coefficient corresponding to the task deadline, the proportion of task resource requirements, the task urgency coefficient, and the sum of the product of the correlation between the task's corresponding state and the effective event and the impact of the effective event on that state, and calculate the task priority in a weighted manner. The resource allocation step involves allocating resources based on the task priority and resource constraints, wherein the resource constraints are that the total resource consumption of all tasks in the aggregated state does not exceed the total system resources. The process execution steps are carried out according to the task priority and resource allocation results.

2. The method for dynamic orchestration of complex robot tasks according to claim 1, characterized in that, The association modeling step further includes merging the state-event association matrix and the event-state influence matrix into a state-event-time three-dimensional tensor. The value of any "state-event-time" element in the three-dimensional tensor is calculated by the geometric mean of the association degree between the state and the event and the influence degree of the event on the state, so as to uniformly quantify the dynamic association relationship between the state, event and time.

3. The method for dynamic orchestration of complex robot tasks according to claim 2, characterized in that, The association modeling step further includes using a low-rank decomposition algorithm to decompose the three-dimensional tensor into a product of a state core tensor, an association core tensor, and a time core tensor. The state core tensor represents the low-dimensional features of the state dimension, the association core tensor represents the low-dimensional features of the interaction between the state and the event, and the time core tensor represents the low-dimensional features of the time dimension, so as to reduce the computational complexity of the three-dimensional tensor from a third-order complexity of the number of states, events, and time steps to a linear complexity.

4. The method for dynamic orchestration of complex robot tasks according to claim 3, characterized in that, The association modeling step also includes dynamically adjusting the rank parameter of the low-rank decomposition based on task priority. For high-priority tasks, the rank parameter is increased to increase the feature dimension of the core tensor and retain detailed information about state-event associations. For low-priority tasks, the rank parameter is decreased to reduce the feature dimension of the core tensor and improve computational efficiency within an acceptable range of information loss.

5. The method for dynamic orchestration of complex robot tasks according to claim 1, characterized in that, The threshold adjustment step also includes autonomously optimizing the threshold coefficient through an intelligent optimization algorithm. The intelligent optimization algorithm constructs a weighted summation objective function with the goals of task completion rate, resource utilization rate and information loss rate. By iteratively adjusting the threshold coefficient, the objective function value is maximized, thereby achieving a multi-objective balance between task efficiency, resource utilization and information retention.

6. The method for dynamically arranging complex robot tasks according to claim 5, characterized in that, The threshold adjustment step further includes constructing a multi-robot interaction graph, with each robot as a graph node. The graph node features include the robot's current threshold coefficient, task list, and fault risk. The similarity of the node features of two robots is calculated using a graph attention mechanism, and the similarity is normalized to obtain the task dependency between the two robots. The threshold conflict correction strategy is dynamically adjusted based on the task dependency, and a smaller conflict correction factor is used for robots with high task dependency.

7. The method for dynamic orchestration of complex robot tasks according to claim 1, characterized in that, It also includes a cross-scene adaptation step, which includes: Extract visual image features, laser point cloud features, and task text features from the new scene; Using the task text features as anchors, the similarity between the visual image features, laser point cloud features and task text features is calculated, and the similarity is normalized to obtain the attention weights of each modality feature. Based on the attention weights, multimodal features are aligned to generate latent scene representations. Calculate the similarity between the latent representation of the current scene and the latent representation of historical scenes, and then weight the parameters of the association model of the historical scene based on the similarity.

8. The method for dynamic orchestration of complex robot tasks according to claim 1, characterized in that, It also includes a pre-planning step, which includes: Extract historical information gain sequences, state-event correlation trends, and task complexity variation features to construct a time series prediction model. Separate the long-term trend features and periodic features of information gain, and use an attention mechanism to focus on key historical time steps to predict the information gain of future time steps. The robot collects features of motor current, temperature, and sensor signal strength. Key fault features are assigned high weights through an attention mechanism. An encoder-decoder model is used to calculate the error between the input features and the reconstructed features, and the deviation between the encoded latent features and the latent features of normal samples. The error and deviation are weighted and mapped to the robot fault probability. When the predicted information gain is lower than a set threshold or the failure probability is higher than a set threshold, an alternative process containing a transition task is generated. The similarity of the state characteristics of the transition task with the currently executed task and the target task is not lower than a set threshold.

9. The method for dynamic orchestration of complex robot tasks according to claim 1, characterized in that, The resource allocation step also includes aggregating the local resource parameters of multiple robots using a federated learning framework. During the aggregation process, extreme parameters that exceed the mean ± 2 standard deviations of the local parameters are first removed, and then the remaining parameters are weighted and averaged using the robot task completion rate as the weight. Before uploading the local parameters, Laplace noise is added to the parameters. The noise scale is determined by the maximum variation range of the parameters, the privacy protection level, and the number of robots.

10. The method for dynamic orchestration of complex robot tasks according to claim 1, characterized in that, It also includes a visualization feedback step, which maps task nodes, event nodes, and resource nodes into a dynamic flowchart. The task node is labeled with the task priority calculated based on claim 1, the proportion of task resource requirements to the total system resources, and the product of the correlation influence between the corresponding task state and the effective event. The event node is labeled with the degree of influence of the event on the state. The proportion of task delay time to the total task duration is used as the increment of the dynamic rank adjustment error threshold in the correlation modeling step, and the proportion of task delay time to the deadline is used as the deduction item of the intelligent optimization algorithm reward function in the threshold adjustment step, so as to realize the feedback optimization of the model parameters by the execution result.

Images