Three-in-one brain robot collaborative control architecture system and control method

Abstract

The application discloses a three-way one-brain robot cooperative control architecture system and a control method, and belongs to the technical field of robot control. The system is connected with multiple robots, a multi-protocol access layer receives data packets of multiple data nodes, analyzes each data packet and adds an extension header, generates a time sequence enhancement data frame corresponding to the data packet, a unified data platform layer calls the time sequence enhancement data frame, analyzes the extension header to obtain an event sequence of the time sequence enhancement data frame, generates multiple core factor matrices according to the event sequence, aggregates the multiple core factor matrices to generate a global data feature map, a smart central brain layer plans tasks for the multiple robots based on the global data feature map to generate a task chain, and a multi-robot execution layer sends task instructions corresponding to the task chain to corresponding robots and receives state information of the robots in response to execution of the task instructions. The method can effectively fuse the event evolution relationship in the time dimension.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of robot control, and particularly relates to a three-in-one brain robot collaborative control architecture system and a control method. BACKGROUND

[0002] With the rapid development of intelligent manufacturing, smart logistics and unmanned operation scenarios, multi-robot collaborative systems are widely used in industrial automation, warehouse sorting, medical assistance and emergency rescue, etc. In practical applications, multiple heterogeneous robots are usually required to collaboratively complete complex tasks in a shared environment.

[0003] Traditional multi-robot systems usually adopt centralized or simple distributed architecture, and task scheduling relies on preset rules or static strategies. Existing systems usually process sensor data, robot state, etc. in isolation or statically aggregate, resulting in a lack of accurate perception of the overall dynamic behavior of the system for the upper layer decision, which is difficult to support complex collaborative tasks.

[0004] Therefore, there is an urgent need for a new collaborative control architecture that can effectively integrate the event evolution relationship in the time dimension. SUMMARY

[0005] The present application aims to at least solve one of the technical problems existing in the prior art. To this end, the present application proposes a three-in-one brain robot collaborative control architecture system and a control method, which can effectively integrate the event evolution relationship in the time dimension.

[0006] In a first aspect, the present application provides a three-in-one brain robot collaborative control architecture system, which is connected with multiple robots, and comprises a multi-protocol access layer, a unified data platform layer, an intelligent central brain layer and a multi-robot execution layer connected in sequence.

[0007] The multi-protocol access layer is configured to receive data packets of multiple data nodes, parse and add an extension header to each data packet, and generate a time series enhanced data frame corresponding to the data packet.

[0008] The unified data platform layer is configured to call the time series enhanced data frame, parse the extension header to obtain an event sequence of the time series enhanced data frame, generate multiple core factor matrices according to the event sequence, aggregate the multiple core factor matrices to generate a global data feature map, and the global data feature map is used to describe the data dynamic change of each data node in the time dimension.

[0009] The intelligent central brain layer is configured to perform task planning on the multiple robots based on the global data feature map, and generate a task chain.

[0010] The multi-robot execution layer is configured to issue task instructions corresponding to the task chain to corresponding robots and receive state information of the robots in response to execution of the task instructions.

[0011] According to an embodiment of the present application, the multi-protocol access layer comprises a data access module, a buffer management module and a timing enhancement module connected in sequence.

[0012] The data access module is configured to receive data packets of the plurality of data nodes.

[0013] The buffer management module is configured to store the data packets in a ring buffer, call a dynamic protocol analysis engine to analyze the data packets, extract fingerprint features of the data packets, determine node types of the data nodes based on the fingerprint features, associate the node types with the data packets, and the node types include synonymous nodes and non-synonymous nodes.

[0014] In the case where the data nodes are the synonymous nodes, the data packets are stored in a synonymous node pool in the ring buffer.

[0015] In the case where the data nodes are the non-synonymous nodes, buffer partitions are divided in the ring buffer according to priority labels of the data nodes and network load to store corresponding data packets.

[0016] The timing enhancement module is configured to read the data packets from the ring buffer, add the extension header to the data packets according to node IDs of the data nodes, sliding window numbers of time windows, and comparison logic clocks.

[0017] According to an embodiment of the present application, the buffer management module is further configured to:

[0018] convert the fingerprint features into semantic particles, the fingerprint features include field names, data types and numerical value ranges, and the semantic particles include semantic vectors, field strength coefficients and spin directions;

[0019] construct local semantic fields for each data node based on the semantic particles, and the interaction relationships between the semantic particles in the local semantic fields include semantic similarity, functional complementarity and semantic conflict;

[0020] calculate field strength interference patterns and wave function overlap degrees between the local semantic fields corresponding to the data nodes;

[0021] perform dimension fusion on the local semantic fields based on the field strength interference patterns and the wave function overlap degrees, determine field distances between the local semantic fields, and the field distances include semantic distances, functional distances, timing distances and causal distances.

[0022] According to the field distance, the node type of the data node is determined.

[0023] According to an embodiment of the present application, the multi-protocol access layer further comprises a time alignment module, which is configured to:

[0024] invoke the time series enhanced data frame, and extract a timestamp;

[0025] in the case that the timestamp comprises a physical timestamp, match the physical timestamp with a time window, and generate an index table for recording the data packets received in each time window;

[0026] in the case that the timestamp comprises a PTP synchronization timestamp, time-align the event units in the time series enhanced data frame;

[0027] in the case that the timestamp packet does not comprise a PTP synchronization timestamp, time-align the event units in the time window according to the index table and the physical timestamp.

[0028] According to an embodiment of the present application, the unified data platform layer comprises a data parsing module, an event sorting module, a tensor decomposition module and a matrix aggregation module connected in sequence;

[0029] the data parsing module is configured to invoke the time series enhanced data frame, parse the extension header to obtain the timestamp in the time series enhanced data frame, determine the time slot number of the event unit according to the timestamp, and divide the event unit based on the event content of the time series enhanced data frame;

[0030] the event sorting module is configured to sort the event units according to the time slot number, and generate the event sequence;

[0031] the tensor decomposition module is configured to construct a high-order event tensor for the event sequence, perform random singular value decomposition on the high-order event tensor, and generate the core factor matrix of each data node, wherein the high-order event tensor comprises a time dimension, a node dimension and an attribute dimension of the event unit;

[0032] the matrix aggregation module is configured to aggregate the core factor matrix, and generate the global data feature map.

[0033] According to an embodiment of the present application, the matrix aggregation module is further configured to:

[0034] model the correlation between the core factor matrices through a time-varying graph attention network, and determine the dynamic interaction features of the data nodes in the time dimension.

[0035] aggregate the plurality of core factor matrices based on the dynamic interaction feature to generate a time-varying system embedding vector;

[0036] construct the global data feature map based on the time-varying system embedding vector, the global data feature map comprising a horizontal dimension and a vertical dimension, the horizontal dimension representing attribute features of the data nodes, and the vertical dimension representing state evolution of the data nodes over time.

[0037] According to an embodiment of the present application, the time-varying graph attention network comprises, which are connected in sequence, a time-aware graph construction layer, a multi-scale time convolution layer, a dynamic graph attention layer, a space-time fusion module, and a dynamic interaction feature extraction layer;

[0038] The time-aware graph construction layer is configured to calculate a basic similarity between the core factor matrices based on a time window, construct a plurality of static relationship networks, analyze time sequence correlation of the static relationship networks, and construct a dynamic relationship graph.

[0039] The multi-scale time convolution layer is configured to extract multi-time scale features from the dynamic relationship graph, and filter time sequence noise through time gating to obtain multi-time scale features.

[0040] The dynamic graph attention layer is configured to perform attention allocation on the multi-time scale features in the time dimension, extract time sequence features in the multi-time scale features through a time attention layer, extract structural correlation strength between data nodes through a structure attention layer, and generate spatial structure features.

[0041] The space-time fusion module is configured to fuse the time sequence features and the spatial structure features to generate an embedding vector.

[0042] The dynamic interaction feature extraction layer is configured to extract the dynamic interaction feature between the data nodes in the time dimension based on the embedding vector.

[0043] According to an embodiment of the present application, the intelligent hub brain layer comprises, which are connected in sequence, a task planning module and a complexity control module.

[0044] The task planning module is configured to perform task planning on a plurality of robots based on a task target and time sequence evolution information of the global data feature map, and state information, response preference, and task allocation granularity of the robots, to generate a task chain, the state information comprising a current position, a remaining power, and an execution task type of the robots, and the response preference being determined by modeling historical task response data through an entropy regularization term.

[0045] The complexity control module is used to calculate the task execution complexity based on the task chain length, the node size of the data node, and the fluctuation range of the excitation trajectory, and to control the task execution complexity by adjusting the regularization coefficient.

[0046] According to one embodiment of this application, the multi-robot execution layer includes an instruction issuing module and an information receiving module that are interconnected;

[0047] The instruction issuing module is used to map the task chain to a low-dimensional latent space and generate task instructions corresponding to each robot. The task instructions include action instructions and path planning instructions.

[0048] The information receiving module is used to receive the status information generated by the robot in response to the task instruction, and generate a reward signal based on the status information. The status information includes task completion degree, energy consumption, and number of collisions.

[0049] Secondly, this application provides a multi-robot cooperative control method, applied to the three-way-one-brain robot cooperative control architecture system as described in the first aspect, the method comprising:

[0050] Data packets from multiple data nodes are received through a multi-protocol access layer, the data packets are stored in a circular buffer, and each data packet is parsed and an extended header is added to generate a timing-enhanced data frame corresponding to the data packet.

[0051] The time-series enhanced data frame is retrieved by the unified data platform layer, and the extended header is parsed to obtain the event sequence of the time-series enhanced data frame. Based on the event sequence, multiple core factor matrices are generated. The multiple core factor matrices are aggregated to generate a global data feature map, which is used to describe the dynamic changes of each data node in the time dimension.

[0052] Based on global data feature maps, the intelligent central brain layer performs task planning for the multiple robots and generates task chains.

[0053] The multi-robot execution layer sends the task instructions corresponding to the task chain to the corresponding robots and receives the status information of the robots in response to the task instructions.

[0054] Additional aspects and advantages of this application will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of this application.

[0055] This application provides a collaborative control architecture system and control method for a three-way, one-brain robot, which has the following advantages over existing technologies:

[0056] (1) By parsing data packets from different data nodes through the multi-protocol access layer and adding extended headers to generate time-enhanced data frames, subsequent processing can make more effective use of time-dimensional information; the unified data platform layer parses the time-enhanced data frames generated by the multi-protocol access layer, extracts event sequences, and converts them into core factor matrices, thereby improving the depth and accuracy of data analysis; the intelligent central brain layer performs task planning based on global data feature maps, generates task chains for different robots, and can optimize task allocation according to the current environment and robot status, thereby improving task execution efficiency and resource utilization, while ensuring the quality of task completion; the multi-robot execution layer issues task instructions to the corresponding robots and collects their status feedback for task execution, ensuring accurate transmission of instructions and real-time monitoring of execution status, which helps to adjust strategies in a timely manner to deal with emergencies; through unified access, dynamic modeling, intelligent planning and closed-loop execution of heterogeneous robot devices, the event evolution relationship in the time dimension can be effectively integrated, improving the collaborative efficiency and task robustness of the multi-robot system in complex dynamic environments, ensuring the continuity and reliability of task execution, and also promoting the convenience of human-computer interaction.

[0057] (2) By leaping from raw protocol data to high-level semantic understanding, the determination of node type not only depends on the protocol format, but also realizes the semantic role and functional positioning of the device in collaborative tasks, enabling more accurate and adaptive data flow management and resource allocation. Attached Figure Description

[0058] The above and / or additional aspects and advantages of this application will become apparent and readily understood from the description of the embodiments taken in conjunction with the following drawings, in which:

[0059] Figure 1 This is a schematic diagram of the three-way-one-brain robot collaborative control architecture system provided in the embodiments of this application;

[0060] Figure 2 This is a flowchart illustrating the multi-robot collaborative control method provided in the embodiments of this application. Detailed Implementation

[0061] The technical solutions of the embodiments of this application will be clearly described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this application. All other embodiments obtained by those skilled in the art based on the embodiments of this application are within the scope of protection of this application.

[0062] The terms "first," "second," etc., used in the specification and claims of this application are used to distinguish similar objects and not to describe a specific order or sequence. It should be understood that such use of data can be interchanged where appropriate so that embodiments of this application can be implemented in orders other than those illustrated or described herein, and the objects distinguished by "first," "second," etc., are generally of the same class and the number of objects is not limited; for example, a first object can be one or more. Furthermore, in the specification and claims, "and / or" indicates at least one of the connected objects, and the character " / " generally indicates that the preceding and following objects are in an "or" relationship.

[0063] The following description, in conjunction with the accompanying drawings, details the three-way, one-brain robot collaborative control architecture system and multi-robot collaborative control method provided in this application through specific embodiments and application scenarios.

[0064] Understandably, the "three connections" in "three connections and one brain" refer to the multi-protocol access layer, the unified data platform layer, and the multi-robot execution layer, while the "one brain" refers to the intelligent central brain layer. When the system coordinates the work of multiple robots, the intelligent central brain layer makes centralized decisions and allocates tasks, while other layers handle data communication and execution.

[0065] like Figure 1 As shown, the three-way one-brain robot collaborative control architecture system is connected to multiple robots. The system includes a multi-protocol access layer 110, a unified data platform layer 120, an intelligent central brain layer 130, and a multi-robot execution layer 140 connected in sequence.

[0066] The multi-protocol access layer 110 is used to receive data packets from multiple data nodes, parse each data packet and add an extension header to generate a timing-enhanced data frame corresponding to the data packet;

[0067] The unified data platform layer 120 is used to retrieve the time-series enhanced data frame and parse the extended header to obtain the event sequence of the time-series enhanced data frame. Based on the event sequence, multiple core factor matrices are generated. The multiple core factor matrices are aggregated to generate a global data feature map. The global data feature map is used to describe the dynamic changes of each data node in the time dimension.

[0068] The intelligent central brain layer 130 is used to perform task planning for the multiple robots and generate task chains based on global data feature maps;

[0069] The multi-robot execution layer 140 is used to send the task instructions corresponding to the task chain to the corresponding robot, and to receive the status information of the robot in response to the task instructions in performing the task.

[0070] Among them, many robots are heterogeneous robots such as autonomous mobile robots, robotic arms, drones, and fixed sensors.

[0071] Understandably, data nodes are autonomous mobile robots, robotic arms, vision sensors, LiDAR, environmental monitoring equipment, etc., and they use different communication protocols, such as ROS, Modbus TCP, CAN bus, MQTT, EtherCAT, etc.

[0072] A data packet is a raw binary stream or a serialized message unit transmitted over a physical network.

[0073] The extended header includes, but is not limited to, physical timestamps, device unique identifiers, data semantic type identifiers, and task context identifiers.

[0074] A time-series augmented data frame is a data structure that contains time-series information and other augmented features.

[0075] In actual implementation, the multi-protocol access layer 110 is configured with multiple protocol adaptation modules, each corresponding to different types of access devices. For example, autonomous mobile robots upload their position, speed, and battery status via the ROS 2 protocol; robotic arm robots upload joint angles and gripper status via the Modbus TCP protocol; and vision cameras upload the detected cargo type and coordinate information via the Message Queuing Telemetry Transport (MQTT) protocol.

[0076] After receiving the node data packet, the multi-protocol access layer 110 identifies the communication protocol type corresponding to the data packet, calls the corresponding parser to perform structured parsing of the data packet, and adds a uniform format extension header to each data packet to enhance the temporal sequence of the data. The original data packet is encapsulated into a temporally enhanced data frame and stored in a buffer queue. Among them, the data semantic type identifier includes pose, object detection result, power level, etc., and the task context identifier includes the ID of the currently executing task.

[0077] An event sequence is a collection of time-enhanced data frames generated by the same data node over a period of time and ordered by timestamps.

[0078] The core factor matrix is ​​represented in the form of a compressed matrix to describe the state changes of data nodes and to reveal and represent potential core factors or hidden structures in the data.

[0079] The global data feature map is represented as a situation map, where each pixel or region represents the collaborative state, potential conflicts, or overall task progress of the entire robot swarm within a specific spatiotemporal range. It characterizes the state evolution of all data nodes over time and their interdependencies. For example, a channel in the feature map might highlight a workspace conflict area where two robots may potentially clash.

[0080] The unified data platform layer 120 periodically retrieves time-series augmented data frames from the buffer queue of the multi-protocol access layer 110, parses the extended header of each time-series augmented data frame, extracts information such as node identification (ID), timestamp, and device identifier, and classifies the data in the time-series augmented data frames into corresponding event categories according to semantic type, forming an event sequence arranged in chronological order. For example, position data from autonomous mobile robots (AMRs), object detection events from vision cameras, and state feedback from robotic arms are organized into position event sequences, perception event sequences, and execution event sequences, respectively.

[0081] The unified data platform layer 120 abstracts key features that significantly impact decision-making from event sequences as core factors, constructing multiple core factor matrices. For state data such as speed and position, numerical values ​​are used as core factors. For sensor data such as images, pre-trained convolutional neural networks (CNNs) extract feature vectors as core factors. For text commands, Natural Language Processing (NLP) models extract intent embedding vectors as core factors. Core factors include, but are not limited to, spatial location factors, motion state factors, task execution state factors, and environmental obstacle factors. Taking spatial location factors as an example, a T×N×3 location factor matrix is ​​constructed with a time step of 100 milliseconds, where T represents the number of time steps, N represents the total number of currently online robots and key sensor nodes, and 3 represents the three-dimensional spatial coordinates.

[0082] Each data node is treated as a vertex in a graph. Multiple core factor matrices are spliced ​​together in the feature dimension or fused through a fully connected network to form a temporary global factor tensor. The dynamic correlation and evolution of all data nodes in the time dimension are captured by a Long Short-Term Memory (LSTM) network. Edge connections are established based on physical proximity or task relevance, and the feature vectors of each factor are fused to generate a global data feature map.

[0083] The task chain is represented as a Directed Acyclic Graph (DAG), which contains multiple ordered task nodes. Each node represents the specified execution subject, action type, and execution parameters of an atomic task, while the edges represent the temporal and logical dependencies between tasks. An atomic task is an indivisible basic operation unit, such as moving to coordinates (X, Y, Z), grabbing workpiece A, or performing assembly operation B.

[0084] The intelligent central brain layer 130 receives the global data feature map, analyzes the task objective, and identifies the sub-tasks to be executed, including path planning, cargo identification, grasping, transportation, and placement. Based on information such as the robot's current position, battery status, path congestion, and robotic arm availability reflected in the global data feature map, the intelligent central brain is trained by interacting with the simulation environment or the actual system. It models the task planning as a mixed-integer linear programming optimization problem, with the goal of minimizing the total task time and energy consumption. Using the global data feature map as the state input, it solves for the optimal task chain. Combining predefined task templates and business rules, it dynamically instantiates and sorts the task chain according to the current global features.

[0085] Furthermore, if a robot reports an anomaly during task execution, the intelligent central brain layer 130 will update the global data feature map in real time and trigger dynamic replanning of the task chain to reallocate tasks or adjust the execution order, ensuring the robust completion of the overall task.

[0086] The multi-robot execution layer 140 converts the corresponding task instructions into underlying control commands supported by the robot based on the execution entity information in the task chain's task nodes. These task instructions are then sent to the target robot via the corresponding protocol channel in the multi-protocol access layer. Simultaneously, the multi-robot execution layer 140 continuously monitors the status feedback information of each robot, including task execution progress, completion status, and fault codes. This status information is encapsulated into new data packets and re-input to the multi-protocol access layer 110, forming a closed-loop feedback mechanism for the unified data platform layer to update the global data feature map.

[0087] According to the three-way, one-brain robot collaborative control architecture system provided in this application embodiment, the multi-protocol access layer parses data packets from different data nodes and adds extended headers to generate time-enhanced data frames, enabling subsequent processing to more effectively utilize information in the time dimension. The unified data platform layer parses the time-enhanced data frames generated by the multi-protocol access layer, extracts event sequences, and converts them into a core factor matrix, improving the depth and accuracy of data analysis. The intelligent central brain layer performs task planning based on global data feature maps, generating task chains for different robots. It can optimize task allocation according to the current environment and robot status, improving task execution efficiency and resource utilization, while ensuring the quality of task completion. The multi-robot execution layer issues task instructions to the corresponding robots and collects their task execution status feedback, ensuring accurate instruction transmission and real-time monitoring of execution status, which helps to adjust strategies in a timely manner to deal with emergencies. Through unified access, dynamic modeling, intelligent planning, and closed-loop execution of heterogeneous robot devices, the system can effectively integrate the event evolution relationship in the time dimension, improve the collaborative efficiency and task robustness of the multi-robot system in complex dynamic environments, ensure the continuity and reliability of task execution, and also promote the convenience of human-computer interaction.

[0088] In some embodiments, the multi-protocol access layer 110 includes a data access module, a buffer management module, and a timing enhancement module connected in sequence;

[0089] The data access module is used to receive data packets from the multiple data nodes;

[0090] The buffer management module is used to store the data packet into a circular buffer, call the dynamic protocol parsing engine to parse the data packet, extract the fingerprint features of the data packet, determine the node type of the data node based on the fingerprint features, and associate the node type with the data packet. The node type includes synonymous nodes and heteronymous nodes.

[0091] If the data node is a synonym node, the data packet is stored in the synonym node pool in the circular buffer;

[0092] In the case where the data node is the heteronym node, a buffer partition is divided in the circular buffer according to the priority label of the data node and the network load to store the corresponding data packet;

[0093] The timing enhancement module is used to read the data packet from the circular buffer and add the extended header to the data packet according to the node ID of the data node, the sliding window number of the time window, and the comparison logic clock.

[0094] Understandably, a circular buffer is a circular memory queue that uses the first-in-first-out (FIFO) principle and serves as the central data pool for the entire module.

[0095] In actual implementation, in the multi-protocol access layer 110, the data access module contains a set of network service instances, each instance listening to a specific network port or topic to receive connection requests and data streams from the corresponding data node.

[0096] The dynamic protocol parsing engine is a pluggable and scalable software engine used for preliminary protocol detection and feature extraction of data packets. It has a built-in fingerprint library of various industrial protocols such as ROS, MODBUS, and PROFINET.

[0097] Fingerprint features include data packet length distribution, key field offset positions, verification methods, and protocol keywords.

[0098] Node types are classified based on the semantics of the data source and the system resource requirements.

[0099] Synonymous nodes are device nodes that generate high-frequency, small-volume data, have similar data content structures or can be processed uniformly, and have the same or interchangeable functional semantics. For example, multiple AMRs of the same model are all used for material handling.

[0100] Heterogeneous nodes are device nodes that generate low-frequency, large-volume data, or have unique protocols and data structures, different functional semantics, and are not interchangeable. For example, an AMR and a six-axis robotic arm, or a vision camera and a temperature and humidity sensor.

[0101] The buffer management module temporarily stores data packets in a preset circular buffer and calls the dynamic protocol parsing engine to parse the data packets in real time. The dynamic protocol parsing engine extracts fingerprint features based on the byte structure, field distribution and communication characteristics of the data packets; and identifies the node type of the data node based on the matching results of the fingerprint features in the fingerprint database.

[0102] The synonym node pool is a logically contiguous memory area with a unified data format, which facilitates batch and efficient subsequent parsing operations and reduces memory fragmentation and context switching overhead.

[0103] When a data node is determined to be a synonym node, the buffer management module stores the corresponding data packet in a pre-defined synonym node pool in the circular buffer. The synonym node pool uses a first-in-first-out strategy for unified scheduling to improve the throughput efficiency of data processing for similar devices.

[0104] When a data node is determined to be a heterodox node, the buffer management module dynamically divides the circular buffer into independent buffer partitions based on the pre-configured priority label of the data node and the current network load status. These partitions are used to store the data packets of the heterodox node. The size and scheduling weight of the buffer partitions are adaptively adjusted according to the network load to ensure low-latency processing of high-priority heterodox node data such as emergency stop signals and obstacle alarms.

[0105] The sliding window sequence number is used to identify the time window sequence number to which the data packet belongs, and is used for subsequent timing alignment;

[0106] The logical clock value is a comparison logical clock generated based on the Lamport logical clock or vector clock mechanism. It is used to establish partial order relationships of events in a distributed environment and resolve timing ambiguities caused by physical clock asynchrony.

[0107] The timing enhancement module reads classified data packets from the circular buffer according to the scheduling policy and adds an extension header to each data packet to generate a timing enhancement data frame. The extension header includes at least the node ID, sliding window number, and logical clock value.

[0108] In this embodiment, the node type of data nodes is determined by data packets, and the data packets are stored separately. This not only realizes unified access and classified caching of heterogeneous protocol data, but also improves the timeliness of processing high-priority and key heterogeneous data through semantic recognition and dynamic buffering strategies, and provides standardized input with strong temporal semantics for the subsequent unified data platform layer.

[0109] In some embodiments, the buffer management module is further configured to:

[0110] The fingerprint features are converted into semantic particles. The fingerprint features include field names, data types, and numerical ranges. The semantic particles include semantic vectors, field strength coefficients, and spin directions.

[0111] Based on the semantic particles, a local semantic field is constructed for each data node. The interaction relationships between the semantic particles in the local semantic field include semantic similarity, functional complementarity, and semantic conflict.

[0112] Calculate the field strength interference mode and wave function overlap between the local semantic fields corresponding to the data nodes;

[0113] Based on the field strength interferometry mode and the wave function overlap, the local semantic fields are dimensionally fused to determine the field distance between the local semantic fields. The field distance includes semantic distance, functional distance, temporal distance and causal distance.

[0114] The node type of the data node is determined based on the field distance.

[0115] It is understandable that semantic particles are basic information units obtained by mathematically and vectorizing fingerprint features.

[0116] Semantic vectors are vector representations generated based on field names and data types using a pre-trained Word2Vec semantic embedding model, used to characterize the functional semantics of the field;

[0117] The field strength coefficient is a weight value assigned based on the importance of the field in device control or sensing;

[0118] Spin direction is used to identify the temporal behavior characteristics of a field in a data stream. For example, continuous output types are marked as positive spin, and event-triggered types are marked as negative spin.

[0119] In actual implementation, the buffer management module is further used to perform deep semantic identification and classification of data nodes to improve the accuracy and intelligence of node type determination.

[0120] After extracting the fingerprint features of data packets, the buffer management module converts these fingerprint features into structured semantic representation units, which serve as semantic particles.

[0121] A local semantic field is a dynamic image region in a multidimensional semantic space composed of semantic particles corresponding to all fields of a data node, used to describe all semantic characteristics and internal structure of the data node.

[0122] Semantic similarity is used to characterize that the angle between the semantic vectors of two semantic particles is less than a threshold; functional complementarity is used to characterize that the spin directions of two semantic particles are opposite and that the semantic vectors satisfy a certain correlation; semantic conflict is used to characterize that the spin directions of two semantic particles are the same, but the semantic vectors indicate that the two semantic particles represent mutually exclusive states.

[0123] For a given data node, the buffer management module places all its semantic particles in the same vector space. Each particle exerts an influence based on the field strength coefficient. The semantic vector determines the center of the influence, and the spin direction determines the polarity of the influence. The resulting local semantic field is the superposition of the semantic potential fields generated by all particles.

[0124] Field strength interferometry is the result of superimposing the potential functions of two fields at various points in vector space. It includes constructive interferometry and destructive interferometry. The region of constructive interferometry indicates that the field strength is enhanced after the superposition of the two nodes, and the functions can work together. The region of destructive interferometry indicates that the field strength is weakened or canceled after superposition. It is achieved by adding the potential function matrices of the two fields and analyzing their extreme points.

[0125] Wavefunction overlap is used to calculate the degree to which two local semantic fields coexist or are confused in a vector space.

[0126] Furthermore, the buffer management module performs inter-field interaction analysis on the local semantic fields corresponding to any two data nodes. Based on the field strength coefficients and spatial distribution of each semantic particle, it evaluates the interference effect of the two local semantic fields on semantic intensity using a weighted superposition method to obtain the field strength interference mode. Each local semantic field is modeled as a high-dimensional probability distribution, and the similarity between the two semantic fields is quantified by calculating the overlap integral of their joint distribution to obtain the wave function overlap.

[0127] Field distance is used to characterize the relationship between nodes from multiple dimensions.

[0128] Semantic distance is determined by the overlap of wave functions and directly reflects the similarity of the meaning of the data.

[0129] Functional distance is determined by the ratio of constructive to destructive interference in the field strength interferometry mode, reflecting functional cooperation or competition.

[0130] Time distance is derived by analyzing the correlation of the arrival time series of data packets between data nodes, reflecting the time coupling relationship of data nodes in actual operation.

[0131] Causal distance is determined by using the Granger causality test to analyze whether the data from one data node helps predict the data from another data node, thereby identifying potential causal relationships.

[0132] Based on the overlap of field strength interferometry and wave function, the buffer management module performs dimensional fusion on multiple local semantic fields to generate a unified semantic comparison space, and calculates the field distance between any two data nodes in this space.

[0133] Data nodes are clustered or classified based on field distance. If the combined field distance between two data nodes is less than the preset synonym threshold, they are determined to be synonym nodes. If the field distance is greater than the dissimilar threshold, or there are significant differences in the dimensions of functional distance or causal distance, they are determined to be dissimilar nodes.

[0134] In this embodiment, by leaping from raw protocol data to high-level semantic understanding, the determination of node type not only depends on the protocol format, but also realizes the semantic role and functional positioning of devices in collaborative tasks, enabling more accurate and adaptive data flow management and resource allocation.

[0135] In some embodiments, the multi-protocol access layer further includes a time alignment module, the time alignment module being configured to:

[0136] Invoke the time-enhanced data frame and extract the timestamp;

[0137] If the timestamp includes a physical timestamp, the physical timestamp is matched with a time window to generate an index table, which is used to record the data packets received within each time window;

[0138] When the timestamp includes a PTP synchronization timestamp, the event units in the timing enhancement data frame are time-aligned.

[0139] If the timestamp package does not include the PTP synchronization timestamp, the event units within the time window are time-aligned according to the index table and the physical timestamp.

[0140] Understandably, timestamps vary depending on the clock mechanism of the data source device, including physical timestamps or PTP synchronization timestamps.

[0141] An event unit is the smallest data object with independent business semantics that is parsed from a time-enhanced data frame.

[0142] The physical timestamp is the actual hardware time when the data packet arrives. The Precision Time Protocol (PTP) synchronizes timestamps to achieve time synchronization across data nodes through a precision clock protocol. The time alignment module ensures the consistency of event units in the spatiotemporal dimension through a dual timestamp mechanism, achieving high-precision time synchronization processing to ensure that multi-source heterogeneous data has a consistent time reference in subsequent processing.

[0143] In actual execution, the time alignment module calls the timing enhancement data frame and extracts the embedded timestamp from it.

[0144] A time window is a series of continuous, fixed-duration, fixed or sliding time intervals that divide a continuous time stream into segments for segmented management of the continuous data stream. Each window has a unique identifier, such as a start timestamp or sequence number.

[0145] The index table is represented by a data structure such as a hash table or a database table, with the key being a time window identifier and the value being a list of metadata for all time-enhanced data frames received within that time window.

[0146] For event units that contain both physical timestamps and PTP synchronization timestamps, the PTP synchronization timestamp is used for alignment first. For event units that lack PTP synchronization timestamps, the time window attribution is determined by the mapping relationship between the physical timestamp and the index table. The generated time alignment result will serve as the input benchmark for the event sorting module, ensuring the continuity and accuracy of the global data feature map on the time axis.

[0147] When the timestamp is a physical timestamp, the time alignment module matches the physical timestamp with a preset time window. Based on the matching result, the time alignment module generates an index table, which records the identification information of all data packets received within each time window, including the data packet ID, source node ID, and original timestamp.

[0148] When the timestamp is a PTP synchronization timestamp, it indicates that the data packet comes from a high-precision device that has been connected to the PTP time synchronization network. The time alignment module directly performs precise time alignment of the event units in the time-enhanced data frame based on the PTP synchronization timestamp, without relying on external indexes.

[0149] When the timestamp does not contain a PTP synchronization timestamp and only contains a regular physical timestamp or no high-precision synchronization information, the time alignment module combines the index table and the physical timestamp to perform intra-window time alignment on multiple event units within the same time window. Taking the start time of the time window as the reference, the physical timestamps of all event units within the window are normalized and offset corrected. Then, an interpolation or nearest neighbor alignment strategy is used to map asynchronously arriving events onto a unified time grid. Logical timestamps are assigned to event units that do not contain PTP synchronization timestamps, thereby achieving logical time consistency between devices that lack PTP support.

[0150] In this embodiment, the time alignment module is compatible with both high-precision synchronous devices and ordinary asynchronous devices. It supports microsecond-level alignment under PTP networks and can achieve logical timing consistency through time windows and index tables in PTP-free environments, effectively solving the problem of data timing misalignment caused by clock heterogeneity in multi-robot systems.

[0151] In some embodiments, the unified data platform layer includes a data parsing module, an event sorting module, a tensor decomposition module, and a matrix aggregation module connected in sequence.

[0152] The data parsing module is used to retrieve the time-series enhanced data frame and parse the extended header to obtain the timestamp in the time-series enhanced data frame, and determine the time slot number of the event unit based on the timestamp. The event unit is divided based on the event content of the time-series enhanced data frame.

[0153] The event sorting module is used to sort the event units according to the time slot number and generate the event sequence;

[0154] The tensor decomposition module is used to construct a higher-order event tensor from the event sequence, perform random singular value decomposition on the higher-order event tensor, and generate a core factor matrix for each data node. The higher-order event tensor includes the time dimension, node dimension, and attribute dimension of the event unit.

[0155] The matrix aggregation module is used to aggregate the core factor matrix to generate the global data feature map.

[0156] Understandably, the time slot numbers are monotonically increasing integers used to identify unique time slots. A continuous timeline is divided into a series of extremely short, consecutive, and non-overlapping time slots, each with a duration much shorter than the time window in a multi-protocol access layer.

[0157] In actual implementation, within the unified data platform layer, high-dimensional data representations supporting intelligent decision-making are generated by performing structured parsing, temporal organization, feature extraction, and global fusion on time-enhanced data frames.

[0158] The data parsing module retrieves the timing-enhanced data frame and parses its extended header to extract the timestamps contained therein. It prioritizes extracting the PTP synchronization timestamp. If the data frame does not contain a PTP synchronization timestamp, it uses the logical timestamp assigned by the time alignment module as a substitute.

[0159] Based on the timestamp, the data parsing module divides the payload in the time-enhanced data frame into one or more event units in one step. When a time-enhanced data frame is received, the node ID and protocol type are extracted from its extended header to load the corresponding protocol pattern description file from the protocol pattern library. The protocol pattern description file is a structured document used to define the binary or serialized format of the payload of this type of data frame. According to the loaded protocol pattern description file, the payload of the time-enhanced data frame is parsed. While parsing the data structure, the boundaries of the event units are identified according to the predefined business semantic annotations in the protocol pattern description file and the event content. For each identified and divided event unit, a corresponding standardized internal data structure is created, including an event header and an event body. The event header includes the node ID, timestamp, event type, and event ID. The event body includes all field data related to the event unit parsed from the payload. Each event unit corresponds to an atomic-level perception or control semantic.

[0160] The data parsing module maps timestamps to corresponding time slot numbers according to the preset time slot granularity, which serves as the positioning identifier of the event unit on the global timeline.

[0161] Event sequences are used to describe the order and concurrency of events on different nodes in the entire robot cluster at a micro time scale, as well as the temporal correlation between events across devices.

[0162] The event sorting module receives all event units and their corresponding time slot numbers, and sorts the event units in ascending order according to the time slot numbers to generate an event sequence organized in chronological order.

[0163] Higher-order event tensors are represented as multidimensional arrays. In a higher-order event tensor, the time dimension corresponds to consecutive time slot numbers, representing the dynamic evolution of the event; the node dimension corresponds to different data nodes, representing the data sources at different spatial locations; and the attribute dimension corresponds to different data fields in the event unit, representing different types of observations.

[0164] The core factor matrix compactly represents the dynamic behavior characteristics of data nodes in the time dimension and their coupling relationship with the attribute dimension. It is used to capture the core dynamic behavior characteristics of data nodes within a specified time window, filtering out noise and redundant information.

[0165] The tensor decomposition module uses a fixed-length time slot sequence (i.e., a time window) as a unit to construct a high-order event tensor from the data in the event sequence. It then uses the Randomized Singular Value Decomposition (Randomized SVD) algorithm for dimensionality reduction and feature extraction, generating a low-rank core factor matrix for each data node.

[0166] The matrix aggregation module, based on the topological relationships, task relevance, or semantic similarity between nodes, employs weighted fusion, graph convolution, or attention mechanisms to aggregate multiple core factor matrices into a unified global data feature map. This global data feature map comprehensively depicts the state evolution, interdependence, and collaborative dynamics of each data node in the system over time in a structured form, serving as input for task planning in the intelligent central brain layer.

[0167] In this embodiment, the unified data platform layer realizes the automatic conversion from raw time-series data to high-dimensional semantic features, and has the capabilities of high-precision time alignment, efficient tensor computation and global semantic fusion, effectively supporting the real-time perception and intelligent decision-making of complex multi-robot systems.

[0168] In some embodiments, the matrix aggregation module is further configured to:

[0169] The correlation between the core factor matrices is modeled by a time-varying graph attention network to determine the dynamic interaction characteristics between the data nodes in the time dimension.

[0170] Based on the dynamic interaction features, multiple core factor matrices are aggregated to generate a time-varying system embedding vector;

[0171] The global data feature map is constructed based on the time-varying system embedding vector. The global data feature map includes a horizontal dimension and a vertical dimension. The horizontal dimension represents the attribute features of the data node, and the vertical dimension is used to represent the state evolution of the data node in the time series.

[0172] Understandably, the Hybrid Time-Varying Graph Neural Network (HTGNN) is a hybrid neural network structure that integrates graph neural networks and time series modeling capabilities, used to jointly model the topological relationships and feature propagation of multi-node systems at continuous time steps.

[0173] Dynamic interaction features are high-dimensional feature vectors extracted from multiple core factor matrices by HTGNN, reflecting the mutual influence and collaborative relationships between data nodes in the time dimension. They not only contain the independent state of a data node at a given moment but also encode its dependency on the states of other data nodes at historical moments. For example, robot A's path planning at time t is influenced by robot B's path planning at time t. The influence of position at time 1.

[0174] In actual implementation, the matrix aggregation module treats each data node as a vertex in a graph using HTGNN. The edge weights are dynamically generated based on semantic similarity, physical proximity, or task dependency between nodes. An attention mechanism is introduced to adaptively learn the importance of different neighboring nodes at different time steps, thereby determining the dynamic interaction characteristics between data nodes over time. Both node connectivity and node features are dynamically updated over time, enabling the capture of non-stationary interaction patterns in the system.

[0175] The embedding vector of a time-varying system is a compact vector representation of the global state of the entire system within a certain time window.

[0176] HTGNN aggregates the core factor matrices of all nodes using weighted summation, max pooling, or attention weighting, with fixed dimensions, to generate a time-varying system embedding vector. This vector is dynamically updated as the time window slides, continuously reflecting the latest state of the system.

[0177] The time-varying system is embedded into vectors and reconstructed into a two-dimensional matrix using a deconvolutional network, which serves as a global data feature map.

[0178] In the global data feature map, the horizontal dimension corresponds to the attribute feature dimension of the data node. Each column represents a data node, and the elements in the column are embedded with multi-dimensional attributes such as its position, speed, task status, and environmental awareness in the current time window. The vertical dimension corresponds to the time series dimension. Each row represents a time slot, and the elements in the row reflect the state snapshot of all nodes at that moment.

[0179] In this embodiment, the matrix aggregation module realizes the static fusion of the core factor matrix. By introducing a time-varying graph attention network, it deeply mines the dynamic and complex interactions within the robot cluster and explicitly models the interaction relationships between data nodes that evolve over time. This generates a global data feature map with strong spatiotemporal expressive capabilities, which not only includes the individual attributes of each robot but also contains the collaborative relationships and temporal evolution patterns between robots. This significantly improves the perception and understanding capabilities of the multi-robot collaborative control system for complex dynamic environments and provides extremely rich and accurate situational awareness information.

[0180] In some embodiments, the time-varying graph attention network includes a time-aware graph construction layer, a multi-scale temporal convolutional layer, a dynamic graph attention layer, a spatiotemporal fusion module, and a dynamic interaction feature extraction layer connected in sequence.

[0181] The time-aware graph construction layer is used to calculate the basic similarity between the core factor matrices based on time windows, construct multiple static relationship networks, analyze the temporal correlation of the static relationship networks, and construct a dynamic relationship graph.

[0182] The multi-scale temporal convolutional layer is used to extract multi-timescale features from the dynamic relationship graph and filter temporal noise through temporal gating to obtain multi-timescale features;

[0183] The dynamic graph attention layer is used to allocate attention to features at multiple time scales in the time dimension. The temporal attention layer extracts the temporal features from the features at multiple time scales, and the structural attention layer extracts the structural correlation strength between data nodes to generate spatial structural features.

[0184] The spatiotemporal fusion module is used to fuse the temporal features and the spatial structure features to generate an embedding vector;

[0185] The dynamic interaction feature extraction layer is used to extract the dynamic interaction features between the data nodes in the time dimension based on the embedding vector.

[0186] Understandably, basic similarity is a scalar index that quantifies the consistency of the orientation of the state vectors of any two data nodes in the feature space at a specific moment. It can be obtained by calculating the cosine similarity between the core factor vectors of two nodes at the same moment. It is used to preliminarily judge the degree of instantaneous association between the states of data nodes. The value range is [-1, 1]. The closer the value is to 1, the more similar the behavior patterns of the two data nodes are at that moment.

[0187] A static relationship network is a graph structure representation formed at a single discrete time point after filtering based on a basic similarity threshold. The network uses data nodes as vertices and connections between data nodes whose basic similarity exceeds a preset threshold at a specific time as edges, forming a static topological structure that describes the instantaneous association between data nodes at that moment.

[0188] Dynamic relational graphs are time-extended graph models that integrate information from multiple static relational networks over a time series. They include nodes and edges, and assign time-series functions to the edges to represent the evolution of relational strength over time. They can describe the stability, trends, and temporal patterns of the relationships between nodes, and fully characterize the dynamic evolution of the system's interactive structure.

[0189] In actual implementation, the time-aware graph construction layer calculates the basic similarity between core factor matrices based on a preset time window, and constructs multiple static relationship networks. By evaluating the changes of the static relationship networks at continuous time points, it analyzes the temporal correlation of the static relationship networks, identifies and connects data nodes with significant temporal correlations, and constructs a dynamic relationship graph that reflects the evolution of the system state.

[0190] Multi-timescale features are sets of features with different temporal granularities extracted from the time-series data of nodes or systems by applying convolutional kernels of different time window sizes in parallel. These features capture system behavior patterns at different time scales, such as short-term fluctuations, medium-term rhythms, and long-term trends, and together constitute a multi-resolution description of system dynamics.

[0191] Time gating is a trainable mechanism in a gated recurrent unit (GRU). Through structures such as update gates and reset gates, it adaptively controls the fusion ratio of historical information and current input, selectively retaining important historical states and filtering irrelevant information, effectively managing the information flow in time series.

[0192] Temporal noise refers to random fluctuations or short-term variations that are not predictive in time series data. Noise may originate from sensor errors, communication interference, or random environmental factors. It can mask the true temporal patterns and regularities in the data and interfere with the model's learning of long-term dependencies.

[0193] Multi-timescale features refer to the set of multi-timescale features filtered and purified by a time-gating mechanism. While retaining the original multi-scale information, the time-series noise is removed through the gating mechanism, which enhances the robustness and information density of the time-series representation and provides high-quality input for subsequent attention allocation.

[0194] Multi-scale temporal convolutional layers use convolutional kernels of different sizes to capture various temporal dependency patterns from short-term to long-term dynamic relationship graphs, extract features containing information at different time scales, and use a temporal gating mechanism to adaptively and selectively transmit important information based on the temporal characteristics of the input signal, filter out noise signals, and improve the quality of feature representation.

[0195] Attention allocation dynamically determines the importance of different elements in the input data to the current task through learnable weight coefficients. In temporal and graph structures, it is applied to the time dimension and node dimension, respectively, to achieve selective information focusing.

[0196] The temporal attention layer is an attention mechanism component that operates on the time dimension. By calculating the attention weight of each time step in the time series, it evaluates the differences in importance of different times to the overall behavior of the nodes, thereby generating a weighted temporal summary that highlights the impact of key time points.

[0197] Temporal features are node feature vectors obtained by weighted aggregation through a temporal attention layer, used to characterize the temporal importance of each data node.

[0198] The structural attention layer is an attention mechanism component that operates on the graph structure. Based on the topological connections of the graph, it calculates the attention weight between each node and its neighboring nodes, thereby quantifying the relative importance of different connections in the graph to the representation of the central node.

[0199] Structural association strength is an attention weight calculated through the structural attention layer. It is a scalar value used to quantify the degree of mutual influence between two connected nodes in the feature space. The higher the weight, the greater the feature contribution of the source node to the target node, and the stronger the structural association between them.

[0200] Spatial structural features are feature vectors obtained by aggregating the information of each node's neighbors through a structural attention layer. This vector integrates the features of the node itself and its topological neighbors, encoding the node's local connectivity patterns and location information within the graph structure.

[0201] The dynamic graph attention layer assigns attention weights to features at multiple time scales in the time dimension. By emphasizing or weakening the feature contributions at different times through the temporal attention layer, it enhances the model's ability to remember key time nodes and extracts temporal features. By quantifying the strength of relationships between nodes through the structural attention layer, it strengthens important local structural patterns and extracts spatial structural features.

[0202] The embedding vector is a low-dimensional dense vector representation generated by the spatiotemporal fusion module to characterize the comprehensive spatiotemporal state of a single data node within a specific time window.

[0203] The spatiotemporal fusion module nonlinearly combines temporal features and spatial structural features into a unified embedding vector that integrates information from both spatiotemporal aspects.

[0204] The dynamic interaction feature extraction layer identifies the interaction patterns between data nodes that evolve over time based on the generated embedding vectors, extracts the dynamic interaction features between data nodes in the time dimension, and maps them to a low-dimensional space through an autoencoder to obtain the final time-varying system embedding vector.

[0205] In this embodiment, a time-varying graph attention network is used to achieve comprehensive and detailed modeling of the dynamic interaction behavior of robot swarms from micro to macro and from time to space.

[0206] In some embodiments, the intelligent central brain layer includes a task planning module and a complexity control module connected in sequence;

[0207] The task planning module is used to perform task planning for multiple robots and generate a task chain based on the task objectives, the temporal evolution information of the global data feature map, the robot's state information, response preferences, and task allocation granularity. The state information includes the robot's current position, remaining battery power, and task type. The response preferences are determined by modeling historical task response data using entropy regularization terms.

[0208] The complexity control module is used to calculate the task execution complexity based on the task chain length, the node size of the data node, and the fluctuation range of the excitation trajectory, and to control the task execution complexity by adjusting the regularization coefficient.

[0209] Understandably, task objectives are used to define the specific tasks that need to be completed or the goals that need to be achieved.

[0210] Temporal evolution information is a dynamic feature extracted from the global data feature map of continuous time windows, which describes the changing trend of the overall state and internal patterns of the system over time, and provides background information on environmental changes.

[0211] State information is a set of data describing the real-time physical state and operational context of a single robot at a specific moment.

[0212] Response preference is a quantitative description of the stable and efficient behavioral tendencies exhibited by a particular robot when faced with different tasks, used to capture the tendency of each robot to handle different types of tasks.

[0213] The entropy regularization term is a constraint added to the loss function of a machine learning model, and its value is proportional to the entropy of the model's output probability distribution.

[0214] Historical task response data is a collection of records accumulated by the system in the past regarding the robot's task execution.

[0215] Task allocation granularity refers to the size or level of detail in which a macro-level task is broken down into the smallest working units that can be assigned to different robots during the task planning process.

[0216] The task planning module is based on mixed-integer linear programming. It generates a task chain based on the task objective, the temporal evolution information of the global data feature map, the robot's state information, response preferences, and task allocation granularity.

[0217] The task chain length is the total number of atomic task nodes contained in the task chain. The longer the task chain, the higher its complexity and resource requirements may be.

[0218] Node size refers to the total number of data nodes involved in executing a task chain. It reflects the degree to which the task chain consumes system resources and the number of entities that need to be coordinated and scheduled. Large-scale data nodes may lead to more complex interaction patterns and higher computing costs.

[0219] The fluctuation range of the excitation trajectory is a quantitative measure of the stability of the excitation signals generated within the system during the simulation or actual execution of the task chain. For example, the excitation signals can be action instructions, changes in the value function, logits of the policy network, etc.

[0220] Task execution complexity is a scalar indicator used to quantify the overall effort, coordination difficulty, and potential risks required to execute a given task chain. It is calculated by weighted fusion of multiple factors, such as task chain length, node size, and the fluctuation range of the incentive trajectory.

[0221] Regularization coefficients control the weight or influence of regularization terms in the objective function. By appropriately adjusting the regularization coefficients, resource utilization efficiency and task execution stability can be optimized while ensuring the quality of task completion.

[0222] The complexity control module calculates the overall complexity of task execution based on factors such as the length of the task chain, the size of the data nodes, and the fluctuation range of the stimulus trajectory, and controls the complexity of task execution by adjusting the regularization coefficient.

[0223] Let the first The robot identifies the task type in its historical tasks. The response probability is Specifically:

[0224]

[0225] in, Represents robots Types of tasks successfully executed in history Total number of times; The total number of task types; It's a robot. The total number of times all task types have been successfully executed in history. .

[0226] Response preferences are modeled using historical task response data through an entropy regularization term; the response preference entropy... for:

[0227]

[0228] In the task planning process, an entropy regularization term is introduced to encourage the assignment of tasks consistent with historical preferences. The objective function can be expressed as:

[0229]

[0230] in, This is a comprehensive cost item, including path length, power consumption, task latency, etc. For the current planning time of the robot Estimation of preference entropy; Currently involved in the planning The sum of the response preference entropy of each robot; For preference regularization coefficients, This is used to adjust the weight of preference consistency in planning; The total number of robots participating in the task.

[0231] The objective function is solved under constraints such as battery threshold and collision-free path to generate the optimal task chain.

[0232]

[0233] in, To execute the robot; For action; For time windows; It represents the total number of atomic tasks contained in the task chain.

[0234] The complexity control module defines the task execution complexity. for:

[0235]

[0236] in, The task chain length, which is the number of subtasks contained in the task chain. , The larger the value, the longer the workflow and the higher the coordination requirements; The data node scale refers to the total number of data nodes (robots) participating in the collaboration. , The larger the value, the greater the communication and scheduling overhead.

[0237] The fluctuation range of the excitation trajectory is defined as the sum of the variances of acceleration or direction changes in all robot path commands, used to quantify the stationarity of control commands:

[0238]

[0239] in, It's a robot. During its task execution period The internal acceleration command sequence, It is the first A robot in time Location, It is the first A robot in time speed, It is the first A robot in time The acceleration is directly related to the required control force / torque, energy consumption, smoothness of motion and mechanical shock, as well as the dynamic performance of trajectory tracking; The larger the value, the more demanding the task chain is on the robot's dynamic control, which may lead to higher energy consumption and instability. These are adjustable weighting coefficients. These are adjustable weighting coefficients, used to reflect the system's sensitivity to chain length, size, and dynamic stability, respectively.

[0240] To control complexity, a complexity regularization term is introduced, expanding the original objective function to:

[0241]

[0242] in, This is the complexity regularization coefficient, dynamically adjusted by the complexity control module based on system resource status (such as CPU load, communication bandwidth, and real-time requirements); when Exceeding the preset threshold At that time, increase This prompts the task planning module to generate shorter, smoother task chains with fewer nodes.

[0243] In this embodiment, the collaboration between the task planning module and the complexity control module enables detailed task planning and precise complexity management. This generates intelligent task chains that fully consider the robot's historical expertise, effectively coordinates cooperation between different robots, and takes into account various practical factors such as power limitations, location distribution, and individual preferences. By evaluating and controlling the complexity of the plan, the system pursues task efficiency while ensuring the success rate of task execution and the overall performance of the system.

[0244] In some embodiments, the multi-robot execution layer includes an instruction issuing module and an information receiving module that are interconnected.

[0245] The instruction issuing module is used to map the task chain to a low-dimensional latent space and generate task instructions corresponding to each robot. The task instructions include action instructions and path planning instructions.

[0246] The information receiving module is used to receive the status information generated by the robot in response to the task instruction, and generate a reward signal based on the status information. The status information includes task completion degree, energy consumption, and number of collisions.

[0247] Understandably, the low-dimensional latent space is a continuous and dense vector space learned by a variational autoencoder from a large number of historical successful task instructions.

[0248] Task instructions are used to control the robot to perform specific operations, such as opening and closing the gripper, starting the lifting platform, activating the sensor, etc.

[0249] Motion commands are the low-level control signals that drive the robot's joints or body, including the desired joint angle trajectory, end effector pose, linear velocity, and angular velocity commands.

[0250] Path planning instructions are used to guide mobile robots to move along an optimized trajectory, including parameters such as target pose, velocity constraints, obstacle avoidance area, and time window.

[0251] In some embodiments, in the multi-robot execution layer, the instruction issuing module is used to convert the abstract actions in the task chain into a point in a low-dimensional latent space through the encoder network, generate the underlying commands that each robot can execute as task instructions, each task instruction is uniquely bound to the corresponding robot, and is issued to the target device through the communication protocol channel adapted in the multi-protocol access layer.

[0252] Status information is quantitative evaluation data for the effectiveness of task execution.

[0253] Task completion rate represents the percentage of achievement of the current task's sub-objectives;

[0254] Energy consumption reflects the energy consumed during the execution process, such as the change in battery charge or the integral value of motor power consumption;

[0255] The number of collisions is the number of physical or virtual collision events that occur to the robot during task execution, and is used to assess path safety.

[0256] The reward signal is used to evaluate the execution quality of the current task chain and feeds back to the intelligent central brain layer 130 to evaluate the effectiveness of the current task allocation and planning strategy. It supports the iterative optimization of the dynamic replanning or long-term strategy optimization of the task chain in the task planning module, so that it can generate task chains that can obtain higher rewards in the future.

[0257] The information receiving module continuously receives status information from each robot during task execution. Based on this status information, it calculates rewards by considering task completion efficiency, energy efficiency, and safety performance according to a preset multi-objective evaluation function. For example, high task completion corresponds to a positive reward, while high energy consumption or collisions result in negative penalties.

[0258] In this embodiment, the multi-robot execution layer not only realizes the semantic conversion and reliable delivery of high-level tasks to low-level controls, but also constructs a closed-loop feedback mechanism with state information as input and reward signals as output, effectively supporting the system's adaptive collaboration and continuous optimization capabilities.

[0259] This application also provides a multi-robot collaborative control method, which is applied to the three-way-one-brain robot collaborative control architecture system in any of the above embodiments.

[0260] like Figure 2 As shown, the multi-robot cooperative control method includes:

[0261] Step 210: Receive data packets from multiple data nodes through the multi-protocol access layer, store the data packets in a circular buffer, parse each data packet and add an extended header to generate a timing-enhanced data frame corresponding to the data packet;

[0262] Step 220: Retrieve the time-series enhanced data frame through the unified data platform layer and parse the extended header to obtain the event sequence of the time-series enhanced data frame. Generate multiple core factor matrices based on the event sequence. Aggregate the multiple core factor matrices to generate a global data feature map. The global data feature map is used to describe the dynamic changes of each data node in the time dimension.

[0263] Step 230: Based on the global data feature map, the intelligent central brain layer performs task planning for the multiple robots and generates a task chain;

[0264] Step 240: Send the task instructions corresponding to the task chain to the corresponding robots through the multi-robot execution layer, and receive the status information of the robots in response to the task instructions to execute the tasks.

[0265] According to the multi-robot collaborative control method provided in this application, a multi-protocol access layer parses data packets from different data nodes and adds extended headers to generate time-enhanced data frames, enabling subsequent processing to more effectively utilize information in the time dimension. A unified data platform layer parses the time-enhanced data frames generated by the multi-protocol access layer, extracts event sequences, and converts them into a core factor matrix, improving the depth and accuracy of data analysis. An intelligent central brain layer performs task planning based on a global data feature map, generating task chains for different robots. This optimizes task allocation based on the current environment and robot status, improving task execution efficiency and resource utilization while ensuring the quality of task completion. A multi-robot execution layer issues task instructions to the corresponding robots and collects their task execution status feedback, ensuring accurate instruction transmission and real-time monitoring of execution, which helps to adjust strategies promptly to deal with emergencies. Through unified access, dynamic modeling, intelligent planning, and closed-loop execution of heterogeneous robot devices, the evolutionary relationships of events in the time dimension can be effectively integrated, improving the collaborative efficiency and task robustness of the multi-robot system in complex dynamic environments, ensuring the continuity and reliability of task execution, and also promoting the convenience of human-machine interaction.

[0266] The embodiments of this application have been described above with reference to the accompanying drawings. However, this application is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make many other forms under the guidance of this application without departing from the spirit and scope of the claims, and all of these forms are within the protection scope of this application.

[0267] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "illustrative embodiment," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of this application. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0268] Although embodiments of this application have been shown and described, those skilled in the art will understand that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of this application, the scope of which is defined by the claims and their equivalents.

Claims

1. A collaborative control architecture system for a three-way, one-brain robot, characterized in that, The system is connected to multiple robots, and the system includes a multi-protocol access layer, a unified data platform layer, an intelligent central brain layer, and a multi-robot execution layer connected in sequence. The multi-protocol access layer is used to receive data packets from multiple data nodes, parse each data packet and add an extension header to generate a timing-enhanced data frame corresponding to the data packet; The unified data platform layer is used to retrieve the time-series enhanced data frame and parse the extended header to obtain the event sequence of the time-series enhanced data frame, and generate multiple core factor matrices based on the event sequence. The multiple core factor matrices are aggregated to generate a global data feature map, which is used to describe the dynamic changes of each data node in the time dimension. The intelligent central brain layer is used to perform task planning for the multiple robots and generate task chains based on global data feature maps; The multi-robot execution layer is used to send the task instructions corresponding to the task chain to the corresponding robot, and to receive the status information of the robot in response to the task instructions in performing the task. The intelligent central brain layer includes a task planning module and a complexity control module connected in sequence; The task planning module is used to perform task planning for multiple robots and generate a task chain based on the task objectives, the temporal evolution information of the global data feature map, the robot's state information, response preferences, and task allocation granularity. The state information includes the robot's current position, remaining battery power, and task type. The response preferences are determined by modeling historical task response data using entropy regularization terms. The complexity control module is used to calculate the task execution complexity based on the task chain length, the node size of the data node, and the fluctuation range of the excitation trajectory, and to control the task execution complexity by adjusting the regularization coefficient.

2. The three-way, one-brain robot collaborative control architecture system according to claim 1, characterized in that, The multi-protocol access layer includes a data access module, a buffer management module, and a timing enhancement module connected in sequence. The data access module is used to receive data packets from the multiple data nodes; The buffer management module is used to store the data packet into a circular buffer, call the dynamic protocol parsing engine to parse the data packet, extract the fingerprint features of the data packet, determine the node type of the data node based on the fingerprint features, and associate the node type with the data packet. The node type includes synonymous nodes and heteronymous nodes. If the data node is a synonym node, the data packet is stored in the synonym node pool in the circular buffer; In the case where the data node is the heteronym node, a buffer partition is divided in the circular buffer according to the priority label of the data node and the network load to store the corresponding data packet; The timing enhancement module is used to read the data packet from the circular buffer and add the extended header to the data packet according to the node ID of the data node, the sliding window number of the time window, and the comparison logic clock.

3. The three-way-one-brain robot collaborative control architecture system according to claim 2, characterized in that, The buffer management module is further used for: The fingerprint features are converted into semantic particles. The fingerprint features include field names, data types, and numerical ranges. The semantic particles include semantic vectors, field strength coefficients, and spin directions. Based on the semantic particles, a local semantic field is constructed for each data node. The interaction relationships between the semantic particles in the local semantic field include semantic similarity, functional complementarity, and semantic conflict. Calculate the field strength interference mode and wave function overlap between the local semantic fields corresponding to the data nodes; Based on the field strength interferometry mode and the wave function overlap, the local semantic fields are dimensionally fused to determine the field distance between the local semantic fields. The field distance includes semantic distance, functional distance, temporal distance and causal distance. The node type of the data node is determined based on the field distance.

4. The three-way, one-brain robot collaborative control architecture system according to claim 2, characterized in that, The multi-protocol access layer further includes a time alignment module, which is used for: Invoke the time-enhanced data frame and extract the timestamp; If the timestamp includes a physical timestamp, the physical timestamp is matched with a time window to generate an index table, which is used to record the data packets received within each time window; When the timestamp includes a PTP synchronization timestamp, the event units in the timing enhancement data frame are time-aligned. If the timestamp package does not include the PTP synchronization timestamp, the event units within the time window are time-aligned according to the index table and the physical timestamp.

5. The three-way, one-brain robot collaborative control architecture system according to claim 1, characterized in that, The unified data platform layer includes a data parsing module, an event sorting module, a tensor decomposition module, and a matrix aggregation module connected in sequence. The data parsing module is used to retrieve the time-series enhanced data frame and parse the extended header to obtain the timestamp in the time-series enhanced data frame, and determine the time slot number of the event unit based on the timestamp. The event unit is divided based on the event content of the time-series enhanced data frame. The event sorting module is used to sort the event units according to the time slot number and generate the event sequence; The tensor decomposition module is used to construct a higher-order event tensor from the event sequence, perform random singular value decomposition on the higher-order event tensor, and generate a core factor matrix for each data node. The higher-order event tensor includes the time dimension, node dimension, and attribute dimension of the event unit. The matrix aggregation module is used to aggregate the core factor matrix to generate the global data feature map.

6. The three-way-one-brain robot collaborative control architecture system according to claim 5, characterized in that, The matrix aggregation module is further used for: The correlation between the core factor matrices is modeled by a time-varying graph attention network to determine the dynamic interaction characteristics between the data nodes in the time dimension. Based on the dynamic interaction features, multiple core factor matrices are aggregated to generate a time-varying system embedding vector; The global data feature map is constructed based on the time-varying system embedding vector. The global data feature map includes a horizontal dimension and a vertical dimension. The horizontal dimension represents the attribute features of the data node, and the vertical dimension is used to represent the state evolution of the data node in the time series.

7. The three-way-one-brain robot collaborative control architecture system according to claim 6, characterized in that, The time-varying graph attention network includes a time-aware graph construction layer, a multi-scale temporal convolutional layer, a dynamic graph attention layer, a spatiotemporal fusion module, and a dynamic interactive feature extraction layer connected in sequence. The time-aware graph construction layer is used to calculate the basic similarity between the core factor matrices based on time windows, construct multiple static relationship networks, analyze the temporal correlation of the static relationship networks, and construct a dynamic relationship graph. The multi-scale temporal convolutional layer is used to extract multi-timescale features from the dynamic relationship graph and filter temporal noise through temporal gating to obtain multi-timescale features; The dynamic graph attention layer is used to allocate attention to features at multiple time scales in the time dimension. The temporal attention layer extracts the temporal features from the features at multiple time scales, and the structural attention layer extracts the structural correlation strength between data nodes to generate spatial structural features. The spatiotemporal fusion module is used to fuse the temporal features and the spatial structure features to generate an embedding vector; The dynamic interaction feature extraction layer is used to extract the dynamic interaction features between the data nodes in the time dimension based on the embedding vector.

8. The three-way-one-brain robot collaborative control architecture system according to claim 1, characterized in that, The multi-robot execution layer includes interconnected instruction issuing modules and information receiving modules; The instruction issuing module is used to map the task chain to a low-dimensional latent space and generate task instructions corresponding to each robot. The task instructions include action instructions and path planning instructions. The information receiving module is used to receive the status information generated by the robot in response to the task instruction, and generate a reward signal based on the status information. The status information includes task completion degree, energy consumption, and number of collisions.

9. A multi-robot cooperative control method, characterized in that, The method, applied to the collaborative control architecture system of the three-way-one-brain robot as described in any one of claims 1-8, comprises: Data packets from multiple data nodes are received through a multi-protocol access layer, the data packets are stored in a circular buffer, and each data packet is parsed and an extended header is added to generate a timing-enhanced data frame corresponding to the data packet. The time-series enhanced data frame is retrieved by the unified data platform layer, and the extended header is parsed to obtain the event sequence of the time-series enhanced data frame. Based on the event sequence, multiple core factor matrices are generated. The multiple core factor matrices are aggregated to generate a global data feature map, which is used to describe the dynamic changes of each data node in the time dimension. Based on global data feature maps, the intelligent central brain layer performs task planning for the multiple robots and generates task chains. The multi-robot execution layer sends the task instructions corresponding to the task chain to the corresponding robots and receives the status information of the robots in response to the task instructions. The intelligent central brain layer includes a task planning module and a complexity control module connected in sequence; The task planning module performs task planning for multiple robots based on the task objectives, the temporal evolution information of the global data feature map, the robot's state information, response preferences, and task allocation granularity, generating a task chain. The state information includes the robot's current position, remaining battery power, and task type. The response preferences are determined by modeling historical task response data using entropy regularization. The complexity control module calculates the task execution complexity based on the task chain length, the node size of the data node, and the fluctuation range of the excitation trajectory, and controls the task execution complexity by adjusting the regularization coefficient.

Images