Three-way one-brain robot cooperative control architecture system and control method

Abstract

The invention discloses a three-way one-brain robot cooperative control architecture system and a control method, and belongs to the technical field of robot control. The system is connected with a plurality of robots, a multi-protocol access layer receives data packets of a plurality of data nodes, analyzes each data packet, adds an extension head, and generates a time sequence enhanced data frame corresponding to the data packet; the unified data platform layer calls the time sequence enhanced data frame, analyzes the extension header to obtain an event sequence of the time sequence enhanced data frame, and generates a plurality of core factor matrixes according to the event sequence; aggregating the plurality of core factor matrixes to generate a global data feature map; the intelligent center brain layer carries out task planning on a plurality of robots based on the global data feature map to generate a task chain; and the multi-robot execution layer issues the task instruction corresponding to the task chain to the corresponding robot, and receives state information that the robot executes the task in response to the task instruction. According to the method, the event evolution relationship in the time dimension can be effectively fused.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of robot control, and particularly relates to a three-in-one brain robot collaborative control architecture system and a control method. BACKGROUND

[0002] With the rapid development of intelligent manufacturing, smart logistics and unmanned operation scenarios, multi-robot collaborative systems are widely used in industrial automation, warehouse sorting, medical assistance and emergency rescue, etc. In practical applications, multiple heterogeneous robots are usually required to collaboratively complete complex tasks in a shared environment.

[0003] Traditional multi-robot systems usually adopt centralized or simple distributed architecture, and task scheduling relies on preset rules or static strategies. Existing systems usually process sensor data, robot state, etc. in isolation or statically aggregate, resulting in a lack of accurate perception of the overall dynamic behavior of the system for the upper layer decision, which is difficult to support complex collaborative tasks.

[0004] Therefore, there is an urgent need for a new collaborative control architecture that can effectively integrate the event evolution relationship in the time dimension. SUMMARY

[0005] The application aims to at least solve one of the technical problems existing in the prior art. To this end, the application provides a three-in-one brain robot collaborative control architecture system and a control method, which can effectively integrate the event evolution relationship in the time dimension.

[0006] In a first aspect, the application provides a three-in-one brain robot collaborative control architecture system, which is connected with multiple robots, and includes a multi-protocol access layer, a unified data platform layer, an intelligent central brain layer and a multi-robot execution layer connected in sequence. The multi-protocol access layer is configured to receive data packets of multiple data nodes, parse and add an extension header to each data packet, and generate a time series enhanced data frame corresponding to the data packet. The unified data platform layer is configured to call the time series enhanced data frame, parse the extension header to obtain an event sequence of the time series enhanced data frame, generate multiple core factor matrices according to the event sequence, aggregate the multiple core factor matrices to generate a global data feature map, and the global data feature map is used to describe the data dynamic change of each data node in the time dimension. The intelligent central brain layer is configured to perform task planning on the multiple robots based on the global data feature map, and generate a task chain. The multi-robot execution layer is configured to issue task instructions corresponding to the task chain to corresponding robots, and receive state information of the robots in response to the execution of the task instructions.

[0007] According to one embodiment of the present application, the multi-protocol access layer comprises a data access module, a buffer management module and a timing enhancement module connected in sequence; The data access module is configured to receive data packets of the plurality of data nodes; The buffer management module is configured to store the data packets in a ring buffer, call a dynamic protocol analysis engine to analyze the data packets, extract fingerprint features of the data packets, determine node types of the data nodes based on the fingerprint features, and associate the node types with the data packets, wherein the node types include synonymous nodes and non-synonymous nodes; In the case where the data nodes are synonymous nodes, the data packets are stored in a synonymous node pool in the ring buffer; In the case where the data nodes are non-synonymous nodes, buffer partitions are divided in the ring buffer according to priority labels of the data nodes and network load to store corresponding data packets; The timing enhancement module is configured to read the data packets from the ring buffer, add the extension header to the data packets according to node IDs of the data nodes, sliding window numbers of time windows, and comparison logic clocks.

[0008] According to one embodiment of the present application, the buffer management module is further configured to: convert the fingerprint features into semantic particles, wherein the fingerprint features include field names, data types and numerical value ranges, and the semantic particles include semantic vectors, field strength coefficients and spin directions; construct local semantic fields for each of the data nodes based on the semantic particles, wherein the interaction relationships between the semantic particles in the local semantic fields include semantic similarity, functional complementarity and semantic conflict; calculate field strength interference patterns and wave function overlap degrees between the local semantic fields corresponding to the data nodes; perform dimension fusion on the local semantic fields based on the field strength interference patterns and the wave function overlap degrees, determine field distances between the local semantic fields, and the field distances include semantic distances, functional distances, timing distances and causal distances; determine the node types of the data nodes according to the field distances.

[0009] According to one embodiment of the present application, the multi-protocol access layer further comprises a time alignment module, and the time alignment module is configured to: call the timing enhancement data frame to extract a timestamp; In a case where the timestamp comprises a physical timestamp, the physical timestamp is matched with a time window, and an index table is generated to record the data packets received within each time window; In a case where the timestamp comprises a PTP synchronization timestamp, event units in the time series enhanced data frame are time-aligned. In a case where the timestamp packet does not comprise a PTP synchronization timestamp, event units within the time window are time-aligned according to the index table and the physical timestamp.

[0010] According to an embodiment of the present application, the unified data platform layer comprises a data parsing module, an event sorting module, a tensor decomposition module and a matrix aggregation module connected in sequence; The data parsing module is configured to call the time series enhanced data frame, parse the extension header to obtain the timestamp in the time series enhanced data frame, determine the time slot number of the event unit according to the timestamp, and divide the event unit based on the event content of the time series enhanced data frame. The event sorting module is configured to sort the event units according to the time slot number to generate the event sequence. The tensor decomposition module is configured to construct a high-order event tensor for the event sequence, perform random singular value decomposition on the high-order event tensor to generate a core factor matrix of each data node, and the high-order event tensor comprises a time dimension, a node dimension and an attribute dimension of the event unit. The matrix aggregation module is configured to aggregate the core factor matrix to generate the global data feature map.

[0011] According to an embodiment of the present application, the matrix aggregation module is further configured to: Model the correlation between the core factor matrices through a time-varying graph attention network to determine the dynamic interaction features of the data nodes in the time dimension; Aggregate a plurality of core factor matrices based on the dynamic interaction features to generate a time-varying system embedding vector; Construct the global data feature map based on the time-varying system embedding vector, wherein the global data feature map comprises a horizontal dimension and a vertical dimension, the horizontal dimension represents the attribute features of the data nodes, and the vertical dimension represents the state evolution of the data nodes in the time sequence.

[0012] According to an embodiment of the present application, the time-varying graph attention network comprises a time-aware graph construction layer, a multi-scale time convolution layer, a dynamic graph attention layer, a space-time fusion module and a dynamic interaction feature extraction layer connected in sequence. The time-aware graph construction layer is configured to calculate a basic similarity between the core factor matrices based on a time window, construct a plurality of static relationship networks, analyze time correlation of the static relationship networks, and construct a dynamic relationship graph. The multi-scale time convolution layer is configured to extract multi-time scale features from the dynamic relationship graph, filter time sequence noise through time gating, and obtain multi-time scale features. The dynamic graph attention layer is configured to perform attention allocation on the multi-time scale features in the time dimension, extract time sequence features in the multi-time scale features through a time attention layer, extract structural correlation strength between data nodes through a structural attention layer, and generate spatial structure features. The space-time fusion module is configured to fuse the time sequence features and the spatial structure features to generate embedding vectors. The dynamic interaction feature extraction layer is configured to extract the dynamic interaction features between the data nodes in the time dimension based on the embedding vectors.

[0013] According to an embodiment of the present application, the intelligent hub brain layer includes a task planning module and a complexity control module connected in sequence. The task planning module is configured to perform task planning on a plurality of robots based on a task target and time sequence evolution information of the global data feature map, and state information, response preference, and task allocation granularity of the robots, to generate a task chain, the state information including a current position, remaining power, and execution task type of the robots, and the response preference being determined by modeling historical task response data through an entropy regularization term. The complexity control module is configured to calculate a task execution complexity according to a task chain length of the task chain, a node scale of the data nodes, and a fluctuation range of an incentive trajectory, and control the task execution complexity by adjusting a regularization coefficient.

[0014] According to an embodiment of the present application, the multi-robot execution layer includes an instruction issuing module and an information receiving module connected to each other. The instruction issuing module is configured to map the task chain to a low-dimensional hidden space to generate a task instruction corresponding to each of the robots, the task instruction including an action instruction and a path planning instruction. The information receiving module is configured to receive state information generated by the robots in response to the task instruction, generate a reward signal according to the state information, and the state information including a task completion degree, energy consumption, and collision frequency.

[0015] In a second aspect, the present application provides a multi-robot cooperative control method applied to the three-in-one brain robot cooperative control architecture system of the first aspect. receiving data packets of a plurality of data nodes through a multi-protocol access layer, storing the data packets into a ring buffer, and parsing each of the data packets and adding an extension header to generate a time-series enhanced data frame corresponding to the data packet; calling the time-series enhanced data frame through a unified data platform layer, parsing the extension header to obtain an event sequence of the time-series enhanced data frame, generating a plurality of core factor matrices according to the event sequence, and aggregating the plurality of core factor matrices to generate a global data feature map, the global data feature map being used to describe data dynamic changes of each of the data nodes in a time dimension; generating a task chain by planning tasks for the plurality of robots based on the global data feature map through an intelligent central brain layer; issuing task instructions corresponding to the task chain to corresponding robots through a multi-robot execution layer, and receiving state information of the robots in response to execution of the task instructions.

[0016] Additional aspects and advantages of the present application will be partially given in the following description, partially will become obvious from the following description, or will be learned by practicing the present application.

[0017] The three-way one-brain robot collaborative control architecture system and control method provided by the present application has the following beneficial effects compared with the prior art: (1) The multi-protocol access layer parses data packets from different data nodes and adds an extension header to generate a time-series enhanced data frame, so that subsequent processing can more effectively utilize information in the time dimension; the unified data platform layer parses the time-series enhanced data frame generated by the multi-protocol access layer, extracts an event sequence, and converts it into a core factor matrix, improving the depth and accuracy of data analysis; the intelligent central brain layer plans tasks based on the global data feature map to generate a task chain for different robots, which can optimize task allocation according to the current environment and robot state, improve task execution efficiency and resource utilization, and at the same time ensure the quality of task completion; the multi-robot execution layer issues task instructions to corresponding robots and collects their state feedback in executing the tasks, ensuring accurate communication of the instructions and real-time monitoring of the execution, which helps to adjust the strategy in a timely manner to cope with unexpected situations; through unified access, dynamic modeling, intelligent planning and closed-loop execution of heterogeneous robot devices, the event evolution relationship in the time dimension can be effectively integrated, the collaborative efficiency and task robustness of the multi-robot system in a complex dynamic environment are improved, the coherence and reliability of task execution are ensured, and the convenience of human-machine interaction is also promoted.

[0018] (2) Through the transition from the original protocol data to the high-level semantic understanding, the determination of the node type is not only dependent on the protocol format, the semantic role and the function positioning of the device in the collaborative task are realized, and the data flow management and the resource allocation are more accurate and more adaptive. BRIEF DESCRIPTION OF DRAWINGS

[0019] The above and / or additional aspects and advantages of the present application will become apparent and be readily appreciated from the description of the embodiments, taken in conjunction with the following drawings in which: Figure 1 FIG. 1 is a structural schematic diagram of a three-in-one brain robot collaborative control architecture system provided by an embodiment of the present application; Figure 2 FIG. 2 is a flow schematic diagram of a multi-robot collaborative control method provided by an embodiment of the present application. DETAILED DESCRIPTION

[0020] The technical solutions in the embodiments of the present application will be apparently and easily understood from the description of the embodiments of the present application in conjunction with the accompanying drawings. Obviously, the described embodiments are some embodiments of the present application, but not all the embodiments of the present application. All other embodiments obtained by those skilled in the art based on the embodiments of the present application belong to the scope of protection of the present application.

[0021] The terms "first", "second", and the like in the specification and claims of the present application are used to distinguish similar objects, and are not used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged under appropriate circumstances, so that the embodiments of the present application can be implemented in an order other than those illustrated or described herein, and the objects distinguished by "first", "second", etc. are usually a class, and are not limited to the number of objects, for example, the first object can be one or more. In addition, "and / or" in the specification and claims means at least one of the connected objects, and the character " / ", generally indicates that the front and rear associated objects are in an "or" relationship.

[0022] The three-in-one brain robot collaborative control architecture system and the multi-robot collaborative control method provided by the embodiments of the present application will be described in detail below in conjunction with the accompanying drawings and specific embodiments and application scenarios.

[0023] It can be understood that "three-in-one" in the three-in-one brain refers to the multi-protocol access layer, the unified data platform layer, and the multi-robot execution layer, and "one brain" refers to the intelligent central brain layer. When the system coordinates the work of multiple robots, centralized decision-making and task allocation are performed through the intelligent central brain layer, and other levels process data communication and execution.

[0024] As shown in FIG. 1, the three-in-one brain robot collaborative control architecture system provided by the embodiment of the present application includes a multi-protocol access layer, a unified data platform layer, a multi-robot execution layer, and an intelligent central brain layer. Figure 1As shown, the three-way brain robot collaborative control architecture system is connected with multiple robots, and the system comprises a multi-protocol access layer 110, a unified data platform layer 120, an intelligent central brain layer 130 and a multi-robot execution layer 140 connected in sequence; The multi-protocol access layer 110 is configured to receive data packets of multiple data nodes, parse each data packet and add an extension header to generate a time sequence enhanced data frame corresponding to the data packet. The unified data platform layer 120 is configured to call the time sequence enhanced data frame, parse the extension header to obtain an event sequence of the time sequence enhanced data frame, generate multiple core factor matrices according to the event sequence, aggregate the multiple core factor matrices to generate a global data feature map, and the global data feature map is used to describe data dynamic changes of each data node in a time dimension. The intelligent central brain layer 130 is configured to perform task planning on the multiple robots based on the global data feature map to generate a task chain. The multi-robot execution layer 140 is configured to issue task instructions corresponding to the task chain to corresponding robots and receive state information of the robots in response to the task instructions.

[0025] Among them, the multiple robots are autonomous mobile robots, mechanical arms, unmanned aerial vehicles, fixed sensors and other heterogeneous robots.

[0026] It can be understood that the data nodes are autonomous mobile robots, mechanical arms, visual sensors, laser radars, environmental monitoring devices and the like, and the communication protocol types are different, for example, ROS, Modbus TCP, CAN bus, MQTT, EtherCAT and the like.

[0027] The data packet is a raw binary stream or a serialized message unit transmitted on a physical network.

[0028] The extension header includes but is not limited to a physical timestamp, a device unique identifier, a data semantic type identifier and a task context identifier.

[0029] The time sequence enhanced data frame is a data structure containing time sequence information and other enhanced features.

[0030] In actual implementation, the multi-protocol access layer 110 is configured with multiple protocol adaptation modules corresponding to different types of access devices. For example, an autonomous mobile robot uploads its position, speed and battery status through the ROS 2 protocol; a robotic arm uploads joint angle and gripper status through the Modbus TCP protocol; a vision camera uploads detected cargo type and coordinate information through the Message Queuing Telemetry Transport (MQTT) protocol.

[0031] After receiving the node data packet, the multi-protocol access layer 110 identifies the communication protocol type corresponding to the data packet, calls the corresponding parser to perform structured analysis on the data packet, and adds an extension header of a uniform format to each data packet to enhance the time sequence of the data. The original data packet is encapsulated into a time sequence enhanced data frame and stored in a buffer queue. The data semantic type identifier includes pose, object detection result, power, etc., and the task context identifier includes the ID of the current execution task.

[0032] An event sequence is a collection of time sequence enhanced data frames generated by the same data node within a period of time and sorted by timestamp.

[0033] The core factor matrix is in the form of a compressed matrix, which is used to describe the state change of the data node and reveal and represent the potential core factor or hidden structure in the data.

[0034] The global data feature map is in the form of a situation map, where each pixel or region represents the coordination state, potential conflict or overall task progress of the entire robot cluster within a certain spatiotemporal range, representing the state evolution of all data nodes in the time dimension and their mutual dependence. For example, a certain channel in the feature map may highlight the workspace conflict area where two robots may occur.

[0035] The unified data platform layer 120 periodically retrieves the time sequence enhanced data frames from the buffer queue of the multi-protocol access layer 110, parses the extension header of each time sequence enhanced data frame, extracts node ID, timestamp and device identifier, etc., and classifies the data in the time sequence enhanced data frame into corresponding event categories according to the semantic type, forming an event sequence arranged in chronological order. For example, the position data from the autonomous mobile robot (AMR), the object detection event from the vision camera and the state feedback from the robotic arm are respectively organized into a position event sequence, a perception event sequence and an execution event sequence.

[0036] The unified data platform layer 120 abstracts characteristic quantities that are critical to decision-making from the event sequence as core factors, constructs multiple core factor matrices, wherein for state data such as speed and position, numerical values are taken as core factors, for sensor data such as images, a pre-trained convolutional neural network (CNN) is used to extract a feature vector as a core factor, and for text instructions, a natural language processing (NLP) model is used to extract an intent embedding vector as a core factor. The core factors include but are not limited to spatial position factors, motion state factors, task execution state factors, and environmental obstacle factors. Taking the spatial position factor as an example, a position factor matrix with a dimension of T x N x 3 is constructed with a time step of 100 milliseconds, wherein T represents the number of time steps, N represents the total number of robots and key sensor nodes currently online, and 3 represents three-dimensional spatial coordinates.

[0037] Each data node is regarded as a vertex in a graph, multiple core factor matrices are spliced or fused through a fully connected network in the feature dimension, forming a temporary global factor tensor, and a long short-term memory (LSTM) network is used to capture the dynamic association and evolution law of all data nodes in the time dimension, an edge connection is established according to physical proximity or task association, and the feature vectors of various factors are fused to generate a global data feature graph.

[0038] The task chain is represented as a directed acyclic graph (DAG) and includes multiple ordered task nodes, wherein a node represents a specified execution subject, action type, and execution parameter of an atomic task, and an edge represents the time sequence and logical dependency relationship between tasks. An atomic task is an indivisible basic operation unit, such as moving to coordinates (X, Y, Z), grabbing workpiece A, and performing assembly operation B.

[0039] The intelligent hub brain layer 130 receives the global data feature graph and parses the task target to identify the subtasks required to be executed, including path planning, cargo identification, grabbing, transportation, and placement. Based on the information such as the current position of the robot, the power state, the path congestion, and the availability of the mechanical arm reflected in the global data feature graph, the intelligent hub brain trains by interacting with the simulation environment or the actual system, models the task planning as an optimization problem of mixed integer linear programming, takes the global data feature graph as the state input, and solves the optimal task chain. The task chain is dynamically instantiated and sorted according to the current global features in combination with the predefined task templates and business rules.

[0040] In addition, during task execution, if a robot reports an exception, the intelligent central brain layer 130 will update the global data feature map in real time and trigger dynamic re-planning of the task chain to re-allocate tasks or adjust the execution order, ensuring robust completion of the overall task.

[0041] The multi-robot execution layer 140 converts the corresponding task instruction into a bottom-layer control command supported by the robot according to the execution subject information in the task node of the task chain, and the task instruction is issued to the target robot through the corresponding protocol channel in the multi-protocol access layer. At the same time, the multi-robot execution layer 140 continuously monitors the state feedback information of each robot, including task execution progress, completion status, fault code, etc. The state information is encapsulated as a new data packet and re-input to the multi-protocol access layer 110 to form a closed-loop feedback mechanism for the unified data platform layer to update the global data feature map.

[0042] According to the three-in-one brain robot collaborative control architecture system provided in the embodiments of the present application, the multi-protocol access layer analyzes the data packets from different data nodes and adds an extension header to generate a time-enhanced data frame, so that the subsequent processing can more effectively utilize the information in the time dimension; the unified data platform layer analyzes the time-enhanced data frame generated by the multi-protocol access layer, extracts the event sequence, and converts it into a core factor matrix, improving the depth and accuracy of data analysis; the intelligent central brain layer plans tasks based on the global data feature map, generates a task chain for different robots, and can optimize task allocation according to the current environment and robot state, improving task execution efficiency and resource utilization, while ensuring the quality of task completion; the multi-robot execution layer issues task instructions to corresponding robots and collects their state feedback for task execution, ensuring accurate communication of instructions and real-time monitoring of execution, which helps to adjust strategies in a timely manner to deal with unexpected situations; through unified access, dynamic modeling, intelligent planning and closed-loop execution of heterogeneous robot devices, the event evolution relationship in the time dimension can be effectively integrated, improving the collaborative efficiency and task robustness of the multi-robot system in a complex dynamic environment, ensuring the continuity and reliability of task execution, and also promoting the convenience of human-machine interaction.

[0043] In some embodiments, the multi-protocol access layer 110 includes a data access module, a buffer management module, and a time enhancement module connected in sequence; The data access module is configured to receive data packets of the plurality of data nodes; The buffer management module is configured to store the data packets in a ring buffer, call a dynamic protocol analysis engine to perform protocol analysis on the data packets, extract fingerprint features of the data packets, determine a node type of the data node based on the fingerprint features, and associate the node type with the data packets, wherein the node type includes synonymous nodes and non-synonymous nodes. In the case that the data node is the synonymous node, the data packet is stored in the synonymous node pool in the ring buffer; In the case that the data node is the synonymous node, the data packet is stored in the synonymous node pool in the ring buffer; The timing enhancement module is configured to read the data packet from the ring buffer, and add the extension header to the data packet according to the node ID of the data node, the sliding window number of the time window, and the comparison logic clock.

[0044] It can be understood that the ring buffer is a circular memory queue adopting the first-in first-out (FIFO) principle, and is used as a central data pool of the entire module.

[0045] In actual execution, in the multi-protocol access layer 110, the data access module includes a group of network service instances, each instance listens to a specific network port or topic to receive connection requests and data streams from corresponding data nodes.

[0046] The dynamic protocol analysis engine is a plug-in and extensible software engine for preliminary protocol detection and feature extraction of data packets, and has a built-in fingerprint library of multiple industrial protocols such as ROS, MODBUS, and PROFINET.

[0047] The fingerprint features include data packet length distribution, key field offset position, verification method, and protocol keyword.

[0048] The node type is classified based on the semantics of the data source and the system resource demand.

[0049] The synonymous node is a device node that generates high frequency, small data volume, and similar or unified data content structure, and has the same or interchangeable function semantics, for example, multiple AMRs of the same model are used for material handling.

[0050] The synonymous node is a device node that generates high frequency, small data volume, and similar or unified data content structure, and has the same or interchangeable function semantics, for example, multiple AMRs of the same model are used for material handling.

[0051] The buffer management module temporarily stores the data packet in the preset ring buffer, and calls the dynamic protocol analysis engine to analyze the data packet in real time. The dynamic protocol analysis engine extracts the fingerprint features based on the byte structure, field distribution, and communication characteristics of the data packet; and identifies the node type of the data node based on the matching result of the fingerprint features in the fingerprint library.

[0052] The synonymous node pool is a logically continuous memory area, and the data format is uniform, which facilitates subsequent analysis operations in batches and efficiently, reduces memory fragmentation and context switching overhead.

[0053] When it is determined that the data node is a synonymous node, the buffer management module stores the corresponding data packet into a preset synonymous node pool in the ring buffer area, and the synonymous node pool adopts a first-in-first-out strategy for unified scheduling, so as to improve the throughput efficiency of data processing of the same type of device.

[0054] When it is determined that the data node is a non-synonymous node, the buffer management module dynamically divides an independent buffer partition in the ring buffer area according to the priority label pre-configured by the data node and the current network load state, and the buffer partition is used to store the data packet of the non-synonymous node. The size and scheduling weight of the buffer partition are adaptively adjusted according to the network load, so as to guarantee low-delay processing of data of high-priority non-synonymous nodes such as safety emergency stop signals and obstacle alarms.

[0055] The sliding window number is used to identify the time window number to which the data packet belongs, and is used for subsequent time alignment; The logical clock value is a comparison logical clock generated based on a Lamport logical clock or a vector clock mechanism, and is used to establish an event partial order relationship in a distributed environment and solve time ambiguity caused by unsynchronized physical clocks.

[0056] The time sequence enhancement module reads the classified data packets from the ring buffer area according to the scheduling strategy, and adds an extension header to each data packet to generate a time sequence enhanced data frame. The extension header at least includes a node ID, a sliding window number and a logical clock value.

[0057] In this embodiment, the node type of the data node is determined through the data packet, and the data packet is stored respectively, which not only realizes unified access and classified caching of heterogeneous protocol data, but also improves the processing timeliness of high-priority and key non-synonymous data through semantic recognition and dynamic buffer strategy, and provides a standardized input with strong time sequence semantics for a subsequent unified data platform layer.

[0058] In some embodiments, the buffer management module is further used for: Converting the fingerprint features into semantic particles, the fingerprint features including a field name, a data type and a numerical value range, and the semantic particles including a semantic vector, a field strength coefficient and a spin direction; Constructing a local semantic field for each data node based on the semantic particles, and an interaction relationship between the semantic particles in the local semantic field including semantic similarity, functional complementarity and semantic conflict; Calculating a field strength interference mode and a wave function overlap degree between the local semantic fields corresponding to the data nodes; Based on the field strength interference mode and the wave function overlap, the local semantic field is dimensionally fused to determine the field distance between the local semantic fields, and the field distance includes semantic distance, functional distance, time sequence distance and causal distance. According to the field distance, the node type of the data node is determined.

[0059] It can be understood that the semantic particle is a basic information unit obtained by mathematizing and vectorizing the fingerprint feature.

[0060] The semantic vector is a vector representation generated by a pre-trained Word2Vec semantic embedding model based on the field name and the data type, and is used to represent the functional semantics of the field. The field strength coefficient is a weight value assigned according to the importance of the field in device control or perception. The spin direction is used to identify the time sequence behavior characteristics of the field in the data stream, for example, a continuous output type is marked as a positive spin, and an event triggered type is marked as a negative spin.

[0061] In actual execution, the buffer management module is further used for semantic level deep identification and classification of the data node to improve the accuracy and intelligent level of node type determination.

[0062] After extracting the fingerprint feature of the data packet, the buffer management module converts the fingerprint feature into a structured semantic representation unit as a semantic particle. The local semantic field is a dynamic image area in a multi-dimensional semantic space composed of semantic particles corresponding to all fields of the data node, and is used to describe all semantic characteristics and internal structures of the data node.

[0063] Semantic similarity is used to represent that the semantic vector angle of two semantic particles is less than a threshold value; functional complementarity is used to represent that the spin directions of two semantic particles are opposite, and the semantic vectors satisfy a certain correlation; semantic conflict is used to represent that the spin directions of two semantic particles are the same, but the semantic vectors indicate that the two semantic particles represent mutually exclusive states.

[0064] For a given data node, the buffer management module places all semantic particles of the data node in the same vector space, each particle exerts an influence according to the field strength coefficient, the semantic vector determines the center of the influence, and the spin direction determines the polarity of the influence, and the local semantic field formed is the superposition of the semantic potential fields generated by all particles.

[0065] The field intensity interference mode is the superposition result of the potential functions of two fields at each point in the vector space, including constructive interference and destructive interference. The region of constructive interference represents the field intensity enhancement after the superposition of two nodes, and the functions can work synergistically. The region of destructive interference represents the field intensity weakening or cancellation function after superposition. The potential function matrix of two fields is added and the extreme points are analyzed to realize it.

[0066] The wave function overlap degree is used to calculate the degree of coexistence or confusion of two local semantic fields in the vector space.

[0067] Further, the buffer management module performs field interaction analysis on the local semantic fields corresponding to any two data nodes, evaluates the interference effect of the two local semantic fields in the semantic intensity based on the field intensity coefficients and spatial distribution of each semantic particle by using a weighted superposition method, and obtains a field intensity interference mode. Each local semantic field is modeled as a high-dimensional probability distribution, and the similarity degree of the two semantic fields is quantified by calculating the overlap integral of the joint distribution, and the wave function overlap degree is obtained.

[0068] The field distance is used to describe the relationship between nodes from multiple dimensions.

[0069] The semantic distance is determined by the wave function overlap degree, and directly reflects the similarity of data meaning.

[0070] The functional distance is determined by the proportion of constructive and destructive interference in the field intensity interference mode, and reflects the collaboration or competition in function.

[0071] The time sequence distance is obtained by analyzing the correlation of the time sequence of data packets between data nodes, and reflects the time coupling relationship of data nodes in actual work.

[0072] The causal distance is obtained by Granger causality test, analyzing whether the data of one data node is helpful for predicting the data of another data node, and judging the potential causal relationship.

[0073] Based on the field intensity interference mode and the wave function overlap degree, the buffer management module performs dimension fusion on multiple local semantic fields, generates a unified semantic comparison space, and calculates the field distance between any two data nodes in the space.

[0074] According to the field distance, the data nodes are clustered or classified. If the comprehensive field distance between two data nodes is less than a preset synonymous threshold, the two data nodes are determined as synonymous nodes. If the field distance is greater than a nonsynonymous threshold, or there is a significant difference in the functional distance and causal distance dimensions, the two data nodes are determined as nonsynonymous nodes.

[0075] In this embodiment, the determination of node type is not only dependent on the protocol format by the transition from the original protocol data to the high-level semantic understanding, so that the semantic role and functional positioning of the device in the collaborative task can be realized, and more accurate and adaptive data flow management and resource allocation can be realized.

[0076] In some embodiments, the multi-protocol access layer further comprises a time alignment module, configured to: call the time-enhanced data frame to extract the timestamp; in the case that the timestamp comprises a physical timestamp, match the physical timestamp with a time window to generate an index table, the index table being configured to record the data packets received in each time window; in the case that the timestamp comprises a PTP synchronization timestamp, time-align the event units in the time-enhanced data frame; in the case that the timestamp packet does not comprise a PTP synchronization timestamp, time-align the event units in the time window according to the index table and the physical timestamp.

[0077] It can be understood that the timestamp comprises a physical timestamp or a PTP synchronization timestamp according to the clock mechanism of the data source device.

[0078] The event unit is the smallest data object with independent business semantics parsed from the time-enhanced data frame.

[0079] The physical timestamp is the hardware time when the data packet actually arrives, and the PTP synchronization timestamp realizes the time synchronization across data nodes through the precision clock protocol. The time alignment module ensures the consistency of the event units in the time and space dimensions through the double timestamp mechanism, realizes the high-precision time synchronization processing, and ensures that the multi-source heterogeneous data has a consistent time reference in subsequent processing.

[0080] In actual execution, the time alignment module calls the time-enhanced data frame and extracts the embedded timestamp therefrom.

[0081] The time window is to divide the continuous time stream into a series of continuous, fixed-length fixed or sliding time intervals, for segmented management of continuous data stream, each window having a unique identifier, such as a start timestamp or a sequence number.

[0082] The index table is a data structure such as a hash table or a database table, the key being the time window identifier, and the value being the metadata list of all time-enhanced data frames received in the time window.

[0083] For event units containing both physical timestamps and PTP synchronization timestamps, the PTP synchronization timestamps are used for alignment; for event units lacking PTP synchronization timestamps, the mapping relationship between physical timestamps and the index table is used to complete time window attribution judgment; the generated time alignment result is used as the input reference of the event sorting module, ensuring the continuity and accuracy of the global data feature map on the time axis.

[0084] When the timestamp is a physical timestamp, the time alignment module matches the physical timestamp with a preset time window. The time alignment module generates an index table according to the matching result, and the index table records the identification information of all data packets received in each time window, including the data packet ID, the source node ID and the original timestamp.

[0085] When the timestamp is a PTP synchronization timestamp, it indicates that the data packet is from a high-precision device that has accessed the PTP time synchronization network, and the time alignment module directly performs accurate time alignment on the event units in the time sequence enhanced data frame according to the PTP synchronization timestamp, without relying on external indexes.

[0086] When the timestamp does not contain a PTP synchronization timestamp, only contains a normal physical timestamp or has no high-precision synchronization information, the time alignment module combines the index table and the physical timestamp to perform in-window time alignment on multiple event units in the same time window, takes the starting time of the time window as the reference, performs normalization offset correction on the physical timestamps of all event units in the window, and uses interpolation or nearest neighbor alignment strategy to map asynchronous events to a unified time grid, thereby assigning logical timestamps to event units that do not include PTP synchronization timestamps, and achieving logical time consistency among devices that lack PTP support.

[0087] In this embodiment, the time alignment module can be compatible with high-precision synchronization devices and ordinary asynchronous devices, supporting microsecond-level alignment under PTP network, and achieving logical time sequence consistency through time windows and index tables in a PTP-free environment, effectively solving the data time sequence misalignment problem caused by clock heterogeneity in a multi-robot system.

[0088] In some embodiments, the unified data platform layer includes a data parsing module, an event sorting module, a tensor decomposition module and a matrix aggregation module connected in sequence. The data parsing module is configured to call the time sequence enhanced data frame, parse the extension header to obtain the timestamp in the time sequence enhanced data frame, determine the time slot number of an event unit based on the timestamp, and divide the event unit based on the event content of the time sequence enhanced data frame. The event sorting module is configured to sort the event units according to the time slot number and generate the event sequence. The tensor decomposition module is configured to construct a high-order event tensor for the event sequence, and perform random singular value decomposition on the high-order event tensor to generate a core factor matrix of each data node, wherein the high-order event tensor comprises a time dimension, a node dimension and an attribute dimension of the event unit. The matrix aggregation module is configured to aggregate the core factor matrix to generate the global data feature map.

[0089] It can be understood that the time slot number is a monotonically increasing integer, which is used to identify a unique time slot. The continuous time axis is divided into a series of extremely short, continuous and non-overlapping time slots, and the duration of each time slot is much smaller than the time window in the multi-protocol access layer.

[0090] In actual execution, in the unified data platform layer, high-dimensional data representation supporting intelligent decision-making is generated by structured parsing, time series organization, feature extraction and global fusion of the time series enhanced data frame.

[0091] The data parsing module calls the time series enhanced data frame and parses the extension header to extract the timestamp contained therein, and preferentially extracts the PTP synchronization timestamp. If the data frame does not contain the PTP synchronization timestamp, the logical timestamp assigned by the time alignment module is used as a substitute.

[0092] Based on the timestamp, the data parsing module divides the payload in the time series enhanced data frame into one or more event units. When a time series enhanced data frame is received, the node ID and protocol type in the extension header are extracted to load the corresponding protocol mode description file from the protocol mode library. The protocol mode description file is a structured document used to define the binary or serialized format of the payload of the data frame. According to the loaded protocol mode description file, the payload of the time series enhanced data frame is parsed. While parsing the data structure, according to the pre-defined business semantic annotation in the protocol mode description file and the event content, the boundaries of the event units are identified. For each identified and divided event unit, a corresponding standardized internal data structure is created, including an event header and an event body. The event header includes the node ID, the timestamp, the event type, and the event ID. The event body includes all field data related to the event unit parsed from the payload. Each event unit corresponds to an atomic-level perception or control semantic.

[0093] The data parsing module maps the timestamp to the corresponding time slot number according to the preset time slot granularity, as the positioning identifier of the event unit on the global time axis.

[0094] The event sequence is used to describe the occurrence order and concurrent relationship of events on different nodes in the entire robot cluster on the micro time scale, as well as the time sequence correlation between cross-device events.

[0095] The event ordering module receives all event units and their corresponding time slot numbers, and arranges the event units in ascending order according to the time slot numbers to generate an event sequence organized in chronological order.

[0096] The high-order event tensor is represented as a multi-dimensional array. In the high-order event tensor, the time dimension corresponds to continuous time slot numbers, representing the dynamic evolution of events; the node dimension corresponds to different data nodes, representing data sources at different spatial positions; and the attribute dimension corresponds to different data fields in the event unit, representing different types of observations.

[0097] The core factor matrix compactly represents the dynamic behavior characteristics of the data nodes in the time dimension and the coupling relationship with the attribute dimension, and is used to capture the core dynamic behavior characteristics of the data nodes within a specified time window, filtering out noise and redundant information.

[0098] The tensor decomposition module constructs data in the event sequence into a high-order event tensor in units of a fixed-length time slot sequence (i.e., a time window), and uses the Randomized Singular Value Decomposition (Randomized SVD) algorithm for dimension reduction and feature extraction to generate a low-rank core factor matrix for each data node.

[0099] The matrix aggregation module uses weighted fusion, graph convolution, or attention mechanism to aggregate multiple core factor matrices into a unified global data feature map based on the topological relationship, task association, or semantic similarity between nodes. The global data feature map comprehensively describes the state evolution, mutual dependence, and collaborative dynamics of each data node in the system in the time dimension in a structured form, serving as input for task planning in the intelligent central brain layer.

[0100] In this embodiment, the unified data platform layer realizes the automatic conversion from raw time series data to high-dimensional semantic features, with high-precision time alignment, efficient tensor calculation, and global semantic fusion capabilities, effectively supporting real-time perception and intelligent decision-making of complex multi-robot systems.

[0101] In some embodiments, the matrix aggregation module is further configured to: model the association between the core factor matrices through a time-varying graph attention network to determine the dynamic interaction features between the data nodes in the time dimension; aggregate multiple core factor matrices based on the dynamic interaction features to generate a time-varying system embedding vector; The global data feature map is constructed based on the time-varying system embedding vector, and the global data feature map includes a horizontal dimension and a vertical dimension, the horizontal dimension representing attribute features of the data nodes, and the vertical dimension representing state evolution of the data nodes over time.

[0102] It can be understood that the Hybrid Time-Varying Graph Neural Network (HTGNN) is a hybrid neural network structure that integrates graph neural networks and time series modeling capabilities, and is used for joint modeling of the topological relationship and feature propagation of a multi-node system at consecutive time steps.

[0103] The dynamic interaction feature is a high-dimensional feature vector extracted from multiple core factor matrices by the HTGNN, reflecting the mutual influence and synergy relationship between data nodes in the time dimension. It not only contains the independent state of the data node at a certain moment, but also encodes the dependence relationship with the state of other data nodes at historical moments, for example, the path planning of robot A at time t is affected by the position of robot B at time t 1.

[0104] In actual execution, the matrix aggregation module regards each data node as a vertex in the graph through the HTGNN, and the weight of the edge is dynamically generated by the semantic similarity, physical proximity or task dependency between nodes, and an attention mechanism is introduced to adaptively learn the importance of different neighbor nodes at different time steps to determine the dynamic interaction feature between data nodes in the time dimension. Among them, the node connection relationship and the node feature are dynamically updated with time, which can capture the non-steady-state interaction mode in the system.

[0105] The time-varying system embedding vector is a compact vector representation of the global state of the entire system within a certain time window.

[0106] The HTGNN is used to aggregate the weighted sum, maximum pooling or attention weighted sum of the core factor matrices of all nodes to generate a time-varying system embedding vector with fixed dimensions, which is dynamically updated with the time window sliding to continuously reflect the latest state of the system.

[0107] The time-varying system embedding vector is reconstructed and reshaped into a two-dimensional matrix by a deconvolution network as a global data feature map.

[0108] In the global data feature map, the horizontal dimension corresponds to the attribute feature dimension of the data node, each column represents a data node, and the elements in the column are the position, speed, task state, environment perception and other multi-dimensional attribute embeddings of the data node in the current time window; the vertical dimension corresponds to the time sequence dimension, each row represents a time slot, and the elements in the row reflect the state snapshot of all nodes at that time.

[0109] In this embodiment, the matrix aggregation module realizes static fusion of the core factor matrix, deep mining of dynamic and complex interactions within the robot cluster is realized by introducing a time-varying graph attention network, and the interaction relationship between data nodes evolving over time is explicitly modeled, thereby generating a global data feature map with strong spatiotemporal expression capability, which not only contains the individual attributes of each robot, but also contains the collaborative relationship and time evolution law between robots, significantly improving the perception and understanding ability of the multi-robot collaborative control system to the complex dynamic environment, and providing extremely rich and accurate situation awareness information.

[0110] In some embodiments, the time-varying graph attention network comprises a time-aware graph construction layer, a multi-scale time convolution layer, a dynamic graph attention layer, a spatiotemporal fusion module and a dynamic interaction feature extraction layer connected in sequence; The time-aware graph construction layer is configured to calculate the basic similarity between the core factor matrices based on a time window, construct a plurality of static relationship networks, analyze the time sequence correlation of the static relationship networks, and construct a dynamic relationship graph. The multi-scale time convolution layer is configured to extract multi-time scale features from the dynamic relationship graph and filter time sequence noise through time gating to obtain multi-time scale features. The dynamic graph attention layer is configured to perform attention allocation on the multi-time scale features in the time dimension, extract time sequence features in the multi-time scale features through the time attention layer, extract the structural correlation strength between data nodes through the structural attention layer, and generate spatial structure features. The spatiotemporal fusion module is configured to fuse the time sequence features and the spatial structure features to generate an embedding vector. The dynamic interaction feature extraction layer is configured to extract the dynamic interaction features between the data nodes in the time dimension based on the embedding vector.

[0111] It can be understood that the basic similarity is a scalar index quantifying the consistency of the state vectors of any two data nodes at a specific time in the feature space, which can be obtained by calculating the cosine similarity between the core factor vectors of the two nodes at the same time, and is used to preliminarily judge the instantaneous correlation degree between data nodes, with a value range of [-1, 1], and the closer the value is to 1, the more similar the behavior patterns of the two data nodes at that time.

[0112] The static relationship network is a graph structure representation formed at a single discrete time point based on the basic similarity threshold filtering. The network takes data nodes as vertices and connections between data nodes with a basic similarity exceeding a preset threshold at a specific time as edges, forming a static topology structure describing the instantaneous association relationship between data nodes at the time.

[0113] The dynamic relationship graph is a time-expanded graph model integrating information of multiple static relationship networks within a time sequence, containing nodes and edges, and assigning a time sequence function representing the evolution of relationship strength over time to the edges, which can describe the stability, change trend and time sequence pattern of the association relationship between nodes, and completely depict the dynamic evolution of the interaction structure of the system.

[0114] In actual execution, the time-aware graph construction layer calculates the basic similarity between the core factor matrices based on a preset time window, constructs multiple static relationship networks, analyzes the time sequence correlation of the static relationship networks by evaluating the changes of the static relationship networks at consecutive time points, identifies and connects data nodes with significant time sequence association, and constructs a dynamic relationship graph reflecting the evolution of the system state.

[0115] The multi-time scale feature is a set of features with different time granularities extracted from the time series data of nodes or systems by applying convolution kernels of different time window sizes in parallel. These features capture different time scale system behavior patterns such as short-term fluctuations, medium-term rhythms and long-term trends, and together form a multi-resolution description of system dynamics.

[0116] Time gating is a trainable mechanism in the Gate Recurrent Unit (GRU), which adaptively controls the fusion ratio of historical information and current input through structures such as update gate and reset gate, and can selectively retain important historical states and filter irrelevant information, effectively managing information flow in time series.

[0117] Time sequence noise refers to random fluctuations or short-term variations embedded in time series data that do not have predictive value. Noise may come from sensor errors, communication interference or environmental random factors, and can mask the true time sequence patterns and rules in the data, interfering with the model's learning of long-term dependencies.

[0118] Multi-time scale features refer to the multi-time scale feature set filtered and purified by the time gating mechanism. On the basis of retaining the original multi-scale information, the time gating mechanism removes time sequence noise, enhancing the robustness and information density of time sequence representation and providing high-quality input for subsequent attention allocation.

[0119] The multi-scale time convolution layer captures various time-dependent patterns from short-term to long-term by using different size convolution kernels on dynamic relational graphs, extracts features containing different time scale information, and uses a time gating mechanism to selectively pass important information according to the time characteristics of the input signal, filter out noise signals, and improve the quality of feature representation.

[0120] Attention allocation dynamically determines the importance of different elements in the input data to the current task through learnable weight coefficients. In the time sequence and graph structure, it is applied to the time dimension and node dimension respectively to achieve selective information focusing.

[0121] The time attention layer is an attention mechanism component that acts on the time dimension. It calculates the attention weights of each time step in the time series to evaluate the importance difference of different time points in representing the overall behavior of the node, thereby generating a weighted time sequence summary that highlights the influence of key time points.

[0122] The time sequence feature is a node feature vector obtained by weighted aggregation through the time attention layer, which is used to represent the importance of each data node in the time sequence.

[0123] The structure attention layer is an attention mechanism component that acts on the graph structure. Based on the topological connection relationship of the graph, it calculates the attention weight between each node and its neighbor node, thereby quantifying the relative importance of different connections in the graph to the representation of the center node.

[0124] The structure correlation strength is the attention weight calculated by the structure attention layer, which is a scalar value used to quantify the mutual influence of two connected nodes in the feature space. The higher the weight, the greater the feature contribution of the source node to the target node, and the stronger the correlation between the two in the structure.

[0125] The spatial structure feature is a feature vector obtained by aggregating the neighbor node information of each node through the structure attention layer. This vector integrates the features of the node itself and its neighbor nodes in the topological relationship, encoding the local connection pattern and position information of the node in the graph structure.

[0126] The dynamic graph attention layer allocates attention weights to multi-time scale features in the time dimension. It emphasizes or weakens the feature contribution of different time points through the time attention layer, enhances the model's memory ability for key time nodes, extracts time sequence features, quantifies the relationship strength between nodes through the structure attention layer, and strengthens important local structure patterns to extract spatial structure features.

[0127] The embedding vector is a low-dimensional dense vector representation generated by the space-time fusion module to represent the comprehensive space-time state of a single data node within a specific time window.

[0128] The spatio-temporal fusion module performs nonlinear combination of the temporal features and the spatial structure features to fuse them into a unified embedding vector that comprehensively reflects the information in both the time and space dimensions.

[0129] The dynamic interaction feature extraction layer identifies the interaction patterns between the data nodes that evolve over time based on the generated embedding vector, extracts the dynamic interaction features between the data nodes in the time dimension, and maps them to a low-dimensional space through an autoencoder to obtain the final time-varying system embedding vector.

[0130] In this embodiment, the dynamic interaction behavior of the robot cluster is comprehensively and finely modeled from the micro to the macro and from the time to the space through the time-varying graph attention network.

[0131] In some embodiments, the intelligent hub brain layer includes a task planning module and a complexity control module connected in sequence. The task planning module is configured to plan tasks for the plurality of robots based on a task target and time-evolution information of the global data feature map, and state information, response preference, and task allocation granularity of the robots, to generate a task chain, wherein the state information includes current positions, remaining power, and types of tasks to be executed of the robots, and the response preference is determined by modeling historical task response data through an entropy regular term. The complexity control module is configured to calculate a task execution complexity according to a task chain length of the task chain, a node size of the data node, and a fluctuation range of an incentive trajectory, and to control the task execution complexity by adjusting a regular coefficient.

[0132] It can be understood that the task target is used to define a specific task to be completed or a target to be achieved.

[0133] The time-evolution information is a dynamic feature extracted from the global data feature map in a continuous time window, which describes the trend of the overall state and internal patterns of the system over time, and provides a background of environmental changes.

[0134] The state information describes a data set of real-time physical conditions and running contexts of a single robot at a specific time.

[0135] The response preference is a quantitative description of the stable and efficient behavior tendency of a specific robot when facing different tasks, and is used to capture the tendency of each robot when processing different types of tasks.

[0136] The entropy regular term is a constraint term added to a loss function of a machine learning model, and its value is proportional to the entropy of a probability distribution of a model output.

[0137] The historical task response data is a record set about the task execution of the robots accumulated by the system in past operations.

[0138] Task allocation granularity is the size or the degree of refinement of the smallest work unit that a macro task is decomposed into in the task planning process.

[0139] The task planning module generates a task chain based on mixed integer linear programming according to the task goal, the time series evolution information of the global data feature map, the state information of the robot, the response preference, and the task allocation granularity.

[0140] Task chain length is the total number of atomic task nodes contained in a task chain. The longer the task chain, the higher the complexity and resource requirements of the task chain.

[0141] Node scale is the total number of data nodes involved in the execution of a certain task chain, reflecting the degree of occupation of system resources and the number of entities that need to be cooperatively scheduled by the task chain. Large-scale data nodes may lead to more complex interaction modes and higher computational costs.

[0142] The fluctuation range of the incentive trajectory is a quantitative measure of the smoothness of the incentive signal generated within the system during the simulation or actual execution of the task chain. For example, the incentive signal can be an action instruction, a change in the value function, logits of the policy network, etc.

[0143] Task execution complexity is a scalar index used to quantify the overall effort, coordination difficulty, and potential risks required to execute a given task chain. It is calculated by weighted fusion of multiple factors such as task chain length, node scale, and fluctuation range of the incentive trajectory.

[0144] The regularization coefficient controls the weight or influence of the regularization term in the objective function. By properly adjusting the regularization coefficient, the resource use efficiency and the stability of task execution can be optimized while ensuring the quality of task completion.

[0145] The complexity control module calculates the overall complexity of task execution based on factors such as the length of the task chain, the scale of the data nodes, and the fluctuation range of the incentive trajectory, and controls the task execution complexity by adjusting the regularization coefficient.

[0146] Let the response probability of the th robot to the task type in the historical task be , which is specifically: wherein represents the total number of times that the robot successfully executed the task type in the history; is the total number of task types; is the robot The sum of the number of times each task type is successfully executed in history, .

[0147] The response preference is modeled by an entropy regularizer on the historical task response data, and the response preference entropy is: In the task planning process, an entropy regularizer is introduced to encourage the assignment of tasks consistent with their historical preferences. The objective function can be expressed as: where, is the comprehensive cost term, including path length, power consumption, task delay, etc. is the current planning time estimate of the robot preference entropy; is the sum of the response preference entropies of the robots currently participating in planning; is the preference regularization coefficient, , used to adjust the weight of preference consistency in planning; is the total number of robots participating in the task.

[0148] The objective function is solved under the constraints of power threshold, collision-free path, etc., to generate the optimal task chain: where, is the execution robot; is the action; is the time window; is the total number of atomic tasks contained in the task chain.

[0149] The complexity control module defines the task execution complexity as: where, is the task chain length, i.e., the number of subtasks contained in the task chain , The larger the value, the longer the workflow and the higher the coordination requirements; is the data node scale, i.e., the total number of data nodes (robots) participating in collaboration , The larger the value, the greater the communication and scheduling overhead; is the fluctuation range of the incentive trajectory, defined as the sum of the variance of acceleration or direction changes in all robot path instructions, used to quantify the smoothness of the control instructions: wherein, is the robot the acceleration instruction sequence within its task execution time period, is the position of the robot at time is the velocity of the robot at time is the acceleration of the robot at time , the acceleration is directly related to the required control force / torque, energy consumption size, motion smoothness and mechanical impact, and dynamic performance of trajectory tracking; The greater the value, the more severe the dynamic control requirements of the task chain on the robot, which can lead to higher energy consumption and instability; is an adjustable weight coefficient, is an adjustable weight coefficient, respectively used to reflect the sensitivity of the system to the length, size and dynamic stability of the chain; is the control complexity, the original planning objective function is extended to: wherein, is the complexity regularization coefficient, which is dynamically adjusted by the complexity control module according to the system resource state (such as CPU load, communication bandwidth, real-time requirement); when exceeds the preset threshold , increase , prompting the task planning module to generate shorter, smoother, and fewer node task chains. In this embodiment, the cooperation of the task planning module and the complexity control module can generate intelligent task chains that fully consider the historical expertise of the robot, effectively coordinate the cooperation between different robots, and take into account various actual factors such as power limitation, position distribution, and individual preferences. By evaluating and controlling the complexity of the plan, the success rate of task execution and the overall performance of the system are guaranteed while pursuing task efficiency. In some embodiments, the multi-robot execution layer includes an instruction issuing module and an information receiving module connected to each other;

[0150] The instruction issuing module is configured to map the task chain to a low-dimensional hidden space to generate a task instruction corresponding to each robot, the task instruction including an action instruction and a path planning instruction;

[0151] In some embodiments, the multi-robot execution layer includes an instruction issuing module and an information receiving module connected to each other; The instruction issuing module is configured to map the task chain to a low-dimensional hidden space to generate a task instruction corresponding to each robot, the task instruction including an action instruction and a path planning instruction; ​The information receiving module is configured to receive state information generated by the robot in response to the task instruction, and generate a reward signal according to the state information, wherein the state information includes a task completion degree, energy consumption and collision times.

[0152] It can be understood that the low-dimensional latent space is a continuous and dense vector space learned from a large number of historical successful task instructions by the variational autoencoder.

[0153] The task instruction is used to control the robot to perform a specific operation, such as opening and closing of a gripper, starting of a lifting platform, activation of a sensor, etc. The action instruction is a bottom-layer control signal for driving a joint or body of the robot, including a desired joint angle trajectory, a pose of an end effector, a linear velocity and an angular velocity instruction.

[0154] The path planning instruction is used to guide a mobile robot to move along an optimized trajectory, and contains parameters such as a target pose, a speed constraint, an obstacle avoidance region and a time window.

[0155] In some embodiments, in the multi-robot execution layer, the instruction issuing module is configured to convert an abstract action in the task chain into a point in the low-dimensional latent space through an encoder network, to generate a bottom-layer command executable by each robot as a task instruction, and each task instruction is uniquely bound to a corresponding robot and is issued to a target device through a communication protocol channel adapted in the multi-protocol access layer.

[0156] The state information is quantitative evaluation data for a task execution effect.

[0157] The task completion degree represents a proportion of achievement of a current task sub-goal. The energy consumption reflects energy consumption in an execution process, such as a battery power change amount or a motor power consumption integral value. The collision times are the number of physical or virtual collision events occurring during task execution of the robot, and are used to evaluate path safety.

[0158] The reward signal is used to evaluate an execution quality of the current task chain, and is fed back to the intelligent core brain layer 130, to evaluate effectiveness of a current task allocation and planning strategy, to support dynamic re-planning of the task chain or long-term strategy optimization in the iterative optimization task planning module, so that the task chain can be generated in the future to obtain a higher reward.

[0159] The information receiving module continuously receives state information fed back by each robot in the process of executing the task instruction, and generates a reward signal based on the state information and according to a preset multi-objective evaluation function considering task completion efficiency, energy efficiency and safety performance. For example, a high task completion degree corresponds to a positive reward, and high energy consumption or collision occurrence introduces a negative penalty term.

[0160] In this embodiment, the multi-robot execution layer not only realizes semantic conversion and reliable delivery of high-level tasks to bottom-layer control, but also constructs a closed-loop feedback mechanism with state information as input and reward signal as output, effectively supporting adaptive collaboration and continuous optimization capability of the system.

[0161] The embodiment of the application further provides a multi-robot collaborative control method applied to the three-in-one brain robot collaborative control architecture system in any of the above embodiments.

[0162] As shown in Figure 2 The multi-robot collaborative control method comprises the following steps: Step 210, receiving data packets of a plurality of data nodes through a multi-protocol access layer, storing the data packets into a ring buffer, and parsing and adding an extension header to each data packet to generate a time sequence enhanced data frame corresponding to the data packet; Step 220, calling the time sequence enhanced data frame through a unified data platform layer, parsing the extension header to obtain an event sequence of the time sequence enhanced data frame, generating a plurality of core factor matrices according to the event sequence, and aggregating the plurality of core factor matrices to generate a global data feature map, wherein the global data feature map is used to describe data dynamic changes of each data node in a time dimension; Step 230, planning a task for the plurality of robots based on the global data feature map through an intelligent central brain layer to generate a task chain; Step 240, delivering a task instruction corresponding to the task chain to a corresponding robot through a multi-robot execution layer, and receiving state information of the robot in response to execution of the task instruction.

[0163] According to the multi-robot cooperative control method provided by the embodiment of the present application, the data packets from different data nodes are parsed and an extension header is added by the multi-protocol access layer to generate a time sequence enhanced data frame, so that the subsequent processing can more effectively utilize the information in the time dimension; the uniform data platform layer parses the time sequence enhanced data frame generated by the multi-protocol access layer, extracts an event sequence, and converts it into a core factor matrix, thereby improving the depth and accuracy of data analysis; the intelligent central brain layer performs task planning based on a global data feature map, generates a task chain for different robots, can optimize task allocation according to the current environment and robot state, improves the task execution efficiency and resource utilization rate, and at the same time ensures the quality of task completion; the multi-robot execution layer issues task instructions to the corresponding robots and collects the state feedback of their task execution, ensures the accurate transmission of instructions and real-time monitoring of the execution situation, and helps to timely adjust the strategy to cope with unexpected situations; through the unified access, dynamic modeling, intelligent planning and closed-loop execution of heterogeneous robot devices, the event evolution relationship in the time dimension can be effectively fused, the cooperative efficiency and task robustness of the multi-robot system in a complex dynamic environment are improved, the coherence and reliability of task execution are ensured, and the convenience of human-computer interaction is also promoted.

[0164] The embodiments of the present application are described above in combination with the drawings, but the present application is not limited to the above-described specific embodiments, and the above-described specific embodiments are only illustrative, rather than limiting, and those of ordinary skill in the art can make many forms under the inspiration of the present application without departing from the purpose of the present application and the scope protected by the claims, all of which belong to the protection of the present application.

[0165] In the description of the present specification, the description of the terms "one embodiment", "some embodiments", "an illustrative embodiment", "an example", "a specific example", or "some examples" means that the specific features, structures, materials or characteristics described in combination with the embodiment or example are included in at least one embodiment or example of the present application. In the present specification, the illustrative description of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any one or more embodiments or examples in a suitable manner.

[0166] Although the embodiments of the present application have been shown and described, those of ordinary skill in the art can understand that various changes, modifications, replacements and variations can be made to these embodiments without departing from the principles and purposes of the present application, and the scope of the present application is defined by the claims and their equivalents.

Claims

1. A collaborative control architecture system for a three-way, one-brain robot, characterized in that, The system is connected to multiple robots, and the system includes a multi-protocol access layer, a unified data platform layer, an intelligent central brain layer, and a multi-robot execution layer connected in sequence. The multi-protocol access layer is used to receive data packets from multiple data nodes, parse each data packet and add an extension header to generate a timing-enhanced data frame corresponding to the data packet; The unified data platform layer is used to retrieve the time-series enhanced data frame and parse the extended header to obtain the event sequence of the time-series enhanced data frame, and generate multiple core factor matrices based on the event sequence. The multiple core factor matrices are aggregated to generate a global data feature map, which is used to describe the dynamic changes of each data node in the time dimension. The intelligent central brain layer is used to perform task planning for the multiple robots and generate task chains based on global data feature maps; The multi-robot execution layer is used to send the task instructions corresponding to the task chain to the corresponding robots, and to receive the status information of the robots in response to the task instructions in performing the tasks.

2. The three-way, one-brain robot collaborative control architecture system according to claim 1, characterized in that, The multi-protocol access layer includes a data access module, a buffer management module, and a timing enhancement module connected in sequence. The data access module is used to receive data packets from the multiple data nodes; The buffer management module is used to store the data packet into a circular buffer, call the dynamic protocol parsing engine to parse the data packet, extract the fingerprint features of the data packet, determine the node type of the data node based on the fingerprint features, and associate the node type with the data packet. The node type includes synonymous nodes and heteronymous nodes. If the data node is a synonym node, the data packet is stored in the synonym node pool in the circular buffer; In the case where the data node is the heteronym node, a buffer partition is divided in the circular buffer according to the priority label of the data node and the network load to store the corresponding data packet; The timing enhancement module is used to read the data packet from the circular buffer and add the extended header to the data packet according to the node ID of the data node, the sliding window number of the time window, and the comparison logic clock.

3. The three-way-one-brain robot collaborative control architecture system according to claim 2, characterized in that, The buffer management module is further used for: The fingerprint features are converted into semantic particles. The fingerprint features include field names, data types, and numerical ranges. The semantic particles include semantic vectors, field strength coefficients, and spin directions. Based on the semantic particles, a local semantic field is constructed for each data node. The interaction relationships between the semantic particles in the local semantic field include semantic similarity, functional complementarity, and semantic conflict. Calculate the field strength interference mode and wave function overlap between the local semantic fields corresponding to the data nodes; Based on the field strength interferometry mode and the wave function overlap, the local semantic fields are dimensionally fused to determine the field distance between the local semantic fields. The field distance includes semantic distance, functional distance, temporal distance and causal distance. The node type of the data node is determined based on the field distance.

4. The three-way, one-brain robot collaborative control architecture system according to claim 2, characterized in that, The multi-protocol access layer further includes a time alignment module, which is used for: Invoke the time-enhanced data frame and extract the timestamp; If the timestamp includes a physical timestamp, the physical timestamp is matched with a time window to generate an index table, which is used to record the data packets received within each time window; When the timestamp includes a PTP synchronization timestamp, the event units in the timing enhancement data frame are time-aligned. If the timestamp package does not include the PTP synchronization timestamp, the event units within the time window are time-aligned according to the index table and the physical timestamp.

5. The three-way, one-brain robot collaborative control architecture system according to claim 1, characterized in that, The unified data platform layer includes a data parsing module, an event sorting module, a tensor decomposition module, and a matrix aggregation module connected in sequence. The data parsing module is used to retrieve the time-series enhanced data frame and parse the extended header to obtain the timestamp in the time-series enhanced data frame, and determine the time slot number of the event unit based on the timestamp. The event unit is divided based on the event content of the time-series enhanced data frame. The event sorting module is used to sort the event units according to the time slot number and generate the event sequence; The tensor decomposition module is used to construct a higher-order event tensor from the event sequence, perform random singular value decomposition on the higher-order event tensor, and generate a core factor matrix for each data node. The higher-order event tensor includes the time dimension, node dimension, and attribute dimension of the event unit. The matrix aggregation module is used to aggregate the core factor matrix to generate the global data feature map.

6. The three-way-one-brain robot collaborative control architecture system according to claim 5, characterized in that, The matrix aggregation module is further used for: The correlation between the core factor matrices is modeled by a time-varying graph attention network to determine the dynamic interaction characteristics between the data nodes in the time dimension. Based on the dynamic interaction features, multiple core factor matrices are aggregated to generate a time-varying system embedding vector; The global data feature map is constructed based on the time-varying system embedding vector. The global data feature map includes a horizontal dimension and a vertical dimension. The horizontal dimension represents the attribute features of the data node, and the vertical dimension is used to represent the state evolution of the data node in the time series.

7. The three-way-one-brain robot collaborative control architecture system according to claim 6, characterized in that, The time-varying graph attention network includes a time-aware graph construction layer, a multi-scale temporal convolutional layer, a dynamic graph attention layer, a spatiotemporal fusion module, and a dynamic interactive feature extraction layer connected in sequence. The time-aware graph construction layer is used to calculate the basic similarity between the core factor matrices based on time windows, construct multiple static relationship networks, analyze the temporal correlation of the static relationship networks, and construct a dynamic relationship graph. The multi-scale temporal convolutional layer is used to extract multi-timescale features from the dynamic relationship graph and filter temporal noise through temporal gating to obtain multi-timescale features; The dynamic graph attention layer is used to allocate attention to features at multiple time scales in the time dimension. The temporal attention layer extracts the temporal features from the features at multiple time scales, and the structural attention layer extracts the structural correlation strength between data nodes to generate spatial structural features. The spatiotemporal fusion module is used to fuse the temporal features and the spatial structure features to generate an embedding vector; The dynamic interaction feature extraction layer is used to extract the dynamic interaction features between the data nodes in the time dimension based on the embedding vector.

8. The three-way-one-brain robot collaborative control architecture system according to claim 1, characterized in that, The intelligent central brain layer includes a task planning module and a complexity control module connected in sequence; The task planning module is used to perform task planning for multiple robots and generate a task chain based on the task objectives, the temporal evolution information of the global data feature map, the robot's state information, response preferences, and task allocation granularity. The state information includes the robot's current position, remaining battery power, and task type. The response preferences are determined by modeling historical task response data using entropy regularization terms. The complexity control module is used to calculate the task execution complexity based on the task chain length, the node size of the data node, and the fluctuation range of the excitation trajectory, and to control the task execution complexity by adjusting the regularization coefficient.

9. The three-way-one-brain robot collaborative control architecture system according to claim 1, characterized in that, The multi-robot execution layer includes interconnected instruction issuing modules and information receiving modules; The instruction issuing module is used to map the task chain to a low-dimensional latent space and generate task instructions corresponding to each robot. The task instructions include action instructions and path planning instructions. The information receiving module is used to receive the status information generated by the robot in response to the task instruction, and generate a reward signal based on the status information. The status information includes task completion degree, energy consumption, and number of collisions.

10. A multi-robot cooperative control method, characterized in that, The method, applied to the collaborative control architecture system of the three-way-one-brain robot as described in any one of claims 1-9, comprises: Data packets from multiple data nodes are received through a multi-protocol access layer, the data packets are stored in a circular buffer, and each data packet is parsed and an extended header is added to generate a timing-enhanced data frame corresponding to the data packet. The time-series enhanced data frame is retrieved by the unified data platform layer, and the extended header is parsed to obtain the event sequence of the time-series enhanced data frame. Based on the event sequence, multiple core factor matrices are generated. The multiple core factor matrices are aggregated to generate a global data feature map, which is used to describe the dynamic changes of each data node in the time dimension. Based on global data feature maps, the intelligent central brain layer performs task planning for the multiple robots and generates task chains. The multi-robot execution layer sends the task instructions corresponding to the task chain to the corresponding robots and receives the status information of the robots in response to the task instructions.

Images