A multi-mobile manipulator collaborative sorting system and method based on GNN and reinforcement learning

Abstract

This application provides a collaborative sorting system and method for multiple mobile robotic arms based on GNN and reinforcement learning. The system includes acquisition, construction, updating, and instruction modules. The acquisition module, through a central scheduling server, acquires environmental information such as the position, speed, and state of the mobile robotic arms, objects, and obstacles from an environmental perception system. The construction module constructs a dynamic graph data structure based on the environmental information according to dynamic edge activation rules; that is, a connection edge is established when the Euclidean distance between two entity nodes is less than a preset threshold or the logical relationship strength is greater than a preset threshold. The updating module inputs the dynamic graph data structure into a graph neural network encoder, employs a collaborative attention mechanism, dynamically selects collaborative nodes, calculates attention weights, and updates node embeddings. The instruction module generates joint action instructions for the mobile robotic arms based on the node embedding vectors. This application unifies the representation of entities and their associations, integrates encoding and decision-making, and synchronously and collaboratively generates instructions, improving the overall efficiency, real-time performance, and safety of the system.

Description

Technical Field

[0001] This application belongs to the field of robotic arm control, and in particular relates to a multi-mobile robotic arm collaborative sorting system and method based on GNN and reinforcement learning. Background Technology

[0002] Multi-mobility robotic arm collaborative sorting systems based on traditional optimization or rules are used to automate handling and stacking operations in warehouse environments.

[0003] Traditional technologies typically break down collaborative tasks into multiple independent, sequential or decoupled modules, such as "task allocation," "path planning," and "motion control." The operation involves a central scheduler using optimization algorithms (such as auction algorithms or integer programming) to assign items to each robot based on orders and robot positions, achieving static or periodic task allocation. Each robot independently calculates its mobile chassis's path using path planning algorithms (such as A* or RRT) based on its assigned task objective. The robotic arm controller performs independent trajectory planning based on the chassis's position after it has reached its designated location to complete the grasping and placement actions. Through this layered, decoupled planning and control approach, existing technologies attempt to simplify system complexity and enable basic multi-robot operations.

[0004] In the existing technology, the sequential decision-making process through "layering and decoupling" suffers from a lack of unified modeling and real-time coordination of global dynamic relationships (such as real-time interaction between robots and task-space coupling) because task allocation, path planning, and motion control modules are independent, information is fragmented, and decision frequencies are inconsistent. Therefore, the existing technology suffers from drawbacks such as delayed decision response, potential conflicts between robot actions, and a high risk of collisions in dynamic environments. Summary of the Invention

[0005] The purpose of this application is to overcome the deficiencies in the prior art and provide a multi-mobile robotic arm collaborative sorting system and method based on GNN and reinforcement learning.

[0006] This application provides a multi-mobile robotic arm collaborative sorting system based on GNN and reinforcement learning, including:

[0007] The acquisition module, the central dispatch server acquires environmental information from the environmental perception system, the environmental information including the position, speed and status of the mobile robotic arm, the object and the obstacle;

[0008] The construction module, based on the environmental information, constructs a dynamic graph data structure according to the dynamic edge activation rules. The dynamic edge activation rules are as follows: for any two entity nodes, when the Euclidean distance between the two entity nodes is less than a preset type-related distance threshold or the logical relationship strength is greater than a preset type-related logical relationship threshold, a connection edge is established between the two entity nodes.

[0009] The update module inputs the dynamic graph data structure into the graph neural network encoder in the integrated collaborative decision module. The graph neural network encoder uses a collaborative attention mechanism to calculate the node embedding vector. The collaborative attention mechanism includes: dynamically selecting a collaborative node from the nodes of the dynamic graph data structure, and calculating attention weights based on the feature vectors of the current node, neighboring nodes, and the collaborative node, thereby updating the node embedding.

[0010] The instruction module generates joint action instructions for the mobile robotic arm from the decoding head in the integrated collaborative decision-making module based on the node embedding vector.

[0011] Optionally, the construction module constructs a dynamic graph data structure based on the environment information and the dynamic heterogeneous relationship graph construction module according to dynamic edge activation rules, including:

[0012] The logical relationship strength is defined based on the task state of the entity in the environmental information, wherein the task state is the item allocation state. When the item has been allocated to the mobile robotic arm, the logical relationship strength is set to a first logical value; otherwise, it is set to a second logical value. The first logical value is greater than the second logical value.

[0013] Optionally, the update module inputs the dynamic graph data structure into the graph neural network encoder in the integrated collaborative decision-making module. The graph neural network encoder uses a collaborative attention mechanism to calculate the node embedding vector, wherein the collaborative attention mechanism includes: dynamically selecting a collaborative node from the nodes of the dynamic graph data structure, including:

[0014] The dynamic selection of the collaborative nodes is based on the priority of the nodes in the dynamic graph data structure, wherein the priority is determined according to the urgency of the entity's task; the higher the urgency of the task, the higher the priority.

[0015] Optionally, the update module inputs the dynamic graph data structure into the graph neural network encoder in the integrated collaborative decision-making module. The graph neural network encoder uses a collaborative attention mechanism to calculate node embedding vectors, wherein the collaborative attention mechanism includes: calculating attention weights based on the feature vectors of the current node, neighboring nodes, and the collaborative nodes, including:

[0016] The feature vectors of the current node, the neighboring nodes, and the collaborating nodes are concatenated, and the attention weights are calculated based on the concatenated vectors.

[0017] Optionally, the instruction module generates joint action instructions for the mobile robotic arm from the decoding head in the integrated collaborative decision-making module based on the node embedding vector, including:

[0018] The decoding head includes a task allocation head, which processes the node embedding vector using the Gumbel-Softmax function and outputs discrete actions to select target items.

[0019] The decoding head includes a path planning head, which concatenates the embedding vector of the moving robotic arm node with the embedding vector of the target item node, and outputs path instructions based on the concatenated vector.

[0020] This application also provides a multi-mobile robotic arm collaborative sorting method based on GNN and reinforcement learning, including:

[0021] The central dispatch server obtains environmental information from the environmental perception system, including the position, speed, and status of the mobile robotic arm, objects, and obstacles.

[0022] Based on the environmental information, the dynamic heterogeneous relation graph construction module constructs a dynamic graph data structure according to the dynamic edge activation rule, wherein the dynamic edge activation rule is: for any two entity nodes, when the Euclidean distance between the two entity nodes is less than a preset type-related distance threshold or the logical relationship strength is greater than a preset type-related logical relationship threshold, a connection edge is established between the two entity nodes.

[0023] The dynamic graph data structure is input into the graph neural network encoder in the integrated collaborative decision-making module. The graph neural network encoder uses a collaborative attention mechanism to calculate the node embedding vector. The collaborative attention mechanism includes: dynamically selecting a collaborative node from the nodes of the dynamic graph data structure, and calculating attention weights based on the feature vectors of the current node, neighboring nodes, and the collaborative node, thereby updating the node embedding.

[0024] Based on the node embedding vector, the decoding head in the integrated collaborative decision-making module generates the joint action commands for the mobile robotic arm.

[0025] Optionally, the step of constructing a dynamic graph data structure by the dynamic heterogeneous relation graph construction module according to the environmental information and dynamic edge activation rules includes:

[0026] The logical relationship strength is defined based on the task state of the entity in the environmental information, wherein the task state is the item allocation state. When the item has been allocated to the mobile robotic arm, the logical relationship strength is set to a first logical value; otherwise, it is set to a second logical value. The first logical value is greater than the second logical value.

[0027] Optionally, the step of inputting the dynamic graph data structure into the graph neural network encoder in the integrated collaborative decision-making module, wherein the graph neural network encoder uses a collaborative attention mechanism to calculate the node embedding vector, and wherein the collaborative attention mechanism includes: dynamically selecting a collaborative node from the nodes of the dynamic graph data structure, including:

[0028] The dynamic selection of the collaborative nodes is based on the priority of the nodes in the dynamic graph data structure, wherein the priority is determined according to the urgency of the entity's task; the higher the urgency of the task, the higher the priority.

[0029] Optionally, the step of inputting the dynamic graph data structure into the graph neural network encoder in the integrated collaborative decision-making module, wherein the graph neural network encoder uses a collaborative attention mechanism to calculate node embedding vectors, and wherein the collaborative attention mechanism includes: calculating attention weights based on the feature vectors of the current node, neighboring nodes, and the collaborative nodes, including:

[0030] The feature vectors of the current node, the neighboring nodes, and the collaborating nodes are concatenated, and the attention weights are calculated based on the concatenated vectors.

[0031] Optionally, generating the joint motion commands for the mobile robotic arm from the decoding head in the integrated collaborative decision-making module based on the node embedding vector includes:

[0032] The decoding head includes a task allocation head, which processes the node embedding vector using the Gumbel-Softmax function and outputs discrete actions to select target items.

[0033] The decoding head includes a path planning head, which concatenates the embedding vector of the moving robotic arm node with the embedding vector of the target item node, and outputs path instructions based on the concatenated vector.

[0034] The beneficial effects of this application are:

[0035] Invention Point 1: The integrated collaboration of dynamic graphs and collaborative attention mechanisms avoids global conflicts and enables real-time decision-making.

[0036] Invention Point 2: Integrated node embedding vectors and parallel multi-decoder collaboration achieve intrinsic consistency at the decision source.

[0037] This application provides a multi-mobile robotic arm collaborative sorting system based on GNN and reinforcement learning, comprising: an acquisition module, wherein a central scheduling server acquires environmental information from an environmental perception system, the environmental information including the position, speed, and state of the mobile robotic arms, items, and obstacles; a construction module, wherein, based on the environmental information, a dynamic heterogeneous relation graph construction module constructs a dynamic graph data structure according to dynamic edge activation rules, wherein the dynamic edge activation rules are: for any two entity nodes, when the Euclidean distance between the two entity nodes is less than a preset type-related distance threshold or the logical relationship strength is greater than a preset type-related logical relationship threshold, a connection edge is established between the two entity nodes; an update module, wherein the dynamic graph data structure is input into a graph neural network encoder in an integrated collaborative decision-making module, the graph neural network encoder using a collaborative attention mechanism to calculate node embedding vectors, wherein the collaborative attention mechanism includes: dynamically selecting a collaborative node from the nodes of the dynamic graph data structure, and calculating attention weights based on the feature vectors of the current node, neighboring nodes, and the collaborative node, thereby updating the node embedding; and an instruction module, wherein, based on the node embedding vectors, a decoder in the integrated collaborative decision-making module generates joint action instructions for the mobile robotic arms. This application constructs a dynamic heterogeneous relationship graph to uniformly represent all entities and their spatiotemporal relationships within the system, and employs a graph neural network based on a collaborative attention mechanism to encode and make decisions on global dynamic relationships in an integrated manner. This enables the synchronous and collaborative generation of task allocation, path planning, and motion control commands, thereby fundamentally avoiding information fragmentation, response delays, and action conflicts caused by hierarchical decoupling decisions, and significantly improving the overall efficiency, real-time performance, and security of the system. Attached Figure Description

[0038] Figure 1 This is a schematic diagram of the multi-mobile robotic arm collaborative sorting system based on GNN and reinforcement learning in this application;

[0039] Figure 2 This is a schematic diagram of the collaborative sorting process of multiple mobile robotic arms based on GNN and reinforcement learning in this application. Detailed Implementation

[0040] Exemplary embodiments of the present disclosure will now be described in more detail with reference to the accompanying drawings. While exemplary embodiments of the present disclosure are shown in the drawings, it is to be understood that various forms of implementation of the present disclosure are intended and should not be limited to the embodiments set forth herein. Rather, the embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the disclosure to those skilled in the art.

[0041] Please refer to Figure 1As shown, this application provides a multi-mobile robotic arm collaborative sorting system based on GNN and reinforcement learning, applied in the field of intelligent warehousing and logistics robot technology. It addresses the technical problems of existing sorting systems in dynamic, large-scale order scenarios, such as poor flexibility, low efficiency, and easy interference between multiple machines, as well as the difficulty of traditional hierarchical planning methods in achieving integrated collaborative decision-making for task allocation, path planning, and motion control. This system enables multiple mobile robotic arms to achieve efficient, collision-free, and real-time integrated collaborative operations.

[0042] The system includes:

[0043] The acquisition module 101, the central dispatch server acquires environmental information from the environmental perception system, the environmental information including the position, speed and status of the mobile robotic arm, the object and the obstacle.

[0044] The environmental perception system includes a global camera deployed on the top of the warehouse and local sensors on each robot. The acquired environmental information includes the position, orientation, speed, and task status of all moving robotic arms, items to be sorted, target shelves, and obstacles.

[0045] Among them, position information is usually represented by a coordinate vector p, attitude information is represented by a quaternion or Euler angle vector q, velocity information is represented by a velocity vector v, and task status information is represented by a status vector s, which are used to describe whether the entity is idle, has been assigned a task, or is executing a task.

[0046] The mobile robotic arm is a robot unit that integrates a mobile chassis and a multi-degree-of-freedom robotic arm. Each robot unit is equipped with a vision sensor, LiDAR, and computing unit. The central scheduling server is connected to all mobile robotic arm units via a wireless network, thus forming a complete collaborative operation system.

[0047] Construction module 102, based on the environmental information, constructs a dynamic graph data structure by the dynamic heterogeneous relation graph construction module according to the dynamic edge activation rule, wherein the dynamic edge activation rule is: for any two entity nodes, when the Euclidean distance between the two entity nodes is less than a preset type-related distance threshold or the logical relationship strength is greater than a preset type-related logical relationship threshold, a connection edge is established between the two entity nodes;

[0048] Based on the acquired environmental information, the dynamic heterogeneous relation graph construction module constructs or updates the dynamic environment graph data structure at the current moment according to preset dynamic edge activation rules:

[0049]

[0050] in, For a set of nodes, Let be the set of edges activated based on the above rules.

[0051] The dynamic edge activation rule is as follows: Each entity in the entire work scenario is abstracted as a node; for any two entity nodes i and j, an edge of type... edge If and only if the following condition is met:

[0052]

[0053] or:

[0054]

[0055] in, This represents the Euclidean distance between entities i and j; This indicates that the edge type is dependent. The distance threshold is determined by the edge type, which includes different types such as robot-object, robot-robot, and robot-obstacle. Different types of edges have different distance activation thresholds. This indicates the strength of the logical relationship between entities i and j; Represents edge type The corresponding logical relationship strength threshold.

[0056] This rule ensures that connections are established only between entities that are spatially close or logically related, thus constructing a dynamic graph that reflects the key relationships in the current environment.

[0057] The strength of the logical relationship The task state is defined based on the entity's task status in the environmental information. Specifically, the task status is the item allocation status. When an item to be sorted has been allocated to a specific mobile robotic arm, the logical relationship strength between the item node and the robot node is determined. A higher first logical value is set to indicate a strong association between the two; if the item is not assigned, then the association between it and the robot node is... It is set to a lower second logical value. With this definition, even if the physical distance between the item and the robot is temporarily far, as long as the assignment relationship is valid, the system will establish connecting edges in the graph, thereby integrating task assignment information into the graph structure and guiding subsequent collaborative decisions.

[0058] The update module 103 inputs the dynamic graph data structure into the graph neural network encoder in the integrated collaborative decision-making module. The graph neural network encoder uses a collaborative attention mechanism to calculate the node embedding vector. The collaborative attention mechanism includes: dynamically selecting a collaborative node from the nodes of the dynamic graph data structure, and calculating attention weights based on the feature vectors of the current node, neighboring nodes, and the collaborative node, thereby updating the node embedding.

[0059] The dynamic graph data structure to be constructed The input is a graph neural network encoder in the integrated collaborative decision-making module. This encoder uses a collaborative attention mechanism to calculate and update the embedding vectors of all nodes. The graph neural network encoder consists of an L-layer graph attention network, with the collaborative attention mechanism at its core.

[0060] First, a coordinating node k is dynamically selected from all nodes in the dynamic graph data structure. Coordinating node k is typically the most critical global objective at present, such as the highest-priority item to be sorted. The dynamic selection of the coordinating node is based on the priority of nodes in the graph, which is determined by the urgency of the task of the entity it represents. For example, an item with the closest delivery date or the highest value to be sorted has a higher task urgency, therefore its corresponding node has a higher priority and is more likely to be selected as coordinating node k. The selection of coordinating node k allows the message passing process to proceed under the guidance of the globally critical objective.

[0061] In each layer of the graph attention network, for node i and one of its neighbor nodes j∈N(i), the collaborative attention mechanism calculates the attention weights based on the feature vector of the current node i, the feature vector of the neighbor node j, and the feature vector of the collaborative node k.

[0062] The specific calculation process involves concatenating the feature vectors of the current node, the neighboring nodes, and the collaborating nodes to form a comprehensive feature vector, and then calculating the attention coefficient based on this vector.

[0063] Attention coefficient in layer l The calculation formula is:

[0064]

[0065] in, and These are the feature vectors of node i and node j in the l-th layer, respectively. It is the feature vector of the dynamically selected collaborative node k in the l-th layer; and It is a learnable weight matrix used to perform linear transformations on node features; It is a learnable attention vector; symbol This represents a vector concatenation operation. It is a non-linear activation function.

[0066] This formula implies that the calculation of attention weights not only depends on the relationship between the current node pairs, but also incorporates global information from collaborating nodes for guidance.

[0067] After obtaining the attention coefficients, the softmax function is used to normalize the coefficients of all neighbors to obtain the final attention weights. The calculation formula is:

[0068]

[0069] in, It is an exponential function, and it is a node. The set of all neighboring nodes of node i. This normalization operation ensures that the sum of the attention weights of node i to all its neighbors is 1, representing the proportion of attention it is assigned.

[0070] Finally, the embedding vector of node i is updated based on the weighted aggregation result of all its neighboring nodes, and the update formula is:

[0071]

[0072] in, It is a non-linear activation function, such as ReLU; after L layers of message passing and aggregation, the final embedding vector of each node is... Each of them encodes its own characteristics, local neighbor information, and global task-related information under the guidance of cooperating nodes.

[0073] The instruction module 104 generates the joint action instructions of the mobile robotic arm by the decoding head in the integrated collaborative decision-making module based on the node embedding vector.

[0074] Based on the obtained final node embedding vector, multiple decoding heads in the integrated collaborative decision-making module generate joint motion commands for each mobile robotic arm. These decoding heads include a task allocation head, a path planning head, and a trajectory planning head, which operate in parallel.

[0075] For the task assignment head, it receives the embedding vector of robot node i. After processing through a multilayer perceptron, a discrete action is output using the Gumbel-Softmax function:

[0076]

[0077] The Gumbel-Softmax function allows for differentiable sampling of discrete selection actions (such as which item to select) during training, facilitating optimization of network parameters via gradient descent. This discrete action indicates which item the robot should select as its target for sorting.

[0078] For the path planning head, it embeds the vector of the moving robot node i. Embedding vector of the target item node selected by the task assignment head The features are then concatenated to form a new joint feature vector.

[0079] Then, the joint feature vector is input into another multilayer perceptron, and after passing through the tanh activation function, it outputs a continuous path instruction vector:

[0080]

[0081] The tanh function restricts the output to the range of [-1, 1]. This continuous vector can guide the direction and speed of the mobile chassis in a two-dimensional plane.

[0082] For the trajectory planning head, it receives the embedding vector of robot node i. Embedding vectors of nodes related to the state of the robot arm After concatenating the two, a multilayer perceptron outputs the trajectory point sequence of the robotic arm joints. The calculation formula is as follows:

[0083]

[0084] in, It is the node embedding of robot i. It is the embedding of nodes related to the state of the robotic arm of robot i. This represents a vector concatenation operation. This represents the multilayer perceptron corresponding to the trajectory planning head.

[0085] Ultimately, the combined action commands for each robot The command is sent to the corresponding mobile robotic arm unit via wireless network for execution.

[0086] The integrated collaborative decision-making module includes a centralized evaluation network that takes as input an aggregated vector of all node embedding vectors, the aggregation operation being, for example, global average pooling. This network evaluates the current global state. Below, all the mobile robotic arms take coordinated actions. Overall value:

[0087]

[0088] in, This value assessment is used for offline training optimization of the system to evaluate the network's parameters.

[0089] The integrated collaborative decision-making module needs to be trained offline on a simulation and training platform before deployment. The training adopts a centralized training and distributed execution framework, and its policy gradient update formula is as follows:

[0090]

[0091] in, Indicates policy parameters The gradient of the objective function; N is the number of moving robotic arms; T is the total number of time steps in the training rounds; It is the policy of agent i, representing the global state. Select action The probability of; It is the global state-action value calculated by a centralized evaluation network; It is the baseline function, which is defined as follows: This indicates the actions of other intelligent agents. Given a fixed value, the expected value of agent i.

[0092] This formula evaluates the marginal contribution of each agent's actions to the global value by subtracting a baseline that is only related to the actions of other agents from the global total value, thereby guiding all agents to learn a cooperative strategy.

[0093] After the commands are generated and issued, each mobile robotic arm receives and executes its respective joint action commands via a wireless network. Simultaneously, the environmental perception system continuously monitors the entire work scene, acquiring the latest position, speed, and status information of the mobile robotic arms, objects, and obstacles in real time, and updating the status accordingly. This method, at a millisecond-level frequency, repeatedly executes the aforementioned environmental information acquisition, dynamic graph construction, node embedding and updating, and command generation and issuance processes based on the updated environmental status, achieving dynamic replanning and closed-loop online real-time control. This enables it to cope with unexpected situations such as dynamic obstacles and new tasks appearing in the environment.

[0094] The system described above achieves deep collaboration among multiple mobile robotic arms in sorting operations. Through the coordinated operation of dynamic graph construction and a collaborative attention mechanism, the system transforms the attention function of the graph neural network from weighting local information to detecting potential conflicts under the guidance of a global objective, thereby achieving integration and avoiding conflict at the decision-making source. The end-to-end decision model avoids the latency associated with traditional hierarchical planning, and the dynamic graph structure naturally adapts to changes in the number of entities in the environment, ensuring strong real-time performance and adaptability. The training formula based on global value and joint action outputs ensures that the system pursues long-term overall efficiency optimization rather than local optima.

[0095] Please refer to Figure 2 As shown, this application also provides a multi-mobile robotic arm collaborative sorting method based on GNN and reinforcement learning, including:

[0096] S201. The central dispatch server obtains environmental information from the environmental perception system, including the position, speed, and status of the mobile robotic arm, the object, and the obstacle.

[0097] S202. Based on the environmental information, the dynamic heterogeneous relation graph construction module constructs a dynamic graph data structure according to the dynamic edge activation rule, wherein the dynamic edge activation rule is: for any two entity nodes, when the Euclidean distance between the two entity nodes is less than a preset type-related distance threshold or the logical relationship strength is greater than a preset type-related logical relationship threshold, a connection edge is established between the two entity nodes.

[0098] S203. Input the dynamic graph data structure into the graph neural network encoder in the integrated collaborative decision-making module. The graph neural network encoder uses a collaborative attention mechanism to calculate the node embedding vector. The collaborative attention mechanism includes: dynamically selecting a collaborative node from the nodes of the dynamic graph data structure, and calculating attention weights based on the feature vectors of the current node, neighboring nodes, and the collaborative node, thereby updating the node embedding.

[0099] S204. Based on the node embedding vector, the decoding head in the integrated collaborative decision-making module generates the joint action command of the mobile robotic arm.

[0100] Optionally, the step of constructing a dynamic graph data structure by the dynamic heterogeneous relation graph construction module according to the environmental information and dynamic edge activation rules includes:

[0101] The logical relationship strength is defined based on the task state of the entity in the environmental information, wherein the task state is the item allocation state. When the item has been allocated to the mobile robotic arm, the logical relationship strength is set to a first logical value; otherwise, it is set to a second logical value. The first logical value is greater than the second logical value.

[0102] Optionally, the step of inputting the dynamic graph data structure into the graph neural network encoder in the integrated collaborative decision-making module, wherein the graph neural network encoder uses a collaborative attention mechanism to calculate the node embedding vector, and wherein the collaborative attention mechanism includes: dynamically selecting a collaborative node from the nodes of the dynamic graph data structure, including:

[0103] The dynamic selection of the collaborative nodes is based on the priority of the nodes in the dynamic graph data structure, wherein the priority is determined according to the urgency of the entity's task; the higher the urgency of the task, the higher the priority.

[0104] Optionally, the step of inputting the dynamic graph data structure into the graph neural network encoder in the integrated collaborative decision-making module, wherein the graph neural network encoder uses a collaborative attention mechanism to calculate node embedding vectors, and wherein the collaborative attention mechanism includes: calculating attention weights based on the feature vectors of the current node, neighboring nodes, and the collaborative nodes, including:

[0105] The feature vectors of the current node, the neighboring nodes, and the collaborating nodes are concatenated, and the attention weights are calculated based on the concatenated vectors.

[0106] Optionally, generating the joint motion commands for the mobile robotic arm from the decoding head in the integrated collaborative decision-making module based on the node embedding vector includes:

[0107] The decoding head includes a task allocation head, which processes the node embedding vector using the Gumbel-Softmax function and outputs discrete actions to select target items.

[0108] The decoding head includes a path planning head, which concatenates the embedding vector of the moving robotic arm node with the embedding vector of the target item node, and outputs path instructions based on the concatenated vector.

[0109] The above description of the embodiments is provided to enable those skilled in the art to understand and apply this application. Those skilled in the art will readily make various modifications to the above embodiments and apply the general principles described herein to other embodiments without inventive effort. Therefore, this application is not limited to the above embodiments, and any improvements and modifications made to this application based on the disclosure thereof should be within the scope of protection of this application.

Claims

1. A multi-mobile robotic arm collaborative sorting system based on GNN and reinforcement learning, characterized in that, include: The acquisition module, the central dispatch server acquires environmental information from the environmental perception system, the environmental information including the position, speed and status of the mobile robotic arm, the object and the obstacle; The construction module, based on the environmental information, constructs a dynamic graph data structure according to the dynamic edge activation rules. The dynamic edge activation rules are as follows: for any two entity nodes, when the Euclidean distance between the two entity nodes is less than a preset type-related distance threshold or the logical relationship strength is greater than a preset type-related logical relationship threshold, a connection edge is established between the two entity nodes. The update module inputs the dynamic graph data structure into the graph neural network encoder in the integrated collaborative decision module. The graph neural network encoder uses a collaborative attention mechanism to calculate the node embedding vector. The collaborative attention mechanism includes: dynamically selecting a collaborative node from the nodes of the dynamic graph data structure, and calculating attention weights based on the feature vectors of the current node, neighboring nodes, and the collaborative node, thereby updating the node embedding. The instruction module generates joint action instructions for the mobile robotic arm from the decoding head in the integrated collaborative decision-making module based on the node embedding vector.

2. The system according to claim 1, characterized in that, The construction module, based on the environmental information, constructs a dynamic graph data structure according to dynamic edge activation rules, including: The logical relationship strength is defined based on the task state of the entity in the environmental information, wherein the task state is the item allocation state. When the item has been allocated to the mobile robotic arm, the logical relationship strength is set to a first logical value; otherwise, it is set to a second logical value. The first logical value is greater than the second logical value.

3. The system according to claim 1, characterized in that, The update module inputs the dynamic graph data structure into the graph neural network encoder in the integrated collaborative decision-making module. The graph neural network encoder uses a collaborative attention mechanism to calculate node embedding vectors, wherein the collaborative attention mechanism includes: dynamically selecting a collaborative node from the nodes of the dynamic graph data structure, including: The dynamic selection of the collaborative nodes is based on the priority of the nodes in the dynamic graph data structure, wherein the priority is determined according to the urgency of the entity's task; the higher the urgency of the task, the higher the priority.

4. The system according to claim 1, characterized in that, The update module inputs the dynamic graph data structure into the graph neural network encoder in the integrated collaborative decision-making module. The graph neural network encoder uses a collaborative attention mechanism to calculate node embedding vectors, wherein the collaborative attention mechanism includes: calculating attention weights based on the feature vectors of the current node, neighboring nodes, and the collaborative nodes, including: The feature vectors of the current node, the neighboring nodes, and the collaborating nodes are concatenated, and the attention weights are calculated based on the concatenated vectors.

5. The system according to claim 1, characterized in that, The instruction module generates joint action instructions for the mobile robotic arm based on the node embedding vector and the decoding head in the integrated collaborative decision-making module, including: The decoding head includes a task allocation head, which processes the node embedding vector using the Gumbel-Softmax function and outputs discrete actions to select target items. The decoding head includes a path planning head, which concatenates the embedding vector of the moving robotic arm node with the embedding vector of the target item node, and outputs path instructions based on the concatenated vector.

6. A collaborative sorting method for multiple mobile robotic arms based on GNN and reinforcement learning, characterized in that, include: The central dispatch server obtains environmental information from the environmental perception system, including the position, speed, and status of the mobile robotic arm, objects, and obstacles. Based on the environmental information, the dynamic heterogeneous relation graph construction module constructs a dynamic graph data structure according to the dynamic edge activation rule, wherein the dynamic edge activation rule is: for any two entity nodes, when the Euclidean distance between the two entity nodes is less than a preset type-related distance threshold or the logical relationship strength is greater than a preset type-related logical relationship threshold, a connection edge is established between the two entity nodes. The dynamic graph data structure is input into the graph neural network encoder in the integrated collaborative decision-making module. The graph neural network encoder uses a collaborative attention mechanism to calculate the node embedding vector. The collaborative attention mechanism includes: dynamically selecting a collaborative node from the nodes of the dynamic graph data structure, and calculating attention weights based on the feature vectors of the current node, neighboring nodes, and the collaborative node, thereby updating the node embedding. Based on the node embedding vector, the decoding head in the integrated collaborative decision-making module generates the joint action commands for the mobile robotic arm.

7. The method according to claim 6, characterized in that, The step of constructing a dynamic graph data structure by the dynamic heterogeneous relation graph construction module according to the dynamic edge activation rules based on the environmental information includes: The logical relationship strength is defined based on the task state of the entity in the environmental information, wherein the task state is the item allocation state. When the item has been allocated to the mobile robotic arm, the logical relationship strength is set to a first logical value; otherwise, it is set to a second logical value. The first logical value is greater than the second logical value.

8. The method according to claim 6, characterized in that, The dynamic graph data structure is input into the graph neural network encoder in the integrated collaborative decision-making module. The graph neural network encoder uses a collaborative attention mechanism to calculate the node embedding vector, wherein the collaborative attention mechanism includes: dynamically selecting a collaborative node from the nodes of the dynamic graph data structure, including: The dynamic selection of the collaborative nodes is based on the priority of the nodes in the dynamic graph data structure, wherein the priority is determined according to the urgency of the entity's task; the higher the urgency of the task, the higher the priority.

9. The method according to claim 6, characterized in that, The dynamic graph data structure is input into the graph neural network encoder in the integrated collaborative decision-making module. The graph neural network encoder uses a collaborative attention mechanism to calculate node embedding vectors, wherein the collaborative attention mechanism includes: calculating attention weights based on the feature vectors of the current node, neighboring nodes, and the collaborative nodes, including: The feature vectors of the current node, the neighboring nodes, and the collaborating nodes are concatenated, and the attention weights are calculated based on the concatenated vectors.

10. The method according to claim 6, characterized in that, The step of generating joint action commands for the mobile robotic arm by the decoding head in the integrated collaborative decision-making module based on the node embedding vector includes: The decoding head includes a task allocation head, which processes the node embedding vector using the Gumbel-Softmax function and outputs discrete actions to select target items. The decoding head includes a path planning head, which concatenates the embedding vector of the moving robotic arm node with the embedding vector of the target item node, and outputs path instructions based on the concatenated vector.

Images