Multi-robot collaborative assembly system and method based on large model and knowledge graph

Abstract

The application discloses a large model and knowledge graph linkage multi-robot collaborative assembly system and method, the application acquires the data information of the assembly environment and the assembly task in real time through the perception layer; the data information is preprocessed and deeply analyzed through the data processing layer, and structured information is obtained; the knowledge graph is constructed according to the structured information through the knowledge graph layer; the task decision instruction is obtained through the task decision layer based on the knowledge graph for task decomposition, distribution and optimized scheduling; the corresponding robot is controlled to execute the distributed target assembly task based on the task decision instruction through the execution layer. The application can better handle the relationship of complex tasks, optimize the task distribution process, realize the multi-robot collaborative assembly system of autonomous intelligent task decomposition and distribution, improve the flexibility, response speed and cooperation efficiency of the system, and can be widely applied to the technical field of data processing.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of data processing, and in particular to a large model and knowledge graph linked multi-robot collaborative assembly system and method. BACKGROUND

[0002] With the rapid development of industrial automation, more and more manufacturing scenarios introduce robots to complete complex assembly tasks. However, in the prior art, multi-robot collaborative assembly systems face many challenges. Traditional task allocation and collaboration methods mainly rely on predefined rules or simple linear reasoning models, which show limitations in the face of complex and variable tasks and environments. Existing assembly systems usually require human programming to define the tasks and paths of each robot, making it difficult to quickly adapt to frequent task changes and dynamic adjustments of the environment, resulting in low production efficiency and increasing the cost of human intervention.

[0003] At the same time, existing multi-robot collaboration systems often rely on a single control center for scheduling in terms of communication and task allocation, lacking autonomous intelligence. This centralized control method can easily become a bottleneck in large-scale, multi-robot systems, limiting the scalability and real-time response capability of the system. In addition, the obstacle avoidance and collaboration between robots often rely on simple sensor feedback, which cannot effectively handle the multi-target obstacle avoidance problem in complex dynamic environments, resulting in low collaboration efficiency and safety hazards.

[0004] In terms of task decomposition and allocation, existing systems mostly use task scheduling algorithms based on fixed rules, which lack flexibility and are difficult to adapt to task dependencies, resource constraints, and other issues in complex assembly tasks. Especially when the task structure is complex and there are multiple levels of dependencies between tasks, traditional task scheduling algorithms are inefficient and difficult to optimize the task load of each robot while ensuring the rationality of the task order. SUMMARY

[0005] The present application proposes a large model and knowledge graph linked multi-robot collaborative assembly system and method, aiming to at least solve one of the technical problems in the related art, the present application embodiment can efficiently and accurately realize the scheduling of multi-robot collaborative assembly.

[0006] On the one hand, the present application embodiment provides a large model and knowledge graph linked multi-robot collaborative assembly system, the system includes a perception layer, a data processing layer, a knowledge graph layer, a task decision layer, a communication and synchronization layer, and an execution layer;

[0007] The perception layer is used to collect data information of the assembly environment and assembly task in real time;

[0008] A data processing layer is configured to preprocess data information collected by the perception layer and perform deep analysis on the data information by using a preset large model to obtain structured information.

[0009] A knowledge graph layer is configured to construct a knowledge graph based on the structured information output by the data processing layer.

[0010] A task decision layer is configured to perform task decomposition, distribution and optimized scheduling based on the knowledge graph, in combination with application data of each robot and a task execution state fed back by the execution layer, to obtain a task decision instruction.

[0011] A communication and synchronization layer is configured to establish a high-speed communication network within the multi-robot collaborative assembly system and implement time synchronization of devices within the system by using a time synchronization protocol.

[0012] An execution layer is configured to control a corresponding robot to execute a distributed target assembly task based on the task decision instruction, and feed back a task execution state of the robot to the task decision layer.

[0013] Optionally, the perception layer comprises a multi-sensor fusion module and a task information acquisition module; the data information comprises environmental information and task data.

[0014] The multi-sensor fusion module is configured to collect environmental information in real time based on preset sensors; the preset sensors comprise a vision sensor, a laser radar and a force sensor.

[0015] The task information acquisition module is configured to acquire real-time updated task data through a system interface based on a standardized data format.

[0016] Optionally, the data processing layer comprises a preprocessing module and a large model processing module.

[0017] The preprocessing module is configured to preprocess the data information output by the perception layer; the preprocessing comprises data cleaning and formatting processing.

[0018] The large model processing module is configured to extract key information from the preprocessed data information by using a pre-trained large-scale language model, and then arrange the key information to obtain structured information; the structured information comprises entity information and relationship information; the entity information comprises assembly components, assembly tools, robots and assembly steps; the relationship information comprises assembly sequences and task dependency relationships.

[0019] Optionally, the structured information comprises entity information and relationship information; the knowledge graph layer comprises a knowledge graph construction module and a knowledge graph update module.

[0020] The knowledge graph construction module is configured to store the structured information as a knowledge graph by using a graph database.

[0021] The nodes in the graph database represent entity information, and the entity information includes assembly components, assembly tools, robots, and assembly steps. The edges in the graph data represent relationship information, and the relationship information includes assembly sequences and task dependency relationships.

[0022] The knowledge graph updating module is configured to listen to the update information about the structured information output by the data processing layer through a real-time listening mechanism, and update the knowledge graph in an incremental manner through a hierarchical updating mechanism based on the update information.

[0023] Optionally, the task decision layer includes a task decomposition module, a task allocation module, and a scheduling optimization module.

[0024] The task decomposition module is configured to decompose the assembly task into a plurality of subtasks based on the knowledge graph and through a topological sorting algorithm.

[0025] The task allocation module is configured to obtain a task allocation scheme by processing a preset optimization algorithm based on the assembly capability, the use state, and the load condition of each robot. The optimization algorithm includes a Hungarian algorithm or a genetic algorithm.

[0026] The scheduling optimization module is configured to optimize and adjust the task allocation scheme according to the task execution state, the load condition of the corresponding robot, the task progress, and the environmental change.

[0027] Optionally, the communication and synchronization layer includes a high-speed communication network module and a time synchronization module.

[0028] The high-speed communication network module is configured to establish a high-speed communication network inside the multi-robot collaborative assembly system based on a preset communication network and a communication protocol.

[0029] The time synchronization module is configured to realize time synchronization of devices inside the multi-robot collaborative assembly system based on a preset precise time protocol.

[0030] Optionally, the data information includes environmental information. The execution layer includes a robot control module, a cooperative obstacle avoidance module, and a state feedback module.

[0031] The robot control module is configured to control the corresponding robot to execute the allocated target assembly task according to the task decision instruction.

[0032] The cooperative obstacle avoidance module is configured to plan an obstacle avoidance path for the robot executing the task through a model predictive control algorithm based on the environmental information.

[0033] The state feedback module is configured to feed back the task execution state of the robot executing the target assembly task to the task decision layer in real time.

[0034] Optionally, the system further includes a human-computer interaction layer.

[0035] The human-machine interaction layer is used to push task data to operators or receive control commands from operators for the multi-robot collaborative assembly system based on a preset interactive interface.

[0036] Optionally, the human-computer interaction layer includes a monitoring and visualization module, a manual intervention module, and an alarm and notification module;

[0037] The monitoring and visualization module is used to push the system status of the multi-robot collaborative assembly system, the task progress of the robot in performing the target assembly task, and the robot's position information to the interactive interface.

[0038] The manual intervention module is used to respond to the control commands received from the operator through the interactive interface, adjust the task decision commands, or stop the operation of the multi-robot collaborative assembly system.

[0039] The alarm and notification module is used to push abnormal situations of the multi-robot collaborative assembly system to the interactive interface for alarm and notification.

[0040] On the other hand, embodiments of the present invention provide a multi-robot collaborative assembly method that links a large model with a knowledge graph, applied to the aforementioned multi-robot collaborative assembly system that links a large model with a knowledge graph. The method includes the following steps:

[0041] The assembly environment and assembly tasks are collected in real time through the perception layer;

[0042] The data processing layer preprocesses the data collected by the perception layer and performs in-depth analysis using a pre-set large model to obtain structured information.

[0043] The knowledge graph is constructed by the knowledge graph layer based on the structured information output by the data processing layer;

[0044] The task decision layer, based on a knowledge graph, combines application data from each robot with the task execution status feedback from the execution layer to decompose, allocate, and optimize scheduling of tasks, thereby obtaining task decision instructions.

[0045] A high-speed communication network is established within the multi-robot collaborative assembly system through a communication and synchronization layer, and time synchronization of each device in the system is achieved through a time synchronization protocol.

[0046] The execution layer controls the corresponding robot to perform the assigned target assembly task based on the task decision instructions; and feeds back the robot's task execution status to the task decision layer.

[0047] Optionally, the method further includes the following steps:

[0048] The human-computer interaction layer pushes task data to an operator or receives a control instruction of the multi-robot collaborative assembly system from the operator according to a preset interaction interface.

[0049] In another aspect, the embodiment of the present application provides an electronic device, comprising a processor and a memory; the memory is used for storing a program; and the processor executes the program to realize the multi-robot collaborative assembly method based on the large model and the knowledge graph.

[0050] In another aspect, the embodiment of the present application provides a computer storage medium, wherein the computer storage medium stores a program executable by a processor, and the program executable by the processor is used for realizing the multi-robot collaborative assembly method based on the large model and the knowledge graph when the program executable by the processor is executed by the processor.

[0051] The embodiment of the present application collects data information of an assembly environment and an assembly task in real time through the perception layer; the data information collected by the perception layer is preprocessed and analyzed in depth by using a preset large model through the data processing layer to obtain structured information; the knowledge graph is constructed according to the structured information output by the data processing layer through the knowledge graph layer; the task decision instruction is obtained by task decomposition, distribution and optimization scheduling based on the knowledge graph, in combination with application data of each robot and a task execution state fed back by the execution layer through the task decision layer; a high-speed communication network is established in the multi-robot collaborative assembly system through the communication and synchronization layer, and time synchronization of devices in the system is realized through a time synchronization protocol; the corresponding robot is controlled to execute the distributed target assembly task based on the task decision instruction through the execution layer; and the task execution state of the robot is fed back to the task decision layer. The embodiment of the present application can better handle the relationship of a complex task and optimize the task distribution process based on the structured expression capability of the knowledge graph; and a multi-robot collaborative assembly system that realizes autonomous intelligent task decomposition and distribution in combination with the application situation of the robot and the real-time task execution state can improve the flexibility, response speed and cooperation efficiency of the system. BRIEF DESCRIPTION OF DRAWINGS

[0052] The accompanying drawings are used to provide a further understanding of the technical solutions of the present application, and constitute a part of the specification, and are used to explain the technical solutions of the present application together with the embodiments of the present application, and do not constitute a limitation to the technical solutions of the present application.

[0053] Figure 1 A schematic diagram of a module composition of the multi-robot collaborative assembly system based on the large model and the knowledge graph provided by the embodiment of the present application;

[0054] Figure 2 A schematic diagram of an overall operation module architecture of the multi-robot collaborative assembly system based on the large model and the knowledge graph provided by the embodiment of the present application;

[0055] Figure 3A schematic diagram of the overall operation flow of the multi-robot collaborative assembly system that links a large model with a knowledge graph, as provided in an embodiment of the present invention.

[0056] Figure 4 This is a schematic diagram of an implementation environment for a multi-robot collaborative assembly method that links a large model with a knowledge graph, as provided in an embodiment of the present invention.

[0057] Figure 5 A flowchart illustrating the multi-robot collaborative assembly method that links a large model with a knowledge graph, as provided in an embodiment of the present invention.

[0058] Figure 6 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention. Detailed Implementation

[0059] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0060] It should be noted that although functional modules are divided in the system diagram and the logical order is shown in the flowchart, in some cases, the steps shown or described may be performed in a different order than the module division in the system or the order in the flowchart. The terms "first / S100," "second / S200," etc., in the specification, claims, and the aforementioned figures are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence.

[0061] In this document, the term "embodiment" means that a particular feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of the invention. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment mutually exclusive with other embodiments. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments.

[0062] On the one hand, such as Figure 1 As shown, this embodiment of the invention provides a multi-robot collaborative assembly system 100 that links a large model with a knowledge graph, which may include a perception layer 110, a data processing layer 120, a knowledge graph layer 130, a task decision layer 140, a communication and synchronization layer 150, and an execution layer 160; specifically:

[0063] The perception layer is used to collect data on the assembly environment and assembly tasks in real time.

[0064] It should be noted that in some embodiments, the perception layer includes a multi-sensor fusion module and a task information acquisition module; the data information includes environmental information and task data;

[0065] The multi-sensor fusion module is configured to collect environmental information in real time based on preset sensors; the preset sensors include a vision sensor, a laser radar, and a force sensor;

[0066] The task information acquisition module is configured to acquire real-time updated task data based on a standardized data format through a system interface.

[0067] By way of example, in some specific embodiments, the perception layer is responsible for collecting environmental, task, and robot state data in real time, providing basic information for subsequent data processing and task decomposition.

[0068] The multi-sensor fusion module: collects environmental information through vision sensors, laser radars, force sensors, etc., including workpiece positions, distance data, and force feedback. Through timestamp synchronization, the consistency of multi-source data is ensured. The collected data is processed through filtering, denoising, etc. and used for subsequent analysis and control.

[0069] Task information acquisition module: acquires new assembly tasks and production plans through system interfaces with ERP, MES, etc., uses standardized data formats (such as XML, JSON) to transfer task information, and ensures accurate transmission and analysis of task data.

[0070] The data processing layer is configured to preprocess the data information collected by the perception layer and perform deep analysis using a preset large model to obtain structured information.

[0071] It should be noted that in some embodiments, the data processing layer includes a preprocessing module and a large model processing module:

[0072] The preprocessing module is configured to preprocess the data information output by the perception layer; preprocessing includes data cleaning and formatting processing.

[0073] The large model processing module is configured to use a pre-trained large-scale language model to extract key information from the preprocessed data information, and then organize structured information; the structured information includes entity information and relationship information; the entity information includes assembly components, assembly tools, robots, and assembly steps; the relationship information includes assembly order and task dependency.

[0074] By way of example, in some specific embodiments, the data processing layer uses a large model to perform deep analysis on perception data, extract task-related information, and structure the data.

[0075] Preprocessing module: Clean and format the data from multiple sensors. Remove noise through algorithms such as Kalman filtering, and unify data formats for subsequent processing.

[0076] Large model processing module: Use Qwen-2-72B large model to extract key information in the environment and tasks, such as assembly steps, component names, dependency relationships, and assembly sequences. The large model generates structured information through pre-training knowledge combined with real-time environmental data, which is convenient for subsequent construction of knowledge graphs.

[0077] Some specific application scenarios:

[0078] 1. The specific training content of the large model is as follows:

[0079] Qwen-2-72B large model is a large-scale pre-training language model, and the main training content includes the following aspects:

[0080] General knowledge pre-training: Qwen-2-72B is pre-trained on a large amount of text data (such as encyclopedic articles, technical documents, academic papers, and open datasets), covering knowledge in a wide range of fields. This enables the model to handle complex language tasks and extract key information.

[0081] Task relevance: Through fine-tuning, the model will further be trained for specific scenarios of assembly tasks. The fine-tuning dataset may include assembly processes, operation steps, component names, and dependency relationships in the manufacturing industry, ensuring the applicability of the model in the manufacturing environment.

[0082] Relationship reasoning: The large model learns how to extract entities (such as components and tools) and their relationships from natural language descriptions through training, and can perform simple reasoning operations. For example, during training, the model can understand the dependency relationship "Component A must be operated after Component B is installed."

[0083] 2. Combined with real-time environmental data:

[0084] The large model combines with real-time environmental data in the following ways:

[0085] Dynamic input: Real-time environmental data (such as dynamic information collected by visual sensors and laser radars) will be filtered for noise and formatted by the preprocessing module, and then input to the large model for analysis. The large model uses pre-training knowledge to understand and reason about input data, extracting information related to task execution.

[0086] Scenario adaptation: The model makes corresponding adjustments based on pre-trained knowledge and real-time data. For example, sensor data transmitted back in real-time during the robot's execution can update the task sequence through the model's inference capabilities, identifying changes in the current assembly task (such as position deviation, step adjustment, etc.).

[0087] 3. Process of generating structured information:

[0088] The large model converts perception data into structured information through the following steps:

[0089] Entity and relationship extraction: When processing task data, the large model identifies entities (such as components, tools, assembly steps, etc.) and their relationships (such as assembly order, dependency relationships, etc.) in the assembly task.

[0090] Graph representation: Extracted entities and relationships are converted into nodes and edges to generate a knowledge graph. Nodes represent specific elements in the assembly task (such as parts, tools, robots), and edges represent their relationships (such as operation order, dependency relationships).

[0091] Dynamic update: When real-time environmental data (such as sensor information) changes, the model updates the information in the knowledge graph in real-time based on these changes, ensuring the continuity and accuracy of task planning. This process ensures data consistency and timeliness through transaction processing.

[0092] 4. Related materials and references:

[0093] Pre-training model principle: The architecture of Qwen-2-72B is based on Transformer, similar to the training process of GPT, BERT, and other general language models. Large-scale unsupervised text data is used for pre-training, and small-scale supervised data is used for fine-tuning.

[0094] Knowledge graph construction: The technology of converting natural language processing results into knowledge graphs is based on relationship extraction and entity linking. Related technologies can refer to the construction process of knowledge graphs, such as RDF (Resource Description Framework), SPARQL query language, etc.

[0095] Knowledge graph layer, used to construct a knowledge graph based on the structured information output by the data processing layer;

[0096] It should be noted that in some embodiments, the structured information includes entity information and relationship information; the knowledge graph layer includes a knowledge graph construction module and a knowledge graph update module;

[0097] Knowledge graph construction module, for storing structured information as a knowledge graph through a graph database;

[0098] Among them, the nodes in the graph database represent entity information, and the entity information includes assembly components, assembly tools, robots and assembly steps; the edges in the graph data represent relationship information, and the relationship information includes assembly sequence and task dependency relationship;

[0099] The knowledge graph updating module is configured to listen to the update information about the structured information output by the data processing layer through a real-time listening mechanism, and update the knowledge graph in an incremental manner through a hierarchical updating mechanism based on the update information.

[0100] For example, in some embodiments, the knowledge graph layer is responsible for organizing structured information into a knowledge graph, which serves as an important basis for task decomposition and optimization.

[0101] The knowledge graph construction module: the task information (such as entities and relationships) extracted by the large model is stored as a knowledge graph through a graph database. The knowledge graph nodes represent components, tools, and task steps, and the edges represent the dependency and sequence relationship between them, ensuring the integrity of the task dependency structure.

[0102] The knowledge graph updating module: through real-time monitoring of the update information output by the large model, the knowledge graph is dynamically updated to ensure the real-time and consistency of the knowledge structure after the task changes, and transaction processing is used to avoid data conflicts.

[0103] In some specific application scenarios, the rapidity and accuracy of knowledge graph updating can be ensured through the following technical means:

[0104] (1) Real-time monitoring mechanism:

[0105] Real-time data input: the knowledge graph updating module listens to dynamic information from sensors, robot feedback and large model inference results in real time. When the task or environment changes (such as robot position adjustment, assembly sequence modification), these changes will be quickly transmitted to the knowledge graph updating module through the data pipeline.

[0106] Event-driven update: the knowledge graph updating is based on an event-driven mechanism. When the system detects changes in task information, environmental data or other task execution-related information, an update operation is triggered. The large model listens to the real-time input data stream and analyzes it immediately to extract key information (such as new task dependency relationships, sequence changes, etc.), and updates the nodes and edges in the knowledge graph.

[0107] (2) Hierarchical updating mechanism:

[0108] Incremental update: in order to improve the updating speed, the knowledge graph updating is performed in an incremental manner, i.e. only the changed part (such as task dependency or robot state) is updated locally, avoiding the high computational overhead caused by full graph updating. Incremental updating identifies the changed part by comparing the new task data with the existing graph.

[0109] Parallel Processing: The update process may involve multiple robots and tasks, and to improve efficiency, the system uses parallel processing technology. The update operations of each task or robot instance can be performed simultaneously, and different update processes do not interfere with each other.

[0110] (3) Graph Database Optimization:

[0111] Index Mechanism: To speed up the query and write operations of nodes and edges, graph databases such as Neo4j use index mechanisms to ensure that the knowledge graph can quickly locate the nodes and edges that need to be modified during updates. This indexing technology ensures rapid response even in large graph structures.

[0112] Cache Strategy: To improve response speed, the knowledge graph update module may use a cache strategy to cache frequently used node and relationship data in memory. When the task allocation module requests data, it is read from the cache first, reducing frequent database access and improving real-time performance.

[0113] 2. Specific mechanisms to avoid data conflicts:

[0114] (1) Transaction Processing Mechanism:

[0115] ACID Properties of Transactions: Graph databases usually support ACID (Atomicity, Consistency, Isolation, Durability) transactions, which means that each graph update is a complete operation that cannot be divided. If a conflict or error occurs during the update process, the system will roll back the operation to ensure that the graph does not enter an inconsistent state.

[0116] Optimistic Locking Mechanism: In concurrent situations, graph databases use optimistic locking to handle conflicts. Each node and edge has a version number, and when multiple processes attempt to update the same node, only the first operation can succeed, and subsequent operations will fail and retry, avoiding data overwrite or conflict.

[0117] (2) Version Control and Rollback Mechanism:

[0118] Version Control: The knowledge graph records the version of each change during the update, and the system can roll back to the previous version in case of conflicts or errors, ensuring data integrity and consistency. This mechanism is particularly suitable for complex task update scenarios.

[0119] Data Snapshot: The system can generate snapshots of the graph at regular intervals, recording the complete state of the graph. In case of major errors or conflicts, the system can use the snapshot to restore to the previous state.

[0120] (3) Concurrency Control:

[0121] Lock mechanism: The system can control access to the knowledge graph by multiple robots or task instances through a fine-grained lock mechanism. For example, when a robot is updating the dependency relationship of a task, other processes will temporarily lock the relevant nodes to avoid data conflicts caused by simultaneous writing.

[0122] Isolation level: By setting different transaction isolation levels (such as read committed, repeatable read, etc.), the system can ensure that the reading and updating of task dependencies do not interfere with each other during concurrent transaction operations, avoiding problems such as "dirty reads" or "phantom reads."

[0123] (4) Consistency checking mechanism:

[0124] Periodic consistency check: The system will periodically perform consistency checks on the knowledge graph to ensure the correctness of task dependencies and order relationships. This process can verify the order and dependencies of tasks through topological sorting or other algorithms, and find and fix inconsistencies.

[0125] Automatic repair: When the system detects data conflicts or inconsistencies, it can automatically repair the knowledge graph according to predefined rules. For example, when a task dependency cycle occurs, the system can automatically adjust the node order or re-plan the task.

[0126] 3. Specific techniques for optimizing the update process:

[0127] Batch processing: In some cases, multiple update operations can be batched, with multiple changes completed in a single transaction, reducing the overhead and conflict probability caused by frequent small updates.

[0128] Delayed update: For some non-urgent update operations, a delayed update strategy can be used to concentrate update operations in low-load periods, avoiding competition and conflicts during peak periods.

[0129] Task decision layer, used to decompose, allocate and optimize scheduling based on the knowledge graph, combined with the application data of each robot and the task execution state feedback from the execution layer, to obtain task decision instructions;

[0130] It should be noted that in some embodiments, the task decision layer includes a task decomposition module, a task allocation module, and a scheduling optimization module;

[0131] Task decomposition module, used to decompose assembly tasks into multiple sub-tasks based on the knowledge graph through a topological sorting algorithm;

[0132] Task allocation module, used to obtain a task allocation scheme by processing a preset optimization algorithm based on the assembly capabilities, usage state, and load situation of each robot; the optimization algorithm includes the Hungarian algorithm or the genetic algorithm;

[0133] A scheduling optimization module is configured to optimize and adjust the task allocation scheme based on the task execution status, the load of the corresponding robot, the task progress, and the environmental changes.

[0134] In some embodiments, the task decision layer decomposes, allocates, and optimizes the scheduling of tasks based on the knowledge graph, ensuring the efficiency of multi-robot collaboration.

[0135] Task decomposition module: Through topological sorting analysis of the knowledge graph, the dependency relationship of the task is identified, and the complex assembly task is decomposed into executable subtasks. This module identifies parallel tasks to optimize the task execution order.

[0136] Task allocation module: Based on the ability, current state, and load of each robot, the task allocation module uses optimization algorithms (such as the Hungarian algorithm) to calculate the optimal task allocation scheme and generate standardized task instructions to issue to the robot.

[0137] Scheduling optimization module: Real-time monitoring of task execution status, dynamic adjustment of task order and allocation based on robot load, task progress, and environmental changes (can be achieved through deep learning algorithms or machine learning algorithms), ensuring efficient operation of the system in a variable production environment.

[0138] Communication and synchronization layer, used to establish a high-speed communication network within the multi-robot collaborative assembly system and achieve time synchronization of devices within the system through a time synchronization protocol.

[0139] It should be noted that in some embodiments, the communication and synchronization layer includes a high-speed communication network module and a time synchronization module.

[0140] High-speed communication network module, used to establish a high-speed communication network within the multi-robot collaborative assembly system based on a pre-set communication network and communication protocol.

[0141] Time synchronization module, used to achieve time synchronization of devices within the multi-robot collaborative assembly system based on a pre-set precise time protocol.

[0142] In some embodiments, the communication and synchronization layer provides high-speed data transmission and time synchronization within the system, ensuring consistency during multi-robot collaboration.

[0143] High-speed communication network module: Through industrial Ethernet or 5G network, real-time data transmission between modules is ensured. OPC UA or MQTT protocol is used to ensure low latency and high reliability of communication.

[0144] Time Synchronization Module: Using IEEE 1588 Precision Time Protocol (PTP), time synchronization is achieved for all robots and devices within the system, ensuring motion consistency when multiple robots collaborate.

[0145] In some specific application scenarios, the following is a detailed explanation and example of the specific application scenarios, configurations, and usage details of OPC UA and MQTT protocols in multi-robot collaboration systems:

[0146] 1. Application and Configuration of OPC UA Protocol:

[0147] (1) OPC UA Overview:

[0148] OPC UA (Unified Architecture) is a platform-independent industrial protocol designed for secure and efficient communication between devices and systems. It not only supports traditional factory automation devices but also handles complex data models, making it particularly suitable for state monitoring, command issuance, and data transmission in multi-robot collaboration systems.

[0149] Application Scenario: In a multi-robot collaboration system, OPC UA is mainly used for state synchronization, task issuance, and real-time feedback between robots and the central control system. Each robot can act as an OPC UA client, sending state information to the central control server and receiving task instructions.

[0150] (2) Configuration Details of OPC UA:

[0151] Server / Client Architecture: OPC UA follows a server / client architecture. In this system, the central control system can be configured as an OPC UA server, providing task scheduling and state query interfaces, while robots act as OPC UA clients, connecting to the server and executing related tasks.

[0152] Server Configuration: Configuring the OPC UA server requires setting specific endpoints to define resources that robots can access, such as real-time state, task list, and sensor data.

[0153] Client Configuration: The robot side needs to configure the OPC UA client to connect to the server's endpoint to obtain task instructions and report state. Typical client configurations include the OPC UA server's address, port, authentication method, etc.

[0154] Node Management: OPC UA uses a node structure to represent data, with each robot task and state information corresponding to a node. When configuring, each robot needs to be assigned a unique node ID, and task instructions and state information are interacted through accessing these nodes. The hierarchical structure of nodes helps organize and query data.

[0155] Example: In the task assignment scenario, the central control system can send task instructions to robot 1 through the / robots / robot1 / task node, and the robot can feedback the task execution progress through the / robots / robot1 / status node.

[0156] (3) Details of OPC UA usage:

[0157] Task assignment and status monitoring: In actual operation, the OPC UA server regularly receives real-time data (such as current tasks, running status, sensor data) from each robot and assigns new task instructions as needed.

[0158] Example: After completing the current task, the robot feedbacks the completion status through the status node, and the OPC UA server assigns the next task to the robot through the corresponding node according to the task scheduling algorithm.

[0159] Security configuration: OPC UA provides secure communication mechanisms, including encryption, identity verification and authorization. To ensure the security of data transmission between robots, the system can configure a certificate authentication mechanism, and the client (robot) must pass the certificate verification to connect to the server, ensuring the reliability of communication.

[0160] Real-time and reliability: OPC UA supports multiple transmission methods (such as TCP), which can run in low-latency and high-reliability network environments. The system can adjust the refresh rate (update frequency) according to specific scene requirements to ensure that robots receive real-time instructions and status feedback.

[0161] 2. Application and configuration of MQTT protocol:

[0162] (1) Overview of MQTT:

[0163] MQTT (Message Queuing Telemetry Transport) is a lightweight publish / subscribe message protocol designed for low-bandwidth and high-latency environments, with low overhead and strong flexibility. It is very suitable for communication between robots and information publishing between control systems, especially for scenarios that require low-latency data exchange.

[0164] Application scenario: In a multi-robot cooperation system, MQTT can be used to transmit task status, perception data and emergency instructions. Each robot can act as a publisher to publish its own status information, or as a subscriber to subscribe to task instructions from the central control system or status information published by other robots.

[0165] (2) Details of MQTT configuration:

[0166] Publish / Subscribe Pattern: MQTT is based on a publish / subscribe communication pattern, where robots can act as both publishers and subscribers, publishing their own state and subscribing to task instructions. When configuring MQTT, a topic needs to be specified, and each robot can have its own topic for communication.

[0167] Example: Robot 1 can subscribe to the task instruction topic / tasks / robot1 and publish its own state information to the topic / status / robot1. The control system subscribes to all robot state information and publishes task instructions.

[0168] QoS (Quality of Service) Settings: MQTT provides three levels of Quality of Service (QoS) to balance the real-time and reliability of transmission:

[0169] QoS 0 (At Most Once): The packet is transmitted only once, and it cannot guarantee that the data will reach the receiver. It is suitable for scenarios such as sensor data that do not require strict arrival guarantees.

[0170] QoS 1 (At Least Once): It guarantees that data will arrive at least once, and the receiver will confirm receipt of the message. It is suitable for task instructions or important state data.

[0171] QoS 2 (Exactly Once): It guarantees that the message will only be delivered once, suitable for extremely critical control instruction scenarios.

[0172] Broker Configuration: MQTT requires a Broker as a message relay station, responsible for receiving messages from publishers and distributing them to subscribers. The central control server in the system usually acts as a Broker role, responsible for managing message exchange between all robots.

[0173] Example: The control system can act as an MQTT Broker, receiving state publications from each robot and publishing task instructions to the corresponding robot topics.

[0174] (3) Details of using MQTT:

[0175] Task Allocation and State Feedback: Robots subscribe to task instruction topics through MQTT, and when new task instructions are published, robots can immediately obtain and execute them. At the same time, robots can also publish state messages (such as / status / robot1) to the system to feedback task execution status.

[0176] Example: When the control system publishes a task on the / tasks / robot1 topic, robot 1 receives the task and executes it, and after execution is completed, it publishes the status to the / status / robot1 topic.

[0177] Efficient broadcast mechanism: Through the topic mechanism of MQTT, the central control system can broadcast task instructions or state information to multiple robots simultaneously. By using wildcards (such as / tasks / #), multiple robots can be assigned tasks simultaneously, greatly simplifying the complexity of instruction issuance.

[0178] Connection reconnection and keep-alive mechanism: MQTT has a connection reconnection mechanism. If the robot loses network connection under certain circumstances, the system will automatically attempt to reconnect and continue publishing and subscribing messages. In addition, MQTT supports a keep-alive mechanism, and the system will ensure the stability of the connection by periodically sending keep-alive packets.

[0179] Lightweight advantage: Due to the small protocol overhead of MQTT, it is very suitable for robot state information transmission in low-bandwidth environments, ensuring effective communication even in poor network conditions.

[0180] The execution layer is configured to execute the assigned target assembly task based on the task decision instruction, and feed back the task execution state of the robot to the task decision layer.

[0181] It should be noted that in some embodiments, the data information includes environmental information; the execution layer includes a robot control module, a cooperative obstacle avoidance module, and a state feedback module.

[0182] The robot control module is configured to control the corresponding robot to execute the assigned target assembly task according to the task decision instruction.

[0183] The cooperative obstacle avoidance module is configured to plan an obstacle avoidance path for the robot executing the task based on the environmental information through a model predictive control algorithm.

[0184] The state feedback module is configured to feed back the task execution state of the robot executing the target assembly task to the task decision layer in real time.

[0185] For example, in some specific embodiments, the execution layer is composed of multiple robots, which are responsible for executing assembly tasks according to the instructions of the task decision layer, and ensuring the safety of the work through cooperation and obstacle avoidance technology.

[0186] Robot control module: According to the instructions of the task allocation module, control the execution mechanism such as the mechanical arm to execute specific assembly operations, including path planning, motion control and grasping operation, to ensure accurate task completion.

[0187] Cooperative obstacle avoidance module: Use multi-sensor data to monitor the surrounding environment in real time, and use predictive obstacle avoidance algorithms to plan robot actions to ensure that robots do not collide and task conflicts during collaborative work.

[0188] State feedback module: The robot provides real-time state feedback to the task decision layer and human-machine interaction layer during execution, including task progress, operation status, position information, etc., to facilitate timely adjustment of the system or handling of faults.

[0189] In some optional embodiments, the system can further include a human-machine interaction layer;

[0190] The human-machine interaction layer is configured to push task data to the operator or receive control instructions of the multi-robot collaborative assembly system from the operator according to a preset interaction interface.

[0191] It should be noted that in some embodiments, the human-machine interaction layer includes a monitoring and visualization module, a manual intervention module, and an alarm and notification module.

[0192] The monitoring and visualization module is configured to push the system state of the multi-robot collaborative assembly system, the task progress of the robot executing the target assembly task, and the position information of the robot to the interaction interface.

[0193] The manual intervention module is configured to adjust the task decision instruction or stop the operation of the multi-robot collaborative assembly system in response to the control instruction of the operator received by the interaction interface.

[0194] The alarm and notification module is configured to push the abnormal condition of the multi-robot collaborative assembly system to the interaction interface for alarm and prompt.

[0195] For example, in some specific embodiments, the human-machine interaction layer provides an operator with a monitoring interface of the system state and supports manual intervention and management of the task.

[0196] The monitoring and visualization module: The system running state, task progress, and robot position are displayed through a visualization interface. The operator can understand the overall situation of the current system through charts and 3D models.

[0197] The manual intervention module: allows the operator to manually adjust the task instruction or stop certain operations in special cases, ensuring that the system can be handled in time in abnormal situations.

[0198] The alarm and notification module: When the system detects an abnormal condition or task failure, the alarm module will notify the relevant personnel for fault handling through various ways such as pop-up windows, sounds, and SMS.

[0199] To explain the principle of the technical scheme of the present application, the overall process of the present application will be described below in conjunction with some specific embodiments. It should be easily understood that the following is an explanation of the technical principle of the present application and cannot be regarded as a limitation of the present application.

[0200] First of all, it needs to be pointed out that the existing production line scheduling technology has the following main shortcomings in multi-robot collaboration and dynamic scheduling:

[0201] Poor flexibility in task decomposition and allocation: Most existing multi-robot collaborative assembly systems rely on predefined rules or simple linear reasoning for task decomposition and allocation, making it difficult to cope with frequent changes in task requirements and complex dependencies. When production tasks or environments change, the system often needs manual intervention to redistribute tasks, resulting in low efficiency and inability to achieve automated dynamic adjustment.

[0202] Lack of intelligent dynamic adjustment capability: Current assembly systems rely on fixed task scheduling models and lack intelligent adaptive mechanisms. The system cannot automatically adjust according to real-time environmental data or task changes, making it difficult to meet the demand for highly flexible and adaptive production in modern manufacturing. Traditional systems show insufficient efficiency in task allocation and load balancing when faced with multi-task, multi-level task structures.

[0203] Low collaboration efficiency, communication and synchronization lag: Existing multi-robot systems rely on a single control center for task scheduling and communication, which can easily cause communication bottlenecks, resulting in low collaboration efficiency between robots. In complex assembly scenarios requiring multiple robots to collaborate simultaneously, existing systems cannot guarantee efficient collaboration of tasks, resulting in problems such as delayed synchronization, task conflicts between robots, and affecting overall production efficiency.

[0204] Insufficient obstacle avoidance and collaboration safety: Traditional systems rely on simple sensor feedback for robot obstacle avoidance, making it difficult to achieve precise multi-robot obstacle avoidance in dynamic and complex environments, leading to collisions or task interruptions during collaboration, and low safety. Especially in dense industrial assembly scenarios, the lag in obstacle avoidance algorithms can pose serious production safety hazards.

[0205] Weak knowledge representation and task association: Existing systems have difficulty effectively representing the association and dependency between tasks when dealing with complex task structures. Simple task lists or tree structures are insufficient to meet the complex task decomposition requirements, and cannot clearly display the multi-level dependencies and order of tasks, which is particularly evident in large-scale production environments.

[0206] The present application provides a multi-robot collaborative assembly system based on large model and knowledge graph linkage, as shown in Figure 2 The overall architecture mainly includes the following levels: perception layer, data processing layer, knowledge graph layer, task decision layer, communication and synchronization layer, execution layer and human-computer interaction layer. Each level exchanges data and functions through standardized interfaces and protocols, and the specific architecture is as follows:

[0207] Perception layer:

[0208] Environment Perception Module: Contains various sensors such as visual sensors (cameras), laser radars, force sensors, etc., to collect real-time dynamic information of the assembly environment, including workpiece positions, equipment states, environmental changes, etc.

[0209] Task Information Acquisition Module: Acquires new assembly task requirements, design changes, and production plans, etc. information, which can come from enterprise resource planning (ERP) systems, manufacturing execution systems (MES), or manual input.

[0210] Data Processing Layer:

[0211] Preprocessing Module: Format conversion, noise filtering, and data cleaning are performed on the raw data obtained by the perception layer to ensure data quality.

[0212] Large Model Processing Module: Deploy large-scale pre-trained models (such as Qwen-2-72B) to perform deep analysis on preprocessed data, extracting key information such as assembly components, operation steps, task dependency relationships, etc.

[0213] Knowledge Graph Layer:

[0214] Knowledge Graph Construction Module: Structurize entities and relationships extracted by the large model to construct the knowledge graph of the assembly task. Nodes represent assembly components, tools, robots, etc., and edges represent relationships between entities such as assembly order, dependency, etc. Knowledge Graph Update Module: Dynamically update the knowledge graph based on real-time new information to ensure its accuracy and timeliness.

[0215] Knowledge Storage and Query Module: Use graph database technology to support efficient knowledge storage and query operations, providing data support for the task decision layer.

[0216] Task Decision Layer:

[0217] Task Decomposition Module: Based on the latest knowledge graph, use topological sorting, depth-first search, etc. algorithms to decompose complex assembly tasks into multiple executable subtasks, and clarify the sequence and dependency of tasks.

[0218] Task Assignment Module: According to the capabilities, current state and load of each robot, use optimization algorithms (such as Hungarian algorithm, genetic algorithm) to reasonably assign subtasks to suitable robots, achieve load balancing and maximize efficiency.

[0219] Scheduling Optimization Module: Consider production priority, resource constraints, etc. to optimize task execution order, reduce waiting time and resource conflicts.

[0220] Communication and Synchronization Layer:

[0221] High-speed communication network: Establish a communication network based on industrial Ethernet or 5G to achieve high-speed data transmission between robots, sensors, and control systems.

[0222] Time synchronization module: Employs Precise Time Protocol (PTP) to ensure time synchronization among devices within the system, guaranteeing the accuracy of collaborative operations.

[0223] Protocol and interface standardization: Use standard communication protocols (such as OPC UA) to achieve seamless connection and data exchange between modules.

[0224] Execution layer:

[0225] Robot control module: Receives instructions from the task decision layer and controls the robot to perform specific assembly operations. It includes sub-modules for motion control, path planning, and operation execution.

[0226] Collaborative obstacle avoidance module: Through multi-sensor fusion, it perceives the surrounding environment and the position of other robots in real time, and uses predictive obstacle avoidance algorithms (such as model predictive control, MPC) to adjust the path and avoid collisions.

[0227] Feedback monitoring module: Monitors the robot's execution status and task progress in real time, and provides feedback information to the task decision-making layer and the human-computer interaction layer.

[0228] Human-computer interaction layer:

[0229] Monitoring and Visualization Module: Provides real-time monitoring and visualization of the system's operating status, including task progress, robot status, environmental information, etc.

[0230] Manual intervention and command input module: Allows operators to manually intervene when necessary, input new task commands or adjust system parameters.

[0231] Alarm and notification module: When the system detects an abnormal situation or malfunction, it will promptly issue an alarm message to prompt the operator to take appropriate measures.

[0232] like Figure 3 As shown, the system operation flow can be implemented as follows:

[0233] Task Acquisition and Environmental Awareness: The system acquires new assembly tasks and environmental information through the perception layer.

[0234] Data processing and knowledge extraction: The data processing layer preprocesses the information, and the large model extracts key information.

[0235] Knowledge graph construction and updating: The knowledge graph layer structures the extracted information to build or update the knowledge graph.

[0236] Task decomposition and distribution: The task decision layer decomposes tasks based on the latest knowledge graph and optimally allocates them to each robot.

[0237] Task execution and coordination: Robots in the execution layer execute tasks according to instructions and work collaboratively and avoid obstacles through the communication and synchronization layer.

[0238] State feedback and adjustment: The execution layer feeds back task execution information to the task decision layer and the human-machine interaction layer, and makes necessary adjustments or manual interventions.

[0239] Loop iteration: The system continuously monitors changes in the environment and tasks, dynamically adjusts the knowledge graph and task plan, and ensures efficient completion of assembly tasks.

[0240] In some specific implementation scenarios, to further illustrate the principle and effect of the technical solutions of the present application, the embodiments of the present application also provide a verification experiment of a multi-robot collaborative assembly system based on large model and knowledge graph linkage, which is implemented as follows:

[0241] 1. Experimental environment:

[0242] In terms of hardware platform, the experiment uses 6 humanoid collaborative robots, each equipped with a mechanical arm, a laser radar, a high-definition camera, a depth sensor, and a force sensor. These robots have high-precision motion control capabilities and support multi-degree-of-freedom operations, enabling them to perform precise operations in assembly tasks. The server used for large model inference and data processing is configured with an Intel Xeon 24-core processor, 512GB DDR4 memory, 2TB SSD storage, and an NVIDIA A100 80GB GPU to support large-scale model inference acceleration. Industrial Ethernet (EtherCAT) is used for low-latency and high-bandwidth data transmission between robots and servers, combined with IEEE 1588 PTP synchronous clock system to achieve time synchronization of all devices.

[0243] In terms of software system, the experiment uses Qwen-2-72B large model as the core knowledge extraction engine, which is deployed on the GPU of the server for real-time inference and information extraction based on assembly task information and environmental data. Neo4j graph database is used to store assembly task entities, dependency relationships, and assembly sequences extracted from the large model, forming a complete knowledge graph. The task scheduling system is based on the Hungarian algorithm to achieve optimization and load balancing of multi-robot task allocation, and the control system is responsible for receiving task instructions and performing assembly operations through ROS (Robot Operating System).

[0244] Data sources, the assembly task data in the experiment is based on the assembly scene of real mechanical parts, including task decomposition and dependence of multiple steps. The robot collects real-time dynamic change information in the working environment through visual sensors, laser radars and force sensors, and performs positioning, obstacle avoidance and operation feedback. The experimental scene simulates the complex assembly line environment in actual production, and each workstation is completed by robots to grab, assemble and install operations. To further test the adaptability and flexibility of the system in complex environments, dynamic changes in workpiece position and work area layout are introduced in the experimental scene to evaluate the performance of the system in real-time task scheduling.

[0245] 2. Experimental steps:

[0246] To verify the performance of the multi-robot collaborative assembly system of the present application, the experiment is divided into the following specific steps. The experimental scene is set as a typical mechanical part assembly line, covering task decomposition, task allocation, robot execution and collaborative operation, etc. The experiment tests the task adaptability and scheduling optimization effect of the system in dynamic environment through multiple rounds of task execution.

[0247] Step 1: Experimental environment initialization

[0248] Robot and equipment start:

[0249] First, start the 6 collaborative robots used in the experiment, ensure that all hardware devices such as mechanical arms, visual sensors, laser radars, force sensors, etc. are in working condition, and establish stable communication connection with the control server.

[0250] System network configuration:

[0251] Configure the industrial Ethernet (EtherCAT) network to ensure low-latency communication between the robot and the server. Complete time synchronization through IEEE 1588 PTP protocol to ensure that the robots in the experiment can accurately coordinate working time.

[0252] Task data import:

[0253] Import the assembly task description file from the experimental task library. The task file contains part names, assembly steps, dependency relationships, etc. Ensure that the task information is accurate and correct, and get the latest task updates through the interface with ERP, MES system.

[0254] Step 2: Large model inference and knowledge graph construction

[0255] Large model information extraction:

[0256] The imported task data is input into the Qwen-2-72B large model, and the model is started for inference. The large model extracts key information from the assembly document based on the task description, including component names, assembly sequence, dependency relationships, etc.

[0257] Knowledge graph construction:

[0258] The assembly information extracted by the model is automatically input into the Neo4j graph database to form a knowledge graph. The nodes in the knowledge graph represent assembly components, tools, and robots, and the edges represent dependencies and sequences between assembly steps. Ensure that the knowledge graph reflects the completeness and structured dependencies of the task.

[0259] Graph update and inspection:

[0260] Check if the data in the knowledge graph is complete and make necessary adjustments. According to the task changes and robot feedback information, the knowledge graph is dynamically updated during operation to ensure the accuracy of task planning.

[0261] Step three: task decomposition and allocation

[0262] Task decomposition:

[0263] The task scheduling system uses the topological sorting algorithm to decompose complex assembly tasks into multiple parallel or dependent subtasks based on the dependency relationships in the knowledge graph. Task decomposition includes multiple steps such as mechanical part grabbing, installation, and precise alignment.

[0264] Task allocation:

[0265] The system optimally allocates the decomposed tasks. Through the Hungarian algorithm, the optimal task load of each robot is calculated to ensure load balancing in the allocated tasks. Task allocation takes into account the current state, skills, and location of the robot, and generates standardized task instructions.

[0266] Step four: task execution and collaboration

[0267] Task execution:

[0268] After each robot receives the allocated task instructions, it starts the ROS control system and executes specific assembly operations according to the path planning and action sequence. The robot adjusts operation parameters in real time through force sensing and visual feedback to ensure accurate task completion.

[0269] Collaborative obstacle avoidance:

[0270] In the process of collaborative work, multiple robots share the workspace. Laser radar and visual sensors monitor other robots and obstacles in the environment in real time, and the system adjusts the motion trajectory of the robot through the model predictive control (MPC) algorithm to avoid collisions and ensure the safety of collaboration.

[0271] State feedback and adjustment:

[0272] Robots provide real-time feedback of task status, position and load information to the server during execution. The task scheduling system dynamically adjusts the allocation of unfinished tasks based on feedback information to ensure the continuity and effectiveness of tasks.

[0273] Step five: dynamic adjustment of experimental scene

[0274] Dynamic scene simulation:

[0275] Dynamic scene changes are introduced in the experiment, including changes in workpiece position and adjustment of work area layout. The system senses environmental changes in real time based on sensor feedback and updates the knowledge graph through large model, automatically adjusting task planning and allocation.

[0276] Task reassignment:

[0277] In a dynamic scene, some robots may not be able to continue their original tasks due to environmental changes. The task scheduling system reassigns unfinished tasks and adjusts the execution order of robots to ensure successful completion of assembly tasks through real-time calculation.

[0278] Step six: experimental data recording and analysis

[0279] Data recording:

[0280] During the experiment, the execution time, task success rate, obstacle avoidance, and number of task allocation adjustments of each robot are recorded as key performance indicators. Sensor data, task scheduling logs, and system feedback information are stored in real time for subsequent analysis.

[0281] Experimental result analysis:

[0282] By comparing the execution efficiency, task allocation balance, and robot collaboration safety of different rounds of experiments, the system's adaptability in dynamic environments is analyzed. The results are used to evaluate the actual performance of the large model and knowledge graph system in complex assembly tasks.

[0283] In summary, the present invention proposes a multi-robot collaborative assembly system based on large model and knowledge graph linkage, successfully solving the problems of poor task decomposition and allocation flexibility, insufficient dynamic adaptability, low communication synchronization efficiency, weak safety and coordination in existing multi-robot collaboration systems. Through the innovative technical design of the present invention, the system exhibits significant superiority in complex and variable assembly scenarios. The present invention introduces the combination of large models and knowledge graphs to build a system with real-time dynamic task decomposition, intelligent task allocation, and multi-robot collaboration. The specific technical problems solved include:

[0284] 1. Achieve dynamic and intelligent task decomposition and allocation according to task and environmental changes, avoid human intervention and improve system adaptability.

[0285] 2. Real-time extraction of key data in assembly tasks using large models, and automatic task decomposition and optimized allocation based on knowledge graph.

[0286] 3. Through high-speed communication and synchronization mechanism, improve the efficiency of multi-robot cooperation, avoid communication delay and task conflict.

[0287] 4. Enhance the safety of robot cooperation, avoid collision and task interruption in dynamic and complex environment.

[0288] 5. Use knowledge graph to structurally represent complex tasks and their dependencies to improve the accuracy and efficiency of task decomposition and execution.

[0289] Compared with the prior art, the present application has at least the following beneficial effects:

[0290] Intelligent task decomposition and optimized allocation: The present application uses large models (such as Qwen-2-72B) for automatic information extraction of assembly tasks, constructs a structured knowledge graph, and realizes intelligent decomposition of tasks through topological sorting algorithm. Compared with existing systems that rely on manual or rule programming for task planning, the present application can dynamically adjust task decomposition when task requirements or environmental changes occur, reducing human intervention and significantly improving the task adaptability and automation of the system. Through Hungarian algorithm to optimize task allocation, ensure robot load balancing, maximize resource utilization efficiency.

[0291] Real-time dynamic adaptation and automatic update capability: The present application combines the powerful information extraction capability of large models and the dynamic update mechanism of knowledge graph, and can automatically update task dependencies and execution plans according to real-time sensor feedback. When the assembly task or environment changes (such as workpiece position change or work area layout adjustment), the system can adjust the task planning and allocation scheme in real time to ensure the continuity and efficiency of task execution, and enhance the flexibility and adaptability of the system to complex environments.

[0292] Efficient communication and synchronization mechanism: In the prior art, multi-robot cooperation systems have bottlenecks in communication delay and task synchronization, especially when multiple robots work in parallel, communication delay and task conflict problems are prone to occur. The present application uses industrial Ethernet (EtherCAT) and IEEE 1588 PTP protocol to ensure high-speed data transmission and precise time synchronization within the system. Each robot can perform task coordination and obstacle avoidance with microsecond-level time accuracy, avoiding task conflict and coordination difficulty caused by delayed communication in traditional systems, greatly improving cooperation efficiency.

[0293] Safe and reliable collaborative obstacle avoidance: Multi-robot collaboration is prone to collision risks, especially in dynamic and complex assembly environments. Existing systems have difficulty in accurately avoiding obstacles. This invention uses multi-sensor data fusion (such as laser radar, vision sensor and force sensor) to perceive the surrounding environment and other robot actions in real time, and combines model predictive control (MPC) algorithm to plan obstacle avoidance path. This scheme not only ensures the safe cooperation of robots in high-density working environment, but also effectively reduces the misoperation and task interruption, thereby improving the safety and production efficiency of the system.

[0294] Modular design and strong scalability: This invention adopts a modular system architecture design, and each layer interacts with data through standardized interfaces, facilitating system expansion and maintenance. In particular, the combination of knowledge graph and task scheduling layer, the system has good scalability and can be flexibly adjusted and optimized according to different task requirements or production environments, adapting to different scale assembly scenarios. Compared with traditional closed systems, this invention greatly improves the scalability and application range of the system.

[0295] Reduce labor costs and improve production efficiency: This invention has the ability of automatic task decomposition, real-time optimization scheduling and dynamic adjustment, reducing the need for human intervention. The system can independently handle complex dependency relationships and changing scenarios in tasks, reducing the dependence on professional technical personnel and improving production efficiency and reducing labor costs in the production process.

[0296] Significantly improve assembly quality and accuracy: Through the combination of large models and knowledge graphs, this invention can provide higher precision control and task execution accuracy in assembly tasks. Multi-sensor such as force sensor provides real-time feedback on operation results, combined with intelligent control algorithms to ensure accurate operation and component installation during assembly. This makes the assembly task not only fast, but also stable in quality, thereby improving the overall product quality.

[0297] In another aspect, the embodiment of the present application also provides a large model and knowledge graph linkage multi-robot collaborative assembly method. It can be understood that the large model and knowledge graph linkage multi-robot collaborative assembly method provided by the embodiment of the present application can be applied to any computer device with data processing and calculation capability. The computer device can be various terminals or servers. When the computer device in the embodiment is a server, the server is an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server providing cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDN (Content Delivery Network), and basic cloud computing services such as big data and artificial intelligence platforms. Alternatively, the terminal is a smart phone, a tablet computer, a notebook computer, and a desktop computer, but is not limited thereto.

[0298] As shown in Figure 4 , it is an implementation environment schematic diagram provided by the embodiment of the present application. Referring to Figure 4 , the implementation environment includes at least one terminal 102 and a server 101. The terminal 102 and the server 101 can be connected through wireless or wired network to complete data transmission and exchange.

[0299] The server 101 can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server providing cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDN (Content Delivery Network), and basic cloud computing services such as big data and artificial intelligence platforms.

[0300] In addition, the server 101 can also be a node server in a blockchain network. The blockchain is a new application mode of distributed data storage, point-to-point transmission, consensus mechanism, encryption algorithm, and other computer technologies.

[0301] The terminal 102 can be a smart phone, a tablet computer, a notebook computer, a desktop computer, a smart speaker, a smart watch, and the like, but is not limited thereto. The terminal 102 and the server 101 can be directly or indirectly connected through wired or wireless communication, which is not limited in the embodiment of the present application.

[0302] Exemplarily based on Figure 4In the illustrated implementation environment, the embodiment of the present application provides a large model and knowledge graph linkage multi-robot collaborative assembly method. The following takes the large model and knowledge graph linkage multi-robot collaborative assembly method applied to the terminal 102 as an example for description. It can be understood that the large model and knowledge graph linkage multi-robot collaborative assembly method can also be applied to the server 101.

[0303] Referring to Figure 5 , Figure 5 The flowchart of the large model and knowledge graph linkage multi-robot collaborative assembly method applied to the terminal provided by the embodiment of the present application. The execution subject of the large model and knowledge graph linkage multi-robot collaborative assembly method can be any one of the aforementioned computer devices (including servers or terminals). Referring to Figure 5 , the method can include the following steps:

[0304] S100, collecting data information of the assembly environment and the assembly task in real time through the perception layer;

[0305] S200, pre-processing the data information collected by the perception layer and performing deep analysis using a preset large model through the data processing layer to obtain structured information;

[0306] S300, constructing a knowledge graph according to the structured information output by the data processing layer through the knowledge graph layer;

[0307] S400, based on the knowledge graph, combining the application data of each robot and the task execution state feedback by the execution layer to perform task decomposition, distribution and optimization scheduling to obtain task decision instructions through the task decision layer;

[0308] S500, establishing a high-speed communication network inside the multi-robot collaborative assembly system through the communication and synchronization layer, and realizing time synchronization of each device in the system through a time synchronization protocol;

[0309] S600, based on the task decision instructions, controlling the corresponding robot to execute the assigned target assembly task through the execution layer; and feeding back the task execution state of the robot to the task decision layer.

[0310] In some embodiments, the method can further include the following steps:

[0311] According to the preset interaction interface, the human-computer interaction layer pushes task data to the operator or receives the control instructions of the operator to the multi-robot collaborative assembly system.

[0312] It should be noted that the content of the system embodiment of the present application is applicable to the method embodiment, and the process steps specifically implemented by the method embodiment are the same as those of the above-mentioned system embodiment, and the beneficial effects achieved are also the same as those of the above-mentioned system. It should be understood that the function logic realized by the specific modules of each part of the above-mentioned system embodiment belongs to the protection scope of the method embodiment of the present application, and will not be repeated here.

[0313] In another aspect, the embodiment of the present application also provides an electronic device, which comprises a memory and a processor, the memory stores a computer program, and the processor realizes the above-mentioned large model and knowledge graph linkage multi-robot collaborative assembly method when executing the computer program. The electronic device can be any intelligent terminal including a tablet computer, a vehicle-mounted computer, etc.

[0314] It can be understood that the content of the above-mentioned method embodiment is applicable to the device embodiment, and the function specifically implemented by the device embodiment is the same as that of the above-mentioned method embodiment, and the beneficial effects achieved are also the same as those of the above-mentioned method embodiment.

[0315] As shown in Figure 6 , a hardware structure specific example of an electronic device 1000 of an embodiment is shown, which comprises: Figure 6 The processor 1001 can be implemented in the form of a general-purpose CPU (Central Processing Unit), a microprocessor, an ASIC (Application Specific Integrated Circuit), or one or more integrated circuits, etc., for executing related programs to realize the technical solutions provided by the embodiments of the present application;

[0316] The memory 1002 can be implemented in the form of a ROM (Read Only Memory), a static storage device, a dynamic storage device, or a RAM (Random Access Memory), etc. The memory 1002 can store an operating system and other application programs, and when the technical solutions provided by the embodiments of the present application are implemented by software or firmware, the related program codes are saved in the memory 1002 and called and executed by the processor 1001 to realize the network node population optimization method of the embodiments of the present application;

[0317] The input / output interface 1003 is used to realize information input and output;

[0318]

[0319] ​The communication interface 1004 is configured to realize the communication interaction between the device and other devices, and can realize the communication through a wired manner (for example, a USB, a network cable and the like) or a wireless manner (for example, a mobile network, WIFI, Bluetooth and the like).

[0320] The bus 1005 is configured to transmit information between various components (for example, the processor 1001, the memory 1002, the input / output interface 1003 and the communication interface 1004) of the device.

[0321] The processor 1001, the memory 1002, the input / output interface 1003 and the communication interface 1004 are connected to each other through the bus 1005 to realize the communication connection between the device.

[0322] The electronic device embodiments described above are only schematic, and the units described as separate components can or can not be physically separate, that is, can be located in one place or can be distributed on multiple network units. According to actual needs, part or all of the modules can be selected to realize the purposes of the embodiments.

[0323] The contents of the method embodiments of the present application are applicable to the electronic device embodiments, the electronic device embodiments specifically realize the functions of the above method embodiments, and the beneficial effects achieved by the electronic device embodiments are the same as the beneficial effects achieved by the above method.

[0324] Another aspect of the embodiments of the present application further provides a computer readable storage medium, the storage medium stores a program, and the program is executed by a processor to realize the above method.

[0325] It should be noted that the computer readable medium shown in the embodiments of the present application can be a computer readable signal medium or a computer readable storage medium or any combination of the two. The computer readable storage medium may, for example, but is not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or component, or any combination of the above. More specific examples of computer readable storage media can include, but are not limited to, an electrical connection having one or more wires, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM), a flash memory, an optical fiber, a portable compact disk read-only memory (Compact Disc Read-Only Memory, CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present application, the computer readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, device or component. In the present application, the computer readable signal medium can include a data signal carried in a baseband or as a part of a carrier wave, which carries computer readable program code. Such a propagated data signal can take many forms, including but not limited to electromagnetic signals, optical signals or any suitable combination thereof. The computer readable signal medium can also be any computer readable medium other than the computer readable storage medium, which can send, propagate or transmit a program for use by or in conjunction with an instruction execution system, device or component. The program code contained on the computer readable medium can be transmitted by any suitable medium, including but not limited to wireless, wired or the like, or any suitable combination thereof.

[0326] The contents of the method embodiments of the present application are applicable to the computer readable storage medium embodiments of the present application. The computer readable storage medium embodiments specifically implement the same functions as the above-mentioned method embodiments, and achieve the same beneficial effects as the above-mentioned methods.

[0327] The embodiments of the present application also disclose a computer program product or computer program, which comprises computer instructions stored in a computer readable storage medium. The processor of the computer device can read the computer instructions from the computer readable storage medium, and the processor executes the computer instructions to make the computer device execute the above method.

[0328] The computer program product of the present application can be a storage medium or a computer program product storing instructions which, when executed by one or more processors, cause the processor or processors to carry out the steps described herein. The computer program product can be embodied in one or more computer- readable media, which can be non-transitory. The computer-readable media can include computer- readable storage media, which can be tangible or in other forms. The computer-readable storage media include, but are not limited to, RAM, ROM, EEPROM, solid state drives (SSDs), flash memory, phase change memory, or any other memory storage technologies, CD ROMs, digital versatile discs (DVDs), etc. The computer program product can also be propagated by a computer data signal embodied in a carrier wave, modulated onto a carrier, or frequencies, a phone call, a telephone conversation, or a voice, modem, radio frequency (RF) or red laser, or other mechanism for transmitting information.

[0329] It should be noted that although several modules of the device for action execution are mentioned in the above detailed description, such a division is not mandatory. Indeed, according to embodiments of the application, the features and functionalities of two or more modules or units described above can be embodied in one module or unit. Conversely, the features and functionalities of one module or unit described above can be further divided into embodiments by a plurality of modules or units.

[0330] From the above description of the embodiments, it is easy for those skilled in the art to understand that the example embodiments described herein can be implemented by software, or by software in combination with necessary hardware. Therefore, the technical solutions according to the embodiments of the present application can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a USB flash disk, a mobile hard disk, etc.) or a network, and includes a number of instructions to enable a computing device (which can be a personal computer, a server, a touch terminal, or a network device, etc.) to perform the methods according to the embodiments of the present application.

[0331] In some alternative embodiments, the functions / operations mentioned in the block diagrams can not occur in the order mentioned in the operation diagrams. For example, depending on the functions / operations involved, two blocks shown in succession can actually be executed substantially concurrently or the blocks can sometimes be executed in reverse order. Furthermore, the embodiments presented and described in the flow diagrams of the present application are only examples. The steps presented and described in the flow diagrams are provided by way of example in order to provide a more thorough understanding of the technology. The disclosed methods are not limited to the operations and logical flows presented in the specification. Alternative embodiments are contemplated, in which the order of various steps is changed and in which sub-steps of a larger step are performed in parallel.

[0332] Furthermore, although the present application is described in the context of functional modules, it is to be understood that one or more of the functions and / or features of'can be integrated in a single physical device and / or software module, or one or more functions and / or features can be implemented in separate physical devices or software modules. It will also be understood that detailed discussion of the actual implementation of each module is unnecessary to an understanding of the present application. Rather, the actual implementation of the modules, in light of the description of the properties, functions and interrelationships of the various functional modules disclosed herein, will be apparent to one of ordinary skill in the art. Accordingly, the present application is not limited to the specific embodiments illustrated herein, but rather is inclusive of various alternatives, modifications and equivalents that can be apparent to one of ordinary skill in the art. The specific embodiments described herein are illustrative of the broader application, but are not to be considered limiting of the present application, the scope of which is defined by the appended claims and their equivalents.

[0333] If the functions are implemented in software, the functions can be stored in or implemented as one or more software modules on a computer-readable storage medium. In terms of this understanding, the technical solutions of the present application, or the parts of the present application that essentially contribute to the prior art, or the parts of the present application can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes a plurality of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the embodiments of the present application. The aforementioned storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), a magnetic disk or an optical disk, and various media that can store program codes.

[0334] The logic and / or steps represented in the flowcharts, or otherwise described herein, for example, can be considered as a list of executable instructions for implementing logic functions, and can be specifically embodied in any computer-readable medium for use by an instruction execution device, device or equipment (such as a computer-based device, a device including a processor, or other devices that can fetch and execute instructions from an instruction execution device, device or equipment), or in conjunction with these instruction execution devices, devices or equipment. For the purpose of this specification, "computer-readable medium" can be any device that can contain, store, communicate, propagate or transport programs for use by an instruction execution device, device or equipment, or in conjunction with these instruction execution devices, devices or equipment.

[0335] More specific examples (a non-exhaustive list) of the computer-readable medium include the following: an electrical connection (electronic) having one or more wires, a portable computer diskette (magnetic), a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), an optical fiber, and a portable compact disc read-only memory (CDROM). Additionally, the computer-readable medium can also be paper or another suitable medium upon which the program is printed, as the program can be electronically captured, for example, via optical scanning of the paper or other medium, then compiled, interpreted, or otherwise processed in a suitable manner, if necessary, and then stored in a computer memory.

[0336] It should be understood that aspects of the application can be implemented in hardware, software, firmware or combinations thereof. In the embodiments described above, various steps or methods can be implemented, for example, by software or firmware stored in memory and executed by a suitable instruction execution device. If implemented in hardware, for example, in another embodiment, any of the following techniques can be used in whole or in part: discrete logic circuitry having logic gates for implementing logic functions upon data signals, application specific integrated circuits having appropriate combinational logic gates, programmable gate arrays (PGA), field programmable gate arrays (FPGA), and so forth, as well as or instead.

[0337] In the description of the specification, the description of the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" and the like means that the specific features, structures, materials or characteristics described in connection with the embodiment or example are included in at least one embodiment or example of the present application. In the specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any appropriate manner in one or more embodiments or examples.

[0338] Although embodiments of the present application have been shown and described, it would be recognized by those of ordinary skill in the art that various changes, modifications, substitutions, and alterations can be made hereto without departing from the principles and spirit of the application, and it is intended that the scope of the application be limited only by the claims and equivalents thereof.

[0339] The above is a specific description of the preferred embodiment of the present application, but the present application is not limited to the embodiment, and those skilled in the art can make various equivalent modifications or replacements without departing from the spirit of the present application, and these equivalent modifications or replacements are all included in the scope defined by the claims of the present application.

Claims

1. A large model and knowledge graph linkage multi-robot collaborative assembly system, characterized in that, The system comprises a perception layer, a data processing layer, a knowledge graph layer, a task decision layer, a communication and synchronization layer, and an execution layer. The perception layer is configured to collect data information of an assembly environment and an assembly task in real time. The data processing layer is configured to pre-process the data information collected by the perception layer and perform deep analysis using a preset large model to obtain structured information. The data processing layer comprises a pre-processing module and a large model processing module. The pre-processing module is configured to pre-process the data information output by the perception layer. The pre-processing module is configured to pre-process the data information output by the perception layer. The large model processing module is configured to extract key information from the pre-processed data information using a pre-trained large-scale language model, and further arrange the structured information. The large-scale language model uses Qwen-2-72B large model. The training content of the Qwen-2-72B large model includes the following aspects: General knowledge pre-training: Qwen-2-72B is pre-trained on massive text data, covering knowledge in various fields, so that the model has the ability to handle complex language tasks and extract key information. Task relevance: through fine-tuning, the model is further trained for specific scenarios of assembly tasks. Relationship reasoning: the large model learns to extract entities and their relationships from natural language descriptions through training. Furthermore, the large-scale combines with real-time environment data in the following ways: Dynamic input: real-time environment data is filtered and formatted by the pre-processing module, and then input to the large model for analysis. Scene adaptation: the large model adjusts according to the pre-trained knowledge and real-time data, including updating the task order through the model's reasoning ability based on real-time sensor data transmitted by the robot during execution, and identifying whether there are changes in the current assembly task. The knowledge graph layer is configured to construct a knowledge graph based on the structured information output by the data processing layer. The task decision layer is configured to perform task decomposition, allocation, and optimized scheduling based on the knowledge graph, combined with the application data of each robot and the task execution state feedback from the execution layer, to obtain task decision instructions. The communication and synchronization layer is configured to establish a high-speed communication network within the multi-robot collaborative assembly system and realize time synchronization of devices within the system through a time synchronization protocol. The execution layer is configured to control the corresponding robot to execute the allocated target assembly task based on the task decision instructions, and feed back the task execution state of the robot to the task decision layer.

2. The multi-robot collaborative assembly system with large model and knowledge graph linkage of claim 1, wherein, The perception layer comprises a multi-sensor fusion module and a task information acquisition module; the data information comprises environmental information and task data; The multi-sensor fusion module is configured to collect the environmental information in real time based on preset sensors; the preset sensors comprise a vision sensor, a laser radar and a force sensor; The task information acquisition module is configured to acquire the task data updated in real time through a system interface based on a standardized data format.

3. The large model and knowledge graph linked multi-robot collaborative assembly system of claim 1, wherein, The structured information comprises entity information and relationship information; the knowledge graph layer comprises a knowledge graph construction module and a knowledge graph update module; The knowledge graph construction module is configured to store the structured information as the knowledge graph through a graph database; The nodes in the graph database represent the entity information, which comprises assembly components, assembly tools, robots and assembly steps; the edges in the graph database represent the relationship information, which comprises assembly sequences and task dependency relationships; The knowledge graph update module is configured to listen to update information about the structured information output by the data processing layer through a real-time listening mechanism, and update the knowledge graph in an incremental manner through a hierarchical update mechanism based on the update information.

4. The large model and knowledge graph linked multi-robot collaborative assembly system of claim 1, wherein, The task decision layer comprises a task decomposition module, a task allocation module and a scheduling optimization module; The task decomposition module is configured to decompose the assembly task into a plurality of subtasks through a topological sorting algorithm based on the knowledge graph; The task allocation module is configured to obtain a task allocation scheme by processing based on a preset optimization algorithm, the optimization algorithm comprising a Hungarian algorithm or a genetic algorithm, the assembly capacity, the use state and the load condition of each robot; The scheduling optimization module is configured to optimize and adjust the task allocation scheme according to the task execution state in combination with the load condition, the task progress and the environmental change of the corresponding robot.

5. The large model and knowledge graph linked multi-robot collaborative assembly system of claim 1, wherein, The communication and synchronization layer comprises a high-speed communication network module and a time synchronization module; The high-speed communication network module is configured to establish the high-speed communication network inside the multi-robot collaborative assembly system based on a preset communication network and a communication protocol; The time synchronization module is configured to realize time synchronization of devices inside the multi-robot collaborative assembly system based on a preset precise time protocol.

6. The large model and knowledge graph linked multi-robot collaborative assembly system of claim 1, wherein, The data information comprises environmental information; the execution layer comprises a robot control module, a cooperative obstacle avoidance module and a state feedback module; The robot control module is configured to control the corresponding robot to execute the allocated target assembly task according to the task decision instruction; The cooperative obstacle avoidance module is configured to plan an obstacle avoidance path for the robot executing a task through a model predictive control algorithm according to the environmental information; The state feedback module is configured to feed back the task execution state of the robot executing the target assembly task to the task decision layer in real time.

7. The large model and knowledge graph linked multi-robot collaborative assembly system of claim 1, wherein, The system further comprises a human-machine interaction layer; The human-machine interaction layer is configured to push task data to an operator or receive control instructions of the multi-robot collaborative assembly system from the operator according to a preset interaction interface.

8. The large model and knowledge graph linked multi-robot collaborative assembly system of claim 7, wherein, The human-computer interaction layer comprises a monitoring and visualization module, a manual intervention module and an alarm and notification module; The monitoring and visualization module is configured to push system status of the multi-robot collaborative assembly system, task progress of the robot in performing the target assembly task and position information of the robot to the interactive interface; The manual intervention module is configured to adjust the task decision instruction or stop operation of the multi-robot collaborative assembly system in response to the control instruction of the operator received by the interactive interface; The alarm and notification module is configured to push abnormal conditions of the multi-robot collaborative assembly system to the interactive interface for alarm and prompt.

9. A large model and knowledge graph linkage multi-robot collaborative assembly method, characterized in that, The method applied to the multi-robot collaborative assembly system of large model and knowledge graph linkage according to any one of claims 1 to 8 comprises the following steps: Real-time collection of data information of an assembly environment and an assembly task through a perception layer; Preprocessing of the data information collected by the perception layer and deep analysis using a preset large model through a data processing layer to obtain structured information; Construction of a knowledge graph according to the structured information output by the data processing layer through a knowledge graph layer; Task decomposition, distribution and optimized scheduling based on the knowledge graph, combined with application data of each robot and task execution status feedback by the execution layer to obtain a task decision instruction through a task decision layer; Establishment of a high-speed communication network within the multi-robot collaborative assembly system through a communication and synchronization layer, and realization of time synchronization of devices within the system through a time synchronization protocol; Control of the corresponding robot to perform the distributed target assembly task based on the task decision instruction through an execution layer; feedback of the task execution status of the robot to the task decision layer.

Images