Large model and knowledge graph linkage multi-robot collaborative assembly system and method

Abstract

The invention discloses a large model and knowledge graph linked multi-robot collaborative assembly system and method. The method comprises the following steps: acquiring data information of an assembly environment and an assembly task in real time through a sensing layer; preprocessing and deeply analyzing the data information through a data processing layer to obtain structured information; constructing a knowledge graph through a knowledge graph layer according to the structured information; task decomposition, distribution and optimization scheduling are carried out based on the knowledge graph through a task decision-making layer, and a task decision-making instruction is obtained; and controlling the corresponding robot to execute the allocated target assembly task through the execution layer based on the task decision instruction. Based on the structured expression ability of the knowledge graph, the multi-robot collaborative assembly system can better process the relation of complex tasks and optimize the task allocation process, realizes autonomous intelligent task decomposition and allocation, can improve the flexibility, response speed and collaborative efficiency of the system, and can be widely applied to the technical field of data processing.

Description

Technical Field

[0001] The present invention relates to the field of data processing technology, and in particular to a multi-robot collaborative assembly system and method that links a large model with a knowledge graph. Background Art

[0002] With the rapid development of industrial automation, robots are being introduced into more and more manufacturing scenarios to complete complex assembly tasks. However, in existing technologies, multi-robot collaborative assembly systems face multiple challenges. Traditional task allocation and collaboration methods mainly rely on predefined rules or simple linear reasoning models, which show limitations when faced with complex and changing tasks and environments. Existing assembly systems usually require human programming to define the tasks and paths of each robot, which makes it difficult to quickly adapt to frequent task changes and dynamic adjustments to the environment, resulting in low production efficiency and increased costs for human intervention.

[0003] At the same time, existing multi-robot collaborative systems often rely on a single control center for scheduling in terms of communication and task allocation, and lack autonomous intelligence. This centralized control method can easily become a bottleneck in large-scale, multi-robot systems, limiting the system's scalability and real-time response capabilities. In addition, obstacle avoidance and collaboration between robots often rely on simple sensor feedback, which cannot effectively handle multi-target obstacle avoidance problems in complex dynamic environments, resulting in low collaboration efficiency and potential safety hazards.

[0004] In terms of task decomposition and allocation, most existing systems use task scheduling algorithms based on fixed rules. These algorithms lack flexibility and are difficult to adapt to issues such as task dependency and resource constraints in complex assembly tasks. Especially when the task structure is complex and there are multiple layers of dependencies between tasks, traditional task scheduling algorithms are inefficient and it is difficult to optimize the task load of each robot while ensuring the rationality of the task sequence. Summary of the invention

[0005] The present invention proposes a multi-robot collaborative assembly system and method that links a large model with a knowledge graph, aiming to solve at least one of the technical problems in the related technology to a certain extent. The embodiments of the present invention can efficiently and accurately realize the scheduling of multi-robot collaborative assembly.

[0006] On the one hand, an embodiment of the present invention provides a multi-robot collaborative assembly system that links a large model with a knowledge graph, the system comprising a perception layer, a data processing layer, a knowledge graph layer, a task decision layer, a communication and synchronization layer, and an execution layer;

[0007] The perception layer is used to collect data information about the assembly environment and assembly tasks in real time;

[0008] The data processing layer is used to pre-process the data information collected by the perception layer and perform in-depth analysis using a preset large model to obtain structured information;

[0009] The knowledge graph layer is used to construct a knowledge graph based on the structured information output by the data processing layer;

[0010] The task decision layer is used to decompose, allocate and optimize tasks based on the knowledge graph, combined with the application data of each robot and the task execution status fed back by the execution layer, to obtain task decision instructions;

[0011] The communication and synchronization layer is used to establish a high-speed communication network within the multi-robot collaborative assembly system and realize the time synchronization of each device in the system through the time synchronization protocol;

[0012] The execution layer is used to control the corresponding robot to perform the assigned target assembly task based on the task decision instructions; and feed back the robot's task execution status to the task decision layer.

[0013] Optionally, the perception layer includes a multi-sensor fusion module and a task information acquisition module; the data information includes environmental information and task data;

[0014] A multi-sensor fusion module is used to collect environmental information in real time based on preset sensors; the preset sensors include visual sensors, laser radars, and force sensors;

[0015] The task information acquisition module is used to obtain real-time updated task data through the system interface based on a standardized data format.

[0016] Optionally, the data processing layer includes a preprocessing module and a large model processing module:

[0017] The preprocessing module is used to preprocess the data information output by the perception layer; the preprocessing includes data cleaning and formatting;

[0018] The large model processing module is used to use the pre-trained large-scale language model to extract key information from the pre-processed data information, and then organize it to obtain structured information; the structured information includes entity information and relationship information; the entity information includes assembly parts, assembly tools, robots and assembly steps; the relationship information includes assembly sequence and task dependencies.

[0019] Optionally, the structured information includes entity information and relationship information; the knowledge graph layer includes a knowledge graph construction module and a knowledge graph update module;

[0020] A knowledge graph building module, which is used to store structured information as a knowledge graph through a graph database;

[0021] The nodes in the graph database represent entity information, including assembly parts, assembly tools, robots, and assembly steps; the edges in the graph data represent relationship information, including assembly sequence and task dependency;

[0022] The knowledge graph update module is used to monitor the update information about structured information output by the data processing layer through a real-time monitoring mechanism, and based on the update information, the knowledge graph is updated in an incremental manner through a hierarchical update mechanism.

[0023] Optionally, the task decision layer includes a task decomposition module, a task allocation module and a scheduling optimization module;

[0024] The task decomposition module is used to decompose the assembly task into multiple subtasks based on the knowledge graph through a topological sorting algorithm;

[0025] The task allocation module is used to obtain a task allocation plan based on the assembly capability, usage status and load condition of each robot using a preset optimization algorithm; the optimization algorithm includes a Hungarian algorithm or a genetic algorithm;

[0026] The scheduling optimization module is used to optimize and adjust the task allocation plan according to the task execution status combined with the corresponding robot load, task progress and environmental changes.

[0027] Optionally, the communication and synchronization layer includes a high-speed communication network module and a time synchronization module;

[0028] A high-speed communication network module, used to establish a high-speed communication network within the multi-robot collaborative assembly system based on a preset communication network and communication protocol;

[0029] The time synchronization module is used to achieve time synchronization of each device in a multi-robot collaborative assembly system based on a preset precise time protocol.

[0030] Optionally, the data information includes environmental information; the execution layer includes a robot control module, a collaborative obstacle avoidance module and a state feedback module;

[0031] A robot control module is used to control the corresponding robot to perform the assigned target assembly task according to the task decision instruction;

[0032] The collaborative obstacle avoidance module is used to plan the obstacle avoidance path of the robot performing the task based on the environmental information through the model predictive control algorithm;

[0033] The state feedback module is used to feed back the task execution status of the robot performing the target assembly task to the task decision layer in real time.

[0034] Optionally, the system further includes a human-computer interaction layer;

[0035] The human-machine interaction layer is used to push task data to the operator or receive the operator's control instructions for the multi-robot collaborative assembly system according to the preset interaction interface.

[0036] Optionally, the human-computer interaction layer includes a monitoring and visualization module, a manual intervention module, and an alarm and notification module;

[0037] The monitoring and visualization module is used to push the system status of the multi-robot collaborative assembly system, the task progress of the robots in executing the target assembly task, and the location information of the robots to the interactive interface;

[0038] A manual intervention module, used to adjust the task decision instruction or stop the multi-robot collaborative assembly system in response to the control instruction of the operator received by the interactive interface;

[0039] The alarm and notification module is used to push abnormal situations of the multi-robot collaborative assembly system to the interactive interface for alarm and prompts.

[0040] On the other hand, an embodiment of the present invention provides a multi-robot collaborative assembly method in which a large model and a knowledge graph are linked, which is applied to the multi-robot collaborative assembly system in which a large model and a knowledge graph are linked. The method includes the following steps:

[0041] Collect data information about assembly environment and assembly tasks in real time through the perception layer;

[0042] The data processing layer pre-processes the data information collected by the perception layer and uses the preset large model for in-depth analysis to obtain structured information;

[0043] The knowledge graph is constructed through the knowledge graph layer according to the structured information output by the data processing layer;

[0044] The task decision layer is based on the knowledge graph, combined with the application data of each robot and the task execution status fed back by the execution layer to perform task decomposition, allocation and optimization scheduling, and obtain task decision instructions;

[0045] A high-speed communication network is established within the multi-robot collaborative assembly system through the communication and synchronization layer, and the time synchronization of each device in the system is achieved through the time synchronization protocol;

[0046] Based on the task decision instructions, the execution layer controls the corresponding robot to execute the assigned target assembly task; and feeds back the robot's task execution status to the task decision layer.

[0047] Optionally, the method further comprises the following steps:

[0048] Through the human-computer interaction layer, task data is pushed to the operator according to the preset interactive interface, or control instructions for the multi-robot collaborative assembly system are received from the operator.

[0049] On the other hand, an embodiment of the present invention provides an electronic device, including: a processor and a memory; the memory is used to store programs; the processor executes the program to implement the multi-robot collaborative assembly method that links the above-mentioned large model with the knowledge graph.

[0050] On the other hand, an embodiment of the present invention provides a computer storage medium, which stores a program executable by a processor. When the program executable by the processor is executed by the processor, it is used to implement the multi-robot collaborative assembly method that links the above-mentioned large model with the knowledge graph.

[0051] The embodiment of the present invention collects data information of assembly environment and assembly tasks in real time through the perception layer; pre-processes the data information collected by the perception layer and uses a preset large model for in-depth analysis to obtain structured information through the data processing layer; constructs a knowledge graph according to the structured information output by the data processing layer through the knowledge graph layer; performs task decomposition, allocation and optimization scheduling based on the knowledge graph by the task decision layer, combined with the application data of each robot and the task execution status fed back by the execution layer, to obtain task decision instructions; establishes a high-speed communication network inside the multi-robot collaborative assembly system through the communication and synchronization layer, and realizes time synchronization of each device in the system through the time synchronization protocol; controls the corresponding robot to execute the assigned target assembly task based on the task decision instruction through the execution layer; and feeds back the task execution status of the robot to the task decision layer. Based on the structured expression ability of the knowledge graph, the present invention can better handle the relationship between complex tasks and optimize the task allocation process; and then realizes the multi-robot collaborative assembly system with autonomous intelligent task decomposition and allocation in combination with the application of the robot and the real-time task execution status, which can improve the flexibility, response speed and collaboration efficiency of the system. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] The accompanying drawings are used to provide a further understanding of the technical solution of the present invention and constitute a part of the specification. Together with the embodiments of the present invention, they are used to explain the technical solution of the present invention and do not constitute a limitation on the technical solution of the present invention.

[0053] Figure 1 A schematic diagram of the module composition of a multi-robot collaborative assembly system linked with a large model and a knowledge graph provided by an embodiment of the present invention;

[0054] Figure 2 A schematic diagram of the overall operation module architecture of an example of a multi-robot collaborative assembly system that links a large model with a knowledge graph provided by an embodiment of the present invention;

[0055] Figure 3A schematic diagram of the overall operation process of a multi-robot collaborative assembly system that is linked to a large model and a knowledge graph provided by an embodiment of the present invention;

[0056] Figure 4 A schematic diagram of an implementation environment of a multi-robot collaborative assembly method for linking a large model with a knowledge graph provided by an embodiment of the present invention;

[0057] Figure 5 A schematic diagram of a process flow of a multi-robot collaborative assembly method that combines a large model with a knowledge graph provided by an embodiment of the present invention;

[0058] Figure 6 A schematic diagram of the structure of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0059] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0060] It should be noted that, although the functional modules are divided in the system schematic diagram and the logical order is shown in the flowchart, in some cases, the steps shown or described may be performed in a different order than the module division in the system or the order in the flowchart. The terms "first / S100", "second / S200", etc. in the specification, claims and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence.

[0061] Reference to "embodiments" herein means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present invention. The appearance of the phrase in various places in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0062] On the one hand, if Figure 1 As shown, the embodiment of the present invention provides a multi-robot collaborative assembly system 100 that links a large model with a knowledge graph, which may include a perception layer 110, a data processing layer 120, a knowledge graph layer 130, a task decision layer 140, a communication and synchronization layer 150, and an execution layer 160; specifically:

[0063] The perception layer is used to collect data information about the assembly environment and assembly tasks in real time;

[0064] It should be noted that, in some embodiments, the perception layer includes a multi-sensor fusion module and a task information acquisition module; the data information includes environmental information and task data;

[0065] A multi-sensor fusion module is used to collect environmental information in real time based on preset sensors; the preset sensors include visual sensors, laser radars, and force sensors;

[0066] The task information acquisition module is used to obtain real-time updated task data through the system interface based on a standardized data format.

[0067] Exemplarily, in some specific implementations, the perception layer is responsible for collecting environment, task and robot status data in real time, and providing basic information for subsequent data processing and task decomposition.

[0068] Multi-sensor fusion module: collects environmental information, including workpiece position, distance data and force feedback, through visual sensors, lidar, force sensors, etc. The consistency of multi-source data is ensured through timestamp synchronization. The collected data is filtered, denoised and processed for subsequent analysis and control.

[0069] Task information acquisition module: obtains new assembly tasks and production plans through interfaces with ERP, MES and other systems, and transmits task information in standardized data formats (such as XML, JSON) to ensure accurate transmission and parsing of task data.

[0070] The data processing layer is used to pre-process the data information collected by the perception layer and perform in-depth analysis using a preset large model to obtain structured information;

[0071] It should be noted that in some embodiments, the data processing layer includes a pre-processing module and a large model processing module:

[0072] The preprocessing module is used to preprocess the data information output by the perception layer; the preprocessing includes data cleaning and formatting;

[0073] The large model processing module is used to use the pre-trained large-scale language model to extract key information from the pre-processed data information, and then organize it to obtain structured information; the structured information includes entity information and relationship information; the entity information includes assembly parts, assembly tools, robots and assembly steps; the relationship information includes assembly sequence and task dependencies.

[0074] Exemplarily, in some specific implementations, the data processing layer uses a large model to perform in-depth analysis on the perception data, extract task-related information, and structure the data.

[0075] Preprocessing module: Cleans and formats the data from multiple sensors. It removes noise through algorithms such as Kalman filtering and unifies the data format for subsequent processing.

[0076] Large model processing module: Use the Qwen-2-72B large model to extract key information in the environment and tasks, such as assembly steps, component names, dependencies, assembly order, etc. The large model generates structured information through pre-training knowledge and real-time environmental data, which is convenient for subsequent use in building knowledge graphs.

[0077] In some specific application scenarios:

[0078] 1. The specific training content of the large model is as follows:

[0079] The Qwen-2-72B large model is a language model based on large-scale pre-training. The main training content includes the following aspects:

[0080] General knowledge pre-training: Qwen-2-72B is pre-trained on massive text data (such as encyclopedia articles, technical documents, academic papers, open data sets, etc.), covering a wide range of knowledge. This enables the model to handle complex language tasks and extract key information.

[0081] Task relevance: Through fine-tuning, the model will be further trained for specific scenarios of assembly tasks. The fine-tuning dataset may contain assembly processes, operation steps, component names, dependencies, etc. in the manufacturing industry to ensure the applicability of the model in the manufacturing environment.

[0082] Relational reasoning: Through training, the large model has learned how to extract entities (such as parts and tools) and the relationships between them from natural language descriptions, and can perform simple reasoning operations. For example, during the training process, the model can understand the dependency relationship that "Part A must be operated after Part B is installed."

[0083] 2. Combined with real-time environmental data:

[0084] The large model is combined with real-time environmental data in the following ways:

[0085] Dynamic input: Real-time environmental data (such as dynamic information collected by visual sensors, lidar, etc.) will be filtered and formatted by the preprocessing module, and then passed to the big model as input for analysis. The big model uses pre-trained knowledge to understand and reason about the input data and extract information related to task execution.

[0086] Scenario Adaptation: The model makes corresponding adjustments based on pre-trained knowledge and real-time data. For example, the sensor data transmitted back in real time during the robot execution can update the task sequence through the model's reasoning ability and identify whether there are changes in the current assembly task (such as position deviation, step adjustment, etc.).

[0087] 3. The process of generating structured information:

[0088] The large model transforms sensory data into structured information through the following steps:

[0089] Entity and relationship extraction: When the large model processes task data, it identifies the entities in the assembly task (such as parts, tools, assembly steps, etc.) and the relationships between them (such as assembly order, dependencies, etc.).

[0090] Graph representation: The extracted entities and relationships are converted into nodes and edges to generate a knowledge graph. Nodes represent specific elements in the assembly task (such as parts, tools, robots), and edges represent the relationships between them (such as operation sequence, dependency).

[0091] Dynamic update: When real-time environmental data (such as sensor information) changes, the model will update the information in the knowledge graph in real time according to these changes to ensure the continuity and accuracy of task planning. This process ensures the consistency and timeliness of data through transaction processing.

[0092] 4. Related materials and references:

[0093] Principle of pre-training model: The architecture of Qwen-2-72B is based on Transformer. Similar language model training processes can refer to the training methods of general language models such as GPT and BERT, using large-scale unsupervised text data for pre-training, and then fine-tuning with small-scale supervised data.

[0094] Knowledge graph construction: The technology that the big model uses to convert the results of natural language processing into a knowledge graph is based on relationship extraction and entity linking. For related technologies, please refer to the knowledge graph construction process, such as RDF (Resource Description Framework), SPARQL query language, etc.

[0095] The knowledge graph layer is used to construct a knowledge graph based on the structured information output by the data processing layer;

[0096] It should be noted that in some embodiments, structured information includes entity information and relationship information; the knowledge graph layer includes a knowledge graph construction module and a knowledge graph update module;

[0097] A knowledge graph building module, which is used to store structured information as a knowledge graph through a graph database;

[0098] The nodes in the graph database represent entity information, including assembly parts, assembly tools, robots, and assembly steps; the edges in the graph data represent relationship information, including assembly sequence and task dependency;

[0099] The knowledge graph update module is used to monitor the update information about structured information output by the data processing layer through a real-time monitoring mechanism, and based on the update information, the knowledge graph is updated in an incremental manner through a hierarchical update mechanism.

[0100] Exemplarily, in some specific implementations, the knowledge graph layer is responsible for organizing structured information into a knowledge graph, which serves as an important basis for task decomposition and optimization.

[0101] Knowledge graph construction module: The task information (such as entities and relationships) extracted by the big model is stored as a knowledge graph through the graph database. The knowledge graph nodes represent parts, tools, and task steps, and the edges represent the dependencies and sequence relationships between them, ensuring the integrity of the task dependency structure.

[0102] Knowledge graph update module: By real-time monitoring of the update information output by the large model, the knowledge graph is dynamically updated to ensure the real-time and consistency of the knowledge structure after task changes, and use transaction processing to avoid data conflicts.

[0103] In some specific application scenarios, the following technical means can be used to ensure the rapidity and accuracy of knowledge graph updates:

[0104] (1) Real-time monitoring mechanism:

[0105] Real-time data input: The knowledge graph update module monitors dynamic information from sensors, robot feedback, and large model reasoning results in real time. When the task or environment changes (such as robot position adjustment, assembly sequence modification), these changes will be quickly transmitted to the knowledge graph update module through the data pipeline.

[0106] Event-driven updates: Knowledge graph updates are based on an event-driven mechanism. When the system detects changes in task information, environmental data, or other changes related to task execution, an update operation is triggered. The large model listens to the real-time input data stream and analyzes it instantly, extracting key information (such as new task dependencies, order changes, etc.) from it and updating the nodes and edges in the knowledge graph.

[0107] (2) Hierarchical update mechanism:

[0108] Incremental update: In order to increase the update speed, the knowledge graph is updated incrementally, that is, only the changed parts (such as task dependencies or robot status) are updated locally to avoid the high computational overhead caused by the full graph update. Incremental updates identify the changed parts by comparing the new task data with the existing graph.

[0109] Parallel processing: The update process may involve multiple robots and tasks. In order to improve efficiency, the system uses parallel processing technology. The update operation of each task or robot instance can be carried out simultaneously, and different update processes do not interfere with each other.

[0110] (3) Graph database optimization:

[0111] Indexing mechanism: In order to speed up the query and write operations of nodes and edges, graph databases (such as Neo4j) use an indexing mechanism to ensure that the nodes and edges that need to be modified can be quickly located when the knowledge graph is updated. This indexing technology ensures a rapid response to updates even in large graph structures.

[0112] Caching strategy: In order to improve the response speed, the knowledge graph update module may adopt a caching strategy to cache commonly used node and relationship data in memory. When the task allocation module requests data, it will be read from the cache first, reducing frequent database access and improving real-time performance.

[0113] 2. Specific mechanisms to avoid data conflicts:

[0114] (1) Transaction processing mechanism:

[0115] ACID properties of transactions: Graph databases usually support ACID (atomicity, consistency, isolation, and durability) transactions, which means that each graph update is a complete operation and cannot be divided. If a conflict or error occurs during the update process, the system will roll back the operation to ensure that the graph does not enter an inconsistent state.

[0116] Optimistic locking mechanism: In concurrent situations, graph databases use optimistic locking to handle conflicts. Each node and edge has a version number. When multiple processes try to update the same node, only the first operation will succeed. Subsequent operations will fail and be retried to avoid data overwriting or conflicts.

[0117] (2) Version control and rollback mechanism:

[0118] Version control: When the knowledge graph is updated, it will record the version of each change. The system can roll back to the previous version when conflicts or errors occur to ensure data integrity and consistency. This mechanism is particularly suitable for complex task update scenarios.

[0119] Data snapshots: The system can periodically generate snapshots of the graph to record the complete state of the graph. In the event of a major error or conflict, the system can use the snapshot to restore to the previous state.

[0120] (3) Concurrency Control:

[0121] Lock mechanism: The system can control the simultaneous access of multiple robots or task instances to the knowledge graph through a fine-grained lock mechanism. For example, when a robot is updating the dependencies of a task, other processes will temporarily lock the relevant nodes to avoid data conflicts caused by simultaneous writing.

[0122] Isolation level: By setting different transaction isolation levels (such as read committed, repeatable read, etc.), the system can ensure that during concurrent transaction operations, the reading and updating of task dependencies will not interfere with each other, avoiding problems such as "dirty reads" or "phantom reads".

[0123] (4)Consistency check mechanism:

[0124] Regular consistency check: The system will regularly check the consistency of the knowledge graph to ensure the correctness of task dependencies and order relationships. This process can verify the order and dependencies of tasks through topological sorting or other algorithms, and find and fix inconsistencies.

[0125] Automatic repair: When the system detects data conflicts or inconsistencies, it can automatically repair the knowledge graph according to predefined rules. For example, when a task dependency loop occurs, the system can automatically adjust the node order or re-plan the task.

[0126] 3. Specific technologies for optimizing the update process:

[0127] Batch processing: In some cases, multiple update operations can be batched to complete multiple changes in one transaction, reducing the overhead and conflict probability caused by frequent small updates.

[0128] Delayed update: For some non-urgent update operations, a delayed update strategy can be adopted to concentrate update operations during time periods with low system load to avoid competition and conflicts during peak hours.

[0129] The task decision layer is used to decompose, allocate and optimize tasks based on the knowledge graph, combined with the application data of each robot and the task execution status fed back by the execution layer, to obtain task decision instructions;

[0130] It should be noted that in some embodiments, the task decision layer includes a task decomposition module, a task allocation module and a scheduling optimization module;

[0131] The task decomposition module is used to decompose the assembly task into multiple subtasks based on the knowledge graph through a topological sorting algorithm;

[0132] The task allocation module is used to obtain a task allocation plan based on the assembly capability, usage status and load condition of each robot using a preset optimization algorithm; the optimization algorithm includes a Hungarian algorithm or a genetic algorithm;

[0133] The scheduling optimization module is used to optimize and adjust the task allocation plan according to the task execution status combined with the corresponding robot load, task progress and environmental changes.

[0134] Exemplarily, in some specific implementations, the task decision layer performs task decomposition, allocation, and optimal scheduling based on the knowledge graph to ensure the efficiency of multi-robot collaboration.

[0135] Task decomposition module: By analyzing the topological sorting of the knowledge graph, it identifies the dependencies of tasks and decomposes complex assembly tasks into executable subtasks. This module also identifies parallel tasks to optimize the order of task execution.

[0136] Task allocation module: Based on the capabilities, current status and load of each robot, the task allocation module uses an optimization algorithm (such as the Hungarian algorithm) to calculate the optimal task allocation plan and generate standardized task instructions to send to the robot.

[0137] Scheduling optimization module: monitors the task execution status in real time, and dynamically adjusts the task sequence and allocation based on robot load, task progress, and environmental changes (this can be achieved through deep learning algorithms or machine learning algorithms) to ensure that the system always maintains efficient operation in a changing production environment.

[0138] The communication and synchronization layer is used to establish a high-speed communication network within the multi-robot collaborative assembly system and realize the time synchronization of each device in the system through the time synchronization protocol;

[0139] It should be noted that in some embodiments, the communication and synchronization layer includes a high-speed communication network module and a time synchronization module;

[0140] A high-speed communication network module, used to establish a high-speed communication network within the multi-robot collaborative assembly system based on a preset communication network and communication protocol;

[0141] The time synchronization module is used to achieve time synchronization of each device in a multi-robot collaborative assembly system based on a preset precise time protocol.

[0142] Exemplarily, in some specific implementations, the communication and synchronization layer provides high-speed data transmission and time synchronization within the system to ensure consistency during multi-robot collaboration.

[0143] High-speed communication network module: through industrial Ethernet or 5G network, ensure real-time data transmission between modules. Use OPC UA or MQTT protocol to ensure low latency and high reliability of communication.

[0144] Time synchronization module: Use IEEE 1588 Precision Time Protocol (PTP) to achieve time synchronization of all robots and devices in the system, ensuring consistency of actions when multiple robots collaborate.

[0145] In some specific application scenarios, the following are detailed descriptions and examples of the specific application scenarios, configurations, and usage details of OPC UA and MQTT protocols in multi-robot collaborative systems:

[0146] 1. Application and configuration of OPC UA protocol:

[0147] (1) OPC UA Overview:

[0148] OPC UA (Unified Architecture)** is a platform-independent industrial protocol designed for secure and efficient communication between devices and systems. It not only supports traditional factory automation equipment, but also handles complex data models, and is particularly suitable for status monitoring, command issuance, and data transmission in multi-robot collaborative systems.

[0149] Application scenario: In a multi-robot collaborative system, OPC UA is mainly used for status synchronization, task delivery, and real-time feedback between robots and the central control system. Each robot can act as an OPC UA client to send status information to the central control server and receive task instructions.

[0150] (2) OPC UA configuration details:

[0151] Server / Client Architecture: OPC UA follows a server / client architecture. In this system, the central control system can be configured as an OPC UA server, providing task scheduling and status query interfaces, and the robot acts as an OPC UA client, connecting to the server and performing related tasks.

[0152] Server-side configuration: To configure the OPC UA server, you need to set specific endpoints and define the resources that the robot can access, such as real-time status, task lists, sensor data, etc.

[0153] Client configuration: The robot needs to configure the OPC UA client to connect to the server endpoint to obtain task instructions and report status. Typical client configuration includes the address, port, authentication method, etc. of the OPC UA server.

[0154] Node management: OPC UA represents data in a structured manner through nodes. Each robot task and status information corresponds to a node. During configuration, a unique node ID needs to be assigned to each robot. Task instructions and status information interact by accessing these nodes. The hierarchical structure of nodes facilitates data organization and query.

[0155] Example: In the task delivery scenario, the central control system can send task instructions to robot 1 through the / robots / robot1 / task node, and the robot can feedback the task execution progress through the / robots / robot1 / status node.

[0156] (3) Details of OPC UA usage:

[0157] Task issuance and status monitoring: In actual operation, the OPC UA server regularly receives real-time data (such as current tasks, operating status, sensor data) from each robot and issues new task instructions as needed.

[0158] Example: After completing the current task, the robot feedbacks the completion status through the status node. The OPC UA server sends the next task to the robot through the corresponding node according to the task scheduling algorithm.

[0159] Security configuration: OPC UA provides a secure communication mechanism, including encryption, authentication, and authorization. To ensure the security of data transmission between robots, the system can configure a certificate authentication mechanism. The client (robot) must pass the certificate verification before connecting to the server to ensure the reliability of communication.

[0160] Real-time and reliability: OPC UA supports multiple transmission methods (such as TCP) and can operate in a low-latency, high-reliability network environment. The system can adjust the refresh rate (update frequency) according to the specific scenario requirements to ensure that the robot receives instructions and status feedback in real time.

[0161] 2.Application and configuration of MQTT protocol:

[0162] (1) MQTT Overview:

[0163] MQTT (Message Queuing Telemetry Transport) is a lightweight publish / subscribe messaging protocol designed for low-bandwidth, high-latency environments with low overhead and high flexibility. It is ideal for communication between robots and information publishing between control systems, especially for scenarios that require low-latency data exchange.

[0164] Application scenario: In a multi-robot collaborative system, MQTT can be used to transmit task status, perception data, and emergency instructions. Each robot can act as a publisher to publish its own status information, or as a subscriber to subscribe to task instructions from the central control system or status information published by other robots.

[0165] (2) MQTT configuration details:

[0166] Publish / Subscribe Mode: MQTT is based on the publish / subscribe communication mode. Robots can act as publishers and subscribers, publishing their own status and subscribing to task instructions respectively. When configuring MQTT, you need to specify a topic. Each robot can have its own topic for communication.

[0167] Example: Robot 1 can subscribe to the task instruction topic / tasks / robot1, publish its own status information to the topic / status / robot1, and the control system subscribes to all robot status information and publishes task instructions.

[0168] QoS (Quality of Service) settings: MQTT provides three Quality of Service (QoS) levels to balance the real-time and reliability of transmission:

[0169] QoS 0 (at most once): The data packet is transmitted only once, and there is no guarantee that the data will reach the receiver. It is suitable for scenarios such as sensor data where arrival does not need to be strictly guaranteed.

[0170] QoS1 (At Least Once): ensures that data arrives at least once and the receiver confirms receipt of the message. It is suitable for task instructions or important status data.

[0171] QoS2 (only once): ensures that the message is delivered only once, which is suitable for extremely critical control instruction scenarios.

[0172] Broker configuration: MQTT requires a Broker to be configured as a message transfer station, which is responsible for receiving messages from publishers and distributing them to subscribers. The central control server in the system usually plays the role of Broker, responsible for managing the message exchange between all robots.

[0173] Example: The control system can act as an MQTT Broker, receiving the status publications of each robot and publishing task instructions to the corresponding robot topics.

[0174] (3) MQTT usage details:

[0175] Task allocation and status feedback: The robot subscribes to the task instruction topic through MQTT. When a new task instruction is released, the robot can immediately obtain and execute it. At the same time, the robot can also feedback the task execution status to the system by publishing status messages (such as / status / robot1).

[0176] Example: When the control system publishes a task on the / tasks / robot1 topic, Robot 1 receives the task and executes it. After the execution is completed, the status is published to the / status / robot1 topic.

[0177] Efficient broadcast mechanism: Through the MQTT topic mechanism, the central control system can broadcast task instructions or status information to multiple robots at the same time. Through wildcards (such as / tasks / #), tasks can be issued to multiple robots at the same time, greatly simplifying the complexity of issuing instructions.

[0178] Disconnection reconnection and keep-alive mechanism: MQTT has a disconnection reconnection mechanism. If the robot loses network connection in some cases, the system will automatically try to reconnect and continue to publish and subscribe messages. In addition, MQTT supports a keep-alive mechanism. The system will ensure the stability of the connection by regularly sending keep-alive packets.

[0179] Lightweight advantage: Due to the low protocol overhead of MQTT, it is very suitable for the transmission of robot status information in low-bandwidth environments, ensuring effective communication even under poor network conditions.

[0180] The execution layer is used to control the corresponding robot to perform the assigned target assembly task based on the task decision instruction; the robot's task execution status is fed back to the task decision layer

[0181] It should be noted that in some embodiments, the data information includes environmental information; the execution layer includes a robot control module, a collaborative obstacle avoidance module, and a state feedback module;

[0182] A robot control module is used to control the corresponding robot to perform the assigned target assembly task according to the task decision instruction;

[0183] The collaborative obstacle avoidance module is used to plan the obstacle avoidance path of the robot performing the task based on the environmental information through the model predictive control algorithm;

[0184] The state feedback module is used to feed back the task execution status of the robot performing the target assembly task to the task decision layer in real time.

[0185] For example, in some specific implementations, the execution layer is composed of multiple robots, which are responsible for performing assembly tasks according to the instructions of the task decision layer and ensuring the safety of the operation through collaboration and obstacle avoidance technology.

[0186] Robot control module: According to the instructions of the task allocation module, it controls the robot arm and other actuators to perform specific assembly operations, including path planning, motion control and grasping operations, to ensure the accurate completion of the task.

[0187] Collaborative obstacle avoidance module: Utilizes multi-sensor data to monitor the surrounding environment in real time, and uses a predictive obstacle avoidance algorithm to plan robot movements to ensure that robots do not collide or have task conflicts during collaborative work.

[0188] Status feedback module: During the execution process, the robot will feed back the real-time status to the task decision layer and the human-computer interaction layer. The information includes task progress, operation status, location information, etc., so that the system can adjust the task or handle faults in time.

[0189] In some optional embodiments, the system may further include a human-computer interaction layer;

[0190] The human-machine interaction layer is used to push task data to the operator or receive the operator's control instructions for the multi-robot collaborative assembly system according to the preset interaction interface.

[0191] It should be noted that in some embodiments, the human-computer interaction layer includes a monitoring and visualization module, a manual intervention module, and an alarm and notification module;

[0192] The monitoring and visualization module is used to push the system status of the multi-robot collaborative assembly system, the task progress of the robots in executing the target assembly task, and the location information of the robots to the interactive interface;

[0193] A manual intervention module, used to adjust the task decision instruction or stop the multi-robot collaborative assembly system in response to the control instruction of the operator received by the interactive interface;

[0194] The alarm and notification module is used to push abnormal situations of the multi-robot collaborative assembly system to the interactive interface for alarm and prompts.

[0195] For example, in some specific implementations, the human-computer interaction layer provides operators with a monitoring interface for system status and supports manual intervention and management of tasks.

[0196] Monitoring and visualization module: Displays system operation status, task progress and robot position through a visual interface. Operators can understand the overall situation of the current system through charts and 3D models.

[0197] Manual intervention module: allows operators to manually adjust task instructions or stop certain operations under special circumstances, ensuring that the system can be handled in a timely manner in abnormal situations.

[0198] Alarm and notification module: When the system detects an abnormal situation or task failure, the alarm module will notify relevant personnel to handle the fault through various methods such as pop-up windows, sounds and text messages.

[0199] In order to explain the principle of the technical solution of the present invention in detail, the overall process of the present invention is described below in combination with some specific embodiments. It is easy to understand that the following is an explanation of the technical principle of the present invention and cannot be regarded as a limitation of the present invention.

[0200] First of all, it should be noted that the existing production line scheduling technology has the following main shortcomings in multi-robot collaboration and dynamic scheduling:

[0201] Poor flexibility in task decomposition and allocation: Most existing multi-robot collaborative assembly systems rely on predefined rules or simple linear reasoning to decompose and allocate tasks, which makes it difficult to cope with frequently changing task requirements and complex dependencies. When production tasks or environments change, the system often requires manual intervention to reallocate tasks, resulting in low efficiency and inability to achieve dynamic adjustment of automation.

[0202] Lack of intelligent dynamic adjustment capabilities: Current assembly systems mostly rely on fixed task scheduling models and lack intelligent adaptive mechanisms. The system cannot automatically adjust according to real-time environmental data or task changes, and it is difficult to meet the needs of highly flexible and adaptive production in modern manufacturing. When faced with multi-task and multi-level task structures, traditional systems show inefficiency in task allocation and load balancing.

[0203] Low collaboration efficiency, communication and synchronization lag: The existing multi-robot system relies on a single control center for task scheduling and communication, which easily leads to communication bottlenecks and low collaboration efficiency between robots. In complex assembly scenarios that require multiple robots to collaborate simultaneously, the existing system cannot guarantee efficient task collaboration, and there are problems such as untimely synchronization and task conflicts between robots, which affect overall production efficiency.

[0204] Insufficient safety in obstacle avoidance and collaboration: Traditional systems rely mainly on simple sensor feedback for robot obstacle avoidance, which makes it difficult to achieve accurate multi-robot obstacle avoidance in dynamic and complex environments. This can easily lead to collisions or task interruptions during collaboration, and is not very safe. Especially in dense industrial assembly scenarios, the lag in obstacle avoidance algorithms can pose serious production safety risks.

[0205] Weak correlation between knowledge expression and tasks: Existing systems have difficulty effectively expressing the correlation and dependency between tasks when dealing with complex task structures. Simple task lists or tree structures are not sufficient to cope with complex task decomposition requirements and cannot clearly show the multi-level dependencies and order of tasks, which is particularly evident in large-scale production environments.

[0206] The present invention provides a multi-robot collaborative assembly system based on the linkage of large models and knowledge graphs, such as Figure 2 As shown in the figure, the overall architecture mainly includes the following layers: perception layer, data processing layer, knowledge graph layer, task decision layer, communication and synchronization layer, execution layer and human-computer interaction layer. Each layer exchanges data and collaborates functions through standardized interfaces and protocols. The specific architecture is as follows:

[0207] Perception layer:

[0208] Environmental perception module: contains a variety of sensors, such as visual sensors (cameras), lidar, force sensors, etc., which are used to collect dynamic information of the assembly environment in real time, including workpiece position, equipment status, environmental changes, etc.

[0209] Task information acquisition module: obtains new assembly task requirements, design changes, production plans and other information, which can come from the enterprise resource planning (ERP) system, manufacturing execution system (MES) or manual input.

[0210] Data processing layer:

[0211] Preprocessing module: performs format conversion, noise filtering and data cleaning on the raw data obtained by the perception layer to ensure data quality.

[0212] Large model processing module: deploy large-scale pre-trained models (such as Qwen-2-72B), conduct in-depth analysis of pre-processed data, and extract key information, including assembly parts, operation steps, task dependencies, etc.

[0213] Knowledge graph layer:

[0214] Knowledge graph construction module: Structuring the entities and relationships extracted from the large model to build a knowledge graph for assembly tasks. Nodes represent entities such as assembly parts, tools, robots, etc., and edges represent relationships between entities, such as assembly sequence, dependencies, etc. Knowledge graph update module: Dynamically update the knowledge graph based on new information obtained in real time to ensure its accuracy and timeliness.

[0215] Knowledge storage and query module: It uses graph database technology to support efficient knowledge storage and query operations, and provide data support for the task decision-making layer.

[0216] Task decision-making layer:

[0217] Task decomposition module: Based on the latest knowledge graph, it uses algorithms such as topological sorting and depth-first search to decompose complex assembly tasks into multiple executable subtasks and clarify the sequence and dependencies of tasks.

[0218] Task allocation module: According to the capabilities, current status and load conditions of each robot, optimization algorithms (such as Hungarian algorithm and genetic algorithm) are used to reasonably allocate subtasks to suitable robots to achieve load balancing and maximize efficiency.

[0219] Scheduling optimization module: Considers factors such as production priority and resource constraints to optimize the order of task execution and reduce waiting time and resource conflicts.

[0220] Communication and synchronization layer:

[0221] High-speed communication network: Establish a communication network based on industrial Ethernet or 5G to achieve high-speed data transmission between robots, sensors, and control systems.

[0222] Time synchronization module: uses the Precision Time Protocol (PTP) to ensure time synchronization of each device in the system and the accuracy of collaborative operations.

[0223] Protocol and interface standardization: Use standard communication protocols (such as OPC UA) to achieve seamless connection and data exchange between modules.

[0224] Execution layer:

[0225] Robot control module: receives instructions from the task decision layer and controls the robot to perform specific assembly operations. It includes sub-modules such as motion control, path planning, and operation execution.

[0226] Collaborative obstacle avoidance module: Through multi-sensor fusion, it can perceive the surrounding environment and the position of other robots in real time, and use predictive obstacle avoidance algorithms (such as model predictive control MPC) to adjust the path and avoid collisions.

[0227] Feedback monitoring module: monitors the robot's execution status and task progress in real time, and provides feedback to the task decision layer and human-computer interaction layer.

[0228] Human-computer interaction layer:

[0229] Monitoring and visualization module: provides real-time monitoring and visualization interface of system operation status, including task progress, robot status, environmental information, etc.

[0230] Manual intervention and command input module: allows operators to manually intervene when necessary, input new task instructions or adjust system parameters.

[0231] Alarm and notification module: When the system detects an abnormal situation or failure, it will promptly issue an alarm message to prompt the operator to take corresponding measures.

[0232] like Figure 3 As shown, the system operation process can be implemented as follows:

[0233] Task acquisition and environmental perception: The system acquires new assembly tasks and environmental information through the perception layer.

[0234] Data processing and knowledge extraction: The data processing layer pre-processes the information, and the large model extracts key information.

[0235] Knowledge graph construction and update: The knowledge graph layer structures the extracted information and constructs or updates the knowledge graph.

[0236] Task decomposition and allocation: The task decision layer decomposes tasks and optimizes their allocation to each robot based on the latest knowledge graph.

[0237] Task execution and coordination: Robots in the execution layer perform tasks according to instructions, and achieve collaborative work and obstacle avoidance through the communication and synchronization layer.

[0238] Status feedback and adjustment: The execution layer feeds back the task execution status to the task decision layer and the human-computer interaction layer, and performs task adjustments or manual intervention when necessary.

[0239] Iteration: The system continuously monitors changes in the environment and tasks, dynamically adjusts the knowledge graph and task plan, and ensures efficient completion of assembly tasks.

[0240] In some specific implementation scenarios, in order to further illustrate the principle and effect of the technical solution of the present invention, the embodiment of the present invention also provides a multi-robot collaborative assembly system verification experiment based on the linkage of a large model and a knowledge graph, which is specifically implemented as follows:

[0241] 1. Experimental environment:

[0242] In terms of hardware platform, the experiment uses 6 humanoid collaborative robots, each equipped with a robotic arm, lidar, high-definition camera, depth sensor and force sensor. These robots have high-precision motion control capabilities, support multi-degree-of-freedom operations, and can perform precision operations in assembly tasks. The server used for large model reasoning and data processing is equipped with an Intel Xeon 24-core processor, 512GB DDR4 memory, 2TB SSD storage and NVIDIA A100 80GB GPU to support reasoning acceleration of large-scale models. The network equipment uses Industrial Ethernet (EtherCAT) for low-latency, high-bandwidth data transmission between robots and servers, combined with the IEEE 1588PTP synchronous clock system to achieve time synchronization of all devices.

[0243] In terms of software systems, the experiment uses the Qwen-2-72B large model as the core knowledge extraction engine. The model is deployed on the GPU of the server, and real-time reasoning and information extraction are performed through assembly task information and environmental data. The Neo4j graph database is used to store assembly task entities, dependencies, and assembly sequences extracted from the large model to form a complete knowledge graph. The task scheduling system is based on the Hungarian algorithm, which optimizes the task allocation and load balancing of multiple robots. The control system is responsible for the robot receiving task instructions and performing assembly operations through ROS (Robot Operating System).

[0244] In terms of data sources, the assembly task data in the experiment is based on the assembly scenario of real mechanical parts, including task decomposition and dependencies of multiple steps. The robot collects dynamic change information in the working environment in real time through visual sensors, lidar and force sensors, and performs positioning, obstacle avoidance and operation feedback. The experimental scenario simulates the complex assembly line environment in actual production, and each workstation is collaboratively completed by robots for grasping, assembly and installation operations. In order to further test the adaptability and flexibility of the system in complex environments, dynamically changing workpiece positions and work area layouts are introduced into the experimental scenario to evaluate the system's performance in real-time task scheduling.

[0245] 2. Experimental steps:

[0246] In order to verify the performance of the multi-robot collaborative assembly system of the present invention, the experiment is divided into the following specific steps. The experimental scene is set as a typical mechanical parts assembly line, covering multiple links such as task decomposition, task allocation, robot execution and collaborative operation. The experiment tests the task adaptability and scheduling optimization effect of the system in a dynamic environment through multiple rounds of task execution.

[0247] Step 1: Initialize the experimental environment

[0248] Robot and equipment startup:

[0249] First, start the six collaborative robots used in the experiment, ensure that all hardware devices such as robotic arms, visual sensors, lidars, force sensors, etc. are in working condition, and establish a stable communication connection with the control server.

[0250] System network configuration:

[0251] An industrial Ethernet (EtherCAT) network was configured to ensure low-latency communication between the robot and the server. Time synchronization was completed through the IEEE 1588PTP protocol to ensure that the robots in the experiment could accurately coordinate their working time.

[0252] Task data import:

[0253] Import assembly task description files from the experimental task library. Task files contain component names, assembly steps, dependencies, etc. Ensure that task information is accurate and obtain the latest task updates through interfaces with ERP and MES systems.

[0254] Step 2: Large model reasoning and knowledge graph construction

[0255] Large model information extraction:

[0256] The imported task data is input into the Qwen-2-72B large model, and the model is started for reasoning. The large model extracts key information from the assembly document based on the task description, including component names, assembly order, dependencies, etc.

[0257] Knowledge graph construction:

[0258] The assembly information extracted by the model is automatically input into the Neo4j graph database to form a knowledge graph. The nodes in the knowledge graph represent assembly parts, tools, and robots, and the edges represent the dependencies and sequences between assembly steps. This ensures that the knowledge graph reflects the completeness and structured dependencies of the task.

[0259] Atlas update and inspection:

[0260] Check whether the data in the knowledge graph is complete and make necessary adjustments. According to task changes and robot feedback information, the knowledge graph is dynamically updated during operation to ensure the accuracy of task planning.

[0261] Step 3: Task decomposition and allocation

[0262] Task breakdown:

[0263] The task scheduling system uses a topological sorting algorithm to decompose complex assembly tasks into multiple parallel or dependent subtasks based on the dependency relationship of the knowledge graph. The task decomposition includes multiple steps such as the grasping, installation, and precise alignment of mechanical parts.

[0264] Task Assignment:

[0265] The system optimizes the distribution of decomposed tasks. The optimal task load for each robot is calculated through the Hungarian algorithm to ensure that each robot is balanced in the assigned tasks. Task allocation takes into account the current state, skills and position of the robot and generates standardized task instructions.

[0266] Step 4: Task Execution and Collaboration

[0267] Task Execution:

[0268] After receiving the assigned task instructions, each robot starts the ROS control system and performs specific assembly operations according to the path planning and action sequence. The robot adjusts the operating parameters in real time through force sensors and visual feedback to ensure the task is completed accurately.

[0269] Collaborative obstacle avoidance:

[0270] During collaborative work, multiple robots share the workspace. LiDAR and vision sensors monitor other robots and obstacles in the environment in real time. The system adjusts the robot's motion trajectory through the model predictive control (MPC) algorithm to avoid collisions and ensure the safety of collaboration.

[0271] Status feedback and adjustment:

[0272] During the execution process, the robot will provide real-time feedback on the task status, location and load information to the server. The task scheduling system will dynamically adjust the allocation of unfinished tasks based on the feedback information to ensure the continuity and effectiveness of the tasks.

[0273] Step 5: Dynamic adjustment of experimental scene

[0274] Dynamic scene change simulation:

[0275] Dynamic scene changes were introduced in the experiment, including changes in the position of the workpiece, adjustments to the layout of the work area, etc. The system perceives environmental changes in real time based on sensor feedback, updates the knowledge graph through a large model, and automatically adjusts task planning and allocation.

[0276] Task reassignment:

[0277] In dynamic scenarios, some robots may not be able to continue to perform their original tasks due to environmental changes. The task scheduling system reallocates unfinished tasks through real-time calculations and adjusts the robot's task execution order to ensure smooth completion of assembly tasks.

[0278] Step 6: Experimental data recording and analysis

[0279] Data Records:

[0280] During the experiment, key performance indicators such as the execution time, task success rate, obstacle avoidance, and number of task allocation adjustments of each robot were recorded. Sensor data, task scheduling logs, and system feedback information were stored in real time for subsequent analysis.

[0281] Experimental results analysis:

[0282] By comparing the execution efficiency, task allocation balance, robot collaboration safety and other indicators of different rounds of experiments, the system's adaptability in a dynamic environment is analyzed. The results are used to evaluate the actual performance of the large model and knowledge graph linkage system in complex assembly tasks.

[0283] In summary, the present invention proposes a multi-robot collaborative assembly system based on the linkage of a large model and a knowledge graph, which successfully solves the problems of poor flexibility in task decomposition and allocation, insufficient dynamic adaptability, low communication synchronization efficiency, weak security and coordination in existing multi-robot collaborative systems. Through the innovative technical design of the present invention, the system has demonstrated significant superiority in complex and changeable assembly scenarios. The present invention introduces the technology of combining a large model with a knowledge graph to construct a system with real-time dynamic task decomposition, intelligent task allocation, and multi-robot collaborative work. The specific technical problems solved include:

[0284] 1. Realize dynamic and intelligent task decomposition and allocation according to task and environment changes, avoid human intervention and improve the system's adaptability.

[0285] 2. Use the big model to extract key data in assembly tasks in real time, and perform automatic task decomposition and optimal allocation based on the knowledge graph.

[0286] 3. Improve the efficiency of multi-robot collaboration through high-speed communication and synchronization mechanisms, and avoid communication delays and task conflicts.

[0287] 4. Enhance the safety of collaboration between robots to avoid collisions and task interruptions in dynamic and complex environments.

[0288] 5. Use knowledge graphs to structuredly represent complex tasks and their dependencies to improve the accuracy and efficiency of task decomposition and execution.

[0289] Compared with the prior art, the present invention has at least the following beneficial effects:

[0290] Intelligent task decomposition and optimized allocation: The present invention uses a large model (such as Qwen-2-72B) to automatically extract information from assembly tasks, constructs a structured knowledge graph, and realizes intelligent task decomposition through a topological sorting algorithm. Compared with existing systems that rely on manual or rule-based programming for task planning, the present invention can dynamically adjust task decomposition when task requirements or environmental changes occur, reducing manual intervention and significantly improving the system's task adaptability and automation. Task allocation is optimized through the Hungarian algorithm to ensure robot load balancing and maximize resource utilization efficiency.

[0291] Real-time dynamic adaptation and automatic update capability: By combining the powerful information extraction capability of the large model and the dynamic update mechanism of the knowledge graph, the present invention can automatically update the task dependencies and execution plans according to real-time sensor feedback. When the assembly task or environment changes (such as the position of the workpiece changes or the layout of the work area is adjusted), the system can adjust the task planning and allocation scheme in real time to ensure the consistency and efficiency of task execution, and enhance the flexibility of the system and its adaptability to complex environments.

[0292] Efficient communication and synchronization mechanism: In the prior art, multi-robot collaborative systems have bottlenecks in communication delay and task synchronization, especially when facing multiple robots working in parallel, communication delay and task conflict problems are prone to occur. The present invention ensures high-speed data transmission and precise time synchronization within the system by adopting industrial Ethernet (EtherCAT) and IEEE 1588PTP protocol. Each robot can perform task coordination and obstacle avoidance with microsecond time accuracy, avoiding task conflicts and coordination difficulties caused by untimely communication in traditional systems, and greatly improving collaboration efficiency.

[0293] Safe and reliable collaborative obstacle avoidance capability: When multiple robots work together, there is a risk of collision, especially in dynamic and complex assembly environments, where existing systems have difficulty avoiding obstacles accurately. The present invention uses multi-sensor data fusion (such as lidar, visual sensors, and force sensors) to perceive the surrounding environment and other robot movements in real time, and combines the model predictive control (MPC) algorithm to plan obstacle avoidance paths. This solution not only ensures the safe collaboration of robots in high-density working environments, but also effectively reduces misoperations and task interruptions, thereby improving the safety and production efficiency of the system.

[0294] Modular design and strong scalability: The present invention adopts a modular system architecture design, and each layer interacts with data through standardized interfaces, which facilitates system expansion and maintenance. In particular, in the combination of the knowledge graph and the task scheduling layer, the system has good scalability and can be flexibly adjusted and optimized according to different task requirements or production environments to adapt to assembly scenarios of different scales. Compared with traditional closed systems, the present invention greatly improves the scalability and application scope of the system.

[0295] Reduce labor costs and improve production efficiency: Since the present invention has the ability of automated task decomposition, real-time optimization scheduling and dynamic adjustment, it reduces the need for human intervention. The system can autonomously handle complex dependencies and changing scenarios in tasks, thereby reducing dependence on professional technicians, improving production efficiency and reducing labor costs in the production process.

[0296] Significantly improve assembly quality and accuracy: Through the combination of large models and knowledge graphs, the present invention can provide higher precision control and task execution accuracy in assembly tasks. Multiple sensors such as force sensors provide real-time feedback of operation results, combined with intelligent control algorithms, to ensure accurate operation and component installation during the assembly process. This makes assembly tasks not only completed quickly, but also with stable quality, thereby improving the overall product quality.

[0297] On the other hand, the embodiment of the present invention also provides a multi-robot collaborative assembly method in which a large model is linked with a knowledge graph. It can be understood that the multi-robot collaborative assembly method in which a large model is linked with a knowledge graph provided in the embodiment of the present invention can be applied to any computer device with data processing and computing capabilities, and this computer device can be various types of terminals or servers. When the computer device in the embodiment is a server, the server is an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN (Content Delivery Network), and big data and artificial intelligence platforms. Optionally, the terminal is a smart phone, a tablet computer, a laptop computer, a desktop computer, etc., but is not limited to this.

[0298] like Figure 4 FIG. 1 is a schematic diagram of an implementation environment provided by an embodiment of the present invention. Figure 4 The implementation environment includes at least one terminal 102 and a server 101. The terminal 102 and the server 101 can be connected to a network wirelessly or wired to complete data transmission and exchange.

[0299] Server 101 can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers. It can also be a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN (Content Delivery Network), as well as big data and artificial intelligence platforms.

[0300] In addition, the server 101 can also be a node server in the blockchain network. Blockchain is a new application model of computer technologies such as distributed data storage, peer-to-peer transmission, consensus mechanism, and encryption algorithm.

[0301] The terminal 102 may be a smart phone, a tablet computer, a laptop computer, a desktop computer, a smart speaker, a smart watch, etc., but is not limited thereto. The terminal 102 and the server 101 may be directly or indirectly connected via wired or wireless communication, which is not limited in the embodiment of the present invention.

[0302] Based on the example Figure 4In the implementation environment shown, an embodiment of the present invention provides a multi-robot collaborative assembly method that links a large model with a knowledge graph. The following is explained using an example of the multi-robot collaborative assembly method that links a large model with a knowledge graph and is applied to the terminal 102. It can be understood that the multi-robot collaborative assembly method that links a large model with a knowledge graph can also be applied to the server 101.

[0303] Reference Figure 5 , Figure 5 A flowchart of a multi-robot collaborative assembly method for a terminal with a large model and a knowledge graph provided in an embodiment of the present invention. The execution subject of the multi-robot collaborative assembly method for a terminal with a large model and a knowledge graph can be any of the aforementioned computer devices (including servers or terminals). Figure 5 , the method may include the following steps:

[0304] S100, collecting data information of assembly environment and assembly tasks in real time through the perception layer;

[0305] S200, preprocessing the data information collected by the perception layer through the data processing layer and performing in-depth analysis using a preset large model to obtain structured information;

[0306] S300, constructing a knowledge graph through the knowledge graph layer according to the structured information output by the data processing layer;

[0307] S400, the task decision layer is based on the knowledge graph, combined with the application data of each robot and the task execution status fed back by the execution layer to perform task decomposition, allocation and optimization scheduling, and obtain task decision instructions;

[0308] S500, establishing a high-speed communication network within the multi-robot collaborative assembly system through the communication and synchronization layer, and realizing time synchronization of various devices in the system through a time synchronization protocol;

[0309] S600, controlling the corresponding robot to execute the assigned target assembly task based on the task decision instruction through the execution layer; and feeding back the task execution status of the robot to the task decision layer.

[0310] In some embodiments, the method may further include the following steps:

[0311] Through the human-computer interaction layer, task data is pushed to the operator according to the preset interactive interface, or control instructions for the multi-robot collaborative assembly system are received from the operator.

[0312] It should be noted that the contents of the system embodiment of the present invention are applicable to the method embodiment, the process steps specifically implemented by the method embodiment are the same as those of the above system embodiment, and the beneficial effects achieved are also the same as those achieved by the above system. It should be understood that the functional logics implemented by the specific modules of each part of the above system embodiment all belong to the protection scope of the method embodiment of the present invention, and will not be described one by one here.

[0313] On the other hand, an embodiment of the present invention further provides an electronic device, the electronic device comprising a memory and a processor, the memory storing a computer program, and the processor implementing the multi-robot collaborative assembly method of the large model and the knowledge graph linkage when executing the computer program. The electronic device can be any intelligent terminal including a tablet computer, a vehicle-mounted computer, etc.

[0314] It can be understood that the contents of the above method embodiments are all applicable to the present device embodiments, the functions specifically implemented by the present device embodiments are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0315] like Figure 6 As shown, Figure 6 A specific example of the hardware structure of an electronic device 1000 according to an embodiment is shown. The electronic device 1000 includes:

[0316] The processor 1001 may be implemented by a general-purpose CPU (Central Processing Unit), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of the present invention.

[0317] The memory 1002 can be implemented in the form of a read-only memory (ROM), a static storage device, a dynamic storage device, or a random access memory (RAM). The memory 1002 can store an operating system and other application programs. When the technical solution provided in the embodiment of this specification is implemented by software or firmware, the relevant program code is stored in the memory 1002, and the processor 1001 calls and executes the network node population optimization method of the embodiment of the present invention;

[0318] Input / output interface 1003, used to implement information input and output;

[0319] The communication interface 1004 is used to realize the communication interaction between the device and other devices. The communication can be realized through a wired manner (such as USB, network cable, etc.) or a wireless manner (such as mobile network, WIFI, Bluetooth, etc.);

[0320] A bus 1005 , which transmits information between various components of the device (e.g., the processor 1001 , the memory 1002 , the input / output interface 1003 , and the communication interface 1004 );

[0321] The processor 1001 , the memory 1002 , the input / output interface 1003 and the communication interface 1004 are connected to each other in communication within the device via the bus 1005 .

[0322] The electronic device embodiments described above are merely illustrative, and the units described as separate components may or may not be physically separated, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0323] The contents of the method embodiments of the present invention are all applicable to the electronic device embodiments. The functions specifically implemented by the electronic device embodiments are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0324] Another aspect of an embodiment of the present invention further provides a computer-readable storage medium, wherein the storage medium stores a program, and the program is executed by a processor to implement the above method.

[0325] It should be noted that the computer-readable medium shown in the embodiment of the present invention may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM), a flash memory, an optical fiber, a portable compact disk read-only memory (Compact Disc Read to Only Memory, CD to ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present invention, a computer-readable storage medium may be any tangible medium containing or storing a program, which may be used by or in combination with an instruction execution system, device or device. In the present invention, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, which carries a computer-readable program code. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. Computer readable signal media may also be any computer readable medium other than computer readable storage media, which may send, propagate, or transmit programs for use by or in conjunction with an instruction execution system, apparatus, or device. The program code contained on the computer readable medium may be transmitted using any suitable medium, including but not limited to: wireless, wired, etc., or any suitable combination of the above.

[0326] The contents of the method embodiments of the present invention are all applicable to the computer-readable storage medium embodiments. The functions specifically implemented by the computer-readable storage medium embodiments are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0327] The embodiment of the present invention also discloses a computer program product or a computer program, which includes a computer instruction stored in a computer-readable storage medium. A processor of a computer device can read the computer instruction from the computer-readable storage medium, and the processor executes the computer instruction, so that the computer device executes the above method.

[0328] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present invention. In this regard, each box in the flow chart or block diagram can represent a module, a program segment, or a part of a code, and the above-mentioned module, program segment, or a part of a code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flow chart, and the combination of the boxes in the block diagram or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0329] It should be noted that, although several modules of the device for action execution are mentioned in the above detailed description, this division is not mandatory. In fact, according to an embodiment of the present invention, the features and functions of two or more modules or units described above can be embodied in one module or unit. On the contrary, the features and functions of one module or unit described above can be further divided into being embodied by multiple modules or units.

[0330] Through the description of the above implementation, it is easy for those skilled in the art to understand that the example implementation described here can be implemented by software, or by software combined with necessary hardware. Therefore, the technical solution according to the implementation of the present invention can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD to ROM, a USB flash drive, a mobile hard disk, etc.) or on a network, including several instructions to enable a computing device (which can be a personal computer, a server, a touch terminal, or a network device, etc.) to execute the method according to the implementation of the present invention.

[0331] In some selectable embodiments, the function / operation mentioned in the block diagram may not occur in the order mentioned in the operation diagram. For example, depending on the function / operation involved, the two boxes shown in succession can actually be executed substantially simultaneously or the boxes can sometimes be executed in reverse order. In addition, the embodiment presented and described in the flow chart of the present invention is provided by way of example, for the purpose of providing a more comprehensive understanding of technology. The disclosed method is not limited to the operation and logic flow presented herein. Selectable embodiments are expected, wherein the order of various operations is changed and the sub-operation of a part for which is described as a larger operation is performed independently.

[0332] In addition, although the present invention is described in the context of functional modules, it should be understood that, unless otherwise specified, one or more of the functions and / or features can be integrated into a single physical device and / or software module, or one or more functions and / or features can be implemented in a separate physical device or software module. It is also understood that a detailed discussion of the actual implementation of each module is unnecessary for understanding the present invention. More specifically, in view of the properties, functions and internal relationships of the various functional modules in the device disclosed herein, the actual implementation of the module will be understood within the conventional skills of the engineer. Therefore, those skilled in the art can implement the present invention set forth in the claims without excessive experimentation using ordinary techniques. It is also understood that the specific concepts disclosed are merely illustrative and are not intended to limit the scope of the present invention, which is determined by the full scope of the appended claims and their equivalents.

[0333] If the function is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art or the part of the solution of the present invention, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the methods of each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program codes.

[0334] The logic and / or steps represented in the flowchart or otherwise described herein, for example, may be considered as an ordered list of executable instructions for implementing logical functions, and may be embodied in any computer-readable medium for use by an instruction execution device, apparatus or device (such as a computer-based device, a device including a processor, or other device that can fetch instructions from an instruction execution device, apparatus or device and execute the instructions), or in conjunction with such instruction execution device, apparatus or device. For purposes of this specification, "computer-readable medium" may be any device that can contain, store, communicate, propagate or transmit a program for use by an instruction execution device, apparatus or device, or in conjunction with such instruction execution device, apparatus or device.

[0335] More specific examples of computer-readable media (a non-exhaustive list) include the following: an electrical connection with one or more wires (electronic device), a portable computer disk case (magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable and programmable read-only memory (EPROM or flash memory), a fiber optic device, and a portable compact disk read-only memory (CDROM). In addition, the computer-readable medium may even be a paper or other suitable medium on which the program is printed, since the program may be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, deciphering or, if necessary, processing in another suitable manner, and then stored in a computer memory.

[0336] It should be understood that the various parts of the present invention can be implemented by hardware, software, firmware or a combination thereof. In the above-mentioned embodiments, a plurality of steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution device. For example, if implemented by hardware, as in another embodiment, it can be implemented by any one of the following technologies known in the art or their combination: a discrete logic circuit having a logic gate circuit for implementing a logic function for a data signal, a dedicated integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0337] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "examples", "specific examples", or "some examples" means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representation of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner.

[0338] Although the embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the claims and their equivalents.

[0339] The above is a specific description of the preferred implementation of the present invention, but the present invention is not limited to the embodiments. Those skilled in the art can make various equivalent modifications or substitutions without violating the spirit of the present invention. These equivalent modifications or substitutions are all included in the scope defined by the claims of the present invention.

Claims

1. A multi-robot collaborative assembly system that integrates a large model with a knowledge graph, characterized in that: It includes perception layer, data processing layer, knowledge graph layer, task decision layer, communication and synchronization layer, and execution layer; The perception layer is used to collect data information of the assembly environment and assembly tasks in real time; The data processing layer is used to pre-process the data information collected by the perception layer and perform in-depth analysis using a preset large model to obtain structured information; The knowledge graph layer is used to construct a knowledge graph according to the structured information output by the data processing layer; The task decision layer is used to perform task decomposition, allocation and optimization scheduling based on the knowledge graph, combined with the application data of each robot and the task execution status fed back by the execution layer, to obtain task decision instructions; The communication and synchronization layer is used to establish a high-speed communication network inside the multi-robot collaborative assembly system and realize time synchronization of various devices in the system through a time synchronization protocol; The execution layer is used to control the corresponding robot to execute the assigned target assembly task based on the task decision instruction; and feed back the task execution status of the robot to the task decision layer.

2. The multi-robot collaborative assembly system based on the linkage between the large model and the knowledge graph according to claim 1 is characterized in that: The perception layer includes a multi-sensor fusion module and a task information acquisition module; the data information includes environmental information and task data; The multi-sensor fusion module is used to collect the environmental information in real time based on preset sensors; the preset sensors include visual sensors, laser radars and force sensors; The task information acquisition module is used to acquire the task data updated in real time through a system interface based on a standardized data format.

3. The multi-robot collaborative assembly system based on the linkage between the large model and the knowledge graph according to claim 1 is characterized in that: The data processing layer includes a pre-processing module and a large model processing module: The preprocessing module is used to preprocess the data information output by the perception layer; the preprocessing includes data cleaning and formatting; The large model processing module is used to use a pre-trained large-scale language model to extract key information from the pre-processed data information, and then organize it to obtain the structured information; the structured information includes entity information and relationship information; the entity information includes assembly parts, assembly tools, robots and assembly steps; the relationship information includes assembly sequence and task dependencies.

4. The multi-robot collaborative assembly system based on the linkage between the large model and the knowledge graph according to claim 1 is characterized in that: The structured information includes entity information and relationship information; the knowledge graph layer includes a knowledge graph construction module and a knowledge graph update module; The knowledge graph construction module is used to store the structured information as the knowledge graph through a graph database; The nodes in the graph database represent the entity information, which includes assembly parts, assembly tools, robots and assembly steps; the edges in the graph database represent the relationship information, which includes assembly sequence and task dependency; The knowledge graph update module is used to monitor the update information about the structured information output by the data processing layer through a real-time monitoring mechanism, and based on the update information, update the knowledge graph in an incremental manner through a hierarchical update mechanism.

5. The multi-robot collaborative assembly system based on the linkage of large model and knowledge graph according to claim 1 is characterized in that: The task decision layer includes a task decomposition module, a task allocation module and a scheduling optimization module; The task decomposition module is used to decompose the assembly task into multiple subtasks based on the knowledge graph by using a topological sorting algorithm; The task allocation module is used to obtain a task allocation plan based on the assembly capability, usage status and load condition of each robot by using a preset optimization algorithm; the optimization algorithm includes a Hungarian algorithm or a genetic algorithm; The scheduling optimization module is used to optimize and adjust the task allocation plan according to the task execution status combined with the corresponding load condition of the robot, task progress and environmental changes.

6. The multi-robot collaborative assembly system based on the linkage of large model and knowledge graph according to claim 1 is characterized in that: The communication and synchronization layer includes a high-speed communication network module and a time synchronization module; The high-speed communication network module is used to establish the high-speed communication network inside the multi-robot collaborative assembly system based on a preset communication network and communication protocol; The time synchronization module is used to achieve time synchronization of various devices in the multi-robot collaborative assembly system based on a preset precise time protocol.

7. The multi-robot collaborative assembly system based on the linkage of large model and knowledge graph according to claim 1 is characterized in that: The data information includes environmental information; the execution layer includes a robot control module, a collaborative obstacle avoidance module and a state feedback module; The robot control module is used to control the corresponding robot to perform the assigned target assembly task according to the task decision instruction; The collaborative obstacle avoidance module is used to plan an obstacle avoidance path for the robot performing the task through a model predictive control algorithm according to the environmental information; The state feedback module is used to feed back the task execution state of the robot executing the target assembly task to the task decision layer in real time.

8. The multi-robot collaborative assembly system based on the linkage of large model and knowledge graph according to claim 1 is characterized in that: The system also includes a human-computer interaction layer; The human-machine interaction layer is used to push task data to the operator according to a preset interaction interface or receive the operator's control instructions for the multi-robot collaborative assembly system.

9. The multi-robot collaborative assembly system based on the linkage between the large model and the knowledge graph according to claim 8 is characterized in that: The human-computer interaction layer includes a monitoring and visualization module, a manual intervention module, and an alarm and notification module; The monitoring and visualization module is used to push the system status of the multi-robot collaborative assembly system, the task progress of the robots in executing the target assembly task, and the position information of the robots to the interactive interface; The manual intervention module is used to adjust the task decision instruction or stop the multi-robot collaborative assembly system in response to the control instruction of the operator received by the interactive interface; The alarm and notification module is used to push abnormal conditions of the multi-robot collaborative assembly system to the interactive interface for alarm and prompt.

10. A multi-robot collaborative assembly method that combines a large model with a knowledge graph, characterized in that: Applied to the multi-robot collaborative assembly system linked with the large model and the knowledge graph as claimed in any one of claims 1 to 9, the method comprises the following steps: Collect data information about assembly environment and assembly tasks in real time through the perception layer; Preprocessing the data information collected by the perception layer through the data processing layer and performing in-depth analysis using a preset large model to obtain structured information; Constructing a knowledge graph according to the structured information output by the data processing layer through the knowledge graph layer; The task decision layer performs task decomposition, allocation and optimization scheduling based on the knowledge graph, combined with the application data of each robot and the task execution status fed back by the execution layer, to obtain task decision instructions; A high-speed communication network is established within the multi-robot collaborative assembly system through a communication and synchronization layer, and time synchronization of each device in the system is achieved through a time synchronization protocol; Based on the task decision instruction, the execution layer controls the corresponding robot to execute the assigned target assembly task; and feeds back the task execution status of the robot to the task decision layer.

Images