Cross-protocol task coordination method and system for multi-source heterogeneous robot cluster

Abstract

The present application relates to the technical field of robot cooperative control, and particularly relates to a cross-protocol task cooperation method and system for a multi-source heterogeneous robot cluster, which comprises the following steps: passively monitoring communication messages, extracting protocol characteristic information from the communication messages, and generating protocol characteristic fingerprints corresponding to each robot; matching the protocol characteristic fingerprints with a preset protocol fingerprint library, loading a protocol adapter matched with the protocol characteristic fingerprints for the corresponding robot when the matching is successful, and actively detecting the corresponding robot when the matching fails, and generating a corresponding protocol adapter according to the detection result; analyzing and mapping the original communication messages of each robot into standardized cooperation messages carrying cooperation semantic information through the protocol adapter, and publishing the standardized cooperation messages to a unified message bus; and performing task decomposition, capability matching and timing constraint determination based on the standardized cooperation messages in the unified message bus, and generating a cooperation execution scheme.

Description

Technical Field

[0001] This invention relates to the field of robot cooperative control technology, and in particular to a cross-protocol task cooperative method and system for multi-source heterogeneous robot swarms. Background Technology

[0002] With the deepening development of flexible manufacturing, collaborative operations of multi-robot clusters in industrial manufacturing scenarios are becoming increasingly common. In automotive welding workshops, welding robots, laser-guided AGVs, and handling robotic arms need to work together to complete the production cycle of "material feeding-welding-transfer," while in electronic SMT production lines, placement robots, inspection robots, and sorting AGVs need to work together to complete the online process of "placement-inspection-sorting." However, these automated devices for different processes often come from different suppliers, and each supplier's robots use their own independent communication protocols, such as ROS2, MQTT, OPC-UA, EtherCAT, CANopen, PROFINET, Modbus TCP, etc., making direct communication and collaboration between them impossible.

[0003] In existing technologies, to enable robots with heterogeneous protocols to collaboratively complete the same batch of production tasks, it is usually necessary to manually develop a dedicated protocol conversion gateway for each pair of "robot A and robot B", or to write instruction adaptation code separately for each robot's protocol in the MES system, or to use a PLC as a protocol relay node.

[0004] These approaches all suffer from a fundamental technical flaw: robots using heterogeneous protocols cannot automatically adapt and connect, failing to achieve plug-and-play functionality. This results in long protocol integration development cycles and high integration costs. For a production line with three protocols and three suppliers, the protocol integration development cycle is approximately two to three weeks. Adding a fourth supplier's robot requires redeveloping an adapter, adding another one to two weeks. In a flexible manufacturing model of "small batches and multiple varieties," protocol configuration becomes the biggest efficiency bottleneck during production line changeovers. Especially when robots use unpublished proprietary protocols, existing solutions cannot automatically adapt, requiring manual development based on protocol documentation provided by the manufacturer, further lengthening the integration cycle.

[0005] The information disclosed in this background section is intended only to enhance the understanding of the general background of the invention and should not be construed as an admission or in any way implying that the information constitutes prior art known to those skilled in the art. Summary of the Invention

[0006] This invention provides a cross-protocol task collaboration method and system for multi-source heterogeneous robot clusters, thereby effectively solving the problems in the background art.

[0007] To achieve the above objectives, the technical solution adopted by this invention is: a cross-protocol task collaboration method and system for multi-source heterogeneous robot swarms, comprising the following steps:

[0008] The communication messages of each robot in the cluster are passively monitored in the communication network. Protocol feature information is extracted from the communication messages to generate a protocol feature fingerprint for each robot.

[0009] The protocol feature fingerprint is matched with a pre-set protocol fingerprint database. When the match is successful, a protocol adapter matching the protocol feature fingerprint is loaded for the corresponding robot. When the match fails, the corresponding robot is actively probed, and the corresponding protocol adapter is generated based on the probe results.

[0010] The protocol adapter parses and maps the native communication messages of each robot into standardized collaborative messages carrying collaborative semantic information, and publishes the standardized collaborative messages to the unified message bus.

[0011] Based on the standardized collaborative messages in the unified message bus, task decomposition, capability matching, and timing constraint determination are performed to generate a collaborative execution plan. The collaborative execution plan is then converted into the native protocol instructions of the corresponding robot by each protocol adapter and then sent out for execution.

[0012] Furthermore, the protocol feature fingerprint includes message header features, field semantic features, and timing features;

[0013] The message header features include the sequence of magic number bytes, the position of the version number, and the position of the message length field. The field semantic features include the positions of the command field, the address field, and the data payload field. The timing features include the request-response interval distribution and the message sending period.

[0014] Furthermore, the standardized collaborative message includes task semantic fields, spatiotemporal attribute fields, collaborative relationship fields, and priority fields;

[0015] The task semantic field is used to characterize the production task information associated with the standardized collaborative message; the spatiotemporal attribute field is used to characterize the position coordinates, time window and motion constraints; the collaborative relationship field is used to characterize the collaborative type, follow-up robot and synchronization point; and the priority field is used to characterize the scheduling priority of the standardized collaborative message in the unified message bus.

[0016] Furthermore, the step of performing task decomposition, capability matching, and timing constraint determination based on the standardized collaborative messages in the unified message bus to generate a collaborative execution plan includes:

[0017] Extract the capability parameters and current status of each robot from the standardized collaborative message;

[0018] Decompose production tasks into a set of subtasks;

[0019] Calculate the capability matching degree between each subtask and each robot, and assign each subtask to the corresponding robot according to the capability matching degree;

[0020] The execution sequence and synchronization point of each subtask are determined based on the collaboration relationship field; the collaborative execution scheme containing the collaborative action sequence of each robot is generated based on the execution sequence and the synchronization point.

[0021] Furthermore, the collaborative execution scheme includes synchronization points;

[0022] After the first robot completes the collaborative action corresponding to the synchronization point, it sends a synchronization notification to the subsequent robot represented by the collaborative relationship field.

[0023] After receiving the synchronization notification, the rear robot begins to execute the corresponding collaborative action.

[0024] Further, matching the protocol feature fingerprint with a pre-set protocol fingerprint database includes:

[0025] Calculate the cosine similarity between the protocol feature fingerprint and the preset protocol template in the protocol fingerprint database. If the cosine similarity is greater than or equal to a preset threshold, the match is considered successful. If the cosine similarity is less than the preset threshold, the match is considered unsuccessful.

[0026] Furthermore, the step of actively probing the corresponding robot and generating the corresponding protocol adapter based on the probing results includes:

[0027] Send a preset sequence of detection messages to the corresponding robot;

[0028] Collect the response messages returned by the robot in response to the sequence of probe messages;

[0029] Based on the response message, the protocol state transition relationship of the robot is inferred, and a protocol state machine is constructed;

[0030] The protocol state machine automatically generates message parsing rules and instruction encoding rules;

[0031] The protocol adapter is generated by assembling the protocol parsing rules and the instruction encoding rules, and the corresponding protocol feature fingerprint is added to the protocol fingerprint database.

[0032] Furthermore, the unified message bus adopts a publish-subscribe mechanism based on cooperative relationships, and is divided into serial cooperative topics, parallel cooperative topics, and master-slave cooperative topics according to the cooperative type;

[0033] Each protocol adapter publishes the standardized collaboration message to the topic corresponding to its collaboration type. The collaboration decision-making end subscribes to the topic to obtain the standardized collaboration message and performs queue scheduling according to the priority field.

[0034] Furthermore, the protocol adapters are cached to form an adapter pool;

[0035] When multiple protocol adapters need to perform message parsing and message mapping operations, they are executed in parallel using a pipelined approach.

[0036] When a new protocol adapter is generated, existing protocol adapters with the same or similar functions in the adapter pool are reused first.

[0037] Furthermore, the protocol fingerprint database and the protocol adapter are deployed on a cloud server, which is shared by multiple factories; when a new protocol adapter is generated, the new protocol adapter is automatically synchronized to each of the factories.

[0038] Furthermore, before passively monitoring the communication messages of each robot in the cluster within the communication network, the method further includes:

[0039] The robot's communication terminal is authenticated with a digital certificate; the robot's declared capability parameters are compared and verified with the actual capability parameters obtained through active detection; only when both authentication and capability comparison verification are passed is passive listening and active detection of the robot permitted.

[0040] Furthermore, the method also includes:

[0041] Receive feedback messages returned by each of the robots after executing the cooperative execution scheme;

[0042] When the feedback message indicates an execution error, the environmental constraint information corresponding to the error is appended to the environmental constraints of the current task.

[0043] The collaborative execution scheme is regenerated based on the updated environmental constraints, and the updated collaborative execution scheme is then deployed for execution.

[0044] Update the capability profile of the corresponding robot based on the feedback message.

[0045] The present invention also includes a cross-protocol task collaboration system for multi-source heterogeneous robot swarms, applied to the method described above, the system comprising:

[0046] The protocol sniffing module is used to passively listen to the communication packets of each robot in the cluster in the communication network, extract protocol feature information from the communication packets, and generate a protocol feature fingerprint corresponding to each robot.

[0047] The adapter management module is used to match the protocol feature fingerprint with a preset protocol fingerprint library. When the match is successful, it loads a protocol adapter that matches the protocol feature fingerprint for the corresponding robot. When the match fails, it actively probes the corresponding robot and generates the corresponding protocol adapter based on the probe results.

[0048] The message bus module is used to parse and map the native communication messages of each robot into standardized collaborative messages carrying collaborative semantic information through each of the protocol adapters, and to uniformly publish and subscribe to the standardized collaborative messages.

[0049] The collaborative decision-making module is used to decompose tasks, match capabilities, and determine timing constraints based on the standardized collaborative messages, generate a collaborative execution plan, and then convert the collaborative execution plan into the native protocol instructions of the corresponding robot through each of the protocol adapters before issuing it for execution.

[0050] Furthermore, the system also includes a feedback optimization module, which is used to receive feedback messages returned by each of the robots after executing the collaborative execution scheme;

[0051] When the feedback message indicates an execution error, the environmental constraint information corresponding to the error is added to the environmental constraints of the current task, and the collaborative decision-making module is triggered to regenerate the collaborative execution plan and update the corresponding robot's capability profile according to the feedback message.

[0052] Furthermore, the system also includes a security authentication module, which is used to perform digital certificate authentication on the robot's communication terminal before passive eavesdropping, and to compare and verify the robot's declared capability parameters with the actual capability parameters obtained through active detection.

[0053] The present invention also includes a computer device comprising a processor and a memory coupled to the processor and the memory, the memory storing program instructions which, when executed by the processor, implement the method described above.

[0054] The present invention also includes a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the above-described method.

[0055] The present invention also includes a computer program product comprising a computer program that, when run on a computer, causes the computer to perform the methods described above.

[0056] The beneficial effects of this invention are as follows: By passively listening to extract protocol feature fingerprints and matching them with a pre-set protocol fingerprint library, known protocols can be directly loaded with adapters, while unknown private protocols can be automatically constructed into protocol state machines and adapters generated through active detection. When new vendor robots are integrated into the production line, there is no need to provide protocol documentation or manually write adaptation programs. Standardized collaborative messages carry four types of collaborative semantic information: task semantics, spatiotemporal attributes, collaborative relationships, and priority. This enables collaborative decisions to be made based on unified semantics, significantly increasing the success rate of collaborative tasks compared to existing message forwarding schemes, and achieving automatic adaptation and deep task collaboration for robots with heterogeneous protocols. Attached Figure Description

[0057] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0058] Figure 1 This is a flowchart of the method in Embodiment 1 of the present invention;

[0059] Figure 2 This is a schematic diagram of the system structure in Embodiment 1 of the present invention;

[0060] Figure 3 This is an overall flowchart of the cross-protocol task collaboration method provided in Embodiment 2 of the present invention;

[0061] Figure 4 This is a schematic diagram of the cross-protocol task collaboration system provided in Embodiment 2 of the present invention;

[0062] Figure 5 This is a schematic diagram of the protocol feature fingerprint extraction and adapter generation process provided in Embodiment 2 of the present invention;

[0063] Figure 6 This is a schematic diagram of the standardized collaborative message structure and publish-subscribe mechanism provided in Embodiment 2 of the present invention;

[0064] Figure 7 This is a schematic diagram of the cross-protocol collaborative decision-making process provided in Embodiment 2 of the present invention;

[0065] Figure 8 This is a schematic diagram of the cross-protocol task collaboration device provided in Embodiment 4 of the present invention.

[0066] Figure 9 This is a schematic diagram of the computer device in Embodiment 6 of the present invention. Detailed Implementation

[0067] The technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments.

[0068] Example 1:

[0069] like Figure 1 As shown: A cross-protocol task collaboration method for multi-source heterogeneous robot swarms includes the following steps:

[0070] The system passively listens to the communication messages of each robot in the cluster in the communication network, extracts protocol feature information from the communication messages, and generates a protocol feature fingerprint for each robot.

[0071] The protocol feature fingerprint is matched with a pre-set protocol fingerprint library. When the match is successful, a protocol adapter that matches the protocol feature fingerprint is loaded for the corresponding robot. When the match fails, the corresponding robot is actively probed, and a corresponding protocol adapter is generated based on the probe results.

[0072] The protocol adapter parses and maps the native communication messages of each robot into standardized collaborative messages carrying collaborative semantic information, and publishes the standardized collaborative messages to the unified message bus.

[0073] Based on standardized collaborative messages in the unified message bus, task decomposition, capability matching, and timing constraint determination are performed to generate a collaborative execution plan. The collaborative execution plan is then converted into the native protocol instructions of the corresponding robot by each protocol adapter and then issued for execution.

[0074] By passively listening to extract protocol feature fingerprints and matching them with a pre-built protocol fingerprint database, known protocols can be directly loaded with adapters, while unknown proprietary protocols can be actively probed to automatically build protocol state machines and generate adapters. When new vendor robots are integrated into the production line, there is no need to provide protocol documentation or manually write adaptation programs. Standardized collaborative messages carry four types of collaborative semantic information: task semantics, spatiotemporal attributes, collaborative relationships, and priority. This enables collaborative decisions to be made based on unified semantics, significantly increasing the success rate of collaborative tasks compared to existing message forwarding schemes, and achieving automatic adaptation and connection of heterogeneous protocol robots and deep task collaboration.

[0075] In this embodiment, the protocol feature fingerprint includes message header features, field semantic features, and timing features;

[0076] The message header features include the sequence of magic number bytes, the position of the version number, and the position of the message length field. The field semantic features include the position of the command field, the position of the address field, and the position of the data payload field. The time series features include the request-response interval distribution and the message sending period.

[0077] Standardized collaborative messages include task semantic fields, spatiotemporal attribute fields, collaborative relationship fields, and priority fields;

[0078] The task semantic field is used to characterize the production task information associated with the standardized collaborative message; the spatiotemporal attribute field is used to characterize the position coordinates, time window and motion constraints; the collaborative relationship field is used to characterize the collaborative type, follow-up robot and synchronization point; and the priority field is used to characterize the scheduling priority of the standardized collaborative message in the unified message bus.

[0079] This includes task decomposition, capability matching, and timing constraint determination based on standardized collaborative messages in the unified message bus, generating a collaborative execution plan, including:

[0080] Extract the capability parameters and current status of each robot from standardized collaborative messages;

[0081] Decompose production tasks into a set of subtasks;

[0082] Calculate the capability matching degree between each subtask and each robot, and assign each subtask to the corresponding robot based on the capability matching degree;

[0083] The execution sequence and synchronization point of each subtask are determined based on the collaboration relationship field; a collaborative execution scheme containing the collaborative action sequence of each robot is generated based on the execution sequence and synchronization point.

[0084] As a preferred embodiment of the above, the collaborative execution scheme includes a synchronization point;

[0085] Once the first robot completes the collaborative action corresponding to the synchronization point, it sends a synchronization notification to the subsequent robot represented by the collaborative relationship field.

[0086] After receiving the synchronization notification, the rear robot begins to execute the corresponding collaborative actions.

[0087] In this embodiment, matching the protocol feature fingerprint with a preset protocol fingerprint database includes:

[0088] Calculate the cosine similarity between the protocol feature fingerprint and the preset protocol template in the protocol fingerprint database. If the cosine similarity is greater than or equal to a preset threshold, the match is considered successful. If the cosine similarity is less than the preset threshold, the match is considered unsuccessful.

[0089] Actively probe the corresponding robot and generate a corresponding protocol adapter based on the probe results, including:

[0090] Send a preset sequence of probe messages to the corresponding robot;

[0091] Collect the response messages returned by the robot in response to the probe message sequence;

[0092] Infer the robot's protocol state transition relationships from the response messages and construct the protocol state machine;

[0093] Automatically generate message parsing rules and instruction encoding rules based on the protocol state machine;

[0094] The protocol adapter is assembled and generated based on the message parsing rules and instruction encoding rules, and the corresponding protocol feature fingerprints are added to the protocol fingerprint database.

[0095] The unified message bus adopts a publish-subscribe mechanism based on collaboration relationships, and is divided into serial collaboration topics, parallel collaboration topics, and master-slave collaboration topics according to collaboration type.

[0096] Each protocol adapter publishes standardized collaboration messages to the topic corresponding to its collaboration type. The collaboration decision-making end subscribes to the topic to obtain standardized collaboration messages and performs queue scheduling according to the priority field.

[0097] As a preferred embodiment of the above, the protocol adapter is cached to form an adapter pool;

[0098] When multiple protocol adapters need to perform message parsing and message mapping operations, they are executed in parallel using a pipelined approach.

[0099] When a new protocol adapter is generated, existing protocol adapters with the same or similar functions in the adapter pool are reused first.

[0100] As a preferred embodiment of the above, the protocol fingerprint library and protocol adapter are deployed on a cloud server, which is shared by multiple factories; when a new protocol adapter is generated, the new protocol adapter is automatically synchronized to each factory.

[0101] In this embodiment, before passively monitoring the communication messages of each robot in the cluster in the communication network, the method further includes:

[0102] The robot's communication terminal is authenticated with a digital certificate; the robot's declared capability parameters are compared and verified with the actual capability parameters obtained through active detection; passive listening and active detection of the robot are only permitted when both authentication and capability comparison verification are passed.

[0103] As a preferred embodiment of the above, the method further includes:

[0104] Receive feedback messages returned by each robot after executing the collaborative execution plan;

[0105] When a feedback message indicates an execution exception, the environmental constraint information corresponding to the exception is appended to the environmental constraints of the current task.

[0106] The collaborative execution plan is regenerated based on the updated environmental constraints, and then the updated collaborative execution plan is deployed for execution.

[0107] Update the capability profile of the corresponding robot based on feedback.

[0108] like Figure 2 As shown, this embodiment also includes a cross-protocol task collaboration system for multi-source heterogeneous robot swarms, applied to the method described above. The system includes:

[0109] The protocol sniffing module is used to passively listen to the communication messages of each robot in the cluster in the communication network, extract protocol feature information from the communication messages, and generate the protocol feature fingerprint corresponding to each robot.

[0110] The adapter management module is used to match protocol feature fingerprints with a pre-set protocol fingerprint library. When a match is successful, it loads a protocol adapter that matches the protocol feature fingerprint for the corresponding robot. When a match fails, it actively probes the corresponding robot and generates a corresponding protocol adapter based on the probe results.

[0111] The message bus module is used to parse and map the native communication messages of each robot into standardized collaborative messages carrying collaborative semantic information through each protocol adapter, and to uniformly publish and subscribe to the standardized collaborative messages.

[0112] The collaborative decision-making module is used to decompose tasks, match capabilities, and determine timing constraints based on standardized collaborative messages, generate collaborative execution plans, and then convert the collaborative execution plans into native protocol instructions for the corresponding robots through various protocol adapters before issuing them for execution.

[0113] The system also includes a feedback optimization module, which is used to receive feedback messages returned by each robot after executing the collaborative execution plan;

[0114] When a feedback message indicates an execution error, the environmental constraint information corresponding to the error is added to the environmental constraints of the current task, and the collaborative decision-making module is triggered to regenerate the collaborative execution plan and update the corresponding robot's capability profile based on the feedback message.

[0115] As a preferred embodiment of the above, the system further includes a security authentication module, which is used to perform digital certificate authentication on the robot's communication terminal before passive eavesdropping, and to compare and verify the robot's declared capability parameters with the actual capability parameters obtained through active detection.

[0116] Example 2: Cross-protocol collaboration scenario in an automotive welding workshop

[0117] like Figure 3 As shown, this embodiment provides a cross-protocol task collaboration method applied to an automotive welding workshop. The workshop is equipped with: 3 six-axis welding robots in Plant A, using the EtherCAT protocol, responsible for welding the B-pillars and side panels of the vehicle body, with a working radius of 2.5m and a welding torch load of 6kg; 5 laser-guided AGVs in Plant B, using the ROS2 protocol, responsible for transporting body parts from the material area to the workstation, with a load of 500kg and a speed of 1.5m / s; 2 six-axis handling robotic arms in Plant C, using the OPC-UA protocol, responsible for transferring welded parts to the final assembly line buffer zone, with a gripping load of 80kg; and 1 online inspection robot in Plant D, using a private TCP protocol, responsible for visual inspection of weld quality, a newly introduced piece of equipment on the production line. A collaboration server is deployed at the edge of the production line, using an industrial PC (Intel i7 processor, 32GB memory), connected to each robot via an industrial switch, such as... Figure 4 As shown.

[0118] Step 1: Protocol Feature Sniffing. The collaborative server passively listens to the communication messages of each robot in the factory network within a 5-minute time window, without actively sending any requests or affecting existing communication. It extracts three dimensions of protocol feature information from the messages: message header features (magic number byte sequence, version number position, message length field position), field semantic features (command field position, address field position, data payload field position), and temporal features (request-response interval distribution, message sending period). These are then concatenated to generate the protocol feature fingerprint for each robot. For example... Figure 5 As shown, taking the EtherCAT welding robot from Factory A as an example, its message header features include the magic number 0x885A, the version number in Byte 2, and the length field in Byte 4. Semantic features include the command field in Byte 8, the address field in Byte 12, and the data payload in Byte 16. Timing features include an average request-response interval of 3.2ms and a message transmission period of 250μs. Because the Industrial Ethernet protocol has a fixed message period and frame structure, its fingerprint features are more stable and have a higher matching accuracy than those of general IoT protocols.

[0119] Step 2: Fingerprint Matching and Adapter Generation. The protocol feature fingerprint extracted in Step 1 is matched with a pre-built protocol fingerprint library (pre-installed with templates for seven common industrial protocols: ROS2, MQTT, OPC-UA, EtherCAT, CANopen, PROFINET, and Modbus TCP) using cosine similarity, with a preset threshold of 0.85. The fingerprint of the welding robot from Factory A has a cosine similarity of 0.97 with the EtherCAT template in the library; the fingerprint of the AGV from Factory B has a similarity of 0.96 with the ROS2 template; and the fingerprint of the handling robot arm from Factory C has a similarity of 0.91 with the OPC-UA template. All three are greater than or equal to 0.85, indicating a successful match, and the corresponding protocol adapter is directly loaded. Taking the EtherCAT welding robot from Factory A as an example, the loaded EtherCAT adapter contains three sub-modules: a message parsing sub-module (parses EtherCAT frames into structured intermediate representations), a message mapping sub-module (maps the intermediate representations into standardized collaborative messages), and an instruction generation sub-module (encodes collaborative actions into EtherCAT instruction frames). The fingerprint of Factory D's inspection robot had a similarity score below 0.85 with all templates in the database (the highest similarity score was 0.61), resulting in a matching failure and triggering an active detection process: The collaborative server sent a sequence of 6 rounds of detection messages (TCP handshake, heartbeat, status query, capability query, movement command, and stop command) to Factory D's inspection robot, collected the response messages of each round, inferred from the response messages that the protocol contained 4 state nodes and 7 state transition edges, automatically constructed a protocol state machine, generated message parsing rules and command encoding rules based on the state machine, assembled and generated a new protocol adapter A4, and added Factory D's protocol feature fingerprint to the protocol fingerprint database. From the time the Factory D inspection robot connected to the production line to its participation in the collaborative task, it took approximately 1.5 hours (including protocol sniffing, active detection, adapter generation, and functional verification), during which no protocol documentation from Factory D was required.

[0120] Step 3: Standardize the unified message bus. Each protocol adapter parses the native messages and maps them to standardized collaborative messages in a unified format, such as... Figure 6As shown, standardized collaborative messages contain four fields: task semantic fields (e.g., {task type: "welding", workpiece number: "B-pillar-056", action: "positioning completed"}), spatiotemporal attribute fields (e.g., {workstation: "WS03", time window: [0,120]s, safety zone: "ZoneA"}), collaborative relationship fields (e.g., {collaboration type: "serial collaboration", follow-up: ["robotic arm C01"], synchronization point: "welding completed"}), and priority fields (e.g., priority: HIGH, used for safety-related messages). The unified message bus adopts a publish-subscribe mechanism based on collaborative relationships, dividing into serial collaborative topics, parallel collaborative topics, and master-slave collaborative topics according to collaborative type. Each adapter publishes standardized collaborative messages to the corresponding topic, and the collaborative decision module, as a subscriber, obtains messages from each topic and performs queue scheduling according to the priority field. The adapter pool caches the generated adapters, achieving a reuse rate of 78.3%. The message parsing and message mapping operations for multiple adapters are executed in parallel using a pipeline, reducing message conversion latency from 45ms to 12ms, a decrease of approximately 73%.

[0121] Step 4, cross-protocol task collaborative decision-making. The collaborative server receives the production task "B-pillar welding batch 056, quantity 3 pieces" issued by MES, and collects the status of each robot through the unified message bus (welding robot A02 idle rate 75%, AGV-B05 battery 82%, located at the material area entrance), such as... Figure 7As shown, the collaborative decision-making module executes the following sub-steps: A. Capability and Status Extraction: Extract the capability parameters (load, speed, working radius, etc.) and current status (busy / idle / faulty / waiting) of each robot from the standardized collaborative messages; B. Task Decomposition and Capability Matching: Decompose the "B-pillar welding" task into a set of sub-tasks: ST1 (AGV picks up B-pillar components from the material area and transports them to workstation WS03), ST2 (welding robot positions, clamps, and performs welding), ST3 (transferring the welded components to the final assembly line buffer area), and ST4 (inspection robot performs visual inspection of the weld quality); Calculate the capability matching degree between each sub-task and each robot: ST1 matches AGV (matching degree 0.92), ST2 matches welding robot (matching degree 0.96), ST3 matches transfer robot (matching degree 0.88), ST4 matches transfer robot (matching degree 0.88), ST5 matches transfer robot (matching degree 0.92), ST6 matches transfer robot (matching degree 0.96), ST3 matches transfer robot (matching degree 0.88), ST4 matches transfer robot (matching degree 0.92), ST5 matches transfer robot (matching degree 0.96), ST2 matches transfer robot (matching degree 0.96), ST3 matches transfer robot (matching degree 0.96), ST4 matches transfer robot (matching degree 0.96), ST2 ...2 matches transfer robot (matching degree 0.96), ST2 matches transfer robot (matching degree 0.96), ST2 matches transfer robot (matching degree 0.96 4. Matching and testing robots (matching degree 0.85), assigning each subtask to the corresponding robot; C. Determining execution timing and collaborative constraints: Based on the collaborative relationship field, the execution timing is determined as ST1→ST2→ST3→ST4 (all are serial collaborations), and the synchronization points are "AGV arrives at WS03" (welding robot starts clamping) and "welding completed and quality inspection passed" (transfer robot starts transporting); D. Generating collaborative execution scheme: Generating a collaborative execution scheme containing the collaborative action sequence and synchronization points of each robot, which is converted into native protocol instructions for each robot via a unified message bus and sent for execution—AGV-B05 receives ROS2 navigation instructions, welding robot A02 receives EtherCAT motion control instructions, transfer robot C01 receives OPC-UA motion instructions, and testing robot receives private TCP instructions.

[0122] During execution, the robots coordinated at synchronization points: After AGV-B05 completed navigation and transport actions to workstation WS03, it sent a synchronization notification to welding robot A02. Upon receiving the notification, welding robot A02 began clamping and executing welding program WELD_12. After welding was completed and quality inspection passed, it sent a synchronization notification to the handling robot C01, which then began gripping and transferring the components to the final assembly line buffer area BAY7. The entire batch of 3 pieces was produced continuously in 12.6 minutes (approximately 4.2 minutes per piece), with synchronization point signal delays all less than 15ms, and a collaborative task success rate of 94.1%.

[0123] Anomaly Handling: During the welding of the second B-pillar, AGV-B05 encountered a temporarily stacked material box blocking its path during transport. AGV-B05 reported a "path blockage" anomaly feedback message via the ROS2 adapter. The feedback optimization module added the obstacle information as an environmental constraint, triggering the collaborative decision-making module to re-determine the issue and generate an updated solution (AGV-B05 reroutes to an alternative path, and the synchronization point of subsequent processes is delayed by 45 seconds). An update command was issued, and welding robot A02 automatically waited for the updated synchronization point. The entire process was completed automatically within 3.2 seconds without manual intervention. At the same time, the capability profile of AGV-B05 was updated based on the feedback message (recording the speed attenuation coefficient caused by the path blockage). In this embodiment, the dynamic environment collaboration success rate was 92.1%, which is 18 percentage points higher than the non-feedback closed-loop solution.

[0124] It should be noted that the order of the above steps does not constitute a limitation on the present invention. Those skilled in the art can adjust the execution order of each step according to the actual application scenario. For example, the security authentication step can be performed before protocol sniffing.

[0125] Example 3: Cross-protocol collaboration scenario in electronic SMT production lines

[0126] This embodiment verifies the transferability of the method of the present invention to other industrial scenarios. For example... Figure 3 As shown, this embodiment is applied to an electronic SMT production line, deploying: a placement robot (PROFINET protocol) in factory D, an AOI inspection robot (MQTT protocol) in factory E, and a sorting AGV (CANopen protocol) in factory F. A collaborative server connects each device through a unified message bus and a protocol adapter pool, such as... Figure 4 As shown, the production line task is "PCB board batch 071: SMT → Online inspection → NG product sorting → OK product warehousing".

[0127] The differences from Example 2 are as follows: First, the protocols of each device are PROFINET, MQTT, and CANopen. The cosine similarity between the fingerprints extracted during the protocol sniffing phase and the corresponding templates in the fingerprint database are 0.95, 0.93, and 0.90, respectively, all of which are successful matches. The adapter is loaded directly without active detection. Second, the task decomposition granularity is finer. The batch task is decomposed into patching sub-tasks, detection sub-tasks, sorting sub-tasks, and warehousing sub-tasks. The detection sub-task has a capability matching degree of 0.94 with the AOI detection robot, and the sorting sub-task has a capability matching degree of 0.91 with the sorting AGV. Third, the collaboration types include both serial collaboration (patch → detection) and parallel collaboration (NG product sorting and OK product warehousing are executed in parallel). Parallel collaboration tasks are published to the parallel collaboration topic, further verifying the support of the publish-subscribe mechanism for multiple collaboration types.

[0128] In this embodiment, the detection result message reported by the MQTT adapter is converted into a standardized collaborative message carrying the detection judgment result through the message mapping submodule. The collaborative decision module then distributes the batch to the sorting subtask based on the judgment result. The collaborative completion time for the entire batch of 100 PCBs is 8.5 minutes, with a collaborative task success rate of 95.2%, verifying the portability of the method of this invention to heterogeneous protocol combinations and different industrial scenarios.

[0129] Example 4: Cross-protocol task collaboration device example

[0130] like Figure 8 As shown, this embodiment provides a cross-protocol task collaboration device, which is deployed on a collaboration server and includes: a security authentication module 61, a protocol sniffing module 62, an adapter management module 63, a message bus module 64, a collaboration decision module 65, and a feedback optimization module 66.

[0131] The security authentication module 61 is used to perform digital certificate authentication on the robot's communication terminal before protocol sniffing, and to compare and verify the robot's declared capability parameters with the actual capability parameters obtained by active detection to prevent unauthorized devices from accessing the network and falsely labeled capabilities from interfering with collaboration. Only when both identity authentication and capability comparison verification are passed can passive listening and active detection of the corresponding robot be allowed.

[0132] The protocol sniffing module 62 is used to passively listen to the communication messages of each robot in the cluster in the communication network, extract protocol feature information from the communication messages, and generate the protocol feature fingerprint corresponding to each robot.

[0133] The adapter management module 63 is used to match the protocol feature fingerprint with the pre-set protocol fingerprint library. When the match is successful, it loads the protocol adapter that matches the protocol feature fingerprint for the corresponding robot. When the match fails, it actively probes the corresponding robot and generates the corresponding protocol adapter based on the probe results. The generated protocol adapters are cached to form an adapter pool. When multiple protocol adapters need to perform message parsing and message mapping operations, they are executed in parallel in a pipeline manner. When a new protocol adapter is generated, the existing protocol adapters with the same or similar functions in the adapter pool are reused first.

[0134] The message bus module 64 is used to parse and map the native communication messages of each robot into standardized collaborative messages carrying collaborative semantic information through each protocol adapter, and to uniformly publish and subscribe to the standardized collaborative messages. It adopts a publish-subscribe mechanism based on collaborative relationships, divides the collaborative topics into serial collaborative topics, parallel collaborative topics and master-slave collaborative topics according to the collaborative type, and performs queue scheduling according to the priority field.

[0135] The collaborative decision-making module 65 is used to decompose tasks, match capabilities, and determine timing constraints based on standardized collaborative messages, generate collaborative execution plans, and then convert the collaborative execution plans into native protocol instructions for the corresponding robots through each protocol adapter before issuing them for execution.

[0136] The feedback optimization module 66 is used to receive feedback messages returned by each robot after executing the collaborative execution plan; when the feedback message indicates an execution abnormality, the environmental constraint information corresponding to the abnormality is added to the environmental constraints of the current task, and the collaborative decision module 65 is triggered to regenerate the collaborative execution plan, and the capability profile of the corresponding robot is updated according to the feedback message.

[0137] The connections between modules are as follows: the output of the security authentication module 61 is connected to the input of the protocol sniffing module 62; the output of the protocol sniffing module 62 is connected to the input of the adapter management module 63; the protocol adapter generated by the adapter management module 63 is deployed on the message bus module 64; the output of the message bus module 64 is connected to the input of the collaborative decision-making module 65; and the output of the collaborative decision-making module 65 is connected to the communication interface of each robot via each protocol adapter. The input of the feedback optimization module 66 is connected to the communication interface of each robot, and the output of the feedback optimization module 66 is connected to both the collaborative decision-making module 65 and the adapter management module 63 (for mapping rule optimization and capability profile updates). Each module can be implemented by a processor deployed on the collaborative server executing computer programs stored in memory.

[0138] Example 5: Alternative Solution Example

[0139] This embodiment provides several alternative implementations of the present invention, all of which fall within the protection scope of the present invention.

[0140] Alternative Solution 1 (Cloud Adapter Management): The protocol fingerprint library and protocol adapters are deployed on a cloud server, which is shared by multiple factories. Each factory's collaborative server downloads the protocol adapters from the cloud. When a factory generates a new protocol adapter, it is automatically synchronized to the cloud server and other factories, forming a "protocol adapter application store" model, which is suitable for multi-factory group deployment scenarios.

[0141] Alternative Solution 2 (Decentralized Collaboration): The collaborative decision-making module is not centralized on a single collaborative server, but rather distributed across the edge computing units of each robot. These edge computing units exchange standardized collaborative messages and reach collaborative decisions through a rumor propagation consensus protocol, eliminating single points of failure for the server and making it suitable for extreme industrial environments with limited communication. In this solution, the publish-subscribe mechanism of the unified message bus is jointly maintained by all edge computing units.

[0142] Alternative Solution 3 (5G+TSN Deterministic Transmission): The underlying transmission of the unified message bus adopts a converged communication of 5G and Time-Sensitive Networking (TSN), ensuring deterministic low latency (less than 1ms) for collaborative synchronization point messages. It is suitable for motion control-level collaborative scenarios, such as cross-vendor collaboration of lithography machines, etching machines, and inspection machines in semiconductor cleanrooms; cross-protocol loading and unloading collaboration of gantry cranes (RTU protocol), AGVs (MQTT protocol), and unmanned trucks (DDS protocol) in smart ports; and cross-domain unmanned collaboration of excavators (CAN protocol), mining trucks (proprietary protocol), and drones (MAVLink protocol) in unmanned mining scenarios.

[0143] Example 6:

[0144] Please see Figure 9 The diagram shows a structural schematic of a computer device provided in an embodiment of this application. An embodiment of this application provides a computer device 400, including a processor 410 and a memory 420. The memory 420 stores a computer program executable by the processor 410. When the computer program is executed by the processor 410, it performs the method described above.

[0145] This application embodiment also provides a storage medium 430, on which a computer program is stored, and the computer program is executed by a processor 410 to perform the above method.

[0146] The storage medium 430 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as Static Random Access Memory (SRAM), Electrically Erasable Programmable Read-Only Memory (EEPROM), Erasable Programmable Read Only Memory (EPROM), Programmable Red-Only Memory (PROM), Read-Only Memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk.

[0147] In the description of this invention, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature. "A plurality of" means two or more, unless otherwise explicitly specified.

[0148] In this invention, unless otherwise explicitly specified and limited, the terms "installation," "connection," "linking," and "fixing," etc., should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral part; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; they can refer to the internal communication of two components or the interaction between two components. Those skilled in the art can understand the specific meaning of the above terms in this invention according to the specific circumstances.

[0149] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the present invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Furthermore, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of different embodiments or examples.

[0150] Any process or method description in the flowchart or otherwise herein can be understood as representing a module, segment, or portion of code comprising one or more executable instructions for implementing a particular logical function or process, and the scope of the preferred embodiments of the invention includes additional implementations in which functions may be performed not in the order shown or discussed, including substantially simultaneously or in reverse order depending on the functions involved, as will be understood by those skilled in the art to which embodiments of the invention pertain.

[0151] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as a ordered list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus, or device (such as a computer-based system, a processor-included system, or other system that can fetch and execute instructions from, an instruction execution system, apparatus, or device). For the purposes of this specification, "computer-readable medium" can be any means that can contain, store, communicate, propagate, or transmit programs for use by, or in conjunction with, an instruction execution system, apparatus, or device. More specific examples (a non-exhaustive list) of computer-readable media include: an electrical connection having one or more wires (electronic device), a portable computer disk drive (magnetic device), random access memory (RAM), read-only memory (ROM), erasable and editable read-only memory (EPROM or flash memory), fiber optic devices, and portable optical disc read-only memory (CDROM). Alternatively, the computer-readable medium may be paper or other suitable media on which the program can be printed, since the program can be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, interpreting, or otherwise processing as necessary, and then stored in a computer memory.

[0152] It should be understood that various parts of the present invention can be implemented in hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented in software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, it can be implemented using any one or a combination of the following techniques known in the art: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.

[0153] Those skilled in the art will understand that all or part of the steps of the methods in the above embodiments can be implemented by a program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, the program includes one or a combination of the steps of the method embodiments.

[0154] The storage medium mentioned above can be a read-only memory, a disk, or an optical disk, etc. Although embodiments of the present invention have been shown and described above, it is to be understood that the above embodiments are exemplary and should not be construed as limiting the present invention. Those skilled in the art can make changes, modifications, substitutions, and variations to the above embodiments within the scope of the present invention.

Claims

1. A cross-protocol task collaboration method for multi-source heterogeneous robot swarms, characterized in that, Includes the following steps: The communication messages of each robot in the cluster are passively monitored in the communication network. Protocol feature information is extracted from the communication messages to generate a protocol feature fingerprint for each robot. The protocol feature fingerprint is matched with a pre-set protocol fingerprint database. When the match is successful, a protocol adapter matching the protocol feature fingerprint is loaded for the corresponding robot. When the match fails, the corresponding robot is actively probed, and the corresponding protocol adapter is generated based on the probe results. The protocol adapter parses and maps the native communication messages of each robot into standardized collaborative messages carrying collaborative semantic information, and publishes the standardized collaborative messages to the unified message bus. Based on the standardized collaborative messages in the unified message bus, task decomposition, capability matching, and timing constraint determination are performed to generate a collaborative execution plan. The collaborative execution plan is then converted into the native protocol instructions of the corresponding robot by each protocol adapter and then sent out for execution.

2. The method according to claim 1, characterized in that, The protocol feature fingerprint includes message header features, field semantic features, and timing features; The message header features include the sequence of magic number bytes, the position of the version number, and the position of the message length field. The field semantic features include the positions of the command field, the address field, and the data payload field. The timing features include the request-response interval distribution and the message sending period.

3. The method according to claim 1, characterized in that, The standardized collaborative message includes task semantic fields, spatiotemporal attribute fields, collaborative relationship fields, and priority fields; The task semantic field is used to characterize the production task information associated with the standardized collaborative message; the spatiotemporal attribute field is used to characterize the position coordinates, time window and motion constraints; the collaborative relationship field is used to characterize the collaborative type, follow-up robot and synchronization point; and the priority field is used to characterize the scheduling priority of the standardized collaborative message in the unified message bus.

4. The method according to claim 3, characterized in that, The step of decomposing tasks, matching capabilities, and determining timing constraints based on the standardized collaborative messages in the unified message bus to generate a collaborative execution plan includes: Extract the capability parameters and current status of each robot from the standardized collaborative message; Decompose production tasks into a set of subtasks; Calculate the capability matching degree between each subtask and each robot, and assign each subtask to the corresponding robot according to the capability matching degree; The execution sequence and synchronization point of each subtask are determined based on the collaboration relationship field; the collaborative execution scheme containing the collaborative action sequence of each robot is generated based on the execution sequence and the synchronization point.

5. The method according to claim 3, characterized in that, The collaborative execution scheme includes synchronization points; After the first robot completes the collaborative action corresponding to the synchronization point, it sends a synchronization notification to the subsequent robot represented by the collaborative relationship field. After receiving the synchronization notification, the rear robot begins to execute the corresponding collaborative action.

6. The method according to claim 1, characterized in that, The step of matching the protocol feature fingerprint with a pre-set protocol fingerprint database includes: Calculate the cosine similarity between the protocol feature fingerprint and the preset protocol template in the protocol fingerprint database. If the cosine similarity is greater than or equal to a preset threshold, the match is considered successful. If the cosine similarity is less than the preset threshold, the match is considered unsuccessful.

7. The method according to claim 6, characterized in that, The step of actively probing the corresponding robot and generating the corresponding protocol adapter based on the probing results includes: Send a preset sequence of detection messages to the corresponding robot; Collect the response messages returned by the robot in response to the sequence of probe messages; Based on the response message, the protocol state transition relationship of the robot is inferred, and a protocol state machine is constructed; The protocol state machine automatically generates message parsing rules and instruction encoding rules; The protocol adapter is generated by assembling the protocol parsing rules and the instruction encoding rules, and the corresponding protocol feature fingerprint is added to the protocol fingerprint database.

8. The method according to claim 1, characterized in that, The unified message bus adopts a publish-subscribe mechanism based on cooperative relationships, and is divided into serial cooperative topics, parallel cooperative topics, and master-slave cooperative topics according to cooperative type; Each protocol adapter publishes the standardized collaboration message to the topic corresponding to its collaboration type. The collaboration decision-making end subscribes to the topic to obtain the standardized collaboration message and performs queue scheduling according to the priority field.

9. The method according to claim 8, characterized in that, The protocol adapters are cached to form an adapter pool; When multiple protocol adapters need to perform message parsing and message mapping operations, they are executed in parallel using a pipelined approach. When a new protocol adapter is generated, existing protocol adapters with the same or similar functions in the adapter pool are reused first.

10. The method according to claim 1, characterized in that, The protocol fingerprint database and the protocol adapter are deployed on a cloud server, which is shared by multiple factories; when a new protocol adapter is generated, the new protocol adapter is automatically synchronized to each of the factories.

11. The method according to any one of claims 1 to 10, characterized in that, Before passively monitoring the communication messages of each robot in the cluster in the communication network, the method further includes: The robot's communication terminal is authenticated with a digital certificate; the robot's declared capability parameters are compared and verified with the actual capability parameters obtained through active detection; only when both authentication and capability comparison verification are passed is passive listening and active detection of the robot permitted.

12. The method according to any one of claims 1 to 10, characterized in that, The method further includes: Receive feedback messages returned by each of the robots after executing the cooperative execution scheme; When the feedback message indicates an execution error, the environmental constraint information corresponding to the error is appended to the environmental constraints of the current task. The collaborative execution scheme is regenerated based on the updated environmental constraints, and the updated collaborative execution scheme is then deployed for execution. Update the capability profile of the corresponding robot based on the feedback message.

13. A cross-protocol task collaboration system for multi-source heterogeneous robot swarms, characterized in that, The system, applied to the method of any one of claims 1 to 12, comprises: The protocol sniffing module is used to passively listen to the communication packets of each robot in the cluster in the communication network, extract protocol feature information from the communication packets, and generate a protocol feature fingerprint corresponding to each robot. The adapter management module is used to match the protocol feature fingerprint with a preset protocol fingerprint library. When the match is successful, it loads a protocol adapter that matches the protocol feature fingerprint for the corresponding robot. When the match fails, it actively probes the corresponding robot and generates the corresponding protocol adapter based on the probe results. The message bus module is used to parse and map the native communication messages of each robot into standardized collaborative messages carrying collaborative semantic information through each of the protocol adapters, and to uniformly publish and subscribe to the standardized collaborative messages. The collaborative decision-making module is used to decompose tasks, match capabilities, and determine timing constraints based on the standardized collaborative messages, generate a collaborative execution plan, and then convert the collaborative execution plan into the native protocol instructions of the corresponding robot through each of the protocol adapters before issuing it for execution.

14. The system according to claim 13, characterized in that, The system also includes a feedback optimization module, which is used to receive feedback messages returned by each robot after executing the collaborative execution scheme; When the feedback message indicates an execution error, the environmental constraint information corresponding to the error is added to the environmental constraints of the current task, and the collaborative decision-making module is triggered to regenerate the collaborative execution plan and update the corresponding robot's capability profile according to the feedback message.

15. The system according to claim 13, characterized in that, The system also includes a security authentication module, which is used to perform digital certificate authentication on the robot's communication terminal before passive eavesdropping, and to compare and verify the robot's declared capability parameters with the actual capability parameters obtained through active detection.

16. A computer device, characterized in that, The method includes a processor and a memory, the processor and the memory being coupled together, the memory storing program instructions, which, when executed by the processor, implement the method of any one of claims 1 to 12.

17. A computer-readable storage medium, characterized in that, It stores a computer program thereon, which, when executed by a processor, implements the method of any one of claims 1 to 12.

18. A computer program product, characterized in that, The computer program product includes a computer program that, when run on a computer, causes the computer to perform the method of any one of claims 1 to 12.

Images