Multi-modal human-robot collaborative job scheduling and control method for flexible manufacturing

Abstract

The application belongs to the technical field of industrial automation and intelligent manufacturing, and particularly relates to a multi-modal man-machine collaborative operation scheduling and control method for flexible manufacturing, which comprises the following steps: a sensing step, in which a plurality of source heterogeneous sensors are integrated through an IoT sensing layer, data synchronization is realized based on a PTP-1588 precision clock protocol, and multi-modal features are extracted by an edge node and uploaded to the cloud after dimension reduction; a cognition step, in which multi-modal features are dynamically fused through a MM-AFNet model via a gated attention mechanism, and a Neo4j knowledge graph correlation reasoning is combined to output a device health degree evaluation. The multi-modal man-machine collaborative operation scheduling and control method for flexible manufacturing realizes complete connection from multi-source data acquisition, edge real-time processing, cloud multi-modal fusion to augmented reality guidance by constructing a four-layer closed-loop architecture of an IoT sensing layer, an edge layer, a cloud AI layer and a man-machine collaborative layer, and effectively solves the problems of traditional monitoring response lag and low man-machine collaborative efficiency.