The invention discloses a mechanical arm
natural language instruction
control system and method based on a large
language model, and belongs to the field of intelligent manufacturing. Aiming at the limitation that traditional mechanical arm control depends on pre-
programming and a static rule
library, a
dynamic mapping mode from a
natural language instruction to an atomic action sequence is designed, an atomic skill
library including detection, grabbing, moving, placement and other operations is constructed, and semantic analysis and a multi-mode cooperation technology are combined, so that the atomic action sequence is obtained. And support is provided for man-
machine cooperation of a flexible
assembly task. The method specifically comprises the steps that a DeepSeek-Distil-Llam-8B
large model and a LoRA
fine tuning technology are adopted, and a
natural language instruction is converted into an
executable atomic action sequence; based on a transfer learning optimized YOLOv8 target detection technology and a
binocular vision positioning technology, a sensing module adaptive to an
assembly scene is constructed and is fused with a mechanical arm
motion planning module, and positioning grabbing of parts and tools is achieved. And an interactive interface is built by combining a voice-to-text
large model and a Gradio front-end framework, so that the convenience of man-
machine interaction is improved. By optimizing
large model reasoning and
motion planning cooperation efficiency,
response delay from instructions to execution is reduced, and an efficient and extensible solution is provided for man-
machine cooperation in intelligent manufacturing.