The present application relates to a kind of flexible cloth manipulation methods based on key point detection, comprising the following steps: step 1, construct dual-arm
robot workspace, the
workspace includes top depth camera, the cloth state and surrounding environment in the manipulation area are captured by the real-time capture of visual data by the depth camera;Step 2, define multiple action primitives;Step 3, decode multiple action primitives using VIT-
Transformer, generate operation strategy, output the action that needs to be executed;Step 4, key point detection and identification are carried out to cloth by Swin-
Transformer, select operation strategy, generate action instruction;Step 5, action instruction is sent to dual-arm
robot, and action instruction is executed by dual-arm
robot.Compared with prior art, the present application can improve operation efficiency, improve adaptability, improve accuracy, reduce implementation cost and the advantages such as wide application.