Robot control method and device based on normal VLA model

Abstract

The invention provides a robot control method and device based on a normal VLA model, and the method comprises the steps: obtaining a training data set which comprises a human activity video, robot historical data and simulation generation data; a normal form VLA model is constructed, the normal form VLA model comprises a cognitive-dynamic world normal form, the cognitive-dynamic world normal form comprises a plurality of cognitive nodes and relation edges representing topology, space or functional dependence among the cognitive nodes, and attributes of each cognitive node comprise space-time poses, semantic anchor points, dynamic intrinsic and functional ontologies; the training data set is input into the normal form VLA model for training, so that a pre-trained robot control model is obtained, and in the training process, action loss and normal form cognitive loss are adopted to conduct parameter updating on the normal form VLA model; deploying the pre-trained robot control model to the robot to perform real-time decision making and control; therefore, the control effect on the robot is improved.

Images