The application discloses a kind of
robot simulation scene and trajectory automatic generation method and
system, receive
natural language operation instruction, and through semantic analysis, visual asset generation, three-dimensional reconstruction and physical credible
layout generation three-
dimensional simulation scene that can be used for
robot operation training;Utilize video generation model to synthesize the continuous
image sequence of target operation task, and extract two-dimensional
visual motion trajectory in target object area;By depth
estimation, three-dimensional motion fusion, grasping posture generation and
inverse kinematics mapping, two-dimensional
visual motion trajectory is converted into
robot end
effector trajectory and corresponding robot joint trajectory, the
simulation scene and expert trajectory generated are paired, form training
data set for robot
imitation learning strategy training, and support the strategy obtained by training is deployed to
simulation environment or real
robot hardware.The application can reduce the cost of robot training
data acquisition and artificial trajectory design, improve the degree of
automation of simulation scene and operation trajectory generation.