Triad acquisition method and device based on video learning and text learning

Abstract

The invention provides a triple acquisition method and device based on video learning and text learning, a co-fusion robot visual programming and online cooperation system and a robot, and the methodcomprises the steps: firstly learning a long video to acquire a main body object, an acceptor object and an action type corresponding to each video segment, and forming a corresponding operation instruction triple; training a natural language model by utilizing the text triple positive and negative sample data set, inputting text information formed by the operation instruction triple into the natural language model to obtain a prediction triple, and correcting the operation instruction triple output by the long video through the prediction triple so as to realize multi-modal fusion; moreover,the accuracy of the target triad obtained through video learning and text learning is relatively high, the accuracy of identifying the long video operation instruction by the robot can be effectivelyimproved, better autonomous learning of the robot is facilitated, and the learning efficiency is improved.

Description

technical field [0001] The present invention relates to the technical field of artificial intelligence, and in particular to a triplet acquisition method and device based on video learning and text learning, a visual programming and online collaboration system for a fusion robot, and a robot. Background technique [0002] With the continuous development of robot technology, the autonomous learning ability of robots is also constantly improving; for example, in the prior art, long videos are usually used to learn operation instruction triplets, and the robot divides the input long videos into video segments according to actions, Then identify the position and type of the main object and the recipient object in each video segment, as well as the action between the main object and the recipient object, thus forming a triplet of operation instructions, and then the robot executes the corresponding operation instructions to complete autonomous learning process, and can make diffe...

AI Technical Summary

Problems solved by technology

[0004] The purpose of the present invention is to at least solve one of the above-mentioned technical defects, especially in the prior art, the robot cannot judge the connection between objects and the connection between actions and objects, thus limiting the autonomous learning ability of the robot, making the robot perform The technical defect of lower accuracy and efficiency of action

Images