Industrial robot debugging method based on combination of natural language and computer vision

Abstract

The invention discloses an industrial robot debugging method based on combination of natural language and computer vision. According to the method, semantic information is generated from natural language description through a word2vec network and a linear layer. And industrial robot environment features are extracted through a three-dimensional cyclic convolutional neural network. And inputting the environment features into a long short-term memory network to generate an intermediate context, and obtaining API recommendation through a recurrent neural network (RNN) by taking the intermediate context as the input of a GRU network. A text generated by natural language description through a word2vec network is embedded and input into a long short-term memory network encoder and a long short-term memory network decoder, an AST construction action sequence is output, and API recommendation and the construction action sequence are combined to generate an industrial robot debugging code. And adding the debugging code into a robot program editor to complete debugging. The development efficiency of robot debugging is effectively improved, and the deployment time of a robot production line in an industrial environment is shortened.

Description

technical field [0001] The invention belongs to the technical field of industrial robots, in particular to an industrial robot debugging method based on the combination of natural language and computer vision. Background technique [0002] In recent years, with the country's advocacy of smart factories, the manufacturing industry has begun to use robot technology on a large scale to assist production, and the concept of intelligent manufacturing has entered the stage of comprehensive promotion from popularization. As a major component of smart factories, robots can help improve factory productivity, perform operations that workers cannot, and be able to quickly adapt to new production needs. [0003] At present, the debugging methods of industrial robots on the market can be divided into online debugging and offline debugging. Online debugging requires the user to control the robot to complete the specified actions and save them. By running these specified actions, the opera...

AI Technical Summary

Problems solved by technology

However, the robot debugging method based on the current deep learning model has shortcomings such as poor robustness of the task code, and the debugging results cannot be applied to the field environment.

The debugging method based on this method cannot be applied to the production environment of the factory

Images