Natural Language-Based Auxiliary Programming Method for Industrial Robots

Abstract

The invention provides an auxiliary programming method for industrial robots based on natural language, which generates corresponding robot execution codes according to language instructions and environment images. The present invention is divided into three parts: 1) using bidirectional recurrent neural network (Bi-RNN) with long short-term memory (LSTM) and fast regional convolutional neural network (F-RCNN) to extract language instructions and characteristics of factory environment respectively. 2) Propose a "multi-attention mechanism" model and machine translation alignment algorithm to correctly match objects in the environment with instructions, thereby identifying the specified object and outputting the coordinate point where the object is placed. 3) Use the output results of the above model and the CoBlox modular programming method to generate the robot code for the operation. The "multi-attention mechanism" model adopted by the invention improves the recognition accuracy and solves the problem that the current method cannot accurately recognize objects in the industrial environment. The modular programming technology solution simplifies the programming complexity of engineers and effectively improves development efficiency.

Description

technical field [0001] The application belongs to the technical field of robot programming, and in particular relates to the robot programming technology based on natural language and machine vision. Background technique [0002] With the rapid development of robot technology in recent decades, the concept of intelligent manufacturing has been deeply rooted in the hearts of the people. Robot arm technology has been widely used in industrial production environments. Collaborative robots combine the advantages of humans and mechanical equipment, and work closely with workers on the production line, which can significantly improve production efficiency. [0003] At present, all mechanical tasks must be carefully designed and coded by engineers in order to assist and replace workers to perform a single mechanical task. Engineers usually use online or offline programming methods to write robot codes. However, these programming methods are too time-consuming, and the timeliness i...

AI Technical Summary

Problems solved by technology

However, such methods only output the behavior and state of the robot, and do not provide the source code needed by industrial engineers

This programming method cannot be modified offline at the code level when the scheme needs to be adjusted in industrial production

Because no code text is generated, it is not conducive to the reuse of similar code in other projects

[0007] In this case, the existing programming technology cannot meet the needs of industrial intelligent manufacturing due to its inherent defects

Images