Three-dimensional convolutional neural network-based workflow identification method

Abstract

The invention discloses a three-dimensional convolutional neural network-based workflow identification method. Only different process tasks are divided in advance and different action behaviors are tagged artificially in a video analysis process, so that the automation demand of intelligent manufacturing is not met. An inter-frame difference method with an adaptive threshold is proposed firstly and is mainly used for segmenting out a region of a moving object from a complex background, so that the time complexity of subsequent feature extraction and model training is lowered; secondly, a 3D convolutional neural network is improved to fully adapt to a factory environment with a plurality of monitoring devices, and for different views, a view pooling layer is adopted for fusing views of different angles according to weights; and finally, a new action division method is proposed for performing automatic division on continuous production actions in a video, so that an automated workflow identification process is realized.

Description

technical field [0001] The invention belongs to the technical field of workflow identification and is used for fast and accurate identification and detection of production and manufacturing processes. Through the camera installed in the manufacturing workshop, the entire process of production scheduling on the production line is photographed, and then the video is calculated and processed, so as to protect the personal safety of employees, reduce production costs, ensure product quality, and optimize production scheduling and process specifications. play an important role Background technique [0002] Intelligent manufacturing is the further development direction of manufacturing automation. It widely applies artificial intelligence technology to engineering design, process design, production scheduling, fault diagnosis and other links of industrial manufacturing process, so as to realize intelligent manufacturing process and greatly improve productivity. . As an important...

AI Technical Summary

Problems solved by technology

[0003] However, workflow identification technology has its complexity and particularity

First of all, because there are many kinds of machines, transport vehicles, auxiliary equipment and other objects in the production workshop, they are often blocked by each other, and the similarity of different process operations, frequent changes in light intensity in the workshop, all these make video and image analysis and Recognition presents challenges

In addition, the dynamic production workflow process makes the recognition process quite complicated, and it is easy to produce deviations: for example, different tasks in the workflow often have different execution times, and there is no clear definition between the start and end of tasks; these Tasks may even contain both human and machine actions, some of which have nothing to do with workflow and must be distinguished from real production tasks

These aspects make it difficult for traditional action / posture recognition methods that rely on object detection and tracking to be applied in complex factory manufacturing environments.

In addition, although some researchers have carried out some research on workflow recognition technology, how to automatically divide the production process / action of the image sequence in the video, these research works have not given a clear definition, most of them are only Divide different process tasks in advance and artificially label different action behaviors in the process of analyzing video, which obviously does not meet the automation requirements of intelligent manufacturing

Images