Convolution operation device, system and image processing device for convolutional neural network

Abstract

The present application discloses a convolution operation device for a convolutional neural network, including an enabling matrix acquisition module, an enabling matrix controller, a matrix to be convoluted controller, a convolution kernel matrix controller, and a convolution operation module. The enabling matrix acquisition module sets the non-zero element values ​​of the matrix X to be convoluted and the convolution kernel matrix W to the first preset value, so as to obtain the enabling matrix X' to be convolved and the convolution kernel enabling matrix W '. The enabling matrix controller, the to-be-convolved matrix controller, and the convolution kernel matrix controller sequentially send the elements of the to-be-convolved matrix X and the convolution kernel matrix W to the convolution operation module. The convolution operation module only treats the convolution matrix X and the convolution kernel matrix W corresponding The elements of are multiplied. Since only non-zero elements are multiplied during the convolution operation, the hardware load is reduced and the convolution operation speed is improved.

Description

technical field [0001] The present invention relates to the technical field of image processing, in particular to a convolution operation device, system and image processing device for convolutional neural networks. Background technique [0002] With the increasing demand for artificial intelligence solutions based on neural networks, some mobile platforms such as drones and robots are built with convolutional neural networks, which are profoundly changing human production and lifestyle. Corresponding to the design and research of dedicated hardware for convolutional neural networks, there are already implementation methods based on new devices such as CPU, GPU, FPGA, ASIC and RRAM. From the cloud to the edge device, different application scenarios put forward different requirements for the computing power of the convolutional neural network. In addition, the structure of the convolutional neural network is diverse, the amount of data is large, and the amount of calculation ...

AI Technical Summary

Problems solved by technology

However, this powerful algorithm encounters great obstacles in the process of moving towards practical applications. This is because many real-world applications where CNN can be useful are implemented on edge devices with few computing resources, such as AI-enabled devices. Smartphones and smart security cameras, and today’s most famous CNN network, ResNet-50, is composed of 50 layers of basic structures stacked, and each layer of basic structures contains a large number of convolution operations. If such a computationally intensive algorithm is not modified or It is difficult to meet the real-time requirements of many applications without designing special acceleration hardware for its characteristics and directly implementing it on the CPU of the edge device

Images