Convolutional neural network processor, implementation method, electronic device and storage medium

Abstract

The present application proposes a convolutional neural network processor, implementation method, electronic device and storage medium. The convolutional neural network processor is connected to the main processor via a bus protocol and includes: an instruction module for receiving and parsing instructions issued by the main processor; the content of the instruction includes at least data related to the image to be processed, data related to the convolutional neural network and data related to the feature map; a buffer module for storing data related to the convolutional neural network operation; a calculation module for implementing the convolutional neural network operation; a control module for controlling data writing and reading from the buffer module according to corresponding instructions; and controlling the calculation module to execute the relevant content of the instruction according to the instruction, and performing the corresponding convolutional neural network operation on the corresponding image to be processed. The present application can perform corresponding convolutional neural network operations according to the instructions sent by the main processor, can realize accelerated calculation of convolutional neural network operations, has versatility and good flexibility.

Description

Technical Field

[0001] The present application belongs to the technical field of convolutional neural networks, and specifically relates to a convolutional neural network processor, an implementation method, an electronic device, and a storage medium. Background Art

[0002] Convolutional Neural Networks (CNNs) are a classic and widely used neural network architecture for deep learning. Their local connections, weight sharing, and pooling operations effectively reduce network complexity and the number of training parameters, making the model somewhat invariant to translation, distortion, and scaling. They also possess strong robustness and fault tolerance, and are easy to train and optimize.

[0003] Because convolutional neural networks (CNNs) are computationally complex and process large amounts of data, most current CNN models suffer from computational latency. To increase the processing speed of CNNs, they are typically offloaded from the CPU and accelerated through heterogeneous processing. Currently, the main heterogeneous processors used include GPUs, FPGAs, and ASICs. However, GPUs can only pipeline instructions, not data, and consume excessively high power. ASICs, on the other hand, often only support specific CNN operations and have long development cycles. In today's era of rapidly evolving neural network algorithms, algorithms are often outdated by the time they're taped out. Summary of the Invention

[0004] The present application proposes a convolutional neural network processor, implementation method, electronic device and storage medium, which can perform corresponding convolutional neural network operations according to the instructions sent by the main processor, can realize the accelerated calculation of convolutional neural network operations, and has versatility and good flexibility.

[0005] The first embodiment of the present application provides a convolutional neural network processor, which is connected to a main processor via a bus protocol, including:

[0006] An instruction module, configured to receive and parse instructions issued by the main processor; the contents of the instructions at least include data related to the image to be processed, data related to the convolutional neural network, and data related to the feature map;

[0007] A first cache module is used to store quantization parameters and weight parameters of the convolutional neural network;

[0008] A second cache module is used to cache the output result data of each layer of the convolutional neural network and notify the control module to read the output result data;

[0009] A third cache module is used to cache the intermediate calculation result data of the convolutional neural network and perform quantization operations, data bit width compression and function activation on the intermediate calculation result data;

[0010] Computing module, used to implement convolutional neural network operations;

[0011] A control module is used to control the quantization parameters and weight parameters to be written into or read out of the first cache module according to corresponding instructions, control the intermediate calculation result data to be written into or read out of the third cache module, and control the calculation module to execute the relevant content of the instruction according to the instruction to perform the corresponding convolutional neural network operation on the corresponding image to be processed.

[0012] In some embodiments of the present application, the instruction module is further used to store the instructions in an instruction queue, and the instructions include configuration instructions for one or more convolutional neural networks.

[0013] In some embodiments of the present application, the first buffer module includes a quantization buffer and a weight buffer;

[0014] The quantization buffer stores the quantization parameters of the convolutional neural network in an on-chip static random access manner;

[0015] The weight buffer uses on-chip static random access to store the weight parameters of the convolutional neural network.

[0016] In some embodiments of the present application, the weight buffer is connected to an external memory, and the weight buffer reads weight data from the external memory in batches according to the size of the weight, and reads a specified amount of weight data each time for cache.

[0017] In some embodiments of the present application, the second buffer module includes a feature map buffer and an output level buffer;

[0018] The output level buffer is used to cache the output result data of each layer of convolutional neural network operation;

[0019] The feature map buffer is used to buffer the output result data returned from the output stage buffer and to notify the control module to read the output result data.

[0020] In some embodiments of the present application, the feature map buffer is also connected to an external memory. If the space occupied by the newly generated output result data is larger than the remaining storage space of the feature map buffer, the feature map buffer transfers the newly generated output result data to the external memory.

[0021] In some embodiments of the present application, the third cache module includes an intermediate level cache and a quantization-compression-activator;

[0022] The intermediate level buffer is used to cache the intermediate calculation result data during the convolutional neural network operation process;

[0023] The quantization-compression-activator is used to receive the intermediate calculation result data transmitted by the intermediate level buffer, and perform quantization operation, data bit width compression and function activation on the intermediate calculation result data.

[0024] In some embodiments of the present application, the computing module includes: a convolution operator, a normalization operator, a pooling operator, an upsampling operator and a fully connected operator, which are respectively used to implement convolution operation, normalization operation, pooling operation, upsampling operation and fully connected operation.

[0025] In some embodiments of the present application, the processor is implemented based on FPGA.

[0026] An embodiment of a second aspect of the present application provides a method for implementing a convolutional neural network, based on the convolutional neural network processor described in the first aspect, the method comprising:

[0027] Receive and parse instructions issued by the main processor; the content of the instructions at least includes data related to the image to be processed, data related to the convolutional neural network, and data related to the feature map;

[0028] Establishing a convolutional neural network model according to the convolutional neural network related data;

[0029] Through the convolutional neural network model, the relevant content of the instruction is executed, and the corresponding convolutional neural network operation is performed on the corresponding image to be processed.

[0030] An embodiment of the third aspect of the present application provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the method described in the second aspect.

[0031] An embodiment of the fourth aspect of the present application provides a computer-readable storage medium having a computer program stored thereon, wherein the program is executed by a processor to implement the method described in the second aspect.

[0032] The technical solutions provided in the embodiments of this application have at least the following technical effects or advantages:

[0033] The convolutional neural network processor provided in the embodiment of the present application is connected to the main processor through a bus, can receive instructions issued by the main processor, and after parsing the instructions, can establish a convolutional neural network model based on the parsed convolutional neural network related data, and execute the relevant content of the instructions through the convolutional neural network model, and perform corresponding convolutional neural network operations on the corresponding image to be processed, thereby realizing accelerated calculation of convolutional neural network operations. The processor has high versatility and flexibility and can support multiple convolutional neural network models. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] Various other advantages and benefits will become apparent to those skilled in the art upon reading the detailed description of the preferred embodiment below. The accompanying drawings are for illustration purposes only and are not to be considered as limiting the present application. Throughout the accompanying drawings, the same reference numerals are used to denote the same components.

[0035] In the attached figure:

[0036] Figure 1 A block diagram of a convolutional neural network processor proposed in an embodiment of the present application is shown;

[0037] Figure 2 A block diagram of another convolutional neural network processor proposed in an embodiment of the present application is shown;

[0038] Figure 3 A flow chart of a convolutional neural network implementation method proposed in an embodiment of the present application is shown;

[0039] Figure 4 A schematic structural diagram of an electronic device provided in one embodiment of the present application is shown;

[0040] Figure 5 A schematic diagram of a storage medium provided in an embodiment of the present application is shown. DETAILED DESCRIPTION

[0041] The following describes exemplary embodiments of the present application in more detail with reference to the accompanying drawings. Although exemplary embodiments of the present application are shown in the accompanying drawings, it should be understood that the present application can be implemented in various forms and should not be limited by the embodiments described herein. Instead, these embodiments are provided to enable a more thorough understanding of the present application and to fully convey the scope of the present application to those skilled in the art.

[0042] It should be noted that, unless otherwise specified, the technical or scientific terms used in this application should have the common meanings understood by those skilled in the art to which this application belongs.

[0043] The following describes a convolutional neural network processor, implementation method, electronic device, and storage medium proposed according to an embodiment of the present application in conjunction with the accompanying drawings.

[0044] In light of the above issues, this embodiment studies the operation of convolutional neural networks on existing processors. The results show that FPGAs offer a promising alternative to GPUs, which can only pipeline instructions but not data, and consume excessive power, and to ASICs, which can only support specific convolutional neural network operations and have a long development cycle. As a programmable hardware, FPGAs offer advantages over GPUs in terms of both instruction and data pipelining and low power consumption, and over ASICs in terms of the ability to rapidly adapt hardware implementations to algorithmic changes.

[0045] Based on the above research, this embodiment proposes a convolutional neural network processor and a method for implementing convolutional neural network operations on the convolutional neural network processor. The convolutional neural network processor is oriented to the system-on-a-chip (SoC), can be connected to the main processor (CPU) through a bus protocol, receives instructions issued by the main processor, and after parsing the instructions, can execute the relevant content of the instructions according to the instructions, and perform corresponding convolutional neural network operations on the corresponding image to be processed, thereby realizing accelerated calculation of the convolutional neural network. The convolutional neural network processor has high versatility and flexibility, and can support multiple convolutional neural network models.

[0046] The main processor, or central processing unit (CPU), is the computing and control core of a computer system and the final execution unit for information processing and program execution. Bus protocols include, but are not limited to, common bus protocols such as AXI, PCI-E, SATA, and USB. As long as the bus protocol can be used to connect the convolutional neural network processor to the main processor, it can be used.

[0047] Please refer to Figure 1 The convolutional neural network processor provided in this embodiment includes an instruction module, a buffer module, a computing module, and a control module. The instruction module is used to receive and parse instructions issued by a main processor; the computing module is used to implement convolutional neural network operations; and the control module is used to control the computing module to execute the relevant content of the instructions according to the instructions, performing the corresponding convolutional neural network operations on the corresponding image to be processed. The buffer module is used to store data related to the convolutional neural network operations, such as specific data corresponding to the above-mentioned instruction content. The control module can control data writing and reading from the buffer module and relocating storage addresses according to the corresponding instructions to implement data storage and transmission during the convolutional neural network operation process.

[0048] Among them, the content of the instruction includes at least data related to the image to be processed, data related to the convolutional neural network and data related to the feature map, such as the input image size, the type of operator to be operated, the number of input channels, the number of output channels, the operation step size, the starting reading address of the input feature map, the storage address of the output feature map, the reading address of the weight data, the reading address of the quantization data, and other required related control instructions.

[0049] In some implementations of this embodiment, the convolutional neural network model on the main processor generates corresponding instructions through a compiler, and the main processor then configures these instructions to the convolutional neural network processor via a bus. After receiving the instructions issued by the main processor, the instruction module of the convolutional neural network processor stores these instructions in an instruction queue, so that the instruction module can parse the received instructions in a first-in-first-out order, and the subsequent computing module can perform the corresponding convolutional neural network operations based on the content of the instructions.

[0050] In this embodiment, the convolutional neural network processor can simultaneously support multiple convolutional neural network operations. Accordingly, the instruction module not only supports the instruction queue of a certain convolutional neural network configuration, but also supports the instruction queue of multiple convolutional neural network configurations. Accordingly, the above instructions can also include one or more convolutional neural network configuration instructions. The main processor can freely switch which convolutional neural network configuration is used for the convolutional neural network operation. The number of instruction queues specifically supported by the instruction module can be set according to the storage space of the module, and this embodiment does not specifically limit it.

[0051] In other implementations of this embodiment, the control module is responsible for reading the content of each instruction from the instruction queue and controlling the computing module to execute the instruction according to the content of the corresponding instruction. The content of the instructions for the control module may include data access, data movement, neural network operations, etc. Among them, data access refers to controlling the reading and writing of data from various buffers. Data movement refers to moving data from one address in the cache to another address. Neural network operations refer to performing convolution operations, pooling operations, upsampling operations, full connection operations, and normalization function operations, etc.

[0052] In a single instruction, the control module may repeatedly fetch data and call the computation module to perform neural network operations. After the computation module completes the operation, it returns the corresponding result to the control module, which then temporarily stores it in the cache module described below.

[0053] In other implementations of this embodiment, the computing module may include but is not limited to a convolution operator, a normalization operator, a pooling operator, an upsampling operator and a fully connected operator, which are used to implement convolution operations, normalization operations, pooling operations, upsampling operations and fully connected operations respectively.

[0054] The buffer module may include multiple sub-modules, which are used to cache data related to the image to be processed (such as specific data of the image to be processed of various sizes), data related to the convolutional neural network model (that is, structural parameters forming the convolutional neural network model, such as convolutional layers, convolution blocks, pooling layers, fully connected layers, etc. of various sizes), and data related to the feature map generated during the convolutional neural network operation of the image to be processed (such as the calculation results of each layer of the convolutional neural network operation).

[0055] Specifically, the cache module may include a first cache module, a second cache module and a third cache module, wherein the first cache module is used to store the quantization parameters and weight parameters of the convolutional neural network; the second cache module is used to cache the output result data of each layer of the convolutional neural network, and notify the control module to read the output result data; the third cache module is used to cache the intermediate calculation result data of the convolutional neural network, and perform quantization operations, data bit width compression and function activation on the intermediate calculation result data.

[0056] Furthermore, the first buffer module may include a quantization buffer and a weight buffer. The quantization buffer is used to store quantization parameters of the convolutional neural network. These quantization parameters are used for data quantization operations and can be read by the control module and sent to the quantization buffer for quantization operations and storage. Because the quantization parameters typically have a small amount of data and require less storage space, they can be configured once by the main processor via the bus during initialization and stored using on-chip static random access memory to increase the speed at which other modules can obtain the quantization parameters.

[0057] The weight buffer is used to store the weight data of the convolutional neural network. The weight parameters of the convolutional neural network can be stored in an on-chip static random access manner to increase the speed at which other modules obtain the quantization parameters.

[0058] Furthermore, since the amount of data of the weight parameters is often large, even when the convolutional neural network has been processed with weight sharing, the storage space required for the weights is still a bottleneck of the design. Therefore, in this embodiment, the weight buffer is also connected to the external memory. The weight buffer can read the weight data from the external memory in batches according to the size of the weights, and each time reads a specified number of weight data for caching, which is then read by the control module and sent to the computing module for calculation. In this way, combined with the on-chip static random access storage method, the weight data is close to the computing resources, which can well solve the bottleneck problem of other modules accessing the weight memory. Among them, the specific specified number can be specifically limited according to the storage space of the weight buffer. The external memory can be, but is not limited to, a double-rate synchronous dynamic random access memory to ensure the speed of data access and transmission.

[0059] In other implementations of this embodiment, the second cache module may include a feature map cache and an output-level cache. The output-level cache is used to cache the output result data of each layer of the convolutional neural network operation. It can also be stored in an on-chip static random access memory and stored in a first-in-first-out queue to save the latest data. The output-level cache can also be used to temporarily store data sent from other caches. When the feature map cache has available storage space, the data is sent to the feature map cache in the order of the first-in-first-out queue.

[0060] The feature map buffer is used to cache the output result data returned from the output stage buffer, and is also stored in an on-chip static random access manner. After storage, the control module is notified to read the output result data.

[0061] The convolutional neural network's input data, such as image data and sound data, is transmitted from the main processor via a bus. After the image buffer module receives the input data, it notifies the control module to read the data. The control module then sends the input data to the computation module, which then controls the computation module to perform the convolutional neural network operations. The calculation results are then transmitted back to the feature map buffer via the output-level buffer.

[0062] Furthermore, the feature map buffer is also connected to the external memory. If the space occupied by the newly generated output result data is larger than the remaining storage space of the feature map buffer, the feature map buffer transmits the newly generated output result data to the external memory.

[0063] like Figure 2 As shown (the arrows in the figure indicate the direction of data flow transmission), the feature map buffer may include 3 input ports and 3 output ports. The 3 input ports are connected to the main processor, external memory and output level buffer respectively. The 3 output ports are connected to the main processor, external memory and control module respectively. If the amount of data of the calculation result is greater than the storage space of the feature map buffer, the feature map buffer will transfer the data to the external memory for storage. When the next layer of convolutional neural network processing is performed, the feature map buffer will import the data from the external memory in batches and process it in batches. When all processing is completed, the final result is generally read from the feature map buffer by the main processor through the bus.

[0064] In some other implementations of this embodiment, the third cache module may include an intermediate cache and a quantization-compression-activator. The intermediate cache is used to cache intermediate calculation result data during the convolutional neural network operation process, and can also be stored in an on-chip static random access memory to improve data access speed. Figure 2As shown, the intermediate-level buffer is connected to the control module and the quantization-compression-activator respectively, and can perform bidirectional data transmission with the control module and unidirectional data output with the quantization-compression-activator, that is, the intermediate-level buffer includes an input port and two output ports, the input port is connected to the control module, and the two output ports are connected to the control module and the quantization-compression-activator respectively.

[0065] During the operation of the convolutional neural network processor, all generated data can be sent to the intermediate-level buffer. If the intermediate-level buffer detects that the received data is an intermediate calculation result, it will cache it, allowing the control module to read the data again and send it to the calculation module for calculation. If the intermediate-level buffer detects that the received data is the final calculation result, it will send the data to the quantization-compression-activation unit.

[0066] The quantization-compression-activator is used to receive the intermediate calculation result data transmitted by the intermediate-level buffer and perform quantization operations, data bit width compression and function activation on the intermediate calculation result data. Typically, after receiving the data sent by the intermediate-level buffer, the quantization-compression-activator immediately performs pipeline processing on it (referring to a series of processes that migrate data among several data, then perform complex data processing on the migrated data, and then load it into the target data system for summary). After processing for several clock cycles, the processed data is sent to the output buffer for temporary storage. When the feature map buffer has available storage space, the data will be sent to the feature map buffer again in the order of the first-in-first-out queue.

[0067] It should be noted that the above-mentioned instruction module, calculation module, control module and cache module are all functional modules of the convolutional neural network processor. This embodiment does not specifically limit its hardware composition, as long as it can achieve the above-mentioned functions.

[0068] In other implementations of this embodiment, the convolutional neural network processor can be implemented based on FPGA. FPGA (Field Programmable Gate Array) is composed of hardware resources such as logic units, RAM, and multipliers. By rationally organizing these hardware resources, hardware circuits such as multipliers, registers, and address generators can be implemented. FPGA can be reprogrammed infinitely, and loading a new design scheme only takes a few hundred milliseconds. New scheme designs can be implemented using reconfiguration, thereby reducing hardware overhead. In addition, the operating frequency of the FPGA is determined by the FPGA chip and design, and certain demanding requirements can be met by modifying the design or replacing a faster chip. Therefore, this embodiment adopts FPGA-based hardware implementation, and the convolutional neural network processor can simultaneously implement multiple convolutional neural network operations, especially when processing multiple images, which can significantly improve the processing speed of the convolutional neural network operations.

[0069] Based on the same concept of the above-mentioned convolutional neural network processor, this embodiment also provides a convolutional neural network implementation method, which is implemented based on the convolutional neural network processor of any of the above-mentioned embodiments, such as Figure 3 As shown, the following steps are included:

[0070] Step S1, receiving and parsing an instruction issued by a main processor; the content of the instruction includes at least data related to the image to be processed, data related to the convolutional neural network, and data related to the feature map;

[0071] Step S2, establishing a convolutional neural network model based on convolutional neural network related data;

[0072] Step S3: Execute the relevant content of the instruction through the convolutional neural network model and perform the corresponding convolutional neural network operation on the corresponding image to be processed.

[0073] The convolutional neural network implementation method provided in this embodiment is based on a convolutional neural network processor connected to a main processor via a bus. It can receive instructions issued by the main processor, parse the instructions, and then establish a convolutional neural network model based on the parsed convolutional neural network related data. The convolutional neural network model executes the relevant content of the instructions and performs corresponding convolutional neural network operations on the corresponding image to be processed, thereby realizing accelerated calculation of convolutional neural network operations. The method has high versatility and flexibility and can support multiple convolutional neural network models.

[0074] The present application also provides an electronic device to implement the above convolutional neural network implementation method. Figure 4 , which shows a schematic diagram of an electronic device provided by some embodiments of the present application. Figure 4 As shown, the electronic device 8 includes: a processor 800, a memory 801, a bus 802 and a communication interface 803. The processor 800, the communication interface 803 and the memory 801 are connected through the bus 802; the memory 801 stores a computer program that can be run on the processor 800, and when the processor 800 runs the computer program, it executes the convolutional neural network implementation method provided in any of the aforementioned embodiments of the present application.

[0075] The memory 801 may include high-speed random access memory (RAM) and may also include non-volatile memory, such as at least one disk storage. The communication connection between the device network element and at least one other network element is achieved through at least one communication interface 803 (which may be wired or wireless), and may use the Internet, a wide area network, a local area network, a metropolitan area network, etc.

[0076] Bus 802 can be an ISA bus, a PCI bus, or an EISA bus. Buses can be divided into address buses, data buses, control buses, etc. Memory 801 is used to store programs, and processor 800 executes the programs after receiving execution instructions. The convolutional neural network implementation method disclosed in any of the aforementioned embodiments of the present application can be applied to processor 800 or implemented by processor 800.

[0077] The processor 800 may be an integrated circuit chip with signal processing capabilities. During implementation, each step of the above method can be completed by hardware integrated logic circuits in the processor 800 or by software instructions. The above processor 800 may be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it may also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), an off-the-shelf field programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. It can implement or execute the various methods, steps, and logic block diagrams disclosed in the embodiments of this application. The general-purpose processor may be a microprocessor or any conventional processor. The steps of the method disclosed in conjunction with the embodiments of this application can be directly implemented and executed by a hardware decoding processor, or by a combination of hardware and software modules in the decoding processor. The software module can be located in a storage medium mature in the art, such as random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, etc. The storage medium is located in the memory 801 , and the processor 800 reads the information in the memory 801 and completes the steps of the above method in combination with its hardware.

[0078] The electronic device provided in the embodiment of the present application and the convolutional neural network implementation method provided in the embodiment of the present application are based on the same inventive concept and have the same beneficial effects as the methods adopted, operated or implemented by them.

[0079] The present application also provides a computer-readable storage medium corresponding to the convolutional neural network implementation method provided in the above embodiment. Figure 5 The computer-readable storage medium shown is a CD 30, on which a computer program (i.e., a program product) is stored. When the computer program is run by the processor, it will execute the convolutional neural network implementation method provided by any of the aforementioned embodiments.

[0080] It should be noted that examples of computer-readable storage media may also include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other optical or magnetic storage media, which are not listed here one by one.

[0081] The computer-readable storage medium provided in the above-mentioned embodiments of the present application and the convolutional neural network implementation method provided in the embodiments of the present application are based on the same inventive concept and have the same beneficial effects as the methods adopted, run or implemented by the application programs stored therein.

[0082] It should be noted that:

[0083] In the description provided herein, a large number of specific details are described. However, it is understood that the embodiments of the present application can be practiced without these specific details. In some instances, well-known structures and technologies are not shown in detail so as not to obscure the understanding of this description.

[0084] Similarly, it should be understood that in order to streamline the present application and aid in understanding one or more of the various inventive aspects, in the above description of the exemplary embodiments of the present application, various features of the present application are sometimes grouped together into a single embodiment, figure, or description thereof. However, this disclosed method should not be interpreted as reflecting the following schematic diagram: the claimed application requires more features than the features expressly recited in each claim. Rather, as reflected in the claims below, inventive aspects lie in less than all the features of the individual embodiments disclosed above. Therefore, the claims following the detailed description are hereby expressly incorporated into the detailed description, with each claim itself serving as a separate embodiment of the present application.

[0085] Furthermore, those skilled in the art will appreciate that although some embodiments herein include certain features included in other embodiments but not other features, combinations of features from different embodiments are intended to be within the scope of this application and to form different embodiments. For example, in the claims below, any of the claimed embodiments may be used in any combination.

[0086] The above are merely preferred embodiments of the present application, but the scope of protection of the present application is not limited thereto. Any changes or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in this application should be included in the scope of protection of the present application. Therefore, the scope of protection of the present application should be based on the scope of protection of the claims.

Claims

1. A convolutional neural network processor, characterized in that The convolutional neural network processor is an FPGA, which is connected to the main processor via a bus protocol and includes: an instruction module, configured to receive and parse instructions issued by the main processor; the contents of the instructions include at least data related to the image to be processed, data related to the convolutional neural network, and data related to the feature map; the instruction module stores the instructions in an instruction queue, and the instructions include configuration instructions for one or more convolutional neural networks; the convolutional neural network processor simultaneously supports multiple convolutional neural network operations; A first cache module is used to store quantization parameters and weight parameters of the convolutional neural network; A second cache module includes a feature map buffer and an output-level buffer, wherein the output-level buffer is used to cache the output result data of each layer of the convolutional neural network, and when the feature map buffer has available storage space, the output result data is sent to the feature map buffer; the feature map buffer is used to cache the output result data returned from the output-level buffer and notify the control module to read the output result data; A third cache module includes an intermediate-level cache and a quantization-compression-activator. All data generated during the operation of the convolutional neural network processor are first sent to the intermediate-level cache. The intermediate-level cache is used to cache the intermediate calculation result data of the convolutional neural network if it is detected that the received data is intermediate calculation result data, so that the control module can read it out again and send it to the calculation module for calculation. If it is detected that the received data is the final calculation result, it is sent to the quantization-compression-activator; the quantization-compression-activator is used to perform quantization calculation, data bit width compression and function activation on the intermediate calculation result data to obtain the output result data; The computing module is used to implement convolutional neural network operations; the computing module includes: a convolution operator, a normalization operator, a pooling operator, a sampling operator and a fully connected operator, which are respectively used to implement convolution operations, normalization operations, pooling operations, upsampling operations and fully connected operations; The control module is used to control the quantization parameters and weight parameters to be written into or read out of the first cache module according to corresponding instructions, control the intermediate calculation result data to be written into the third cache module or read out from the intermediate-level cache to the calculation module, and control the calculation module to execute the relevant content of the instruction according to the instruction, and perform corresponding convolutional neural network operations on the corresponding image to be processed.

2. The processor according to claim 1, wherein: The first buffer module includes a quantization buffer and a weight buffer; The quantization buffer stores the quantization parameters of the convolutional neural network in an on-chip static random access manner; The weight buffer uses on-chip static random access to store the weight parameters of the convolutional neural network.

3. The processor according to claim 2, wherein: The weight buffer is connected to the external memory, and the weight buffer reads weight data from the external memory in batches according to the size of the weight, and reads a specified amount of weight data each time for buffering.

4. The processor according to claim 1, wherein: The feature map buffer is also connected to an external memory. If the space occupied by the newly generated output result data is larger than the remaining storage space of the feature map buffer, the feature map buffer transfers the newly generated output result data to the external memory. The processor according to claim 1 , wherein: The intermediate level buffer is used to cache the intermediate calculation result data during the convolutional neural network operation process; The quantization-compression-activator is used to receive the intermediate calculation result data transmitted by the intermediate level buffer, and perform quantization operation, data bit width compression and function activation on the intermediate calculation result data.

6. A convolutional neural network implementation method, characterized in that: Implementation based on the convolutional neural network processor according to any one of claims 1 to 5, the method comprising: Receive and parse instructions issued by the main processor; the content of the instructions includes at least data related to the image to be processed, data related to the convolutional neural network, and data related to the feature map, the instructions are stored in the form of an instruction queue, and the instructions include configuration instructions for one or more convolutional neural networks; Establishing a convolutional neural network model according to the convolutional neural network related data; Through the convolutional neural network model, the relevant content of the instruction is executed, and the corresponding convolutional neural network operation is performed on the corresponding image to be processed.

7. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: The processor executes the computer program to implement the method according to claim 6.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that: The program is executed by a processor to implement the method according to claim 6.

Images