A neural network small memory chip architecture

Abstract

The application relates to a kind of neural network small memory chip architecture.Currently, the miniaturization trend of neural network is popular, and the volume disadvantage of MCU and FPGA scheme is obvious. The application proposes a chip hardware architecture, including input collection module, processor CPU, memory FLASH, running memory SRAM, neural network module and output module for communication through bus. SRAM is used as system running memory to cache all data during neural network runtime, neural network command FIFO, neural network command corresponding address information table and CPU runtime data; FLASH is used as system memory to store neural network weight and structure instruction as well as CPU program and read-only data. In the chip architecture of the application, processor CPU and neural network module run in parallel, improving the running efficiency. The application reuses memory space during neural network runtime, achieving the effect of reducing memory space.

Description

Technical Field

[0001] This invention belongs to the field of integrated circuit technology and relates to a small memory chip architecture for neural networks, which is applied to the field of artificial intelligence. Background Technology

[0002] With the development of neural network technology in recent years, some large-scale neural network models have been able to rival or even surpass human experts in certain fields, such as AlphaGo. Meanwhile, neural network applications for processing natural language, such as ChatGpt, can already communicate normally with humans and assist them in completing various tasks. At the same time, the application of neural networks in various niche fields is becoming increasingly widespread, such as voice control of appliances, speech noise reduction, and facial recognition.

[0003] Unlike large-scale neural network applications, those in niche domains are mostly embedded, making them highly sensitive to power consumption, size, and cost. Embedded systems offer advantages over large-scale neural network applications hosted on cloud servers, including lower latency and better privacy protection. For many niche domains, the application scenario is singular, and the trained model requires minimal modification. Therefore, many current neural network solutions separate training and deployment, deploying only the trained neural network within embedded systems.

[0004] Currently, many studies are miniaturizing neural networks and implementing them using MCUs or FPGAs. While these solutions are flexible and can be adapted to various application scenarios, they have significant disadvantages in terms of material cost, performance, power consumption, and size compared to ASICs in large-scale applications. Summary of the Invention

[0005] The purpose of this invention is to provide a small memory chip architecture for neural networks in response to the trend of neural network miniaturization.

[0006] The specific technical solution of this invention is as follows:

[0007] It includes an input acquisition module that communicates via a bus, a CPU processor, a FLASH memory, an SRAM memory, a neural network module, and an output module.

[0008] The input acquisition module is used to acquire data to be processed, such as images and sounds.

[0009] The CPU is used to perform preprocessing of input data and postprocessing of output data from neural network modules.

[0010] The aforementioned FLASH memory is used to store the weights and structural instructions of the neural network, as well as the program and read-only data of the processor CPU.

[0011] The aforementioned FLASH memory supports XIP (Execute In Place) technology, enabling the CPU to directly execute programs stored in the FLASH memory, and allowing the neural network module to read data byte-by-byte from the FLASH memory. Even though the read speed in XIP mode is slower than SRAM, the performance is almost unaffected because the time spent reading instructions and weights is very small compared to the computation time. During system operation, the FLASH memory is read-only; when the system is not in use, it becomes read-write, allowing updates to the internal CPU program, neural network weights, and structural instructions.

[0012] One or more neural network structure instructions and weights are stored in the FLASH memory, which can be directly read without needing to be moved to the SRAM, thus reducing storage costs. Multiple neural network structure instructions and weights are stored independently in the FLASH memory, but can share and reuse the same memory space during runtime. The cached data between the neural network structure layers is released after execution, reducing the runtime SRAM space requirement. The neural network structure instructions consist of multiple neural network operator instructions, each corresponding to a specific neural network operation.

[0013] The SRAM is used to cache all data during neural network operation, the neural network command FIFO, the address information table corresponding to the neural network commands, and the data during CPU operation.

[0014] The address information table includes address information for neural network structure instructions and a mapping table of neural network execution addresses. The operator instructions within the neural network structure instructions contain virtual addresses, which the neural network module converts into physical addresses according to the mapping table. The input and output addresses of each operator instruction in the neural network structure instructions are virtual addresses, which are fixed after the neural network training is completed; however, the corresponding physical addresses are dynamically variable, and their mapping table is configured by the processor CPU each time the neural network is run.

[0015] The neural network module receives commands from the CPU and performs neural network operations. In addition to supporting normal neural network operators, the module also supports copy operations. Adding a copy operation to the neural network structure, copying weights from the FLASH memory to the SRAM memory, speeds up the network's operation and avoids the efficiency loss caused by repeatedly reading weight data during complex calculations such as convolution, where the FLASH memory access speed is slower than the SRAM memory.

[0016] The output module is used to perform specific output operations, such as displaying images, playing sounds, and transmitting commands.

[0017] In the chip architecture of this invention, the CPU and the neural network module run in parallel, improving system performance. Wherein:

[0018] The specific operation of the processor CPU is as follows:

[0019] C1. The CPU reads the input data collected by the input acquisition module from the SRAM (Support RAM).

[0020] C2. The CPU preprocesses the input data, such as cropping the input image data and performing a Fast Fourier Transform (FFT) on the input speech data, which is a time-domain to frequency-domain conversion.

[0021] C3. The CPU determines whether the free space in the SRAM is sufficient for running a neural network command. If it is, it executes C4; otherwise, it waits.

[0022] C4. The CPU writes the spatial information of this neural network command into the address information table;

[0023] C5. The CPU determines whether the free space in the command FIFO is sufficient to write the data for this neural network command. If it is sufficient, C6 is executed; otherwise, it waits.

[0024] C6. Write the neural network command data, i.e. the starting address and length of the address information table, into the command FIFO;

[0025] C7. During the execution of C1-C6, if the CPU receives an interrupt from the neural network module, it will release the corresponding neural network runtime space and call the output module to complete the final operation.

[0026] The specific operations of the neural network module are as follows:

[0027] S1. The neural network module determines whether there is any readable command information in the command FIFO. If there is, it executes S2; otherwise, it waits.

[0028] S2. The neural network module obtains command information from the command FIFO and reads the address information table based on the command information;

[0029] S3. The neural network module reads the neural network structure instructions from the address information table and executes the corresponding neural network operations;

[0030] S4. After the neural network module completes all neural network operations for this neural network command, it sends an interrupt to the processor CPU.

[0031] This invention's architecture stores the fixed structure instructions and weights of the trained neural network in FLASH memory and reuses this memory space during neural network operation, thus reducing memory usage. The architecture utilizes XIP technology to allow the CPU and neural network modules to directly read data from the FLASH memory, eliminating the need to move data to SRAM, further reducing memory space. Simultaneously, due to the characteristics of neural networks, the slow FLASH read speed does not significantly impact overall operating efficiency. Furthermore, the architecture uses an address information table, allowing the neural network's operating address to change in real time, enabling better parallel operation of the CPU and neural network modules and improving overall efficiency. Attached Figure Description

[0032] Figure 1 This is a schematic diagram of the neural network small memory chip architecture of the present invention;

[0033] Figure 2 This is a schematic diagram of the operation flow of the CPU in the chip architecture of this invention;

[0034] Figure 3 This is a schematic diagram of the operation flow of the neural network module in the chip architecture of the present invention. Detailed Implementation

[0035] To make the objectives and advantages of this invention clearer, the specific implementation methods of this invention will be further described in detail below. It should be noted that these implementation methods are merely for explaining this invention and do not limit the scope of its application.

[0036] like Figure 1 As shown, the chip architecture of a neural network small memory system includes an input acquisition module 10 that communicates via a bus, a processor CPU 20, a FLASH memory 30, a running memory SRAM 40, a neural network module 50, and an output module 60.

[0037] The input acquisition module 10 is used to acquire data to be processed, such as images and sounds.

[0038] The CPU 20 is used to perform preprocessing of input data and post-processing of output data from the neural network module. Its scheduling process for the neural network module 50 is as follows: Figure 2 As shown, specifically:

[0039] C1. The CPU reads the input data collected by the input acquisition module from the SRAM (Support RAM).

[0040] C2. The CPU preprocesses the input data, such as cropping the input image data and performing a Fast Fourier Transform (FFT) on the input speech data, which is a time-domain to frequency-domain conversion.

[0041] C3. The CPU determines whether the free space in the SRAM is sufficient for running a neural network command. If it is, it executes C4; otherwise, it waits.

[0042] C4. The CPU writes the spatial information of this neural network command into the address information table;

[0043] C5. The CPU determines whether the free space in the command FIFO is sufficient to write the data for this neural network command. If it is sufficient, C6 is executed; otherwise, it waits.

[0044] C6. Write the neural network command data, i.e. the starting address and length of the address information table, into the command FIFO;

[0045] C7. During the execution of C1-C6, if the CPU receives an interrupt from the neural network module, it will release the corresponding neural network runtime space and call the output module to complete the final operation.

[0046] The FLASH memory (FLASH30) is used to store the weights and structural instructions of the neural network, as well as the CPU's program and read-only data. The FLASH memory supports XIP (Execute In Place) technology, allowing the CPU to directly run programs stored in the FLASH memory, and the neural network module to read data byte-by-byte from the FLASH memory. Even though the read speed in XIP mode is slower than SRAM, the performance is almost unaffected because reading instructions and weights takes up a very small percentage of the computation time. During system operation, the FLASH memory is read-only; when the system is not in use, the FLASH memory is read-write, allowing updates to the internal CPU program, neural network weights, and structural instructions.

[0047] One or more neural network structure instructions and weights are stored in FLASH memory, which can be directly read without being moved to SRAM, thus reducing storage costs. Multiple neural network structure instructions and weights are stored independently in FLASH memory, but can share and reuse the same memory space during runtime. Cache data between neural network layers is released after execution, reducing the demand for SRAM space during runtime. Neural network structure instructions consist of multiple neural network operator instructions, each corresponding to a specific neural network operation.

[0048] The SRAM40 is used to cache all data during neural network operation, the neural network command FIFO, the address information table corresponding to the neural network commands, and the data during CPU operation.

[0049] The address information table includes address information for neural network structure instructions and a mapping table of neural network execution addresses.

[0050] The operator instructions in a neural network architecture contain virtual addresses. The neural network module translates these virtual addresses into physical addresses using a mapping table. The input and output addresses of each operator instruction in the neural network architecture are virtual addresses, which are fixed after the neural network training is complete; however, the corresponding physical addresses are dynamically variable, and their mapping table is configured by the processor CPU each time the neural network is run. Specifically, an example of such a mapping table is introduced here:

[0051] The mapping table contains multiple sets of base addresses and sequence numbers; the virtual address in the operator instruction is an offset address and a sequence number; the neural network module obtains the physical address by adding the corresponding base address to the sequence number. For example: if the input data space sequence number of the current operator instruction is 0x1, the offset address is 0x123, the input weight space sequence number is 0x2, the offset address is 0x23, the output data space sequence number is 0x3, the offset address is 0x12, the base address of space sequence number 0x1 is 0x1000, the base address of space sequence number 0x2 is 0x5000, and the base address of space sequence number 0x3 is 0x2000, then the input data address of the operator instruction is 0x1123, the input weight address is 0x5023, and the output data address is 0x2012.

[0052] The neural network module 50 receives commands from the CPU and performs neural network operations. In addition to supporting normal neural network operators, the module also supports copy operations. Adding a copy operation to the neural network structure copies weights from FLASH memory to SRAM, which speeds up network operation and avoids the efficiency loss caused by repeatedly reading weight data during complex calculations such as convolution, where FLASH access speed is slower than SRAM access speed. The neural network module 50 completes the scheduling response process as follows... Figure 3 As shown, specifically:

[0053] S1. The neural network module determines whether there is any readable command information in the command FIFO. If there is, it executes S2; otherwise, it waits.

[0054] S2. The neural network module obtains command information from the command FIFO and reads the address information table based on the command information;

[0055] S3. The neural network module reads the neural network structure instructions from the address information table and executes the corresponding neural network operations;

[0056] S4. After the neural network module completes all neural network operations for this neural network command, it sends an interrupt to the processor CPU.

[0057] The CPU can continuously generate the input data required for the operation of the neural network module during its operation. When the output conditions are met, the CPU calls the output module to perform the corresponding operation to achieve the expected results.

[0058] The output module 60 is used to perform specific output operations, such as displaying images, playing sounds, and transmitting commands.

[0059] Specifically, in a small-memory neural network system, since the read speed of FLASH memory is slower than that of SRAM, for data that is repeatedly read, copy operator instructions can be inserted into the neural network structure instructions to copy the weights stored in FLASH memory to SRAM, thereby speeding up the network's operation. However, this operation will increase the demand for SRAM space, which needs to be determined during network training based on the size of the neural network model and the available SRAM space.

[0060] Because the neural network structure is read-only and cannot be modified during system operation, virtual addresses are used within the neural network's operator commands. The mapping relationship between virtual addresses and physical addresses is dynamically configured when the processor (CPU) prepares input data, enabling flexible address changes and allowing the processor (CPU) and neural network modules to run in parallel.

Claims

1. A small memory chip architecture for neural networks, comprising an input acquisition module communicating via a bus, a CPU, a FLASH memory, an SRAM memory, a neural network module, and an output module; characterized in that: The input acquisition module is used to acquire the data to be processed; The CPU is used to perform preprocessing of input data and postprocessing of output data from neural network modules. The aforementioned FLASH memory is used to store the weights and structural instructions of the neural network, as well as the program and read-only data of the processor CPU. The SRAM is used to cache all data during neural network operation, the neural network command FIFO, the address information table corresponding to the neural network commands, and the data during processor CPU operation; the address information table includes the address information of neural network structure instructions and a mapping table of neural network running addresses; The neural network module is used to receive commands from the processor CPU and complete neural network operations; The output module is used to perform specific output operations, such as displaying images, playing sounds, and transmitting commands.

2. The neural network small memory chip architecture as described in claim 1, characterized in that: The aforementioned FLASH memory supports XIP technology, enabling the CPU to directly run programs on the FLASH memory, and the neural network module to directly read data byte by byte from the FLASH memory. When the system is running, the FLASH memory is a read-only memory; when the system is not working, the FLASH memory is a read-write memory. The FLASH memory stores one or more neural network structure instructions and weights, which can be read directly without being moved to the SRAM memory. Multiple neural network structure instructions and weights are stored independently in the FLASH memory, and can share and reuse the same memory space during runtime; the cached data between the layers of the neural network structure is released after the operation is completed, reducing the demand for SRAM space during runtime; the neural network structure instructions consist of multiple neural network operator instructions, and each operator instruction corresponds to a specific neural network operation.

3. The neural network small memory chip architecture as described in claim 1, characterized in that: The operator instructions in the neural network structure instructions contain virtual addresses, and the neural network module converts the virtual addresses into physical addresses according to the mapping table. The input and output addresses of each operator instruction in the neural network structure instructions are virtual addresses, which are fixed after the neural network training is completed. The corresponding physical addresses are dynamically variable, and their mapping table is configured by the processor CPU each time the neural network is run.

4. The neural network small memory chip architecture as described in claim 1, characterized in that: In addition to supporting operators for normal neural network calculations, the neural network module also supports copy operations; a copy operation is added to the neural network structure to copy the weights stored in the FLASH memory to the SRAM memory.

5. A neural network small memory chip architecture as described in claim 1, 2, 3, or 4, characterized in that, The CPU and neural network module run in parallel, wherein: The specific operation of the processor CPU is as follows: C1. The CPU reads the input data collected by the input acquisition module from the SRAM (Support RAM). C2. The CPU performs preprocessing on the input data; C3. The CPU determines whether the free space in the SRAM is sufficient for running a neural network command. If it is, it executes C4; otherwise, it waits. C4. The CPU writes the spatial information of this neural network command into the address information table; C5. The CPU determines whether the free space in the command FIFO is sufficient to write the data for this neural network command. If it is sufficient, C6 is executed; otherwise, it waits. C6. Write the neural network command data, i.e. the starting address and length of the address information table, into the command FIFO; C7. During the execution of C1-C6, if the CPU receives an interrupt from the neural network module, it will release the corresponding neural network running space and call the output module to complete the final operation. The specific operations of the neural network module are as follows: S1. The neural network module determines whether there is any readable command information in the command FIFO. If there is, it executes S2; otherwise, it waits. S2. The neural network module obtains command information from the command FIFO and reads the address information table based on the command information; S3. The neural network module reads the neural network structure instructions from the address information table and executes the corresponding neural network operations; S4. After the neural network module completes all neural network operations for this neural network command, it sends an interrupt to the processor CPU.

Images