Processing system of neural network algorithm, data processing method and electronic device

Abstract

The present disclosure includes a processing system of a neural network algorithm, a data processing method and an electronic device. The processing system comprises a first processor, a second processor, a first memory, a first cache module and a system bus; the first processor and the first memory are connected with the system bus respectively; the second processor is connected with the system bus through the first cache module; the first memory is used for storing operation parameters of the neural network algorithm; the first processor is used for sending first address information to the first cache module, the first address information being address information of the operation parameters in the first memory; the first cache module is used for obtaining the operation parameters from the first memory according to the first address information; and the second processor is used for reading the operation parameters through the first cache module and performing operation of the neural network algorithm according to the operation parameters.

Description

Technical Field

[0001] This disclosure relates to computer technology, and more specifically, to processing systems, data processing methods, and electronic devices for neural network algorithms. Background Technology

[0002] Artificial intelligence technology is developing rapidly and its applications are becoming increasingly widespread. With the advancement of artificial intelligence technology, higher demands are being placed on the processing performance of neural network algorithm processing systems. Figure 1 This paper illustrates a conventional embedded neural network algorithm processing system. Due to the limited resources of embedded systems, data transmission is typically handled via the system bus. Mounting the neural network processor on the same system bus places a significant load on it, resulting in low overall system efficiency. Therefore, it is necessary to provide a novel neural network algorithm processing system architecture. Summary of the Invention

[0003] The neural network algorithm processing system and electronic device of the present disclosure provide a new neural network algorithm processing system architecture that can be applied to embedded system architecture.

[0004] According to a first aspect of this disclosure, a processing system for a neural network algorithm is provided. The processing system includes a first processor, a second processor, a first memory, a first cache module, and a system bus; the first processor and the first memory are respectively connected to the system bus; the second processor is connected to the system bus through the first cache module; the first memory is used to store the computational parameters of the neural network algorithm; the first processor is used to send first address information to the first cache module, the first address information being the address information of the computational parameters in the first memory; the first cache module is used to retrieve the computational parameters from the first memory according to the first address information; the second processor is used to read the computational parameters through the first cache module and perform the computation of the neural network algorithm according to the computational parameters.

[0005] According to a second aspect of this disclosure, a data processing method is provided, applied to a neural network algorithm processing system. The neural network algorithm processing system includes a first processor, a second processor, a first memory, a first cache module, and a system bus. The first processor and the first memory are respectively connected to the system bus. The second processor is connected to the system bus through the first cache module. The first memory is used to store the operation parameters of the neural network algorithm. The data processing method includes: the first processor sending first address information to the first cache module, the first address information being the address information of the operation parameters in the first memory; the first cache module obtaining the operation parameters from the first memory according to the first address information; and the second processor reading the operation parameters through the first cache module and performing the operation of the neural network algorithm according to the operation parameters.

[0006] According to a third aspect of this disclosure, an electronic device is provided having a processing system for the neural network algorithm described in any of the first aspects of this disclosure.

[0007] The neural network algorithm processing system, data processing method, and electronic device disclosed herein propose a new neural network algorithm processing system architecture, which adds a first cache module. The first cache module obtains the operation parameters of the neural network algorithm from a first memory. The second processor reads the operation parameters through the first cache module and performs neural network algorithm operations according to the operation parameters, thereby reducing the system memory occupation and not affecting the overall system efficiency.

[0008] The features and advantages of the embodiments of this specification will become clear from the following detailed description of exemplary embodiments with reference to the accompanying drawings. Attached Figure Description

[0009] The accompanying drawings, which are incorporated in and form a part of this specification, illustrate embodiments of this specification and, together with their description, serve to explain the principles of these embodiments.

[0010] Figure 1 This is a block diagram of a neural network algorithm processing system in the prior art;

[0011] Figure 2(a) is a block diagram of the processing system of the neural network algorithm provided in the first embodiment of this disclosure;

[0012] Figure 2(b) is a block diagram of the processing system of the neural network algorithm provided in the third embodiment of this disclosure;

[0013] Figure 2(c) is a block diagram of the processing system of the neural network algorithm provided in the fourth embodiment of this disclosure;

[0014] Figure 2(d) is a block diagram of the processing system of the neural network algorithm provided in the fifth embodiment of this disclosure;

[0015] Figure 3(a) is a schematic flowchart of the data processing method provided in the first embodiment of this disclosure;

[0016] Figure 3(b) is a flowchart illustrating the data processing method provided in the second embodiment of this disclosure.

[0017] Figure 3(c) is a flowchart illustrating the data processing method provided in the third embodiment of this disclosure;

[0018] Figure 3(d) is a flowchart illustrating the data processing method provided in the fourth embodiment of this disclosure;

[0019] Figure 3(e) is a flowchart illustrating the data processing method provided in the fifth embodiment of this disclosure;

[0020] Figure 4 This is a schematic flowchart illustrating the operation of a neural network algorithm by the second processor provided in this embodiment. Detailed Implementation

[0021] Various exemplary embodiments of this specification will now be described in detail with reference to the accompanying drawings.

[0022] The following description of at least one exemplary embodiment is merely illustrative and is in no way intended to limit the embodiments of this specification or their application or use.

[0023] It should be noted that similar labels and letters in the following figures indicate similar items; therefore, once an item is defined in one figure, it does not need to be discussed further in subsequent figures.

[0024] First, the definitions of the technical terms and English abbreviations in the embodiments of this disclosure are explained:

[0025] MCU: Microcontroller Unit, also known as a single-chip microcomputer or microcontroller.

[0026] NPU: Neural-network Processing Unit, is a processor specifically designed for performing neural network calculations.

[0027] Flash memory is a semiconductor storage chip that retains information even when power is off. It has the advantages of small size, low power consumption, and resistance to physical damage.

[0028] DMA: Direct Memory Access. Direct memory access is an interface technology that enables high-speed data transfer between peripherals and memory, or between memory devices.

[0029] FIFO storage: First Input First Output is a first-in, first-out storage technology where data that enters the storage area first is removed first.

[0030] Ping-pong type storage: The memory uses two storage areas. Data is first stored in the first storage area. When the first storage area is full, the data is then stored in the second storage area. At the same time, the data in the first storage area is retrieved through logic control. This process continues, with the two storage areas taking turns caching the data.

[0031] CNN: Convolutional Neural Networks.

[0032] TOPS: Tera Operations Per Second, a unit of computing power. 1 TOPS means that the processor can perform one trillion operations per second.

[0033] MACs unit: Multiply–Accumulate Operations.

[0034] The inventors of this application have conducted extensive research and analysis on the processing systems of embedded neural network algorithms in the prior art, and have found that the processing systems of embedded neural network algorithms in the prior art have the following problems:

[0035] Common neural network algorithms require a large number of computational parameters; for example, a common image classification algorithm can involve hundreds of millions of weight parameters. Neural network processors need to use a large number of weight parameters when running neural network algorithms (such as CNN network algorithms for image recognition).

[0036] When the MCU powers on, it reads the instructions and computational parameters required by the neural network processor from flash memory and stores them in system memory. The large amount of computational parameters requires significant system memory. After the neural network processor starts up, it reads instructions and computational parameters from system memory via the system bus, also putting considerable pressure on the system bus. For example, in an embedded system running a typical neural network algorithm, the computational parameters required by the algorithm can account for about 80% of the system memory usage, leaving only about 20% of the system memory for the MCU and the cache space used by the neural network processor. For these reasons, neural network algorithm processing systems require large-capacity, high-performance memory. However, embedded systems are sensitive to cost and power consumption; using large-capacity system memory increases both cost and power consumption.

[0037] Typical neural network algorithms operate layer by layer. These large amounts of computational parameters are used only once per round of neural network computation, yet they continuously occupy system memory, resulting in significant memory consumption. Neural network processors are limited by system cost and power consumption, with most having less than 4 TOPS, and some even below 0.5 TOPS. For neural network processors with limited computing power, even if all computational parameters are loaded into system memory, the processor can only read and process them bit by bit, wasting system memory space.

[0038] The arithmetic logic circuits (ALUs) of neural network processors primarily perform matrix convolution operations, which are computationally intensive and require high system memory bandwidth. The conventional approach is for the ALUs to access system memory via the system bus and use that memory to store intermediate data. This method is relatively flexible but inefficient. There are two reasons for this inefficiency: First, the ALUs perform convolution operations in parallel, resulting in large data read / write operations. Therefore, the ALUs are high-bandwidth, such as 128-bit, 256-bit, or even higher. However, the system bus is only 32-bit wide. This necessitates a bit-width converter in the neural network processor to convert the 128-bit width to 32-bit for data read / write operations. Since 32-bit is only one-quarter the width of 128-bit, it means that for every data read / write operation performed by the ALUs, the system bus needs to perform four data read / write operations. This forces the ALUs to read data from system memory before processing, as the data cannot keep up. Second, the frequent data reads on the system bus cause it to become busy, affecting other devices and even the system core's data access, thus reducing the overall system efficiency.

[0039] For the above reasons, the system consumes a lot of memory, requiring the use of large-capacity, high-performance system memory to store the computational parameters of the neural network processor, which leads to increased cost and power consumption, slow startup time, and affects product competitiveness.

[0040] Based on the analysis of the above-mentioned reasons for the large system memory consumption, this disclosure proposes a new processing system for neural network algorithms, which will be described below:

[0041] <First Embodiment>

[0042] Referring to Figure 2(a), a processing system for a neural network algorithm according to a first embodiment of this disclosure is illustrated. The processing system for the neural network algorithm includes a first processor, a second processor, a first memory, a first cache module, and a system bus.

[0043] The first processor and the first memory are connected to the system bus. The second processor is connected to the system bus through the first cache module.

[0044] The first memory is used to store the computational parameters of the neural network algorithm.

[0045] The first processor is used to send first address information to the first cache module. The first address information is the address information of the operation parameters in the first memory.

[0046] The first cache module is used to obtain the operation parameters from the first memory according to the first address information.

[0047] The second processor is used to read the operation parameters through the first cache module and perform neural network algorithm operations based on the operation parameters.

[0048] In one example, the second processor is a neural network processor. In one example, the first processor can be a microcontroller unit. In one example, the first memory is non-volatile memory. Non-volatile memory is memory whose information is not lost after power failure. In one example, the first memory can be flash memory. In one example, the first cache module includes a DMA circuit. Based on the DMA circuit, the first cache module can automatically retrieve operation parameters from the first memory via the system bus, achieving high transmission efficiency and fast transmission speed.

[0049] In one example, the second processor includes control logic circuitry and arithmetic logic circuitry. The control logic circuitry reads arithmetic parameters from the first cache module and sends these parameters to the arithmetic logic circuitry. The arithmetic logic circuitry then performs neural network algorithm calculations based on the arithmetic parameters. In another example, the second processor has a cache space. The control logic circuitry stores the read arithmetic parameters in the cache space, and the arithmetic logic circuitry retrieves the arithmetic parameters from this cache space to perform the neural network algorithm calculations.

[0050] The neural network algorithm processing system of this disclosure proposes a new neural network algorithm processing system architecture, which adds a first cache module. The first cache module obtains the operation parameters of the neural network algorithm from the first memory. The second processor reads the operation parameters through the first cache module and performs the operation of the neural network algorithm according to the operation parameters, thereby reducing the system memory occupation and not affecting the overall system efficiency.

[0051] <Second Embodiment>

[0052] The processing system for the neural network algorithm according to the second embodiment of this disclosure has the same system architecture as the first embodiment. The processing system for the neural network algorithm includes a first processor, a second processor, a first memory, a first cache module, and a system bus.

[0053] The first processor and the first memory are connected to the system bus. The second processor is connected to the system bus through the first cache module.

[0054] The first memory is used to store the computational parameters of the neural network algorithm. The first memory is also used to store the raw data, which is the input data for the neural network algorithm.

[0055] The first processor is used to send first address information to the first cache module, the first address information being the address information of the operation parameters in the first memory. The first processor is also used to send second address information to the first cache module, the second address information being the address information of the original data in the first memory.

[0056] The first cache module is used to retrieve operation parameters from the first memory based on the first address information. The first cache module is also used to retrieve raw data from the first memory based on the second address information.

[0057] The second processor is used to read the operation parameters and raw data through the first cache module, and to perform the operation of the neural network algorithm based on the operation parameters and raw data.

[0058] For example, if the original data is a regular image, this image is input into a neural network algorithm, which then classifies the image. Alternatively, if the original data is a radar point cloud image, this radar point cloud image is input into a neural network algorithm, which then classifies the radar point cloud image.

[0059] In one example, the second processor is a neural network processor. In one example, the first processor can be a microcontroller unit. In one example, the first memory is non-volatile memory. Non-volatile memory is memory whose information is not lost after power failure. In one example, the first memory can be flash memory. In one example, the first cache module includes a DMA circuit. Based on the DMA circuit, the first cache module can automatically retrieve operation parameters from the first memory via the system bus, achieving high transmission efficiency and fast transmission speed.

[0060] In one example, the second processor may include control logic circuitry and arithmetic logic circuitry. The control logic circuitry reads the computational parameters and raw data from the first cache module and sends them to the arithmetic logic circuitry. The arithmetic logic circuitry performs neural network algorithm calculations based on the computational parameters and raw data. In another example, the second processor has a cache space. The control logic circuitry stores the read computational parameters and raw data in the cache space, and the arithmetic logic circuitry retrieves the computational parameters and raw data from this cache space to perform the neural network algorithm calculations.

[0061] The neural network algorithm processing system of this disclosure proposes a new neural network algorithm processing system architecture, which adds a first cache module. The first cache module obtains the operation parameters and raw data of the neural network algorithm from the first memory. The second processor reads the operation parameters and raw data through the first cache module and performs neural network algorithm operation based on the operation parameters and raw data, thereby reducing the system memory occupation and not affecting the overall system efficiency.

[0062] <Third Embodiment>

[0063] Referring to Figure 2(b), a processing system for a neural network algorithm according to a third embodiment of this disclosure is illustrated. The processing system for the neural network algorithm includes a first processor, a second processor, a first memory, a first cache module, system memory, and a system bus.

[0064] The first processor, the first memory, and the system memory are all connected to the system bus. The second processor is connected to the system bus through the first cache module.

[0065] The first memory is used to store the computational parameters of the neural network algorithm. The system memory is used to store the raw data, which is the input data of the neural network algorithm.

[0066] The first processor is used to send first address information to the first cache module. The first address information is the address information of the operation parameters in the first memory. The first processor is also used to send second address information to the first cache module. The second address information is the address information of the original data in the system memory.

[0067] The first cache module is used to retrieve operation parameters from the first memory based on the first address information. The first cache module is also used to retrieve raw data from system memory based on the second address information.

[0068] The second processor is used to read the operation parameters and raw data through the first cache module, and to perform neural network algorithm operations based on the operation parameters and raw data.

[0069] For example, if the original data is a regular image, this image is input into a neural network algorithm, which then classifies the image. Alternatively, if the original data is a radar point cloud image, this radar point cloud image is input into a neural network algorithm, which then classifies the radar point cloud image.

[0070] In one example, the second processor is a neural network processor. In one example, the first processor can be a microcontroller unit. In one example, the first memory is non-volatile memory. Non-volatile memory is memory whose information is not lost after power failure. In one example, the first memory can be flash memory. In one example, the first cache module includes a DMA circuit. Based on the DMA circuit, the first cache module can automatically retrieve operation parameters from the first memory via the system bus, achieving high transmission efficiency and fast transmission speed.

[0071] The second processor may include control logic circuitry and arithmetic logic circuitry. The control logic circuitry reads the computational parameters and raw data from the first cache module and sends them to the arithmetic logic circuitry. The arithmetic logic circuitry performs neural network algorithm calculations based on the computational parameters and raw data. In one example, the second processor has a cache space. The control logic circuitry stores the read computational parameters and raw data into the cache space, and the arithmetic logic circuitry retrieves the computational parameters and raw data from this cache space to perform the neural network algorithm calculations.

[0072] The neural network algorithm processing system of this disclosure proposes a new neural network algorithm processing system architecture, which adds a first cache module. The first cache module obtains the operation parameters of the neural network algorithm from the first memory. The second processor reads the operation parameters through the first cache module and performs the operation of the neural network algorithm according to the operation parameters, thereby reducing the system memory occupation and not affecting the overall system efficiency.

[0073] The neural network algorithm processing system may further include peripherals, with the raw data required by the second processor for neural network algorithm operations originating from the peripherals. Peripherals may be, for example, radar or cameras. Specifically, the peripherals are connected to the system bus, collect external information, and generate raw data. The first processor or a first cache module stores the raw data generated by the peripherals via the system bus, for example, in system memory or a first storage device. After storage, the first cache module is also used to retrieve the raw data from system memory or the first storage device. The second processor is used to read the operation parameters and raw data through the first cache module, and perform the neural network algorithm operation based on the operation parameters and raw data. Based on this, this disclosure proposes a fourth and a fifth embodiment of the neural network algorithm processing system.

[0074] <Fourth Embodiment>

[0075] Referring to Figure 2(c), the processing system of the neural network algorithm of the fourth embodiment of this disclosure is illustrated.

[0076] The processing system for the neural network algorithm includes a first processor, a second processor, a first memory, a first cache module, peripherals, and a system bus.

[0077] The first processor, the first memory, and the peripherals are each connected to the system bus. The second processor is connected to the system bus through the first cache module.

[0078] Peripherals are used to collect external information and generate raw data. The first processor stores the raw data generated by the peripherals into the first memory through the system bus.

[0079] The first memory is used to store the computational parameters of the neural network algorithm. The first memory is also used to store the raw data generated by the peripheral device; this raw data serves as the input data for the neural network algorithm.

[0080] The first processor is used to send first address information to the first cache module, the first address information being the address information of the operation parameters in the first memory. The first processor is also used to send second address information to the first cache module, the second address information being the address information of the original data in the first memory.

[0081] The first cache module is used to retrieve operation parameters from the first memory based on the first address information. The first cache module is also used to retrieve raw data from the first memory based on the second address information.

[0082] The second processor is used to read the operation parameters and raw data through the first cache module, and to perform the operation of the neural network algorithm based on the operation parameters and raw data.

[0083] For example, if the original data is a regular image, this image is input into a neural network algorithm, which then classifies the image. Alternatively, if the original data is a radar point cloud image, this radar point cloud image is input into a neural network algorithm, which then classifies the radar point cloud image.

[0084] In one example, the second processor is a neural network processor. In one example, the first processor can be a microcontroller unit. In one example, the first memory is non-volatile memory. Non-volatile memory is memory whose information is not lost after power failure. In one example, the first memory can be flash memory. In one example, the first cache module includes a DMA circuit. Based on the DMA circuit, the first cache module can automatically retrieve operation parameters from the first memory via the system bus, achieving high transmission efficiency and fast transmission speed.

[0085] In one example, the second processor may include control logic circuitry and arithmetic logic circuitry. The control logic circuitry reads the computational parameters and raw data from the first cache module and sends them to the arithmetic logic circuitry. The arithmetic logic circuitry performs neural network algorithm calculations based on the computational parameters and raw data. In another example, the second processor has a cache space. The control logic circuitry stores the read computational parameters and raw data in the cache space, and the arithmetic logic circuitry retrieves the computational parameters and raw data from this cache space to perform the neural network algorithm calculations.

[0086] The neural network algorithm processing system of this disclosure proposes a new neural network algorithm processing system architecture, which adds a first cache module. The first cache module obtains the operation parameters and raw data of the neural network algorithm from the first memory. The second processor reads the operation parameters and raw data through the first cache module and performs neural network algorithm operation based on the operation parameters and raw data, thereby reducing the system memory occupation and not affecting the overall system efficiency.

[0087] <Fifth Embodiment>

[0088] Referring to Figure 2(d), a processing system for a neural network algorithm according to a fifth embodiment of this disclosure is illustrated. This processing system includes a first processor, a second processor, a first memory, a first cache module, system memory, peripherals, and a system bus.

[0089] The first processor, the first memory, the system memory, and the peripherals are all connected to the system bus. The second processor is connected to the system bus through the first cache module.

[0090] Peripherals are used to collect external information and generate raw data. The first processor stores the raw data generated by the peripherals into the system memory via the system bus.

[0091] The first memory is used to store the computational parameters of the neural network algorithm. The system memory is used to store the raw data, which is the input data of the neural network algorithm.

[0092] The first processor is used to send first address information to the first cache module. The first address information is the address information of the operation parameters in the first memory. The first processor is also used to send second address information to the first cache module. The second address information is the address information of the original data in the system memory.

[0093] The first cache module is used to retrieve operation parameters from the first memory based on the first address information. The first cache module is also used to retrieve raw data from system memory based on the second address information.

[0094] The second processor is used to read the operation parameters and raw data through the first cache module, and to perform neural network algorithm operations based on the operation parameters and raw data.

[0095] For example, if the original data is a regular image, this image is input into a neural network algorithm, which then classifies the image. Alternatively, if the original data is a radar point cloud image, this radar point cloud image is input into a neural network algorithm, which then classifies the radar point cloud image.

[0096] In one example, the second processor is a neural network processor. In one example, the first processor can be a microcontroller unit. In one example, the first memory is non-volatile memory. Non-volatile memory is memory whose information is not lost after power failure. In one example, the first memory can be flash memory. In one example, the first cache module includes a DMA circuit. Based on the DMA circuit, the first cache module can automatically retrieve operation parameters from the first memory via the system bus, achieving high transmission efficiency and fast transmission speed.

[0097] The second processor may include control logic circuitry and arithmetic logic circuitry. The control logic circuitry reads the computational parameters and raw data from the first cache module and sends them to the arithmetic logic circuitry. The arithmetic logic circuitry performs neural network algorithm calculations based on the computational parameters and raw data. In one example, the second processor has a cache space. The control logic circuitry stores the read computational parameters and raw data into the cache space, and the arithmetic logic circuitry retrieves the computational parameters and raw data from this cache space to perform the neural network algorithm calculations.

[0098] The neural network algorithm processing system of this disclosure proposes a new neural network algorithm processing system architecture, which adds a first cache module. The first cache module obtains the operation parameters of the neural network algorithm from the first memory. The second processor reads the operation parameters through the first cache module and performs the operation of the neural network algorithm according to the operation parameters, thereby reducing the system memory occupation and not affecting the overall system efficiency.

[0099] The primary function of the second processor is to perform matrix convolution operations. Taking the VGG16 network as an example, VGG16 is a commonly used 16-layer convolutional neural network. The input to the first layer of the neural network algorithm is a 224*224 RGB color image. Because the RGB colors are processed through three channels, with each channel representing only one color, the input is transformed into 224*224*3 data points. The kernel size of the first convolutional layer is 3*3. Since convolution is performed on each of the three channels, there are three 3*3 kernels, each with a size of 3*3*3, for a total of 64 such kernels. Therefore, the number of weight parameters in the first layer of the neural network algorithm is 3*3*3*64. Each kernel also has a bias parameter, which is added to each number in the result matrix of the convolution kernel. So, the number of operation parameters for the first layer is 3*3*3*64 + 64 = 1792. When the second processor runs, it first extracts the first convolution kernel and performs the convolution operation on the input image. Each convolutional kernel is 3x3, and the input image size is 224x224. The convolutional kernels perform convolution operations point by point on the image. Each convolution operation is a 3x3 multiplication and addition operation, and each image is processed 224x224 times. The computational cost per channel is 3x3x224x224, and for three channels it is 3x3x224x224x3. There are 64 such convolutional kernels. Therefore, the computational cost of the first layer of the neural network algorithm is 3x3x224x224x3x64 = 86,704,128, approximately 86.7M of computation. However, the computing power of the second processor in an embedded system is limited, sometimes only having 256 or even 128 MACs. If there are only 128 MACs, then only 4 convolutional kernels can be used for computation, and 64 convolutional kernels would require 16 rounds to complete all the computations.

[0100] The characteristic of a convolutional neural network is that the input of the current layer is convolved to obtain the output, and the output of the current layer becomes the input of the next layer. During computation, the input of the current layer remains unchanged; the second processor simply uses different convolution kernels to produce different outputs. The second processor uses the convolution kernel of the first layer to convolve the original data, obtaining the result of the first layer's computation. When the first layer is fully computed, its output is used as the input of the second layer. At this point, the input of the first layer (i.e., the original data) is no longer needed and can be released, as are the computation parameters and convolution kernels. The second processor then uses the convolution kernel of the second layer to convolve the output of the first layer, obtaining the result of the second layer's computation. When the second layer is fully computed, its output is used as the input of the third layer. Again, the input of the second layer (i.e., the output of the first layer) is no longer needed and can be released, as are the computation parameters and convolution kernels. This process continues until the current layer's computation is complete, at which point the input, convolution kernel, and computation parameters of the current layer are no longer needed and can be released.

[0101] Based on this consideration, the first cache module can read the operation parameters from the first memory in any of the following ways in the embodiments of this disclosure:

[0102] In one example, the first cache module can obtain the calculation parameters required for the current operation and the next operation of the second processor from the first memory, so that the second processor can quickly read the calculation parameters required for the current operation without having to put all the calculation parameters into the system memory, thus saving a lot of system memory space and thus saving costs.

[0103] In one example, the first cache module can be configured in first-in, first-out (FIFO) mode. In another example, the first cache module can be configured in ping-pong mode.

[0104] In one example, the first cache module includes multiple cache units configured to be used alternately. For instance, the first cache module includes a first cache unit and a second cache unit, configured to be used alternately. That is, the second processor alternately reads a first operation parameter from the first cache unit and a second operation parameter from the second cache unit, where the first operation parameter is the operation parameter currently stored in the first cache unit, and the second operation parameter is the operation parameter currently stored in the second cache unit. After the first operation parameter is read, the first cache module clears the first cache unit, retrieves the next operation parameter from the first memory, and stores it in the first cache unit. After the second operation parameter is read, the first cache module clears the second cache unit, retrieves the next operation parameter from the first memory, and stores it in the first cache unit.

[0105] In one example, the first cache module obtains the operation parameters based on the first address information, including: the first cache module obtains the operation parameters from the first memory in stages according to the reading progress of the second processor based on the first address information.

[0106] In one example, the first cache module retrieves the operation parameters from the first memory in stages according to the first address information and the reading progress of the second processor, including: the first cache module retrieves the operation parameters from the first memory in a first-in-first-out or ping-pong manner according to the first address information.

[0107] In one example, the first cache module transfers the operation parameters from the first memory to the second processor in a first-in, first-out (FIFO) manner based on the first address information. After obtaining the first address information, the first cache module automatically reads the operation parameters stored in the first memory via the system bus. When the second processor reads the operation parameters required for the current operation from the first cache module, the first cache module automatically discards the operation parameters read by the second processor and reads the operation parameters required for subsequent operations from the first memory, so that the second processor can quickly read the required operation parameters in subsequent accesses.

[0108] In one example, the first cache module transfers the operation parameters from the first memory to the second processor in a ping-pong manner based on the first address information. For example: Assuming the first cache module is 4KB in size, the first processor sends the first address information to the first cache module, and the first cache module automatically loads 4KB of operation parameters from the first memory based on the first address information. When the second processor starts working, it first reads a portion of the operation parameters for calculation. Assuming the second processor reads the first 2KB of operation parameters, once it has finished reading the first 2KB, the first cache module knows that the second processor has already read these 2KB of operation parameters, and will then read another 2KB of operation parameters from the first memory to overwrite the first 2KB of operation parameters previously read by the second processor. At this point, the 4KB of operation parameters in the first cache module are all operation parameters that the second processor has not yet read. Similarly, after the second processor has read the last 2KB of operation parameters, the first cache module will read another 2KB of operation parameters from the first memory to overwrite the last 2KB of operation parameters previously read by the second processor. The second processor alternately reads the first 2KB and then the last 2KB of computational parameters from the first cache module. The first cache module continuously loads new computational parameters from the first memory, overwriting the older parameters already read by the second processor. This ensures that the second processor always reads the computational parameters for the current layer of the neural network algorithm. This allows the second processor to operate continuously without storing all computational parameters in system memory, saving significant memory space and reducing costs. Furthermore, it saves considerable time spent reading computational parameters from the first memory during system initialization, further reducing costs and improving efficiency.

[0109] In one example, the two cache units of the first cache module ping-pong load the computational parameters of the current layer's neural network algorithm and the next layer's neural network algorithm, respectively. After the current layer's neural network algorithm finishes its computation, its computational parameters are cleared and the computational parameters of the layer after that are loaded, thus enabling the convolutional neural network to run in a pipelined manner.

[0110] In one example, if the computational parameters of a single-layer neural network algorithm are too large, the parameters can be ping-pong loaded individually. The second processor has limited computing power; even if all the computational parameters of a single-layer neural network algorithm are loaded at once, the second processor will still perform the computations individually. Therefore, ping-pong loading can be used to process the computational parameters of a single-layer neural network algorithm.

[0111] Through the above examples, the first cache module can load only a small number of calculation parameters each time, which reduces the storage capacity requirement of the first cache module and can further reduce costs.

[0112] See Figures 2(b)-2(d) As shown in the figure, the neural network algorithm processing system of this disclosure embodiment may further include a second cache module. The second cache module is connected to the second processor. The second cache module is used to store intermediate data generated during the operation of the neural network algorithm in the second processor.

[0113] In one example, the second processor has a first interface circuit, and the second cache module has a second interface circuit. The second interface circuit is connected to the first interface circuit and has the same bit width. When the second processor performs convolution operations, it simultaneously performs multiplication and addition operations on a large amount of data. For example, with 128 MAC units, it can perform 128 pairs of multiplication and addition operations simultaneously, which places higher demands on data bandwidth. In this embodiment, a second cache module is placed outside the second processor. The second processor has a first interface circuit, and the second cache module has a second interface circuit. The second interface circuit is connected to the first interface circuit and has the same bit width. This allows the second processor to read the intermediate data it needs within one clock cycle without needing to perform high-bit-width to low-bit-width conversion, thereby significantly improving the data reading speed and increasing the efficiency of the second processor. At the same time, this intermediate data does not need to pass through the system bus, reducing the load on the system bus and improving the efficiency of the entire system, thus increasing the system response speed. Based on this concept, in one example, a bit-width converter can be omitted in the second processor, thus reducing the cost of the second processor.

[0114] The neural network algorithm processing system of this disclosure, based on the first cache module and the second cache module, can optimize the problems of increased cost due to excessive system memory usage and efficiency impacted by excessive access volume. Thus, it can achieve a system with a larger computational load with lower system memory cost, increase product competitiveness, and accelerate product launch.

[0115] In this embodiment of the disclosure, the processing system for the neural network algorithm can be an embedded system. In another example, the neural network system can also be applied to a non-embedded system.

[0116] The following describes the data processing method provided in this disclosure, which can be applied to the processing system of the neural network algorithm in the foregoing embodiments.

[0117] <First Embodiment>

[0118] The neural network algorithm processing system includes a first processor, a second processor, a first memory, a first cache module, and a system bus. The first processor and the first memory are both connected to the system bus. The second processor is connected to the system bus through the first cache module. The first memory is used to store the computational parameters of the neural network algorithm.

[0119] Referring to Figure 3(a), the first embodiment of this disclosure provides a data processing method, which includes steps S106-S110.

[0120] Step S106: The first processor sends first address information to the first cache module. The first address information is the address information of the operation parameters in the first memory.

[0121] Step S108: The first cache module obtains the operation parameters from the first memory according to the first address information.

[0122] Step S110: The second processor reads the operation parameters through the first cache module and performs neural network algorithm operations based on the operation parameters.

[0123] In the data processing method of this embodiment, the first cache module obtains the operation parameters of the neural network algorithm from the first memory, and the second processor reads the operation parameters through the first cache module and performs the operation of the neural network algorithm according to the operation parameters, thereby reducing the occupation of system memory and not affecting the overall efficiency of the system.

[0124] <Second Embodiment>

[0125] The neural network algorithm processing system includes a first processor, a second processor, a first memory, a first cache module, and a system bus. The first processor and the first memory are both connected to the system bus. The second processor is connected to the system bus through the first cache module. The first memory stores the computational parameters of the neural network algorithm. The first memory also stores the raw data, which is the input data for the neural network algorithm.

[0126] Referring to 3(b), a second embodiment of this disclosure provides a data processing method, which includes steps S206-S210.

[0127] Step S206: The first processor sends first address information to the first cache module. The first address information is the address information of the operation parameters in the first memory. The first processor sends second address information to the first cache module. The second address information is the address information of the original data in the first memory.

[0128] Step S208: The first cache module retrieves the operation parameters from the first memory according to the first address information. The first cache module retrieves the raw data from the first memory according to the second address information.

[0129] Step S210: The second processor reads the operation parameters and raw data through the first cache module, and performs neural network algorithm operations based on the operation parameters and raw data.

[0130] In the data processing method of this embodiment, the first cache module obtains the operation parameters and raw data of the neural network algorithm from the first memory, and the second processor reads the operation parameters and raw data through the first cache module and performs the operation of the neural network algorithm according to the operation parameters and raw data, thereby reducing the occupation of system memory and not affecting the overall efficiency of the system.

[0131] <Third Embodiment>

[0132] The neural network algorithm processing system includes a first processor, a second processor, a first memory, a first cache module, system memory, and a system bus. The first processor, the first memory, and the system memory are all connected to the system bus. The second processor is connected to the system bus through the first cache module. The first memory stores the computational parameters of the neural network algorithm. The system memory stores the raw data, which is the input data for the neural network algorithm.

[0133] Referring to 3(c), a third embodiment of this disclosure provides a data processing method, which includes steps S306-S310.

[0134] Step S306: The first processor sends first address information to the first cache module. The first address information is the address information of the operation parameters in the first memory. The first processor sends second address information to the first cache module. The second address information is the address information of the original data in the system memory.

[0135] Step S308: The first cache module retrieves the operation parameters from the first memory according to the first address information. The first cache module retrieves the raw data from the system memory according to the second address information.

[0136] Step S310: The second processor reads the operation parameters and raw data through the first cache module, and performs neural network algorithm operation based on the operation parameters and raw data.

[0137] In the data processing method of this embodiment, the first cache module obtains the operation parameters of the neural network algorithm from the first memory, and the second processor reads the operation parameters through the first cache module and performs the operation of the neural network algorithm according to the operation parameters, thereby reducing the occupation of system memory and not affecting the overall efficiency of the system.

[0138] The neural network algorithm processing system may also include peripherals, and the raw data required by the second processor to perform the neural network algorithm operation comes from the peripherals. Peripherals may be, for example, radar or cameras. Specifically, the peripherals are connected to the system bus, collect external information and generate raw data, and the first processor or the first cache module stores the raw data generated by the peripherals through the system bus, for example, in system memory or a first storage device. After storage, the first cache module is also used to retrieve the raw data from system memory or the first storage device. The second processor is used to read the operation parameters and raw data through the first cache module, and perform the neural network algorithm operation based on the operation parameters and raw data. Based on this, this disclosure proposes a fourth and a fifth embodiment of the data processing method.

[0139] <Fourth Embodiment>

[0140] The neural network algorithm processing system includes a first processor, a second processor, a first memory, a first cache module, peripherals, and a system bus. The first processor, the first memory, and the peripherals are all connected to the system bus. The second processor is connected to the system bus through the first cache module. The first memory is used to store the computational parameters of the neural network algorithm.

[0141] Referring to 3(d), a fourth embodiment of this disclosure provides a data processing method, which includes steps S202-S210.

[0142] Step S202: The peripheral device collects external information and generates raw data, which is the input data of the neural network algorithm.

[0143] Step S204: The first processor stores the raw data generated by the peripheral device in the first memory via the system bus.

[0144] Step S206: The first processor sends first address information to the first cache module. The first address information is the address information of the operation parameters in the first memory. The first processor sends second address information to the first cache module. The second address information is the address information of the original data in the first memory.

[0145] Step S208: The first cache module retrieves the operation parameters from the first memory according to the first address information. The first cache module retrieves the raw data from the first memory according to the second address information.

[0146] Step S210: The second processor reads the operation parameters and raw data through the first cache module, and performs neural network algorithm operations based on the operation parameters and raw data.

[0147] In the data processing method of this embodiment, the first cache module obtains the operation parameters and raw data of the neural network algorithm from the first memory, and the second processor reads the operation parameters and raw data through the first cache module and performs the operation of the neural network algorithm according to the operation parameters and raw data, thereby reducing the occupation of system memory and not affecting the overall efficiency of the system.

[0148] <Fifth Embodiment>

[0149] The processing system for this neural network algorithm includes a first processor, a second processor, a first memory, a first cache module, system memory, peripherals, and a system bus. The first processor, first memory, system memory, and peripherals are all connected to the system bus. The second processor is connected to the system bus through the first cache module. The first memory is used to store the computational parameters of the neural network algorithm.

[0150] Referring to 3(e), a fifth embodiment of this disclosure provides a data processing method, which includes steps S302-S310.

[0151] Step S302: The peripheral device collects external information and generates raw data, which is the input data of the neural network algorithm.

[0152] Step S304: The first processor stores the raw data generated by the peripheral device in the system memory via the system bus.

[0153] Step S306: The first processor sends first address information to the first cache module. The first address information is the address information of the operation parameters in the first memory. The first processor sends second address information to the first cache module. The second address information is the address information of the original data in the system memory.

[0154] Step S308: The first cache module retrieves the operation parameters from the first memory according to the first address information. The first cache module retrieves the raw data from the system memory according to the second address information.

[0155] Step S310: The second processor reads the operation parameters and raw data through the first cache module, and performs neural network algorithm operation based on the operation parameters and raw data.

[0156] In the data processing method of this embodiment, the first cache module obtains the operation parameters of the neural network algorithm from the first memory, and the second processor reads the operation parameters through the first cache module and performs the operation of the neural network algorithm according to the operation parameters, thereby reducing the occupation of system memory and not affecting the overall efficiency of the system.

[0157] In the above method embodiments, the process by which the first cache module obtains the operation parameters from the first memory according to the first address information can be as follows: the first cache module obtains the operation parameters from the first memory in stages according to the first address information and the reading progress of the second processor. For example, the first cache module obtains the operation parameters from the first memory in a first-in-first-out or ping-pong manner according to the first address information and the reading progress of the second processor. Two specific examples are given below.

[0158] <Example 1>

[0159] In steps S110, S210, and S310, the second processor reads the first target operation parameter from the first cache module, wherein the first target operation parameter is the operation parameter in the first cache module that was stored earliest.

[0160] In steps S108, S208, and S308, after the first target operation parameter is read, the first cache module deletes the first target operation parameter, retrieves the next operation parameter of the second target operation parameter from the first memory, and stores it. The second target operation parameter is the operation parameter in the first cache module that was stored the latest.

[0161] In other words, when the second processor reads the operation parameters from the first cache module, it reads the earliest operation parameter stored in the first cache module, which is the first target operation parameter. After the first target operation parameter is read by the second processor, the first cache module deletes the first target operation parameter and retrieves the next operation parameter after the latest operation parameter stored in the first cache module from the first memory, which is the second target operation parameter.

[0162] In the approach described in Example 1, the second processor automatically reads the necessary computation parameters from the first cache module based on its computation progress. This eliminates the need to store all computation parameters in system memory, saving significant memory space and reducing overall cost. Furthermore, it saves considerable time spent reading data from the first memory during system initialization, further reducing both cost and efficiency.

[0163] The data processing method of this disclosure can release the no longer needed operation parameters in the first cache module in a timely manner, so that a smaller first cache module can be used to load operation parameters, further reducing the overall cost.

[0164] <Example 2>

[0165] In this example 2, the first cache module includes a first cache unit and a second cache unit.

[0166] In steps S110, S210, and S310, the second processor alternately reads the first operation parameter from the first cache unit and the second operation parameter from the second cache unit, wherein the first operation parameter is the operation parameter currently stored in the first cache unit, and the second operation parameter is the operation parameter currently stored in the second cache unit.

[0167] In steps S108, S208, and S308, after the first operation parameter is read, the first cache module clears the first cache unit, retrieves the next operation parameter of the second operation parameter from the first memory, and stores it in the first cache unit. After the second operation parameter is read, the first cache module clears the second cache unit, retrieves the next operation parameter of the first operation parameter from the first memory, and stores it in the first cache unit.

[0168] In this approach of Example 2, the first cache module includes a first cache unit and a second cache unit. The second processor alternately reads the required operation parameters from the first cache unit and the second cache unit according to its operation progress. This eliminates the need to store all operation parameters in system memory, saving significant memory space and reducing system memory usage, thus lowering costs. Furthermore, it also saves a considerable amount of time spent reading data from the first memory during system initialization, further reducing costs and improving efficiency.

[0169] The data processing method of this disclosure embodiment uses a first cache module to process operation parameters in a ping-pong manner, which can release operation parameters that are no longer needed in the first cache module in a timely manner. This allows a smaller first cache module to be used to load operation parameters, further reducing the overall cost.

[0170] In a neural network algorithm processing system that includes a second cache module connected to a second processor, the second cache module can be used to store intermediate data generated during the neural network algorithm's computation in the second processor. Based on this, see [link to relevant documentation]. Figure 4 As shown, the process of the second processor performing neural network algorithm operations may include steps S402-S406.

[0171] Step S402: During the operation of the current layer neural network algorithm, the second processor stores the first intermediate data in the second cache module. The first intermediate data is the intermediate data generated by the second processor during the operation of the current layer neural network algorithm.

[0172] Step S404: After the operation of the neural network algorithm in the current layer is completed, the second processor reads the first intermediate data from the second cache module.

[0173] Step S406: The second processor uses the first intermediate data it reads as the input data for the next layer of the neural network algorithm and performs the operation of the next layer of the neural network algorithm.

[0174] In one example, in step S402, the second processor stores the first intermediate data in the second cache module in the form of overwriting the second intermediate data, wherein the second intermediate data is the intermediate data generated by the second processor in the operation of the previous layer neural network algorithm.

[0175] The data processing method of this disclosure embodiment does not require storing intermediate data generated by the neural network algorithm in the system memory via the system bus, but instead stores it in the second cache module, thereby reducing the load on the system bus and system memory, improving the efficiency of the entire system, and increasing the system response speed.

[0176] In one example, before starting the neural network algorithm, the first processor sends a configuration file to the first cache module via the system bus. This configuration file may include the instructions for each layer of the neural network algorithm. It may also include the parameter values ​​for each layer. The first cache module can then send the configuration file to the second processor.

[0177] The second processor performs the calculations for each layer of the neural network algorithm according to the instructions of each layer. After the calculations for each layer of the neural network algorithm are completed, the second processor can release the instructions for that layer of the neural network algorithm.

[0178] The first cache module can sequentially retrieve the computational parameters required by each layer of the neural network algorithm from the first memory, according to the number of parameters required for each layer. In this example, the first cache module knows the address information of the computational parameters in the first memory and the number of parameters required by each layer of the neural network algorithm. After reading the computational parameters from the first memory once, it can increment the address range according to the number of parameters to be read next time, thereby determining the address range for the next reading of computational parameters from the first memory, so as to facilitate the next reading of computational parameters from the first memory.

[0179] The following is a specific example illustrating the data processing method mentioned in the above embodiments. In this example, the first cache module has a first cache unit and a second cache unit.

[0180] The first processor sends a configuration file to the first cache module via the system bus. The configuration file includes the parameter values ​​and instructions for each layer of the neural network algorithm. The second processor retrieves the configuration file from the first cache module.

[0181] The first processor sends first address information, which is the address information of the operation parameters in the first memory, to the first cache module via the system bus. The first address information is the address information of the original data in the system memory, which is the address information of the original data.

[0182] The first cache module retrieves raw data from system memory and sends it to the second processor based on the second address information via the system bus.

[0183] The first cache module retrieves the operation parameters of the first-layer neural network algorithm from the first memory based on the first address information and the number of parameters of the first-layer neural network algorithm, and stores the operation parameters of the first-layer neural network algorithm in the first cache unit. The first cache module then retrieves the operation parameters of the second-layer neural network algorithm from the first memory based on the number of parameters of the second-layer neural network algorithm, and stores the operation parameters of the second-layer neural network algorithm in the second cache unit.

[0184] The second processor reads the operation parameters of the first-layer neural network algorithm from the first cache module. Based on these parameters, it performs the first-layer neural network algorithm operation on the original data to obtain the output data of the first-layer neural network algorithm. This output data is then stored in the second cache module. After the second processor reads the operation parameters from the first cache module, the first cache module clears its cache units, that is, it removes the operation parameters of the first-layer neural network algorithm that have been read by the second processor. Then, based on the number of parameters for the third-layer neural network algorithm, it retrieves the operation parameters for the third-layer neural network algorithm from the first memory and stores them in the first cache unit.

[0185] After the first layer of the neural network algorithm is completed, the second processor reads the operation parameters of the second layer from the first cache module and the output data of the first layer from the second cache module. Based on the read operation parameters, it performs the second layer operation on the output data of the first layer, obtaining the output data of the second layer. This output data is then stored in the second cache module. After the second processor reads the operation parameters of the second layer from the first cache module, the first cache module clears the second cache unit, that is, it removes the operation parameters of the second layer that have been read by the second processor. Then, based on the number of parameters for the fourth layer of the neural network algorithm, it retrieves the operation parameters of the fourth layer from the first memory and stores them in the second cache unit.

[0186] After the second layer of the neural network algorithm is completed, the second processor reads the operation parameters of the third layer from the first cache module and the output data of the second layer from the second cache module. Based on the read operation parameters, it performs the third layer operation on the output data of the second layer, obtaining the output data of the third layer. This output data of the third layer is then stored in the second cache module, overwriting the output data of the first layer. After the second processor reads the operation parameters of the third layer from the first cache module, the first cache module clears its first cache unit, that is, it removes the operation parameters of the third layer that have been read by the second processor. Then, based on the number of parameters for the fifth layer, it retrieves the operation parameters of the fifth layer from the first memory and stores them in the first cache unit.

[0187] After the computation of the third layer neural network algorithm is completed, the second processor reads the computation parameters of the fourth layer neural network algorithm from the first cache module and the output data of the third layer neural network algorithm from the second cache module. Based on the read computation parameters, it performs the fourth layer neural network algorithm computation on the output data of the third layer neural network algorithm to obtain the output data of the fourth layer neural network algorithm. This output data of the fourth layer neural network algorithm is then stored in the second cache module, overwriting the output data of the second layer neural network algorithm. After the second processor reads the computation parameters of the fourth layer neural network algorithm from the first cache module, the first cache module clears the second cache unit, that is, it clears the computation parameters of the fourth layer neural network algorithm that have been read by the second processor. Then, based on the number of parameters of the sixth layer neural network algorithm, it retrieves the computation parameters of the sixth layer neural network algorithm from the first memory and stores them in the second cache unit.

[0188] By analogy, we can conclude that:

[0189] After the second processor reads the operation parameters of the Nth layer neural network algorithm from the first cache module, the first cache module clears the target cache unit, which is the cache unit where the operation parameters of the Nth layer neural network algorithm are located. Then, the first cache module retrieves the operation parameters of the (N+2)th layer neural network algorithm from the first memory and stores the operation parameters of the (N+2)th layer neural network algorithm in the target cache unit. Here, N is an integer greater than or equal to 1.

[0190] After the N+1 layer neural network algorithm completes its operation, the second processor reads the operation parameters of the N+2 layer neural network algorithm from the first cache module and the output data of the N+1 layer neural network algorithm from the second cache module. Based on the read operation parameters, it performs the N+2 layer neural network algorithm operation on the output data of the N+1 layer neural network algorithm to obtain the output data of the N+2 layer neural network algorithm. The output data of the N+2 layer neural network algorithm is then stored in the second cache module, overwriting the output data of the Nth layer neural network algorithm. Here, N is an integer greater than or equal to 1.

[0191] The data processing method of this disclosure, based on the first cache module and the second cache module, can optimize the problems of increased cost due to excessive system memory usage and reduced efficiency due to excessive access volume. This allows for the implementation of a system with a larger computational load at a lower system memory cost, thereby increasing product competitiveness and accelerating product launch.

[0192] The data processing method of this disclosure can be implemented based on the neural network algorithm processing system in the foregoing embodiments. Related parts can be referred to each other and will not be described again here.

[0193] This disclosure also provides an electronic device including a processing system for the neural network algorithm disclosed in any of the foregoing embodiments.

[0194] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. For the electronic device embodiments, relevant details can be found in the description of the neural network algorithm processing system embodiments.

[0195] The foregoing has described specific embodiments of this specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims may be performed in a different order than that shown in the embodiments and may still achieve the desired result. Furthermore, the processes depicted in the drawings do not necessarily require the specific or sequential order shown to achieve the desired result. In some embodiments, multitasking and parallel processing are possible or may be advantageous.

[0196] Embodiments of this specification may be systems, methods, and / or computer program products. A computer program product may include a computer-readable storage medium having computer instructions stored thereon for causing a processor to implement various aspects of the embodiments of this specification.

[0197] Computer-readable storage media can be tangible devices capable of holding and storing computer instructions for use by computer instruction execution devices. Computer-readable storage media can be, for example—but not limited to—electrical storage devices, magnetic storage devices, optical storage devices, electromagnetic storage devices, semiconductor storage devices, or any suitable combination thereof. More specific examples (a non-exhaustive list) of computer-readable storage media include: portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), static random access memory (SRAM), portable compact disc read-only memory (CD-ROM), digital multifunction disc (DVD), memory sticks, floppy disks, mechanical encoding devices, such as punch cards or recessed protrusions storing computer instructions thereon, and any suitable combination thereof. The computer-readable storage media used herein are not to be construed as transient signals themselves, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through waveguides or other transmission media (e.g., light pulses through fiber optic cables), or electrical signals transmitted through wires.

[0198] The computer instructions described herein can be downloaded from computer-readable storage media to various computing / processing devices, or downloaded via a network, such as the Internet, local area network, wide area network, and / or wireless network, to an external computer or external storage device. The network may include copper cables, fiber optic cables, wireless transmission, routers, firewalls, switches, gateway computers, and / or edge servers. A network adapter card or network interface in each computing / processing device receives computer instructions from the network and forwards them to computer-readable storage media within the respective computing / processing device.

[0199] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this specification. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of computer instructions, which contains one or more executable computer instructions for implementing a specified logical function. In some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, may be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions. It will be known to those skilled in the art that implementation in hardware, implementation in software, and implementation using a combination of software and hardware are equivalent.

[0200] Various embodiments of this specification have been described above. These descriptions are exemplary and not exhaustive, nor are they limited to the disclosed embodiments. Many modifications and variations will be apparent to those skilled in the art without departing from the scope of the described embodiments. The terminology used herein is chosen to best explain the principles, practical application, or improvement of the technology in the market, or to enable others skilled in the art to understand the embodiments disclosed herein.

Claims

1. A processing system for a neural network algorithm, characterized in that, The processing system includes a first processor, a second processor, a first memory, a first cache module, and a system bus; The first processor and the first memory are respectively connected to the system bus; The second processor is connected to the system bus through the first cache module; The first memory is used to store the computational parameters of the neural network algorithm; The first processor is used to send first address information to the first cache module, wherein the first address information is the address information of the operation parameter in the first memory; The first cache module is used to obtain the operation parameters from the first memory according to the first address information; The second processor is used to read the operation parameters through the first cache module and perform the operation of the neural network algorithm according to the operation parameters.

2. The processing system for the neural network algorithm according to claim 1, characterized in that, The second processor includes control logic circuitry and arithmetic logic circuitry; The control logic circuit is used to read the operation parameters through the first cache module and send the operation parameters to the operation logic circuit; The arithmetic logic circuit is used to perform the neural network algorithm operation according to the arithmetic parameters.

3. The processing system for the neural network algorithm according to claim 1, characterized in that, The second processor is a neural network processor; the first memory is a non-volatile memory.

4. The processing system for the neural network algorithm according to claim 1, characterized in that, The processing system of the neural network algorithm includes system memory, which is connected to the system bus; The system memory is used to store raw data, which is the input data of the neural network algorithm; The first processor is used to send second address information to the first cache module, wherein the second address information is the address information of the original data in the system memory; The first cache module is used to retrieve the original data from the system memory according to the second address information; The second processor is used to read the original data through the first cache module and perform the neural network algorithm operation according to the operation parameters and the original data.

5. The processing system for the neural network algorithm according to claim 1, characterized in that, The first memory is used to store raw data, which is the input data of the neural network algorithm; The first processor is used to send second address information to the first cache module, wherein the second address information is the address information of the original data in the first memory; The first cache module is used to retrieve the original data from the first memory according to the second address information; The second processor is used to read the original data through the first cache module and perform the neural network algorithm operation according to the operation parameters and the original data.

6. The processing system for the neural network algorithm according to claim 4 or 5, characterized in that, The second processor includes control logic circuitry and arithmetic logic circuitry; The control logic circuit is used to read the operation parameters and the original data through the first cache module, and send the operation parameters and the original data to the operation logic circuit; The computational logic circuit is used to perform the neural network algorithm operation based on the computational parameters and the original data.

7. The processing system for the neural network algorithm according to claim 4 or 5, characterized in that, The processing system includes peripherals, which are connected to the system bus. The peripheral device is used to collect external information and generate the raw data; The first processor or the first cache module is used to store the raw data generated by the peripheral device via the system bus.

8. The processing system for the neural network algorithm according to claim 1, characterized in that, The first cache module includes a DMA circuit.

9. The processing system for the neural network algorithm according to claim 1, characterized in that, The first cache module is configured to either first-in-first-out (FIFO) mode or ping-pong mode.

10. The processing system for the neural network algorithm according to claim 1, characterized in that, The first cache module includes multiple cache units, which are configured to be used alternately.

11. The processing system for the neural network algorithm according to claim 1, characterized in that, The first cache module obtains the operation parameters based on the first address information, including: The first cache module retrieves the operation parameters from the first memory in stages according to the first address information and the reading progress of the second processor.

12. The processing system for the neural network algorithm according to claim 11, characterized in that, The first cache module retrieves the operation parameters from the first memory in stages according to the first address information and the read progress of the second processor, including: The first cache module retrieves the operation parameters from the first memory in a first-in-first-out or ping-pong manner based on the first address information.

13. The processing system for the neural network algorithm according to claim 1, characterized in that, The processing system includes a second cache module; The second cache module is connected to the second processor; The second cache module is used to store intermediate data generated during the operation of the neural network algorithm in the second processor.

14. The processing system for the neural network algorithm according to claim 13, characterized in that, The second processor has a first interface circuit, and the second cache module has a second interface circuit. The second interface circuit is connected to the first interface circuit and has the same bit width.

15. The processing system for the neural network algorithm according to any one of claims 1-5, characterized in that, The first processor is a microcontroller unit.

16. A data processing method, characterized in that, A processing system for neural network algorithms includes a first processor, a second processor, a first memory, a first cache module, and a system bus; the first processor and the first memory are respectively connected to the system bus. The second processor is connected to the system bus via the first cache module; The first memory is used to store the computational parameters of the neural network algorithm; The data processing method includes: The first processor sends first address information to the first cache module, wherein the first address information is the address information of the operation parameter in the first memory; The first cache module obtains the operation parameters from the first memory based on the first address information; The second processor reads the operation parameters through the first cache module and performs the neural network algorithm operation based on the operation parameters.

17. The data processing method according to claim 16, characterized in that, The processing system of the neural network algorithm includes system memory, which is connected to the system bus; The system memory is used to store raw data, which is the input data of the neural network algorithm; The data processing method includes: The first processor sends second address information to the first cache module, the second address information being the address information of the original data in the system memory; The first cache module retrieves the original data from the system memory based on the second address information; The second processor reads the original data through the first cache module and performs the neural network algorithm operation based on the operation parameters and the original data.

18. The data processing method according to claim 16, characterized in that, The first memory is used to store raw data, which is the input data of the neural network algorithm; The data processing method includes: The first processor sends second address information to the first cache module, the second address information being the address information of the original data in the first memory; The first cache module retrieves the original data from the first memory based on the second address information; The second processor reads the original data through the first cache module and performs the neural network algorithm operation based on the operation parameters and the original data.

19. The data processing method according to claim 17 or 18, characterized in that, The processing system of the neural network algorithm includes peripherals, which are connected to the system bus. The data processing method includes: The peripheral device collects external information and generates the raw data; The first processor or the first cache module stores the raw data generated by the peripheral device through the system bus.

20. The data processing method according to claim 16, characterized in that, The first cache module obtains the operation parameters based on the first address information, including: The first cache module retrieves the operation parameters from the first memory in stages according to the first address information and the reading progress of the second processor.

21. The data processing method according to claim 20, characterized in that, The first cache module retrieves the operation parameters from the first memory in stages according to the first address information and the read progress of the second processor, including: The first cache module retrieves the operation parameters from the first memory in a first-in-first-out or ping-pong manner based on the first address information.

22. The data processing method according to claim 20, characterized in that, The second processor reads the operation parameters through the first cache module, including: The second processor reads the first target operation parameter from the first cache module, wherein the first target operation parameter is the operation parameter in the first cache module that has been stored the earliest. The first cache module retrieves the operation parameters from the first memory in stages according to the first address information and the read progress of the second processor, including: After the first target operation parameter is read, the first cache module deletes the first target operation parameter, retrieves the next operation parameter of the second target operation parameter from the first memory, and stores it. The second target operation parameter is the operation parameter in the first cache module that was stored the latest.

23. The data processing method according to claim 20, characterized in that, The first cache module includes a first cache unit and a second cache unit; The second processor reads the operation parameters through the first cache module, including: The second processor alternately reads a first operation parameter from the first cache unit and a second operation parameter from the second cache unit, wherein the first operation parameter is the operation parameter currently stored in the first cache unit, and the second operation parameter is the operation parameter currently stored in the second cache unit; The first cache module retrieves the operation parameters from the first memory in stages according to the first address information and the read progress of the second processor, including: After the first operation parameter is read, the first cache module clears the first cache unit, retrieves the next operation parameter of the second operation parameter from the first memory, and stores it in the first cache unit; After the second operation parameter is read, the first cache module clears the second cache unit, retrieves the next operation parameter from the first memory, and stores it in the first cache unit.

24. The data processing method according to claim 16, characterized in that, The processing system of the neural network algorithm includes a second cache module; the second cache module is connected to the second processor; The data processing method includes: During the operation of the neural network algorithm of the current layer, the second processor stores the first intermediate data in the second cache module, wherein the first intermediate data is the intermediate data generated by the second processor during the operation of the neural network algorithm of the current layer; After the second processor finishes the operation of the neural network algorithm in the current layer, it reads the first intermediate data from the second cache module; The second processor uses the first intermediate data it reads as input data for the next layer of the neural network algorithm and performs the operation of the next layer of the neural network algorithm.

25. The data processing method according to claim 24, characterized in that, During the computation of the neural network algorithm in the current layer, the second processor stores the first intermediate data in the second cache module, including: The second processor stores the first intermediate data in the second cache module in a form that overwrites the second intermediate data, wherein the second intermediate data is the intermediate data generated by the second processor during the operation of the neural network algorithm of the previous layer.

26. An electronic device, characterized in that, A processing system including the neural network algorithm described in any one of claims 1-15.

Images