Data processing method and apparatus, terminal device, and storage medium

Abstract

The embodiment of the application provides a data processing method and device, terminal equipment and storage medium, the data processing method comprises the following steps: an NPU determines M SRAMs allocated to to-be-input data according to the channel number of the to-be-input data, M is less than or equal to N, M is an integer multiple of the channel number of the to-be-input data; the NPU reads overlapping data in first channel data from a system cache through a bus, and loads the overlapping data into P SRAMs corresponding to the first channel, the first channel data comprising data of any channel in the to-be-input data; the NPU reads non-overlapping data in the first channel data from a DRAM through the bus, and loads the non-overlapping data into a corresponding SRAM; the reading power consumption of the system cache is less than the reading power consumption of the DRAM; and the NPU performs data processing on the data loaded in the P SRAMs through a data processing unit matrix. The embodiment of the application can reduce the power consumption of the NPU.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence technology, specifically to a data processing method, apparatus, terminal device, and storage medium. Background Technology

[0002] In current mobile phones, tablets, and other terminal devices, to enhance their artificial intelligence capabilities, a Neural Network Processing Unit (NPU) is typically integrated. The NPU is a storage-intensive processor. In the initial stages of NPU computation, a large amount of data is transferred to the NPU via a bus. The hardware design usually includes the NPU, the bus, and Dynamic Random Access Memory (DRAM). To ensure sufficient bandwidth for NPU data read and write operations, more bandwidth is provided to the NPU on the bus, requiring either wider DRAM bandwidth or increased DRAM frequency. However, both widening the DRAM bandwidth and increasing the DRAM frequency lead to a significant increase in power consumption. Summary of the Invention

[0003] This application provides a data processing method, apparatus, terminal device, and storage medium that can reduce the power consumption of the NPU.

[0004] A first aspect of this application provides a data processing method applied to a data processing system. The data processing system includes a neural network processor (NPU), a system cache, dynamic random access memory (DRAM), and a bus. The NPU includes a data processing unit matrix and N static random access memories (SRAMs). The method includes:

[0005] The NPU determines the number of SRAMs allocated to the input data based on the number of channels of the input data, where M is less than or equal to N and M is an integer multiple of the number of channels of the input data.

[0006] The NPU reads overlapping data from the system cache of the first channel data through the bus, loads the overlapping data into P SRAMs corresponding to the first channel, Q*P=M, where Q is the number of channels, and the first channel data includes data from any one of the channels in the data to be input.

[0007] The NPU reads non-overlapping data from the first channel data in the DRAM via the bus and loads the non-overlapping data into the corresponding SRAM; the read power consumption of the system cache is less than the read power consumption of the DRAM;

[0008] The NPU processes the data loaded in the P SRAMs through the data processing unit matrix.

[0009] A second aspect of this application provides a data processing apparatus, wherein the method of the data processing apparatus is applied to a data processing system, the data processing system including a neural network processor (NPU), a system cache, dynamic random access memory (DRAM), and a bus, the NPU including a data processing unit matrix and N static random access memories (SRAMs), and the data processing apparatus including:

[0010] The determining unit is used to determine the M SRAMs allocated to the data to be input based on the number of channels of the data to be input, where M is less than or equal to N and M is an integer multiple of the number of channels of the data to be input.

[0011] The reading unit is used to read overlapping data from the system cache of the first channel data through the bus, load the overlapping data into P SRAMs corresponding to the first channel, Q*P=M, where Q is the number of channels, and the first channel data includes data from any one of the channels in the data to be input.

[0012] The read unit is further configured to read non-overlapping data from the first channel data in the DRAM via the bus, and load the non-overlapping data into the corresponding SRAM; the read power consumption of the system cache is less than the read power consumption of the DRAM;

[0013] The data processing unit processes the data loaded in the P SRAMs through the data processing unit matrix.

[0014] A third aspect of this application provides a terminal device including a processor and a memory, the memory being used to store a computer program, the computer program including program instructions, and the processor being configured to invoke the program instructions to execute the step instructions as described in the first aspect of this application.

[0015] A fourth aspect of this application provides a computer-readable storage medium storing a computer program for electronic data interchange, wherein the computer program causes a computer to perform some or all of the steps described in the first aspect of this application.

[0016] A fifth aspect of this application provides a computer program product, wherein the computer program product includes a computer program operable to cause a computer to perform some or all of the steps described in the first aspect of this application. The computer program product may be a software installation package.

[0017] In this embodiment, the data processing method is applied to a data processing system, which includes a neural network processor (NPU), a system cache, dynamic random access memory (DRAM), and a bus. The NPU includes a data processing unit matrix and N static random access memories (SRAMs). The data processing method includes: the NPU determining M SRAMs allocated to the input data based on the number of channels of the input data, where M is less than or equal to N and M is an integer multiple of the number of channels of the input data; the NPU reading overlapping data from the first channel data from the system cache via the bus and loading the overlapping data into P SRAMs corresponding to the first channel, where Q*P = M, Q is the number of channels, and the first channel data includes data from any one channel of the input data; the NPU reading non-overlapping data from the first channel data from the DRAM via the bus and loading the non-overlapping data into the corresponding SRAMs; the read power consumption of the system cache is less than the read power consumption of the DRAM; and the NPU performing data processing on the data loaded in the P SRAMs through the data processing unit matrix. The amount of data stored in an SRAM is limited. Since an SRAM cannot store all the input data required for convolution operations, and due to the characteristics of convolution operations in neural networks, to ensure that each row of data can undergo convolution, adjacent inputs to the SRAM will contain duplicate data, i.e., overlapping data. The overlapping data in the first channel data is the data that needs to be read repeatedly. In the data processing method of this application embodiment, for the overlapping data that needs to be read repeatedly in the first channel data, it can read it from the system cache, which has relatively low power consumption, avoiding reading duplicate data from the relatively high power consumption DRAM. Without widening the DRAM bandwidth or increasing the DRAM frequency, reading the overlapping data that needs to be read repeatedly from the system cache can reduce the NPU's read power consumption while maintaining the NPU's read speed, thereby reducing the NPU's overall power consumption. Attached Figure Description

[0018] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0019] Figure 1 This is a schematic diagram of the structure of a data processing system provided in an embodiment of this application;

[0020] Figure 2 This is a schematic diagram of the structure of an NPU provided in an embodiment of this application;

[0021] Figure 3This is a schematic diagram of the structure of a data processing system provided in an embodiment of this application;

[0022] Figure 4 This is a flowchart illustrating a data processing method provided in an embodiment of this application;

[0023] Figure 5 This is a flowchart illustrating another data processing method provided in an embodiment of this application;

[0024] Figure 6 This is a schematic diagram of a circular cache space in an SRAM provided in an embodiment of this application;

[0025] Figure 7 This is a schematic diagram of a distributed storage provided in an embodiment of this application;

[0026] Figure 8 This is a schematic diagram of the power consumption distribution of a graph calculated using different methods, provided in the embodiments of this application;

[0027] Figure 9 This is a schematic diagram of the structure of a data processing device provided in an embodiment of this application;

[0028] Figure 10 This is a schematic diagram of the structure of a terminal device provided in an embodiment of this application. Detailed Implementation

[0029] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0030] The terms "first," "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish different objects, not to describe a specific order. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or apparatus that includes a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units not listed, or may optionally include other steps or units inherent to these processes, methods, products, or apparatuses.

[0031] In this application, the reference to "embodiment" means that a specific feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of this application. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a mutually exclusive, independent, or alternative embodiment. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described in this application can be combined with other embodiments.

[0032] The terminal devices involved in the embodiments of this application may include various handheld devices, vehicle-mounted devices, wearable devices, computing devices or other processing devices connected to a wireless modem with wireless communication functions, as well as various forms of user equipment (UE), mobile station (MS), terminal device, etc. For ease of description, the devices mentioned above are collectively referred to as terminal devices.

[0033] This application provides a data processing method, apparatus, terminal device, and storage medium that can reduce the power consumption of an NPU. These are described in detail below.

[0034] Please see Figure 1 , Figure 1 This is a schematic diagram of the structure of a data processing system provided in an embodiment of this application. For example... Figure 1 As shown, the data processing system 100 may include a neural network processing unit (NPU) 10, a system buffer (SysBuf) 20, a dynamic random access memory (DRAM) 30, and a bus 40. The NPU 10, system buffer 20, and DRAM 30 are all connected to the bus 40.

[0035] Please see Figure 2 , Figure 2 This is a schematic diagram of the structure of an NPU provided in an embodiment of this application. Figure 2 As shown, the NPU10 includes a data processing unit matrix 11 and an on-chip cache, the on-chip cache including N static random access memories (SRAMs) (e.g., ...). Figure 2 (As shown in 121 to 12N). N SRAMs can read data from system cache 20 or DRAM 30 via the bus. The data processing unit matrix 11 can perform calculations on the data input from the N SRAMs and output the calculation results. Figure 2The data processing unit matrix comprises multiple processing elements (PEs). To facilitate neural network-related calculations, these PEs are arranged in a PE matrix. In NPU architecture design, multi-level caches are typically used to improve system bandwidth. PEs are usually arithmetic logic units (ALUs) with storage, and this storage is called the level 0 cache (L0 cache). Generally, data processed for neural networks (e.g., convolutional neural networks) is divided into multiple channels. In a systolic array architecture for convolutional neural networks, a common approach is to split the on-chip cache into independent small caches corresponding to different channels of SRAM, each corresponding to a different PE to increase bandwidth. Each SRAM corresponds to one row of PE units. As shown in the diagram, SRAM 0 provides data to the first row of PEs, SRAM 2 provides data to the second row of PEs, and SRAM N-1 provides data to the Nth row of PEs. The capacities of the N SRAMs can be equal. Each PE in the data processing unit matrix 11 can also be identical.

[0036] In this embodiment of the application, the system cache 20 may be composed of SRAM, and the storage space of the system cache 20 is larger than the storage space in the NPU 10.

[0037] In this embodiment, when the NPU10 performs neural network operations, it can read overlapping data from the system cache 20 through the bus 40 and non-overlapping data from the DRAM30 through the bus 40. This avoids reading duplicate data from the relatively high-power DRAM30. Without widening the bandwidth of the DRAM30 or increasing its frequency, the overlapping data that needs to be read repeatedly is read from the system cache 20. This can reduce the NPU's reading power consumption while ensuring the NPU's reading speed, thereby reducing the NPU's power consumption.

[0038] Optional, such as Figure 1 As shown, the data processing system 100 may further include a central processing unit (CPU) 50 and an image signal processor (ISP) 60. Both the CPU 50 and the ISP 60 are connected to the bus 40.

[0039] In this embodiment of the application, before the NPU10 performs neural network operations, the CPU50 can send instructions to the NPU10 and the system cache20 via the bus40. The NPU10 can perform neural network operations according to the instructions sent by the CPU50 and execute the method in the following method embodiment. The system cache20 can read the data that needs to be performed on the neural network from the DRAM30 in advance according to the instructions sent by the CPU50.

[0040] Figure 1 The DRAM and system cache are independent of each other, and both are connected to the bus. The system cache can give the DRAM data read instructions in advance, and can read overlapping data from the DRAM into the system cache in advance when the DRAM bandwidth is relatively idle.

[0041] Please see Figure 3 , Figure 3 This is a schematic diagram of the structure of a data processing system provided in an embodiment of this application. For example... Figure 3 As shown, the data processing system 100 may include a neural network processing unit (NPU) 10, a system buffer (SysBuf) 20, dynamic random access memory (DRAM) 30, and a bus 40. The NPU 10 and system buffer 20 are both connected to the bus 40. The DRAM 30 is connected to the system buffer 20.

[0042] Optional, such as Figure 3 As shown, the data processing system 100 may further include a central processing unit (CPU) 50 and an image signal processor (ISP) 60. Both the CPU 50 and the ISP 60 are connected to the bus 40.

[0043] Figure 3 The system cache is connected to the bus, while the DRAM is not connected to the bus but is connected to the system cache. When the NPU needs to read data from the DRAM, it can do so through the bus and the system cache. For example, when the NPU sends a data read command to the system cache, it first checks if the required data exists in the system cache. If it does, it reads the data directly from the system cache; otherwise, it reads the data from the DRAM.

[0044] based on Figure 1 or Figure 3 The data processing system shown and Figure 2The NPU shown in this application provides a data processing method. Please refer to [link / reference]. Figure 4 , Figure 4 This is a flowchart illustrating a data processing method provided in an embodiment of this application. Figure 4 As shown, the data processing method may include the following steps.

[0045] 401. The NPU determines the number of SRAMs allocated to the input data based on the number of channels to be input, where M is less than or equal to N and M is an integer multiple of the number of channels to be input.

[0046] In this embodiment, the input data can be image data. For example, the input data can be all the data of an image, or it can be a portion of the data in an image. When the input data is a portion of the data in an image, the amount of data in that portion of the image can be exactly equal to the maximum amount of data that the NPU can process in one operation.

[0047] The number of channels in the input data can be one channel (e.g., grayscale image or binarized image), or two channels, three channels (e.g., RGB image), four channels (e.g., RGB + transparency), etc.

[0048] M is an integer multiple of the number of input data channels. Each channel can be allocated at least one SRAM, and the number of SRAMs allocated to each channel is equal. For example, if there are 3 input data channels and the NPU's on-chip cache has a total of N = 6 SRAMs, then 2 SRAMs can be allocated to each channel. If there are 2 input data channels and the NPU's on-chip cache has a total of N = 6 SRAMs, then 3 SRAMs can be allocated to each channel. If there are 3 input data channels and the NPU's on-chip cache has a total of N = 8 SRAMs, then 2 SRAMs can be allocated to each channel.

[0049] Optionally, in step 401, the NPU determines the M SRAMs allocated to the input data based on the number of channels to be input, which may specifically include the following steps:

[0050] When N is greater than or equal to twice Q, the NPU determines P SRAMs corresponding to each channel of the input data, where P is greater than or equal to 2;

[0051] When N is greater than Q and less than twice Q, the NPU determines one SRAM corresponding to each channel of the input data;

[0052] When N is less than Q, the NPU determines one SRAM corresponding to each channel of the data to be input, and the data to be input is input to the NPU multiple times.

[0053] In this embodiment, the number of channels is Q, which can be a positive integer; N is the number of SRAMs contained in the NPU's on-chip cache. If N is divisible by Q, all N SRAMs in the on-chip cache can be used to read input data from the system cache or DRAM via the bus, fully utilizing the on-chip cache to process the input data, improving the utilization rate of the SRAMs in the cache, and thus improving the NPU's processing efficiency for the input data. If N is not divisible by Q, but N is greater than Q, at least one SRAM cannot be used to process the input data. When N is greater than Q, the NPU can process all channel data contained in the input data at once. If N is less than Q, the NPU can only process a portion of the channel data contained in the input data at once, and the input data is input to the NPU multiple times.

[0054] In one possible implementation, N can be equal to 2 of the number of channels Q. n The number of times N can be 4, 8, 16, etc., when Q = 2.

[0055] 402. The NPU reads overlapping data from the system cache of the first channel data via the bus, loads the overlapping data into the P SRAMs corresponding to the first channel, where Q*P = M, and Q is the number of channels. The first channel data includes data from any one of the channels in the input data. The first channel data can be data from just one of the channels in the input data.

[0056] In this embodiment, the overlapping data in the first channel data can include two cases: the first case is that data input to the same SRAM in two adjacent inputs is repeated; the second case is that data input to the same channel of two or more SRAMs in the same input is repeated.

[0057] In the first scenario, since the data stored in a single SRAM is limited, the number of SRAMs in the NPU is fixed after manufacturing, and the size of each SRAM is also fixed. If a single SRAM cannot store all the input data required for convolution operations, the input data needs to be input into the NPU's SRAM multiple times for processing. Due to the characteristics of neural network convolution operations, to ensure that each row of image data can undergo convolution, there will be duplicate data in adjacent inputs to the SRAM. Since the overlapping data in the first channel needs to be read twice between adjacent operations, both reads are from the system cache. Compared to reading from DRAM, this significantly reduces read power consumption and lowers the bandwidth and frequency requirements of DRAM.

[0058] In the second scenario, P is an integer greater than or equal to 2. If P equals 2, one channel can correspond to two SRAMs. When the first channel corresponds to two SRAMs, due to the characteristics of convolution operations in neural networks, to ensure that each row of data in the image data can undergo convolution, there will be duplicate data between the two SRAMs corresponding to the first channel, i.e., overlapping data in the first channel data. In this case, the NPU can read the overlapping data in the first channel twice from the system cache via the bus and load it into the two SRAMs corresponding to the first channel respectively. Compared to P=1, when P equals 2, the NPU can process more data for the same channel at once (it can process the data in the two SRAMs corresponding to the same channel), improving the NPU's data processing efficiency.

[0059] 403. The NPU reads non-overlapping data from the first channel of DRAM via the bus and loads the non-overlapping data into the corresponding SRAM; the power consumption of the system cache read is less than that of the DRAM read.

[0060] In this embodiment, the NPU only needs to read the non-overlapping data in the first channel data once, without having to read it multiple times. The NPU reads the non-overlapping data in the first channel data from the DRAM via the bus and loads the non-overlapping data into the corresponding SRAM. The requirements for the bandwidth and frequency of the DRAM are relatively low.

[0061] The input data is generally placed in the DRAM of the data processing system. Before executing step 402, the overlapping data in the first channel data contained in the input data can be read from the DRAM into the system cache. When the NPU needs to process the first channel data contained in the input data, it reads the data that needs to be read repeatedly in the first channel data (overlapping data in the first channel data) from the system cache and reads the data that does not need to be read repeatedly in the first channel data (non-overlapping data in the first channel data) from the DRAM. Since the reading power consumption of the system cache is less than that of the DRAM, the reading power consumption of the NPU can be significantly reduced compared with reading all data (overlapping data and non-overlapping data in the first channel data) from the DRAM.

[0062] It should be noted that this application embodiment uses the first channel data contained in the data to be input as an example for illustration. For other channel data contained in the data to be input, the reading method of the first channel data can also be referred to, and will not be repeated here.

[0063] 404. The NPU processes data loaded in P SRAMs through a data processing unit matrix.

[0064] In this embodiment of the application, the NPU can perform convolution operations on the data loaded in P SRAMs through a data processing unit matrix.

[0065] Optionally, step 404 may include the following steps:

[0066] The NPU performs convolution operations on the convolution kernel and the data loaded in the P SRAMs through the data processing unit matrix to obtain the convolution operation result.

[0067] In this embodiment, the convolution kernel can be convolved with the data loaded in each of the P SRAMs to obtain the convolution result. Figure 2 The data loaded in the SRAM corresponding to each row has the same convolution kernel size. The weights of the convolution kernels can be the same or different, depending on the algorithm of the neural network.

[0068] Optionally, the amount of overlapping data in the first channel data is positively correlated with the size of the convolution kernel.

[0069] In this embodiment, the larger the size of the convolution kernel, the larger the amount of overlapping data in the first channel data. For example, if the size of the convolution kernel is 3*3 (3 rows and 3 columns, with a total of 9 weights) and the stride is 1, then the number of rows of overlapping data in the first channel data is 2 rows (the convolution operation is a row-by-row dot product) or 2 columns (the convolution operation is a column-by-column dot product). If the size of the convolution kernel is 5*5 (5 rows and 5 columns, with a total of 25 weights) and the stride is 1, then the number of rows of overlapping data in the first channel data is 4 rows (the convolution operation is a row-by-row dot product) or 4 columns (the convolution operation is a column-by-column dot product).

[0070] The following example illustrates line-by-line dot product. If both SRAM1 and SRAM2 contain 20 rows of data, and the data in both SRAM1 and SRAM2 is from the same channel, then the last two rows of SRAM1 are identical to the first two rows of SRAM2. If an image contains 1000 rows of data, and the convolution kernel size is 3x3, the kernel first performs convolution operations on rows 1-3. After the convolution operations on rows 1-3 are completed, it then performs convolution operations on rows 2-4. After the convolution operations on rows 2-4 are completed, it then performs convolution operations on rows 3-5, and so on, until the convolution operations on rows 998-1000 are completed. Since the amount of data in an image is often very large, a single SRAM often cannot store all the data, so multiple SRAMs are needed for multiple reads to perform the operation. If the data in the two SRAMs do not overlap, assuming that the capacity of one SRAM is 20 rows, then if rows 1 to 20 are stored in SRAM1 and rows 21 to 40 are stored in SRAM2, then the convolution operation of the convolution kernel in rows 19 to 21 and the convolution operation in rows 20 to 22 will be lost.

[0071] In this embodiment of the application, when processing these 1000 rows of data using the NPU, rows 1 to 20 of the 1000 rows can be stored in rows 1 to 20 of SRAM1, and rows 19 to 38 of the 1000 rows can be stored in rows 1 to 20 of SRAM2 (the last two rows of SRAM1 have the same data as the first two rows of SRAM2, which are overlapping data). After the NPU performs one processing cycle, rows 37 to 56 of the 1000 rows can be stored in rows 1 to 20 of SRAM1, and rows 55 to 74 of the 1000 rows can be stored in rows 1 to 20 of SRAM2, and so on, until all 1000 rows have been convolved. After the NPU performs the first processing, and the 3*3 convolution kernel performs convolution operation in rows 18 to 20 of SRAM1 (equivalent to rows 18 to 20 out of 1000 rows), the convolution kernel can continue to perform convolution operation in rows 1 to 3 of SRAM2 (equivalent to rows 19 to 21 out of 1000 rows).

[0072] In this embodiment, for overlapping data that needs to be read repeatedly in the first channel data, it can be read from the system cache with relatively low reading power consumption, avoiding reading duplicate data from the DRAM with relatively high power consumption. Without widening the DRAM bandwidth or increasing the DRAM frequency, reading the overlapping data that needs to be read repeatedly from the system cache can reduce the NPU's reading power consumption while ensuring the NPU's reading speed, thereby reducing the NPU's power consumption.

[0073] based on Figure 1 or Figure 3 The data processing system shown and Figure 2 The NPU shown in this application provides another data processing method; please refer to [link to relevant documentation]. Figure 5 , Figure 5 This is a flowchart illustrating another data processing method provided in an embodiment of this application. For example... Figure 5 As shown, the data processing method may include the following steps.

[0074] 501. The NPU determines the number of SRAMs allocated to the input data based on the number of channels to be input, where M is less than or equal to N and M is an integer multiple of the number of channels to be input; where N is greater than or equal to twice the number of channels Q, the NPU determines the number of SRAMs corresponding to each channel of the input data, where P is greater than or equal to 2 and Q*P = M.

[0075] 502. When P is greater than or equal to 2, the NPU allocates corresponding circular cache spaces in the P SRAMs corresponding to the first channel data.

[0076] In this embodiment of the application, the NPU can allocate a circular cache space in each of the P SRAMs corresponding to the first channel data, thus obtaining P circular cache spaces.

[0077] Please see Figure 6 , Figure 6 This is a schematic diagram of a ring buffer space in an SRAM provided in an embodiment of this application. This embodiment of the application can allocate ring buffer spaces corresponding to different network layers in a neural network within each SRAM block. Figure 6 There are n network layers. Each SRAM can allocate a circular cache space in each network layer. Each of the P SRAMs allocates space in units of the even-numbered rows of pixels in the image, which can ensure the address alignment of the circular cache spaces of the same network layer in different SRAMs.

[0078] 503, the NPU concatenates P ring cache spaces in the address space to obtain a concatenated ring cache space, and maps the overlapping regions in the P ring cache spaces to shadow spaces.

[0079] In this embodiment, P circular cache spaces can be concatenated in the address space to obtain a concatenated circular cache space. The overlapping region in the P circular cache spaces can be a region where duplicate data is stored between the P circular cache spaces.

[0080] Please see Figure 7 , Figure 7 This is a schematic diagram of a distributed storage system provided in an embodiment of this application. For example... Figure 7As shown, assume the on-chip cache consists of four SRAMs that cannot access each other, numbered core 0 to core 3. Assume the input image of a convolutional layer with a kernel size of 3x3 and a stride of 1 has an even number of rows and two channels (channel 0 and channel 1). Without the distributed data storage method of this embodiment, the image data of the two channels is distributed in two SRAMs, Core 0 to Core 1. The two SRAMs, Core 2 to Core 3, are idle. This embodiment can evenly distribute the image data of the same channel in the two SRAMs. For example, the image data corresponding to channel 0 is stored in Core 0 and Core 2, and the image data corresponding to channel 1 is stored in Core 1 and Core 3. An overlapping space is designed in the ring buffer of each SRAM. The data stored in this overlapping space will be used between different cores. The space where this part of the data is repeatedly stored is called the shadow space, and the overlapping space corresponding to the shadow space is called the source space, thereby improving the SRAM utilization of the on-chip cache in scenarios where the number of image channels is less than the number of SRAMs in the on-chip cache.

[0081] Optionally, in step 503, the NPU concatenates the P circular cache spaces in the address space to obtain a concatenated circular cache space, which may specifically include the following steps:

[0082] The NPU concatenates P ring cache spaces sequentially end to end in the address space, with overlapping areas between two adjacent concatenated ring cache spaces.

[0083] Optionally, in step 503, the NPU maps the overlapping regions in the P ring-shaped cache spaces to shadow spaces, which may specifically include the following steps:

[0084] The NPU maps the addresses of overlapping regions in the P ring cache spaces to addresses in the shadow space according to the address mapping table.

[0085] The address mapping table can be pre-stored in the NPU. This table can include the mapping relationship between addresses of overlapping regions and addresses in the shadow space. The shadow space can be mapped to a real address, and the source space corresponding to the shadow space can be found through the address mapping table.

[0086] In the embodiments of this application, such as Figure 7As shown, the image data corresponding to channel 0 is stored in Core 0 and Core 2. The data in the overlapping area of ​​Core 0 (row 6, row 7) is mapped to the shadow space before address space (0x20) of Core 2, and the first two rows of Core 2 (row 8, row 9) are mapped to the shadow space of Core 0. Similarly, the last two rows of Core 2 (row 14, row 15) are mapped to the shadow space at the beginning of Core 0, and the first two rows of Core 0 are mapped to the corresponding shadow space in Core 2. The address spaces of Core 0 and Core 2 become a circular space connected end to end. For the data in the shadow space, this data is read from the low-power system cache to avoid repeated readings from high-power DRAM.

[0087] Figure 7 The overlapping region size is the space corresponding to two rows of image data. This is because after the convolutional layer has processed three rows of data, it needs to retrieve two new rows of data from SRAM. Similarly, once the data in Core 0 has been processed, Figure 2 The PE matrix needs to read data from Core 2 because the convolution kernel size is 3x3, meaning three rows of data are needed to generate one row of data in the feature image. Therefore, Core 2 needs to store the last two rows of data from Core 0 to ensure that the PE corresponding to channel 0 can read the data required for calculation in Core 2.

[0088] The following is combined Figure 7 To illustrate the computation required for the overlapping region, suppose the image has 4000 rows of data across two channels. The allocated core 0 Ring buffer 0 can store 20 rows of data, and the corresponding core 2 Ring buffer 0 can also store 20 rows. Therefore, 40 rows of data can be retrieved at a time. Based on the previous analysis, with a 5x5 convolution kernel, retrieving 40 rows of data results in 16 rows of overlapping data. It would take 100 rounds to retrieve all the image data. These 100 rounds require an additional 1600 rows of overlapping data to be read. This 1600 rows represent 1600 / 4000 = 0.4 of the total image rows, meaning 40% of the data is repeated data from the overlapping region. Since DRAM read energy is much higher than SRAM read energy, reading the repeated 1280 rows of data from the system cache can save a significant amount of energy.

[0089] Please see Figure 8 , Figure 8 This is a schematic diagram of the power consumption distribution of a graph calculated using different methods, provided in the embodiments of this application. Figure 8The dashed line in the diagram represents the power consumption distribution when all data is read from DRAM. Figure 8 The solid lines in the diagram represent the power consumption distribution of the method described in this embodiment. In this embodiment, overlapping data is read from the system cache, and non-overlapping data is read from DRAM. Figure 8 It can be seen that the method of this application embodiment can reduce the power consumption of computing a single image. Figure 8 The x-axis represents the position of the input image data, and the y-axis represents the computational power consumption corresponding to that position. Image positions can include the head, middle, and tail of the image. During convolution, the computation begins at the head position, then proceeds to the middle position, and finally to the tail position. For example, for an image with 3000 rows, rows 1-1000 can be named the head, rows 1001-2000 the middle, and rows 2001-3000 the tail.

[0090] according to Figure 8 (This is just one possible scenario, but not limited to this phenomenon.) Analysis shows that because the overlapping area (the shaded area in the diagram represents the data that needs to be repeatedly read due to the shadow space) requires repeated data readings within a short period, if all data is read from DRAM, it will cause an increase in DRMA data reads within a short time, thereby increasing DRAM power consumption. The embodiments proposed in this application... Figure 1 or Figure 3 The data processing system can utilize the system cache structure to store this part of the data in the system cache and retrieve the data in the overlapping area by accessing the system cache, thereby reducing the increase in system power consumption caused by retrieving the overlapping data.

[0091] The embodiments proposed in this application Figure 1 or Figure 3 The data processing system stores this data in a low-power system cache in advance. This duplicate data can be retrieved from the low-power system cache, thereby reducing the NPU's read power consumption.

[0092] 504. The NPU reads overlapping data from the system cache of the first channel data via the bus and loads the overlapping data into the shadow space of the spliced ​​ring cache space. The first channel data includes data from any one channel of the data to be input. The first channel data can be data from just one channel of the data to be input.

[0093] 505, the NPU reads non-overlapping data from the first channel of DRAM via the bus and loads the non-overlapping data into the corresponding SRAM; the power consumption of the system cache read is less than that of the DRAM read.

[0094] 506. The NPU processes data loaded in P SRAMs through a data processing unit matrix.

[0095] For details on the implementation of steps 504 to 506, please refer to [link / reference]. Figure 4 Steps 402 to 404 shown are not repeated here.

[0096] The embodiments of this application can improve the SRAM utilization rate of on-chip cache in scenarios where the number of channels is less than the number of SRAMs in the on-chip cache, thereby improving data processing efficiency.

[0097] Optionally, before the NPU reads the overlapping data from the system cache of the first channel data via the bus, the following steps may also be included.

[0098] The NPU determines whether the spliced ​​ring cache space has a mapped shadow space;

[0099] If it exists, the NPU requests the data required for the shadow space from the system cache via the bus according to the size of the shadow space, and the NPU performs the step of reading the overlapping data in the first channel data from the system cache via the bus.

[0100] In this embodiment, before the NPU reads the overlapping data in the first channel data from the system cache via the bus, it needs to determine whether there is a mapped shadow space in the spliced ​​ring cache space. Only when there is a mapped shadow space in the spliced ​​ring cache space is it considered that there is overlapping data in the first channel data that needs to be read repeatedly. Then, it requests data from the system cache and reads data, thus avoiding invalid data access.

[0101] The above primarily describes the solutions of the embodiments of this application from the perspective of the method execution process. It is understood that, in order to achieve the above functions, the terminal device includes the corresponding hardware structure and / or software modules for executing each function. Those skilled in the art should readily recognize that, based on the units and algorithm steps of the various examples described in the embodiments provided herein, this application can be implemented in hardware or a combination of hardware and computer software. Whether a function is executed in hardware or by computer software driving hardware depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0102] This application embodiment can divide the terminal device into functional units according to the above method example. For example, each function can be divided into a separate functional unit, or two or more functions can be integrated into one processing unit. The integrated unit can be implemented in hardware or as a software functional unit. It should be noted that the unit division in this application embodiment is illustrative and only represents one logical functional division. In actual implementation, there may be other division methods.

[0103] Please see Figure 9 , Figure 9 This is a schematic diagram of a data processing device 900 provided in an embodiment of this application. The data processing device 900 is applied to a data processing system, which includes a neural network processor (NPU), a system cache, dynamic random access memory (DRAM), and a bus. The NPU includes a data processing unit matrix and N static random access memories (SRAMs). The data processing device 900 may include a determination unit 901, a reading unit 902, and a data processing unit 903, wherein:

[0104] The determining unit 901 is used to determine the M SRAMs allocated to the data to be input based on the number of channels of the data to be input, where M is less than or equal to N and M is an integer multiple of the number of channels of the data to be input.

[0105] The reading unit 902 is used to read overlapping data from the system cache of the first channel data through the bus, load the overlapping data into P SRAMs corresponding to the first channel, Q*P=M, where Q is the number of channels, and the first channel data includes data from any one of the channels in the data to be input.

[0106] The read unit 902 is further configured to read non-overlapping data from the first channel data in the DRAM via the bus, and load the non-overlapping data into the corresponding SRAM; the read power consumption of the system cache is less than the read power consumption of the DRAM;

[0107] The data processing unit 903 processes the data loaded in the P SRAMs through the matrix of the data processing unit 903.

[0108] Optionally, the determining unit 901 determines the M SRAMs allocated to the input data based on the number of channels of the input data, including:

[0109] If N is greater than or equal to twice Q, determine P SRAMs corresponding to each channel of the input data, where P is greater than or equal to 2;

[0110] When N is greater than Q and less than twice Q, determine one SRAM corresponding to each channel of the input data;

[0111] When N is less than Q, one SRAM is determined for each channel of the data to be input, and the data to be input is input to the NPU multiple times.

[0112] Optionally, the data processing device 900 may further include a cache allocation unit 904 and a splicing unit 905;

[0113] The cache allocation unit 904 is used to allocate corresponding circular cache spaces in the P SRAMs corresponding to the first channel data when P is greater than or equal to 2.

[0114] The splicing unit 905 is used to splice P ring cache spaces in the address space to obtain a spliced ​​ring cache space, and to map the overlapping area in the P ring cache spaces to a shadow space.

[0115] The reading unit 902 loads the overlapping data into P SRAMs corresponding to the first channel, including: the reading unit 902 loads the overlapping data into the shadow space of the spliced ​​ring cache space.

[0116] Optionally, the splicing unit 905 splices P ring cache spaces in the address space to obtain a spliced ​​ring cache space, including: the splicing unit 905 splices P ring cache spaces sequentially end to end in the address space, and there is an overlapping area between two adjacent spliced ​​ring cache spaces;

[0117] The splicing unit 905 maps the overlapping regions in the P ring-shaped cache spaces to shadow spaces, including: the splicing unit 905 maps the addresses of the overlapping regions in the P ring-shaped cache spaces to the addresses of the shadow spaces according to the address mapping table.

[0118] Optionally, the data processing device 900 may further include a judgment unit 906 and an application unit 907;

[0119] The judgment unit 906 is used to determine whether the spliced ​​ring buffer space has a mapped shadow space;

[0120] The application unit 907 is used to apply for the data required for the shadow space from the system cache through the bus, based on the size of the shadow space, when the judgment unit 906 determines that the spliced ​​ring cache space has a mapped shadow space;

[0121] The reading unit 902 is further configured to read overlapping data from the first channel data from the system cache via the bus after the requesting unit 907 requests the data required for the shadow space from the system cache via the bus according to the size of the shadow space.

[0122] Optionally, the data processing unit 903 performs data processing on the data loaded in the P SRAMs through the data processing unit 903 matrix, including: the data processing unit 903 performs convolution operation on the convolution kernel and the data loaded in the P SRAMs through the data processing unit 903 matrix to obtain the convolution operation result.

[0123] Optionally, the amount of overlapping data in the first channel data is positively correlated with the size of the convolution kernel.

[0124] In this embodiment, the determining unit 901, reading unit 902, data processing unit 903, cache allocation unit 904, splicing unit 905, judgment unit 906, and application unit 907 can be the NPU in the terminal device.

[0125] In this embodiment, for overlapping data that needs to be read repeatedly in the first channel data, it can be read from the system cache with relatively low reading power consumption, avoiding reading duplicate data from the DRAM with relatively high power consumption. Without widening the DRAM bandwidth or increasing the DRAM frequency, reading the overlapping data that needs to be read repeatedly from the system cache can reduce the NPU's reading power consumption while ensuring the NPU's reading speed, thereby reducing the NPU's power consumption.

[0126] Please see Figure 10 , Figure 10 This is a schematic diagram of the structure of a terminal device provided in an embodiment of this application, such as... Figure 10 As shown, the terminal device 1000 includes a processor 1001 and a memory 1002, which are interconnected via a communication bus 1003. The communication bus 1003 can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. The communication bus 1003 can be divided into an address bus, a data bus, a control bus, etc. For ease of illustration, Figure 10 The bus is represented by a single thick line, but this does not indicate that there is only one bus or one type of bus. Memory 1002 stores computer programs, which include program instructions. Processor 1001 is configured to invoke these program instructions, which include instructions for execution. Figures 4-5 Some or all of the steps in the method shown.

[0127] Processor 1001 may be a general-purpose central processing unit (CPU), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits used to control the execution of programs in the above scheme. Processor 1001 may also include an NPU and an ISP.

[0128] The memory 1002 may be a read-only memory (ROM) or other type of static storage device capable of storing static information and instructions, random access memory (RAM) or other type of dynamic storage device capable of storing information and instructions, or electrically erasable programmable read-only memory (EEPROM), compact disc read-only memory (CD-ROM) or other optical disc storage, optical disc storage (including compressed optical discs, laser discs, optical discs, digital versatile optical discs, Blu-ray discs, etc.), magnetic disk storage media or other magnetic storage devices, or any other medium capable of carrying or storing desired program code in the form of instructions or data structures and accessible by a computer, but not limited thereto. The memory may exist independently and be connected to the processor via a bus. The memory may also be integrated with the processor.

[0129] In addition, the terminal device 1000 may also include general components such as communication interfaces and antennas, which will not be described in detail here.

[0130] In this embodiment, for overlapping data that needs to be read repeatedly in the first channel data, it can be read from the system cache with relatively low reading power consumption, avoiding reading duplicate data from the DRAM with relatively high power consumption. Without widening the DRAM bandwidth or increasing the DRAM frequency, reading the overlapping data that needs to be read repeatedly from the system cache can reduce the NPU's reading power consumption while ensuring the NPU's reading speed, thereby reducing the NPU's power consumption.

[0131] This application also provides a computer-readable storage medium storing a computer program for electronic data interchange, which causes a computer to perform some or all of the steps of any of the data processing methods described in the above method embodiments.

[0132] It should be noted that, for the sake of simplicity, the foregoing method embodiments are all described as a series of actions. However, those skilled in the art should understand that this application is not limited to the described order of actions, as some steps may be performed in other orders or simultaneously according to this application. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are preferred embodiments, and the actions and modules involved are not necessarily essential to this application.

[0133] In the above embodiments, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions in other embodiments.

[0134] In the several embodiments provided in this application, it should be understood that the disclosed apparatus can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical or other forms.

[0135] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0136] Furthermore, the functional units in the various embodiments of the application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software program module.

[0137] If the integrated unit is implemented as a software program module and sold or used as an independent product, it can be stored in a computer-readable storage device (CMD). Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a memory and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned memory includes various media capable of storing program code, such as USB flash drives, read-only memory (ROM), random access memory (RAM), portable hard drives, magnetic disks, or optical disks.

[0138] Those skilled in the art will understand that all or part of the steps in the various methods of the above embodiments can be implemented by a program instructing related hardware. The program can be stored in a computer-readable storage device, which may include: a flash drive, a read-only memory, a random access memory, a magnetic disk, or an optical disk, etc.

[0139] The embodiments of this application have been described in detail above. Specific examples have been used to illustrate the principles and implementation methods of this application. The description of the above embodiments is only for the purpose of helping to understand the method and core ideas of this application. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of this application. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. A data processing method, characterized in that, The method is applied to a data processing system, which includes a neural network processor (NPU), a system cache, dynamic random access memory (DRAM), and a bus. The NPU includes a data processing unit matrix and N static random access memories (SRAMs). The method includes: The NPU determines the number of SRAMs allocated to the input data based on the number of channels of the input data, where M is less than or equal to N and M is an integer multiple of the number of channels of the input data. The NPU reads overlapping data from the system cache of the first channel data through the bus, loads the overlapping data into P SRAMs corresponding to the first channel, Q*P=M, where Q is the number of channels, and the first channel data includes data from any one of the channels in the data to be input. The NPU reads non-overlapping data from the first channel data in the DRAM via the bus and loads the non-overlapping data into the corresponding SRAM; the read power consumption of the system cache is less than the read power consumption of the DRAM; The NPU processes the data loaded in the P SRAMs through the data processing unit matrix.

2. The method according to claim 1, characterized in that, The NPU determines the M SRAMs allocated to the input data based on the number of channels to be input, including: When N is greater than or equal to twice Q, the NPU determines P SRAMs corresponding to each channel of the input data, where P is greater than or equal to 2; When N is greater than Q and less than twice Q, the NPU determines one SRAM corresponding to each channel of the input data; When N is less than Q, the NPU determines one SRAM corresponding to each channel of the data to be input, and the data to be input is input to the NPU multiple times.

3. The method according to claim 2, characterized in that, Before the NPU loads the overlapping data into the P SRAMs corresponding to the first channel, the method further includes: When P is greater than or equal to 2, the NPU allocates corresponding circular cache spaces in the P SRAMs corresponding to the first channel data. The NPU concatenates P ring cache spaces in the address space to obtain a concatenated ring cache space, and maps the overlapping areas in the P ring cache spaces to a shadow space. The NPU loads the overlapping data into P SRAMs corresponding to the first channel, including: The NPU loads the overlapping data into the shadow space of the spliced ​​circular cache space.

4. The method according to claim 3, characterized in that, The NPU concatenates P circular cache spaces in the address space to obtain a concatenated circular cache space, including: The NPU concatenates P ring cache spaces sequentially end to end in the address space, with overlapping areas between two adjacent concatenated ring cache spaces. The NPU maps overlapping regions in the P ring-shaped cache spaces to shadow spaces, including: The NPU maps the addresses of overlapping regions in the P ring cache spaces to addresses in the shadow space according to the address mapping table.

5. The method according to claim 3, characterized in that, Before the NPU reads overlapping data from the system cache of the first channel data via the bus, the method further includes: The NPU determines whether the spliced ​​ring cache space has a mapped shadow space; If it exists, the NPU requests the data required for the shadow space from the system cache via the bus according to the size of the shadow space, and the NPU performs the step of reading the overlapping data in the first channel data from the system cache via the bus.

6. The method according to any one of claims 1 to 5, characterized in that, The NPU processes the data loaded in the P SRAMs through the data processing unit matrix, including: The NPU performs convolution operations on the convolution kernel and the data loaded in the P SRAMs through the data processing unit matrix to obtain the convolution operation result.

7. The method according to claim 6, characterized in that, The amount of overlapping data in the first channel data is positively correlated with the size of the convolution kernel.

8. A data processing apparatus, characterized in that, The method described in the data processing apparatus is applied to a data processing system, which includes a neural network processor (NPU), a system cache, dynamic random access memory (DRAM), and a bus. The NPU includes a data processing unit matrix and N static random access memories (SRAMs). The data processing apparatus includes: The determining unit is used to determine the M SRAMs allocated to the data to be input based on the number of channels of the data to be input, where M is less than or equal to N and M is an integer multiple of the number of channels of the data to be input. The reading unit is used to read overlapping data from the system cache of the first channel data through the bus, load the overlapping data into P SRAMs corresponding to the first channel, Q*P=M, where Q is the number of channels, and the first channel data includes data from any one of the channels in the data to be input. The read unit is further configured to read non-overlapping data from the first channel data in the DRAM via the bus, and load the non-overlapping data into the corresponding SRAM; the read power consumption of the system cache is less than the read power consumption of the DRAM; The data processing unit processes the data loaded in the P SRAMs through the data processing unit matrix.

9. A terminal device, characterized in that, The device includes a processor and a memory, the memory being used to store a computer program, the computer program including program instructions, and the processor being configured to invoke the program instructions to perform the method as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program, the computer program including program instructions that, when executed by a processor, cause the processor to perform the method as described in any one of claims 1 to 7.

Images