Data processing methods, processors, chips and electronic devices

Abstract

This disclosure relates to a data processing method, processor, chip, and electronic device. The method includes: a control logic unit sequentially reading cyclic blocks of data to be processed from memory to a dot product unit array according to acquired control instructions; the dot product unit array performing multiplication and addition operations on each received cyclic block of data to be processed to determine the cyclic block result of each received data; and the control logic unit determining the logical operation result of the data to be processed based on the multiple cyclic block results obtained from the dot product unit array. Embodiments of this disclosure can convert the reading and logical operation of data to be processed into the reading and logical operation of multiple cyclic blocks of data to be processed, which is beneficial for processing larger data sizes without changing processor hardware resources and reducing the pressure on storage bandwidth.

Description

[Technical Field]

[0001] This disclosure relates to the field of computer technology, and in particular to a data processing method, processor, chip and electronic device. [Previous Technology]

[0002] In the field of artificial intelligence technology, neural network algorithms are a very popular machine learning algorithm that has achieved excellent results in various fields, such as image recognition, speech recognition, and natural language processing. With the development of neural network algorithms, their complexity is increasing, and the model size is gradually increasing to improve recognition accuracy. Processing these large-scale models using processors such as Central Processing Units (CPUs) and Graphics Processing Units (GPUs) requires a significant amount of computation time and consumes a large amount of electricity.

[0003] Processors and AI accelerators incorporate numerous proprietary modules to accelerate training and inference. Matrix multiplication and convolution are common operators in AI algorithms. Related technologies utilize processors with Single Instruction Multiple Data (SIMD) or Single Instruction Multiple Thread (SIMT) architectures, relying on multiplication and accumulation instructions to implement corresponding algorithms. This results in limited data sharing between different threads, requiring multiple reads of repetitive data, leading to poor processor performance and energy consumption. [Summary of the Invention]

[0004] This disclosure proposes a data processing technology solution.

[0005] According to one aspect of this disclosure, a data processing method is provided, comprising: the data processing method being applied to a processor, the processor including a control logic unit and a multiplication unit array, the multiplication unit array including a plurality of multiplication units for performing multiplication-accumulation operations, the method comprising: the control logic unit sequentially reading cyclic block data of data to be processed from memory to the multiplication unit array according to acquired control instructions; the multiplication unit array performing multiplication-accumulation operations on each received cyclic block data of the data to be processed to determine the cyclic block result of the data to be processed each time; the control logic unit determining the logical operation result of the data to be processed based on the plurality of cyclic block results obtained from the multiplication unit array.

[0006] In one possible implementation, the control logic unit sequentially reads the cyclic block data of the data to be processed from memory to the dot product unit array according to the acquired control instructions, including: the control logic unit parses the access information of the data to be processed and the logical operation type of the data to be processed according to the acquired control instructions; determines the cyclic block order of the data to be processed according to the logical operation type of the data to be processed; and sequentially reads the cyclic block data of the data to be processed from memory to the dot product unit array according to the access information and the preset size, in accordance with the cyclic block order of the data to be processed, wherein the size of the cyclic block data is less than or equal to the preset size, and the preset size is determined by the number of dot product units in the dot product unit array that are in an idle state.

[0007] In one possible implementation, the access information of the data to be processed includes access information of a first matrix and a second matrix, wherein the number of columns of the first matrix is ​​the same as the number of rows of the second matrix, the logical operation type includes matrix multiplication of the first matrix and the second matrix, and the cyclic block order determined by the matrix multiplication operation type includes: a first outer cyclic order in the direction of the number of rows of the first matrix, a second outer cyclic order in the direction of the number of columns of the second matrix, and an inner cyclic order in the direction of the number of columns of the first matrix and the direction of the number of rows of the second matrix; or, a first outer cyclic order in the direction of the number of columns of the second matrix, a second outer cyclic order in the direction of the number of rows of the first matrix, and an inner cyclic order in the direction of the number of columns of the first matrix and the direction of the number of rows of the second matrix.

[0008] In one possible implementation, the memory includes a local memory disposed outside the processor and a reuse cache disposed within the processor for caching cyclic block data. The step of sequentially reading the cyclic block data of the data to be processed from the memory into the dot product unit array according to the cyclic block order of the data to be processed includes: the control logic unit storing the first cyclic block data of the first matrix read from the local memory into the reuse cache according to the first outer cyclic order and the inner cyclic order, so that the control logic unit reuses the first cyclic block data stored in the reuse cache according to the second outer cyclic order; or, the control logic unit storing the second cyclic block data of the second matrix read from the local memory into the reuse cache according to the second outer cyclic order and the inner cyclic order, so that the control logic unit reuses the second cyclic block data stored in the reuse cache according to the first outer cyclic order.

[0009] In one possible implementation, the dot product unit array performs multiplication and accumulation operations on the cyclic block data of the received data to be processed each time, and determines the cyclic block result of the received data to be processed each time, including: the dot product unit array performs multiplication and accumulation operations on the first cyclic block data and the second cyclic block data obtained each time, to obtain the cyclic block result corresponding to each multiplication and accumulation operation; the control logic unit determines the logical operation result of the data to be processed based on the multiple cyclic block results obtained from the dot product unit array, including: the control logic unit performs accumulation operations on the multiple cyclic block results obtained from the dot product unit array in any round of cyclic block order, to obtain a logical operation sub-result; the control logic unit writes the logical operation sub-result into a temporary register heap; the control logic unit determines the logical operation result based on the multiple logical operation sub-results obtained from the temporary register heap.

[0010] In one possible implementation, the access information of the data to be processed includes access information of the data to be convolved and the convolution kernel; the logical operation type includes the convolution operation of the data to be convolved and the convolution kernel; the cyclic block order determined by the convolution operation type includes the coordinate order of the elements in the convolution kernel; the step of sequentially reading the cyclic block data of the data to be processed from memory to the dot product unit array according to the cyclic block order of the data to be processed includes: the control logic unit, according to the coordinate order of the elements in the convolution kernel, forming a third cyclic block data by reading multiple elements with the same coordinates from multiple convolution kernels in memory each time; and, according to the coordinates of the current element in each convolution kernel and the access information, reading multiple elements from the data to be convolved in memory to form a fourth cyclic block data; the control logic unit writes the third cyclic block data and the fourth cyclic block data into the dot product unit array.

[0011] In one possible implementation, the memory includes a local memory located outside the processor and a reuse cache located within the processor for caching cyclic block data. The method further includes: during the calculation of the fourth cyclic block data corresponding to the same row in the convolution output result, the control logic unit writes M elements of the fourth cyclic block number corresponding to the coordinates of the element in the r-th row and s-th column of the convolution kernel into the reuse cache according to the coordinate order of the elements in the convolution kernel, where r, s, and M are positive integers; the control logic unit reads M-1 elements from the reuse cache and reads 1 element from the data to be convolved in the local memory to determine the fourth cyclic block data corresponding to the coordinates of the element in the r-th row and s+1-th column of the convolution kernel.

[0012] In one possible implementation, the method further includes: during the calculation of the fourth cyclic block data corresponding to adjacent rows in the convolution output result, in response to the control logic unit calculating the fourth cyclic block data of the current row of the convolution output result, writing M elements of the fourth cyclic block number corresponding to the coordinates of the element in the r-th row and s-th column of the convolution kernel into a reuse cache, where r, s, and M are positive integers; in response to the control logic unit calculating the fourth cyclic block data of the next row of the current row of the convolution output result, reading M elements from the reuse cache to form the fourth cyclic block data corresponding to the coordinates of the element in the (r-1)-th row and s-th column of the convolution kernel.

[0013] In one possible implementation, the memory includes a local memory located outside the processor and a reuse cache located within the processor for caching cyclic block data. The control logic unit, according to the coordinate order of the elements in the convolution kernel, constructs a third cyclic block data by taking multiple elements with the same coordinates read from multiple convolution kernels in the memory each time. This includes: the control logic unit reading multiple convolution kernels from the local memory to the reuse cache; and the control logic unit constructing a third cyclic block data by taking multiple elements with the same coordinates read from multiple convolution kernels in the reuse cache each time, according to the coordinate order of the elements in the convolution kernel.

[0014] In one possible implementation, the dot product unit array performs multiplication and accumulation operations on the cyclic block data of the data to be processed received each time, and determines the cyclic block result of the data to be processed received each time, including: the dot product unit array performs multiplication and accumulation operations on the third cyclic block data and the fourth cyclic block data received each time, and determines the cyclic block result.

[0015] According to one aspect of this disclosure, a processor is provided, the processor including a control logic unit and a multiplication unit array, the multiplication unit array including a plurality of multiplication units for performing multiplication-accumulation operations, the processor being configured to: the control logic unit sequentially read cyclic block data of data to be processed from memory to the multiplication unit array according to acquired control instructions; the multiplication unit array performing multiplication-accumulation operations on each received cyclic block data of the data to be processed to determine the cyclic block result of the data to be processed each time; the control logic unit determining the logical operation result of the data to be processed based on the plurality of cyclic block results obtained from the multiplication unit array.

[0016] In one possible implementation, the control logic unit sequentially reads the cyclic block data of the data to be processed from the memory to the dot product unit array according to the acquired control instructions, including: the control logic unit parses the access information of the data to be processed and the logical operation type of the data to be processed according to the acquired control instructions; determines the cyclic block order of the data to be processed according to the logical operation type of the data to be processed; and sequentially reads the cyclic block data of the data to be processed from the memory to the dot product unit array according to the access information and the preset size, in accordance with the cyclic block order of the data to be processed, wherein the size of the cyclic block data is less than or equal to the preset size, and the preset size is determined by the number of dot product units in the dot product unit array that are in an idle state.

[0017] In one possible implementation, the access information of the data to be processed includes access information of a first matrix and a second matrix, wherein the number of columns of the first matrix is ​​the same as the number of rows of the second matrix, the logical operation type includes matrix multiplication of the first matrix and the second matrix, and the cyclic block order determined by the matrix multiplication operation type includes: a first outer cyclic order in the direction of the number of rows of the first matrix, a second outer cyclic order in the direction of the number of columns of the second matrix, and an inner cyclic order in the direction of the number of columns of the first matrix and the direction of the number of rows of the second matrix; or, a first outer cyclic order in the direction of the number of columns of the second matrix, a second outer cyclic order in the direction of the number of rows of the first matrix, and an inner cyclic order in the direction of the number of columns of the first matrix and the direction of the number of rows of the second matrix.

[0018] In one possible implementation, the memory includes a local memory disposed outside the processor and a reuse cache disposed within the processor for caching cyclic block data. The step of sequentially reading the cyclic block data of the data to be processed from the memory to the dot product unit array according to the cyclic block order of the data to be processed includes: the control logic unit storing the first cyclic block data of the first matrix read from the local memory into the reuse cache according to the first outer cyclic order and the inner cyclic order, so that the control logic unit reuses the first cyclic block data stored in the reuse cache according to the second outer cyclic order; or, the control logic unit storing the second cyclic block data of the second matrix read from the local memory into the reuse cache according to the second outer cyclic order and the inner cyclic order, so that the control logic unit reuses the second cyclic block data stored in the reuse cache according to the first outer cyclic order.

[0019] In one possible implementation, the dot product unit array performs multiplication and accumulation operations on the cyclic block data of the received data to be processed each time, and determines the cyclic block result of the received data to be processed each time, including: the dot product unit array performs multiplication and accumulation operations on the first cyclic block data and the second cyclic block data obtained each time, to obtain the cyclic block result corresponding to each multiplication and accumulation operation; the control logic unit determines the logical operation result of the data to be processed based on the multiple cyclic block results obtained from the dot product unit array, including: the control logic unit performs accumulation operations on the multiple cyclic block results obtained from the dot product unit array in any round of cyclic block order, to obtain a logical operation sub-result; the control logic unit writes the logical operation sub-result into a temporary register heap; the control logic unit determines the logical operation result based on the multiple logical operation sub-results obtained from the temporary register heap.

[0020] In one possible implementation, the access information of the data to be processed includes access information of the data to be convolved and the convolution kernel; the logical operation type includes the convolution operation of the data to be convolved and the convolution kernel; the cyclic block order determined by the convolution operation type includes the coordinate order of the elements in the convolution kernel; the step of sequentially reading the cyclic block data of the data to be processed from memory to the dot product unit array according to the cyclic block order of the data to be processed includes: the control logic unit, according to the coordinate order of the elements in the convolution kernel, forming a third cyclic block data by reading multiple elements with the same coordinates from multiple convolution kernels in memory each time; and, according to the coordinates of the current element in each convolution kernel and the access information, reading multiple elements from the data to be convolved in memory to form a fourth cyclic block data; the control logic unit writes the third cyclic block data and the fourth cyclic block data into the dot product unit array.

[0021] In one possible implementation, the memory includes a local memory located outside the processor and a reuse cache located within the processor for caching cyclic block data. The processor is further configured to: during the calculation of the fourth cyclic block data corresponding to the same row in the convolution output result, the control logic unit writes M elements of the fourth cyclic block number corresponding to the coordinates of the element in the r-th row and s-th column of the convolution kernel into the reuse cache according to the coordinate order of the elements in the convolution kernel, where r, s, and M are positive integers; the control logic unit reads M-1 elements from the reuse cache and reads 1 element from the data to be convolved in the local memory to determine the fourth cyclic block data corresponding to the coordinates of the element in the r-th row and s+1-th column of the convolution kernel.

[0022] In one possible implementation, the processor is further configured to: during the calculation of the fourth cyclic block data corresponding to adjacent rows in the convolution output result, in response to the control logic unit calculating the fourth cyclic block data of the current row of the convolution output result, write M elements of the fourth cyclic block number corresponding to the coordinates of the element in the r-th row and s-th column of the convolution kernel into a reuse cache, where r, s, and M are positive integers; in response to the control logic unit calculating the fourth cyclic block data of the next row of the current row of the convolution output result, read M elements from the reuse cache to form the fourth cyclic block data corresponding to the coordinates of the element in the (r-1)-th row and s-th column of the convolution kernel.

[0023] In one possible implementation, the memory includes a local memory located outside the processor and a reuse cache located within the processor for caching cyclic block data. The control logic unit, according to the coordinate order of the elements in the convolution kernel, constructs a third cyclic block data by taking multiple elements with the same coordinates read from multiple convolution kernels in the memory each time. This includes: the control logic unit reading multiple convolution kernels from the local memory to the reuse cache; and the control logic unit constructing a third cyclic block data by taking multiple elements with the same coordinates read from multiple convolution kernels in the reuse cache each time, according to the coordinate order of the elements in the convolution kernel.

[0024] In one possible implementation, the dot product unit array performs multiplication and accumulation operations on the cyclic block data of the data to be processed received each time, and determines the cyclic block result of the data to be processed received each time, including: the dot product unit array performs multiplication and accumulation operations on the third cyclic block data and the fourth cyclic block data received each time, and determines the cyclic block result.

[0025] According to one aspect of this disclosure, an artificial intelligence chip is provided, the artificial intelligence chip including the processor as described above.

[0026] According to one aspect of this disclosure, an electronic device is provided, the electronic device including the artificial intelligence chip as described above.

[0027] In this embodiment of the present disclosure, the control logic unit sequentially reads the cyclic block data of the data to be processed from the memory to the dot product unit array according to the acquired control instructions. The dot product unit array then performs multiplication and addition operations on each received cyclic block data of the data to be processed, determining the cyclic block result for each received data. The control logic unit then determines the logical operation result of the data to be processed based on the multiple cyclic block results obtained from the dot product unit array. In this way, the reading and logical operation of the data to be processed (e.g., large-size image data) can be converted into the reading and logical operation of multiple cyclic block data of the data to be processed (small-size image data). This is beneficial for processing larger-size data without changing the processor hardware resources, and reduces the pressure on storage bandwidth.

[0028] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this disclosure. Other features and aspects of this disclosure will become clear from the following detailed description of exemplary embodiments with reference to the accompanying drawings.

Implementation Method

[0030] Various exemplary embodiments, features, and aspects of this disclosure will now be described in detail with reference to the accompanying drawings. The same reference numerals in the drawings denote elements that have the same or similar functions. Although various aspects of the embodiments are shown in the drawings, they are not necessarily drawn to scale unless specifically indicated otherwise.

[0031] The term “exemplary” as used herein means “serving as an example, embodiment or illustration”. Any embodiment illustrated herein as “exemplary” is not necessarily to be construed as superior to or better than other embodiments.

[0032] The term "and / or" in this document is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent three cases: A exists alone, A and B exist simultaneously, and B exists alone. In addition, the term "at least one" in this document means any combination of at least two of any one or more of a plurality of elements. For example, including at least one of A, B, and C can mean including any one or more elements selected from the set consisting of A, B, and C.

[0033] Furthermore, to better illustrate this disclosure, numerous specific details are set forth in the following detailed description. Those skilled in the art should understand that this disclosure can be practiced without certain specific details. In some instances, methods, means, components, and circuits well known to those skilled in the art have not been described in detail in order to highlight the main points of this disclosure.

[0034] FIG1 shows a schematic diagram of a processor according to an embodiment of the present disclosure. As shown in FIG1, the processor includes a control logic unit and a multiplication unit array, the multiplication unit array including a plurality of multiplication units for performing multiplication-accumulation operations.

[0035] The dot product unit may also include an arithmetic unit to perform a specified operation, such as calculating multiple multiplications and accumulating them. The dot product unit may include multipliers, adders, etc., and the specific structures of each dot product unit may be the same or different; this disclosure does not limit this. The dot product unit may also include other types of arithmetic units to adapt to various different operation processes; this disclosure does not limit the number and type of arithmetic units included in the dot product unit.

[0036] The control logic unit can connect to each dot product unit in the dot product unit array. In some embodiments, the control logic unit can number each dot product unit in the form of a two-dimensional matrix or a multi-dimensional matrix so that multiple dot product units can be logically arranged in the form of a two-dimensional matrix or a multi-dimensional matrix, thereby better adapting to matrix logical operations.

[0037] As shown in Figure 1, the memory corresponding to the processor may include a local memory disposed outside the processor. The control logic unit in the processor may be connected to the local memory. The control logic unit may be used for address calculation to load data from the memory into the dot product unit array and control the dot product unit array to process the data to be processed.

[0038] In one possible implementation, the local memory can be an on-chip cache. The control logic unit can load the executable program and the data to be processed (e.g., the input matrix) on the off-chip flash memory into the local memory (on-chip cache) and then perform the logical operation on the data to be processed.

[0039] In one possible implementation, the local memory may store data to be processed and an executable program. The executable program may include control instructions. The processor can execute the control instructions to perform logical operations on the data to be processed, such as matrix multiplication, convolution, and other operations related to multiplication and accumulation.

[0040] In order to reduce the reading of duplicate data, reduce processor power consumption, and increase processor communication bandwidth, the memory may also include a reuse buffer set inside the processor. The reuse buffer can be set inside the dot product unit array (not shown in Figure 1) or outside the dot product unit array (see Figure 1). Compared with setting the reuse buffer outside the dot product unit array, setting the reuse buffer inside the dot product unit array can have higher data reading efficiency. The control logic unit includes a loader and a decoder. The loader loads data to be processed or a portion of the data to be processed from local memory into the processor's reuse cache. The decoder decodes the control instructions for accessing data in the executable program based on the change in the storage address of the data to be processed after loading. For example, for a control instruction to access data X in local memory, since data X is cached in the reuse cache, the address of data X stored in the reuse cache can be obtained through decoding. The decoder can then convert the control instruction to access data X in local memory into a control instruction to access data X in the reuse cache. This facilitates the subsequent control logic unit directly sending the data cached in the reuse cache to the dot product unit, where the dot product unit performs multiplication and addition operations. The control logic unit can also directly load data from off-chip memory into the reuse cache; this disclosure does not limit this approach.

[0041] In one possible implementation, to facilitate the processor's processing of large-scale data, as shown in Figure 1, a corresponding register file can be set up for the processor to read, store, process, and transfer data without being coupled to memory. This significantly reduces instruction execution time when the processor processes large amounts of data. The register file includes, but is not limited to, memory address registers, memory data registers, instruction registers, opcodeword registers, accumulators, and flag registers. In the example, the control logic unit can be used for address calculation to facilitate data transfer between the dot product unit array and the register file.

[0042] In one possible implementation, the processor of this disclosure embodiment may be a completely new design or an improvement on an existing processor chip. The type of processor chip may include, but is not limited to: a central processing unit (CPU), a graphics processing unit (GPU), a general-purpose graphics processing unit (GPGPU), a digital signal processor (DSP), an application specific integrated circuit (ASIC), a tensor processing unit (TPU), a field programmable gate array (FPGA), or other programmable logic devices, and may also include a microprocessor or other conventional processor.

[0043] FIG2 shows a flowchart of a data processing method according to an embodiment of the present disclosure. As shown in FIG2, the data processing method is applied to a processor and may include the following steps:

[0044] In step S11, the control logic unit reads the cyclic block data of the data to be processed from the memory to the dot multiplication unit array according to the acquired control instructions. The control instructions are used to indicate the access information of the data to be processed and the logical operation type of the data to be processed, such as matrix multiplication operation, convolution operation and other multiplication-accumulation related operations of the data to be processed.

[0045] In one possible implementation, the control logic unit can obtain the address information of the data to be processed according to the control instructions, and determine the address information of each cyclic block of the data to be processed in sequence according to the address information of the data to be processed, and read the cyclic block of the data to be processed from the memory to the dot product unit array.

[0046] Optionally, the memory may include a local memory disposed outside the processor, and the control logic unit may read the cyclic block data of the data to be processed from the local memory and put it into the dot product unit array according to the acquired control instructions.

[0047] Optionally, since the cyclic block data of the data to be processed may be repeatedly read, the memory may include, in addition to the local memory located outside the processor, a reuse cache located within the processor for caching the cyclic block data. The control logic unit reads data from the reuse cache faster than it reads data from the local memory. Thus, for cyclic block data in the data to be processed that needs to be repeatedly read, in response to the control logic unit transferring the cyclic block data to be repeatedly read from the local memory to the dot product unit array, the control logic unit may also simultaneously store the cyclic block data to be repeatedly read into the reuse cache located within the processor, so that the control logic unit can subsequently read the cyclic block data from the reuse cache to the dot product unit array.

[0048] In one possible implementation, the data to be processed includes feature data from a deep learning task, and the feature data includes at least one of image feature data, speech feature data, and text feature data.

[0049] For example, in the scenario of using deep neural networks to perform face recognition on a target object, the data to be processed can be image feature data, and the image feature data of the target object (e.g., a face feature map) can be stored in memory; in the scenario of using deep neural networks to perform speech recognition on a target object, the data to be processed can be speech feature data, and the speech feature data of the target object can be stored in memory; in the scenario of using deep neural networks to perform text recognition on a target document, the data to be processed can be text feature data, and the text feature data of the target document can be stored in memory; the embodiments of this application do not limit the type of data to be processed.

[0050] In one possible implementation, the control logic unit can obtain the data structure information and address information of the data to be processed through control instructions; wherein, the data structure information of the data to be processed includes, for example, the dimension of the data to be processed, the size of the data to be processed, and the data type of the elements in the data to be processed (e.g., integer type, single-precision floating-point type, double-precision floating-point type, character type, etc.) used to describe the data to be processed; the address information of the data to be processed includes, for example, the base address of the data to be processed in memory, the address space, and other address-related information.

[0051] In step S12, the dot product unit array performs multiplication and accumulation operations on the cyclic block data of the data to be processed received each time, and determines the cyclic block result of the data to be processed received each time.

[0052] For example, the dot product unit array may include tile_M×tile_N dot product units, each dot product unit can calculate tile_K multiplications and accumulate them, wherein tile_M, tile_N and tile_K are positive integers. This disclosure does not limit the specific values ​​of tile_M, tile_N and tile_K, and they can be set according to the actual application scenario.

[0053] The dot product unit array can perform matrix multiplication or convolution operations by performing multiplication and summation operations on the cyclically segmented data of the received data to be processed each time. For matrix multiplication, the dot product unit array can perform multiplication operations on a matrix of size tile_M×tile_K and a matrix of size tile_K×tile_N. For convolution, the dot product unit array can process tile_M elements to be convolved and tile_N kernel elements at one time, where the number of channels of the tile_M elements to be convolved and the tile_N kernel elements is tile_K.

[0054] In step S13, the control logic unit determines the logical operation result of the data to be processed based on the multiple cyclic block results obtained from the dot product unit array.

[0055] For example, the control logic unit can concatenate and / or add the acquired multiple cyclic block results to determine the logical operation result of the data to be processed. Specifically, the control logic unit can concatenate and / or add each received cyclic block data with the previously received cyclic block data; alternatively, the control logic unit can receive all the cyclic block data and then concatenate and / or add all the cyclic block data. This disclosure does not limit this process.

[0056] In the data processing method of the embodiments of this disclosure, the control logic unit sequentially reads the cyclic block data of the data to be processed from memory to the dot product unit array according to the acquired control instructions for instructing logical operations on the data to be processed. The dot product unit array performs multiplication and accumulation operations on each received cyclic block data of the data to be processed, determining the cyclic block result of each received data. The control logic unit determines the logical operation result of the data to be processed based on the multiple cyclic block results obtained from the dot product unit array. In this way, the reading and logical operation of the data to be processed (e.g., large-size image data) can be converted into the reading and logical operation of multiple cyclic block data of the data to be processed (small-size image data), which is beneficial for processing larger-size data without changing the processor hardware resources and reducing the pressure on storage bandwidth.

[0057] The data processing method of the present disclosure will be described by way of example below.

[0058] In step S11, the control logic unit reads the cyclic block data of the data to be processed from the memory and puts it into the dot product unit array according to the acquired control instructions.

[0059] The memory may include local memory disposed outside the processor; or, in addition to local memory disposed outside the processor, the memory may also include a reuse cache disposed within the processor for caching cyclic block data. The embodiments of this disclosure do not impose specific limitations on this.

[0060] In one possible implementation, step S11 may include: the control logic unit parses the access information of the data to be processed and the logical operation type of the data to be processed according to the acquired control instructions; determines the cyclic block order of the data to be processed according to the logical operation type of the data to be processed; and reads the cyclic block data of the data to be processed from the memory into the dot product unit array in sequence according to the access information and the preset size, in accordance with the cyclic block order of the data to be processed, wherein the size of the cyclic block data is less than or equal to the preset size, and the preset size is determined by the number of dot product units in the dot product unit array that are in an idle state.

[0061] For example, assuming the control instruction is for instructing a matrix multiplication operation on the data to be processed, the control logic unit parses the acquired control instruction to obtain the access information of the data to be processed and the logical operation type of the data to be processed. This access information may include, for example, the size information, address information, and layout information of the data to be processed. The layout information may be row major or column major. Row major indicates that elements in the same row of the data to be processed are adjacent in memory, and column major indicates that elements in the same column of the data to be processed are adjacent in memory. For example, if the logical operation type of the data to be processed is a matrix multiplication operation of a first matrix and a second matrix, the access information may include the size information of the first matrix and the second matrix, address information (e.g., starting address), layout information, etc.

[0062] The control logic unit can determine the cyclic block order of the data to be processed according to the logical operation type of the data to be processed; for example, if the logical operation type of the data to be processed is the matrix multiplication operation of the first matrix and the second matrix, the control logic unit can select a pre-stored cyclic block order that matches the matrix multiplication operation.

[0063] The control logic unit can determine the size of the data to be processed based on the access information of the parsed data to be processed, and can determine whether it is necessary to perform cyclic block processing on the data to be processed based on the size of the data to be processed. If the size of the data to be processed is less than or equal to the preset size, it means that the number of idle dot product units in the dot product unit array can meet the needs of processing the data to be processed. The control logic unit can directly read all the data to be processed from memory into the dot product unit array so that the dot product unit array can perform multiplication and addition operations on the data to be processed.

[0064] If the size of the data to be processed is larger than the preset size, it means that the number of idle dot product units in the dot product unit array cannot meet the needs of processing the data to be processed. The control logic unit can determine the cyclic block order of the data to be processed according to the logical operation type of the data to be processed; and determine the cyclic block data of the data to be processed according to the access information and the preset size of the data to be processed. Then, according to the cyclic block order of the data to be processed, the cyclic block data of the data to be processed is read from the memory to the dot product unit array in sequence. The multiplication and addition operation of multiple cyclic block data is performed by the dot product unit array to realize the multiplication and addition operation of the data to be processed.

[0065] For example, the dot product unit array contains tile_M × tile_N idle dot product units, each of which can calculate tile_K multiplications and accumulate them. For matrix multiplication, the dot product unit array can perform multiplication operations between a matrix of size tile_M × tile_K and a matrix of size tile_K × tile_N. For matrices larger than tile_M, tile_N, and tile_K, the control logic unit is responsible for loop control, sequentially reading the loop block data of the appropriate size from the data to be processed stored in memory and adding it to the dot product unit array according to the loop block order.

[0066] For example, assuming the control instruction is used to instruct a convolution operation on the data to be processed, the control logic unit parses the acquired control instruction and can parse the access information of the data to be processed. This access information includes, for example, the size information, address information, and convolution description information (e.g., the stride and padding of the convolution kernel) of the data to be convolved, and the dimension information of the convolution output result. For example, if the logical operation type of the data to be processed is a convolution operation on the data to be convolved and the convolution kernel, the access information may include convolution description information (e.g., the stride and padding of the convolution kernel), the dimension information of the convolution output result, the size information, address information, and layout information of the data to be convolved and the convolution kernel, etc.

[0067] The control logic unit can determine the cyclic block order of the data to be processed according to the logical operation type of the data to be processed; for example, if the logical operation type of the data to be processed is the convolution operation of the data to be convolved and the convolution kernel, the control logic unit can select a pre-stored cyclic block order that matches the convolution operation.

[0068] Based on the access information of the parsed data to be processed, the control logic unit can determine the size of the data to be convolved and the convolution kernel. Based on the size of the data to be convolved and the convolution kernel, it can determine whether cyclic block processing is required. If the size of both the data to be convolved and the convolution kernel is less than or equal to a preset size, it indicates that the number of idle dot product units in the dot product unit array is sufficient to meet the requirements for convolution operations between the data to be convolved and the convolution kernel. The control logic unit can then directly read the data to be convolved and the convolution kernel from memory into the dot product unit array, so that the dot product unit array performs multiplication and addition operations on them.

[0069] If the size of any data in the data to be convolved or the convolution kernel is larger than the preset size, it means that the number of idle dot product units in the dot product unit array cannot meet the requirements of the data to be convolved and the convolution kernel to perform convolution operation. The control logic unit can determine the cyclic block order of the data to be processed according to the logical operation type of the data to be processed; and determine the cyclic block data from the data to be convolved and / or the convolution kernel that are larger than the preset size according to the access information of the data to be processed and the preset size. Then, according to the cyclic block order of the data to be processed, the corresponding cyclic block data is read from the memory to the dot product unit array in sequence. The multiplication and addition operations of multiple cyclic block data are performed by the dot product unit array to realize the convolution operation between the data to be convolved and the convolution kernel.

[0070] Wherein, the size of the cyclic block data is less than or equal to the preset size, and this disclosure does not impose specific restrictions on the specific size of the cyclic block data.

[0071] For example, the dot product unit array contains tile_M × tile_N idle dot product units, each of which calculates and accumulates tile_K multiplications. For convolution operations, the dot product unit array can process tile_M elements to be convolved and tile_N convolution kernel elements at a time, where the number of channels for the tile_M elements to be convolved and the tile_N convolution kernel elements is tile_K. For data to be processed that is larger than tile_M, tile_N, and tile_K, the control logic unit is responsible for loop control, sequentially reading the looped block data of the appropriate size from the data to be processed stored in memory and adding it to the dot product unit array according to the loop block order.

[0072] In this way, it is beneficial to convert large-sized data to be processed into cyclic block data that meets the size requirements of the dot product unit array computing structure, thereby improving the computing performance of the processor.

[0073] In one possible implementation, the access information of the data to be processed includes access information of a first matrix and a second matrix, wherein the number of columns of the first matrix is ​​the same as the number of rows of the second matrix, the logical operation includes matrix multiplication of the first matrix and the second matrix, and the cyclic block order determined by the matrix multiplication operation type includes: a first outer loop order in the direction of the number of rows of the first matrix, a second outer loop order in the direction of the number of columns of the second matrix, and an inner loop order in the direction of the number of columns of the first matrix and the direction of the number of rows of the second matrix.

[0074] Due to the limited hardware resources of the processor, when the amount of data accessed by the processor is too large, the control logic unit cannot load the required data into the dot product unit array at once. By setting the cyclic block order, it is beneficial to process larger data sizes without changing the processor hardware resources, thereby reducing the pressure on storage bandwidth.

[0075] For example, FIG3 shows a schematic diagram of the cyclic block sequence according to an embodiment of the present disclosure. As shown in FIG3, the control instruction received by the control logic unit may be to perform a matrix multiplication operation on a first matrix A of size M×K and a second matrix B of size K×N, wherein the number of columns K of the first matrix A is the same as the number of rows K of the second matrix B.

[0076] Where C represents the result of matrix multiplication of the first matrix A and the second matrix B, tile_A represents the first cyclic block data of the first matrix A with a size of tile_M×tile_K, tile_B represents the second cyclic block data of the second matrix B with a size of tile_K×tile_N, and tile_C represents the result of matrix multiplication of the first cyclic block data tile_A and the second cyclic block data tile_B.

[0077] The cyclic block order can be multi-layered (e.g., including three nested cyclic layers), the first outer cyclic order according to the row number direction of the first matrix A (as shown in the M direction of Figure 3), the second outer cyclic order according to the column number direction of the second matrix B (as shown in the N direction of Figure 3), and the inner cyclic order according to the column number direction of the first matrix A and the row number direction of the second matrix B (as shown in the K direction of Figure 3).

[0078] Thus, in step S11, the control logic unit can sequentially read the first cyclic block data tile_A of the first matrix A and the second cyclic block data tile_B of the second matrix B from the memory according to the cyclic block order, and then read them into the dot product unit array. In step S12, the dot product unit array performs a multiplication and accumulation operation on each acquired first cyclic block data tile_A and second cyclic block data tile_B to obtain the corresponding cyclic block result tile_C. In step S13, the control logic unit performs an accumulation operation on multiple cyclic block results tile_C acquired sequentially from the dot product unit array in any round of the cyclic block order to obtain a logical operation sub-result. The pseudocode is as follows: For m=1:M / tile_M \\Cycle in the M direction, where m represents the current iteration of the M direction cycle. For n=1:N / tile_N \\Cycle in the N direction, where n represents the current iteration of the N direction cycle. For k=1:K / tile_K \\ Loop in the K direction, k represents the current iteration of the K-direction loop. tile_C mn=tile_C mn+tile_A mk×tile_B kn\\ The initial value of tile_C mn is 0.

[0079] The first outer loop (the loop in the M direction in Figure 3) can loop M / tile_M times, the second outer loop (the loop in the N direction in Figure 3) can loop N / tile_N times, and the inner loop (the loop in the K direction in Figure 3) can loop K / tile_K times. Thus, the first matrix A can be divided into M / tile_M rows and K / tile_K columns, and the second matrix B can be divided into K / tile_K rows and N / tile_N columns. Each row and column in the first matrix A corresponds to a first loop block data tile_A, and tile_A mk represents the first loop block data tile_A in the m-th row and k-th column of the first matrix A. Each row and column in the second matrix B corresponds to a second loop block data tile_B, and tile_B kn represents the second loop block data tile_B in the k-th row and n-th column of the second matrix B.

[0080] tile_A mk×tile_B kn represents the matrix multiplication operation of the first cyclic block data tile_A mk in the m-th row and k-th column of the first matrix A and the second cyclic block data tile_B kn in the k-th row and n-th column of the second matrix B. The dot product array performs multiplication and accumulation operations on the first cyclic block data tile_A mk and the second cyclic block data tile_B kn to obtain the cyclic block result tile_C=tile_A mk×tile_B kn corresponding to the m-th, n-th, and k-th multiplication and accumulation operations.

[0081] The cyclic block order may include m×n rounds of loops. The mth time of any first outer loop (as shown in the M direction in Figure 3) and the nth time of the second outer loop (as shown in the N direction in Figure 3) correspond to all K / tile_K inner loops (as shown in the K direction in Figure 3) as one round of loops, referred to as the mth and nth rounds. The control logic unit can perform cumulative operations on multiple cyclic block results tile_A mk×tile_B kn obtained from the dot product unit array in any round of loop block order to obtain the logical operation sub-result C mn= of the mth and nth rounds.

[0082] Optionally, the cyclic block order determined by the matrix multiplication operation type may also include: a first outer cyclic order according to the column direction of the second matrix, a second outer cyclic order according to the row direction of the first matrix, and an inner cyclic order according to the column direction of the first matrix and the row direction of the second matrix. For example, the cyclic block order may be a first outer cyclic order according to the column direction of the second matrix B (circular in the N direction in Figure 3), a second outer cyclic order according to the row direction of the first matrix A (circular in the M direction in Figure 3), and an inner cyclic order according to the column direction of the first matrix A and the row direction of the second matrix B (circular in the K direction in Figure 3).

[0083] Thus, in step S11, the control logic unit can sequentially read the first cyclic block data tile_A of the first matrix A and the second cyclic block data tile_B of the second matrix B from the memory according to the cyclic block order, and then read them into the dot product unit array. In step S12, the dot product unit array performs a multiplication and accumulation operation on each acquired first cyclic block data tile_A and second cyclic block data tile_B to obtain the corresponding cyclic block result tile_C for each multiplication and accumulation operation. In step S13, the control logic unit performs an accumulation operation on multiple cyclic block results tile_C acquired sequentially from the dot product unit array in any round of the cyclic block order to obtain a logical operation sub-result. The pseudocode is as follows: For m=1:N / tile_N \\Cycle in the N direction, where n represents the current iteration of the N-direction cycle. For n=1:M / tile_M \\Cycle in the M direction, where m represents the current iteration of the M-direction cycle. For k=1:K / tile_K \\ Loop in the K direction, k represents the current iteration of the K-direction loop. tile_C mn=tile_C mn+tile_A mk×tile_B kn\\ The initial value of tile_C mn is 0.

[0084] The specific process can be referred to above, and will not be repeated here.

[0085] In this way, users can implement the cyclic block order with a few simple loop codes, reducing the complexity of software implementation for users.

[0086] In one possible implementation, the previous loop block result can be stored in the accumulator cache (e.g., a part of the reuse cache). When the current loop block data is obtained from the dot product unit array, the previous loop block result can be read from the accumulator cache and added to the current loop block data. The added result is then used to update the current loop block data as the next loop block data. By setting up an accumulator cache for storing intermediate data (e.g., loop block data) of memory loops (e.g., a loop in the K direction), it is beneficial to reduce the number of memory accesses and reduce the pressure on storage bandwidth. In this way, multiple loop block results are accumulated based on the accumulator cache to obtain logical operation sub-results. Then, when the inner loop ends (e.g., the loop in the K direction ends), the control logic unit writes the logical operation sub-results into the temporary register heap; the control logic unit determines the logical operation result based on the multiple logical operation sub-results obtained from the temporary register heap. For example, the control logic unit can write the logical operation sub-result Cmn obtained after each round of all K / tile_K inner loops to the temporary register heap; the control logic unit then concatenates the logical operation sub-results Cmn from the temporary register heap for a total of M / tile_M×N / tile_N rounds to obtain the logical operation results of the first matrix A and the first matrix B. By setting up the temporary register heap, it is beneficial to further reduce the number of accesses to local memory located outside the processor.

[0087] Optionally, if the memory is a local memory located outside the processor, during the process of the control logic unit reading the cyclic block data of the data to be processed from the local memory in a cyclic block order to the dot product unit array, there may be a situation where the cyclic block data is read repeatedly. Figure 4 shows a schematic diagram of the situation where the cyclic block data is read repeatedly according to an embodiment of the present disclosure. As shown in Figure 4, assuming that the first matrix A can be divided into 3×2 blocks, namely: first cyclic block data tile_A 11, first cyclic block data tile_A 12, first cyclic block data tile_A 21, first cyclic block data tile_A 22, first cyclic block data tile_A 31, and first cyclic block data tile_A 32.

[0088] Assume that the second matrix B can be divided into 2×2 blocks, namely: second cyclic block data tile_B 11, second cyclic block data tile_B 12, second cyclic block data tile_B 21, and second cyclic block data tile_B 22.

[0089] According to the cyclic block order, after the first outer loop (e.g., the loop in the row direction of the first matrix A in Figure 4) to the first time, and the second outer loop (e.g., the loop in the column direction of the second matrix B in Figure 4) to the first time, after one round of inner loop order (e.g., the loop in the column direction of the first matrix A and the row direction of the second matrix B in Figure 4, one round of inner loop may include 2 loops), the results of each inner loop are accumulated. After 2 inner loops, the logical operation sub-result C 11 = tile_A 11 × tile_B 11 + tile_A 12 × tile_B 21 can be obtained.

[0090] After the first outer loop iterates to the 1st iteration, the second outer loop iterates to the 2nd iteration, and after one round of inner loop sequence, the results of each inner loop are accumulated. After two inner loops, the logical operation sub-result C12 = tile_A11 × tile_B12 + tile_A12 × tile_B22 can be obtained. Among them, the first loop block data tile_A11 and the first loop block data tile_A12 will be reused. The control logic unit will repeatedly read the first loop block data tile_A11 and the first loop block data tile_A12 from the local memory.

[0091] After the first outer loop reaches its second iteration, and the second outer loop reaches its first iteration, and after one round of inner loop sequence, the results of each inner loop are accumulated. After two inner loops, the logical operation sub-result C 21 = tile_A 21 × tile_B 11 + tile_A 22 × tile_B 21 can be obtained. Among them, the second loop block data tile_B 11 and the second loop block data tile_B 21 will be reused. The control logic unit will repeatedly read the second loop block data tile_B 11 and the second loop block data tile_B 21 from the local memory.

[0092] After the first outer loop reaches its second iteration, and the second outer loop reaches its second iteration, and after one round of inner loop sequence, the results of each inner loop are accumulated. After two inner loops, the logical operation sub-result C22 = tile_A21 × tile_B12 + tile_A22 × tile_B22 can be obtained. Among them, the first loop block data tile_A21 and the first loop block data tile_A22 will be reused, and the second loop block data tile_B12 and the second loop block data tile_B22 will be reused. The control logic unit will repeatedly read the first loop block data tile_A21, the first loop block data tile_A22, the second loop block data tile_B12, and the second loop block data tile_B22 from the local memory.

[0093] After the first outer loop reaches its third iteration and the second outer loop reaches its first iteration, and after one round of inner loop sequence, the results of each inner loop are accumulated. After two inner loops, the logical operation sub-result C 31 = tile_A 31 × tile_B 11 + tile_A 32 × tile_B 21 can be obtained. Among them, the second loop block data tile_B 11 and the second loop block data tile_B 21 will be reused. The control logic unit will repeatedly read the second loop block data tile_B 11 and the second loop block data tile_B 21 from the local memory.

[0094] After the first outer loop reaches its third iteration and the second outer loop reaches its second iteration, and after one round of inner loop sequence, the results of each inner loop are accumulated. After two inner loops, the logical operation sub-result C32 = tile_A31 × tile_B12 + tile_A32 × tile_B22 can be obtained. Among them, the first loop block data tile_A31 and the first loop block data tile_A32 will be reused, and the second loop block data tile_B12 and the second loop block data tile_B22 will be reused. The control logic unit will repeatedly read the first loop block data tile_A31, the first loop block data tile_A32, the second loop block data tile_B12, and the second loop block data tile_B22 from the local memory.

[0095] Optionally, in order to further save the processor's memory access bandwidth, reduce repeated access to the same circular block data in the local memory outside the processor, and improve the processor's computing performance, in one possible implementation, the memory includes a local memory disposed outside the processor and a reuse cache disposed inside the processor for caching circular block data. The step of sequentially reading the circular block data of the data to be processed from the memory to the dot product unit array according to the circular block order of the data to be processed includes: the control logic unit storing the first circular block data of the first matrix read from the local memory into the reuse cache according to the first outer circular order and the inner circular order, so that the control logic unit reuses the first circular block data stored in the reuse cache according to the second outer circular order; or, the control logic unit storing the second circular block data of the second matrix read from the local memory into the reuse cache according to the second outer circular order and the inner circular order, so that the control logic unit reuses the second circular block data stored in the reuse cache according to the first outer circular order.

[0096] For example, as shown in FIG4, according to the cyclic block order, after the first outer loop (e.g., the loop in the row direction of the first matrix A in FIG4) to the first time, the second outer loop (e.g., the loop in the column direction of the second matrix B in FIG4) to the first time, and the inner loop (e.g., the loop in the column direction of the first matrix A and the row direction of the second matrix B in FIG4, one round of inner loop may include 2 loops) to the first time (e.g., M=1, N=1, K=1), the control logic unit can read the first cyclic block data tile_A 11 of the first matrix A and the second cyclic block data tile_B 11 of the second matrix B from the local memory to obtain the cyclic block result tile_A 11×tile_B 11. The first cyclic block data tile_A 11 of the first matrix A and the second cyclic block data tile_B 11 of the second matrix B can be stored in the reuse cache.

[0097] In this way, when the first outer loop reaches the first iteration, the second outer loop reaches the second iteration, and the inner loop reaches the first iteration (e.g., M=1, N=2, K=1), the first loop block data tile_A 11 can be read from the reuse cache, and the second loop block data tile_B 12 can be read from the local memory to obtain the loop block result tile_A 11 × tile_B 12.

[0098] When the first outer loop reaches the second iteration, the second outer loop reaches the first iteration, and the inner loop reaches the first iteration (e.g., M=2, N=1, K=1), the first loop block data tile_A 21 can be read from the local memory, and the second loop block data tile_B 11 can be read from the reuse cache to obtain the loop block result tile_A 21 × tile_B 11.

[0099] When the first outer loop reaches the 3rd iteration, the second outer loop reaches the 1st iteration, and the inner loop reaches the 1st iteration (e.g., M=3, N=1, K=1), the first loop block data tile_A 31 can be read from the local memory, and the second loop block data tile_B 11 can be read from the reuse cache to obtain the loop block result tile_A 31 × tile_B 11.

[0100] Similarly, when the first outer loop reaches the first iteration, the second outer loop reaches the first iteration, and the inner loop reaches the second iteration (e.g., M=1, N=1, K=2), the control logic unit can read the first cyclic block data tile_A 12 of the first matrix A and the second cyclic block data tile_B 21 of the second matrix B from the local memory to obtain the cyclic block result tile_A 12×tile_B 21. The first cyclic block data tile_A 12 of the first matrix A and the second cyclic block data tile_B 21 of the second matrix B can be stored in the reuse cache.

[0101] In this way, when the first outer loop reaches the first iteration, the second outer loop reaches the second iteration, and the inner loop reaches the second iteration (e.g., M=1, N=2, K=2), the first loop block data tile_A 12 can be read from the reuse cache, and the second loop block data tile_B 22 can be read from the local memory to obtain the loop block result tile_A 12 × tile_B 22.

[0102] When the first outer loop reaches the second iteration, the second outer loop reaches the second iteration, and the inner loop reaches the second iteration (e.g., M=2, N=2, K=2), the first loop block data tile_A 22 can be read from the local memory, and the second loop block data tile_B 21 can be read from the reuse cache to obtain the loop block result tile_A 22× tile_B 21.

[0103] When the first outer loop reaches the 3rd iteration, the second outer loop reaches the 2nd iteration, and the inner loop reaches the 2nd iteration (e.g., M=3, N=2, K=2), the first loop block data tile_A 32 can be read from the local memory, and the second loop block data tile_B 21 can be read from the reuse cache, resulting in the loop block result tile_A 32 × tile_B 21.

[0104] It should be understood that the first outer loop, the second outer loop, and the inner loop iterate to other numbers, as described above, and will not be repeated here. How to reuse the first and second loop data blocks based on cache reuse during the multi-layered loop process can be set according to the actual application scenario, and this disclosure does not impose any limitations on this.

[0105] For example, in scenarios where hardware resources are scarce, to reduce the consumption of hardware resources, the reuse cache will not be set too large. For instance, the size of the reuse cache can be set to cache one first loop block of data and one second loop block of data. Only one first loop block of data and one second loop block of data can be cached at a time. Each time the order of the first outer loop or the second outer loop is switched, the loop block data can be read again from the reuse cache. For example, the control logic unit can read the first loop block of the first matrix from the local memory and store it into the reuse cache in response to each update of the first outer loop order, so that the control logic unit can reuse the first loop block of data stored in the reuse cache during the traversal of the second outer loop order. Alternatively, the control logic unit can read the second loop block of the second matrix from the local memory and store it into the reuse cache in response to each update of the second outer loop order, so that the control logic unit can reuse the second loop block of data stored in the reuse cache during the traversal of the first outer loop order.

[0106] By setting up a reuse cache inside the processor, it is beneficial to reduce the number of accesses to local memory outside the processor and reduce the pressure on storage bandwidth.

[0107] In one possible implementation, the access information of the data to be processed includes access information of the data to be convolved and the convolution kernel; the logical operation includes the convolution operation of the data to be convolved and the convolution kernel; the cyclic block order determined by the convolution operation type includes the coordinate order of the elements in the convolution kernel; the step of sequentially reading the cyclic block data of the data to be processed from memory to the dot product unit array according to the cyclic block order of the data to be processed includes: the control logic unit, according to the coordinate order of the elements in the convolution kernel, forming a third cyclic block data by reading multiple elements with the same coordinates from multiple convolution kernels in memory each time; and, according to the coordinates of the current element in each convolution kernel and the access information, reading multiple elements from the data to be convolved in memory to form a fourth cyclic block data; the control logic unit writes the third cyclic block data and the fourth cyclic block data into the dot product unit array.

[0108] The memory may include local memory disposed outside the processor; or, in addition to local memory disposed outside the processor, the memory may also include a reuse cache disposed within the processor for caching cyclic block data. The embodiments of this disclosure do not impose specific limitations on this.

[0109] For example, the access information for the data to be convolved and the convolution kernel includes convolution description information, which can be used to characterize the dimensionality information of the data to be convolved, the dimensionality information of the convolution kernel, the dimensionality information of the convolution output result, layout information, etc. The control logic unit can read the convolution description information and perform a mapping operation on the data to be convolved and the convolution kernel according to the convolution description information. Each mapping operation can read the third loop block data and the fourth loop block data once, and then associate the third loop block data and the fourth loop block data with the dot product unit array and perform calculation. In this way, by traversing each element of the convolution kernel and accumulating the results, the final convolution output result can be obtained.

[0110] In this way, the control logic unit can expand the matrix operation instructions to implement the convolution function according to the read convolution description information, and convert the convolution operation into matrix multiplication, so that the processor has stronger versatility.

[0111] Figure 5 shows a schematic diagram of the third cyclic block data of the convolutional kernel according to an embodiment of the present disclosure. As shown in Figure 5, the memory stores a convolutional kernel C×R×S×K, where C represents the channel dimension of the convolutional kernel, R represents the height dimension of the convolutional kernel, S represents the width dimension of the convolutional kernel, and K represents the number dimension of the convolutional kernel. Each convolutional kernel may include R×S×C / tile_K convolutional kernel elements. Each convolutional kernel element occupies one unit of space in the height dimension R and the width dimension S, and occupies tile_K units of space in the channel dimension C.

[0112] The control logic unit can, according to the coordinate order of the elements in the convolution kernel, for example from the coordinate order of the convolution kernel elements (r=0, s=0, c=0) to (r=R-1, s=S-1, c=C / tile_K-1), firstly read the elements of the first row and first column of each convolution kernel in the channel dimension 0 to tile_K from memory, and map the 8 (K=0~7) elements with the same coordinates (r=0, s=0, c=0) in the 8 (K=0~7) convolution kernels to the first third loop block data tile_B 000; secondly, read the elements of the first row and second column of each convolution kernel in the channel dimension 0 to tile_K from memory, and map the 8 (K=0~7) elements with the same coordinates (r=0, s=1, c=0) in the 8 (K=0~7) convolution kernels to the second third loop block data tile_B. 010; and so on, the last row and last column of each convolutional kernel read from memory for the last time in channel dimension (C-tile_K)~C will be mapped to the last third loop block data tile_B (R-1)(S-1)(C / tile_K-1) with the same coordinates (r=R-1, s=S-1, c=C / tile_K-1) read from the 8 (K=0~7) convolutional kernels in memory.

[0113] As shown in Figure 5, the size of each third loop block data tile_B rsc is tile_K×tile_N, where tile_N represents the number of convolutional kernels and tile_K represents the size of each convolutional kernel element in the channel dimension.

[0114] Meanwhile, Figure 6 shows a schematic diagram of the fourth cyclic block data of the convolution kernel according to an embodiment of the present disclosure. As shown in Figure 6, the memory stores the data to be convolved, C×H×W (e.g., including an input image of size C×H×W), where C represents the channel dimension of the data to be convolved, H represents the height dimension of the data to be convolved, and W represents the width dimension of the kernel to be convolved. The data to be convolved may include C×H×W / tile_K elements to be convolved, each element occupying one unit of space in the height dimension H and the width dimension W, and occupying tile_K units of space in the channel dimension C.

[0115] Since the dimension of the convolution output is determined by the number of times the convolution kernel slides through the data to be convolved, the control logic unit can calculate the coordinates of the data to be convolved according to the coordinate order of the elements in the convolution kernel, based on the coordinates (r, s, c) of the current element in each convolution kernel and the dimension information of the convolution output included in the access information. Then, based on the coordinates of the data to be convolved, it reads `tile_M` elements from the data to be convolved in memory and maps them to the fourth loop block data. The number of `tile_M` elements can be determined by the dimension information of the convolution output.

[0116] For example, the coordinates of the current element in the convolution kernel are (r=0, s=0, c=0). The element in the first row and first column of the convolution output corresponds to the element in the first row and first column of the data to be convolved, with coordinates (h=0, w=0, c=0). The element with coordinates (h=0, w=0, c=0) in the data to be convolved can be used as the first element among the tile_M elements of the fourth cyclic block data. This process continues until tile_M elements are read from the data to be convolved in memory and mapped to the fourth cyclic block data.

[0117] The control logic unit can write the third and fourth loop block data corresponding to the coordinates of each element in the convolution kernel into the dot product unit array, so that the dot product unit array performs matrix multiplication on the third and fourth loop block data. For details, please refer to the matrix multiplication operation above, which will not be repeated here.

[0118] In the embodiments of this disclosure, the third loop block data of the convolution kernel and the fourth loop block data of the data to be convolved can be determined according to the coordinate order of the elements in the convolution kernel. This eliminates the need for im2col (e.g., sliding the convolution kernel on the data to be convolved, converting the data contained in each convolution kernel window into column vectors, and finally arranging them into a new matrix by columns) to expand in memory, thus reducing the pressure on storage bandwidth. Furthermore, this method facilitates the reuse of matrix calculation structures to implement a convolution engine, so as to use convolution description information to extend matrix operation instructions and realize convolution functionality.

[0119] In one possible implementation, the memory includes a local memory located outside the processor and a reuse cache located within the processor for caching cyclic block data. The method further includes: during the calculation of the fourth cyclic block data corresponding to the same row in the convolution output result, the control logic unit writes M elements of the fourth cyclic block number corresponding to the coordinates of the element in the r-th row and s-th column of the convolution kernel into the reuse cache according to the coordinate order of the elements in the convolution kernel, where M is a positive integer; the control logic unit reads M-1 elements from the reuse cache and reads 1 element from the data to be convolved in the local memory to determine the fourth cyclic block data corresponding to the coordinates of the element in the r-th row and s+1-th column of the convolution kernel.

[0120] Figure 7 shows a schematic diagram of a reuse cache according to an embodiment of the present disclosure. As shown in Figure 7, assuming the stride of the convolution kernel is 1, the control logic unit can write the fourth cyclic block data [0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15] corresponding to the coordinates of the element in the r=1 row and s=0 column of the convolution kernel into the reuse cache.

[0121] For the fourth cyclic block data corresponding to the coordinates of the element in the r=1 row and s=1 column of the convolution kernel, the control logic unit only needs to read one more element from the data to be convolved in the local memory. The other 15 elements can be reused from the cache [1,2,3,4,5,6,7,8,9,10,11,12,13,14,15] to form the fourth cyclic block data [1,2,3,4,5,6,7,8,9,10, 11,12,13,14,15,16]. The control logic unit writes the extra element 16 into the cache, so that the cache continues to store [0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16].

[0122] For the fourth cyclic block data corresponding to the coordinates of the element in the r=1 row and s=2 column of the convolution kernel, the control logic unit only needs to read one more element from the data to be convolved in the local memory. The other 15 elements can be reused from the cache [2,3,4,5,6,7,8,9,10,11,12,13,14,15,16] to form the fourth cyclic block data [2,3,4,5,6,7,8,9,10, 11,12,13,14,15,16,17]. The control logic unit writes the extra element 17 into the reuse cache to obtain [0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17].

[0123] By setting up a reuse cache inside the processor, the input data of convolution is reused, which further reduces the number of accesses to the local memory outside the processor and the bandwidth pressure on the local memory outside the processor.

[0124] In one possible implementation, during the calculation of the fourth cyclic block data corresponding to adjacent rows in the convolution output result, in response to the control logic unit calculating the fourth cyclic block data of the current row of the convolution output result, M elements corresponding to the coordinates of the fourth cyclic block number of the element in the r-th row and s-th column of the convolution kernel are written into the reuse cache; in response to the control logic unit calculating the fourth cyclic block data of the next row of the current row of the convolution output result, M elements are read from the reuse cache to form the fourth cyclic block data corresponding to the coordinates of the element in the (r-1)-th row and s-th column of the convolution kernel.

[0125] Figure 8 shows a schematic diagram of another reuse cache according to an embodiment of the present disclosure. As shown in Figure 8, assuming the stride of the convolution kernel is 1, the reuse cache stores data [0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17], wherein [0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15] is the fourth cyclic block data determined by the coordinates of the element in the r=1 row and s=0 column of the convolution kernel when calculating the first row of the convolution output result (see the first row of the convolution result oh_0 in Figure 8), based on the second row of the data to be convolved (see ih_1 in Figure 8); [1,2,3,4,5,6,7,8,9,10,11,12] is the fourth cyclic block data determined by the coordinates of the element in the r=1 row and s=0 column of the convolution kernel; [1,2,3,4,5,6,7,8,9,10,11,12] is the element in the r=1 row and s=0 column of the convolution kernel. [13,14,15,16] refers to the fourth cyclic block data determined by the coordinates of the element in the r=1 row and s=1 column of the convolution kernel when calculating the first row of the convolution output (see the first row of the convolution result oh_0 in Figure 8), based on the second row of the data to be convolved (see ih_1 in Figure 8). [2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17] refers to the fourth cyclic block data determined by the coordinates of the element in the r=1 row and s=2 column of the convolution kernel when calculating the first row of the convolution output (see the first row of the convolution result oh_0 in Figure 8), based on the second row of the data to be convolved (see ih_1 in Figure 8).

[0126] It should be understood that the fourth cyclic block data [1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16] determined by the coordinates of the element in the r=1th row and s=1st column of the convolution kernel can be reused by utilizing the reuse cache, and the fourth cyclic block data [0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15] determined by the coordinates of the element in the r=1th row and s=0th column of the convolution kernel can be reused; similarly, the fourth cyclic block data [2,3,4,5,6,7,8,9,10,...] determined by the coordinates of the element in the r=1th row and s=2nd column of the convolution kernel can be reused. [11,12,13,14,15,16,17] can utilize the reuse cache to reuse the fourth cyclic block data [1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16] determined by the coordinates of the element in the r=1 row and s=1 column of the convolution kernel. For details, please refer to the description related to Figure 7, which will not be repeated here.

[0127] Considering that adjacent rows in the convolution output result may reuse data, for example, when calculating the second row of the convolution output result (see the second row of the convolution result oh_1 in Figure 8), the element to be convolved corresponding to the r=0 row of the convolution kernel (see the second row of the data to be convolved in Figure 8 ih_1) is the same as the element to be convolved corresponding to the r=1 row of the convolution kernel in the previous row of the convolution output result (see the first row of the convolution result oh_0 in Figure 8) (also the second row of the data to be convolved in Figure 8 ih_1), which can be reused to reduce the amount of data read from local memory.

[0128] For example, when calculating the second row of the convolution output (see the second row oh_1 of the convolution result in Figure 8), the element to be convolved corresponding to the element in the r=0 row and s=0 column of the convolution kernel (see the second row ih_1 of the data to be convolved in Figure 8) is the same as the element to be convolved corresponding to the r=1 row and s=0 column of the convolution kernel in the previous row of the convolution output (see the first row oh_0 of the convolution result in Figure 8) (also the second row ih_1 of the data to be convolved in Figure 8). The reuse cache can be utilized to reuse the fourth cyclic block data [0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15] determined by the r=1 row and s=0 column of the convolution kernel when calculating the first row of the convolution output (see the first row of the convolution output oh_0 in Figure 8), and the fourth cyclic block data [0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15] determined by the r=0 row and s=0 column of the convolution kernel when calculating the second row of the convolution output (see the second row of the convolution output oh_1 in Figure 8).

[0129] Similarly, the reuse cache can be used to reuse the fourth cyclic block data [1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16] determined by the r=1 row and s=1 column of the convolution kernel when calculating the first row of the convolution output (see the first row of the convolution result oh_0 in Figure 8), as the fourth cyclic block data [1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16] determined by the element in the r=0 row and s=1 column of the convolution kernel when calculating the second row of the convolution output (see the second row of the convolution result oh_1 in Figure 8). Furthermore, when reusing the first row of the convolution output (see the first row of the convolution result oh_0 in Figure 8), the fourth cyclic block data [2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17] determined by the r=1 row and s=2 column of the convolution kernel is used as the second row of the convolution output (see the second row of the convolution result oh_1 in Figure 8), and the fourth cyclic block data [2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17] determined by the elements in the r=0 row and s=2 column of the convolution kernel is used as the second row of the convolution output (see the second row of the convolution result oh_1 in Figure 8).

[0130] Compared with related technologies, implementing convolution operations requires the addition of software im2col (e.g., sliding the convolution kernel on the data to be convolved, then converting the data contained in each convolution kernel window into column vectors, and finally arranging them into a new matrix by column), which introduces additional overhead. For example, a common 3x3 convolution kernel requires 9 times the amount of data, putting significant pressure on storage bandwidth. The embodiments of this disclosure can determine the third loop block data of the convolution kernel and the fourth loop block data of the data to be convolved according to the coordinate order of the elements in the convolution kernel. This allows for the reuse of the matrix calculation structure to implement the convolution engine, eliminating the need for im2col to expand in memory, thus reducing the pressure on storage bandwidth. Furthermore, by setting up a reuse cache within the processor, the input data for convolution is reused, further reducing the number of accesses to external local memory and the bandwidth pressure on external local memory.

[0131] In one possible implementation, the control logic unit, according to the coordinate order of the elements in the convolution kernel, forms a third cyclic block data by reading multiple elements with the same coordinates from multiple convolution kernels in the memory each time, including: the control logic unit reads multiple convolution kernels from the local memory to the reuse cache; according to the coordinate order of the elements in the convolution kernel, the control logic unit forms a third cyclic block data by reading multiple elements with the same coordinates from multiple convolution kernels in the reuse cache each time.

[0132] For example, assuming that the local memory stores convolution kernels [S1,S2], [S3,S4], and [S5,S6], the control logic unit can read these three convolution kernels [S1,S2], [S3,S4], and [S5,S6] from the local memory into the reuse cache; in this way, the logic control unit reads multiple elements S1, S3, and S5 at coordinate 1 from the reuse cache in the first order according to the coordinate order of the elements in the convolution kernels [S1,S2], [S3,S4], and [S5,S6], forming the third cyclic block data [S1,S3,S5]; and reads multiple elements S2, S4, and S6 at coordinate 2 from the reuse cache in the second order, forming the third cyclic block data [S2,S4,S6]. It should be understood that this disclosure uses convolution kernels [S1,S2], [S3,S4], and [S5,S6] as examples only, and does not limit the size or number of convolution kernels.

[0133] In this way, the convolution kernel has been read into the reuse cache inside the processor, and the convolution kernel stored in the reuse cache can be reused, avoiding repeated reading of local memory outside the processor.

[0134] In one possible implementation, the dot product unit array performs multiplication and accumulation operations on the cyclic block data of the received data to be processed each time to determine the cyclic block result of the received data to be processed each time, including: the dot product unit array performs multiplication and accumulation operations on the third and fourth cyclic block data each time to determine the cyclic block result. For details, please refer to the above description of the dot product unit array performing multiplication and accumulation operations on the first and second cyclic block data each time, which will not be repeated here.

[0135] The subsequent control logic unit can perform cumulative operations on multiple cyclic block results obtained from the dot product unit array in the cyclic block order to obtain logical operation sub-results and write them into the temporary storage heap; the control logic unit can determine the convolution output result based on the multiple logical operation sub-results obtained from the temporary storage heap.

[0136] This approach facilitates the reuse of matrix computation structures to implement convolution engines, which in turn reduces the complexity of software implementation for users.

[0137] In the data processing method of the embodiments of this disclosure, the control logic unit sequentially reads the cyclic block data of the data to be processed from memory to the dot product unit array according to the acquired control instructions for instructing logical operations on the data to be processed. The dot product unit array performs multiplication and addition operations on each received cyclic block data of the data to be processed, determining the cyclic block result of each received data. The control logic unit determines the logical operation result of the data to be processed based on the multiple cyclic block results obtained from the dot product unit array. In this way, the reading and logical operation of the data to be processed (e.g., large-size image data) can be converted into the reading and logical operation of multiple cyclic block data of the data to be processed (small-size image data), which is beneficial for processing larger-size data without changing the processor hardware resources and reducing the pressure on storage bandwidth.

[0138] It is understood that the various method embodiments mentioned above in this disclosure can be combined with each other to form combined embodiments without violating the principle and logic. Due to space limitations, this disclosure will not elaborate further. Those skilled in the art will understand that in the above methods of specific implementation, the specific execution order of each step should be determined by its function and possible internal logic.

[0139] In addition, this disclosure also provides processors, electronic devices, computer-readable storage media, and programs, all of which can be used to implement any of the data processing methods provided in this disclosure. The corresponding technical solutions and descriptions are described in the corresponding records in the method section, and will not be repeated here.

[0140] In one possible implementation, as shown in FIG1, the processor includes a control logic unit and a multiplication unit array. The multiplication unit array includes multiple multiplication units for performing multiplication-accumulation operations. The processor is configured to: the control logic unit sequentially read cyclic block data of the data to be processed from memory to the multiplication unit array according to the acquired control instructions, wherein the control instructions are used to instruct the logical operation of the data to be processed; the multiplication unit array performs multiplication-accumulation operations on each received cyclic block data of the data to be processed to determine the cyclic block result of the data to be processed each time; and the control logic unit determines the logical operation result of the data to be processed based on the multiple cyclic block results acquired from the multiplication unit array.

[0141] In one possible implementation, the control logic unit sequentially reads the cyclic block data of the data to be processed from the memory to the dot product unit array according to the acquired control instructions, including: the control logic unit parses the access information of the data to be processed and the logical operation type of the data to be processed according to the acquired control instructions; determines the cyclic block order of the data to be processed according to the logical operation type of the data to be processed; and sequentially reads the cyclic block data of the data to be processed from the memory to the dot product unit array according to the access information and the preset size, in accordance with the cyclic block order of the data to be processed, wherein the size of the cyclic block data is less than or equal to the preset size, and the preset size is determined by the number of dot product units in the dot product unit array that are in an idle state.

[0142] In one possible implementation, the access information of the data to be processed includes access information of a first matrix and a second matrix, wherein the number of columns of the first matrix is ​​the same as the number of rows of the second matrix, the logical operation type includes matrix multiplication of the first matrix and the second matrix, and the cyclic block order determined by the matrix multiplication operation type includes: a first outer cyclic order in the direction of the number of rows of the first matrix, a second outer cyclic order in the direction of the number of columns of the second matrix, and an inner cyclic order in the direction of the number of columns of the first matrix and the direction of the number of rows of the second matrix; or, a first outer cyclic order in the direction of the number of columns of the second matrix, a second outer cyclic order in the direction of the number of rows of the first matrix, and an inner cyclic order in the direction of the number of columns of the first matrix and the direction of the number of rows of the second matrix.

[0143] In one possible implementation, the memory includes a local memory disposed outside the processor and a reuse cache disposed within the processor for caching cyclic block data. The step of sequentially reading the cyclic block data of the data to be processed from the memory to the dot product unit array according to the cyclic block order of the data to be processed includes: the control logic unit storing the first cyclic block data of the first matrix read from the local memory into the reuse cache according to the first outer cyclic order and the inner cyclic order, so that the control logic unit reuses the first cyclic block data stored in the reuse cache according to the second outer cyclic order; or, the control logic unit storing the second cyclic block data of the second matrix read from the local memory into the reuse cache according to the second outer cyclic order and the inner cyclic order, so that the control logic unit reuses the second cyclic block data stored in the reuse cache according to the first outer cyclic order.

[0144] In one possible implementation, the dot product unit array performs multiplication and accumulation operations on the cyclic block data of the received data to be processed each time to determine the cyclic block result of the received data to be processed each time, including: the dot product unit array performs multiplication and accumulation operations on the first cyclic block data and the second cyclic block data obtained each time to obtain the cyclic block result corresponding to each multiplication and accumulation operation; the control logic unit determines the logical operation result of the data to be processed based on the multiple cyclic block results obtained from the dot product unit array, including: the control logic unit performs accumulation operations on the multiple cyclic block results obtained from the dot product unit array in any round of cyclic block order to obtain a logical operation sub-result; the control logic unit writes the logical operation sub-result into a temporary register heap; the control logic unit determines the logical operation result based on the multiple logical operation sub-results obtained from the temporary register heap.

[0145] In one possible implementation, the access information of the data to be processed includes access information of the data to be convolved and the convolution kernel, the logical operation type includes the convolution operation of the data to be convolved and the convolution kernel, the cyclic block order determined by the convolution operation type includes the coordinate order of the elements in the convolution kernel, and the step of sequentially reading the cyclic block data of the data to be processed from the memory to the dot product unit array according to the cyclic block order of the data to be processed includes: the control logic unit, according to the coordinate order of the elements in the convolution kernel, forming a third cyclic block data by reading multiple elements with the same coordinates from multiple convolution kernels in the memory each time, and, according to the coordinates of the current element in each convolution kernel and the access information, reading multiple elements from the data to be convolved in the memory to form a fourth cyclic block data; the control logic unit writes the third cyclic block data and the fourth cyclic block data into the dot product unit array.

[0146] In one possible implementation, the memory includes a local memory located outside the processor and a reuse cache located within the processor for caching cyclic block data. The processor is further configured to: during the calculation of the fourth cyclic block data corresponding to the same row in the convolution output result, the control logic unit writes M elements of the fourth cyclic block number corresponding to the coordinates of the element in the r-th row and s-th column of the convolution kernel into the reuse cache according to the coordinate order of the elements in the convolution kernel, where r, s, and M are positive integers; the control logic unit reads M-1 elements from the reuse cache and reads 1 element from the data to be convolved in the local memory to determine the fourth cyclic block data corresponding to the coordinates of the element in the r-th row and s+1-th column of the convolution kernel.

[0147] In one possible implementation, the processor is further configured to: during the calculation of the fourth cyclic block data corresponding to adjacent rows in the convolution output result, in response to the control logic unit calculating the fourth cyclic block data of the current row of the convolution output result, write M elements of the fourth cyclic block number corresponding to the coordinates of the element in the r-th row and s-th column of the convolution kernel into a reuse cache, where r, s, and M are positive integers; in response to the control logic unit calculating the fourth cyclic block data of the next row of the current row of the convolution output result, read M elements from the reuse cache to form the fourth cyclic block data corresponding to the coordinates of the element in the (r-1)-th row and s-th column of the convolution kernel.

[0148] In one possible implementation, the memory includes a local memory located outside the processor and a reuse cache located within the processor for caching cyclic block data. The control logic unit, according to the coordinate order of the elements in the convolution kernel, constructs a third cyclic block data by taking multiple elements with the same coordinates read from multiple convolution kernels in the memory each time. This includes: the control logic unit reading multiple convolution kernels from the local memory to the reuse cache; and the control logic unit constructing a third cyclic block data by taking multiple elements with the same coordinates read from multiple convolution kernels in the reuse cache each time, according to the coordinate order of the elements in the convolution kernel.

[0149] In one possible implementation, the dot product unit array performs multiplication and accumulation operations on the cyclic block data of the data to be processed received each time, and determines the cyclic block result of the data to be processed received each time, including: the dot product unit array performs multiplication and accumulation operations on the third cyclic block data and the fourth cyclic block data received each time, and determines the cyclic block result.

[0150] In some embodiments, the processor provided in this disclosure has functions or includes modules that can be used to execute the methods described in the above method embodiments. The specific implementation can be referred to the description of the above method embodiments. For the sake of brevity, it will not be repeated here.

[0151] This disclosure also provides a computer-readable storage medium storing computer program instructions thereon, which, when executed by a processor, implement the above-described method. The computer-readable storage medium may be volatile or non-volatile.

[0152] This disclosure also proposes an artificial intelligence chip, which includes the processor described above.

[0153] This disclosure also proposes an electronic device, which includes the processor described above. The electronic device may include user equipment (UE), mobile device, user terminal, terminal, cellular phone, cordless phone, personal digital assistant (PDA), handheld device, computing device, in-vehicle device, wearable device, etc.

[0154] This disclosure also provides a computer program product, including computer-readable code, or a non-volatile computer-readable storage medium carrying computer-readable code, wherein when the computer-readable code is run in a processor of an electronic device, the processor in the electronic device performs the above-described method.

[0155] Electronic devices may be provided as terminals, servers or other forms of devices.

[0156] FIG9 shows a block diagram of an electronic device 1900 according to an embodiment of the present disclosure. For example, the electronic device 1900 may be provided as a server or terminal device. Referring to FIG9, the electronic device 1900 includes a processing component 1922, which further includes one or more processors, and memory resources represented by memory 1932 for storing instructions executable by the processing component 1922, such as applications. The applications stored in memory 1932 may include one or more modules, each corresponding to a set of instructions. Furthermore, the processing component 1922 is configured to execute instructions to perform the methods described above.

[0157] Electronic device 1900 may also include a power supply component 1926 configured to perform power management of electronic device 1900, a wired or wireless network interface 1950 configured to connect electronic device 1900 to a network, and an input / output (I / O) interface 1958. Electronic device 1900 can operate on an operating system stored in memory 1932, such as Microsoft Windows Server™, Apple's graphical user interface-based operating system (Mac OS X™), a multi-user, multi-process computer operating system (Unix™), a free and open-source Unix-like operating system (Linux™), an open-source Unix-like operating system (FreeBSD™), or similar.

[0158] In an exemplary embodiment, a non-volatile computer-readable storage medium is also provided, such as a memory 1932 including computer program instructions, which can be executed by a processing component 1922 of an electronic device 1900 to perform the above method.

[0159] This disclosure may be a system, method, and / or computer program product. A computer program product may include a computer-readable storage medium having computer-readable program instructions loaded thereon for causing a processor to implement various aspects of this disclosure.

[0160] A computer-readable storage medium can be a tangible device capable of holding and storing instructions for use by an instruction execution device. A computer-readable storage medium can be, for example, (but not limited to) an electrical storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination thereof. More specific examples (a non-exhaustive list) of computer-readable storage media include: portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), static random access memory (SRAM), portable compact disc read-only memory (CD-ROM), digital multifunction disc (DVD), memory sticks, floppy disks, mechanical encoding devices, such as punch cards or recessed protrusions storing instructions thereon, and any suitable combination thereof. The computer-readable storage medium as used herein is not to be construed as a transient signal itself, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through waveguides or other transmission media (e.g., light pulses through fiber optic cables), or electrical signals transmitted through wires.

[0161] The computer-readable program instructions described herein can be downloaded from computer-readable storage media to various computing / processing devices, or downloaded via a network, such as the Internet, local area network, wide area network, and / or wireless network, to an external computer or external storage device. The network may include copper transmission cables, fiber optic transmission, wireless transmission, routers, firewalls, switches, gateway computers, and / or edge servers. A network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards them to computer-readable storage media stored in the respective computing / processing device.

[0162] The computer program instructions used to perform the operations of this disclosure may be combination instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, status setting data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Smalltalk, C++, etc., and conventional procedural programming languages ​​such as the "C" language or similar programming languages. The computer-readable program instructions may be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider). In some embodiments, electronic circuits, such as programmable logic circuits, field-programmable gate arrays (FPGAs), or programmable logic arrays (PLAs), are personalized by utilizing state information from computer-readable programmable instructions. These electronic circuits can execute computer-readable programmable instructions to implement various aspects of this disclosure.

[0163] Various aspects of this disclosure are described herein with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this disclosure. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer-readable program instructions.

[0164] These computer-readable program instructions may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus to produce a machine such that, when executed by the processor of the computer or other programmable data processing apparatus, they create means for implementing the functions / actions specified in one or more blocks of the flowchart and / or block diagram. These computer-readable program instructions may also be stored in a computer-readable storage medium that causes a computer, programmable data processing apparatus, and / or other device to operate in a particular manner, thereby comprising an article of manufacture including instructions for implementing aspects of the functions / actions specified in one or more blocks of the flowchart and / or block diagram.

[0165] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions executed on the computer, other programmable data processing apparatus, or other device to perform the functions / actions specified in one or more blocks of a flowchart and / or block diagram.

[0166] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of instructions containing one or more executable instructions for implementing a specified logical function. In some alternative implementations, the functions marked in the blocks may occur in a different order than those marked in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, may be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions.

[0167] The computer program product can be implemented specifically through hardware, software, or a combination thereof. In one optional embodiment, the computer program product is specifically embodied as a computer storage medium; in another optional embodiment, the computer program product is specifically embodied as a software product, such as a software development kit (SDK), etc.

[0168] The description of the various embodiments above tends to emphasize the differences between the various embodiments. The similarities or similarities can be referred to each other. For the sake of brevity, they will not be repeated here.

[0169] Those skilled in the art will understand that in the above method of the specific implementation, the order of writing each step does not mean a strict execution order and does not constitute any limitation on the implementation process. The specific execution order of each step should be determined by its function and possible internal logic.

[0170] The various embodiments of this disclosure have been described above. These descriptions are exemplary and not exhaustive, and are not limited to the disclosed embodiments. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described embodiments. The terminology used herein is chosen to best explain the principles, practical application, or improvement of the technology in the market, or to enable others skilled in the art to understand the embodiments disclosed herein. [Simplified Explanation of the Diagram]

[0029] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with this disclosure and, together with the description, serve to illustrate the technical solutions of this disclosure. Figure 1 shows a schematic diagram of a processor according to an embodiment of this disclosure. Figure 2 shows a flowchart of a data processing method according to an embodiment of this disclosure. Figure 3 shows a schematic diagram of a cyclic block sequence according to an embodiment of this disclosure. Figure 4 shows a schematic diagram of repeated reading of cyclic block data according to an embodiment of this disclosure. Figure 5 shows a schematic diagram of the third cyclic block data of a convolution kernel according to an embodiment of this disclosure. Figure 6 shows a schematic diagram of the fourth cyclic block data of a convolution kernel according to an embodiment of this disclosure. Figure 7 shows a schematic diagram of a reuse cache according to an embodiment of this disclosure. Figure 8 shows a schematic diagram of another reuse cache according to an embodiment of this disclosure. Figure 9 shows a block diagram of an electronic device according to an embodiment of this disclosure.

Claims

1. A data processing method, characterized in that the data processing method is applied to a processor, the processor comprising a control logic unit and a multiplication unit array, the multiplication unit array comprising a plurality of multiplication units for performing multiplication-accumulation operations, the method comprising: The control logic unit, according to the acquired control instructions, sequentially reads the cyclic block data of the data to be processed from the memory to the dot product unit array. The memory includes a local memory located outside the processor and a reuse cache located within the processor for caching the cyclic block data. In response to the control logic unit transmitting the cyclic block data that needs to be repeatedly read from the local memory to the dot product unit array, the control logic unit also simultaneously stores the cyclic block data that needs to be repeatedly read into the reuse cache. The dot product unit array performs multiplication and addition operations on each received cyclic block data of the data to be processed to determine the cyclic block result of each received data. The control logic unit determines the logical operation result of the data to be processed based on the multiple cyclic block results obtained from the dot product unit array.

2. The method as described in claim 1, characterized in that the control logic unit sequentially reads the cyclic block data of the data to be processed from memory to the dot product unit array according to the acquired control instructions, including: The control logic unit parses the access information of the data to be processed and the logical operation type of the data to be processed according to the acquired control instructions. Based on the logical operation type of the data to be processed, determine the cyclic block order of the data to be processed; based on the access information and the preset size, read the cyclic block data of the data to be processed from the memory sequentially into the dot product unit array according to the cyclic block order of the data to be processed, wherein the size of the cyclic block data is less than or equal to the preset size, and the preset size is determined by the number of dot product units in the dot product unit array that are in an idle state.

3. The method as described in claim 2, characterized in that the access information of the data to be processed includes access information of a first matrix and a second matrix, the number of columns of the first matrix is ​​the same as the number of rows of the second matrix, the logical operation type includes matrix multiplication of the first matrix and the second matrix, and the cyclic block order determined by the matrix multiplication operation type includes: The first outer loop order is based on the row direction of the first matrix, the second outer loop order is based on the column direction of the second matrix, and the inner loop order is based on both the column direction of the first matrix and the row direction of the second matrix; or, the first outer loop order is based on the column direction of the second matrix, the second outer loop order is based on the row direction of the first matrix, and the inner loop order is based on both the column direction of the first matrix and the row direction of the second matrix.

4. The method as described in claim 3, characterized in that, the step of sequentially reading the cyclic block data of the data to be processed from memory into the dot product unit array according to the cyclic block order of the data to be processed includes: The control logic unit stores the first cyclic block data of the first matrix read from the local memory into the reuse cache according to the first outer loop order and the inner loop order, so that the control logic unit reuses the first cyclic block data stored in the reuse cache according to the second outer loop order; or, the control logic unit stores the second cyclic block data of the second matrix read from the local memory into the reuse cache according to the second outer loop order and the inner loop order, so that the control logic unit reuses the second cyclic block data stored in the reuse cache according to the first outer loop order.

5. The method as described in claim 4, characterized in that the dot product unit array performs multiplication and accumulation operations on the cyclically segmented data of the received data to be processed each time, and determines the cyclically segmented result of the received data to be processed each time, including: The dot product unit array performs multiplication and accumulation operations on the first and second cyclic block data acquired each time to obtain the cyclic block result corresponding to each multiplication and accumulation operation; the control logic unit determines the logical operation result of the data to be processed based on the multiple cyclic block results acquired from the dot product unit array, including: the control logic unit performs accumulation operations on the multiple cyclic block results acquired from the dot product unit array in any round of cyclic block order to obtain a logical operation sub-result; the control logic unit writes the logical operation sub-result into a temporary register heap; the control logic unit determines the logical operation result based on the multiple logical operation sub-results acquired from the temporary register heap.

6. The method as described in claim 2, characterized in that the access information of the data to be processed includes access information of the data to be convolved and the convolution kernel, the logical operation type includes the convolution operation of the data to be convolved and the convolution kernel, the cyclic block order determined by the convolution operation type includes the coordinate order of the elements in the convolution kernel, and the step of sequentially reading the cyclic block data of the data to be processed from memory to the dot product unit array according to the cyclic block order of the data to be processed includes: The control logic unit, according to the coordinate order of the elements in the convolution kernel, reads multiple elements with the same coordinates from multiple convolution kernels in the memory each time to form a third cyclic block data; and, according to the coordinates of the current element in each convolution kernel and the access information, reads multiple elements from the data to be convolved in the memory to form a fourth cyclic block data; the control logic unit writes the third cyclic block data and the fourth cyclic block data into the dot product unit array.

7. The method as described in claim 6, characterized in that the method further comprises: During the calculation of the fourth loop block data corresponding to the same row in the convolution output result, the control logic unit writes M elements corresponding to the fourth loop block number in the r-th row and s-th column of the convolution kernel into the reuse cache according to the coordinate order of the elements in the convolution kernel, where r, s, and M are positive integers; the control logic unit reads M-1 elements from the reuse cache and reads 1 element from the data to be convolved in the local memory to determine the fourth loop block data corresponding to the coordinates of the r-th row and s+1-th column of the convolution kernel.

8. The method as described in claim 6, characterized in that the method further comprises: During the calculation of the fourth loop block data corresponding to adjacent rows in the convolution output result, in response to the control logic unit calculating the fourth loop block data of the current row of the convolution output result, M elements corresponding to the coordinates of the fourth loop block number of the element in the r-th row and s-th column of the convolution kernel are written into the reuse cache, where r, s, and M are positive integers; in response to the control logic unit calculating the fourth loop block data of the next row of the convolution output result, M elements are read from the reuse cache to form the fourth loop block data corresponding to the coordinates of the element in the (r-1)-th row and s-th column of the convolution kernel.

9. The method as described in claim 6, characterized in that the memory includes a local memory disposed outside the processor, and a reuse cache disposed within the processor for caching cyclic block data, wherein the control logic unit, according to the coordinate order of the elements in the convolution kernel, constructs a third cyclic block data by taking multiple elements with the same coordinates read from multiple convolution kernels in the memory each time, including: The control logic unit reads multiple convolution kernels from the local memory into the reuse cache; According to the coordinate order of the elements in the convolution kernel, the control logic unit will each read multiple elements with the same coordinates from multiple convolution kernels in the reuse cache to form a third cyclic block of data.

10. The method according to any one of claims 6-9, characterized in that the dot product unit array performs multiplication and accumulation operations on the cyclically segmented data of the data to be processed received each time, and determines the cyclically segmented result of the data to be processed received each time, including: The dot product unit array performs multiplication and accumulation operations on each received third and fourth cyclic block data to determine the cyclic block result.

11. A processor, characterized in that the processor comprises a control logic unit and a multiplication unit array, the multiplication unit array comprising a plurality of multiplication units for performing multiplication-accumulation operations, the processor being configured to: the control logic unit sequentially read cyclic blocks of data to be processed from memory and load them into the multiplication unit array according to acquired control instructions, wherein, The memory includes a local memory located outside the processor and a reuse cache located within the processor for caching cyclic block data. In response to the control logic unit transmitting repeatedly read cyclic block data obtained from the local memory to the dot product unit array, the control logic unit also simultaneously stores the repeatedly read cyclic block data into the reuse cache. The dot product unit array performs multiplication and addition operations on the cyclic block data of the data to be processed each time it is received, determining the cyclic block result of the data to be processed each time. The control logic unit determines the logical operation result of the data to be processed based on the multiple cyclic block results obtained from the dot product unit array.

12. An artificial intelligence chip, characterized in that the artificial intelligence chip includes a processor as described in claim 11.

13. An electronic device, characterized in that the electronic device comprises an artificial intelligence chip as described in claim 12.

Images