Data processing method, processor, chip, and electronic device

Abstract

The present disclosure relates to a data processing method, a processor, a chip, and an electronic device. The method comprises: on the basis of an obtained control instruction, a control logic unit sequentially reads, from a memory to a dot product unit array, loop tiling data of data to be processed; the dot product unit array performs multiply-accumulate operation on the loop tiling data of the data to be processed that is received each time, and determines a loop tiling result of the data to be processed that is received each time; and on the basis of a plurality of loop tiling results obtained from the dot product unit array, the control logic unit determines a logic operation result of the data to be processed. The embodiments of the present disclosure can convert, into the reading and logic operation of multiple pieces of loop tiling data of the data to be processed, the reading and logic operation of the data to be processed, so that data with larger size can be processed under the condition that the hardware resources of the processor are not changed, and the pressure on the storage bandwidth is reduced.

Description

Data processing method, processor, chip and electronic device

[0001] The present application claims priority to the Chinese patent application No. 202410472589.9, filed on April 18, 2024, and entitled "Data processing method, processor, chip and electronic device", the whole content of which is incorporated herein by reference. TECHNICAL FIELD

[0002] The present disclosure relates to the field of computer technology, and particularly relates to a data processing method, a processor, a chip and an electronic device. BACKGROUND

[0003] In the field of artificial intelligence technology, neural network algorithm is a very popular machine learning algorithm in recent years, which has achieved very good results in various fields, such as image recognition, speech recognition, natural language processing, etc. With the development of neural network algorithm, the complexity of the algorithm is also getting higher and higher, and in order to improve the recognition degree, the size of the model is also gradually increasing. Using central processing unit (CPU), graphic processing unit (GPU) and other processors to process these large-scale models will take a lot of computing time and consume a lot of power.

[0004] A large number of special modules are added to the processor and artificial intelligence accelerator to accelerate training and inference. Among them, matrix multiplication and convolution are common operators in artificial intelligence algorithms. The single instruction multiple data (SIMD) or single instruction multiple thread (SIMT) architecture of the processor in the related art relies on the multiply-accumulate instruction to implement the corresponding algorithm, and the data sharing between different threads is less, and the repeated data needs to be read multiple times, so the performance and energy consumption of the processor are poor. SUMMARY

[0005] The present disclosure provides a data processing technical solution.

[0006] According to an aspect of the present disclosure, a data processing method is provided, which is applied to a processor including a control logic unit and a point multiplication unit array including a plurality of point multiplication units for performing a product-sum operation. The method includes: sequentially reading, by the control logic unit, cyclic block data of to-be-processed data from a memory to the point multiplication unit array according to a control instruction obtained; performing, by the point multiplication unit array, a product-sum operation on each received cyclic block data of the to-be-processed data to determine a cyclic block result of the to-be-processed data; and determining, by the control logic unit, a logical operation result of the to-be-processed data according to a plurality of cyclic block results obtained from the point multiplication unit array.

[0007] In a possible implementation, the control logic unit sequentially reads cyclic block data of to-be-processed data from a memory to the point multiplication unit array according to a control instruction obtained, including: parsing, by the control logic unit, access information of the to-be-processed data and a logical operation type of the to-be-processed data according to the control instruction obtained; determining a cyclic block sequence of the to-be-processed data according to the logical operation type of the to-be-processed data; and sequentially reading, by the control logic unit, cyclic block data of the to-be-processed data from the memory to the point multiplication unit array according to the access information and a preset size, in the cyclic block sequence of the to-be-processed data, the size of the cyclic block data being less than or equal to the preset size, the preset size being determined by a number of point multiplication units in an idle state in the point multiplication unit array.

[0008] In a possible implementation, the access information of the to-be-processed data includes access information of a first matrix and a second matrix, the number of columns of the first matrix being the same as the number of rows of the second matrix, and the logical operation type includes a matrix multiplication operation of the first matrix and the second matrix. The cyclic block sequence determined according to the matrix multiplication operation type includes: a first outer loop sequence in a row number direction of the first matrix, a second outer loop sequence in a column number direction of the second matrix, and an inner loop sequence in a column number direction of the first matrix and a row number direction of the second matrix; or a first outer loop sequence in a column number direction of the second matrix, a second outer loop sequence in a row number direction of the first matrix, and an inner loop sequence in a column number direction of the first matrix and a row number direction of the second matrix.

[0009] In a possible implementation, the memory includes a local memory disposed outside the processor and a reuse cache disposed inside the processor for caching cyclic block data, and the reading of the cyclic block data of the to-be-processed data from the memory to the point multiplication unit array in the cyclic block order of the to-be-processed data includes: the control logic unit stores the first cyclic block data of the first matrix read from the local memory into the reuse cache according to the first outer loop order and the inner loop order, so that the control logic unit reuses the first cyclic block data stored in the reuse cache according to the second outer loop order; or the control logic unit stores the second cyclic block data of the second matrix read from the local memory into the reuse cache according to the second outer loop order and the inner loop order, so that the control logic unit reuses the second cyclic block data stored in the reuse cache according to the first outer loop order.

[0010] In a possible implementation, the point multiplication unit array performs product accumulation operation on the cyclic block data of the to-be-processed data received each time to determine the cyclic block result of the to-be-processed data received each time, which includes: the point multiplication unit array performs product accumulation operation on the first cyclic block data and the second cyclic block data obtained each time to obtain the cyclic block result corresponding to each product accumulation operation; the control logic unit determines the logical operation result of the to-be-processed data according to the plurality of cyclic block results obtained from the point multiplication unit array, which includes: the control logic unit performs accumulation operation on the plurality of cyclic block results obtained from the point multiplication unit array in the cyclic block order of any round to obtain a logical operation sub-result; the control logic unit writes the logical operation sub-result into a register stack; and the control logic unit determines the logical operation result according to the plurality of logical operation sub-results obtained from the register stack.

[0011] In a possible implementation, the access information of the to-be-processed data includes access information of to-be-convolved data and a convolution kernel, the logical operation type includes a convolution operation of the to-be-convolved data and the convolution kernel, and the loop blocking sequence determined according to the convolution operation type includes a coordinate sequence of elements in the convolution kernel. The step of sequentially reading, from the memory, loop blocking data of the to-be-processed data to the point multiplication unit array according to the loop blocking sequence of the to-be-processed data includes: the control logic unit reads, from the memory, a plurality of elements with the same coordinates in a plurality of the convolution kernel each time according to the coordinate sequence of the elements in the convolution kernel, to form third loop blocking data, and reads a plurality of elements from the to-be-convolved data in the memory according to the coordinates of a current element in the convolution kernel each time and the access information, to form fourth loop blocking data; and the control logic unit writes the third loop blocking data and the fourth loop blocking data to the point multiplication unit array.

[0012] In a possible implementation, the memory includes a local memory disposed outside the processor and a reuse cache disposed inside the processor and used to cache loop blocking data. The method further includes: in a process of calculating fourth loop blocking data corresponding to a same row in a convolution output result, the control logic unit writes, according to the coordinate sequence of the elements in the convolution kernel, M elements of the fourth loop blocking data corresponding to the coordinates of an element in the rth row and the sth column of the convolution kernel to the reuse cache, where r, s, and M are positive integers; and the control logic unit reads M-1 elements from the reuse cache, reads one element from the to-be-convolved data in the local memory, and determines the fourth loop blocking data corresponding to the coordinates of an element in the rth row and the (s+1)th column of the convolution kernel.

[0013] In a possible implementation, the method further includes: in a process of calculating fourth loop blocking data corresponding to adjacent rows in a convolution output result, in response to the control logic unit calculating the fourth loop blocking data of a current row of the convolution output result, the control logic unit writes, to the reuse cache, M elements of the fourth loop blocking data corresponding to the coordinates of an element in the rth row and the sth column of the convolution kernel, where r, s, and M are positive integers; and in response to the control logic unit calculating the fourth loop blocking data of a next row of the current row of the convolution output result, the control logic unit reads M elements from the reuse cache to form the fourth loop blocking data corresponding to the coordinates of an element in the (r-1)th row and the sth column of the convolution kernel.

[0014] In a possible implementation, the memory includes a local memory disposed outside the processor and a reuse cache disposed inside the processor for caching cyclic block data, and the control logic unit forms, in the order of coordinates of elements in the convolution kernel, a plurality of elements with the same coordinates read from a plurality of the convolution kernel each time from the memory as third cyclic block data, including: the control logic unit reads a plurality of convolution kernels from the local memory to the reuse cache; in the order of coordinates of elements in the convolution kernel, the control logic unit forms, in the order of coordinates of elements in the convolution kernel, a plurality of elements with the same coordinates read from a plurality of the convolution kernel each time from the reuse cache as third cyclic block data.

[0015] In a possible implementation, the point multiplication unit array performs product accumulation operation on the cyclic block data of the to-be-processed data received each time to determine the cyclic block result of the to-be-processed data received each time, including: the point multiplication unit array performs product accumulation operation on the third cyclic block data and the fourth cyclic block data received each time to determine the cyclic block result.

[0016] According to an aspect of the present disclosure, a processor is provided, which includes a control logic unit and a point multiplication unit array including a plurality of point multiplication units for performing product accumulation operation, and the processor is configured to: the control logic unit reads cyclic block data of to-be-processed data from a memory to the point multiplication unit array in sequence according to acquired control instructions; the point multiplication unit array performs product accumulation operation on the cyclic block data of the to-be-processed data received each time to determine the cyclic block result of the to-be-processed data received each time; and the control logic unit determines a logical operation result of the to-be-processed data according to a plurality of the cyclic block results acquired from the point multiplication unit array.

[0017] In a possible implementation, the control logic unit reads cyclic block data of to-be-processed data from a memory to a point multiplication unit array in sequence according to acquired control instructions, including: the control logic unit analyzes access information of the to-be-processed data and a logical operation type of the to-be-processed data according to acquired control instructions; determines a cyclic block order of the to-be-processed data according to the logical operation type of the to-be-processed data; and reads the cyclic block data of the to-be-processed data from the memory to the point multiplication unit array in sequence according to the access information and a preset size and in the cyclic block order of the to-be-processed data, the size of the cyclic block data being less than or equal to the preset size, the preset size being determined by the number of point multiplication units in an idle state in the point multiplication unit array.

[0018] In a possible implementation, the access information of the to-be-processed data includes access information of a first matrix and a second matrix, the number of columns of the first matrix is the same as the number of rows of the second matrix, the type of the logical operation includes a matrix multiplication operation of the first matrix and the second matrix, and the loop blocking sequence determined according to the type of the matrix multiplication operation includes: a first outer loop sequence in a row number direction of the first matrix, a second outer loop sequence in a column number direction of the second matrix, and an inner loop sequence in a column number direction of the first matrix and a row number direction of the second matrix; or a first outer loop sequence in a column number direction of the second matrix, a second outer loop sequence in a row number direction of the first matrix, and an inner loop sequence in a column number direction of the first matrix and a row number direction of the second matrix.

[0019] In a possible implementation, the memory includes a local memory disposed outside the processor and a reuse cache disposed inside the processor and used for caching loop blocking data, and reading the loop blocking data of the to-be-processed data from the memory to the dot multiplication unit array according to the loop blocking sequence of the to-be-processed data includes: storing, by the control logic unit, first loop blocking data of the first matrix read from the local memory into the reuse cache according to the first outer loop sequence and the inner loop sequence, so that the control logic unit reuses the first loop blocking data stored in the reuse cache according to the second outer loop sequence; or storing, by the control logic unit, second loop blocking data of the second matrix read from the local memory into the reuse cache according to the second outer loop sequence and the inner loop sequence, so that the control logic unit reuses the second loop blocking data stored in the reuse cache according to the first outer loop sequence.

[0020] In a possible implementation, the dot multiplication unit array performs a product accumulation operation on the loop blocking data of the to-be-processed data received each time to determine the loop blocking result of the to-be-processed data received each time, including: performing, by the dot multiplication unit array, a product accumulation operation on the first loop blocking data and the second loop blocking data obtained each time to obtain a loop blocking result corresponding to each product accumulation operation; determining, by the control logic unit, the logical operation result of the to-be-processed data according to a plurality of loop blocking results obtained from the dot multiplication unit array, including: performing, by the control logic unit, an accumulation operation on a plurality of loop blocking results obtained from the dot multiplication unit array in any round of the loop blocking sequence to obtain a logical operation sub-result; writing, by the control logic unit, the logical operation sub-result into a register stack; and determining, by the control logic unit, the logical operation result according to a plurality of logical operation sub-results obtained from the register stack.

[0021] In a possible implementation, the access information of the to-be-processed data includes access information of to-be-convolved data and a convolution kernel, the logical operation type includes a convolution operation of the to-be-convolved data and the convolution kernel, and the loop blocking sequence determined according to the convolution operation type includes a coordinate sequence of elements in the convolution kernel. The reading, from the memory, of the loop blocking data of the to-be-processed data to the point multiplication unit array in sequence according to the loop blocking sequence of the to-be-processed data includes: the control logic unit reads, from the memory, a plurality of elements with the same coordinates in a plurality of the convolution kernel each time, to form third loop blocking data according to the coordinate sequence of the elements in the convolution kernel, and reads a plurality of elements from the to-be-convolved data in the memory according to the coordinates of a current element in the convolution kernel each time and the access information, to form fourth loop blocking data; and the control logic unit writes the third loop blocking data and the fourth loop blocking data into the point multiplication unit array.

[0022] In a possible implementation, the memory includes a local memory disposed outside the processor and a reuse cache disposed inside the processor for caching loop blocking data. The processor is further configured to: in a process of determining the fourth loop blocking data corresponding to the same row in the convolution output result, the control logic unit writes, according to the coordinate sequence of the elements in the convolution kernel, M elements of the fourth loop blocking data corresponding to the coordinates of an element in the rth row and the sth column of the convolution kernel into the reuse cache, where r, s, and M are positive integers; and the control logic unit reads M-1 elements from the reuse cache, reads one element from the to-be-convolved data in the local memory, and determines the fourth loop blocking data corresponding to the coordinates of an element in the rth row and the (s+1)th column of the convolution kernel.

[0023] In a possible implementation, the processor is further configured to: in a process of determining the fourth loop blocking data corresponding to adjacent rows in the convolution output result, in response to the control logic unit determining the fourth loop blocking data of a current row of the convolution output result, the control logic unit writes, into the reuse cache, M elements of the fourth loop blocking data corresponding to the coordinates of an element in the rth row and the sth column of the convolution kernel, where r, s, and M are positive integers; and in response to the control logic unit determining the fourth loop blocking data of a next row of the current row of the convolution output result, the control logic unit reads M elements from the reuse cache to form the fourth loop blocking data corresponding to the coordinates of an element in the (r-1)th row and the sth column of the convolution kernel.

[0024] In a possible implementation, the memory includes a local memory disposed outside the processor and a reuse cache disposed inside the processor for caching cyclic block data, and the control logic unit forms, in the order of coordinates of elements in the convolution kernel, a plurality of elements with the same coordinates read from a plurality of the convolution kernel each time from the memory into the third cyclic block data, including: the control logic unit reads a plurality of convolution kernels from the local memory into the reuse cache; in the order of coordinates of elements in the convolution kernel, the control logic unit forms, in the order of coordinates of elements in the convolution kernel, a plurality of elements with the same coordinates read from a plurality of the convolution kernel each time from the reuse cache into the third cyclic block data.

[0025] In a possible implementation, the point multiplication unit array performs product accumulation operation on the cyclic block data of the to-be-processed data received each time to determine the cyclic block result of the to-be-processed data received each time, including: the point multiplication unit array performs product accumulation operation on the third cyclic block data and the fourth cyclic block data received each time to determine the cyclic block result.

[0026] According to an aspect of the present disclosure, there is provided an artificial intelligence chip including the processor as described above.

[0027] According to an aspect of the present disclosure, there is provided an electronic device including the artificial intelligence chip as described above.

[0028] In the embodiments of the present disclosure, the control logic unit reads, according to the obtained control instruction, the cyclic block data of the to-be-processed data from the memory to the point multiplication unit array in sequence, so that the point multiplication unit array performs product accumulation operation on the cyclic block data of the to-be-processed data received each time to determine the cyclic block result of the to-be-processed data received each time, and the control logic unit determines the logical operation result of the to-be-processed data according to a plurality of cyclic block results obtained from the point multiplication unit array. In this way, the reading and logical operation on the to-be-processed data (for example, image data with a large size) can be converted into the reading and logical operation on a plurality of cyclic block data (image data with a small size) of the to-be-processed data, which is conducive to processing data with a larger size under the condition that the hardware resources of the processor remain unchanged, and reducing the pressure on the storage bandwidth.

[0029] It should be understood that the above general description and the following detailed description are only exemplary and explanatory, but not limiting the present disclosure. Other features and aspects of the present disclosure will become apparent from the following detailed description of exemplary embodiments with reference to the accompanying drawings. BRIEF DESCRIPTION OF DRAWINGS

[0030] The accompanying drawings, which are incorporated herein and constitute part of the specification, illustrate embodiments consistent with the present disclosure and, together with the description, further serve to explain the principles of the present disclosure.

[0031] FIG. 1 shows a schematic diagram of a processor according to an embodiment of the present disclosure.

[0032] FIG. 2 shows a flowchart of a data processing method according to an embodiment of the present disclosure.

[0033] FIG. 3 shows a schematic diagram of a loop tiling sequence according to an embodiment of the present disclosure.

[0034] FIG. 4 shows a schematic diagram of a loop tiling data in a repeated reading situation according to an embodiment of the present disclosure.

[0035] FIG. 5 shows a schematic diagram of third loop tiling data of a convolution kernel according to an embodiment of the present disclosure.

[0036] FIG. 6 shows a schematic diagram of fourth loop tiling data of a convolution kernel according to an embodiment of the present disclosure.

[0037] FIG. 7 shows a schematic diagram of a reuse buffer according to an embodiment of the present disclosure.

[0038] FIG. 8 shows a schematic diagram of another reuse buffer according to an embodiment of the present disclosure.

[0039] FIG. 9 shows a block diagram of an electronic device according to an embodiment of the present disclosure. DETAILED DESCRIPTION

[0040] Various exemplary embodiments, features, and aspects of the present disclosure will be described in detail below with reference to the accompanying drawings. The same reference numbers in different drawings represent the same or similar elements. Although various aspects of embodiments are illustrated in the drawings, the drawings are not necessarily drawn to scale unless specifically indicated.

[0041] The term "exemplary" is used herein to mean "serving as an example, instance, or illustration." Any implementation described herein as "exemplary" is not necessarily to be construed as preferred or advantageous over other implementations.

[0042] The term "and / or" used herein only means an association relationship of associated objects, and means that there can be three relationships, for example, A and / or B can mean that A exists alone, A and B exist together, and B exists alone. In addition, the term "at least one" herein means any one of a plurality or any combination of at least two of a plurality, for example, including at least one of A, B, and C can mean including any one or more elements selected from the set consisting of A, B, and C.

[0043] In addition, for a better illustration of the present disclosure, numerous specific details are set forth in the following detailed description. Those skilled in the art will understand that the present disclosure can be practiced without certain specific details. In some instances, well-known methods, schemes, elements, and circuits have not been described in detail in order to avoid obscuring the present disclosure.

[0044] Figure 1 shows a schematic diagram of a processor according to an embodiment of the present disclosure. As shown in Figure 1, the processor includes a control logic unit, an array of point multiplication units including a plurality of point multiplication units for performing product-sum operations.

[0045] The point multiplication unit can also include an operator to complete a specified operation, such as calculating a plurality of multiplications and performing accumulation. Among them, the point multiplication unit can include a multiplier, an adder, etc., and the specific structure in each point multiplication unit can be the same or different, which is not limited by the present disclosure. The point multiplication unit can also include other types of operators to adapt to various different operation processes, and the number and type of operators included in the point multiplication unit are not limited by the present disclosure.

[0046] The control logic unit can be connected to each point multiplication unit in the array of point multiplication units. In some embodiments, the control logic unit can number the point multiplication units in the form of a two-dimensional matrix or a multi-dimensional matrix, so that the plurality of point multiplication units can be logically arranged in the form of a two-dimensional matrix or a multi-dimensional matrix, thereby better adapting to the logical operation of the matrix.

[0047] As shown in Figure 1, the memory corresponding to the processor can include a local memory disposed outside the processor. The control logic unit in the processor can be connected to the local memory, and the control logic unit can be used for address calculation to load data from the memory to the array of point multiplication units and control the array of point multiplication units to process the data to be processed.

[0048] In one possible implementation, the local memory can be an on-chip cache, and the control logic unit can load executable programs and data to be processed (e.g., input matrix) on the off-chip flash memory into the above local memory (on-chip cache), and then perform the subsequent logical operation process of the data to be processed.

[0049] In one possible implementation, the local memory can store data to be processed and executable programs. The executable programs can include control instructions, and the processor executing the control instructions can implement the logical operation of the data to be processed, such as matrix multiplication, convolution operation, and other multiplication and accumulation related operations.

[0050] In order to reduce the reading of repeated data, reduce the power consumption of the processor, and increase the communication bandwidth of the processor, the memory can further include a reuse buffer arranged in the processor. The reuse buffer can be arranged in the dot product unit array (not shown in FIG. 1) or arranged outside the dot product unit array (see FIG. 1). Compared with arranging the reuse buffer outside the dot product unit array, arranging the reuse buffer inside the dot product unit array can have higher data reading efficiency. The control logic unit is provided with a loader and a decoder. The loader can be used to load the data to be processed in the local memory or part of the processor into the reuse buffer in the processor. The decoder can decode the control instruction for accessing data in the executable program according to the change of the storage address of the data to be processed after loading. For example, for the control instruction for accessing data X in the local memory, since the data X is cached in the reuse buffer, the address of the data X stored in the reuse buffer can be obtained by decoding. The decoder can convert the control instruction for accessing the data X in the local memory into the control instruction for accessing the data X in the reuse buffer, which is conducive to the subsequent control logic unit directly sending the data cached in the reuse buffer to the dot product unit for performing the product-sum operation by the dot product unit. The control logic unit can also directly load data from the off-chip memory to the reuse buffer, which is not limited in the present disclosure.

[0051] In a possible implementation, in order to facilitate the processor to process large-scale data, as shown in FIG. 1, a corresponding register file can be arranged for the processor, which is used to read, store, process and transmit data without combining with the memory. In this way, when the processor processes a large amount of data, the instruction execution time can be greatly reduced. The register file includes, for example, a memory address register, a memory data register, an instruction register, an operation code word register, an accumulator, a flag register, etc., which are not limited in the present disclosure. In an example, the control logic unit can be used for address calculation, so as to carry data between the dot product unit array and the register file.

[0052] In a possible implementation, the processor of the embodiments of the present disclosure can be newly designed or improved from an existing processor chip. The processor chip can include, but is not limited to, a central processing unit (CPU), a graphic processing unit (GPU), a general-purpose computing on graphics processing units (GPGPU), a digital signal processor (DSP), an application specific integrated circuit (ASIC), a tensor processing unit (TPU), a field programmable gate array (FPGA) or other programmable logic device, and can further include a microprocessor or other conventional processor.

[0053] FIG. 2 shows a flowchart of a data processing method according to an embodiment of the present disclosure. As shown in FIG. 2, the data processing method is applied to a processor, and the data processing method can include the following steps:

[0054] In step S11, the control logic unit reads the cyclic block data of the to-be-processed data from the memory to the dot multiplication unit array in sequence according to the obtained control instruction, and the control instruction is used to indicate the access information of the to-be-processed data and the logical operation type of the to-be-processed data, for example, including matrix multiplication operation, convolution operation and other multiplication accumulation related operations.

[0055] In a possible implementation, the control logic unit can obtain the address information of the to-be-processed data according to the control instruction, and determine the address information of each cyclic block data of the to-be-processed data according to the address information of the to-be-processed data, and read the cyclic block data of the to-be-processed data from the memory to the dot multiplication unit array in sequence.

[0056] Optionally, the memory can include a local memory arranged outside the processor, and the control logic unit can read the cyclic block data of the to-be-processed data from the local memory to the dot multiplication unit array in sequence according to the obtained control instruction.

[0057] Optionally, since the cyclically blocked data to be processed may exist a scenario of being repeatedly read, the memory can be provided with a reuse cache inside the processor for caching the cyclically blocked data to be reused, in addition to the local memory provided outside the processor. Wherein, the control logic unit reads data from the reuse cache at a speed faster than reading data from the local memory. Thus, for the cyclically blocked data to be repeatedly read in the data to be processed, in response to the control logic unit transmitting the cyclically blocked data to be repeatedly read obtained from the local memory to the dot product unit array, the control logic unit can also synchronously store the cyclically blocked data to be repeatedly read into the reuse cache provided inside the processor, so that the control logic unit can subsequently read the cyclically blocked data from the reuse cache to the dot product unit array.

[0058] In a possible implementation, the data to be processed includes feature data in a deep learning task, and the feature data includes at least one of image feature data, speech feature data, and text feature data.

[0059] For example, in a scenario of performing face recognition on a target object using a deep neural network, the data to be processed can be image feature data, and image feature data (for example, a face feature map) of the target object can be stored in the memory; in a scenario of performing speech recognition on a target object using a deep neural network, the data to be processed can be speech feature data, and speech feature data of the target object can be stored in the memory; in a scenario of performing character recognition on a target document using a deep neural network, the data to be processed can be text feature data, and text feature data of the target document can be stored in the memory; embodiments of the present application do not limit the type of data to be processed.

[0060] In a possible implementation, the control logic unit can obtain data structure information of the data to be processed and address information of the data to be processed through a control instruction; wherein the data structure information of the data to be processed includes, for example, dimensions of the data to be processed, sizes of the data to be processed, data types (for example, integer type, single-precision floating-point number type, double-precision floating-point number type, character type, etc.) of elements in the data to be processed, and other information for describing the data to be processed; and the address information of the data to be processed includes, for example, a base address (Base Address) of the data to be processed in the memory, addressing space, and other address-related information.

[0061] In step S12, the dot product unit array performs a product accumulation operation on the cyclically blocked data of the data to be processed received each time, to determine the cyclically blocked result of the data to be processed received each time.

[0062] Exemplarily, the point multiplication unit array can include tile_M* tile_N point multiplication units, each of which can calculate tile_K multiplications and perform accumulation, where tile_M, tile_N and tile_K are positive integers, and the present disclosure does not limit the specific values of tile_M, tile_N and tile_K, which can be set according to actual application scenarios.

[0063] The point multiplication unit array can implement matrix multiplication or convolution operation by performing product accumulation operation on each received cyclic block data of the to-be-processed data. For matrix multiplication, the point multiplication unit array can complete multiplication operation of a matrix with a size of tile_M* tile_K and a matrix with a size of tile_K* tile_N. For convolution operation, the point multiplication unit array can process tile_M to-be-convolved elements and tile_N convolution kernel elements at a time, where the number of channels of the tile_M to-be-convolved elements and the tile_N convolution kernel elements is tile_K.

[0064] In step S13, the control logic unit determines the logical operation result of the to-be-processed data according to the plurality of cyclic block results obtained from the point multiplication unit array.

[0065] Exemplarily, the control logic unit can perform splicing processing and / or addition processing on the plurality of cyclic block results to determine the logical operation result of the to-be-processed data. The control logic unit can perform splicing processing and / or addition processing on the last received cyclic block data every time a cyclic block data is received, or the control logic unit can perform splicing processing and / or addition processing on all the received cyclic block data, and the present disclosure does not limit this.

[0066] In the data processing method of the embodiment of the present disclosure, the control logic unit reads the cyclic block data of the to-be-processed data from the memory to the point multiplication unit array according to the obtained control instruction for indicating logical operation of the to-be-processed data, so that the point multiplication unit array performs product accumulation operation on each received cyclic block data of the to-be-processed data, determines the cyclic block result of each received cyclic block data of the to-be-processed data, and the control logic unit determines the logical operation result of the to-be-processed data according to the plurality of cyclic block results obtained from the point multiplication unit array. In this way, the reading and logical operation of the to-be-processed data (for example, large-size image data) can be converted into the reading and logical operation of a plurality of cyclic block data (small-size image data) of the to-be-processed data, which is beneficial to processing larger-size data under the condition that the hardware resources of the processor remain unchanged, and reducing the pressure on the storage bandwidth.

[0067] The data processing method of the embodiment of the present disclosure is exemplarily described below.

[0068] In step S11, the control logic unit reads the loop chunk data of the to-be-processed data from the memory to the dot multiplication unit array according to the obtained control instruction.

[0069] The memory can include a local memory disposed outside the processor, or the memory can include the local memory disposed outside the processor and a reuse cache disposed inside the processor for caching the loop chunk data, and embodiments of the present disclosure do not make a specific limitation.

[0070] In a possible implementation, step S11 can include that the control logic unit analyzes access information of the to-be-processed data and a logical operation type of the to-be-processed data according to the obtained control instruction, determines a loop chunk order of the to-be-processed data according to the logical operation type of the to-be-processed data, and reads the loop chunk data of the to-be-processed data from the memory to the dot multiplication unit array according to the access information and the preset size in the loop chunk order of the to-be-processed data, the size of the loop chunk data being less than or equal to the preset size, the preset size being determined by the number of the dot multiplication units in the idle state in the dot multiplication unit array.

[0071] Exemplarily, assuming that the control instruction is used to indicate a matrix multiplication operation of the to-be-processed data, the control logic unit analyzes the obtained control instruction, and can analyze the access information of the to-be-processed data and the logical operation type of the to-be-processed data. The access information can include, for example, size information, address information, layout information of the to-be-processed data. The layout information can be RowMajor or ColMajor, RowMajor indicating that elements of the same row of the to-be-processed data are adjacent in the memory, and ColMajor indicating that elements of the same column of the to-be-processed data are adjacent in the memory. For example, if the logical operation type of the to-be-processed data is a matrix multiplication operation of a first matrix and a second matrix, the access information can include size information, address information (for example, a starting address), layout information and the like of the first matrix and the second matrix.

[0072] The control logic unit can determine the loop chunk order of the to-be-processed data according to the logical operation type of the to-be-processed data. For example, if the logical operation type of the to-be-processed data is a matrix multiplication operation of a first matrix and a second matrix, the control logic unit can select a pre-stored loop chunk order matched with the matrix multiplication operation.

[0073] The control logic unit can determine the size of the to-be-processed data according to the parsed access information of the to-be-processed data, and can determine whether the to-be-processed data needs to be subjected to the loop blocking processing according to the size of the to-be-processed data. If the size of the to-be-processed data is less than or equal to a preset size, it indicates that the number of the idle point multiplication units in the point multiplication unit array can meet the demand of processing the to-be-processed data, and the control logic unit can directly read all the to-be-processed data from the memory to the point multiplication unit array, so that the point multiplication unit array performs the multiplication and addition operation on the to-be-processed data.

[0074] If the size of the to-be-processed data is greater than the preset size, it indicates that the number of the idle point multiplication units in the point multiplication unit array cannot meet the demand of processing the to-be-processed data, and the control logic unit can determine the loop blocking sequence of the to-be-processed data according to the logical operation type of the to-be-processed data, and determine the loop blocking data of the to-be-processed data according to the access information of the to-be-processed data and the preset size, and then read the loop blocking data of the to-be-processed data from the memory to the point multiplication unit array in sequence according to the loop blocking sequence of the to-be-processed data, and perform the multiplication and addition operation on the plurality of loop blocking data by the point multiplication unit array, so as to realize the multiplication and addition operation on the to-be-processed data.

[0075] For example, there are tile_Mx tile_N idle point multiplication units in the point multiplication unit array, and each point multiplication unit can calculate tile_K multiplications and perform accumulation. For matrix multiplication, the point multiplication unit array can complete the multiplication operation of a matrix with a size of tile_Mx tile_K and a matrix with a size of tile_Kx tile_N. For matrices with sizes greater than tile_M, tile_N and tile_K, the control logic unit is responsible for loop control, and reads the loop blocking data conforming to the size from the to-be-processed data stored in the memory to the point multiplication unit array in sequence according to the loop blocking sequence.

[0076] For example, it is assumed that the control instruction is used to instruct the convolution operation of the to-be-processed data, and the control logic unit parses the obtained control instruction to parse the access information of the to-be-processed data, which for example includes the size information, address information, convolution description information (for example, including the stride of the convolution kernel, the padding number) of the to-be-convoluted data and the convolution kernel, and the dimension information of the convolution output result. For example, if the logical operation type of the to-be-processed data is the convolution operation of the to-be-convoluted data and the convolution kernel, the access information can include the convolution description information (for example, including the stride of the convolution kernel, the padding number), the dimension information of the convolution output result, the size information, address information, layout information, etc. of the to-be-convoluted data and the convolution kernel.

[0077] The control logic unit can determine the loop blocking sequence of the to-be-processed data according to the logical operation type of the to-be-processed data. For example, if the logical operation type of the to-be-processed data is a convolution operation of the to-be-convolved data and the convolution kernel, the control logic unit can select the loop blocking sequence stored in advance that matches the convolution operation.

[0078] The control logic unit can determine the size of the to-be-convolved data and the convolution kernel according to the parsed access information of the to-be-processed data, and can determine whether loop blocking processing needs to be performed on the to-be-convolved data and the convolution kernel according to the size of the to-be-convolved data and the convolution kernel. If the size of the to-be-convolved data and the convolution kernel is less than or equal to the preset size, it indicates that the number of idle point multiplication units in the point multiplication unit array can meet the demand of the convolution operation of the to-be-convolved data and the convolution kernel, and the control logic unit can directly read the to-be-convolved data and the convolution kernel from the memory to the point multiplication unit array to enable the point multiplication unit array to perform multiplication and addition operation thereon.

[0079] If the size of any data in the to-be-convolved data or the convolution kernel is greater than the preset size, it indicates that the number of idle point multiplication units in the point multiplication unit array cannot meet the demand of the convolution operation of the to-be-convolved data and the convolution kernel. The control logic unit can determine the loop blocking sequence of the to-be-processed data according to the logical operation type of the to-be-processed data, and determine the loop blocking data from the to-be-convolved data and / or the convolution kernel that is greater than the preset size according to the access information of the to-be-processed data and the preset size, and then read the corresponding loop blocking data from the memory to the point multiplication unit array in sequence according to the loop blocking sequence of the to-be-processed data, so as to realize the convolution operation of the to-be-convolved data and the convolution kernel through the multiplication and addition operation of the point multiplication unit array on the plurality of loop blocking data.

[0080] The size of the loop blocking data is less than or equal to the preset size, and the specific size of the loop blocking data is not limited in the present disclosure.

[0081] For example, there are tile_Mx tile_N idle point multiplication units in the point multiplication unit array, and each point multiplication unit calculates tile_K multiplications and performs accumulation. For the convolution operation, the point multiplication unit array can process tile_M to-be-convolved elements and tile_N convolution kernel elements at a time, and the number of channels of the tile_M to-be-convolved elements and the tile_N convolution kernel elements is tile_K. For to-be-processed data with a size greater than tile_M, tile_N and tile_K, the control logic unit is responsible for loop control, and reads the loop blocking data with a size in sequence from the to-be-processed data stored in the memory to the point multiplication unit array according to the loop blocking sequence.

[0082] In this way, it is beneficial to convert the large size to-be-processed data into cyclic block data meeting the size requirement of the dot multiplication unit array calculation structure, and the calculation performance of the processor is improved.

[0083] In a possible implementation, the access information of the to-be-processed data includes access information of a first matrix and a second matrix, the number of columns of the first matrix is the same as the number of rows of the second matrix, the logical operation includes a matrix multiplication operation of the first matrix and the second matrix, and the cyclic block sequence determined according to the matrix multiplication operation type includes a first outer loop sequence in a row direction sequence of the first matrix, a second outer loop sequence in a column direction of the second matrix, and an inner loop sequence in a column direction of the first matrix and a row direction of the second matrix.

[0084] Since the hardware resource of the processor is limited, when the amount of data accessed by the processor is too large, the control logic unit cannot load the required data into the dot multiplication unit array at one time. By setting the cyclic block sequence, it is beneficial to process larger size data under the condition that the hardware resource of the processor remains unchanged, and to reduce the pressure on the storage bandwidth.

[0085] Exemplarily, FIG. 3 shows a schematic diagram of a cyclic block sequence according to an embodiment of the present disclosure. As shown in FIG. 3, the control instruction received by the control logic unit can be to perform a matrix multiplication operation on a first matrix A with a size of MxK and a second matrix B with a size of KxN, and the number of columns K of the first matrix A is the same as the number of rows K of the second matrix B.

[0086] Wherein, C represents the matrix multiplication result of the first matrix A and the second matrix B, tile_A represents the first cyclic block data of the first matrix A, with a size of tile_Mx tile_K, tile_B represents the second cyclic block data of the second matrix B, with a size of tile_Kx tile_N, and tile_C represents the matrix multiplication result of the first cyclic block data tile_A and the second cyclic block data tile_B.

[0087] The cyclic block sequence can be a multi-layer loop (for example, including three-layer loop nesting), a first outer loop sequence in a row direction sequence of the first matrix A (for example, the loop in the M direction in FIG. 3), a second outer loop sequence in a column direction of the second matrix B (for example, the loop in the N direction in FIG. 3), and an inner loop sequence in a column direction of the first matrix A and a row direction of the second matrix B (for example, the loop in the K direction in FIG. 3).

[0088] Thus, in step S11, the control logic unit can read the first cyclic tile data tile_A of the first matrix A, the second cyclic tile data tile_B of the second matrix B in turn from the memory to the point multiplication unit array in a cyclic tile order, so that in step S12, the point multiplication unit array performs product accumulation operation on each acquired first cyclic tile data tile_A and second cyclic tile data tile_B, to obtain the cyclic tile result tile_C corresponding to each product accumulation operation. And in step S13, the control logic unit performs accumulation operation on the plurality of cyclic tile results tile_C acquired from the point multiplication unit array in any round of the cyclic tile order in turn, to obtain the logical operation result; the pseudo code is as follows:

[0089] Wherein, the first outer loop (such as the loop in M direction in FIG. 3) can be cyclic M / tile_M times, the second outer loop (such as the loop in N direction in FIG. 3) can be cyclic N / tile_N times, and the inner loop (such as the loop in K direction in FIG. 3) can be cyclic K / tile_K times, so that the first matrix A can be divided into M / tile_M rows and K / tile_K columns, and the second matrix B can be divided into K / tile_K rows and N / tile_N columns, each row and column in the first matrix A corresponds to a first cyclic tile data tile_A, tile_A mk represents the first cyclic tile data tile_A in the mth row and kth column of the first matrix A, and each row and column in the second matrix B corresponds to a second cyclic tile data tile_B, tile_B kn represents the second cyclic tile data tile_B in the kth row and nth column of the second matrix B.

[0090] tile_A mk ×tile_B kn represents the first cyclic tile data tile_A in the mth row and kth column of the first matrix A mk , and the second cyclic tile data tile_B in the kth row and nth column of the second matrix B kn , the point multiplication unit array performs product accumulation operation on the first cyclic tile data tile_A mk and the second cyclic tile data tile_B kn , to obtain the cyclic tile result tile_C corresponding to the mth, nth and kth product accumulation operation, tile_C mk = tile_A kn ×tile_B .

[0091] The loop tiling sequence can include m*n rounds of loops, the mth round of any first outer loop (such as the loop in the M direction in FIG. 3) and the nth round of a second outer loop (such as the loop in the N direction in FIG. 3) correspond to one round of loops (referred to as the mth and nth rounds for short) of all K / tile_K inner loops (such as the loop in the K direction in FIG. 3), and the control logic unit can sequentially obtain, from the point multiplication unit array, a plurality of loop tiling results tile_A in the loop tiling sequence of any round mk ×tile_B kn perform an accumulation operation to obtain the logical operator result of the mth and nth rounds

[0092] Alternatively, the loop tiling sequence determined according to the matrix multiplication operation type can also include: a first outer loop sequence in the column direction of the second matrix, a second outer loop sequence in the row direction of the first matrix, and an inner loop sequence in the column direction of the first matrix and the row direction of the second matrix. For example, the loop tiling sequence can include a first outer loop sequence in the column direction of the second matrix B (such as the loop in the N direction in FIG. 3), a second outer loop sequence in the row direction of the first matrix A (such as the loop in the M direction in FIG. 3), and an inner loop sequence in the column direction of the first matrix A and the row direction of the second matrix B (such as the loop in the K direction in FIG. 3).

[0093] In this way, in step S11, the control logic unit can sequentially read, from the memory, the first loop tiling data tile_A of the first matrix A and the second loop tiling data tile_B of the second matrix B to the point multiplication unit array according to the loop tiling sequence, so that in step S12, the point multiplication unit array performs a product accumulation operation on the first loop tiling data tile_A and the second loop tiling data tile_B obtained each time to obtain a loop tiling result tile_C corresponding to each product accumulation operation. In step S13, the control logic unit performs an accumulation operation on a plurality of loop tiling results tile_C sequentially obtained from the point multiplication unit array in the loop tiling sequence of any round to obtain a logical operator result; the pseudo code is as follows:

[0094] The specific process can be referred to in the foregoing, which will not be described here.

[0095] In this way, the user can implement the loop tiling sequence through a few simple loop codes, reducing the complexity of the software user implementation.

[0096] In a possible implementation, the last time's loop blocking result can be stored in an accumulator cache (for example, which can be part of a reuse cache), and when the current time's loop blocking data is obtained from the point multiplication unit array, the last time's loop blocking result can be read from the accumulator cache, added to the current time's loop blocking data, and the addition result can be used to update the current time's loop blocking data as the next time's loop blocking data. By setting the accumulator cache for storing the intermediate data (for example, the loop blocking data) of the memory loop (for example, a certain time's loop in the K direction), the number of memory access times can be reduced, and the pressure on the storage bandwidth can be reduced. In this way, the accumulator cache is used to accumulate a plurality of loop blocking results to obtain a logical operator result. Then, the inner loop ends (for example, the K direction loop ends), and the control logic unit writes the logical operator result into a register stack; and the control logic unit determines the logical operation result according to a plurality of logical operator results obtained from the register stack. For example, the control logic unit can write the logical operator result C mn obtained after each round of the inner loop with a total of K / tile_K times into the register stack; and the control logic unit determines the logical operation result according to a total of M / tile_M×N / tile_N rounds of logical operator results C mn obtained from the register stack, and splices the logical operation results to obtain the logical operation result of the first matrix A and the first matrix B. By setting the register stack, the number of access times to the local memory arranged outside the processor can be further reduced.

[0097] Optionally, if the memory is a local memory arranged outside the processor, during the process in which the control logic unit reads the loop blocking data of the to-be-processed data from the local memory to the point multiplication unit array in sequence according to the loop blocking order, the loop blocking data can be repeatedly read. FIG. 4 shows a schematic diagram of the repeated reading of the loop blocking data according to an embodiment of the present disclosure. As shown in FIG. 4, it is assumed that the first matrix A can be divided into 3×2 blocks, that is, the first loop blocking data tile_A 11 , the first loop blocking data tile_A 12 , the first loop blocking data tile_A 21 , the first loop blocking data tile_A 22 , the first loop blocking data tile_A 31 , the first loop blocking data tile_A 32 .

[0098] It is assumed that the second matrix B can be divided into 2×2 blocks, that is, the second loop blocking data tile_B 11 , the second loop blocking data tile_B 12 , the second loop blocking data tile_B 21, the second loop tile_B 22 .

[0099] According to the loop tile order, the result of each inner loop is accumulated after the first outer loop (for example, the first matrix A row direction loop in Figure 4) to the first time, the second outer loop (for example, the second matrix B column direction loop in Figure 4) to the first time, and a round of inner loop order (for example, the first matrix A column direction and the second matrix B row direction loop in Figure 4, a round of inner loop can include 2 loops). After 2 inner loops, the logical operator result C 11 = tile_A 11 × tile_B 11 + tile_A 12 × tile_B 21 .

[0100] In the first outer loop to the first time, the second outer loop to the second time, the result of each inner loop is accumulated after a round of inner loop order. After 2 inner loops, the logical operator result C 12 = tile_A 11 × tile_B 12 + tile_A 12 × tile_B 22 . Wherein, the first loop tile_A 11 and the first loop tile_A 12 will be reused, and the control logic unit will repeatedly read the first loop tile_A 11 and the first loop tile_A 12 from the local memory.

[0101] In the first outer loop to the second time, the second outer loop to the first time, the result of each inner loop is accumulated after a round of inner loop order. After 2 inner loops, the logical operator result C 21 = tile_A 21 × tile_B 11 + tile_A 22 × tile_B 21 . Wherein, the second loop tile_B 11 and the second loop tile_B 21 will be reused, and the control logic unit will repeatedly read the second loop tile_B 11 and the second loop tile_B 21 from the local memory.

[0102] In the first outer loop to the second time, the second outer loop to the second time, through a round of inner loop sequence, the results of each inner loop are accumulated, and after two inner loops, the logic operator result C can be obtained 22 = tile_A 21 * tile_B 12 + tile_A 22 * tile_B 22 . Wherein, the first loop block data tile_A 21 and the first loop block data tile_A 22 will be reused, the second loop block data tile_B 12 and the second loop block data tile_B 22 will be reused, and the control logic unit will repeatedly read the first loop block data tile_A21, the first loop block data tile_A22, the second loop block data tile_B12 and the second loop block data tile_B22 from the local memory.

[0103] In the first outer loop to the third time, the second outer loop to the first time, through a round of inner loop sequence, the results of each inner loop are accumulated, and after two inner loops, the logic operator result C can be obtained 31 = tile_A 31 * tile_B 11 + tile_A 32 * tile_B 21 . Wherein, the second loop block data tile_B 11 and the second loop block data tile_B 21 will be reused, and the control logic unit will repeatedly read the second loop block data tile_B 11 and the second loop block data tile_B 21 from the local memory.

[0104] In the first outer loop to the third time, the second outer loop to the second time, through a round of inner loop sequence, the results of each inner loop are accumulated, and after two inner loops, the logic operator result C can be obtained 32 = tile_A 31 * tile_B 12 + tile_A 32 * tile_B 22 . Wherein, the first loop block data tile_A 31 and the first loop block data tile_A 32 will be reused, the second loop block data tile_B 12 and the second loop block data tile_B22 The control logic unit will repeatedly read the first loop tile data tile_A from the local memory 31 , the first loop tile data tile_A 32 , the second loop tile data tile_B 12 , the second loop tile data tile_B 22 .

[0105] Optionally, in order to further save the memory bandwidth of the processor, reduce the repeated access to the same loop tile data in the local memory outside the processor, and improve the computing performance of the processor, in a possible implementation manner, the memory includes a local memory arranged outside the processor and a reuse cache arranged inside the processor for caching the loop tile data, and the reading of the loop tile data of the to-be-processed data from the memory to the dot multiplication unit array in the order of the loop tile order of the to-be-processed data includes: the control logic unit stores the first loop tile data of the first matrix read from the local memory into the reuse cache according to the first outer loop order and the inner loop order, so that the control logic unit reuses the first loop tile data stored in the reuse cache according to the second outer loop order; or the control logic unit stores the second loop tile data of the second matrix read from the local memory into the reuse cache according to the second outer loop order and the inner loop order, so that the control logic unit reuses the second loop tile data stored in the reuse cache according to the first outer loop order.

[0106] For example, as shown in FIG. 4, according to the loop tile order, in the first outer loop (for example, the loop in the row direction of the first matrix A in FIG. 4) to the first time, the second outer loop (for example, the loop in the column direction of the second matrix B in FIG. 4) to the first time, and the inner loop (for example, the loop in the column direction of the first matrix A and the row direction of the second matrix B in FIG. 4, one round of the inner loop can include two loops) to the first time (for example, M = 1, N = 1, and K = 1), the control logic unit can read the first loop tile data tile_A 11 of the first matrix A and the second loop tile data tile_B 11 of the second matrix B from the local memory, to obtain the loop tile result tile_A 11 × tile_B 11 The first loop tile data tile_A 11 of the first matrix A and the second loop tile data tile_B 11 of the second matrix B can be stored in the reuse cache.

[0107] Thus, in the first outer loop to the first time, the second outer loop to the second time, and the inner loop to the first time (e.g., M=1, N=2, K=1), the first loop-tiled data tile_A can be read from the reuse cache 11 , the second loop-tiled data tile_B is read from the local memory 12 , the loop-tiled result tile_A 11 ×tile_B 12 is obtained.

[0108] In the first outer loop to the second time, the second outer loop to the first time, and the inner loop to the first time (e.g., M=2, N=1, K=1), the first loop-tiled data tile_A can be read from the local memory 21 , the second loop-tiled data tile_B is read from the reuse cache 11 , the loop-tiled result tile_A 21 ×tile_B 11 is obtained.

[0109] In the first outer loop to the third time, the second outer loop to the first time, and the inner loop to the first time (e.g., M=3, N=1, K=1), the first loop-tiled data tile_A can be read from the local memory 31 , the second loop-tiled data tile_B is read from the reuse cache 11 , the loop-tiled result tile_A 31 ×tile_B 11 is obtained.

[0110] Similarly, in the first outer loop to the first time, the second outer loop to the first time, and the inner loop to the second time (e.g., M=1, N=1, K=2), the control logic unit can read the first loop-tiled data tile_A of the first matrix A 12 , the second loop-tiled data tile_B of the second matrix B 21 from the local memory, obtain the loop-tiled result tile_A 12 ×tile_B 21 , and store the first loop-tiled data tile_A of the first matrix A 12 and the second loop-tiled data tile_B of the second matrix B 21 in the reuse cache.

[0111] Thus, in the first outer loop to the first time, the second outer loop to the second time, and the inner loop to the second time (e.g., M=1, N=2, K=2), the first loop-tiled data tile_A can be read from the reuse cache 12 , the second loop-tiled data tile_B is read from the local memory22 tile_A 12 × tile_B 22 .

[0112] In the first outer loop to the second time, the second outer loop to the second time, and the inner loop to the second time (for example, M=2, N=2, and K=2), the first loop-tiled data tile_A 22 can be read from the local memory, the second loop-tiled data tile_B 21 can be read from the reuse cache, and the loop-tiled result tile_A 22 × tile_B 21 .

[0113] In the first outer loop to the third time, the second outer loop to the second time, and the inner loop to the second time (for example, M=3, N=2, and K=2), the first loop-tiled data tile_A 32 can be read from the local memory, the second loop-tiled data tile_B 21 can be read from the reuse cache, and the loop-tiled result tile_A 32 × tile_B 21 .

[0114] It should be understood that the first outer loop, the second outer loop, and the inner loop are respectively looped to other times, which can be referred to the above and will not be described here. How to reuse the first loop-tiled data and the second loop-tiled data based on the reuse cache in the multi-loop process can be set according to actual application scenarios, and the embodiments of the present disclosure do not limit this.

[0115] For example, in a hardware resource-constrained scenario, in order to reduce the consumption of hardware resources, the reuse cache is not set to be very large, for example, the size of the reuse cache can be set to cache one first loop-tiled data and one second loop-tiled data, and only one first loop-tiled data and one second loop-tiled data can be cached at the same time. In each time of switching the order of the first outer loop or the second outer loop, the loop-tiled data can be read from the reuse cache again. For example, the control logic unit can read the first loop-tiled data of the first matrix from the local memory and store it in the reuse cache in response to each update of the first outer loop order, so that the control logic unit reuses the first loop-tiled data stored in the reuse cache in the process of traversing the second outer loop order. Alternatively, the control logic unit can read the second loop-tiled data of the second matrix from the local memory and store it in the reuse cache in response to each update of the second outer loop order, so that the control logic unit reuses the second loop-tiled data stored in the reuse cache in the process of traversing the first outer loop order.

[0116] By setting the reuse cache inside the processor, the number of accesses to the local memory outside the processor can be reduced, and the pressure on the storage bandwidth can be reduced.

[0117] In a possible implementation, the access information of the to-be-processed data includes access information of to-be-convolved data and a convolution kernel, the logical operation includes a convolution operation of the to-be-convolved data and the convolution kernel, and the loop blocking sequence determined according to the type of the convolution operation includes a coordinate sequence of elements in the convolution kernel. The reading, according to the loop blocking sequence of the to-be-processed data, of loop blocking data of the to-be-processed data from the memory to the point multiplication unit array includes: according to the coordinate sequence of the elements in the convolution kernel, the control logic unit constructs third loop blocking data from a plurality of elements with the same coordinates read from a plurality of the convolution kernel in the memory each time; and according to the coordinates of a current element in the convolution kernel each time and the access information, the control logic unit reads a plurality of elements from the to-be-convolved data in the memory to construct fourth loop blocking data; and the control logic unit writes the third loop blocking data and the fourth loop blocking data to the point multiplication unit array.

[0118] The memory can include a local memory arranged outside the processor, or the memory can include a local memory arranged outside the processor and a reuse cache arranged inside the processor for caching loop blocking data, and embodiments of the present disclosure do not make a specific limitation in this regard.

[0119] Exemplarily, the access information of the to-be-convolved data and the convolution kernel includes convolution description information, which can be used to represent dimension information of the to-be-convolved data, dimension information of the convolution kernel, dimension information of a convolution output result, layout information, and the like. The control logic unit can read the convolution description information and perform a mapping operation on the to-be-convolved data and the convolution kernel according to the convolution description information. Each time the mapping operation is performed, the third loop blocking data and the fourth loop blocking data can be read, and then the third loop blocking data and the fourth loop blocking data are associated with the point multiplication unit array and are calculated. In this way, each element of the convolution kernel is traversed, and the final convolution output result can be obtained by accumulating the results.

[0120] In this way, the control logic unit can expand the matrix operation instruction to implement the convolution function according to the read convolution description information, and convert the convolution operation into matrix multiplication, so that the processor has stronger versatility.

[0121] Figure 5 shows a schematic diagram of third loop tile data of a convolution kernel according to an embodiment of the present disclosure. As shown in Figure 5, a convolution kernel C x R x S x K is stored in a memory, where C represents a channel dimension of the convolution kernel, R represents a height dimension of the convolution kernel, S represents a width dimension of the convolution kernel, and K represents a number dimension of the convolution kernel. Each of the convolution kernels can include R x S x C / tile_K convolution kernel elements, each of which occupies one unit of space in the height dimension R and the width dimension S, and tile_K units of space in the channel dimension C.

[0122] The control logic unit can read the elements of each of the convolution kernels in the first row and the first column in the channel dimension 0-tile_K from the memory in a coordinate order of the elements in the convolution kernel, for example, from the coordinates of the convolution kernel element (r=0, s=0, c=0) to (r=R-1, s=S-1, c=C / tile_K-1), and map the eight (K=0-7) elements with the same coordinates (r=0, s=0, c=0) in the eight (K=0-7) convolution kernels to the first third loop tile data tile_B(0)(0)(0). 000 The control logic unit can read the elements of each of the convolution kernels in the second row and the second column in the channel dimension 0-tile_K from the memory in a coordinate order of the elements in the convolution kernel, for example, from the coordinates of the convolution kernel element (r=0, s=1, c=0) to (r=R-1, s=S-1, c=C / tile_K-1), and map the eight (K=0-7) elements with the same coordinates (r=0, s=1, c=0) in the eight (K=0-7) convolution kernels to the second third loop tile data tile_B(0)(1)(0). 010 The control logic unit can read the elements of each of the convolution kernels in the last row and the last column in the channel dimension (C-tile_K)-C from the memory in a coordinate order of the elements in the convolution kernel, for example, from the coordinates of the convolution kernel element (r=R-1, s=S-1, c=C / tile_K-1) to (r=0, s=0, c=0), and map the eight (K=0-7) elements with the same coordinates (r=R-1, s=S-1, c=C / tile_K-1) in the eight (K=0-7) convolution kernels read from the memory to the last third loop tile data tile_B(R-1)(S-1)(C / tile_K-1).

[0123] As shown in Figure 5, each of the third loop tile data tile_B rsc has a size of tile_K x tile_N, where tile_N represents the number of the convolution kernels, and tile_K represents the size of each of the convolution kernel elements in the channel dimension.

[0124] Meanwhile, FIG. 6 shows a schematic diagram of fourth loop-tiled data of a convolution kernel according to an embodiment of the present disclosure. As shown in FIG. 6, the memory stores the data to be convolved CxHxW (for example, including an input image with a size of CxHxW), C represents the channel dimension of the data to be convolved, H represents the height dimension of the data to be convolved, and W represents the width dimension of the convolution kernel. The data to be convolved can include CxHxW / tile_K elements to be convolved, and each element to be convolved occupies one unit of space in the height dimension H and the width dimension W, and can occupy tile_K units of space in the channel dimension C.

[0125] Since the dimension of the convolution output result is determined by the data to be convolved and the number of times the convolution kernel slides in the data to be convolved, based on this, the control logic unit can calculate the coordinates of the data to be convolved according to the coordinates of the current element in the convolution kernel (r, s, c) each time, and the dimension information of the convolution output result included in the access information, and read tile_M elements from the data to be convolved in the memory according to the coordinates of the data to be convolved, and map them as the fourth loop-tiled data. The number of tile_M elements can be determined by the dimension information of the convolution output result.

[0126] For example, the coordinates of the current element in the convolution kernel are (r=0, s=0, c=0), the first row and the first column element in the convolution output result correspond to the first row and the first column element in the data to be convolved, and the coordinates of the element are (h=0, w=0, c=0). The element with coordinates (h=0, w=0, c=0) in the data to be convolved can be taken as the first element of the tile_M elements of the fourth loop-tiled data. Similarly, until tile_M elements are read from the data to be convolved in the memory and mapped as the fourth loop-tiled data.

[0127] The control logic unit can write the third loop-tiled data and the fourth loop-tiled data corresponding to the coordinates of each element in the convolution kernel into the point multiplication unit array, so that the point multiplication unit array performs matrix multiplication on the third loop-tiled data and the fourth loop-tiled data. For details, refer to the matrix multiplication operation described above, which will not be described here.

[0128] According to an embodiment of the present disclosure, the third loop-tiled data of the convolution kernel and the fourth loop-tiled data of the data to be convolved can be determined in the order of the coordinates of the elements in the convolution kernel, without im2col (for example, sliding the convolution kernel on the data to be convolved, converting the data contained in each convolution kernel window into a column vector, and finally arranging the column into a new matrix) in the memory. This reduces the pressure on the storage bandwidth. Furthermore, in this way, it is beneficial to reuse the matrix calculation structure to implement the convolution engine, so as to extend the matrix operation instruction by using the convolution description information to realize the convolution function.

[0129] In a possible implementation, the memory includes a local memory disposed outside the processor and a reuse cache disposed inside the processor for caching cyclic block data, and the method further includes: in a process of calculating fourth cyclic block data corresponding to a same row in a convolution output result, the control logic unit writes M elements corresponding to a coordinate of an element in an rth row and an sth column in the convolution kernel into the reuse cache in a coordinate order of the elements in the convolution kernel, where M is a positive integer; the control logic unit reads M-1 elements from the reuse cache, reads 1 element from the to-be-convolved data in the local memory, and determines fourth cyclic block data corresponding to a coordinate of an element in an rth row and an s+1th column in the convolution kernel.

[0130] FIG. 7 shows a schematic diagram of a reuse cache according to an embodiment of the present disclosure. As shown in FIG. 7, assuming that the step of the convolution kernel is 1, the control logic unit can write the fourth cyclic block data [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15] corresponding to the coordinate of the element in the r=1th row and the s=0th column in the convolution kernel into the reuse cache.

[0131] For the fourth cyclic block data corresponding to the coordinate of the element in the r=1th row and the s=1th column in the convolution kernel, the control logic unit only needs to read one more element from the to-be-convolved data in the local memory, and the other 15 elements can reuse [1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15] in the reuse cache to form the fourth cyclic block data [1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16]; wherein the control logic unit writes the read element 16 into the reuse cache, so that the reuse cache continues to store [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16].

[0132] For the fourth cyclic block data corresponding to the coordinate of the element in the r=1th row and the s=2th column in the convolution kernel, the control logic unit only needs to read one more element from the to-be-convolved data in the local memory, and the other 15 elements can reuse [2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16] in the reuse cache to form the fourth cyclic block data [2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17]; wherein the control logic unit writes the read element 17 into the reuse cache to obtain [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17].

[0133] The reuse cache is arranged in the processor, multiplexing of input data is realized in the convolution, and the number of accesses to the external local memory of the processor and the bandwidth pressure on the external local memory of the processor are further reduced.

[0134] In a possible implementation, in the process of calculating the fourth loop block data corresponding to adjacent rows in the convolution output result, in response to the control logic unit calculating the fourth loop block data of the current row of the convolution output result, M elements of the fourth loop block data corresponding to the coordinates of the element in the rth row and the sth column of the convolution kernel are written into the reuse cache; in response to the control logic unit calculating the fourth loop block data of the next row of the current row of the convolution output result, M elements are read from the reuse cache to form the fourth loop block data corresponding to the coordinates of the element in the (r-1)th row and the sth column of the convolution kernel.

[0135] FIG. 8 shows a schematic diagram of another reuse cache according to an embodiment of the present disclosure. As shown in FIG. 8, it is assumed that the step of the convolution kernel is 1, and the data [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17] is stored in the reuse cache, where [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15] is the fourth loop block data determined by the coordinates of the element in the r=1th row and the s=0th column of the convolution kernel according to the second row (see FIG. 8, ih_1) of the data to be convolved when the first row (see FIG. 8, oh_0) of the convolution output result is calculated; [1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16] is the fourth loop block data determined by the coordinates of the element in the r=1th row and the s=1th column of the convolution kernel according to the second row (see FIG. 8, ih_1) of the data to be convolved when the first row (see FIG. 8, oh_0) of the convolution output result is calculated, and [2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17] is the fourth loop block data determined by the coordinates of the element in the r=1th row and the s=2th column of the convolution kernel according to the second row (see FIG. 8, ih_1) of the data to be convolved when the first row (see FIG. 8, oh_0) of the convolution output result is calculated.

[0136] It should be understood that the fourth loop block data [1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16] determined by the coordinates of the element in the first row r = 1 and the first column s = 1 in the convolution kernel can reuse the fourth loop block data [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15] determined by the coordinates of the element in the first row r = 1 and the zeroth column s = 0 in the convolution kernel. Similarly, the fourth loop block data [2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17] determined by the coordinates of the element in the first row r = 1 and the second column s = 2 in the convolution kernel can reuse the fourth loop block data [1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16] determined by the coordinates of the element in the first row r = 1 and the first column s = 1 in the convolution kernel. For details, refer to the description related to FIG. 7, which will not be described here again.

[0137] Considering that two adjacent rows in the convolution output result have data reuse, for example, when calculating the second row of the convolution output result (see the second row oh_1 of the convolution result in FIG. 8), the to-be-convolved element corresponding to the element in the zeroth row r = 0 in the convolution kernel (see the second row ih_1 of the to-be-convolved data in FIG. 8) is consistent with the to-be-convolved element corresponding to the element r = 1 in the convolution kernel when calculating the previous row (see the first row oh_0 of the convolution result in FIG. 8) of the convolution output result (also the second row ih_1 of the to-be-convolved data in FIG. 8), which can be reused to reduce the amount of data read from the local memory.

[0138] For example, when calculating the second row of the convolution output result (see the second row oh_1 of the convolution result in FIG. 8), the to-be-convolved element corresponding to the element in the zeroth row r = 0 and the zeroth column s = 0 in the convolution kernel (see the second row ih_1 of the to-be-convolved data in FIG. 8) is consistent with the to-be-convolved element corresponding to the element in the first row r = 1 and the zeroth column s = 0 in the convolution kernel when calculating the previous row (see the first row oh_0 of the convolution result in FIG. 8) of the convolution output result (also the second row ih_1 of the to-be-convolved data in FIG. 8). The fourth loop block data [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15] determined by the element in the first row r = 1 and the zeroth column s = 0 in the convolution kernel when calculating the first row (see the first row oh_0 of the convolution result in FIG. 8) of the convolution output result can be reused as the fourth loop block data [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15] determined by the element in the zeroth row r = 0 and the zeroth column s = 0 in the convolution kernel when calculating the second row (see the second row oh_1 of the convolution result in FIG. 8) of the convolution output result.

[0139] Similarly, the fourth loop block data [1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16] determined by the element in the r = 1th row and the s = 1th column of the convolution kernel when calculating the first row of the convolution output result (see the first row oh_0 of the convolution result in FIG. 8) can be reused as the fourth loop block data [1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16] determined by the element in the r = 0th row and the s = 1th column of the convolution kernel when calculating the second row of the convolution output result (see the second row oh_1 of the convolution result in FIG. 8), and the fourth loop block data [2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17] determined by the element in the r = 1th row and the s = 2th column of the convolution kernel when calculating the first row of the convolution output result (see the first row oh_0 of the convolution result in FIG. 8) can be reused as the fourth loop block data [2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17] determined by the element in the r = 0th row and the s = 2th column of the convolution kernel when calculating the second row of the convolution output result (see the second row oh_1 of the convolution result in FIG. 8).

[0140] Compared with the related art, the calculation logic of adding software im2col (for example, sliding the convolution kernel on the data to be convolved, converting the data contained in each convolution kernel window into a column vector, and finally arranging the new matrix in columns) for implementing the convolution operation in the related art can introduce additional overhead, for example, 9 times the amount of data is required for a common 3x3 size convolution kernel, which has a large pressure on the storage bandwidth. According to the embodiments of the present disclosure, the third loop block data of the convolution kernel and the fourth loop block data of the data to be convolved can be determined in the order of the coordinates of the elements in the convolution kernel, and the convolution engine can be implemented by reusing the matrix calculation structure without im2col expansion in the memory, thereby reducing the pressure on the storage bandwidth. By setting the reuse cache in the processor, the input data of the convolution is reused, and the number of accesses to the local memory outside the processor and the bandwidth pressure on the local memory outside the processor are further reduced.

[0141] In a possible implementation, the control logic unit, according to the coordinate order of the elements in the convolution kernel, constructs third loop block data from a plurality of elements with the same coordinates read from a plurality of the convolution kernels in the memory each time, including: the control logic unit reads a plurality of convolution kernels from the local memory to the reuse cache; according to the coordinate order of the elements in the convolution kernel, the control logic unit constructs third loop block data from a plurality of elements with the same coordinates read from a plurality of the convolution kernels in the reuse cache each time.

[0142] For example, assuming that the local storage stores the convolution kernels [S1, S2], [S3, S4], and [S5, S6], the control logic unit can read the three convolution kernels [S1, S2], [S3, S4], and [S5, S6] from the local storage to the reuse cache. In this way, the control logic unit reads the elements S1, S3, and S5 at the coordinate 1 from the reuse cache in the first time according to the coordinate order of the elements in the convolution kernels [S1, S2], [S3, S4], and [S5, S6], to form the third loop blocking data [S1, S3, S5]; and reads the elements S2, S4, and S6 at the coordinate 2 from the reuse cache in the second time, to form the third loop blocking data [S2, S4, S6]. It should be understood that the disclosure only takes the convolution kernels [S1, S2], [S3, S4], and [S5, S6] as an example, and does not limit the size and number of the convolution kernels.

[0143] In this way, the convolution kernel has been read to the reuse cache inside the processor, and the convolution kernel stored in the reuse cache can be reused, so that repeated reading of the local storage outside the processor is avoided.

[0144] In a possible implementation, the point multiplication unit array performs product accumulation operation on the loop blocking data of the received data each time to determine the loop blocking result of the received data each time, including: the point multiplication unit array performs product accumulation operation on the third loop blocking data and the fourth loop blocking data received each time to determine the loop blocking result. For details, refer to the above description of the point multiplication unit array performing product accumulation operation on the first loop blocking data and the second loop blocking data obtained each time, which will not be repeated here.

[0145] The subsequent control logic unit can perform accumulation operation on the loop blocking results obtained from the point multiplication unit array in sequence according to the loop blocking order to obtain the logic operator result, and write the logic operator result to the register stack; and the control logic unit can determine the convolution output result according to the logic operator results obtained from the register stack.

[0146] In this way, it is beneficial to reuse the matrix calculation structure to implement the convolution engine, and beneficial to reduce the complexity of software users in the subsequent implementation.

[0147] In the data processing method of the embodiments of the present disclosure, the control logic unit reads the cyclic block data of the to-be-processed data from the memory to the dot multiplication unit array in sequence according to the control instruction obtained for indicating the logical operation of the to-be-processed data, so that the dot multiplication unit array performs the product accumulation operation on the cyclic block data of the to-be-processed data received each time to determine the cyclic block result of the to-be-processed data received each time, and the control logic unit determines the logical operation result of the to-be-processed data according to the plurality of cyclic block results obtained from the dot multiplication unit array. In this way, the reading and logical operation of the to-be-processed data (for example, large-size image data) can be converted into the reading and logical operation of the plurality of cyclic block data (small-size image data) of the to-be-processed data, which is beneficial to processing larger-size data under the condition that the hardware resources of the processor remain unchanged and reducing the pressure on the storage bandwidth.

[0148] It can be understood that the above-mentioned various method embodiments of the present disclosure can be combined with each other to form combined embodiments without deviating from the principle logic. Limited by the length of the present disclosure, the present disclosure will not be described again. It can be understood by those skilled in the art that the specific execution order of each step in the above-mentioned method should be determined according to its function and possible internal logic.

[0149] In addition, the present disclosure also provides a processor, an electronic device, a computer readable storage medium, and a program, which can be used to implement any one of the data processing methods provided by the present disclosure. The corresponding technical solutions and descriptions are described in the method part and will not be described again.

[0150] In a possible implementation, as shown in FIG. 1, the processor includes a control logic unit and a dot multiplication unit array including a plurality of dot multiplication units for performing product accumulation operation. The processor is configured to: the control logic unit reads the cyclic block data of the to-be-processed data from the memory to the dot multiplication unit array in sequence according to the control instruction obtained for indicating the logical operation of the to-be-processed data; the dot multiplication unit array performs the product accumulation operation on the cyclic block data of the to-be-processed data received each time to determine the cyclic block result of the to-be-processed data received each time; and the control logic unit determines the logical operation result of the to-be-processed data according to the plurality of cyclic block results obtained from the dot multiplication unit array.

[0151] In a possible implementation, the control logic unit reads the cyclic block data of the to-be-processed data from the memory to the dot multiplication unit array in sequence according to the obtained control instruction, including: the control logic unit parses the access information of the to-be-processed data and the logical operation type of the to-be-processed data according to the obtained control instruction; the cyclic block sequence of the to-be-processed data is determined according to the logical operation type of the to-be-processed data; and the cyclic block data of the to-be-processed data is read from the memory to the dot multiplication unit array in sequence according to the access information and the preset size, in the cyclic block sequence of the to-be-processed data, the size of the cyclic block data is less than or equal to the preset size, and the preset size is determined by the number of dot multiplication units in the idle state in the dot multiplication unit array.

[0152] In a possible implementation, the access information of the to-be-processed data includes the access information of a first matrix and a second matrix, the number of columns of the first matrix is the same as the number of rows of the second matrix, the logical operation type includes a matrix multiplication operation of the first matrix and the second matrix, and the cyclic block sequence determined by the matrix multiplication operation type includes: a first outer loop sequence in the row number direction of the first matrix, a second outer loop sequence in the column number direction of the second matrix, and an inner loop sequence in the column number direction of the first matrix and the row number direction of the second matrix; or a first outer loop sequence in the column number direction of the second matrix, a second outer loop sequence in the row number direction of the first matrix, and an inner loop sequence in the column number direction of the first matrix and the row number direction of the second matrix.

[0153] In a possible implementation, the memory includes a local memory arranged outside the processor and a reuse cache arranged inside the processor for caching cyclic block data, and the reading of the cyclic block data of the to-be-processed data from the memory to the dot multiplication unit array in the cyclic block sequence of the to-be-processed data includes: the control logic unit stores the first cyclic block data of the first matrix read from the local memory into the reuse cache according to the first outer loop sequence and the inner loop sequence, so that the control logic unit reuses the first cyclic block data stored in the reuse cache according to the second outer loop sequence; or the control logic unit stores the second cyclic block data of the second matrix read from the local memory into the reuse cache according to the second outer loop sequence and the inner loop sequence, so that the control logic unit reuses the second cyclic block data stored in the reuse cache according to the first outer loop sequence.

[0154] In a possible implementation, the point multiplication unit array performs a product accumulation operation on the received cyclic block data of the to-be-processed data each time, and determines a cyclic block result of the to-be-processed data each time, including: the point multiplication unit array performs a product accumulation operation on the received first cyclic block data and the second cyclic block data each time, to obtain a cyclic block result corresponding to each product accumulation operation; and the control logic unit determines a logical operation result of the to-be-processed data according to the cyclic block results obtained from the point multiplication unit array, including: the control logic unit performs an accumulation operation on the cyclic block results obtained from the point multiplication unit array in the cyclic block order each time, to obtain a logical operation sub-result; the control logic unit writes the logical operation sub-result into a register stack; and the control logic unit determines the logical operation result according to the logical operation sub-results obtained from the register stack.

[0155] In a possible implementation, the access information of the to-be-processed data includes access information of to-be-convoluted data and a convolution kernel, the logical operation type includes a convolution operation of the to-be-convoluted data and the convolution kernel, and the cyclic block order determined according to the convolution operation type includes a coordinate order of elements in the convolution kernel. The reading of the cyclic block data of the to-be-processed data from the memory into the point multiplication unit array according to the cyclic block order of the to-be-processed data includes: the control logic unit reads, according to the coordinate order of the elements in the convolution kernel, a plurality of elements with the same coordinates from a plurality of the convolution kernels in the memory each time, to form third cyclic block data, and reads a plurality of elements from the to-be-convoluted data in the memory according to the coordinates of a current element in the convolution kernel and the access information, to form fourth cyclic block data; and the control logic unit writes the third cyclic block data and the fourth cyclic block data into the point multiplication unit array.

[0156] In a possible implementation, the memory includes a local memory arranged outside the processor and a reuse cache arranged inside the processor and used for caching cyclic block data. The processor is further configured to: in a process of determining fourth cyclic block data corresponding to the same row in the calculation of a convolution output result, the control logic unit writes, according to the coordinate order of the elements in the convolution kernel, M elements of the fourth cyclic block data corresponding to the coordinates of an element in the rth row and the sth column of the convolution kernel into the reuse cache, where r, s, and M are positive integers; the control logic unit reads M-1 elements from the reuse cache and reads one element from the to-be-convoluted data in the local memory, to determine the fourth cyclic block data corresponding to the coordinates of an element in the rth row and the (s+1)th column of the convolution kernel.

[0157] In a possible implementation, the processor is further configured to: in a fourth loop tiling data process corresponding to adjacent rows in the calculation of the convolution output result, in response to the control logic unit calculating the fourth loop tiling data of a current row of the convolution output result, write M elements of the fourth loop tiling data corresponding to the coordinates of an element in the rth row and the sth column of the convolution kernel into a reuse cache, where r, s, and M are positive integers; and in response to the control logic unit calculating the fourth loop tiling data of a next row of the current row of the convolution output result, read the M elements from the reuse cache to form the fourth loop tiling data corresponding to the coordinates of an element in the (r-1)th row and the sth column of the convolution kernel.

[0158] In a possible implementation, the memory includes a local memory disposed outside the processor and a reuse cache disposed inside the processor for caching loop tiling data, and the control logic unit is configured to form, in the order of the coordinates of the elements in the convolution kernel, the plurality of elements with the same coordinates read from the plurality of convolution kernels in the memory each time as the third loop tiling data, including: the control logic unit reading a plurality of convolution kernels from the local memory to the reuse cache; and the control logic unit forming, in the order of the coordinates of the elements in the convolution kernel, the plurality of elements with the same coordinates read from the plurality of convolution kernels in the reuse cache each time as the third loop tiling data.

[0159] In a possible implementation, the point multiplication unit array performs product accumulation operation on the loop tiling data of the to-be-processed data received each time to determine the loop tiling result of the to-be-processed data received each time, including: the point multiplication unit array performing product accumulation operation on the third loop tiling data and the fourth loop tiling data received each time to determine the loop tiling result.

[0160] In some embodiments, the processor provided by the embodiments of the present disclosure has functions or includes modules that can be used to perform the methods described in the above method embodiments, and the specific implementation can refer to the description of the above method embodiments. For brevity, details are not repeated here.

[0161] The embodiments of the present disclosure also provide a computer readable storage medium having computer program instructions stored therein, and the computer program instructions are executed by a processor to implement the above method. The computer readable storage medium can be a volatile or non-volatile computer readable storage medium.

[0162] The embodiments of the present disclosure also provide an artificial intelligence chip, which includes the processor as described above.

[0163] The electronic device can be provided as a terminal, a server or other forms of device.

[0164] The embodiments of the present disclosure further provide a computer program product, including a computer readable code or a non-volatile computer readable storage medium carrying the computer readable code, when the computer readable code is executed in a processor of an electronic device, the processor in the electronic device performs the above method.

[0165] The electronic device can be provided as a terminal, a server or other forms of device.

[0166] FIG. 9 shows a block diagram of an electronic device 1900 according to an embodiment of the present disclosure. For example, the electronic device 1900 can be provided as a server or a terminal device. Referring to FIG. 9, the electronic device 1900 includes a processing component 1922, further including one or more processors, and a memory resource represented by a memory 1932, for storing instructions executable by the processing component 1922, such as an application program. The application program stored in the memory 1932 can include one or more than one module each corresponding to a set of instructions. In addition, the processing component 1922 is configured to execute the instructions to perform the above method.

[0167] The electronic device 1900 can further include a power supply component 1926 configured to perform power management of the electronic device 1900, a wired or wireless network interface 1950 configured to connect the electronic device 1900 to a network, and an input / output (I / O) interface 1958. The electronic device 1900 can operate based on an operating system stored in the memory 1932, such as Microsoft Windows Server operating system (Windows Server TM ), Apple's graphical user interface-based operating system (Mac OS X TM ), multi-user multi-process computer operating system (Unix TM ), free and open source Unix-like operating system (Linux TM ), open source Unix-like operating system (FreeBSD TM ) or the like.

[0168] In an example embodiment, a non-transitory computer-readable storage medium, e.g., memory 1932 including computer program instructions, is also provided that can be executed by the processing component(s) 1922 of the electronic device 1900 to implement the above-described methods.

[0169] The present disclosure can be a system, a method, and / or a computer program product. The computer program product can include a computer readable storage medium (or media) having computer readable program instructions thereon for causing a processor to carry out aspects of the present disclosure.

[0170] The computer readable storage medium can be a tangible device that can retain and store instructions for use by an instruction execution device. The computer readable storage medium can be, for example, but is not limited to, an electronic storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the foregoing. A non-exhaustive list of more specific examples of the computer readable storage medium includes the following: a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), a static random access memory (SRAM), a portable compact disc read-only memory (CD-ROM), a digital versatile disk (DVD), a memory stick, a floppy disk, a mechanically encoded device such as punch-cards or punched tape, a

[0171] The computer readable program instructions described herein can be downloaded to respective computing / processing devices from a computer readable storage medium or to an external computer or external storage device via a network, for example, the Internet, a local area network, a wide area network and / or a wireless network. The network can comprise copper transmission cables, optical transmission fibers, wireless transmission, routers, firewalls, switches, gateway computers and / or edge servers. A network adapter card or network interface in each computing / processing device receives computer readable program instructions from the network and forwards the computer readable program instructions for storage in a computer readable storage medium within the respective computing / processing device.

[0172] Computer readable program instructions for carrying out operations of the present disclosure can be assembler instructions, instruction-set-architecture (ISA) instructions, machine instructions, machine dependent instructions, microcode, firmware instructions, state-setting data, or either source code or object code written in any combination of one or more programming languages, including an object oriented programming language such as Smalltalk, C++ or the like and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The computer readable program instructions can execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the latter scenario, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection can be made to an external computer (for example, through the Internet using an Internet Service Provider). In some embodiments, electronic circuitry including, for example, programmable logic circuitry, field-programmable gate array (FPGA), or programmable logic array (PLA) can execute the computer readable program instructions by utilizing state information of the computer readable program instructions to personalize the electronic circuitry, in order to perform aspects of the present disclosure.

[0173] The computer readable program instructions can also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable apparatus or other device to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide processes for implementing the functions / acts specified in the flowchart and / or block diagram block or blocks.

[0174] These computer readable program instructions can be provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions / acts specified in the flowchart and / or block diagram block or blocks. These computer readable program instructions can also be stored in a computer readable storage medium that can include random access memory (RAM), read only memory (ROM), electrically erasable programmable read only memory (EEPROM), flash memory or other data storage device. When the computer readable program instructions are loaded into the computer and other programmable data processing apparatus, a series of operational steps are implemented that provide processes such that the instructions which operate on the computer or other programmable data processing apparatus provide processes for implementing the functions / acts specified in the flowchart and / or block diagram block or blocks.

[0175] The computer readable program instructions can also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable data processing apparatus or other device to produce a computer implemented process such that the instructions which execute on the computer, other programmable data processing apparatus, or other device implement the functions / acts specified in the flowchart and / or block diagram block or blocks.

[0176] The flow diagrams and the block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flow diagrams and the block diagrams can represent a module, segment, or portion of instructions, which comprises one or more executable instructions for implementing the specified logic functions. In some alternative implementations, the functions noted in the blocks can occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and / or flow diagrams, and combinations thereof, can be implemented by special purpose hardware-based systems that perform the specified functions or acts, or combinations of special purpose hardware and

[0177] The computer program product can be embodied in a tangible medium of

[0178] The above description of the various embodiments is intended to be illustrative in all aspects, rather than being restrictive. Those skilled in the art can refer to the description of the various embodiments to make modifications and / or improvements.

[0179] Those skilled in the art can understand that, in the above-described method of the specific embodiments, the writing order of each step does not mean a strict execution order and does not constitute any limitation on the implementation process. The specific execution order of each step should be determined by its function and possible inherent logic.

[0180] Having described above several embodiments of the disclosure, any modifications and variations that fall within the scope of the described embodiments are also intended to be within the scope of the disclosure. As will be apparent to those skilled in the art, some modifications and variations to the embodiments described above can be practiced while staying within the scope and spirit of the described embodiments. The foregoing description of the described embodiments has been presented for the purposes of illustration and description. It is not intended to be exhaustive or to limit the described embodiments to the precise form disclosed. Many modifications and variations are possible in light of the above teachings. It is intended that the disclosed embodiments be limited only by the claims.

Claims

1. A data processing method, characterized by, The data processing method is applied to a processor, the processor comprising a control logic unit and a point multiplication unit array comprising a plurality of point multiplication units for performing a product-sum operation, the method comprising: The control logic unit reads cyclic block data of to-be-processed data from a memory to the point multiplication unit array according to a control instruction obtained; The point multiplication unit array performs a product-sum operation on the cyclic block data of the to-be-processed data received each time to determine a cyclic block result of the to-be-processed data received each time; The control logic unit determines a logical operation result of the to-be-processed data according to a plurality of cyclic block results obtained from the point multiplication unit array.

2. The method of claim 1, wherein, The control logic unit reads cyclic block data of to-be-processed data from a memory to the point multiplication unit array according to a control instruction obtained, comprising: The control logic unit parses access information of the to-be-processed data and a logical operation type of the to-be-processed data according to a control instruction obtained; determining a cyclic block sequence of the to-be-processed data according to the logical operation type of the to-be-processed data; reading cyclic block data of the to-be-processed data from the memory to the point multiplication unit array according to the cyclic block sequence of the to-be-processed data and the access information and a preset size, the size of the cyclic block data being less than or equal to the preset size, the preset size being determined by the number of point multiplication units in an idle state in the point multiplication unit array.

3. The method of claim 2, wherein, The access information of the to-be-processed data comprises access information of a first matrix and a second matrix, the number of columns of the first matrix being the same as the number of rows of the second matrix, and the logical operation type comprising a matrix multiplication operation of the first matrix and the second matrix, The cyclic block sequence determined according to the matrix multiplication operation type comprises a first outer loop sequence in the row direction of the first matrix, a second outer loop sequence in the column direction of the second matrix, and an inner loop sequence in the column direction of the first matrix and the row direction of the second matrix. Or, a first outer loop sequence in the column direction of the second matrix, a second outer loop sequence in the row direction of the first matrix, and an inner loop sequence in the column direction of the first matrix and the row direction of the second matrix.

4. The method of claim 3, wherein, The memory comprises a local memory arranged outside the processor and a reuse cache arranged inside the processor for caching cyclic block data, and reading cyclic block data of the to-be-processed data from the memory to the point multiplication unit array according to the cyclic block sequence of the to-be-processed data comprises: The control logic unit stores first cyclic block data of the first matrix read from the local memory into the reuse cache according to the first outer loop sequence and the inner loop sequence, so that the control logic unit reuses the first cyclic block data stored in the reuse cache according to the second outer loop sequence. Or, The control logic unit stores the second loop block data of the second matrix read from the local memory into the reuse cache according to the second outer loop sequence and the inner loop sequence, so that the control logic unit reuses the second loop block data stored in the reuse cache according to the first outer loop sequence.

5. The method of claim 4, wherein, The point multiplication unit array performs product accumulation operation on the loop block data of the to-be-processed data received each time to determine the loop block result of the to-be-processed data received each time, including: The point multiplication unit array performs product accumulation operation on the first loop block data and the second loop block data obtained each time to obtain the loop block result corresponding to each product accumulation operation; The control logic unit determines the logical operation result of the to-be-processed data according to the plurality of loop block results obtained from the point multiplication unit array, including: The control logic unit performs accumulation operation on the plurality of loop block results obtained from the point multiplication unit array in the loop block sequence of any round to obtain a logical operation sub-result; The control logic unit writes the logical operation sub-result into the register stack; The control logic unit determines the logical operation result according to the plurality of logical operation sub-results obtained from the register stack.

6. The method of claim 2, wherein, The access information of the to-be-processed data includes access information of to-be-convoluted data and a convolution kernel, and the logical operation type includes convolution operation of the to-be-convoluted data and the convolution kernel, The loop block sequence determined according to the convolution operation type includes a coordinate sequence of elements in the convolution kernel, and the reading of the loop block data of the to-be-processed data from the memory to the point multiplication unit array according to the loop block sequence of the to-be-processed data includes: The control logic unit reads a plurality of elements with the same coordinates from a plurality of the convolution kernels in the memory each time according to the coordinate sequence of the elements in the convolution kernel to form third loop block data, and reads a plurality of elements from the to-be-convoluted data in the memory according to the coordinates of the current element in the convolution kernel each time and the access information to form fourth loop block data; The control logic unit writes the third loop block data and the fourth loop block data into the point multiplication unit array.

7. The method of claim 6, wherein, The memory includes a local memory arranged outside the processor and a reuse cache arranged inside the processor for caching loop block data, and the method further includes: In the process of calculating the fourth loop block data corresponding to the same row in the convolution output result, the control logic unit writes M elements of the fourth loop block data corresponding to the coordinates of the element in the rth row and the sth column of the convolution kernel into the reuse cache according to the coordinate sequence of the elements in the convolution kernel, where r, s and M are positive integers; The control logic unit reads M-1 elements from the reuse cache and reads 1 element from the to-be-convoluted data in the local memory to determine the fourth loop block data corresponding to the coordinates of the element in the rth row and the s+1th column of the convolution kernel.

8. The method of claim 6, wherein, The method further comprises: in a fourth loop blocking data process corresponding to adjacent rows in calculating a convolution output result, in response to the control logic unit calculating fourth loop blocking data of a current row of the convolution output result, writing M elements corresponding to a fourth loop blocking number of a coordinate of an element in an rth row and sth column of the convolution kernel into a reuse cache, r, s, and M being positive integers; in response to the control logic unit calculating fourth loop blocking data of a next row of the current row of the convolution output result, reading M elements from the reuse cache to form fourth loop blocking data corresponding to a coordinate of an element in an (r-1)th row and sth column of the convolution kernel.

9. The method of claim 6, wherein, The memory comprises a local memory disposed outside the processor and a reuse cache disposed inside the processor for caching loop blocking data, and the control logic unit forms, in order of coordinates of elements in the convolution kernel, third loop blocking data from a plurality of elements with the same coordinates read from a plurality of the convolution kernels each time from the memory. The control logic unit reads a plurality of convolution kernels from the local memory to the reuse cache; The control logic unit forms, in order of coordinates of elements in the convolution kernel, third loop blocking data from a plurality of elements with the same coordinates read from a plurality of the convolution kernels each time from the reuse cache.

10. The method according to any one of claims 6-9, characterized in that, The point multiplication unit array performs product accumulation operation on loop blocking data of the to-be-processed data received each time to determine loop blocking results of the to-be-processed data received each time. The point multiplication unit array performs product accumulation operation on the third loop blocking data and the fourth loop blocking data received each time to determine loop blocking results.

11. A processor, comprising: The processor comprises a control logic unit and a point multiplication unit array, and the point multiplication unit array comprises a plurality of point multiplication units for performing product accumulation operation, The processor is configured to: The control logic unit reads loop blocking data of to-be-processed data from the memory to the point multiplication unit array in sequence according to the obtained control instruction; The point multiplication unit array performs product accumulation operation on loop blocking data of the to-be-processed data received each time to determine loop blocking results of the to-be-processed data received each time; The control logic unit determines a logical operation result of the to-be-processed data according to a plurality of the loop blocking results obtained from the point multiplication unit array.

12. An artificial intelligence chip, characterized by, The artificial intelligence chip comprises the processor of claim 11.

13. An electronic device, comprising: The electronic device comprises the artificial intelligence chip of claim 12.

Images