Data processing method, processor, chip and electronic device

Abstract

The present disclosure relates to a data processing method, a processor, a chip and an electronic device, the method comprising: a control logic unit sequentially reads the cyclic block data of the data to be processed from the memory to the dot multiplication unit array according to the acquired control instruction, the dot multiplication unit array performs multiplication and accumulation operations on the cyclic block data of the data to be processed each time received, and determines the cyclic block result of the data to be processed each time received, and the control logic unit determines the logical operation result of the data to be processed according to the multiple cyclic block results obtained from the dot multiplication unit array. The embodiment of the present disclosure can convert the reading and logical operation of the data to be processed into the reading and logical operation of multiple cyclic block data of the data to be processed, which is conducive to processing larger size data without changing the processor hardware resources and reducing the pressure on the storage bandwidth.

Description

Technical Field

[0001] The present disclosure relates to the field of computer technology, and in particular to a data processing method, a processor, a chip, and an electronic device. Background Art

[0002] In the field of artificial intelligence technology, neural network algorithms are a very popular machine learning algorithm recently, and have achieved very good results in various fields, such as image recognition, speech recognition, natural language processing, etc. With the development of neural network algorithms, the complexity of the algorithms is getting higher and higher. In order to improve the recognition, the scale of the model is also gradually increasing. Using processors such as central processing units (CPUs) and graphic processing units (GPUs) to process these large-scale models takes a lot of computing time and consumes a lot of power.

[0003] A large number of proprietary modules have been added to processors and AI accelerators to accelerate training and reasoning. Matrix multiplication and convolution are common operators in AI algorithms. The Single Instruction Multiple Data (SIMD) or Single Instruction Multiple Thread (SIMT) architecture of the processor in the related art relies on multiplication and accumulation instructions to implement the corresponding algorithm. There is little data sharing between different threads, and repeated data needs to be read multiple times. The performance and energy consumption of the processor are relatively poor. Summary of the invention

[0004] The present disclosure proposes a data processing technology solution.

[0005] According to one aspect of the present disclosure, a data processing method is provided, comprising: the data processing method is applied to a processor, the processor comprising a control logic unit and a dot multiplication unit array, the dot multiplication unit array comprising a plurality of dot multiplication units for performing multiplication and accumulation operations, the method comprising: the control logic unit sequentially reads cyclic block data of data to be processed from a memory to the dot multiplication unit array according to an acquired control instruction; the dot multiplication unit array performs a multiplication and accumulation operation on the cyclic block data of the data to be processed received each time, and determines a cyclic block result of the data to be processed received each time; the control logic unit determines a logic operation result of the data to be processed according to the plurality of cyclic block results acquired from the dot multiplication unit array.

[0006] In a possible implementation, the control logic unit reads the cyclic block data of the data to be processed from the memory in sequence to the dot multiplication unit array according to the acquired control instruction, including: the control logic unit parses the access information of the data to be processed and the logical operation type of the data to be processed according to the acquired control instruction; determines the cyclic block order of the data to be processed according to the logical operation type of the data to be processed; and reads the cyclic block data of the data to be processed from the memory in sequence to the dot multiplication unit array according to the access information and the preset size and the cyclic block order of the data to be processed, wherein the size of the cyclic block data is less than or equal to the preset size, and the preset size is determined by the number of dot multiplication units in the dot multiplication unit array that are in an idle state.

[0007] In a possible implementation, the access information of the data to be processed includes access information of a first matrix and a second matrix, the number of columns of the first matrix is ​​the same as the number of rows of the second matrix, the logical operation type includes a matrix multiplication operation of the first matrix and the second matrix, and the loop block order determined by the matrix multiplication operation type includes: a first outer loop order in the direction of the number of rows of the first matrix, a second outer loop order in the direction of the number of columns of the second matrix, and an inner loop order in the direction of the number of columns of the first matrix and the number of rows of the second matrix; or, a first outer loop order in the direction of the number of columns of the second matrix, a second outer loop order in the direction of the number of rows of the first matrix, and an inner loop order in the direction of the number of columns of the first matrix and the number of rows of the second matrix.

[0008] In a possible implementation, the memory includes a local memory arranged outside the processor, and a reuse cache arranged in the processor for caching loop block data, and the loop block data of the data to be processed are read from the memory in sequence to the dot product unit array according to the loop block order of the data to be processed, including: the control logic unit stores the first loop block data of the first matrix read from the local memory into the reuse cache according to the first outer loop order and the inner loop order, so that the control logic unit reuses the first loop block data stored in the reuse cache according to the second outer loop order; or, the control logic unit stores the second loop block data of the second matrix read from the local memory into the reuse cache according to the second outer loop order and the inner loop order, so that the control logic unit reuses the second loop block data stored in the reuse cache according to the first outer loop order.

[0009] In a possible implementation, the dot product unit array performs a multiplication and accumulation operation on the circular block data of the data to be processed each time it is received, and determines the circular block result of the data to be processed each time it is received, including: the dot product unit array performs a multiplication and accumulation operation on the first circular block data and the second circular block data obtained each time, and obtains the circular block result corresponding to each multiplication and accumulation operation; the control logic unit determines the logical operation result of the data to be processed according to the multiple circular block results obtained from the dot product unit array, including: the control logic unit performs an accumulation operation on the multiple circular block results obtained from the dot product unit array in any round of the circular block sequence, and obtains a logical operator result; the control logic unit writes the logical operator result into a register stack; the control logic unit determines the logical operation result according to the multiple logical operator results obtained from the register stack.

[0010] In a possible implementation, the access information of the data to be processed includes access information of the data to be convolved and the convolution kernel, the logical operation type includes the convolution operation of the data to be convolved and the convolution kernel, the circular block order determined by the convolution operation type includes the coordinate order of the elements in the convolution kernel, and the circular block data of the data to be processed are read from the memory in sequence to the dot product unit array according to the circular block order of the data to be processed, including: the control logic unit, according to the coordinate order of the elements in the convolution kernel, converts multiple elements with the same coordinates read from multiple convolution kernels in the memory each time to form third circular block data, and, according to the coordinates of the current element in each convolution kernel and the access information, reads multiple elements from the data to be convolved in the memory to form fourth circular block data; the control logic unit writes the third circular block data and the fourth circular block data into the dot product unit array.

[0011] In one possible implementation, the memory includes a local memory arranged outside the processor, and a reuse cache arranged in the processor for caching circular block data. The method also includes: in the process of calculating the fourth circular block data corresponding to the same row in the convolution output result, the control logic unit writes M elements of the fourth circular block number corresponding to the coordinates of the element in the rth row and sth column of the convolution kernel into the reuse cache according to the coordinate order of the elements in the convolution kernel, where r, s, and M are positive integers; the control logic unit reads M-1 elements from the reuse cache, reads 1 element from the data to be convolved in the local memory, and determines the fourth circular block data corresponding to the coordinates of the element in the rth row and s+1th column of the convolution kernel.

[0012] In one possible implementation, the method further includes: in the process of calculating the fourth circular block data corresponding to adjacent rows in the convolution output result, in response to the control logic unit calculating the fourth circular block data of the current row of the convolution output result, writing M elements of the fourth circular block number corresponding to the coordinates of the element in the rth row and sth column of the convolution kernel into a reuse cache, where r, s, and M are positive integers; in response to the control logic unit calculating the fourth circular block data of the next row of the current row of the convolution output result, reading M elements from the reuse cache to constitute the fourth circular block data corresponding to the coordinates of the element in the r-1th row and sth column of the convolution kernel.

[0013] In a possible implementation, the memory includes a local memory arranged outside the processor, and a reuse cache arranged in the processor for caching circular block data, and the control logic unit, according to the coordinate order of the elements in the convolution kernel, converts multiple elements with the same coordinates read from multiple convolution kernels in the memory each time into third circular block data, including: the control logic unit reads multiple convolution kernels from the local memory to the reuse cache; according to the coordinate order of the elements in the convolution kernel, the control logic unit converts multiple elements with the same coordinates read from multiple convolution kernels in the reuse cache each time into third circular block data.

[0014] In a possible implementation, the dot product unit array performs a multiplication and accumulation operation on the circular block data of the data to be processed each time received to determine the circular block result of the data to be processed each time received, including: the dot product unit array performs a multiplication and accumulation operation on the third circular block data and the fourth circular block data each time received to determine the circular block result.

[0015] According to one aspect of the present disclosure, a processor is provided, the processor comprising a control logic unit and a dot multiplication unit array, the dot multiplication unit array comprising a plurality of dot multiplication units for performing multiplication and accumulation operations, the processor being used for: the control logic unit sequentially reading the cyclic block data of the data to be processed from the memory to the dot multiplication unit array according to the acquired control instruction; the dot multiplication unit array performing a multiplication and accumulation operation on the cyclic block data of the data to be processed received each time, and determining the cyclic block result of the data to be processed received each time; the control logic unit determining the logic operation result of the data to be processed according to the plurality of cyclic block results acquired from the dot multiplication unit array.

[0016] In a possible implementation, the control logic unit reads the cyclic block data of the data to be processed from the memory in sequence to the dot multiplication unit array according to the acquired control instruction, including: the control logic unit parses the access information of the data to be processed and the logical operation type of the data to be processed according to the acquired control instruction; determines the cyclic block order of the data to be processed according to the logical operation type of the data to be processed; and reads the cyclic block data of the data to be processed from the memory in sequence to the dot multiplication unit array according to the access information and the preset size and the cyclic block order of the data to be processed, wherein the size of the cyclic block data is less than or equal to the preset size, and the preset size is determined by the number of dot multiplication units in the dot multiplication unit array that are in an idle state.

[0017] In a possible implementation, the access information of the data to be processed includes access information of a first matrix and a second matrix, the number of columns of the first matrix is ​​the same as the number of rows of the second matrix, the logical operation type includes a matrix multiplication operation of the first matrix and the second matrix, and the loop block order determined by the matrix multiplication operation type includes: a first outer loop order in the direction of the number of rows of the first matrix, a second outer loop order in the direction of the number of columns of the second matrix, and an inner loop order in the direction of the number of columns of the first matrix and the number of rows of the second matrix; or, a first outer loop order in the direction of the number of columns of the second matrix, a second outer loop order in the direction of the number of rows of the first matrix, and an inner loop order in the direction of the number of columns of the first matrix and the number of rows of the second matrix.

[0018] In a possible implementation, the memory includes a local memory arranged outside the processor, and a reuse cache arranged in the processor for caching loop block data, and the loop block data of the data to be processed are read from the memory in sequence to the dot product unit array according to the loop block order of the data to be processed, including: the control logic unit stores the first loop block data of the first matrix read from the local memory into the reuse cache according to the first outer loop order and the inner loop order, so that the control logic unit reuses the first loop block data stored in the reuse cache according to the second outer loop order; or, the control logic unit stores the second loop block data of the second matrix read from the local memory into the reuse cache according to the second outer loop order and the inner loop order, so that the control logic unit reuses the second loop block data stored in the reuse cache according to the first outer loop order.

[0019] In a possible implementation, the dot product unit array performs a multiplication and accumulation operation on the circular block data of the data to be processed each time it is received, and determines the circular block result of the data to be processed each time it is received, including: the dot product unit array performs a multiplication and accumulation operation on the first circular block data and the second circular block data obtained each time, and obtains the circular block result corresponding to each multiplication and accumulation operation; the control logic unit determines the logical operation result of the data to be processed according to the multiple circular block results obtained from the dot product unit array, including: the control logic unit performs an accumulation operation on the multiple circular block results obtained from the dot product unit array in any round of the circular block sequence, and obtains a logical operator result; the control logic unit writes the logical operator result into a register stack; the control logic unit determines the logical operation result according to the multiple logical operator results obtained from the register stack.

[0020] In a possible implementation, the access information of the data to be processed includes access information of the data to be convolved and the convolution kernel, the logical operation type includes the convolution operation of the data to be convolved and the convolution kernel, the circular block order determined by the convolution operation type includes the coordinate order of the elements in the convolution kernel, and the circular block data of the data to be processed are read from the memory in sequence to the dot product unit array according to the circular block order of the data to be processed, including: the control logic unit, according to the coordinate order of the elements in the convolution kernel, converts multiple elements with the same coordinates read from multiple convolution kernels in the memory each time to form third circular block data, and, according to the coordinates of the current element in each convolution kernel and the access information, reads multiple elements from the data to be convolved in the memory to form fourth circular block data; the control logic unit writes the third circular block data and the fourth circular block data into the dot product unit array.

[0021] In one possible implementation, the memory includes a local memory arranged outside the processor, and a reuse cache arranged in the processor for caching circular block data, and the processor is also used for: in the process of calculating the fourth circular block data corresponding to the same row in the convolution output result, the control logic unit writes M elements of the fourth circular block number corresponding to the coordinates of the element in the rth row and sth column of the convolution kernel into the reuse cache according to the coordinate order of the elements in the convolution kernel, where r, s, and M are positive integers; the control logic unit reads M-1 elements from the reuse cache, reads 1 element from the data to be convolved in the local memory, and determines the fourth circular block data corresponding to the coordinates of the element in the rth row and s+1th column of the convolution kernel.

[0022] In one possible implementation, the processor is also used to: in the process of calculating the fourth circular block data corresponding to adjacent rows in the convolution output result, in response to the control logic unit calculating the fourth circular block data of the current row of the convolution output result, write M elements of the fourth circular block number corresponding to the coordinates of the element in the rth row and sth column in the convolution kernel into a reuse cache, where r, s, and M are positive integers; in response to the control logic unit calculating the fourth circular block data of the next row of the current row of the convolution output result, read M elements from the reuse cache to constitute the fourth circular block data corresponding to the coordinates of the element in the r-1th row and sth column in the convolution kernel.

[0023] In a possible implementation, the memory includes a local memory arranged outside the processor, and a reuse cache arranged in the processor for caching circular block data, and the control logic unit, according to the coordinate order of the elements in the convolution kernel, converts multiple elements with the same coordinates read from multiple convolution kernels in the memory each time into third circular block data, including: the control logic unit reads multiple convolution kernels from the local memory to the reuse cache; according to the coordinate order of the elements in the convolution kernel, the control logic unit converts multiple elements with the same coordinates read from multiple convolution kernels in the reuse cache each time into third circular block data.

[0024] In a possible implementation, the dot product unit array performs a multiplication and accumulation operation on the circular block data of the data to be processed each time received to determine the circular block result of the data to be processed each time received, including: the dot product unit array performs a multiplication and accumulation operation on the third circular block data and the fourth circular block data each time received to determine the circular block result.

[0025] According to one aspect of the present disclosure, an artificial intelligence chip is provided, comprising the processor as described above.

[0026] According to one aspect of the present disclosure, an electronic device is provided, comprising the artificial intelligence chip as described above.

[0027] In the disclosed embodiment, the control logic unit sequentially reads the cyclic block data of the data to be processed from the memory to the dot multiplication unit array according to the acquired control instruction, so that the dot multiplication unit array performs multiplication and accumulation operations on the cyclic block data of the data to be processed each time received, and determines the cyclic block result of the data to be processed each time received, and the control logic unit determines the logical operation result of the data to be processed according to the multiple cyclic block results acquired from the dot multiplication unit array. In this way, the reading and logical operation of the data to be processed (such as large-sized image data) can be converted into the reading and logical operation of multiple cyclic block data of the data to be processed (small-sized image data), which is conducive to processing larger-sized data without changing the processor hardware resources and reducing the pressure on the storage bandwidth.

[0028] It should be understood that the above general description and the following detailed description are exemplary and explanatory only and do not limit the present disclosure. Other features and aspects of the present disclosure will become clear from the following detailed description of exemplary embodiments with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] The drawings herein are incorporated into the specification and constitute a part of the specification. These drawings illustrate embodiments consistent with the present disclosure and are used to illustrate the technical solutions of the present disclosure together with the specification.

[0030] Figure 1 A schematic diagram showing a processor according to an embodiment of the present disclosure.

[0031] Figure 2 A flow chart of a data processing method according to an embodiment of the present disclosure is shown.

[0032] Figure 3 A schematic diagram showing a cyclic blocking sequence according to an embodiment of the present disclosure.

[0033] Figure 4 A schematic diagram showing a situation in which repeated reading of cyclically divided block data occurs according to an embodiment of the present disclosure.

[0034] Figure 5 A schematic diagram showing the third cycle blocking data of the convolution kernel according to an embodiment of the present disclosure.

[0035] Figure 6 A schematic diagram showing the fourth cycle blocking data of the convolution kernel according to an embodiment of the present disclosure.

[0036] Figure 7 A schematic diagram of a reuse cache according to an embodiment of the present disclosure is shown.

[0037] Figure 8 A schematic diagram showing another reuse cache according to an embodiment of the present disclosure.

[0038] Fig. 9 A block diagram of an electronic device according to an embodiment of the present disclosure is shown. DETAILED DESCRIPTION

[0039] Various exemplary embodiments, features and aspects of the present disclosure will be described in detail below with reference to the accompanying drawings. The same reference numerals in the accompanying drawings represent elements with the same or similar functions. Although various aspects of the embodiments are shown in the accompanying drawings, the drawings are not necessarily drawn to scale unless otherwise specified.

[0040] The word “exemplary” is used exclusively herein to mean “serving as an example, example, or illustration.” Any embodiment described herein as “exemplary” is not necessarily to be construed as preferred or advantageous over other embodiments.

[0041] The term "and / or" herein is only a description of the association relationship of the associated objects, indicating that there may be three relationships. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, the term "at least one" herein represents any combination of at least two of any one or more of a plurality of. For example, including at least one of A, B, and C can represent including any one or more elements selected from the set consisting of A, B, and C.

[0042] In addition, in order to better illustrate the present disclosure, numerous specific details are given in the following specific embodiments. It should be understood by those skilled in the art that the present disclosure can also be implemented without certain specific details. In some examples, methods, means, components and circuits well known to those skilled in the art are not described in detail in order to highlight the subject matter of the present disclosure.

[0043] Figure 1 FIG. 1 is a schematic diagram showing a processor according to an embodiment of the present disclosure. Figure 1 As shown, the processor includes a control logic unit and a dot multiplication unit array, wherein the dot multiplication unit array includes a plurality of dot multiplication units for performing product-addition operations.

[0044] The dot multiplication unit may also include an operator to complete a specified operation, such as calculating multiple multiplications and accumulating them. The dot multiplication unit may include a multiplier, an adder, etc. The specific structures in each dot multiplication unit may be the same or different, and the present disclosure does not limit this. The dot multiplication unit may also include other types of operators to adapt to various different operation processes, and the present disclosure does not limit the number and type of operators included in the dot multiplication unit.

[0045] The control logic unit can connect each dot multiplication unit in the dot multiplication unit array. In some embodiments, the control logic unit can number each dot multiplication unit in the form of a two-dimensional matrix or a multi-dimensional matrix so that multiple dot multiplication units can be logically arranged in the form of a two-dimensional matrix or a multi-dimensional matrix to better adapt to the logical operations of the matrix.

[0046] like Figure 1 As shown, the memory corresponding to the processor may include a local memory arranged outside the processor, and the control logic unit in the processor can be connected to the local memory. The control logic unit can be used for address calculation to load data from the memory to the dot multiplication unit array and control the dot multiplication unit array to process the data to be processed.

[0047] In one possible implementation, the local memory may be an on-chip cache, and the control logic unit may load the executable program and the data to be processed (for example, the input matrix) on the off-chip flash memory into the above-mentioned local memory (on-chip cache), and then perform subsequent logical operations on the data to be processed.

[0048] In one possible implementation, the local memory may store data to be processed and an executable program. The executable program may include control instructions. The processor executes the control instructions to implement logical operations on the data to be processed, such as matrix multiplication operations, convolution operations, and other operations related to multiplication and accumulation of the data to be processed.

[0049] In order to reduce the reading of repeated data, reduce the power consumption of the processor, and increase the communication bandwidth of the processor, the memory may also include a reuse buffer set inside the processor, and the reuse buffer may be set inside the dot product unit array ( Figure 1 Not shown), the reuse buffer can also be set outside the dot product unit array (see Figure 1), wherein, compared with setting the reuse cache outside the dot multiplication unit array, setting the reuse cache inside the dot multiplication unit array can have a higher data reading efficiency. A loader, a decoder, etc. are set in the control logic unit, wherein the loader can be used to load the data to be processed or part of the data to be processed in the local memory into the reuse cache in the processor, and the decoder can decode the control instruction for accessing the data in the executable program according to the change of the storage address of the data to be processed after loading. For example, for the control instruction for accessing data X in the local memory, since the data X is cached in the reuse cache, the address of the data X stored in the reuse cache can be obtained by decoding. The decoder can convert the control instruction for accessing data X in the local memory into the control instruction for accessing data X in the reuse cache, which is beneficial for the subsequent control logic unit to directly send the data cached in the reuse cache to the dot multiplication unit, and the dot multiplication unit performs multiplication and accumulation operations on it. Among them, the control logic unit can also directly load data from the off-chip memory to the reuse cache, and the present disclosure does not limit this.

[0050] In a possible implementation, in order to facilitate the processor to process large-scale data, such as Figure 1 As shown, a corresponding register file (Register File) can also be set for the processor to read, store, process and transmit data without combining with the memory. In this way, when the processor processes a large amount of data, the instruction execution time can be greatly reduced. Among them, the register file includes such as memory address registers, memory data registers, instruction registers, operation code word registers, accumulators, flag registers, etc., and the present disclosure does not limit this. In the example, the control logic unit can be used for address calculation so as to transfer data between the dot multiplication unit array and the register file.

[0051] In one possible implementation, the processor of the embodiment of the present disclosure may be a newly designed one, or may be obtained by improving an existing processor chip. The types of processor chips may include, but are not limited to: a central processing unit (CPU), a graphics processing unit (GPU), a general-purpose graphics processing unit (GPGPU), a digital signal processor (DSP), an application specific integrated circuit (ASIC), a tensor processor (TPU), a field programmable gate array (FPGA) or other programmable logic devices, and may also include a microprocessor or other conventional processor processor.

[0052] Figure 2 FIG. 1 is a flow chart showing a data processing method according to an embodiment of the present disclosure. Figure 2 As shown, the data processing method is applied to a processor, and the data processing method may include the following steps:

[0053] In step S11, the control logic unit reads the cyclic block data of the data to be processed from the memory to the dot multiplication unit array in sequence according to the acquired control instructions, and the control instructions are used to indicate the access information of the data to be processed and the type of logical operation of the data to be processed, for example, matrix multiplication operation, convolution operation and other multiplication-accumulation related operations of the data to be processed.

[0054] In one possible implementation, the control logic unit can obtain the address information of the data to be processed according to the control instruction, and determine the address information of each cyclic block data of the data to be processed in sequence according to the address information of the data to be processed, and read the cyclic block data of the data to be processed from the memory to the dot product unit array.

[0055] Optionally, the memory may include a local memory arranged outside the processor, and the control logic unit may read the cyclic block data of the data to be processed from the local memory to the dot product unit array in sequence according to the acquired control instruction.

[0056] Optionally, since the cyclic block data of the data to be processed may be read repeatedly, the memory may include, in addition to the local memory disposed outside the processor, a reuse cache disposed within the processor for caching the cyclic block data. The control logic unit reads data from the reuse cache faster than the speed of reading data from the local memory. In this way, for the cyclic block data that needs to be read repeatedly in the data to be processed, in response to the control logic unit transmitting the cyclic block data that needs to be read repeatedly obtained from the local memory to the dot multiplication unit array, the control logic unit may also synchronously store the cyclic block data that needs to be read repeatedly into the reuse cache disposed within the processor, so that the control logic unit may subsequently read the cyclic block data from the reuse cache to the dot multiplication unit array.

[0057] In a possible implementation, the data to be processed includes feature data in a deep learning task, and the feature data includes at least one of image feature data, speech feature data, and text feature data.

[0058] For example, in a scenario where a deep neural network is used to perform face recognition on a target object, the data to be processed may be image feature data, and the image feature data of the target object (e.g., a face feature map) may be stored in a memory; in a scenario where a deep neural network is used to perform speech recognition on a target object, the data to be processed may be speech feature data, and the speech feature data of the target object may be stored in a memory; in a scenario where a deep neural network is used to perform text recognition on a target document, the data to be processed may be text feature data, and the text feature data of the target document may be stored in a memory; the embodiments of the present application do not impose any restrictions on the type of data to be processed.

[0059] In a possible implementation, the control logic unit can obtain data structure information of the data to be processed and address information of the data to be processed through control instructions; wherein the data structure information of the data to be processed includes, for example, the dimension of the data to be processed, the size of the data to be processed, the data type of the elements in the data to be processed (for example, integer type, single-precision floating point type, double-precision floating point type, character type, etc.), and other information used to describe the data to be processed; the address information of the data to be processed includes, for example, the base address (Base Address) of the data to be processed in the memory, the addressing space, and other address-related information.

[0060] In step S12, the dot product unit array performs a multiplication and accumulation operation on the cyclic block data of the to-be-processed data received each time, and determines the cyclic block result of the to-be-processed data received each time.

[0061] Exemplarily, the dot multiplication unit array may include tile_M×tile_N dot multiplication units, each of which may calculate tile_K multiplications and accumulate them, wherein tile_M, tile_N, and tile_K are positive integers. The present disclosure does not impose any restrictions on the specific values ​​of tile_M, tile_N, and tile_K, and they may be set according to actual application scenarios.

[0062] Among them, the dot product unit array can realize matrix multiplication or convolution operation by performing multiplication and accumulation operation on the circular block data of the received data to be processed each time. For matrix multiplication operation, the dot product unit array can complete the multiplication operation of a matrix of size tile_M×tile_K and a matrix of size tile_K×tile_N. For convolution operation, the dot product unit array can process tile_M elements to be convolved and tile_N convolution kernel elements at a time, where the number of channels of tile_M elements to be convolved and tile_N convolution kernel elements is tile_K.

[0063] In step S13, the control logic unit determines the logic operation result of the data to be processed according to the multiple loop block results obtained from the dot product unit array.

[0064] Exemplarily, the control logic unit may perform concatenation and / or addition processing on the obtained multiple cyclic block results to determine the logical operation result of the data to be processed. The control logic unit may perform concatenation and / or addition processing on the cyclic block data received last time each time the cyclic block data is received; the control logic unit may also receive all the cyclic block data and then perform concatenation and / or addition processing on all the cyclic block data, which is not limited in the present disclosure.

[0065] In the data processing method of the embodiment of the present disclosure, the control logic unit reads the cyclic block data of the data to be processed from the memory to the dot multiplication unit array in sequence according to the control instruction obtained for indicating the logical operation of the data to be processed, so that the dot multiplication unit array performs multiplication and accumulation operations on the cyclic block data of the data to be processed each time received, and determines the cyclic block result of the data to be processed each time received, and the control logic unit determines the logical operation result of the data to be processed according to the multiple cyclic block results obtained from the dot multiplication unit array. In this way, the reading and logical operation of the data to be processed (such as large-sized image data) can be converted into the reading and logical operation of multiple cyclic block data (small-sized image data) of the data to be processed, which is conducive to processing larger-sized data without changing the processor hardware resources and reducing the pressure on the storage bandwidth.

[0066] The data processing method of the embodiment of the present disclosure is exemplarily described below.

[0067] In step S11, the control logic unit reads the cyclic block data of the data to be processed from the memory in sequence to the dot product unit array according to the acquired control instruction.

[0068] The memory may include a local memory disposed outside the processor; or, in addition to the local memory disposed outside the processor, the memory may also include a reuse cache disposed within the processor for caching cyclic block data, and the embodiments of the present disclosure do not impose specific restrictions on this.

[0069] In a possible implementation, step S11 may include: the control logic unit parses the access information of the data to be processed and the logical operation type of the data to be processed according to the acquired control instruction; determines the cyclic blocking order of the data to be processed according to the logical operation type of the data to be processed; and reads the cyclic blocking data of the data to be processed from the memory in sequence to the dot multiplication unit array according to the access information and the preset size and the cyclic blocking order of the data to be processed, wherein the size of the cyclic blocking data is less than or equal to the preset size, and the preset size is determined by the number of idle dot multiplication units in the dot multiplication unit array.

[0070] Exemplarily, assuming that the control instruction is used to indicate a matrix multiplication operation of the data to be processed, the control logic unit parses the acquired control instruction, and can parse out the access information of the data to be processed, as well as the logical operation type of the data to be processed. The access information includes, for example, size information, address information, and layout information of the data to be processed. Among them, the layout information can be a row-major sequence (RowMajor) or a column-major sequence (ColMajor), and the row-major sequence indicates that the elements of the same row of the data to be processed are adjacent in the memory, and the column-major sequence indicates that the elements of the same column of the data to be processed are adjacent in the memory. For example, if the logical operation type of the data to be processed is a matrix multiplication operation of a first matrix and a second matrix, the access information may include size information, address information (such as a starting address), layout information, etc. of the first matrix and the second matrix.

[0071] The control logic unit can determine the loop block order of the data to be processed according to the logic operation type of the data to be processed; for example, if the logic operation type of the data to be processed is a matrix multiplication operation of a first matrix and a second matrix, the control logic unit can select a pre-stored loop block order that matches the matrix multiplication operation.

[0072] The control logic unit can determine the size of the data to be processed based on the access information of the data to be processed obtained by parsing, and can determine whether it is necessary to perform cyclic block processing on the data to be processed based on the size of the data to be processed. If the size of the data to be processed is less than or equal to the preset size, it means that the number of idle dot multiplication units in the dot multiplication unit array can meet the demand for processing the data to be processed, and the control logic unit can directly read all the data to be processed from the memory to the dot multiplication unit array, so that the dot multiplication unit array performs multiplication and addition operations on the data to be processed.

[0073] If the size of the data to be processed is larger than the preset size, it means that the number of idle dot multiplication units in the dot multiplication unit array cannot meet the demand for processing the data to be processed. The control logic unit can determine the cyclic blocking order of the data to be processed according to the logic operation type of the data to be processed; and determine the cyclic blocking data of the data to be processed according to the access information and the preset size of the data to be processed, and then read the cyclic blocking data of the data to be processed from the memory to the dot multiplication unit array in sequence according to the cyclic blocking order of the data to be processed, and realize the multiplication and addition operation of the data to be processed by the dot multiplication unit array on multiple cyclic blocking data.

[0074] For example, there are tile_M×tile_N idle dot multiplication units in the dot multiplication unit array, and each dot multiplication unit can calculate tile_K multiplications and accumulate them. For matrix multiplication operations, the dot multiplication unit array can complete the multiplication operation of a matrix of size tile_M×tile_K and a matrix of size tile_K×tile_N. For matrices larger than tile_M, tile_N, and tile_K, the control logic unit is responsible for loop control, and reads the loop block data of the size from the data to be processed stored in the memory to the dot multiplication unit array in sequence according to the loop block order.

[0075] Exemplarily, assuming that the control instruction is used to indicate the convolution operation of the data to be processed, the control logic unit parses the acquired control instruction and can parse out the access information of the data to be processed, the access information includes, for example, the size information of the data to be convolved and the convolution kernel, address information, convolution description information (for example, including the stride of the convolution kernel, the padding amount), and the dimension information of the convolution output result. For example, if the logical operation type of the data to be processed is the convolution operation of the data to be convolved and the convolution kernel, the access information may include the convolution description information (for example, including the stride of the convolution kernel, the padding amount), the dimension information of the convolution output result, the size information of the data to be convolved and the convolution kernel, address information, layout information, etc.

[0076] The control logic unit can determine the circular blocking order of the data to be processed according to the logical operation type of the data to be processed; for example, if the logical operation type of the data to be processed is a convolution operation of the data to be convolved and a convolution kernel, the control logic unit can select a pre-stored circular blocking order that matches the convolution operation.

[0077] The control logic unit can determine the size of the data to be convolved and the convolution kernel according to the access information of the data to be processed, and can judge whether it is necessary to perform cyclic block processing on the data to be convolved and the convolution kernel according to the size of the data to be convolved and the convolution kernel. If the size of the data to be convolved and the convolution kernel is less than or equal to the preset size, it means that the number of idle point multiplication units in the point multiplication unit array can meet the needs of convolution operation of the data to be convolved and the convolution kernel, and the control logic unit can directly read the data to be convolved and the convolution kernel from the memory to the point multiplication unit array, so that the point multiplication unit array performs multiplication and addition operation on them.

[0078] If the size of any data in the data to be convolved or the convolution kernel is larger than the preset size, it means that the number of idle dot multiplication units in the dot multiplication unit array cannot meet the demand for convolution operation of the data to be convolved and the convolution kernel. The control logic unit can determine the circular blocking order of the data to be processed according to the logic operation type of the data to be processed; and determine the circular blocking data from the data to be convolved and / or the convolution kernel that is larger than the preset size according to the access information and the preset size of the data to be processed, and then read the corresponding circular blocking data from the memory to the dot multiplication unit array in sequence according to the circular blocking order of the data to be processed, and realize the convolution operation of the data to be convolved and the convolution kernel through the multiplication and addition operation of multiple circular blocking data by the dot multiplication unit array.

[0079] The size of the cyclic block data is less than or equal to a preset size, and the present disclosure does not impose any specific limitation on the specific size of the cyclic block data.

[0080] For example, there are tile_M×tile_N idle dot multiplication units in the dot multiplication unit array, and each dot multiplication unit calculates tile_K multiplications and accumulates them. For convolution operations, the dot multiplication unit array can process tile_M elements to be convolved and tile_N convolution kernel elements at a time, where the number of channels of tile_M elements to be convolved and tile_N convolution kernel elements is tile_K. For data to be processed that is larger than tile_M, tile_N, and tile_K in size, the control logic unit is responsible for loop control, and reads the loop block data that meets the size from the data to be processed stored in the memory to the dot multiplication unit array in sequence according to the loop block order.

[0081] In this way, it is helpful to convert large-sized data to be processed into cyclic block data of a size that meets the required size of the dot multiplication unit array calculation structure, thereby improving the computing performance of the processor.

[0082] In a possible implementation, the access information of the data to be processed includes access information of a first matrix and a second matrix, the number of columns of the first matrix is ​​the same as the number of rows of the second matrix, the logical operation includes a matrix multiplication operation of the first matrix and the second matrix, and the loop block order determined by the matrix multiplication operation type includes: a first outer loop order in the direction of the number of rows of the first matrix, a second outer loop order in the direction of the number of columns of the second matrix, and an inner loop order in the direction of the number of columns of the first matrix and the direction of the number of rows of the second matrix.

[0083] Due to the limited hardware resources of the processor, when the amount of data accessed by the processor is too large, the control logic unit cannot load the required data into the point product unit array at one time. By setting the loop block order, it is beneficial to process larger size data while keeping the processor hardware resources unchanged, reducing the pressure on the storage bandwidth.

[0084] For example, Figure 3 A schematic diagram showing the order of loop blocks according to an embodiment of the present disclosure is shown. Figure 3 As shown, the control instruction received by the control logic unit may be to perform a matrix multiplication operation on a first matrix A of size M×K and a second matrix B of size K×N, where the number of columns K of the first matrix A is the same as the number of rows K of the second matrix B.

[0085] Among them, C represents the matrix multiplication result of the first matrix A and the second matrix B, tile_A represents the first loop block data of the first matrix A, whose size is tile_M×tile_K, tile_B represents the second loop block data of the second matrix B, whose size is tile_K×tile_N, and tile_C represents the matrix multiplication result of the first loop block data tile_A and the second loop block data tile_B.

[0086] The loop block sequence can be a multi-layer loop (for example, including three layers of loop nesting), and the first outer loop sequence (such as Figure 3 The loop in the M direction), the second outer loop order in the column number direction of the second matrix B (such as Figure 3 The inner loop sequence is based on the number of columns of the first matrix A and the number of rows of the second matrix B (e.g. Figure 3 K direction cycle).

[0087] Thus, in step S11, the control logic unit can read the first circular block data tile_A of the first matrix A and the second circular block data tile_B of the second matrix B from the memory in sequence to the dot multiplication unit array in accordance with the circular block order, so that in step S12, the dot multiplication unit array performs a multiplication and accumulation operation on the first circular block data tile_A and the second circular block data tile_B obtained each time, and obtains the circular block result tile_C corresponding to each multiplication and accumulation operation. And in step S13, the control logic unit performs a cumulative operation on the multiple circular block results tile_C obtained in sequence from the dot multiplication unit array in any round of the circular block order to obtain a logical operator result; the pseudo code is as follows:

[0088]

[0089] Among them, the first outer cycle (such as Figure 3 The loop in the M direction) can be cycled M / tile_M times, and the second outer loop (such as Figure 3 The loop in the N direction) can be looped N / tile_N times, and the inner loop (such as Figure 3 The loop in the K direction can be looped K / tile_K times, so that the first matrix A can be divided into M / tile_M rows and K / tile_K columns, and the second matrix B can be divided into K / tile_K rows and N / tile_N columns. Each row and column in the first matrix A corresponds to a first loop block data tile_A, tile_A mk represents the first cyclic block data tile_A in the mth row and kth column of the first matrix A, and each row and column in the second matrix B corresponds to a second cyclic block data tile_B, tile_B kn Represents the second cyclic block data tile_B at the k-th row and n-th column in the second matrix B.

[0090] tile_A mk ×tile_B kn Represents the first loop block data tile_A in the mth row and kth column of the first matrix A mk , and the second loop block data tile_B in the kth row and nth column of the second matrix B kn The matrix multiplication operation is to multiply the cell array by the first loop tile_A mk The second loop blocks the data tile_B kn Perform multiplication and accumulation operations to obtain the loop block results corresponding to the mth, nth, and kth multiplication and accumulation operations tile_C = tile_A mk ×tile_B kn .

[0091] The loop block sequence may include m×n rounds of loops, and any first outer loop (such as Figure 3 The mth cycle in the M direction) and the second outer cycle (such as Figure 3 The nth time of the N-direction loop in the N direction) corresponds to all K / tile_K inner loops (such as Figure 3 The cycle in the K direction is a cycle, referred to as the mth and nth cycles. The control logic unit can obtain multiple cycle block results tile_A from the dot product unit array in any cycle block sequence. mk ×tile_B kn Perform accumulation operations to obtain the results of the logical operators in the mth and nth rounds.

[0092] Optionally, the loop block order determined by the matrix multiplication operation type may also include: a first outer loop order in the direction of the number of columns of the second matrix, a second outer loop order in the direction of the number of rows of the first matrix, and an inner loop order in the direction of the number of columns of the first matrix and the number of rows of the second matrix. For example, the loop block order may be in the direction of the first outer loop order in the direction of the number of columns of the second matrix B (such as Figure 3 The second outer loop sequence of the first matrix A in the N-direction loop sequence (such as Figure 3 The inner loop sequence is based on the number of columns of the first matrix A and the number of rows of the second matrix B (e.g. Figure 3 K direction cycle).

[0093] Thus, in step S11, the control logic unit can read the first circular block data tile_A of the first matrix A and the second circular block data tile_B of the second matrix B from the memory in sequence to the dot multiplication unit array in accordance with the circular block order, so that in step S12, the dot multiplication unit array performs a multiplication and accumulation operation on the first circular block data tile_A and the second circular block data tile_B obtained each time, and obtains the circular block result tile_C corresponding to each multiplication and accumulation operation. And in step S13, the control logic unit performs a cumulative operation on the multiple circular block results tile_C obtained in sequence from the dot multiplication unit array in any round of the circular block order to obtain a logical operator result; the pseudo code is as follows:

[0094]

[0095] The specific process can be found above and will not be repeated here.

[0096] In this way, users can implement the loop block sequence through a few simple loop codes, reducing the complexity of software user implementation.

[0097] In a possible implementation, the last loop block result can be stored in an accumulator cache (for example, it can be part of a reused cache). When the current loop block data is obtained from the dot product unit array, the last loop block result can be read from the accumulator cache and added to the current loop block data, and the addition result can be used to update the current loop block data as the next loop block data. By setting an accumulator cache for storing intermediate data (for example, loop block data) of a memory loop (for example, a certain loop in the K direction), it is helpful to reduce the number of accesses to the memory and reduce the pressure on the memory bandwidth. In this way, multiple loop block results are accumulated based on the accumulator cache to obtain a logical operator result. Then, the inner loop ends (for example, the loop in the K direction ends), and the control logic unit writes the logical operator result into a register stack; the control logic unit determines the logical operation result based on the multiple logical operator results obtained from the register stack. For example, the control logic unit can obtain the logical operator result C after each round of inner loops of all K / tile_K times. mn Write to the register file; the control logic unit obtains the result C of the total M / tile_M×N / tile_N rounds of logical operators from the register file. mn The concatenation is performed to obtain the logic operation result of the first matrix A and the first matrix B. By setting the register file, it is helpful to further reduce the number of accesses to the local memory set outside the processor.

[0098] Optionally, if the memory is a local memory set outside the processor, the control logic unit reads the cyclic block data of the to-be-processed data from the local memory to the dot product unit array in sequence according to the cyclic block order, and the cyclic block data may be repeatedly read. Figure 4 A schematic diagram showing a situation where repeated reading of cyclic block data occurs according to an embodiment of the present disclosure. Figure 4 As shown, it is assumed that the first matrix A can be divided into 3×2 blocks, that is, the first loop block data tile_A 11 , the first cycle block data tile_A 12 , the first cycle block data tile_A 21 , the first cycle block data tile_A 22 , the first cycle block data tile_A 31 , the first cycle block data tile_A 32 .

[0099] Assume that the second matrix B can be divided into 2×2 blocks, that is: the second loop block data tile_B 11 , the second loop block data tile_B 12, the second loop block data tile_B 21 , the second loop block data tile_B 22 .

[0100] According to the loop block order, in the first outer loop (for example Figure 4 The first loop in the row direction of the first matrix A) to the first time, the second outer loop (for example Figure 4 The second matrix B column number direction loop) to the first time, after a round of inner loop sequence (for example Figure 4 The inner loop is a loop of the first matrix A in the column direction and the second matrix B in the row direction. One inner loop may include 2 loops. The result of each inner loop is accumulated. After 2 inner loops, the logical operator result C can be obtained. 11 =tile_A 11 ×tile_B 11 +tile_A 12 ×tile_B 21 .

[0101] The first outer loop is looped for the first time, the second outer loop is looped for the second time, and after a round of inner loop sequence, the results of each inner loop are accumulated. After two inner loops, the logical operator result C can be obtained. 12 =tile_A 11 ×tile_B 12 +tile_A 12 ×tile_B 22 Among them, the first cycle block data tile_A 11 And the first loop tile data tile_A 12 Will be reused, the control logic unit will repeatedly read the first cycle block data tile_A from the local memory 11 And the first loop tile data tile_A 12 .

[0102] The first outer loop is looped for the second time, the second outer loop is looped for the first time, and after a round of inner loop sequence, the results of each inner loop are accumulated. After two inner loops, the logical operator result C can be obtained. 21 =tile_A 21 ×tile_B 11 +tile_A 22 ×tile_B 21 Among them, the second cycle block data tile_B 11 And the second loop blocks the data tile_B 21 Will be reused, the control logic unit will repeatedly read the second cycle block data tile_B from the local memory 11And the second loop blocks the data tile_B 21 .

[0103] The first outer loop is looped for the second time, the second outer loop is looped for the second time, and after a round of inner loop sequence, the results of each inner loop are accumulated. After two inner loops, the logical operator result C can be obtained. 22 =tile_A 21 ×tile_B 12 +tile_A 22 ×tile_B 22 Among them, the first cycle block data tile_A 21 And the first loop tile data tile_A 22 Will be reused, the second loop block data tile_B 12 And the second loop blocks the data tile_B 22 Will be reused, the control logic unit will repeatedly read the first loop block data tile_A21, the first loop block data tile_A22, the second loop block data tile_B12, and the second loop block data tile_B22 from the local memory.

[0104] The first outer loop is looped for the third time, the second outer loop is looped for the first time, and after a round of inner loop sequence, the results of each inner loop are accumulated. After two inner loops, the logical operator result C can be obtained. 31 =tile_A 31 ×tile_B 11 +tile_A 32 ×tile_B 21 Among them, the second cycle block data tile_B 11 And the second loop blocks the data tile_B 21 Will be reused, the control logic unit will repeatedly read the second cycle block data tile_B from the local memory 11 And the second loop blocks the data tile_B 21 .

[0105] The first outer loop is looped for the third time, the second outer loop is looped for the second time, and after a round of inner loop sequence, the results of each inner loop are accumulated. After two inner loops, the logical operator result C can be obtained. 32 =tile_A 31 ×tile_B 12 +tile_A 32 ×tile_B 22 Among them, the first cycle block data tile_A 31 And the first loop tile data tile_A 32Will be reused, the second loop block data tile_B 12 And the second loop blocks the data tile_B 22 Will be reused, the control logic unit will repeatedly read the first cycle block data tile_A from the local memory 31 , the first cycle block data tile_A 32 , the second loop block data tile_B 12 , the second loop block data tile_B 22 .

[0106] Optionally, in order to further save the memory access bandwidth of the processor, reduce repeated access to the same loop block data in the local memory outside the processor, and improve the computing performance of the processor, in a possible implementation, the memory includes a local memory arranged outside the processor, and a reuse cache arranged in the processor for caching loop block data, and the loop block data of the data to be processed are read from the memory in sequence to the dot product unit array according to the loop block order of the data to be processed, including: the control logic unit stores the first loop block data of the first matrix read from the local memory into the reuse cache according to the first outer loop order and the inner loop order, so that the control logic unit reuses the first loop block data stored in the reuse cache according to the second outer loop order; or, the control logic unit stores the second loop block data of the second matrix read from the local memory into the reuse cache according to the second outer loop order and the inner loop order, so that the control logic unit reuses the second loop block data stored in the reuse cache according to the first outer loop order.

[0107] For example, Figure 4 As shown, according to the loop block order, in the first outer loop (for example Figure 4 The first loop in the row direction of the first matrix A) to the first time, the second outer loop (for example Figure 4 The second matrix B column number direction loop) to the first time, the inner loop (for example Figure 4 The control logic unit can read the first cycle block data tile_A of the first matrix A from the local memory. 11 , the second loop block data tile_B of the second matrix B 11 , get the loop block result tile_A 11 ×tile_B 11 , the first loop of the first matrix A can be divided into tile_A11 The second loop of the second matrix B blocks the data tile_B 11 Store in reuse cache.

[0108] In this way, when the first outer layer loops to the first time, the second outer layer loops to the second time, and the inner layer loops to the first time (for example, M=1, N=2, K=1), the first loop block data tile_A can be read from the reuse cache 11 , read the second loop block data tile_B from the local memory 12 , get the loop block result tile_A 11 ×tile_B 12 .

[0109] When the first outer layer loops to the second time, the second outer layer loops to the first time, and the inner layer loops to the first time (for example, M=2, N=1, K=1), the first loop block data tile_A can be read from the local memory 21 , read the second loop tile data tile_B from the reuse cache 11 , get the loop block result tile_A 21 ×tile_B 11 .

[0110] When the first outer layer loops to the third time, the second outer layer loops to the first time, and the inner layer loops to the first time (for example, M=3, N=1, K=1), the first loop block data tile_A can be read from the local memory. 31 , read the second loop tile data tile_B from the reuse cache 11 , get the loop block result tile_A 31 ×tile_B 11 .

[0111] Similarly, when the first outer layer loops to the first time, the second outer layer loops to the first time, and the inner layer loops to the second time (for example, M=1, N=1, K=2), the control logic unit can read the first loop block data tile_A of the first matrix A from the local memory. 12 , the second loop block data tile_B of the second matrix B 21 , get the loop block result tile_A 12 ×tile_B 21 , the first loop of the first matrix A can be divided into tile_A 12 The second loop of the second matrix B blocks the data tile_B 21 The data is stored in the reuse cache.

[0112] In this way, when the first outer layer loops to the first time, the second outer layer loops to the second time, and the inner layer loops to the second time (for example, M=1, N=2, K=2), the first loop block data tile_A can be read from the reuse cache 12 , read the second loop block data tile_B from the local memory 22 , get the loop block result tile_A 12 ×tile_B 22 .

[0113] When the first outer layer loops to the second time, the second outer layer loops to the second time, and the inner layer loops to the second time (for example, M=2, N=2, K=2), the first loop block data tile_A can be read from the local memory. 22 , read the second loop tile data tile_B from the reuse cache 21 , get the loop block result tile_A 22 ×tile_B 21 .

[0114] When the first outer layer loops to the third time, the second outer layer loops to the second time, and the inner layer loops to the second time (for example, M=3, N=2, K=2), the first loop block data tile_A can be read from the local memory. 32 , read the second loop tile data tile_B from the reuse cache 21 , get the loop block result tile_A 32 ×tile_B 21 .

[0115] It should be understood that the first outer loop, the second outer loop, and the inner loop can be cycled to other times respectively, and can refer to the above, which will not be repeated here. How to reuse the first loop block data and the second loop block data based on the reuse cache in the multi-layer loop process can be set according to the actual application scenario, and the embodiment of the present disclosure does not limit this.

[0116] For example, in a scenario where hardware resources are relatively tight, in order to reduce the consumption of hardware resources, the reuse cache will not be set very large. For example, the size of the reuse cache can be set to cache a first loop block data and a second loop block data. Only one first loop block data and a second loop block data can be cached at the same time. Each time the order of the first outer loop or the second outer loop is switched, the loop block data can be read from the reuse cache again. For example, the control logic unit can respond to each update of the first outer loop sequence, read the first loop block data of the first matrix from the local memory and store it in the reuse cache, so that the control logic unit reuses the first loop block data stored in the reuse cache during the process of traversing the second outer loop sequence. Alternatively, the control logic unit can respond to each update of the second outer loop sequence, read the second loop block data of the second matrix from the local memory and store it in the reuse cache, so that the control logic unit reuses the second loop block data stored in the reuse cache during the process of traversing the first outer loop sequence.

[0117] By setting up a reuse cache inside the processor, it is helpful to reduce the number of accesses to the local memory outside the processor and reduce the pressure on the storage bandwidth.

[0118] In a possible implementation, the access information of the data to be processed includes access information of the data to be convolved and the convolution kernel, the logical operation includes the convolution operation of the data to be convolved and the convolution kernel, the circular block order determined by the convolution operation type includes the coordinate order of the elements in the convolution kernel, and the circular block data of the data to be processed are read from the memory in sequence to the dot product unit array according to the circular block order of the data to be processed, including: the control logic unit, according to the coordinate order of the elements in the convolution kernel, converts multiple elements with the same coordinates read from multiple convolution kernels in the memory each time to form third circular block data; and, according to the coordinates of the current element in each convolution kernel and the access information, reads multiple elements from the data to be convolved in the memory to form fourth circular block data; the control logic unit writes the third circular block data and the fourth circular block data into the dot product unit array.

[0119] The memory may include a local memory disposed outside the processor; or, in addition to the local memory disposed outside the processor, the memory may also include a reuse cache disposed within the processor for caching cyclic block data, and the embodiments of the present disclosure do not impose specific restrictions on this.

[0120] Exemplarily, the access information of the data to be convolved and the convolution kernel includes convolution description information, which can be used to characterize the dimension information of the data to be convolved, the dimension information of the convolution kernel, the dimension information of the convolution output result, layout information, etc. The control logic unit can read the convolution description information, and perform mapping operations on the convolution data and the convolution kernel according to the convolution description information. Each mapping operation can read the third loop block data and the fourth loop block data once, and then associate the third loop block data and the fourth loop block data with the dot product unit array and perform calculations. In this way, traversing each element of the convolution kernel and accumulating the results can obtain the final convolution output result.

[0121] In this way, the control logic unit can expand the matrix operation instructions to implement the convolution function according to the read convolution description information, and convert the convolution operation into matrix multiplication, making the processor more versatile.

[0122] Figure 5 FIG. 4 is a schematic diagram showing the third cycle block data of the convolution kernel according to an embodiment of the present disclosure. Figure 5 As shown, the memory stores a convolution kernel C×R×S×K, where C represents the channel dimension of the convolution kernel, R represents the height dimension of the convolution kernel, S represents the width dimension of the convolution kernel, and K represents the number dimension of the convolution kernel. Each convolution kernel may include R×S×C / tile_K convolution kernel elements, and each convolution kernel element occupies one unit of space in the height dimension R and the width dimension S respectively, and may occupy tile_K units of space in the channel dimension C.

[0123] The control logic unit may read the elements of the first row and first column of each convolution kernel in the channel dimension 0 to tile_K from the memory for the first time according to the coordinate order of the elements in the convolution kernel, for example, from the coordinate order of the convolution kernel elements (r=0, s=0, c=0) to (r=R-1, s=S-1, c=C / tile_K-1), and map the 8 (K=0-7) elements with the same coordinates (r=0, s=0, c=0) in the 8 (K=0-7) convolution kernels to the first third loop block data tile_B 000 ; The second time, read the elements of the first row and second column of each convolution kernel in the channel dimension 0 to tile_K from the memory, and map the 8 (K=0~7) elements with the same coordinates (r=0, s=1, c=0) in the 8 (K=0~7) convolution kernels to the second third loop block data tile_B 010; Similarly, the last row and column elements of each convolution kernel in the channel dimension (C-tile_K)~C are read from the memory for the last time, and the 8 (K=0~7) elements with the same coordinates (r=R-1, s=S-1, c=C / tile_K-1) read from the 8 (K=0~7) convolution kernels in the memory are mapped to the last third cycle block data tile_B (R-1)(S-1)(C / tile_K-1) .

[0124] like Figure 5 As shown, each third cycle block data tile_B rsc The size is tile_K×tile_N, where tile_N represents the number of convolution kernels and tile_K represents the size of each convolution kernel element in the channel dimension.

[0125] at the same time, Figure 6 FIG. 4 is a schematic diagram showing the fourth cycle block data of the convolution kernel according to an embodiment of the present disclosure. Figure 6 As shown, the memory stores C×H×W data to be convolved (for example, including an input image of size C×H×W), where C represents the channel dimension of the data to be convolved, H represents the height dimension of the data to be convolved, and W represents the width dimension of the kernel to be convolved. The data to be convolved may include C×H×W / tile_K elements to be convolved, each element to be convolved occupies one unit of space in the height dimension H and the width dimension W respectively, and may occupy tile_K units of space in the channel dimension C.

[0126] Since the dimension of the convolution output result is determined by the data to be convolved and the number of times the convolution kernel slides in the data to be convolved, based on this, the control logic unit can calculate the coordinates of the data to be convolved according to the coordinate order of the elements in the convolution kernel, according to the coordinates (r, s, c) of the current element in each convolution kernel, and the dimension information of the convolution output result included in the access information, and read tile_M elements from the data to be convolved in the memory according to the coordinates of the data to be convolved, and map them to the fourth loop block data. Among them, the number of tile_M elements can be determined by the dimension information of the convolution output result.

[0127] For example, the coordinates of the current element in the convolution kernel are (r=0, s=0, c=0), and the first row and first column element in the convolution output result corresponds to the first row and first column element in the data to be convolved, whose coordinates are (h=0, w=0, c=0). The element with coordinates (h=0, w=0, c=0) in the data to be convolved can be used as the first element of tile_M elements of the fourth circular block data. And so on, until tile_M elements are read from the data to be convolved in the memory and mapped to the fourth circular block data.

[0128] The control logic unit can write the third circular block data and the fourth circular block data corresponding to the coordinates of each element in the convolution kernel into the dot multiplication unit array, so that the dot multiplication unit array performs matrix multiplication operations on the third circular block data and the fourth circular block data. For details, please refer to the matrix multiplication operation above, which will not be repeated here.

[0129] The embodiments of the present disclosure can respectively determine the third loop block data of the convolution kernel and the fourth loop block data of the data to be convolved according to the coordinate order of the elements in the convolution kernel, without the need for im2col (for example, sliding the convolution kernel on the data to be convolved, then converting the data contained in each convolution kernel window into a column vector, and finally arranging them into a new matrix by column) to expand in the memory, thereby reducing the pressure on the storage bandwidth. Moreover, in this way, it is beneficial to reuse the matrix calculation structure to realize the convolution engine, so as to use the convolution description information to expand the matrix operation instructions and realize the convolution function.

[0130] In one possible implementation, the memory includes a local memory arranged outside the processor, and a reuse cache arranged in the processor for caching circular block data. The method also includes: in the process of calculating the fourth circular block data corresponding to the same row in the convolution output result, the control logic unit writes M elements of the fourth circular block number corresponding to the coordinates of the element in the rth row and sth column of the convolution kernel into the reuse cache according to the coordinate order of the elements in the convolution kernel, where M is a positive integer; the control logic unit reads M-1 elements from the reuse cache, reads 1 element from the data to be convolved in the local memory, and determines the fourth circular block data corresponding to the coordinates of the element in the rth row and s+1th column of the convolution kernel.

[0131] Figure 7 FIG. 2 is a schematic diagram showing a reuse cache according to an embodiment of the present disclosure. Figure 7 As shown, assuming that the step size of the convolution kernel is 1, the control logic unit can write the fourth loop block data [0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15] corresponding to the coordinates of the element in the r=1th row and s=0th column in the convolution kernel into the reuse cache.

[0132] For the fourth loop block data corresponding to the coordinates of the element in the r=1th row and s=1th column in the convolution kernel, the control logic unit only needs to read one more element from the data to be convolved in the local memory, and the other 15 elements can reuse [1,2,3,4,5,6,7,8,9,10,11,12,13,14,15] in the reuse cache to form the fourth loop block data [1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16]; wherein the control logic unit writes the extra read element 16 into the reuse cache, so that the reuse cache continues to store [0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16].

[0133] For the fourth circular block data corresponding to the coordinates of the element in the r=1th row and s=2nd column in the convolution kernel, the control logic unit only needs to read one more element from the data to be convolved in the local memory, and the other 15 elements can reuse [2,3,4,5,6,7,8,9,10,11,12,13,14,15,16] in the reuse cache to form the fourth circular block data [2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17]; wherein the control logic unit writes the extra read element 17 into the reuse cache to obtain [0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17].

[0134] By setting a reuse cache inside the processor, the input data of the convolution can be reused, further reducing the number of accesses to the local memory outside the processor and the bandwidth pressure on the local memory outside the processor.

[0135] In one possible implementation, in the process of calculating the fourth circular block data corresponding to adjacent rows in the convolution output result, in response to the control logic unit calculating the fourth circular block data of the current row of the convolution output result, M elements of the fourth circular block number corresponding to the coordinates of the element in the rth row and sth column in the convolution kernel are written into a reuse cache; in response to the control logic unit calculating the fourth circular block data of the next row of the current row of the convolution output result, M elements are read from the reuse cache to constitute the fourth circular block data corresponding to the coordinates of the element in the r-1th row and sth column in the convolution kernel.

[0136] Figure 8 FIG. 2 is a schematic diagram showing another reuse cache according to an embodiment of the present disclosure. Figure 8As shown in Figure 2, assuming that the stride of the convolution kernel is 1, the reuse buffer stores the data [0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17], where [0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15] is when calculating the first row of the convolution output (see Figure 8 The first row of the convolution result in oh_0), according to the second row of the convolution data (see Figure 8 ih_1), the fourth loop block data determined by the coordinates of the element in the r=1th row and s=0th column in the convolution kernel; [1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16] is when calculating the first row of the convolution output result (see Figure 8 The first row of the convolution result in oh_0), according to the second row of the convolution data (see Figure 8 ih_1), the fourth loop block data is determined by the coordinates of the element in the r=1th row and s=1th column of the convolution kernel, [2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17] is when calculating the first row of the convolution output result (see Figure 8 The first row of the convolution result in oh_0), according to the second row of the convolution data (see Figure 8 In ih_1), the fourth loop block data is determined by the coordinates of the element in the r=1th row and s=2th column in the convolution kernel.

[0137] It should be understood that the fourth loop block data [1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16] determined by the coordinates of the element in the r=1th row and the s=1th column in the convolution kernel can utilize the reuse cache to reuse the fourth loop block data [0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15] determined by the coordinates of the element in the r=1th row and the s=0th column in the convolution kernel; similarly , the fourth loop block data [2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17] determined by the coordinates of the element in the r=1th row and s=2nd column in the convolution kernel can use the reuse cache to reuse the fourth loop block data [1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16] determined by the coordinates of the element in the r=1th row and s=1th column in the convolution kernel. For details, please refer to Figure 7 The relevant description will not be repeated here.

[0138] Considering that two adjacent rows in the convolution output result may have data reuse, for example, when calculating the second row of the convolution output result (see Figure 8The second row of the convolution result in oh_1), the convolution kernel r = 0 row elements corresponding to the elements to be convolved (see Figure 8 The second row of data to be convolved (ih_1) and the previous row of the convolution output (see Figure 8 The first row of the convolution result oh_0) has the same convolution kernel r=1 as the convolution element (also Figure 8 The second row of data to be convolved (ih_1) can be reused to reduce the amount of data read from the local memory.

[0139] For example, when computing the second row of the convolution output (see Figure 8 The second row of the convolution result in oh_1), the element to be convolved corresponding to the element in the r=0th row and s=0th column of the convolution kernel (see Figure 8 The second row of data to be convolved (ih_1) and the previous row of the convolution output (see Figure 8 The first row of the convolution result in oh_0) has the same convolution element corresponding to the r=1 row and s=0 column (also Figure 8 The second row of data to be convolved in ih_1 can be reused to reuse the first row of the convolution output result (see Figure 8 When the first row of the convolution result is oh_0), the fourth cycle block data [0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15] determined by the r=1 row and s=0 column in the convolution kernel is used as the second row of the convolution output result (see Figure 8 The second row of the convolution result in oh_1) is the fourth cycle block data [0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15] determined by the elements in the r=0th row and s=0th column of the convolution kernel.

[0140] Similarly, we can use the reuse cache to reuse the first row of the convolution output (see Figure 8 When the first row of the convolution result is oh_0), the fourth cycle block data [1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16] determined by the r=1 row and s=1 column in the convolution kernel is used as the second row of the convolution output result (see Figure 8 The second row of the convolution result in oh_1), the fourth cycle block data [1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16] determined by the elements in the r=0th row and s=1th column of the convolution kernel. And, reuse the first row of the convolution output result (see Figure 8When the first row of the convolution result is oh_0), the fourth cycle block data [2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17] determined by the r=1 row and s=2 column in the convolution kernel is used as the second row of the convolution output result (see Figure 8 The second row of the convolution result in oh_1) is the fourth loop block data [2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17] determined by the elements in the r=0th row and s=2nd column of the convolution kernel.

[0141] Compared with the related art, the implementation of convolution operation requires the addition of the software im2col (for example, sliding the convolution kernel on the data to be convolved, and then converting the data contained in each convolution kernel window into a column vector, and finally arranging them into a new matrix by column) calculation logic, which will introduce additional overhead. For example, for a common 3x3 size convolution kernel, 9 times the amount of data is required, which has a greater pressure on the storage bandwidth. The embodiment of the present disclosure can determine the third loop block data of the convolution kernel and the fourth loop block data of the data to be convolved according to the coordinate order of the elements in the convolution kernel, respectively, and can reuse the matrix calculation structure to implement the convolution engine, without the need for im2col to be expanded in the memory, reducing the pressure on the storage bandwidth. And by setting a reuse cache in the processor, the convolution implements the reuse of input data, further reducing the number of accesses to the local memory outside the processor and the bandwidth pressure on the local memory outside the processor.

[0142] In a possible implementation, the control logic unit, according to the coordinate order of the elements in the convolution kernel, forms third circular block data with multiple elements with the same coordinates read from the multiple convolution kernels in the memory each time, including: the control logic unit reads multiple convolution kernels from the local memory to the reuse cache; according to the coordinate order of the elements in the convolution kernel, the control logic unit forms third circular block data with multiple elements with the same coordinates read from the multiple convolution kernels in the reuse cache each time.

[0143] Exemplarily, assuming that the local memory stores convolution kernels [S1, S2], [S3, S4], [S5, S6], the control logic unit can read the three convolution kernels [S1, S2], [S3, S4], [S5, S6] from the local memory to the reuse cache; in this way, the logic control unit reads multiple elements S1, S3, S5 at coordinate 1 from the reuse cache for the first time according to the coordinate order of the elements in the convolution kernels [S1, S2], [S3, S4], [S5, S6], to form the third loop block data [S1, S3, S5]; and reads multiple elements S2, S4, S6 at coordinate 2 from the reuse cache for the second time, to form the third loop block data [S2, S4, S6]. It should be understood that the present disclosure only takes the convolution kernels [S1, S2], [S3, S4], [S5, S6] as examples, and does not limit the size and number of the convolution kernels.

[0144] In this way, the convolution kernel has been read into the reuse cache inside the processor, and the convolution kernel stored in the reuse cache can be reused, avoiding repeated reading of the local memory outside the processor.

[0145] In a possible implementation, the dot product unit array performs a multiplication and accumulation operation on the cyclic block data of the to-be-processed data received each time to determine the cyclic block result of the to-be-processed data received each time, including: the dot product unit array performs a multiplication and accumulation operation on the third cyclic block data and the fourth cyclic block data received each time to determine the cyclic block result. For details, please refer to the above dot product unit array performing a multiplication and accumulation operation on the first cyclic block data and the second cyclic block data obtained each time, which will not be repeated here.

[0146] The subsequent control logic unit can perform accumulation operations on multiple loop block results obtained from the point multiplication unit array in sequence according to the loop block order, obtain the logical operator result, and write it into the register stack; the control logic unit can determine the convolution output result based on the multiple logical operator results obtained from the register stack.

[0147] In this way, it is helpful to reuse the matrix calculation structure to implement the convolution engine, which is helpful to reduce the complexity of software users' implementation later.

[0148] In the data processing method of the embodiment of the present disclosure, the control logic unit reads the cyclic block data of the data to be processed from the memory to the dot multiplication unit array in sequence according to the control instruction obtained for indicating the logical operation of the data to be processed, so that the dot multiplication unit array performs multiplication and accumulation operations on the cyclic block data of the data to be processed each time received, and determines the cyclic block result of the data to be processed each time received, and the control logic unit determines the logical operation result of the data to be processed according to the multiple cyclic block results obtained from the dot multiplication unit array. In this way, the reading and logical operation of the data to be processed (such as large-sized image data) can be converted into the reading and logical operation of multiple cyclic block data (small-sized image data) of the data to be processed, which is conducive to processing larger-sized data without changing the processor hardware resources and reducing the pressure on the storage bandwidth.

[0149] It can be understood that the above-mentioned various method embodiments mentioned in the present disclosure can be combined with each other to form a combined embodiment without violating the principle logic. Due to space limitations, the present disclosure will not repeat them. It can be understood by those skilled in the art that in the above-mentioned method of the specific implementation method, the specific execution order of each step should be determined according to its function and possible internal logic.

[0150] In addition, the present disclosure also provides a processor, an electronic device, a computer-readable storage medium, and a program, all of which can be used to implement any data processing method provided by the present disclosure. The corresponding technical solutions and descriptions are referred to the corresponding records in the method part and will not be repeated here.

[0151] In one possible implementation, Figure 1 The processor shown includes a control logic unit and a dot multiplication unit array, wherein the dot multiplication unit array includes a plurality of dot multiplication units for performing multiplication and accumulation operations, and the processor is used for: the control logic unit sequentially reads the cyclic block data of the data to be processed from the memory to the dot multiplication unit array according to the control instructions obtained, and the control instructions are used to indicate the logical operation of the data to be processed; the dot multiplication unit array performs a multiplication and accumulation operation on the cyclic block data of the data to be processed received each time, and determines the cyclic block result of the data to be processed received each time; the control logic unit determines the logical operation result of the data to be processed according to the plurality of cyclic block results obtained from the dot multiplication unit array.

[0152] In a possible implementation, the control logic unit reads the cyclic block data of the data to be processed from the memory in sequence to the dot multiplication unit array according to the acquired control instruction, including: the control logic unit parses the access information of the data to be processed and the logical operation type of the data to be processed according to the acquired control instruction; determines the cyclic block order of the data to be processed according to the logical operation type of the data to be processed; and reads the cyclic block data of the data to be processed from the memory in sequence to the dot multiplication unit array according to the access information and the preset size and the cyclic block order of the data to be processed, wherein the size of the cyclic block data is less than or equal to the preset size, and the preset size is determined by the number of dot multiplication units in the dot multiplication unit array that are in an idle state.

[0153] In a possible implementation, the access information of the data to be processed includes access information of a first matrix and a second matrix, the number of columns of the first matrix is ​​the same as the number of rows of the second matrix, the logical operation type includes a matrix multiplication operation of the first matrix and the second matrix, and the loop block order determined by the matrix multiplication operation type includes: a first outer loop order in the direction of the number of rows of the first matrix, a second outer loop order in the direction of the number of columns of the second matrix, and an inner loop order in the direction of the number of columns of the first matrix and the number of rows of the second matrix; or, a first outer loop order in the direction of the number of columns of the second matrix, a second outer loop order in the direction of the number of rows of the first matrix, and an inner loop order in the direction of the number of columns of the first matrix and the number of rows of the second matrix.

[0154] In a possible implementation, the memory includes a local memory arranged outside the processor, and a reuse cache arranged in the processor for caching loop block data, and the loop block data of the data to be processed are read from the memory in sequence to the dot product unit array according to the loop block order of the data to be processed, including: the control logic unit stores the first loop block data of the first matrix read from the local memory into the reuse cache according to the first outer loop order and the inner loop order, so that the control logic unit reuses the first loop block data stored in the reuse cache according to the second outer loop order; or, the control logic unit stores the second loop block data of the second matrix read from the local memory into the reuse cache according to the second outer loop order and the inner loop order, so that the control logic unit reuses the second loop block data stored in the reuse cache according to the first outer loop order.

[0155] In a possible implementation, the dot product unit array performs a multiplication and accumulation operation on the circular block data of the data to be processed each time it is received, and determines the circular block result of the data to be processed each time it is received, including: the dot product unit array performs a multiplication and accumulation operation on the first circular block data and the second circular block data obtained each time, and obtains the circular block result corresponding to each multiplication and accumulation operation; the control logic unit determines the logical operation result of the data to be processed according to the multiple circular block results obtained from the dot product unit array, including: the control logic unit performs an accumulation operation on the multiple circular block results obtained from the dot product unit array in any round of the circular block sequence, and obtains a logical operator result; the control logic unit writes the logical operator result into a register stack; the control logic unit determines the logical operation result according to the multiple logical operator results obtained from the register stack.

[0156] In a possible implementation, the access information of the data to be processed includes access information of the data to be convolved and the convolution kernel, the logical operation type includes the convolution operation of the data to be convolved and the convolution kernel, the circular block order determined by the convolution operation type includes the coordinate order of the elements in the convolution kernel, and the circular block data of the data to be processed are read from the memory in sequence to the dot product unit array according to the circular block order of the data to be processed, including: the control logic unit, according to the coordinate order of the elements in the convolution kernel, converts multiple elements with the same coordinates read from multiple convolution kernels in the memory each time to form third circular block data, and, according to the coordinates of the current element in each convolution kernel and the access information, reads multiple elements from the data to be convolved in the memory to form fourth circular block data; the control logic unit writes the third circular block data and the fourth circular block data into the dot product unit array.

[0157] In one possible implementation, the memory includes a local memory arranged outside the processor, and a reuse cache arranged in the processor for caching circular block data, and the processor is also used for: in the process of calculating the fourth circular block data corresponding to the same row in the convolution output result, the control logic unit writes M elements of the fourth circular block number corresponding to the coordinates of the element in the rth row and sth column of the convolution kernel into the reuse cache according to the coordinate order of the elements in the convolution kernel, where r, s, and M are positive integers; the control logic unit reads M-1 elements from the reuse cache, reads 1 element from the data to be convolved in the local memory, and determines the fourth circular block data corresponding to the coordinates of the element in the rth row and s+1th column of the convolution kernel.

[0158] In one possible implementation, the processor is also used to: in the process of calculating the fourth circular block data corresponding to adjacent rows in the convolution output result, in response to the control logic unit calculating the fourth circular block data of the current row of the convolution output result, write M elements of the fourth circular block number corresponding to the coordinates of the element in the rth row and sth column in the convolution kernel into a reuse cache, where r, s, and M are positive integers; in response to the control logic unit calculating the fourth circular block data of the next row of the current row of the convolution output result, read M elements from the reuse cache to constitute the fourth circular block data corresponding to the coordinates of the element in the r-1th row and sth column in the convolution kernel.

[0159] In a possible implementation, the memory includes a local memory arranged outside the processor, and a reuse cache arranged in the processor for caching circular block data, and the control logic unit, according to the coordinate order of the elements in the convolution kernel, converts multiple elements with the same coordinates read from multiple convolution kernels in the memory each time into third circular block data, including: the control logic unit reads multiple convolution kernels from the local memory to the reuse cache; according to the coordinate order of the elements in the convolution kernel, the control logic unit converts multiple elements with the same coordinates read from multiple convolution kernels in the reuse cache each time into third circular block data.

[0160] In a possible implementation, the dot product unit array performs a multiplication and accumulation operation on the circular block data of the data to be processed each time received to determine the circular block result of the data to be processed each time received, including: the dot product unit array performs a multiplication and accumulation operation on the third circular block data and the fourth circular block data each time received to determine the circular block result.

[0161] In some embodiments, the functions or modules included in the processor provided in the embodiments of the present disclosure can be used to execute the method described in the above method embodiments. The specific implementation thereof can refer to the description of the above method embodiments, and for the sake of brevity, it will not be repeated here.

[0162] The embodiment of the present disclosure also provides a computer-readable storage medium on which computer program instructions are stored, and the computer program instructions implement the above method when executed by a processor. The computer-readable storage medium can be a volatile or non-volatile computer-readable storage medium.

[0163] The disclosed embodiment also provides an artificial intelligence chip, which includes the processor as described above.

[0164] The present disclosure also provides an electronic device, which includes the processor described above. The electronic device may include user equipment (UE), mobile device, user terminal, terminal, cellular phone, cordless phone, personal digital assistant (PDA), handheld device, computing device, vehicle-mounted device, wearable device, etc.

[0165] The embodiments of the present disclosure also provide a computer program product, including a computer-readable code, or a non-volatile computer-readable storage medium carrying the computer-readable code. When the computer-readable code runs in a processor of an electronic device, the processor in the electronic device executes the above method.

[0166] The electronic device may be provided as a terminal, a server, or a device in other forms.

[0167] Fig. 9 1 is a block diagram of an electronic device 1900 according to an embodiment of the present disclosure. For example, the electronic device 1900 may be provided as a server or a terminal device. Fig. 9 , the electronic device 1900 includes a processing component 1922, which further includes one or more processors, and a memory resource represented by a memory 1932 for storing instructions executable by the processing component 1922, such as an application. The application stored in the memory 1932 may include one or more modules, each corresponding to a set of instructions. In addition, the processing component 1922 is configured to execute instructions to perform the above method.

[0168] The electronic device 1900 may also include a power supply component 1926 configured to perform power management of the electronic device 1900, a wired or wireless network interface 1950 configured to connect the electronic device 1900 to a network, and an input / output (I / O) interface 1958. The electronic device 1900 may operate based on an operating system stored in the memory 1932, such as a Microsoft Server operating system (Windows Server 2003). TM ), a graphical user interface operating system launched by Apple (Mac OSX TM ), a multi-user, multi-process computer operating system (Unix TM ), a free and open source Unix-like operating system (Linux TM ), an open source Unix-like operating system (FreeBSD TM ) or similar.

[0169] In an exemplary embodiment, a non-volatile computer-readable storage medium is also provided, such as a memory 1932 including computer program instructions, which can be executed by the processing component 1922 of the electronic device 1900 to perform the above method.

[0170] The present disclosure may be a system, a method and / or a computer program product. The computer program product may include a computer-readable storage medium carrying computer-readable program instructions for causing a processor to implement various aspects of the present disclosure.

[0171] Computer readable storage medium can be a tangible device that can hold and store instructions used by an instruction execution device. Computer readable storage medium can be, for example, (but not limited to) an electrical storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination thereof. More specific examples (non-exhaustive list) of computer readable storage medium include: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a static random access memory (SRAM), a portable compact disk read-only memory (CD-ROM), a digital versatile disk (DVD), a memory stick, a floppy disk, a mechanical encoding device, for example, a punch card or a convex structure in a groove on which instructions are stored, and any suitable combination thereof. The computer readable storage medium used here is not interpreted as a transient signal itself, such as a radio wave or other freely propagating electromagnetic wave, an electromagnetic wave propagated by a waveguide or other transmission medium (for example, a light pulse by an optical fiber cable), or an electrical signal transmitted by a wire.

[0172] The computer-readable program instructions described herein can be downloaded from a computer-readable storage medium to each computing / processing device, or downloaded to an external computer or external storage device via a network, such as the Internet, a local area network, a wide area network, and / or a wireless network. The network can include copper transmission cables, optical fiber transmissions, wireless transmissions, routers, firewalls, switches, gateway computers, and / or edge servers. The network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards the computer-readable program instructions for storage in the computer-readable storage medium in each computing / processing device.

[0173] The computer program instructions for performing the operation of the present disclosure may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-related instructions, microcode, firmware instructions, state setting data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages, such as Smalltalk, C++, etc., and conventional procedural programming languages, such as "C" language or similar programming languages. Computer-readable program instructions may be executed completely on a user's computer, partially on a user's computer, as an independent software package, partially on a user's computer, partially on a remote computer, or completely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., using an Internet service provider to connect via the Internet). In some embodiments, an electronic circuit, such as a programmable logic circuit, a field programmable gate array (FPGA), or a programmable logic array (PLA), may be customized by utilizing the state information of the computer-readable program instructions, and the electronic circuit may execute the computer-readable program instructions, thereby realizing various aspects of the present disclosure.

[0174] Various aspects of the present disclosure are described herein with reference to the flowcharts and / or block diagrams of the methods, devices (systems) and computer program products according to the embodiments of the present disclosure. It should be understood that each box in the flowchart and / or block diagram and the combination of each box in the flowchart and / or block diagram can be implemented by computer-readable program instructions.

[0175] These computer-readable program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, thereby producing a machine, so that when these instructions are executed by the processor of the computer or other programmable data processing device, a device that implements the functions / actions specified in one or more boxes in the flowchart and / or block diagram is generated. These computer-readable program instructions can also be stored in a computer-readable storage medium, and these instructions cause the computer, programmable data processing device, and / or other equipment to work in a specific manner, so that the computer-readable medium storing the instructions includes a manufactured product, which includes instructions for implementing various aspects of the functions / actions specified in one or more boxes in the flowchart and / or block diagram.

[0176] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device so that a series of operating steps are performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions executed on the computer, other programmable data processing apparatus, or other device to implement the functions / actions specified in one or more boxes in the flowchart and / or block diagram.

[0177] The flow chart and block diagram in the accompanying drawings show the possible architecture, function and operation of the system, method and computer program product according to multiple embodiments of the present disclosure. In this regard, each square box in the flow chart or block diagram can represent a part of a module, program segment or instruction, and a part of the module, program segment or instruction includes one or more executable instructions for realizing the specified logical function. In some alternative implementations, the function marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two continuous square boxes can actually be executed substantially in parallel, and they can sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs the specified function or action, or can be implemented with a combination of special hardware and computer instructions.

[0178] The computer program product may be implemented in hardware, software or a combination thereof. In one optional embodiment, the computer program product is embodied as a computer storage medium, and in another optional embodiment, the computer program product is embodied as a software product, such as a software development kit (SDK) and the like.

[0179] The above description of various embodiments tends to emphasize the differences between the various embodiments. The same or similar aspects can be referenced to each other, and for the sake of brevity, they will not be repeated herein.

[0180] Those skilled in the art will appreciate that, in the above method of specific implementation, the order in which the steps are written does not imply a strict execution order and does not constitute any limitation on the implementation process. The specific execution order of the steps should be determined by their functions and possible internal logic.

[0181] The embodiments of the present disclosure have been described above, and the above description is exemplary, not exhaustive, and is not limited to the disclosed embodiments. Many modifications and changes will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments. The selection of terms used herein is intended to best explain the principles of the embodiments, practical applications, or improvements to the technology in the market, or to enable other persons of ordinary skill in the art to understand the embodiments disclosed herein.

Claims

1. A data processing method, characterized in that: The data processing method is applied to a processor, the processor includes a control logic unit and a dot multiplication unit array, the dot multiplication unit array includes a plurality of dot multiplication units for performing a multiplication-addition operation, and the method includes: The control logic unit sequentially reads the cyclic block data of the data to be processed from the memory to the dot product unit array according to the acquired control instruction, wherein the control instruction is used to indicate the access information of the data to be processed and the logical operation type of the data to be processed; The dot product unit array performs a multiplication and accumulation operation on the cyclic block data of the to-be-processed data received each time, and determines the cyclic block result of the to-be-processed data received each time; The control logic unit determines the logic operation result of the to-be-processed data according to the plurality of loop block results obtained from the dot product unit array; Wherein, in a case where the access information of the data to be processed includes the access information of the data to be convolved and the convolution kernel, and the logic operation type includes the convolution operation of the data to be convolved and the convolution kernel, the control logic unit sequentially reads the cyclic block data of the data to be processed from the memory to the dot product unit array according to the acquired control instruction, including: The control logic unit forms third circular block data with multiple elements having the same coordinates read from multiple convolution kernels in the memory each time according to the coordinate order of the elements in the convolution kernel, and reads multiple elements from the data to be convolved in the memory according to the coordinates of the current element in each convolution kernel and the access information to form fourth circular block data; The control logic unit writes the third circulation block data and the fourth circulation block data into the dot product unit array.

2. The method according to claim 1, characterized in that: The control logic unit sequentially reads the cyclic block data of the data to be processed from the memory to the dot multiplication unit array according to the acquired control instruction, including: The control logic unit parses the access information of the data to be processed and the logic operation type of the data to be processed according to the acquired control instruction; Determining a cyclic block order of the data to be processed according to a logical operation type of the data to be processed; According to the access information and the preset size, in accordance with the circular blocking order of the data to be processed, the circular block data of the data to be processed are read from the memory in sequence to the dot multiplication unit array, the size of the circular block data is less than or equal to the preset size, and the preset size is determined by the number of idle dot multiplication units in the dot multiplication unit array.

3. The method according to claim 2, characterized in that The access information of the data to be processed includes access information of a first matrix and a second matrix, the number of columns of the first matrix is ​​the same as the number of rows of the second matrix, and the logical operation type includes a matrix multiplication operation of the first matrix and the second matrix, The loop block order determined by the matrix multiplication operation type includes: a first outer loop order in the order of the row number direction of the first matrix, a second outer loop order in the order of the column number direction of the second matrix, and an inner loop order in the order of the column number direction of the first matrix and the row number direction of the second matrix; Alternatively, a first outer loop order according to the column number direction of the second matrix, a second outer loop order according to the row number direction of the first matrix, and an inner loop order according to the column number direction of the first matrix and the row number direction of the second matrix.

4. The method according to claim 3, characterized in that The memory includes a local memory arranged outside the processor, and a reuse cache arranged in the processor for caching cyclic block data, and the cyclic block data of the data to be processed are sequentially read from the memory to the dot multiplication unit array according to the cyclic block order of the data to be processed, including: The control logic unit stores the first loop block data of the first matrix read from the local memory into the reuse cache according to the first outer loop sequence and the inner loop sequence, so that the control logic unit reuses the first loop block data stored in the reuse cache according to the second outer loop sequence; or, The control logic unit stores the second loop block data of the second matrix read from the local memory into the reuse cache according to the second outer loop sequence and the inner loop sequence, so that the control logic unit reuses the second loop block data stored in the reuse cache according to the first outer loop sequence.

5. The method according to claim 4, characterized in that The dot product unit array performs a multiplication and accumulation operation on the cyclic block data of the to-be-processed data received each time to determine the cyclic block result of the to-be-processed data received each time, including: The dot product unit array performs a multiplication and accumulation operation on the first cyclic block data and the second cyclic block data obtained each time, to obtain a cyclic block result corresponding to each multiplication and accumulation operation; The control logic unit determines the logic operation result of the to-be-processed data according to the plurality of loop block results obtained from the dot product unit array, including: The control logic unit performs accumulation operation on a plurality of the loop block results obtained from the dot product unit array in sequence in any round of the loop block sequence to obtain a logic operation sub-result; The control logic unit writes the logic operator result into a register file; The control logic unit determines the logic operation result according to the plurality of logic operator results obtained from the register file.

6. The method according to claim 1, characterized in that The memory includes a local memory arranged outside the processor, and a reuse cache arranged in the processor for caching cyclic block data, and the method further includes: In the process of calculating the fourth circular block data corresponding to the same row in the convolution output result, the control logic unit writes M elements of the fourth circular block number corresponding to the coordinates of the elements in the rth row and sth column in the convolution kernel into the reuse cache according to the coordinate order of the elements in the convolution kernel, where r, s, and M are positive integers; The control logic unit reads M-1 elements from the reuse cache, reads 1 element from the data to be convolved in the local memory, and determines the fourth loop block data corresponding to the coordinates of the element in the rth row and s+1th column in the convolution kernel.

7. The method according to claim 1, characterized in that The method further includes: in the process of calculating the fourth circular block data corresponding to the adjacent row in the convolution output result, in response to the control logic unit calculating the fourth circular block data of the current row of the convolution output result, writing M elements of the fourth circular block number corresponding to the coordinates of the element in the rth row and sth column in the convolution kernel into the reuse cache, where r, s, and M are positive integers; In response to the control logic unit calculating the fourth circular block data of the next row of the current row of the convolution output result, M elements are read from the reuse cache to form the fourth circular block data corresponding to the coordinates of the element in the r-1th row and sth column in the convolution kernel.

8. The method according to claim 1, characterized in that The memory includes a local memory arranged outside the processor, and a reuse cache arranged in the processor for caching cyclic block data, and the control logic unit forms third cyclic block data from multiple elements with the same coordinates read from multiple convolution kernels in the memory each time according to the coordinate order of the elements in the convolution kernel, including: The control logic unit reads a plurality of convolution kernels from the local memory to the reuse cache; According to the coordinate order of the elements in the convolution kernel, the control logic unit reads multiple elements with the same coordinates from multiple convolution kernels in the reuse cache each time to form third cyclic block data.

9. The method according to any one of claims 1, 6-8, characterized in that: The dot product unit array performs a multiplication and accumulation operation on the cyclic block data of the to-be-processed data received each time to determine the cyclic block result of the to-be-processed data received each time, including: The dot product unit array performs a multiplication and accumulation operation on the third cyclic block data and the fourth cyclic block data received each time to determine a cyclic block result.

10. A processor, characterized in that: The processor includes a control logic unit and a dot multiplication unit array, wherein the dot multiplication unit array includes a plurality of dot multiplication units for performing multiplication and accumulation operations. The processor is used to: The control logic unit sequentially reads the cyclic block data of the data to be processed from the memory to the dot product unit array according to the acquired control instruction, wherein the control instruction is used to indicate the access information of the data to be processed and the logical operation type of the data to be processed; The dot product unit array performs a multiplication and accumulation operation on the cyclic block data of the to-be-processed data received each time, and determines the cyclic block result of the to-be-processed data received each time; The control logic unit determines the logic operation result of the to-be-processed data according to the plurality of loop block results obtained from the dot product unit array; Wherein, in a case where the access information of the data to be processed includes the access information of the data to be convolved and the convolution kernel, and the logic operation type includes the convolution operation of the data to be convolved and the convolution kernel, the control logic unit sequentially reads the cyclic block data of the data to be processed from the memory to the dot product unit array according to the acquired control instruction, including: The control logic unit forms third circular block data with multiple elements having the same coordinates read from multiple convolution kernels in the memory each time according to the coordinate order of the elements in the convolution kernel, and reads multiple elements from the data to be convolved in the memory according to the coordinates of the current element in each convolution kernel and the access information to form fourth circular block data; The control logic unit writes the third circulation block data and the fourth circulation block data into the dot product unit array.

11. An artificial intelligence chip, characterized in that: The artificial intelligence chip includes the processor as described in claim 10.

12. An electronic device, characterized in that: The electronic device comprises the artificial intelligence chip as claimed in claim 11.

Images