Processing method, processing device and related products

Abstract

The present disclosure discloses a processing method, a processing device and related products. The processing device can be implemented as a computing device included in a combined processing device, which can further include an interface device and other processing devices. The computing device interacts with the other processing devices to jointly complete a user-specified computing operation. The combined processing device can further include a storage device connected to the computing device and the other processing devices respectively, for storing data of the computing device and the other processing devices. The scheme of the present disclosure provides a solution of instruction parallelism, which can improve the instruction parallelism and thus improve the processing efficiency of the machine.

Description

Technical Field

[0001] The present disclosure relates to the field of processors, and in particular to a processing method, a processing device, a chip, and a board. Background Art

[0002] The instruction set is the interface between computer software and hardware and is a crucial component of computer system architecture. With the continuous advancement of artificial intelligence (AI) technology, the amount and dimensionality of data that must be processed are increasing. Therefore, how to rationally and scientifically control the execution of instructions, particularly increasing the degree of instruction parallelism and improving machine performance, is a crucial issue in instruction set design. Summary of the Invention

[0003] In order to solve one or more of the technical problems mentioned above, the present disclosure proposes solutions to enhance instruction parallelism in multiple aspects. Through the instruction system disclosed in the present disclosure, the degree of instruction parallelism can be increased, thereby improving the processing efficiency of the machine.

[0004] In a first aspect, the present disclosure provides a processing method, comprising: obtaining a first operation of a decoded instruction; determining a first coordinate space range of tensor data that the first operation is allowed to use; determining a second coordinate space range of the tensor data to be used when performing the first operation; and performing the first operation within a third coordinate space range determined by the intersection of the first coordinate space range and the second coordinate space range; wherein the first coordinate space range and the second coordinate space range are determined at least in part based on a predetermined partitioning of a shape coordinate space of the tensor data.

[0005] In a second aspect, the present disclosure provides a processing device, comprising: an operation acquisition unit configured to acquire a first operation of a decoded instruction; a first determination unit configured to determine a first coordinate space range of tensor data allowed to be used by the first operation; a second determination unit configured to determine a second coordinate space range of the tensor data to be used when performing the first operation; and an execution unit configured to execute the first operation within a third coordinate space range determined by the intersection of the first coordinate space range and the second coordinate space range; wherein the first coordinate space range and the second coordinate space range are determined at least in part based on a predetermined division of the shape coordinate space of the tensor data.

[0006] In a third aspect, the present disclosure provides a chip comprising the processing device of any embodiment of the aforementioned second aspect.

[0007] In a fourth aspect, the present disclosure provides a board comprising the chip of any one of the embodiments of the third aspect.

[0008] Through the processing device, processing method, chip and board provided above, the disclosed embodiment limits the coordinate space range used for the operation during the execution of the instruction operation, so that when the hardware is executed in parallel, it can not only ensure the consistency of the execution order but also improve the degree of parallelism of the operation, thereby ensuring the accuracy and efficiency of the processing. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] The above and other objects, features and advantages of the exemplary embodiments of the present disclosure will become readily understood by reading the detailed description below with reference to the accompanying drawings. In the accompanying drawings, several embodiments of the present disclosure are shown in an exemplary and non-limiting manner, and the same or corresponding reference numerals represent the same or corresponding parts.

[0010] Figure 1A A schematic diagram showing a data storage space according to an embodiment of the present disclosure;

[0011] Figure 1B A schematic diagram showing data blocks in a data storage space according to an embodiment of the present disclosure;

[0012] Figure 2 A schematic block diagram showing a processing device according to an embodiment of the present disclosure;

[0013] Figure 3A A schematic flow chart showing a processing method according to an embodiment of the present disclosure;

[0014] Figure 3B A schematic block diagram showing a processing device according to an embodiment of the present disclosure;

[0015] Figure 4A-4B A schematic diagram illustrating a coordinate space range according to an embodiment of the present disclosure;

[0016] Figure 5 A structural diagram showing a combined processing device according to an embodiment of the present disclosure; and

[0017] Figure 6 A schematic structural diagram of a board card according to an embodiment of the present disclosure is shown. DETAILED DESCRIPTION

[0018] The following will clearly and completely describe the technical solutions in the embodiments of this disclosure in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of this disclosure, not all of them. Based on the embodiments of this disclosure, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of this disclosure.

[0019] It should be understood that the terms "first," "second," "third," and "fourth," etc., which may be used in the claims, specification, and drawings of this disclosure, are used to distinguish different objects rather than to describe a specific order. The terms "include" and "comprising" used in the specification and claims of this disclosure indicate the presence of the described features, integers, steps, operations, elements, and / or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or combinations thereof.

[0020] It should also be understood that the terminology used in this disclosure is for the purpose of describing specific embodiments only and is not intended to limit the disclosure. As used in this disclosure and the claims, the singular forms "a," "an," and "the" are intended to include the plural forms unless the context clearly indicates otherwise. It should be further understood that the term "and / or" as used in this disclosure and the claims refers to any and all possible combinations of one or more of the associated listed items, including and including these combinations.

[0021] As used in this specification and claims, the term "if" can be interpreted as "when" or "upon" or "in response to determining" or "in response to detecting," depending on the context. Similarly, the phrase "if it is determined" or "if [described condition or event] is detected" can be interpreted as meaning "upon determination" or "in response to determining" or "upon detection of [described condition or event]" or "in response to detecting [described condition or event]," depending on the context.

[0022] Computers process data by executing instructions. To indicate the source of the data, the destination of the results, and the operation performed, an instruction typically contains the following information:

[0023] (1) Operation Code (OP): This is used to indicate the operation to be performed by the instruction (e.g., addition, subtraction, multiplication, division, data transfer, etc.). It specifies the nature and function of the operation. A computer may have dozens to hundreds of instructions, each of which has a corresponding operation code. The computer performs different operations by identifying the operation code.

[0024] (2) Operands, which are used to describe the object of the instruction. Operands can involve the data type, memory address, addressing mode, etc. of the object being operated on. Operands can directly give the object being operated on, or indicate the memory address or register address (i.e., register name) of the object being operated on.

[0025] Conventional processor instructions are designed to perform basic single data scalar operations. Here, single data scalar operation refers to an instruction whose each operand is a scalar data. However, with the development of artificial intelligence technology, in tasks such as image processing and pattern recognition, the operand tends to be a multi-dimensional vector (i.e., tensor data) data type, and only using scalar operations cannot make the hardware efficiently complete the operation task. Therefore, how to efficiently perform multi-dimensional tensor data processing is also a problem that needs to be solved in the current computing field.

[0026] In embodiments of the present disclosure, an instruction system is provided, wherein a descriptor is included in the operand of the instruction, through which information related to the tensor data can be quickly obtained. Specifically, the descriptor can indicate at least one of the following information: shape information of the tensor data, spatial information of the tensor data. The shape information of the tensor data can be used to determine the data address of the tensor data corresponding to the operand in the data storage space. The spatial information of the tensor data can be used to determine the dependency relationship between instructions, and in turn can determine, for example, the execution order of the instructions. The spatial information of the tensor data can be indicated by a spatial identifier (ID). The spatial ID can also be referred to as a spatial alias, which refers to a spatial region for storing the corresponding tensor data, which can be a continuous space or multiple spaces, and the present disclosure does not limit the specific form of the spatial region. Different spatial IDs represent that the spatial regions pointed to by the different spatial IDs do not have a dependency relationship. For example, the dependency relationship can be ensured by making the spatial regions pointed to by different spatial IDs not overlap with each other.

[0027] Various possible implementations of the shape information of the tensor data will be described in detail below with reference to the accompanying drawings.

[0028] The tensor can contain various forms of data composition. The tensor can be of different dimensions, such as a scalar can be regarded as a 0-dimensional tensor, a vector can be regarded as a 1-dimensional tensor, and a matrix can be a 2-dimensional or higher-dimensional tensor. The shape of the tensor includes information such as the dimensions of the tensor and the size of each dimension of the tensor. For example, for a three-dimensional tensor:

[0029] x3 = [[[1, 2, 3], [4, 5, 6]]; [[7, 8, 9], [10, 11, 12]]]

[0030] The shape or dimensions of the tensor can be expressed as X3=(2, 2, 3), which means that the three parameters indicate that the tensor is a three-dimensional tensor, with the size of the first dimension being 2, the size of the second dimension being 2, and the size of the third dimension being 3. When storing tensor data in memory, the shape of the tensor data cannot be determined based on its data address (or storage area), and further, related information such as the relationship between multiple tensor data cannot be determined, resulting in low processor access efficiency to the tensor data.

[0031] In one possible implementation, a descriptor can be used to indicate the shape of N-dimensional tensor data, where N is a positive integer, such as N=1, 2, or 3, or zero. The three-dimensional tensor in the above example can be represented by a descriptor as (2, 2, 3). It should be noted that this disclosure does not limit the manner in which a descriptor indicates the shape of a tensor.

[0032] In one possible implementation, the value of N can be determined according to the dimension (also called order) of the tensor data, or it can be set according to the usage requirements of the tensor data. For example, when the value of N is 3, the tensor data is three-dimensional tensor data, and the descriptor can be used to indicate the shape of the three-dimensional tensor data in three dimensions (such as offset, size, etc.). It should be understood that those skilled in the art can set the value of N according to actual needs, and this disclosure does not limit this.

[0033] Although tensor data can be multi-dimensional, because the memory layout is always one-dimensional, there is a correspondence between tensors and storage on the memory. Tensor data is usually allocated in a continuous memory space, that is, tensor data can be expanded one-dimensionally (for example, row-major) and stored in the memory.

[0034] This relationship between a tensor and its underlying storage can be expressed through dimensions such as offset, size, and stride. A dimension's offset refers to the offset relative to a reference position in that dimension. A dimension's size refers to the size of that dimension, or the number of elements in that dimension. A dimension's stride refers to the spacing between adjacent elements in that dimension. For example, the stride of the three-dimensional tensor above is (6, 3, 1), meaning the stride of the first dimension is 6, the stride of the second dimension is 3, and the stride of the third dimension is 1.

[0035] Figure 1A Schematic diagram showing the data storage space according to the embodiment of the present disclosure. Figure 1AAs shown, the data storage space 21 stores a two-dimensional data in a row-major manner, which can be represented by (x, y) (wherein the X-axis is horizontally to the right, and the Y-axis is vertically downward). The size in the X-axis direction (the size of each row, or the total number of columns) is ori_x (not shown in the figure), the size in the Y-axis direction (the total number of rows) is ori_y (not shown in the figure), and the starting address PA_start (reference address) of the data storage space 21 is the physical address of the first data block 22. The data block 23 is part of the data in the data storage space 21, and the offset 25 in the X-axis direction is offset_x, the offset 24 in the Y-axis direction is offset_y, the size in the X-axis direction is size_x, and the size in the Y-axis direction is size_y.

[0036] In a possible implementation, when the data block 23 is defined using the descriptor, the data reference point of the descriptor can use the first data block of the data storage space 21, and it can be agreed that the reference address of the descriptor is the starting address PA_start of the data storage space 21. Then, the content of the descriptor of the data block 23 can be determined in combination with the size ori_x of the data storage space 21 in the X-axis direction, the size ori_y in the Y-axis direction, and the offset offset_y of the data block 23 in the Y-axis direction, the offset offset_x in the X-axis direction, the size size_x in the X-axis direction, and the size size_y in the Y-axis direction.

[0037] In a possible implementation, the content of the descriptor can be represented by the following formula (1):

[0038]

[0039] It should be understood that, although the content of the descriptor in the above example represents a two-dimensional space, a person skilled in the art can set the specific dimension represented by the content of the descriptor according to actual conditions, and the disclosure does not limit this.

[0040] In a possible implementation, it can be agreed that the data reference point of the descriptor is at a reference address in the data storage space, and on the basis of the reference address, the content of the descriptor of the tensor data is determined according to the positions of at least two vertices at the diagonal positions in the N-dimensional directions relative to the data reference point.

[0041] For example, the data reference point of the descriptor can be agreed to be the reference address PA_base in the data storage space. For example, a data (e.g., data at position (2, 2)) can be selected in the data storage space 21 as the data reference point, and the physical address of the data in the data storage space can be used as the reference address PA_base. The position of the two vertices at the diagonal position relative to the data reference point can be used to determine the reference address PA_base. Figure 1A The content of the descriptor of data block 23 in the data block 23 is determined. First, the positions of at least two vertices at the diagonal positions of the data block 23 relative to the data reference point are determined. For example, the positions of the diagonal vertices from the upper left to the lower right relative to the data reference point are used, where the relative position of the upper left vertex is (x_min, y_min) and the relative position of the lower right vertex is (x_max, y_max). Then, the content of the descriptor of data block 23 can be determined based on the reference address PA_base, the relative position of the upper left vertex (x_min, y_min), and the relative position of the lower right vertex (x_max, y_max).

[0042] In a possible implementation, the following formula (2) can be used to express the content of the descriptor (the base address is PA_base):

[0043]

[0044] It should be understood that although the vertices at the upper left corner and the lower right corner are used in the above example to determine the content of the descriptor, those skilled in the art can set the specific vertices of at least two diagonal positions according to actual needs, and this disclosure does not limit this.

[0045] In one possible implementation, the content of the tensor data descriptor can be determined based on the reference address of the descriptor's data reference point in the data storage space and the mapping relationship between the data description position and the data address of the tensor data indicated by the descriptor. The mapping relationship between the data description position and the data address can be set according to actual needs. For example, when the tensor data indicated by the descriptor is three-dimensional spatial data, the function f(x, y, z) can be used to define the mapping relationship between the data description position and the data address.

[0046] In a possible implementation, the following formula (3) can be used to express the content of the descriptor:

[0047]

[0048] In one possible implementation, the descriptor is further used to indicate the address of N-dimensional tensor data, wherein the content of the descriptor further includes at least one address parameter representing the address of the tensor data. For example, the content of the descriptor may be the following formula (4):

[0049]

[0050] Where PA is the address parameter. The address parameter can be a logical address or a physical address. When parsing the descriptor, PA can be used as any vertex, midpoint, or preset point of the vector shape, combined with the shape parameters in the X and Y directions to obtain the corresponding data address.

[0051] In a possible implementation, the address parameter of the tensor data includes a reference address of a data reference point of the descriptor in the data storage space of the tensor data, and the reference address includes a starting address of the data storage space.

[0052] In a possible implementation, the descriptor may further include at least one address parameter representing the address of the tensor data. For example, the content of the descriptor may be the following formula (5):

[0053]

[0054] PA_start is the base address parameter and will not be described in detail.

[0055] It should be understood that those skilled in the art can set the mapping relationship between the data description location and the data address according to actual conditions, and this disclosure does not limit this.

[0056] In one possible implementation, a predetermined reference address can be set within a task. All descriptors in instructions within this task use this reference address, and the descriptor content can include shape parameters based on this reference address. This reference address can be determined by setting the environment parameters for this task. A description of the reference address and its use can be found in the above embodiments. In this implementation, the descriptor content can be mapped to data addresses more quickly.

[0057] In one possible implementation, the base address can be included in the content of each descriptor, so that the base address of each descriptor can be different. Compared with the method of using environmental parameters to set a common base address, each descriptor in this method can describe data more flexibly and use a larger data address space.

[0058] In one possible implementation, the data address of the data corresponding to the operand of the processing instruction in the data storage space can be determined based on the content of the descriptor. The data address is calculated automatically by hardware, and the calculation method of the data address varies depending on the representation of the descriptor content. This disclosure does not limit the specific method for calculating the data address.

[0059] For example, the content of the descriptor in the operand is expressed using formula (1). The offsets of the tensor data indicated by the descriptor in the data storage space are offset_x and offset_y, and the size is size_x*size_y. Then, the starting data address PA1 of the tensor data indicated by the descriptor in the data storage space is (x,y) It can be determined using the following formula (6):

[0060] PA1 (x,y) =PA_start+(offset_y-1)*ori_x+offset_x (6)

[0061] The data starting address PA1 is determined according to the above formula (6) (x,y) , combined with the offsets offset_x and offset_y, and the sizes size_x and size_y of the storage area, the storage area of ​​the tensor data indicated by the descriptor in the data storage space can be determined.

[0062] In one possible implementation, when the operand also includes a data description location for a descriptor, the data address of the data corresponding to the operand in the data storage space can be determined based on the content of the descriptor and the data description location. In this way, partial data (e.g., one or more data) in the tensor data indicated by the descriptor can be processed.

[0063] For example, the content of the descriptor in the operand is expressed using formula (2). The offsets of the tensor data indicated by the descriptor in the data storage space are offset_x and offset_y respectively, and the size is size_x*size_y. The data description position for the descriptor included in the operand is (x q ,y q ), then the data address PA2 of the tensor data indicated by the descriptor in the data storage space (x,y) It can be determined using the following formula (7):

[0064] PA2 (x,y) =PA_start+(offset_y+y q -1)*ori_x+(offset_x+x q ) (7)

[0065] In a possible implementation, the descriptor can indicate the data blocks. Data blocks can effectively speed up the operation and improve the processing efficiency in many applications. For example, in graphics processing, convolution operation often uses data blocks for fast operation processing.

[0066] Figure 1B A schematic diagram of data blocks in a data storage space according to an embodiment of the present disclosure is shown. As shown, the data storage space 26 also stores two-dimensional data in a row-major manner, which can be represented by (x, y) (where the X axis is horizontal to the right, and the Y axis is vertical downward). The size in the X axis direction (the size of each row, or the total number of columns) is ori_x (not shown in the figure), and the size in the Y axis direction (the total number of rows) is ori_y (not shown in the figure). Unlike the tensor data Figure 1B , Figure 1A the tensor data stored in the data storage space 26 includes a plurality of data blocks. Figure 1B In this case, the descriptor needs more parameters to represent these data blocks. Taking the X axis (X dimension) as an example, the following parameters can be involved: ori_x, x.tile.size (size 27 in the block), x.tile.stride (stride in the block, i.e. the distance between the first point of the first small block and the first point of the second small block), x.tile.num (the number of blocks, which is 3 in the example shown in

[0067] Figure 1B In a possible implementation, the descriptor can include an identifier of the descriptor and / or content of the descriptor. The identifier of the descriptor is used to distinguish the descriptor, for example, the identifier of the descriptor can be its number; the content of the descriptor can include at least one shape parameter representing the shape of the tensor data. For example, the tensor data is 3-dimensional data, and in the three dimensions of the tensor data, the shape parameters of two dimensions are fixed, and the content of the descriptor of the tensor data can include a shape parameter representing the shape of the other dimension of the tensor data.

[0068] In a possible implementation, the data address of the data storage space corresponding to each descriptor can be a fixed address. For example, a separate data storage space can be allocated for the tensor data, and the starting address of each tensor data in the data storage space corresponds to the descriptor one by one. In this case, the circuit or module responsible for parsing the computing instruction (for example, an entity outside the computing device of the present disclosure) can determine the data address of the data corresponding to the operand in the data storage space according to the descriptor.

[0069] In a possible implementation, the data address of the data storage space corresponding to each descriptor can be a fixed address. For example, a separate data storage space can be allocated for the tensor data, and the starting address of each tensor data in the data storage space corresponds to the descriptor one by one. In this case, the circuit or module responsible for parsing the computing instruction (for example, an entity outside the computing device of the present disclosure) can determine the data address of the data corresponding to the operand in the data storage space according to the descriptor. ​

[0070] In one possible implementation, when the data address of the data storage space corresponding to the descriptor is a variable address, the descriptor can also be used to indicate the address of N-dimensional tensor data, wherein the content of the descriptor can also include at least one address parameter representing the address of the tensor data. For example, the tensor data is 3-dimensional data. When the descriptor points to the address of the tensor data, the content of the descriptor may include an address parameter representing the address of the tensor data, such as the starting physical address of the tensor data, or may include multiple address parameters of the address of the tensor data, such as the starting address + address offset of the tensor data, or the address parameters of the tensor data based on each dimension. Those skilled in the art can set the address parameters according to actual needs, and this disclosure does not limit this.

[0071] In one possible implementation, the address parameter of the tensor data may include the reference address of the descriptor's data reference point in the data storage space of the tensor data. The reference address may vary depending on the data reference point. This disclosure does not limit the selection of the data reference point.

[0072] In one possible implementation, the reference address may include the starting address of the data storage space. When the data reference point of the descriptor is the first data block in the data storage space, the reference address of the descriptor is the starting address of the data storage space. When the data reference point of the descriptor is data other than the first data block in the data storage space, the reference address of the descriptor is the address of the data block in the data storage space.

[0073] In one possible implementation, the shape parameters of the tensor data include at least one of the following: the size of the data storage space in at least one direction of the N-dimensional directions, the size of the storage area in at least one direction of the N-dimensional directions, the offset of the storage area in at least one direction of the N-dimensional directions, the positions of at least two vertices at diagonal positions in the N-dimensional directions relative to the data reference point, and the mapping relationship between the data description position of the tensor data indicated by the descriptor and the data address. The data description position is the mapping position of the point or area in the tensor data indicated by the descriptor. For example, when the tensor data is 3D data, the descriptor can use three-dimensional space coordinates (x, y, z) to represent the shape of the tensor data, and the data description position of the tensor data can be the position of the point or area mapped in the three-dimensional space represented by the three-dimensional space coordinates (x, y, z).

[0074] It should be understood that those skilled in the art can select shape parameters representing tensor data according to actual circumstances, and this disclosure does not limit this. By using descriptors in the data access process, associations between data can be established, thereby reducing the complexity of data access and improving instruction processing efficiency.

[0075] Figure 2 Schematic block diagram of a processing device according to an embodiment of the present disclosure is shown. Figure 2 As shown, the processing device 200 includes a control module 210 , a calculation module 220 and a storage module 230 .

[0076] The control module 210 can be configured to control the operation of the processing device 200, such as reading instructions from memory or external devices, decoding the instructions via the decoder 211, and issuing micro-operation control signals to corresponding components. Specifically, the control module 210 can be configured to control the execution unit 220 to perform corresponding processing based on the received instructions. Instructions may include, but are not limited to, data access instructions, operation instructions, descriptor management instructions, and synchronization instructions. This disclosure does not limit the specific types of instructions or the specific decoding methods.

[0077] The decoded instruction includes an opcode and an operand. When the instruction involves processing tensor data, at least one operand of the instruction may include at least one descriptor indicating at least one of the following information: shape information of the tensor data and spatial information of the tensor data.

[0078] The operation module 220 is configured to execute specific instructions or operations under the control of the control module 210. The operation module 220 may include, for example, but is not limited to, an arithmetic and logic unit (ALU), a memory access unit (MAU), a neural functional unit (NFU), etc. Figure 2 Schematically showing two functional units 221 and 222. The present disclosure does not limit the specific hardware types of the functional units.

[0079] The storage module 230 can be configured to store various information, including but not limited to instructions, information associated with descriptors, tensor data, etc. The storage module 230 may include various storage resources, including but not limited to internal memory and external memory. Internal memory may include, for example, registers, on-chip SRAM, or other media cache. External memory may include, for example, off-chip memory. This disclosure does not limit the specific implementation of the storage module.

[0080] Optionally or additionally, the processing device 200 may further include a tensor interface unit (TIU) 240. The tensor interface unit 240 may be configured to implement operations associated with descriptors under the control of the control module 210. These operations may include, but are not limited to, registering, modifying, deregistering, and parsing descriptors; reading and writing descriptor contents, etc. The present disclosure does not limit the specific hardware type of the tensor interface unit. In this way, operations associated with descriptors can be implemented through dedicated hardware, further improving the access efficiency of tensor data.

[0081] In some embodiments of the present disclosure, the tensor interface unit 240 may be configured to parse a descriptor included in an operand of an instruction. For example, the tensor interface unit may parse the shape information of the tensor data included in the descriptor to determine the data address of the data corresponding to the operand in the data storage space.

[0082] Despite Figure 2 The control module 210 and the tensor interface unit 240 are shown as two separate modules, but those skilled in the art will appreciate that these two modules / units may also be implemented as one module or more modules, and the present disclosure is not limited in this respect.

[0083] The data processing device 200 can be implemented using a general-purpose processor (such as a central processing unit CPU, a graphics processing unit GPU) and / or a special-purpose processor (such as an artificial intelligence processor, a scientific computing processor or a digital signal processor, etc.). This disclosure does not limit the specific type of the data processing device.

[0084] When the hardware executes instructions in parallel, if there is a dependency between the instructions executed in parallel, it may cause an execution error. For example, if two instructions executed in parallel access the same storage unit or the same data, and at least one of the two instructions is an instruction to write to the storage unit, then there is a dependency between the two instructions, such as a read-after-write dependency, a write-after-write dependency, or a read-after-write dependency. In this case, if the latter instruction is executed before the previous instruction, it will cause an execution error. Therefore, the sequential consistency of these instructions must be guaranteed, for example, by forcing sequential execution, that is, the latter instruction must wait for the previous instruction to complete before it can be executed.

[0085] As can be known from the foregoing description of the tensor data, the tensor data is usually a multi-dimensional array and has a large amount of data, and thus the processing time of the instruction for the tensor data is usually longer than that of the scalar data. At this time, if the tensor data is still processed according to the previous order execution manner, the processing time is too long and the efficiency is low. In view of this, in the embodiments of the present disclosure, an instruction parallel scheme is provided, in which the parallel execution of the operation is limited based on the coordinate space range of the tensor data used by the operation of the instruction, so as to ensure the consistency of the execution order and improve the parallel degree of the operation when the hardware is executed in parallel, thereby ensuring the accuracy and efficiency of the processing.

[0086] Figure 3A An exemplary flowchart of a processing method 300 according to an embodiment of the present disclosure is shown. The processing method 300 may, for example, be implemented by the processing apparatus 200 of Figure 2 .

[0087] As shown in Figure 3A , the method 300 starts at step S310, obtaining a first operation of a decoded instruction. This step may, for example, be performed by the decoder 211 in the control module 210 of the processing apparatus 200. In some embodiments, the first operation can involve processing of tensor data. Figure 2

[0088] It should be noted that the operation involved in the present disclosure can be a basic operation supported by the processor hardware, or a microinstruction (such as a request signal, etc.) parsed from the basic operation. The present disclosure does not limit the specific type of operation. The processing apparatus of the present disclosure can execute two operations in parallel, or more than two operations in parallel, and the present disclosure does not limit the number of operations executed in parallel. The two operations executed in parallel can belong to the same instruction, or belong to different instructions, and the present disclosure does not limit this aspect.

[0089] Next, in step S320, a first coordinate space range of tensor data allowed to be used by the first operation is determined. This step may, for example, be performed by the control module 210 of the processing apparatus 200, or determined by the control module 210 controlling the tensor interface unit 240. Figure 2

[0090] Next, in step S330, a second coordinate space range of the tensor data to be used when executing the first operation is determined. In one implementation, this step may, for example, be performed by the control module 210 of the processing apparatus 200, or determined by the control module 210 controlling the tensor interface unit 240. In another implementation, this step may, for example, be performed by the operation module 220 of the processing apparatus 200, for example, determined by the corresponding functional unit 221 or 222. Figure 2 Figure 2

[0091] ​​​​Finally, in step S340, the first operation is performed within the third coordinate space range determined by the intersection of the first coordinate space range and the second coordinate space range. Figure 2 The operation module 220 executes the operation, specifically, the corresponding functional units 221 and 222 execute the operation.

[0092] In some embodiments, when the third coordinate space range is empty, the first operation can be blocked. By blocking the subsequent operation, multiple operations with dependencies can be forced to execute in a predetermined order, thereby ensuring the correctness of the result.

[0093] In the disclosed embodiments, by limiting the coordinate space range of the tensor data that can be used when executing an operation, for example, limiting the operation to the third coordinate space range, it is possible to ensure that the instructions access each coordinate space range sequentially during the parallel execution of the instructions, thereby ensuring accurate and efficient processing. In some embodiments, the first coordinate space range and the second coordinate space range are determined at least in part based on a predetermined partitioning of the shape coordinate space of the tensor data targeted by the operation.

[0094] In some embodiments, the above-mentioned first operation involves the processing of tensor data. Accordingly, the first coordinate space range and the second coordinate space range can be a part of the shape coordinate space of the corresponding dimension of the tensor data, respectively. The shape coordinate space is mapped to the data storage area of ​​the tensor data on the storage module 230. By dividing the shape coordinate space of the tensor data into several coordinate space ranges, and constraining the parallel execution of instructions based on the restrictions of the coordinate space range (for example, the coordinate space ranges of the previous and next operations do not overlap), the parallelism of the processing is improved and the processing time is reduced. Furthermore, since software-side programming usually uses spatial coordinates to reference data points or data blocks in tensor data, constraining the parallel execution of operations by the coordinate space range of tensor data can simplify the code programming on the software side and be more conducive to the execution of instructions.

[0095] This disclosure also provides methods for implementing Figure 3A An exemplary processing device of the processing method 300 is provided. Figure 3B A schematic functional block diagram of a processing device according to an embodiment of the present disclosure is shown.

[0096] like Figure 3B As shown, the processing device 30 includes an operation acquiring unit 31 , a first determining unit 32 , a second determining unit 33 and an executing unit 34 .

[0097] The operation acquisition unit 31 is configured to acquire the first operation of the decoded instruction. The first determination unit 32 is configured to determine a first coordinate space range of tensor data allowed to be used by the first operation. The second determination unit 33 is configured to determine a second coordinate space range of tensor data to be used when performing the first operation. The execution unit 34 is configured to execute the first operation within a third coordinate space range determined by the intersection of the first coordinate space range and the second coordinate space range.

[0098] Those skilled in the art will understand that Figure 3B The various units shown are divided according to functional implementation. This division is only exemplary. In actual implementation, two or more functions can be implemented in the same hardware unit, and one function may also be implemented in two hardware units.

[0099] For example, in one implementation, the operation acquisition unit 31 and the first determination unit 32 may be included in Figure 2 The second determining unit 33 and the executing unit 34 may be included in the control module 210 of the processing device 200 , while the second determining unit 33 and the executing unit 34 may be included in the computing module 220 of the processing device 200 .

[0100] For example, in another implementation, the operation acquisition unit 31, the first determination unit 32 and the second determination unit 33 may be included in Figure 2 The processing device 200 is shown in the control module 210 , while the execution unit 34 is included in the operation module 220 of the processing device 200 .

[0101] It should also be understood that the units included in the processing device 30 are similar to those in the reference Figure 3A Therefore, the operations and features described herein for the method are also applicable to the processing device 30 and the units included therein, and will not be described in detail.

[0102] In some embodiments of the present disclosure, the first coordinate space range and the second coordinate space range may be determined based at least in part on a predetermined partitioning of the shape coordinate space of the tensor data.

[0103] More specifically, in some embodiments, the processing method further includes determining a prior operation being executed that accesses the same tensor data as the first operation and has a dependency relationship, and determining the first coordinate space range based at least in part on the operating state of the prior operation. As mentioned above, there may be three types of dependencies between the current operation and the prior operation, such as read-after-write dependency, write-after-write dependency, or read-after-write dependency. At this time, the sequential consistency of the execution of these instructions must be guaranteed, so the first coordinate space range allowed for the current operation (here, the first operation) is determined based on the operating state of the prior operation.

[0104] Figure 4A The diagram schematically illustrates the division of the coordinate space range according to an embodiment of the present disclosure. Figure 4A Two-dimensional data is used as an example for illustrative illustration. However, those skilled in the art will appreciate that the same solution can be similarly applied to three-dimensional or even higher-dimensional tensor data.

[0105] like Figure 4A As shown, the shape coordinate space 400A of the two-dimensional tensor data is divided into four spatial blocks, namely 41, 42, 43, and 44. In each spatial block, access to the data at each coordinate point within it is guaranteed to be sequential. That is, for each coordinate point, the previous operations with dependencies are executed first, and then the first operation is executed, thereby ensuring the correctness of the results.

[0106] In some embodiments, Figure 3A Step S320 may further include: determining a spatial block in the shape coordinate space of the tensor data, on which the previous operation has been completed, as a first coordinate space range.

[0107] In these embodiments, for example, when the previous operation has completed access to spatial blocks 41 and 43 and spatial block 42 is being used, the spatial range allowed for the first operation to use (i.e., the first coordinate spatial range) may include spatial block 41 and spatial block 43, such as the area indicated by the diagonal shading.

[0108] Alternatively or additionally, in some embodiments, Figure 3A Step S330 may further include determining a spatial block determined based on the coordinates of the tensor data to be accessed by the first operation as a second coordinate space range.

[0109] In these embodiments, for example, when it is expected that the first operation will use spatial block 41 and spatial block 42 (for example, estimated based on the coordinates of the tensor data to be accessed), the spatial range to be used to perform the first operation (i.e., the second coordinate spatial range) can be determined as spatial block 41 and spatial block 42, as shown in the dotted area.

[0110] Then, according to the embodiment of the present disclosure, the range in which the first operation can be operated when it is actually executed, that is, the range of the third coordinate space, is the intersection of the first coordinate space range and the second coordinate space range. Figure 4A As shown, in the current example, the third coordinate space range is the area with both oblique line shadows and dot fills, that is, Figure 4A The space block 41 in.

[0111] Alternatively or additionally, in some embodiments, within the third coordinate space range, the first operation may be performed based on at least one of the following orders: a predetermined spatial block order; and / or a predetermined spatial coordinate order.

[0112] In some implementations, after the shape coordinate space of the tensor data to be operated is pre-divided into blocks, for example Figure 4A For the four spatial blocks, a spatial block order can be predetermined, that is, the order of operations on each spatial block in the coordinate space, for example, in the order of spatial blocks 41, 42, 43 and 44. In this case, if the operation objects or usage spaces of the two dependent instructions are the entire tensor data, the instructions can be allowed to operate on the spatial blocks one by one in this order. For example, assuming that the previous instruction 1 is to write tensor data and the subsequent instruction 2 is to read the tensor data, then instruction 1 can first write to spatial block 41 and then write to spatial block 42. At this time, instruction 2 can start to read spatial block 41. If the division of spatial blocks makes the execution rhythm of instruction 2 consistent with that of instruction 1, then in the subsequent time, when instruction 1 starts to write to spatial block 43, instruction 2 has also completed the read operation on spatial block 41 and starts to read spatial block 42; and so on. It can be seen from this that the division of spatial blocks is conducive to the parallel execution of instructions, and the agreement on the order of spatial blocks is conducive to simplifying operation scheduling, shortening processing time, and improving processing efficiency.

[0113] Alternatively or additionally, in some implementations, when executing the first operation within a single spatial block, the operation may also be performed in a predetermined spatial coordinate order. When the operating range of the instructions executed in parallel is further controlled based on the coordinate point of the current operation within the single spatial block, this method of executing in a predetermined spatial coordinate order facilitates simplified operation scheduling. The principle behind this method is similar to the principle of executing in a predetermined spatial block order described above and will not be further elaborated here.

[0114] Despite Figure 4A , four equally divided spatial blocks are shown, but the spatial blocks may also be divided into various numbers of unequally sized spatial blocks. The present disclosure has no limitation on the specific division method.

[0115] In some embodiments, the shape coordinate space of the tensor data can be pre-divided based on at least one of: a processing capability of the hardware; a pre-set parameter; and a size of the shape coordinate space of the tensor data. The processing capability of the hardware may, for example, include but is not limited to a data bit width that the hardware can process. Based on the data bit width that the hardware can process, the shape coordinate space of the tensor data can be divided to fully exploit the processing capability of the hardware and improve the parallel processing efficiency. The pre-set parameter may, for example, directly specify a number of space blocks to be divided, a size of each dimension of the space blocks, and the like. Based on the size / dimension of the shape coordinate space of the tensor data, the shape coordinate space of the tensor data can be divided. For example, when the tensor data is a two-dimensional matrix with a size of M rows * N columns (M and N are both positive integers), each row can be divided into m parts on average and each column can be divided into n parts on average, so that a total of m * n space blocks are obtained.

[0116] In some embodiments, the first coordinate space range, the second coordinate space range, and the third coordinate space range can be characterized using the identifiers of the respective included space blocks. For example, in the example shown in FIG. 4, the first coordinate space range can be characterized using the identifiers of the space block 41 and the space block 43; the second coordinate space range can be characterized using the identifiers of the space block 41 and the space block 42; and the third coordinate space range can be characterized using the identifier of the space block 41. Figure 4A In the example shown in FIG. 4, the first coordinate space range can be characterized using the identifiers of the space block 41 and the space block 43; the second coordinate space range can be characterized using the identifiers of the space block 41 and the space block 42; and the third coordinate space range can be characterized using the identifier of the space block 41.

[0117] It is considered that in most cases, the access to the tensor data is usually in a certain dimension, and the coordinates are gradually increased from front to back to traverse the data units at each coordinate point in the tensor data.

[0118] Therefore, in some other embodiments, the first coordinate space range is characterized using an upper bound of a coordinate in one or more dimensions of the tensor data on which the space block or the partial space block allowed to be used by the first operation; and / or the second coordinate space range is characterized using a lower bound of a coordinate in one or more dimensions of the tensor data on which the space block or the partial space block expected to be used by the first operation. By taking advantage of the characteristics of such dimension-ordered access of the tensor data, the first coordinate space range or the second coordinate space range can be characterized using only the upper bound of the coordinate or the lower bound of the coordinate, so that the control information and the corresponding control method can be simplified.

[0119] Figure 4B The characterization of the coordinate space range according to embodiments of the present disclosure is schematically shown. Figure 4B Still taking a two-dimensional data as an example for illustrative purposes, however, it will be appreciated by those skilled in the art that the same scheme can be similarly applied to three-dimensional or even higher-dimensional tensor data.

[0120] As shown in FIG. 4, the first coordinate space range can be characterized using the identifiers of the space block 41 and the space block 43; the second coordinate space range can be characterized using the identifiers of the space block 41 and the space block 42; and the third coordinate space range can be characterized using the identifier of the space block 41. Figure 4BAs shown, the shape coordinate space 400B of the two-dimensional tensor data is divided into 12 space blocks. In each space block, it is guaranteed that multiple instructions with interdependent relationships access the data at each coordinate point therein in sequence. Any data element (e.g., a data point) on the tensor data can be represented by a two-dimensional space coordinate (x, y) (where the X axis is horizontal to the right and the Y axis is vertically downward). Obviously, the coordinates of any data element on the tensor data will not exceed the maximum size of the shape coordinate space 400B.

[0121] As mentioned above, the first coordinate space range can be characterized by the upper bound of the coordinates of the space block that the first operation is allowed to use on one or more dimensions of the tensor data. For example, when the previous operation has completed access to the upper left two rows and three columns, a total of 6 space blocks, and is using the rightmost column and the bottom row, a total of 6 space blocks, the space range allowed to be used by the first operation (that is, the first coordinate space range) can include the upper left two rows and three columns, a total of 6 space blocks, as shown by the diagonal shadow area. At this time, Figure 4B In the example, the first coordinate space range can be represented by an X upper bound 411 on the X axis and a Y upper bound 421 on the Y axis. In this example, the X upper bound and the Y upper bound indicate that the data coordinates accessed by the first operation cannot exceed the X upper bound 312A in the X dimension and cannot exceed the Y upper bound 322A in the Y dimension. It can be seen that these upper bounds correspond to the upper bounds of the spatial blocks that can be used by the first operation in each dimension.

[0122] Similarly, the second coordinate space range can be characterized by the lower bound of the coordinates of the space blocks that the first operation is expected to use on one or more dimensions of the tensor data. For example, when it is determined that the first operation will use space blocks other than the two left space blocks of the first row based on the coordinates of the tensor data to be accessed by the first operation, it can be determined that the second coordinate space range includes the remaining 10 space blocks, as shown in the area of ​​the dot-filled portion. At this time, Figure 4B In the example, the second coordinate space range can be represented by the X lower bound 412 on the X axis and the Y lower bound 422 on the Y axis. In this example, the X lower bound and the Y lower bound indicate that when the first operation is performed, data in the tensor data whose X coordinate is lower than the X lower bound and whose Y coordinate is lower than the Y lower bound will not be accessed. Figure 4B It can be seen that the second coordinate space range corresponds to the range of data in the shape coordinate space 400B whose X coordinate and Y coordinate exceed the corresponding lower bound (X lower bound or Y lower bound). These lower bounds correspond to the lower bounds of the spatial block to be used by the first operation in each dimension.

[0123] The range that can be operated when the first operation is actually executed is the third coordinate space range, which is the intersection of the first coordinate space range and the second coordinate space range. Figure 4BAs shown, in the current example, the third coordinate space range is the area with both oblique line shadows and dot fills, that is, Figure 4B The "reverse L-shaped" area in the

[0124] In some embodiments, each coordinate space range may only restrict some of its dimensions. For example, for two-dimensional tensor data, only the limits of the X dimension or the Y dimension may be restricted.

[0125] The previous description describes determining the first coordinate space range and the second coordinate space range based on a predetermined division of the shape coordinate space of the tensor data. Additionally, other factors can be considered to determine the first coordinate space range and the second coordinate space range. When the spatial block division is large, the access time for operations within a spatial block will also be longer. In this case, more detailed division of the spatial range within a single spatial block can further improve parallel processing efficiency.

[0126] In this embodiment, when a previous operation exists for a spatial block in which only a portion of the coordinates are completed, a partial spatial block consisting of the partial coordinates is dynamically determined based on the operational status of the previous operation, and the partial spatial block is included in the first coordinate space range. Thus, the first coordinate space range may include a complete spatial block in which the previous operation was fully completed, or may include a partial spatial block in which the previous operation was only partially completed.

[0127] Similarly, when a spatial block containing partial coordinates is accessed by the first operation, the partial spatial block consisting of the partial coordinates is dynamically determined based on the operational state of the first operation, and the partial spatial block is included in the second coordinate space range. Thus, the second coordinate space range may include the entire spatial block accessed by the first operation, or may include only a partial spatial block accessed by the first operation.

[0128] Here, the operation status may include at least one of the following information: the completion status of the operation relative to the coordinate points in the spatial block; the execution range of the operation; and the access mode of the operation. For example, for the first coordinate space range, the coordinate points in the spatial block that are previously operated to complete the access can be included in the first coordinate space range. For another example, for the second coordinate space range, the coordinate points in the spatial block that will not be accessed by the first operation can be excluded from the second coordinate space range. The access mode of the operation may include, for example, sequential access, regular access, and so on. Based on the various access modes of the operation, it can be determined accordingly which coordinate points in the spatial block can be included in or excluded from the first and / or second coordinate space ranges. For the case of partial spatial blocks, in addition to using spatial block identifiers for characterization, additional information is required to assist in identifying the coordinate points in the partial spatial blocks. In some embodiments, reference can be made to Figure 4BThe coordinate space range that includes the partial space block is represented by the coordinate upper and lower bounds of the coordinate space. It can be understood that when the coordinate space is accessed continuously in a certain order, the partial space block is usually located at the edge of the coordinate space range to which it belongs. Therefore, the coordinate upper and lower bounds of the complete space block or the partial space block at the edge of the coordinate space range can be used to represent the coordinate space range.

[0129] For example, in an embodiment that uses the coordinate space upper and lower bounds to characterize the coordinate space range, the coordinate space lower bound of the tensor data used in the previous operation or instruction can be used as the upper bound of the tensor data used by the current new instruction.

[0130] In one example, when the first operation (ie, the current operation) is a read operation, the coordinate space upper bound is the coordinate space lower bound of the most recent (ie, the previous operation or the preceding operation) write operation on the tensor data.

[0131] In another example, when the first operation is a write operation, the coordinate space upper bound is the minimum of the coordinate space lower bound of the most recent write operation on the tensor data and the coordinate space lower bounds of all read operations on the tensor data between the two write operations. By selecting the minimum value, it is ensured that the execution of the first operation does not affect the execution of any previous operations.

[0132] Alternatively or additionally, the second coordinate space range can be determined based on at least one of the following: the execution range of the operation; the access mode of the operation; and the current execution state of the operation. For example, in an embodiment where the coordinate space range is characterized by an upper bound and a lower bound of the coordinate space, the second coordinate space range can be determined by comprehensively considering the above factors to ensure that when accessing tensor data according to a dimension, the coordinates on the corresponding dimension are not less than the lower bound of the coordinate space. Furthermore, by providing the maximum value of the lower bound of the coordinate space as much as possible, the accessible space range left for subsequent operations or instructions will be larger.

[0133] In one example, when the access mode of the first operation is sequential and continuous access, the coordinate space lower bound can be determined based on the minimum access coordinate of the first operation. For example, the coordinate space lower bound can be determined as the lower bound of the space block where the minimum access coordinate is located. Figure 4B As shown, when the first operation accesses data according to the X dimension, assuming that the minimum X coordinate of the accessed data is A, which is located in the second spatial block from the left, the X lower bound can be determined as the lower bound of the second spatial block; when the first operation accesses data according to the Y dimension, assuming that the minimum Y coordinate of the accessed data is B, which falls in the third spatial block from the top, the Y lower bound can be determined as the lower bound of the third spatial block.

[0134] In another example, when the access pattern of the first operation is regular access, the coordinate space lower bound can be determined based on the regularity. For example, in a convolution operation, it may be necessary to access tensor data in blocks, so the coordinate space lower bound can be determined based on the block regularity of the convolution operation.

[0135] In another example, when the access mode of the first operation cannot be determined, the coordinate space lower bound may be determined based on a predetermined setting. For example, the coordinate space lower bound may be a default value, such as the size of 0 or 1 or more space blocks.

[0136] Although Figure 4B This section shows how the coordinate bounds are represented when the coordinate space range includes a complete spatial block. However, this representation can be applied similarly to partial spatial blocks. In other words, the coordinate bounds are not the bounds of the complete spatial block, but rather lie within the spatial block. The specific method for determining these bounds is similar to that described above and will not be repeated here.

[0137] The above describes a scheme for constraining the spatial range actually used by operations to ensure sequential consistency of data processing and improve parallel processing efficiency when hardware executes operations in parallel. Those skilled in the art will appreciate that the current operation (e.g., the aforementioned first operation) and the previous operation (or preceding operation) can be operations in different instructions executed in parallel; and the current operation and the previous operation can also be different operations in the same instruction executed in parallel, and this disclosure is not limited in this respect.

[0138] The processing method performed by the processing device of the embodiment of the present disclosure has been described above with reference to the flow chart. Those skilled in the art will understand that, since the operations performed in parallel are constrained based on the coordinate space range of the processed data, it is possible to improve the parallel degree of the operations while ensuring the consistency of the order in which the operations are performed, thereby improving processing efficiency. It should be noted that, for the aforementioned various method embodiments, for the sake of simplicity of description, they are all expressed as a series of action combinations, but those skilled in the art should know that the present disclosure is not limited by the described order of actions, because according to the present disclosure, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also know that the embodiments described in the specification are all optional embodiments, and the actions and modules involved are not necessarily required by the present disclosure.

[0139] It is further noted that, although the steps of the method flowchart are shown in a sequence as indicated by the arrows, the steps need not necessarily be performed in the order as indicated by the arrows. Unless explicitly stated otherwise herein, the steps need not necessarily be performed in the order as indicated by the arrows. The steps can be performed in other orders. Moreover, at least some of the steps of the method flowchart can include multiple sub-steps or multiple stages, which need not necessarily be performed at the same time, but can be performed at different times. The sub-steps or stages of the steps need not necessarily be performed sequentially, but can be performed in rotation or alternation with other steps or sub-steps or stages of other steps.

[0140] Figure 5 is a structural diagram illustrating a combined processing device 500 according to an embodiment of the present disclosure. As shown in Figure 5 The combined processing device 500 includes a computing processing device 502, an interface device 504, other processing devices 506, and a storage device 508. According to different application scenarios, the computing processing device can include one or more computing devices 510, which can be configured to Figure 2 The processing device 200 shown in FIG. 4 can be used to perform the operations described herein in connection with FIG. 4.

[0141] In different embodiments, the computing processing device of the present disclosure can be configured to perform user-specified operations. In exemplary applications, the computing processing device can be implemented as a single-core artificial intelligence processor or a multi-core artificial intelligence processor. Similarly, the one or more computing devices included in the computing processing device can be implemented as an artificial intelligence processor core or a partial hardware structure of an artificial intelligence processor core. When multiple computing devices are implemented as an artificial intelligence processor core or a partial hardware structure of an artificial intelligence processor core, the computing processing device of the present disclosure can be considered to have a single-core structure or a homogeneous multi-core structure.

[0142] In exemplary operation, the computing processing device of the present disclosure can interact with other processing devices through interface means, to jointly complete the operation specified by the user. Depending on the difference in implementation, the other processing devices of the present disclosure may include one or more types of processors in general and / or special processors such as central processing unit (Central Processing Unit, CPU), graphics processing unit (Graphics Processing Unit, GPU), artificial intelligence processor. These processors may include but are not limited to digital signal processor (Digital Signal Processor, DSP), application specific integrated circuit (Application Specific Integrated Circuit, ASIC), field programmable gate array (Field-Programmable Gate Array, FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc., and their number can be determined according to actual needs. As previously mentioned, only with respect to the computing processing device of the present disclosure, it can be regarded as having a single-core structure or a homogeneous multi-core structure. However, when the computing processing device and other processing devices are considered together, the two can be regarded as forming a heterogeneous multi-core structure.

[0143] In one or more embodiments, the other processing device may serve as an interface between the computing device disclosed herein (which may be embodied as an artificial intelligence computing device such as a neural network computing device) and external data and control, performing basic control including but not limited to data transfer, starting and / or stopping the computing device, and so on. In other embodiments, the other processing device may also collaborate with the computing device to jointly complete computing tasks.

[0144] In one or more embodiments, the interface device can be used to transmit data and control instructions between the computing and processing device and other processing devices. For example, the computing and processing device can obtain input data from other processing devices via the interface device and write it to the storage device (or memory) on the computing and processing device chip. Furthermore, the computing and processing device can obtain control instructions from other processing devices via the interface device and write them to the control cache on the computing and processing device chip. Alternatively or optionally, the interface device can also read data from the storage device of the computing and processing device and transmit it to other processing devices.

[0145] Additionally or optionally, the combined processing device of the present disclosure may further include a storage device. As shown in the figure, the storage device is connected to the computing processing device and the other processing device, respectively. In one or more embodiments, the storage device may be used to store data of the computing processing device and / or the other processing device. For example, the data may be data that cannot be fully stored in the internal or on-chip storage device of the computing processing device or other processing device.

[0146] In some embodiments, the present disclosure also discloses a chip (e.g. Figure 6 In one implementation, the chip is a system on chip (SoC) and integrates one or more components such as Figure 5 The chip can be connected to the external interface device (such as Figure 6 The external interface device 606 shown in the figure is connected to other related components. The related components can be, for example, a camera, a display, a mouse, a keyboard, a network card or a wifi interface. In some application scenarios, other processing units (such as video codecs) and / or interface modules (such as DRAM interfaces) can be integrated on the chip. In some embodiments, the present disclosure also discloses a chip packaging structure, which includes the above-mentioned chip. In some embodiments, the present disclosure also discloses a board card, which includes the above-mentioned chip packaging structure. The following will be combined with Figure 6 The board is described in detail.

[0147] Figure 6 FIG. 1 is a schematic diagram showing the structure of a board 600 according to an embodiment of the present disclosure. Figure 6 As shown in , the board includes a storage device 604 for storing data, which includes one or more storage units 610. The storage device can be connected to the control device 608 and the chip 602 described above and transmit data by means of, for example, a bus. Further, the board also includes an external interface device 606, which is configured for data relay or transfer function between the chip (or the chip in the chip packaging structure) and the external device 612 (such as a server or computer, etc.). For example, the data to be processed can be passed to the chip by the external device through the external interface device. For another example, the calculation result of the chip can be transmitted back to the external device via the external interface device. According to different application scenarios, the external interface device can have different interface forms, for example, it can adopt a standard PCIE interface, etc.

[0148] In one or more embodiments, the control device in the disclosed board can be configured to regulate the state of the chip. To this end, in one application scenario, the control device can include a microcontroller unit (MCU) for regulating the working state of the chip.

[0149] According to the above combination Figure 5 and Figure 6 Based on the description, those skilled in the art can understand that the present disclosure also discloses an electronic device or apparatus, which may include one or more of the above-mentioned boards, one or more of the above-mentioned chips and / or one or more of the above-mentioned combined processing devices.

[0150] According to different application scenarios, the electronic devices or devices disclosed herein may include servers, cloud servers, server clusters, data processing devices, robots, computers, printers, scanners, tablet computers, smart terminals, PC devices, Internet of Things terminals, mobile terminals, mobile phones, driving recorders, navigators, sensors, cameras, cameras, video cameras, projectors, watches, headphones, mobile storage, wearable devices, visual terminals, automatic driving terminals, vehicles, household appliances, and / or medical equipment. The vehicles include airplanes, ships and / or vehicles; the household appliances include televisions, air conditioners, microwave ovens, refrigerators, rice cookers, humidifiers, washing machines, electric lights, gas stoves, and range hoods; the medical equipment includes magnetic resonance imaging (MRI), ultrasound machines and / or electrocardiographs. The electronic devices or devices disclosed herein may also be applied to the Internet, Internet of Things, data centers, energy, transportation, public administration, manufacturing, education, power grids, telecommunications, finance, retail, construction sites, medical care and other fields. Furthermore, the electronic devices or devices disclosed herein may also be used in cloud, edge, terminal and other application scenarios related to artificial intelligence, big data and / or cloud computing. In one or more embodiments, electronic devices or apparatuses with high computing power according to the disclosed solution can be applied to cloud devices (such as cloud servers), while electronic devices or apparatuses with low power consumption can be applied to terminal devices and / or edge devices (such as smartphones or cameras). In one or more embodiments, the hardware information of the cloud device and the hardware information of the terminal device and / or edge device are compatible with each other, so that according to the hardware information of the terminal device and / or edge device, appropriate hardware resources can be matched from the hardware resources of the cloud device to simulate the hardware resources of the terminal device and / or edge device, so as to complete the unified management, scheduling and collaborative work of end-to-end or cloud-edge-to-end.

[0151] It should be noted that, for the purpose of clarity, the disclosure describes some methods and embodiments thereof as a series of acts and / or combinations thereof, but those skilled in the art will understand that the present disclosure is not limited to the order of the acts described. Those skilled in the art will understand and appreciate that some steps of the methods can be decided to be executed in other orders or at the same time with other steps. Further, those skilled in the art will understand and appreciate that some of the embodiments described in the disclosure can be considered optional, i.e., the acts or modules involved therein are not necessarily essential for the implementation of one or more of the aspects of the present disclosure. In addition, the disclosure describes some embodiments with different focuses according to different aspects. In view of this, those skilled in the art will understand that the parts not described in detail in some embodiments of the disclosure can also be seen from the relevant description of other embodiments.

[0152] In specific implementation aspects, based on the disclosure and teachings of the present disclosure, those skilled in the art will understand that some of the embodiments disclosed in the present disclosure can also be implemented in other ways not disclosed herein. For example, as for each unit in the electronic device or apparatus embodiments described above, the units are divided herein on the basis of logical functions, and other division manners can also be used in actual implementation. For another example, a plurality of units or components can be combined or integrated into another system, or some features or functions of the units or components can be selectively disabled. As for the connection relationship between different units or components, the connections discussed above in conjunction with the drawings can be direct or indirect coupling between the units or components. In some scenarios, the aforementioned direct or indirect coupling involves a communication connection using an interface, wherein the communication interface can support electrical, optical, acoustic, magnetic or other forms of signal transmission.

[0153] In the present disclosure, the units described as separate components can or can not be physically separated, and the components shown as units can or can not be physical units. The aforementioned components or units can be located in the same place or distributed on a plurality of network units. In addition, according to actual needs, some or all of the units can be selected to achieve the purpose of the aspects described in the embodiments of the present disclosure. In addition, in some scenarios, a plurality of units in the embodiments of the present disclosure can be integrated into one unit or each unit physically exists separately.

[0154] In some implementation scenarios, the above-mentioned integrated unit can be implemented in the form of a software program module. If implemented in the form of a software program module and sold or used as an independent product, the integrated unit can be stored in a computer-readable memory. Based on this, when the scheme of the present disclosure is embodied in the form of a software product (such as a computer-readable storage medium), the software product can be stored in a memory, which may include several instructions to enable a computer device (such as a personal computer, a server or a network device, etc.) to perform some or all of the steps of the method described in the embodiment of the present disclosure. The aforementioned memory may include, but is not limited to, various media that can store program code, such as a USB flash drive, a flash drive, a read-only memory (ROM), a random access memory (RAM), a mobile hard disk, a magnetic disk or an optical disk.

[0155] In some other implementation scenarios, the above-mentioned integrated unit can also be implemented in the form of hardware, that is, a specific hardware circuit, which may include digital circuits and / or analog circuits, etc. The physical implementation of the hardware structure of the circuit may include but is not limited to physical devices, and the physical devices may include but are not limited to devices such as transistors or memristors. In view of this, the various devices described herein (such as computing devices or other processing devices) can be implemented by appropriate hardware processors, such as CPUs, GPUs, FPGAs, DSPs, and ASICs. Furthermore, the aforementioned storage unit or storage device can be any appropriate storage medium (including magnetic storage media or magneto-optical storage media, etc.), which can be, for example, resistive random access memory (RRAM), dynamic random access memory (DRAM), static random access memory (SRAM), enhanced dynamic random access memory (EDRAM), high bandwidth memory (HBM), hybrid memory cube (HMC), ROM and RAM, etc.

[0156] The foregoing content can be better understood in accordance with the following terms:

[0157] Clause 1. A method of treatment, comprising:

[0158] Get the first operation of the decoded instruction;

[0159] determining a first coordinate space range of the tensor data allowed to be used by the first operation;

[0160] determining a second coordinate space range of the tensor data to be used when performing the first operation; and

[0161] Performing the first operation within a third coordinate space range determined by the intersection of the first coordinate space range and the second coordinate space range;

[0162] The first coordinate space range and the second coordinate space range are determined based at least in part on a predetermined partitioning of a shape coordinate space of the tensor data.

[0163] Clause 2. The processing method according to Clause 1, wherein the first coordinate space range and the second coordinate space range are respectively part of the shape coordinate space of the tensor data, and the shape coordinate space is mapped to the data storage area of ​​the tensor data on the storage module.

[0164] Clause 3. The method according to any of Clauses 1-2, further comprising:

[0165] Determining a previous operation that is being executed and that accesses the same tensor data as the first operation and has a dependency relationship with the first operation; and

[0166] The first coordinate space range is determined based at least in part on an operational state of the prior operation.

[0167] Clause 4. The processing method according to clause 3, wherein the shape coordinate space of the tensor data is pre-divided into a plurality of spatial blocks, the method further comprising:

[0168] Determining the space block in the shape coordinate space where the previous operation has been completed as the first coordinate space range; and / or

[0169] A spatial block determined based on the coordinates of the tensor data to be accessed by the first operation is determined as the second coordinate space range.

[0170] Clause 5. The method according to clause 4, further comprising:

[0171] When there is a spatial block in which the previous operation has been completed only with respect to part of the coordinates, determining a partial spatial block consisting of the partial coordinates based on the operation state of the previous operation, and including the partial spatial block in the first coordinate space range; and / or

[0172] When there is a spatial block in which the first operation will access partial coordinates, a partial spatial block composed of the partial coordinates is determined based on an operation state of the first operation, and the partial spatial block is included in the second coordinate space range.

[0173] Clause 6. The processing method according to clause 5, wherein the operating status includes at least one of the following information:

[0174] The completion status of the operation relative to the coordinate point in the spatial block;

[0175] the scope of the operation; and

[0176] The access mode of the operation.

[0177] Clause 7. The processing method according to any one of clauses 5-6, wherein the first coordinate space range and the second coordinate space range are characterized by at least one of the following:

[0178] Identifications of the space blocks respectively included in the first coordinate space range and the second coordinate space range;

[0179] The upper bound of the coordinates of the spatial block or part of the spatial block within the first coordinate space in one or more dimensions of the tensor data; and / or

[0180] The spatial block or partial spatial block within the second coordinate space range is a lower bound of coordinates in one or more dimensions of the tensor data.

[0181] Clause 8. The method according to any of Clauses 4 to 7, further comprising:

[0182] Within the third coordinate space, the first operation is performed based on at least one of the following sequences:

[0183] a predetermined sequence of spatial blocks; and / or

[0184] A predetermined order of spatial coordinates.

[0185] Clause 9. The method according to any one of Clauses 1 to 8, further comprising:

[0186] When the third coordinate space range is empty, the first operation is blocked.

[0187] Clause 10. The processing method according to any one of clauses 1 to 9, wherein the pre-division of the shape coordinate space of the tensor data is performed based on at least one of the following:

[0188] The processing power of the hardware;

[0189] Pre-set parameters; and

[0190] The size of the shape coordinate space of the tensor data.

[0191] Clause 11. A method according to any one of clauses 3 to 10, wherein:

[0192] The first operation and the previous operation are operations in different instructions executed in parallel; or

[0193] The first operation and the previous operation are different operations executed in parallel in the same instruction.

[0194] Clause 12. A processing device comprising:

[0195] an operation acquisition unit configured to acquire a first operation of the decoded instruction;

[0196] a first determining unit configured to determine a first coordinate space range of the tensor data allowed to be used by the first operation;

[0197] a second determining unit configured to determine a second coordinate space range of the tensor data to be used when performing the first operation; and

[0198] an execution unit configured to execute the first operation within a third coordinate space range determined by the intersection of the first coordinate space range and the second coordinate space range;

[0199] The first coordinate space range and the second coordinate space range are determined based at least in part on a predetermined partitioning of a shape coordinate space of the tensor data.

[0200] Clause 13. A processing device according to Clause 12, wherein the first coordinate space range and the second coordinate space range are respectively part of the shape coordinate space of the tensor data, and the shape coordinate space is mapped to the data storage area of ​​the tensor data on the storage unit.

[0201] Clause 14. The processing device according to any one of Clauses 12-13, further comprising:

[0202] A third determining unit is configured to determine a previous operation being executed that accesses the same tensor quantity as the first operation and has a dependency relationship with the first operation; and

[0203] The first determining unit is configured to determine the first coordinate space range based at least in part on an operating state of the preceding operation.

[0204] Clause 15. The processing apparatus according to clause 14, wherein the shape coordinate space of the tensor data is pre-divided into a plurality of space blocks, and

[0205] The first determining unit is further configured to determine, as the first coordinate space range, a space block in the shape coordinate space in which the preceding operation has completed; and / or

[0206] The second determining unit is further configured to determine, as the second coordinate space range, a space block determined based on coordinates of the tensor data to be accessed by the first operation.

[0207] Clause 16. The processing apparatus according to clause 15, wherein:

[0208] The first determining unit is further configured to, when there is a space block in which only part of coordinates have been completed with respect to the preceding operation, determine a partial space block composed of the part of coordinates based on an operation state of the preceding operation, and include the partial space block in the first coordinate space range; and / or

[0209] The second determining unit is further configured to, when there is a space block in which part of coordinates are to be accessed by the first operation, determine a partial space block composed of the part of coordinates based on an operation state of the first operation, and include the partial space block in the second coordinate space range.

[0210] Clause 17. The processing apparatus according to clause 16, wherein the operation state comprises at least one of the following information:

[0211] a completion state of the operation with respect to a coordinate point in the space block;

[0212] an execution range of the operation; and

[0213] an access pattern of the operation.

[0214] Clause 18. The processing apparatus according to any one of clauses 16-17, wherein the first coordinate space range and the second coordinate space range are characterized using at least one of the following:

[0215] an identification of a space block included in each of the first coordinate space range and the second coordinate space range;

[0216] an upper bound of a coordinate in one or more dimensions of the tensor data of a space block or a partial space block in the first coordinate space range; and / or

[0217] a lower bound of a coordinate in one or more dimensions of the tensor data of a space block or a partial space block in the second coordinate space range.

[0218] Clause 19. The processing apparatus according to any one of clauses 15-18, wherein:

[0219] In the third coordinate space range, the first operation is executed based on at least one of the following sequences:

[0220] a predetermined spatial block sequence; and / or

[0221] a predetermined spatial coordinate sequence.

[0222] Clause 20. The processing device according to any one of clauses 12-19, wherein the pre-division of the shape coordinate space of the tensor data is performed based on at least one of:

[0223] a processing capability of hardware;

[0224] a pre-set parameter; and

[0225] a size of the shape coordinate space of the tensor data.

[0226] Clause 21. The processing device according to any one of clauses 14-20, wherein:

[0227] the first operation and the prior operation are operations in different instructions respectively executed in parallel; or

[0228] the first operation and the prior operation are different operations in a same instruction respectively executed in parallel.

[0229] Clause 22. The processing device according to any one of clauses 12-21, wherein the operation acquisition unit and the first determination unit are included in a control module of the processing device, and the second determination unit and the execution unit are included in an operation module of the processing device.

[0230] Clause 23. The processing device according to any one of clauses 12-21, wherein the operation acquisition unit, the first determination unit and the second determination unit are included in a control module of the processing device, and the execution unit is included in an operation module of the processing device.

[0231] Clause 24. A chip, characterized in that the chip comprises the processing device according to any one of clauses 12-23.

[0232] Clause 25. A board card, characterized in that the board card comprises the chip according to clause 24.

Claims

1. A treatment method, comprising: Get the first operation of the decoded instruction; determining a first coordinate space range of the tensor data allowed to be used by the first operation; determining a second coordinate space range of the tensor data to be used when performing the first operation; as well as Performing the first operation within a third coordinate space range determined by the intersection of the first coordinate space range and the second coordinate space range, so as to improve the degree of parallelism of the operation while ensuring the sequential consistency of the operation execution; wherein the first coordinate space range and the second coordinate space range are determined based at least in part on a predetermined partitioning of a shape coordinate space of the tensor data; The shape coordinate space of the tensor data is pre-divided into a number of space blocks; The determining of a first coordinate space range of tensor data allowed to be used by the first operation includes: determining a space block in the shape coordinate space where a previous operation has been completed as the first coordinate space range; The determining of the second coordinate space range of the tensor data to be used when performing the first operation includes: determining a spatial block determined based on the coordinates of the tensor data to be accessed by the first operation as the second coordinate space range.

2. The processing method according to claim 1, wherein the first coordinate space range and the second coordinate space range are respectively part of the shape coordinate space of the tensor data, and the shape coordinate space is mapped to the data storage area of ​​the tensor data on the storage module.

3. The processing method according to any one of claims 1-2, further comprising: Determine a previous operation that is being executed and that accesses the same tensor data as the first operation and has a dependency relationship with the first operation.

4. The processing method according to claim 1, further comprising: When there is a spatial block in which the previous operation has been completed only with respect to a portion of the coordinates, determining a partial spatial block composed of the partial coordinates based on an operation state of the previous operation, and including the partial spatial block in the first coordinate space range; and / or When there is a spatial block in which the first operation will access partial coordinates, a partial spatial block composed of the partial coordinates is determined based on an operation state of the first operation, and the partial spatial block is included in the second coordinate space range.

5. The processing method according to claim 4, wherein the operation status includes at least one of the following information: The completion status of the operation relative to the coordinate point in the spatial block; the scope of the operation; and The access mode of the operation.

6. The processing method according to any one of claims 4 to 5, wherein the first coordinate space range and the second coordinate space range are characterized by at least one of the following: Identifications of the space blocks respectively included in the first coordinate space range and the second coordinate space range; The upper bound of the coordinates of the spatial block or part of the spatial block within the first coordinate space in one or more dimensions of the tensor data; and / or The spatial block or partial spatial block within the second coordinate space range is a lower bound of coordinates in one or more dimensions of the tensor data.

7. The processing method according to claim 1, further comprising: Within the third coordinate space, the first operation is performed based on at least one of the following sequences: a predetermined order of spatial blocks; and / or A predetermined order of spatial coordinates.

8. The processing method according to claim 1, further comprising: When the third coordinate space range is empty, the first operation is blocked.

9. The processing method according to claim 1 , wherein the pre-division of the shape coordinate space of the tensor data is performed based on at least one of the following: The processing power of the hardware; Pre-set parameters; and The size of the shape coordinate space of the tensor data.

10. The processing method according to claim 3, wherein: The first operation and the previous operation are operations in different instructions executed in parallel; or The first operation and the previous operation are different operations executed in parallel in the same instruction.

11. A processing device comprising: an operation acquisition unit configured to acquire a first operation of the decoded instruction; a first determining unit configured to determine a first coordinate space range of the tensor data allowed to be used by the first operation; a second determining unit configured to determine a second coordinate space range of the tensor data to be used when performing the first operation; as well as an execution unit configured to execute the first operation within a third coordinate space range determined by the intersection of the first coordinate space range and the second coordinate space range, so as to improve the degree of parallelism of the operation while ensuring the sequential consistency of the operation execution; wherein the first coordinate space range and the second coordinate space range are determined based at least in part on a predetermined partitioning of a shape coordinate space of the tensor data; The shape coordinate space of the tensor data is pre-divided into several space blocks, and The first determining unit is further configured to: determine a space block in the shape coordinate space where the previous operation has been completed as the first coordinate space range; and / or The second determining unit is further configured to: determine a spatial block determined based on the coordinates of the tensor data to be accessed by the first operation as the second coordinate space range.

12. The processing device according to claim 11, wherein the first coordinate space range and the second coordinate space range are respectively part of a shape coordinate space of the tensor data, and the shape coordinate space is mapped to a data storage area of ​​the tensor data on a storage unit.

13. The processing device according to any one of claims 11-12, further comprising: The third determining unit is configured to determine a previous operation being executed that accesses the same tensor quantity as the first operation and has a dependency relationship with the first operation.

14. The processing device according to claim 11, wherein: The first determining unit is further configured to: when there is a spatial block in which the previous operation is completed only with respect to part of the coordinates, determine a partial spatial block consisting of the partial coordinates based on the operation state of the previous operation, and include the partial spatial block in the first coordinate space range; and / or The second determining unit is further configured to: when there is a spatial block in which the first operation will access partial coordinates, determine a partial spatial block composed of the partial coordinates based on the operation state of the first operation, and include the partial spatial block in the second coordinate space range.

15. The processing device according to claim 14, wherein the operating status includes at least one of the following information: The completion status of the operation relative to the coordinate point in the spatial block; the scope of the operation; and The access mode of the operation.

16. The processing device according to any one of claims 14 to 15, wherein the first coordinate space range and the second coordinate space range are characterized by at least one of the following: Identifications of the space blocks respectively included in the first coordinate space range and the second coordinate space range; The upper bound of the coordinates of the spatial block or part of the spatial block within the first coordinate space in one or more dimensions of the tensor data; and / or The spatial block or partial spatial block within the second coordinate space range is a lower bound of coordinates in one or more dimensions of the tensor data.

17. The processing device according to claim 11, wherein: Within the third coordinate space, the first operation is performed based on at least one of the following sequences: a predetermined sequence of spatial blocks; and / or A predetermined order of spatial coordinates.

18. The processing apparatus according to claim 11, wherein the pre-division of the shape coordinate space of the tensor data is performed based on at least one of the following: The processing power of the hardware; Pre-set parameters; and The size of the shape coordinate space of the tensor data.

19. The processing device according to claim 13, wherein: The first operation and the previous operation are operations in different instructions executed in parallel; or The first operation and the previous operation are different operations executed in parallel in the same instruction. 20 . The processing device according to claim 11 , wherein the operation acquisition unit and the first determination unit are included in a control module of the processing device, and the second determination unit and the execution unit are included in a calculation module of the processing device. 21 . The processing device according to claim 11 , wherein the operation acquiring unit, the first determining unit, and the second determining unit are included in a control module of the processing device, and the executing unit is included in a calculation module of the processing device.

22. A chip, characterized in that: The chip includes the processing device according to any one of claims 11-21.

23. A board, characterized in that: The board includes the chip according to claim 22.

Images