Processing apparatus, processing method and related products

Abstract

The present disclosure discloses a processing device, a processing method and related products. The processing device can be implemented as a computing device included in a combined processing device, which can further include an interface device and other processing devices. The computing device interacts with the other processing devices to jointly complete a user-specified computing operation. The combined processing device can further include a storage device connected to the computing device and the other processing devices respectively, for storing data of the computing device and the other processing devices. The scheme of the present disclosure provides a solution of instruction parallelism, which can improve the instruction parallelism and thus improve the processing efficiency of the machine.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to the field of processors, and in particular, to a processing device, a processing method, a chip and a board card. BACKGROUND

[0002] An instruction system is an interface for computer software and hardware interaction, and is a very important part in computer system structure. With the continuous development of artificial intelligence technology, the amount and dimension of data to be processed are increasing. Therefore, how to reasonably and scientifically control the execution of instructions, especially to improve the degree of instruction parallelism and improve the performance of the machine, is an important problem in the field of processors. SUMMARY

[0003] In order to solve one or more technical problems as mentioned above, the present disclosure proposes a solution to enhance instruction parallelism in multiple aspects. Through the instruction system of the present disclosure, the degree of instruction parallelism can be improved, thereby improving the processing efficiency of the machine.

[0004] In a first aspect, the present disclosure provides a processing device comprising a control unit and an execution unit, wherein: the control unit is configured to: obtain a first operation of an instruction; and send the first operation and information indicating a first coordinate space range of tensor data allowed to be used by the first operation to the execution unit; and the execution unit is configured to: send information indicating a second coordinate space range of the tensor data to be used when executing the first operation to the control unit; and execute the first operation in a third coordinate space range determined by the intersection of the first coordinate space range and the second coordinate space range.

[0005] In a second aspect, the present disclosure provides a chip comprising the processing device of any one of the preceding first aspect.

[0006] In a third aspect, the present disclosure provides a board card comprising the chip of any one of the preceding second aspect.

[0007] In a fourth aspect, the present disclosure provides a processing method comprising: obtaining a first operation of an instruction; determining a first coordinate space range of tensor data allowed to be used by the first operation; determining a second coordinate space range of the tensor data to be used when executing the first operation; and executing the first operation in a third coordinate space range determined by the intersection of the first coordinate space range and the second coordinate space range.

[0008] By means of the processing device, the processing method, the chip and the board card provided as above, the coordinate space range of the tensor data used in the operation during the operation execution of the instruction is limited, so that the consistency of the execution sequence can be ensured and the parallel degree of the operation can be improved when the hardware is executed in parallel, thereby ensuring the accuracy and efficiency of the processing. Further, the use of the coordinate space can simplify the code writing on the software side and facilitate the execution of the instruction. BRIEF DESCRIPTION OF DRAWINGS

[0009] The above and other objects, features and advantages of the exemplary embodiments of the present disclosure will be more apparent from the following detailed description taken in conjunction with the accompanying drawings, in which:

[0010] FIG. 1A A schematic diagram of a data storage space according to an embodiment of the present disclosure is shown;

[0011] FIG. 1B A schematic diagram of data chunking in a data storage space according to an embodiment of the present disclosure is shown;

[0012] FIG. 2 A schematic block diagram of a processing device according to an embodiment of the present disclosure is shown;

[0013] FIGS. 3A-3B A schematic diagram of a coordinate space range according to an embodiment of the present disclosure is shown;

[0014] FIG. 4 A schematic flow chart of a processing method according to an embodiment of the present disclosure is shown;

[0015] FIG. 5 A structural diagram of a combined processing device according to an embodiment of the present disclosure is shown; and

[0016] FIG. 6 A structural schematic diagram of a board card according to an embodiment of the present disclosure is shown. DETAILED DESCRIPTION

[0017] The technical solutions in the embodiments of the present disclosure will be described clearly and completely below in conjunction with the drawings in the embodiments of the present disclosure. Obviously, the described embodiments are part of, rather than all of, the embodiments of the present disclosure. Based on the embodiments in the present disclosure, all other embodiments obtained by those skilled in the art without creative work fall within the scope of protection of the present disclosure.

[0018] It should be understood that the terms "first", "second", "third", and "fourth" and the like in the description and in the claims of the present disclosure are used for distinguishing between similar elements and not necessarily for describing a particular sequential or chronological order. The terms "including", "containing", "comprising" and the like in the description and the claims of the present disclosure are used synonymously to refer to the inclusion of a stated feature, integer, step, operation, element, component or the like to the exclusion of not more than one of the described features, integers, steps, operations, elements, components or the like.

[0019] It should also be understood that the terms used in the specification and the claims of the present disclosure are for the purpose of describing particular embodiments and are not intended to be limiting of the disclosure. As used in the specification and the claims of the present disclosure, the singular forms "a", "an" and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will be further understood that the terms "and / or" as used herein refers to and encompasses any and all possible combinations of one or more of the associated listed items, and that the term "at least one of' is used to mean one, two, three, four, or more than one, or even a finite number of items, and that the term "one or more of' is used to mean one, two, three, four, or more than one, or even a finite number of items.

[0020] As used in the specification and claims, the term "if can be construed to mean "when" or "upon" or "in response to determining" or "in response to detecting" depending on the context. Similarly, the phrase "if it is determined" or "if [a described condition or event] is detected can be construed to mean "upon determining" or "in response to determining" or "upon detecting [the described condition or event]" or "in response to detecting [the described condition or event]".

[0021] A computer processes various data by executing instructions. In order to indicate the source of data, the destination of operation results, and the operation performed, an instruction usually contains the following information:

[0022] (1) Operation code (OP), which is used to represent the operation (e.g., addition, subtraction, multiplication, division, data transfer, etc.) to be completed by the instruction, and specifically describes the nature and function of the operation. A computer can have tens to hundreds of instructions, each of which has a corresponding operation code, and the computer completes different operations by identifying the operation code.

[0023] (2) Operation number, which is used to describe the operation object of the instruction. The operation number can involve the data type of the operation object, the memory address, the addressing mode, etc. The operation number can directly give the operation object, or indicate the memory address or register address (i.e., register name) of the operation object.

[0024] Conventional processor instructions are designed to perform basic single data scalar operations. Here, single data scalar operation refers to an instruction whose each operand is a scalar data. However, with the development of artificial intelligence technology, in tasks such as image processing and pattern recognition, the operand tends to be a multi-dimensional vector (i.e., tensor data) data type, and only using scalar operations cannot make the hardware efficiently complete the operation task. Therefore, how to efficiently perform multi-dimensional tensor data processing is also a problem that needs to be solved in the current computing field.

[0025] In embodiments of the present disclosure, an instruction system is provided, wherein a descriptor is included in the operand of the instruction, through which information related to the tensor data can be quickly obtained. Specifically, the descriptor can indicate at least one of the following information: shape information of the tensor data, spatial information of the tensor data. The shape information of the tensor data can be used to determine the data address of the tensor data corresponding to the operand in the data storage space. The spatial information of the tensor data can be used to determine the dependency relationship between instructions, and in turn can determine, for example, the execution order of the instructions. The spatial information of the tensor data can be indicated by a spatial identifier (ID). The spatial ID can also be referred to as a spatial alias, which refers to a spatial region for storing the corresponding tensor data, which can be a continuous space or multiple spaces, and the present disclosure does not limit the specific form of the spatial region. Different spatial IDs represent that the spatial regions pointed to by the different spatial IDs do not have a dependency relationship. For example, the dependency relationship can be ensured by making the spatial regions pointed to by different spatial IDs not overlap with each other.

[0026] The various possible implementations of the shape information of the tensor data will be described in detail below with reference to the accompanying drawings.

[0027] The tensor can contain various forms of data composition. The tensor can be of different dimensions, such as a scalar can be regarded as a 0-dimensional tensor, a vector can be regarded as a 1-dimensional tensor, and a matrix can be a 2-dimensional or higher-dimensional tensor. The shape of the tensor includes information such as the dimensions of the tensor and the size of each dimension of the tensor. For example, for a three-dimensional tensor:

[0028] x3 = [[[1, 2, 3], [4, 5, 6]]; [[7, 8, 9], [10, 11, 12]]]

[0029] The shape or dimension of the tensor can be represented as X3 = (2, 2, 3), i.e. the tensor is a three-dimensional tensor represented by three parameters, and the size of the first dimension of the tensor is 2, the size of the second dimension is 2, and the size of the third dimension is 3. When storing tensor data in a memory, the shape of the tensor data cannot be determined from its data address (or storage area), and thus the relationship between multiple tensor data and other related information cannot be determined, resulting in low efficiency of the processor in accessing the tensor data.

[0030] In a possible implementation, the shape of the N-dimensional tensor data can be indicated by a descriptor, where N is a positive integer, for example, N = 1, 2, or 3, or zero. The three-dimensional tensor in the above example can be represented by the descriptor as (2, 2, 3). It should be noted that the present disclosure does not limit the way the descriptor indicates the shape of the tensor.

[0031] In a possible implementation, the value of N can be determined according to the dimension (also referred to as the order) of the tensor data, or can be set according to the use requirement of the tensor data. For example, when the value of N is 3, the tensor data is three-dimensional tensor data, and the descriptor can be used to indicate the shape (such as offset, size, etc.) of the three-dimensional tensor data in three dimensions. It should be understood that the value of N can be set according to actual needs by those skilled in the art, and the present disclosure does not limit this.

[0032] Although the tensor data can be multi-dimensional, because the layout of the memory is always one-dimensional, there is a correspondence between the tensor and the storage on the memory. The tensor data is usually allocated in a continuous storage space, i.e. the tensor data can be one-dimensionally unfolded (for example, in a row-major manner) and stored on the memory.

[0033] This relationship between the tensor and the underlying storage can be represented by the offset of the dimension, the size of the dimension, the stride of the dimension, etc. The offset of the dimension refers to the offset of the reference position in this dimension. The size of the dimension refers to the size of the dimension, i.e. the number of elements in this dimension. The stride of the dimension refers to the interval between adjacent elements in this dimension, for example, the stride of the three-dimensional tensor above is (6, 3, 1), i.e. the stride of the first dimension is 6, the stride of the second dimension is 3, and the stride of the third dimension is 1.

[0034] FIG. 1A A schematic diagram of a data storage space according to an embodiment of the present disclosure is shown. As shown in FIG. 1, the data storage space includes a memory 100, a tensor data 101, and a descriptor 102. The tensor data 101 is stored in the memory 100, and the descriptor 102 is used to indicate the shape of the tensor data 101. FIG. 1AAs shown, the data storage space 21 stores a two-dimensional data in a row-major manner, which can be represented by (x, y) (wherein the X-axis is horizontally to the right, and the Y-axis is vertically downward). The size in the X-axis direction (the size of each row, or the total number of columns) is ori_x (not shown in the figure), the size in the Y-axis direction (the total number of rows) is ori_y (not shown in the figure), and the starting address PA_start (reference address) of the data storage space 21 is the physical address of the first data block 22. The data block 23 is part of the data in the data storage space 21, and the offset 25 in the X-axis direction is offset_x, the offset 24 in the Y-axis direction is offset_y, the size in the X-axis direction is size_x, and the size in the Y-axis direction is size_y.

[0035] In a possible implementation, when the data block 23 is defined using the descriptor, the data reference point of the descriptor can use the first data block of the data storage space 21, and it can be agreed that the reference address of the descriptor is the starting address PA_start of the data storage space 21. Then the content of the descriptor of the data block 23 can be determined in combination with the size ori_x of the data storage space 21 in the X-axis direction, the size ori_y in the Y-axis direction, and the offset offset_y of the data block 23 in the Y-axis direction, the offset offset_x in the X-axis direction, the size size_x in the X-axis direction, and the size size_y in the Y-axis direction.

[0036] In a possible implementation, the content of the descriptor can be represented by the following formula (1):

[0037]

[0038] It should be understood that, although in the above example, the content of the descriptor represents a two-dimensional space, a person skilled in the art can set the specific dimension represented by the content of the descriptor according to actual conditions, and the present disclosure does not limit this.

[0039] In a possible implementation, it can be agreed that the data reference point of the descriptor is at a reference address in the data storage space, and on the basis of the reference address, the content of the descriptor of the tensor data is determined according to the positions of at least two vertices at the diagonal positions in the N-dimensional directions relative to the data reference point.

[0040] For example, a data reference point of the descriptor can be agreed to have a reference address PA_base in the data storage space. For example, a data (e.g., the data at position (2, 2)) in the data storage space 21 can be selected as the data reference point, and a physical address of the data in the data storage space can be taken as the reference address PA_base. The content of the descriptor of the data block 23 can be determined according to positions of at least two vertices of the diagonal position relative to the data reference point, for example, positions of vertices of the diagonal position in the left-up to right-down direction relative to the data reference point, wherein a relative position of the left-up corner vertex is (x_min, y_min), and a relative position of the right-down corner vertex is (x_max, y_max). Then, the content of the descriptor of the data block 23 can be determined according to the reference address PA_base, the relative position (x_min, y_min) of the left-up corner vertex, and the relative position (x_max, y_max) of the right-down corner vertex. FIG. 1A The content of the descriptor of the data block 23 can be determined according to positions of at least two vertices of the diagonal position relative to the data reference point, for example, positions of vertices of the diagonal position in the left-up to right-down direction relative to the data reference point, wherein a relative position of the left-up corner vertex is (x_min, y_min), and a relative position of the right-down corner vertex is (x_max, y_max). Then, the content of the descriptor of the data block 23 can be determined according to the reference address PA_base, the relative position (x_min, y_min) of the left-up corner vertex, and the relative position (x_max, y_max) of the right-down corner vertex.

[0041] In a possible implementation, the content of the descriptor (the reference address is PA_base) can be represented by the following formula (2):

[0042]

[0043] It should be understood that, although the content of the descriptor is determined by using the left-up corner and the right-down corner in the above examples, a person skilled in the art can set specific vertices of at least two vertices of the diagonal position according to actual needs, and the disclosure does not limit the specific vertices.

[0044] In a possible implementation, the content of the descriptor of the tensor data can be determined according to a reference address of a data reference point of the descriptor in the data storage space, and a mapping relationship between a data description position and a data address of the tensor data indicated by the descriptor. The mapping relationship between the data description position and the data address can be set according to actual needs. For example, when the tensor data indicated by the descriptor is three-dimensional space data, a function f(x, y, z) can be used to define the mapping relationship between the data description position and the data address.

[0045] In a possible implementation, the content of the descriptor can be represented by the following formula (3):

[0046]

[0047] In a possible implementation, the descriptor is also used to indicate an address of the N-dimensional tensor data, wherein the content of the descriptor further comprises at least one address parameter representing the address of the tensor data, for example, the content of the descriptor can be as follows (4):

[0048] D:

[0049] wherein PA is the address parameter. The address parameter can be a logical address or a physical address. When the descriptor is parsed, the PA can be any one of a vertex, a middle point or a preset point of a vector shape, and the corresponding data address can be obtained in combination with the shape parameters in the X direction and the Y direction.

[0050] In a possible implementation, the address parameter of the tensor data comprises a reference address of a data reference point of the descriptor in a data storage space of the tensor data, and the reference address comprises a start address of the data storage space.

[0051] In a possible implementation, the descriptor can further comprise at least one address parameter representing the address of the tensor data, for example, the content of the descriptor can be as follows (5):

[0052] D:

[0053] wherein PA_start is the reference address parameter, which will not be repeated.

[0054] It should be understood that a person skilled in the art can set the mapping relationship between the data description position and the data address according to actual conditions, and the present disclosure does not limit this.

[0055] In a possible implementation, a reference address can be set in a task, and the descriptor in the instruction of the task uses the reference address, and the shape parameter based on the reference address can be included in the content of the descriptor. The reference address can be determined by setting an environment parameter of the task. For related descriptions and use modes of the reference address, refer to the above embodiments. In this implementation, the content of the descriptor can be mapped to the data address more quickly.

[0056] In a possible implementation, the reference address can be included in the content of each descriptor, and the reference addresses of the descriptors can be different. Compared with the manner of setting a common reference address by using an environment parameter, the descriptors in this manner can describe data more flexibly and use a larger data address space.

[0057] In a possible implementation, the data address of the data corresponding to the operand of the processing instruction in the data storage space can be determined according to the content of the descriptor. The calculation of the data address is automatically completed by hardware, and the calculation method of the data address is different when the representation of the content of the descriptor is different. The specific calculation method of the data address is not limited in the disclosure.

[0058] For example, the content of the descriptor in the operand is represented by formula (1), the offsets of the tensor data indicated by the descriptor in the data storage space are offset x and offset y respectively, and the size is size x * size y. The starting data address PA1 of the tensor data indicated by the descriptor in the data storage space is (x，y) The following formula (6) can be used to determine:

[0059] PA1 (x，y) = PA start + (offset y - 1) * ori x + offset x (6)

[0060] The data starting address PA1 determined according to the above formula (6) (x，y) , combined with the offsets offset x and offset y, and the size size x and size y of the storage region, the storage region of the tensor data indicated by the descriptor in the data storage space can be determined.

[0061] In a possible implementation, when the operand further includes a data description position for the descriptor, the data address of the data corresponding to the operand in the data storage space can be determined according to the content of the descriptor and the data description position. In this way, part of the data (for example, one or more data) in the tensor data indicated by the descriptor can be processed.

[0062] For example, the content of the descriptor in the operand is represented by formula (2), the offsets of the tensor data indicated by the descriptor in the data storage space are offset x and offset y respectively, the size is size x * size y, and the data description position for the descriptor included in the operand is (x q , y q ). The data address PA2 of the tensor data indicated by the descriptor in the data storage space is (x，y) The following formula (7) can be used to determine:

[0063] PA2 (x，y) = PA start + (offset y + y q - 1) * ori x + (offset x + x q ) (7)

[0064] In a possible implementation, the descriptor can indicate the data blocks. Data blocks can effectively speed up the operation and improve the processing efficiency in many applications. For example, in graphics processing, convolution operation often uses data blocks for fast operation processing.

[0065] FIG. 1B A schematic diagram of data blocks in a data storage space according to an embodiment of the present disclosure is shown. As shown, the data storage space 26 also stores two-dimensional data in a row-major manner, which can be represented by (x, y) (where the X-axis is horizontal to the right, and the Y-axis is vertical downward). The size in the X-axis direction (the size of each row, or the total number of columns) is ori_x (not shown in the figure), and the size in the Y-axis direction (the total number of rows) is ori_y (not shown in the figure). Unlike the tensor data, FIG. 1B FIG. 1A The tensor data stored in the data storage space 26 includes a plurality of data blocks. FIG. 1B

[0066] In this case, the descriptor needs more parameters to represent these data blocks. Taking the X-axis (X dimension) as an example, the following parameters can be involved: ori_x, x.tile.size (the size in the block 27), x.tile.stride (the step in the block, that is, the distance between the first point of the first small block and the first point of the second small block), x.tile.num (the number of blocks, which is 3 in the example shown in the figure), x.stride (the overall step, that is, the distance between the first point of the first row and the first point of the second row), and the like. Other dimensions can similarly include corresponding parameters. FIG. 1B

[0067] In a possible implementation, the descriptor can include an identifier of the descriptor and / or content of the descriptor. The identifier of the descriptor is used to distinguish the descriptor, for example, the identifier of the descriptor can be its number; the content of the descriptor can include at least one shape parameter representing the shape of the tensor data. For example, the tensor data is three-dimensional data, and in the three dimensions of the tensor data, the shape parameters of two dimensions are fixed, and the content of the descriptor of the tensor data can include a shape parameter representing the shape of the other dimension of the tensor data.

[0068] In a possible implementation, the data address of the data storage space corresponding to each descriptor can be a fixed address. For example, a separate data storage space can be allocated for the tensor data, and the starting address of each tensor data in the data storage space corresponds to the descriptor one by one. In this case, the circuit or module responsible for parsing the computing instruction (for example, an entity outside the computing device of the present disclosure) can determine the data address of the data corresponding to the operand in the data storage space according to the descriptor. ​​​

[0069] In a possible implementation, when the data address of the data storage space corresponding to the descriptor is a variable address, the descriptor can also be used to indicate the address of the N-dimensional tensor data, wherein the content of the descriptor can further include at least one address parameter representing the address of the tensor data. For example, the tensor data is 3-dimensional data, and when the descriptor points to the address of the tensor data, the content of the descriptor can include one address parameter representing the address of the tensor data, for example, the starting physical address of the tensor data, or can include multiple address parameters of the address of the tensor data, for example, the starting address of the tensor data + address offset, or the address parameters of the tensor data based on each dimension. A person skilled in the art can set the address parameter according to actual needs, and the present disclosure does not limit this.

[0070] In a possible implementation, the address parameter of the tensor data can include the reference address of the data reference point of the descriptor in the data storage space of the tensor data. The reference address can be different according to the change of the data reference point. The present disclosure does not limit the selection of the data reference point.

[0071] In a possible implementation, the reference address can include the starting address of the data storage space. When the data reference point of the descriptor is the first data block of the data storage space, the reference address of the descriptor is the starting address of the data storage space. When the data reference point of the descriptor is other data in the data storage space except the first data block, the reference address of the descriptor is the address of the data block in the data storage space.

[0072] In a possible implementation, the shape parameter of the tensor data includes at least one of the following: the size of the data storage space in at least one direction of the N-dimensional direction, the size of the storage region in at least one direction of the N-dimensional direction, the offset of the storage region in at least one direction of the N-dimensional direction, the position of at least two vertices at the diagonal position of the N-dimensional direction relative to the data reference point, and the mapping relationship between the data description position of the tensor data indicated by the descriptor and the data address. The data description position is the mapping position of the point or region in the tensor data indicated by the descriptor. For example, when the tensor data is 3-dimensional data, the descriptor can use a three-dimensional coordinate space (x, y, z) to represent the shape of the tensor data, and the data description position of the tensor data can be the position of the point or region in the three-dimensional space mapped by the tensor data represented by the three-dimensional space coordinates (x, y, z).

[0073] It should be understood that a person skilled in the art can select the shape parameter representing the tensor data according to actual conditions, and the present disclosure does not limit this. By using the descriptor in the data access process, the association between data can be established, thereby reducing the complexity of data access and improving the instruction processing efficiency.

[0074] When hardware executes instructions in parallel, dependencies between these instructions can lead to incorrect results. For example, if two parallel instructions access the same memory location or the same data, and at least one of them writes to that memory location, a dependency exists between them—such as a write-after-read dependency, a write-after-write dependency, or a read-after-write dependency. In this case, if the later instruction executes before the earlier one, an error will occur. Therefore, it is essential to ensure the consistency of the execution order of these instructions. This can be achieved, for example, by forcing sequential execution, meaning that the later instruction must wait for the earlier instruction to complete before it can execute.

[0075] As described above, tensor data is typically a multi-dimensional array with a large data volume. Therefore, instruction processing time for tensor data is usually longer than that for scalar data. If tensor data is still processed sequentially as before, the processing time will be excessively long and inefficient. Therefore, this embodiment provides an instruction parallelism scheme, in which the parallel execution of operations is limited based on the coordinate space range of the tensor data used by the instruction operation. This ensures both consistency of execution order and increases the degree of parallelism during hardware parallel execution, thereby ensuring both processing accuracy and efficiency.

[0076] FIG. 2 A schematic block diagram of a processing apparatus according to an embodiment of this disclosure is shown. FIG. 2 As shown, the processing device 200 includes a control unit 210 and an execution unit 220.

[0077] The control unit 210 can be configured to control the operation of the processing device 200, such as reading instructions from memory or external sources, decoding the instructions via the decoder 211, and sending micro-operation control signals to corresponding components. Specifically, the control unit 210 can be configured to control the execution unit 220 to perform corresponding processing based on received instructions. Instructions may include, but are not limited to, data access instructions, arithmetic instructions, descriptor management instructions, and synchronization instructions. This disclosure does not limit the specific type of instructions or the specific method of decoding.

[0078] The decoded instruction includes an opcode and operands. When the instruction involves processing tensor data, at least one operand of the instruction may include at least one descriptor indicating at least one of the following: shape information and spatial information of the tensor data.

[0079] The execution unit 220 is configured to execute specific instructions or operations under the control of the control unit 210. The execution unit 220 can include, but is not limited to, an arithmetic and logic unit (ALU), a memory access unit (MAU), a neural functional unit (NFU), etc. FIG. 2 Two functional units 221 and 222 are shown in the figure for illustration. The present disclosure does not limit the specific hardware type of the execution unit.

[0080] The processing apparatus 200 can further include a storage unit 230, which can be configured to store various information including, but not limited to, instructions, descriptor-associated information, tensor data, etc. The storage unit 230 can include various storage resources including, but not limited to, internal memory and external memory. The internal memory can include, for example, registers, on-chip SRAM or other media cache. The external memory can include, for example, off-chip memory. The present disclosure does not limit the specific implementation of the storage unit.

[0081] In some embodiments, the control unit 210 can be configured to: obtain a first operation of an instruction; and send, to the execution unit 220, the first operation and information indicating a first coordinate space range of tensor data allowed to be used by the first operation.

[0082] It should be noted that the operations involved in the present disclosure can be basic operations supported by the processor hardware, or micro-instructions (such as request signals, etc.) parsed from the basic operations. The present disclosure does not limit the specific type of the operations. The processing apparatus of the present disclosure can execute two operations in parallel, or more than two operations in parallel, and the present disclosure does not limit the number of operations executed in parallel. The two operations executed in parallel can belong to the same instruction, or belong to different instructions, and the present disclosure does not limit in this respect.

[0083] In some embodiments, the execution unit 220 can be configured to: send, to the control unit 210, information indicating a second coordinate space range of the tensor data to be used when executing the first operation. Further, the execution unit 220 can be configured to execute the first operation within a third coordinate space range determined by the intersection of the first coordinate space range and the second coordinate space range.

[0084] In some embodiments, when the third coordinate space range is empty, the control unit 210 can block the first operation. By blocking the subsequent operations, the multiple operations with a dependency relationship can be forced to be executed in a predetermined order, thereby ensuring the correctness of the results.

[0085] In embodiments of the present disclosure, by limiting the coordinate space range of the tensor data that can be used when the operation is performed, for example, by limiting the operation to be performed within the third coordinate space range as described above, it can be ensured that the access of the instruction on each coordinate space range is sequential when the parallel execution of the instruction is performed, thereby ensuring the accuracy and efficiency of the processing.

[0086] In some embodiments, the first operation described above involves processing of the tensor data. Accordingly, the first coordinate space range and the second coordinate space range can be respectively a part of the shape coordinate space of the corresponding dimension of the tensor data. The shape coordinate space is mapped to the data storage area of the tensor data on the storage unit 230. By dividing the shape coordinate space of the tensor data into a plurality of coordinate space ranges, and restricting the parallel execution of the instruction based on the limitation of the coordinate space ranges (for example, the coordinate space ranges of the preceding and subsequent operations do not overlap), the parallelism of the processing is improved, and the processing time is reduced.

[0087] As described above with respect to the form information of the tensor data, the data description position can be used in the descriptor to indicate the mapping position of the point or region in the tensor data. For example, when the tensor data is 3-dimensional data, the descriptor can use three-dimensional space coordinates (x, y, z) to represent the shape of the tensor data, and the data description position of the tensor data can be the position of the point or region in the three-dimensional space to which the tensor data is mapped, which is represented using three-dimensional space coordinates (x, y, z). Therefore, in embodiments of the present disclosure, the first, second, and third coordinate space ranges can also be characterized by such data description positions, and further can be characterized by N-dimensional space coordinates. Since the programming on the software side usually uses space coordinates to refer to the data points or data blocks in the tensor data, by restricting the parallel execution of the operation by the coordinate space range of the tensor data, the code programming on the software side can be simplified, and the execution of the instruction is more conducive.

[0088] In most cases, the access to the tensor data is usually according to a certain dimension, and the data units on each coordinate point in the tensor data are traversed from front to back with gradually increasing coordinates. Therefore, in some embodiments, the information indicating the first coordinate space range can include the upper bound of the coordinate space of one or more dimensions of the tensor data that is allowed to be used by the first operation. Alternatively or additionally, in some embodiments, the information indicating the second coordinate space range can include the lower bound of the coordinate space of one or more dimensions of the tensor data that is expected to be used by the first operation. By taking advantage of the characteristic of such dimension-ordered access of the tensor data, only the upper bound of the coordinate space or the lower bound of the coordinate space can be used to characterize the first coordinate space range or the second coordinate space range, thereby simplifying the control information and the corresponding control method.

[0089] FIGS. 3A-3BA mechanism for judging a coordinate space range is illustratively shown. FIG. 3A The example is shown with two-dimensional data, however, it is understood by those skilled in the art that the same scheme can be similarly applied to three-dimensional or even higher dimensional tensor data.

[0090] As shown in FIG. 3A , a two-dimensional tensor data occupies a data storage area 300A on a storage unit, which is stored in a row-major manner. The size of the data storage area 300A in the X-axis direction is represented by an X maximum coordinate 311A, and the size in the Y-axis direction is represented by a Y maximum coordinate 321A. Any data element (e.g., data point) on the tensor data can be represented by a two-dimensional space coordinate (x, y) (where the X-axis is horizontal to the right, and the Y-axis is vertical downward). Obviously, the coordinate of any data element on the tensor data will not exceed the X maximum coordinate 311A and the Y maximum coordinate 321A.

[0091] As mentioned before, the information indicating the first coordinate space range that the first operation is allowed to use, sent by the control unit 210, can be the coordinate space upper bound of one or more dimensions of the tensor data that the first operation is allowed to use. For example, in FIG. 3A , the first coordinate space range can be characterized by an X upper bound 312A on the X-axis and a Y upper bound 322A on the Y-axis. In this example, the X upper bound and the Y upper bound indicate that the data coordinate accessed by the first operation cannot exceed the X upper bound 312A in the X dimension, and cannot exceed the Y upper bound 322A in the Y dimension. FIG. 3A The first coordinate space range is shown with diagonal shading in

[0092] The information indicating the second coordinate space range that will be used when executing the first operation, sent by the execution unit 220, can be the coordinate space lower bound of one or more dimensions of the tensor data that the first operation is expected to use. For example, in FIG. 3A , the second coordinate space range can be characterized by an X lower bound 313A on the X-axis and a Y lower bound 323A on the Y-axis. In this example, the X lower bound and the Y lower bound indicate that, when executing the first operation, the data whose X coordinate is lower than the X lower bound and whose Y coordinate is lower than the Y lower bound in the tensor data will not be accessed. FIG. 3A The second coordinate space range is shown with dot filling in FIG. 3A It can be seen that the second coordinate space range corresponds to the range of data in the data storage area 300A whose X coordinate and Y coordinate either exceed the corresponding lower bound (X lower bound or Y lower bound).

[0093] The range that the first operation can actually operate on is the third coordinate space range, which is the intersection of the first coordinate space range and the second coordinate space range. As FIG. 3AAs shown, in the current example, the third coordinate space range is the area where both the diagonal line shading and the dot filling exist, i.e. FIG. 3B the "anti-L-shaped" area in

[0094] In some embodiments, the information indicating the above-mentioned respective coordinate space ranges can only indicate information in one dimension.

[0095] FIG. 3B Still taking the two-dimensional data as an example for illustrative purposes, but only information in one dimension, e.g. the Y-axis, is given.

[0096] Similarly, as shown in FIG. 3B , the two-dimensional tensor data occupies a data storage area 300B on the storage unit, which is stored in a row-major manner. The size of the data storage area 300B in the X-axis direction is represented by an X maximum coordinate 311B, and the size in the Y-axis direction is represented by a Y maximum coordinate 321B.

[0097] In this example, the information indicating the first coordinate space range that the first operation is allowed to use only gives the upper bound of the coordinate space in one dimension (e.g. the Y-axis) of the tensor data. For example, in FIG. 3B , the first coordinate space range is only represented by a Y upper bound 322B on the Y-axis. At this time, the other dimension (e.g. the X-axis) cannot be determined, and therefore can not be limited. In this example, the Y upper bound indicates that the data coordinates accessed by the first operation cannot exceed the Y upper bound 322B in the Y dimension. FIG. 3B The first coordinate space range is shown in diagonal line shading in

[0098] In this example, the information indicating the second coordinate space range that will be used when the first operation is executed also only gives the lower bound of the coordinate space in one dimension (e.g. the Y-axis) of the tensor data that is expected to be used by the first operation. For example, in FIG. 3B , the second coordinate space range is only represented by a Y lower bound 323B on the Y-axis. At this time, the other dimension (e.g. the X-axis) cannot be determined, and therefore can not be limited. In this example, the Y lower bound indicates that, when the first operation is executed, data whose Y coordinate is lower than the Y lower bound in the tensor data will not be accessed. FIG. 3B The second coordinate space range is shown in dot filling in FIG. 3B It can be seen that the second coordinate space range corresponds to the range of data in the data storage area 300B whose Y coordinate exceeds the Y lower bound.

[0099] In this example, the range that the first operation can actually operate in, i.e. the third coordinate space range, is the intersection of the first coordinate space range and the second coordinate space range. As FIG. 3BAs shown, in the current example, the third coordinate space range is a region that has both diagonal line shading and dot filling, i.e. FIG. 3A the rectangular region in the middle.

[0100] Those skilled in the art can understand that, when the first coordinate space range and the second coordinate space range are represented by the upper bound of the coordinate space and the lower bound of the coordinate space, it is intended to provide a range space as large as possible that is available for the current operation. In this regard, the first coordinate space range can be the upper bound of each coordinate dimension (for example, the maximum X coordinate and the maximum Y coordinate in FIG. 3B and FIG. 3A ), or can be smaller than the upper bound; the second coordinate space range can be the lower bound of each coordinate dimension (for example, the coordinate origin in FIG. 3B and FIG. 3A ), or can be larger than the lower bound.

[0101] Although the above describes an embodiment in which the first coordinate space range and the second coordinate space range are represented by the upper bound of the coordinate space and the lower bound of the coordinate space, those skilled in the art can understand that other ways can also be adopted to represent the first coordinate space range and the second coordinate space range, and the present disclosure is not limited in this regard.

[0102] The first coordinate space range and the second coordinate space range can be determined in various ways.

[0103] In some embodiments, the first and second coordinate space ranges can be dynamically determined according to the execution of the operation. In some implementations, the first coordinate space range can be determined based on at least one of the following: the order of the operation; the operand involved in the operation; information indicating the second coordinate space range of the preceding operation received from the execution unit; and the pre-partition of the shape coordinate space of the tensor data. In some embodiments, the preceding operation refers to an operation that is being executed, accesses the same tensor data as the first operation, and has a dependency relationship. As mentioned earlier, there can be three types of dependency relationships between the current operation and the preceding operation, such as a write-after-read dependency, a write-after-write dependency, or a read-after-write dependency. At this time, the order consistency of the execution of these instructions must be guaranteed, so the first coordinate space range that can be used by the current operation (here, the first operation) can be determined based on the operation range, operation state, etc. of the preceding operation. In yet other embodiments, for example, in embodiments in which the coordinate space range is represented by the upper bound and lower bound of the coordinate space, the lower bound of the coordinate space of the tensor data used by the preceding operation can be used as the upper bound of the coordinate space of the tensor data used by the current new instruction.

[0104] In one example, when the first operation (i.e. the current operation) is a read operation, the upper bound of the coordinate space is determined by the lower bound of the coordinate space of the write operation (i.e. the previous operation or the preceding operation) that is not ended yet. Preferably, the minimum value among the relevant lower bounds is determined.

[0105] In another example, when the first operation is a write operation, the upper bound of the coordinate space is determined by the lower bound of the coordinate space of the write operation that is not ended yet and the lower bound of the coordinate space of the read operation. Preferably, the minimum value among the relevant lower bounds is determined. By selecting the minimum value, it can be ensured that the execution of the first operation will not affect the execution of any preceding operation.

[0106] Optionally or additionally, the second coordinate space range can be determined based on at least one of: the execution range of the operation; the access pattern of the operation; the current execution state of the operation; and the pre-partition of the shape coordinate space of the tensor data. For example, in the embodiment where the coordinate space range is characterized by the upper bound and the lower bound of the coordinate space, the second coordinate space range can be determined by taking into account the above factors, to ensure that when the tensor data is accessed in a dimension, the coordinate in the corresponding dimension is not less than the lower bound of the coordinate space. Further, the maximum value of the lower bound of the coordinate space is provided as much as possible, so that the accessible space range of the subsequent operation or instruction will also be larger.

[0107] In one example, when the access pattern of the first operation is sequential and continuous access, the lower bound of the coordinate space can be determined based on the minimum access coordinate of the subsequent access of the first operation. For example, the lower bound of the coordinate space can be determined as the minimum access coordinate - 1. As shown in FIG. 3B For example, when the first operation accesses data in the X dimension, assuming that the minimum X coordinate of the accessed data is A, the X lower bound can be determined as A - 1; when the first operation accesses data in the Y dimension, assuming that the minimum Y coordinate of the accessed data is B, the Y lower bound can be determined as B - 1.

[0108] In another example, when the access pattern of the first operation is regular access, the lower bound of the coordinate space can be determined based on the regularity. For example, in a convolution operation, it can be necessary to access the tensor data in blocks, and therefore the lower bound of the coordinate space can be determined according to the block regularity of the convolution operation.

[0109] In yet another example, when the access pattern of the first operation cannot be determined, the lower bound of the coordinate space can be determined based on a predetermined setting. For example, the lower bound of the coordinate space can be a default value. In FIG. 2 For example, the X lower bound of the X dimension is 0 by default.

[0110] The first and second coordinate space ranges mentioned above can also be determined in combination with a pre-partition of the shape coordinate space of the tensor data. Specifically, the shape coordinate space of the tensor data can be first partitioned into a plurality of spatial blocks, for example, uniformly or non-uniformly in each dimension. Then, within each spatial block, the first and second coordinate space ranges can be dynamically determined based on the execution of the operation. The specific determination manner can refer to the foregoing description, which will not be repeated here. At this time, the shape coordinate space 300A, 300B, etc. in the foregoing embodiments can be replaced by the region of the current spatial block accordingly. In these implementations, when the exact position of the second coordinate space range within a spatial block cannot be determined, the range corresponding to the spatial block can be defaulted.

[0111] In some embodiments, the pre-partition of the shape coordinate space of the tensor data can be based on at least one of: the processing capability of the hardware; a pre-set parameter; and the size of the shape coordinate space of the tensor data. The processing capability of the hardware may, for example, include but is not limited to the data bit width that the hardware can process. Based on the data bit width that the hardware can process, the shape coordinate space of the tensor data is partitioned, which can fully exploit the processing capability of the hardware and improve the parallel processing efficiency. The pre-set parameter may, for example, directly specify the number of spatial blocks to be partitioned, the size of each dimension of the spatial blocks, etc. Based on the size / dimension of the shape coordinate space of the tensor data, the shape coordinate space of the tensor data is partitioned. For example, when the tensor data is a two-dimensional matrix, the size of which is M rows * N columns (M, N are both positive integers), each row can be divided into m parts on average and each column can be divided into n parts on average, so as to total m*n spatial blocks.

[0112] By this way of dynamically determining the coordinate space ranges, the potential parallelism between operations can be exploited as much as possible, thereby improving the processing efficiency.

[0113] The above describes a scheme for restricting the spatial range actually used by an operation when the operation is executed in parallel by hardware, so as to ensure the sequential consistency of data processing while improving the parallel processing efficiency. Those skilled in the art can understand that the current operation (for example, the first operation described above) and the prior operation (or the preceding operation) can be operations in different instructions executed in parallel; and the current operation and the prior operation can also be different operations in the same instruction executed in parallel, which are not limited by the present disclosure in this respect.

[0114] Returning to FIG. 2Optionally or additionally, the processing apparatus 200 can further comprise a tensor interface unit (TIU) 240. The tensor interface unit 240 can be configured to implement operations associated with descriptors under the control of the control unit 210. These operations can include, but are not limited to, registration, modification, deregistration, resolution of descriptors; read and write of descriptor contents, etc. The present disclosure does not limit the specific hardware type of the tensor interface unit. In this way, the operations associated with descriptors can be implemented by dedicated hardware, further improving the access efficiency of tensor data.

[0115] In some embodiments of the present disclosure, the tensor interface unit 240 can be configured to resolve a descriptor included in an operand of an instruction. For example, the tensor interface unit can resolve shape information of tensor data included in the descriptor, so as to determine, for example, the aforementioned second coordinate space range.

[0116] Although the control unit 210 and the tensor interface unit 240 are shown as two separate modules in FIG. 4 , those skilled in the art can understand that the two units can also be implemented as one module or more modules, and the present disclosure does not limit in this respect.

[0117] The data processing apparatus 200 can be implemented by a general-purpose processor (such as a central processing unit CPU, a graphics processing unit GPU) and / or a special-purpose processor (such as an artificial intelligence processor, a scientific computing processor, or a digital signal processor, etc.), and the present disclosure does not limit the specific type of the data processing apparatus.

[0118] FIG. 2 An exemplary flowchart of a processing method 400 according to an embodiment of the present disclosure is shown. The processing method 400 can be implemented, for example, by the processing apparatus 200 of FIG. 4 .

[0119] As shown in FIG. 2 , the method 400 starts at step S410, obtaining a first operation of an instruction. This step can be performed, for example, by the control unit 210 of FIG. 2 . In some embodiments, the first operation can involve processing of tensor data.

[0120] Next, in step S420, a first coordinate space range of a number of tensors allowed to be used by the first operation is determined. This step can be performed, for example, by the control unit 210 of FIG. 2 . The first coordinate space range can be, for example, a part of the shape coordinate space of the tensor data involved in the first operation.

[0121] In some embodiments, the determination of the first coordinate space range can be dynamically determined based on the execution of the operation. For example, the first coordinate space range can be dynamically determined based on at least one of: the order of the operation; the operands involved in the operation; the second coordinate space range of a preceding operation; and a pre-partition of the shape coordinate space of the tensor data.

[0122] In some embodiments, the determination of the first coordinate space range can comprise determining a coordinate space upper bound of one or more dimensions of the tensor data allowed to be used by the first operation.

[0123] For example, in one example, when the first operation is a read operation, the coordinate space upper bound of the first operation is determined according to the coordinate space lower bounds of preceding unended write operations on the tensor data. Preferably, the minimum of the relevant coordinate space lower bounds is determined.

[0124] In another example, when the first operation is a write operation, the coordinate space upper bound of the first operation is determined according to the coordinate space lower bounds of preceding unended write operations on the tensor data and read operations on the tensor data. Preferably, the minimum of the relevant coordinate space lower bounds is determined.

[0125] Those skilled in the art can understand that various rules can be taken to determine the coordinate space upper bound to represent the first coordinate space range, as long as the order consistency when executing the instructions that rely on the relationship can be ensured, and the disclosure is not limited to the above examples.

[0126] Next, in step S430, a second coordinate space range of the tensor data to be used when executing the first operation is determined. This step can be performed, for example, by the execution unit 220 of FIG. 2 The second coordinate space range can be, for example, a part of the shape coordinate space of the tensor data involved in the first operation.

[0127] In some embodiments, the determination of the second coordinate space range can be dynamically determined based on the execution of the operation. For example, the second coordinate space range can be dynamically determined based on at least one of: the execution range of the operation; the access pattern of the operation; the current execution state of the operation; and a pre-partition of the shape coordinate space of the tensor data.

[0128] In some embodiments, the determination of the second coordinate space range can comprise determining a coordinate space lower bound of one or more dimensions of the tensor data expected to be used by the first operation.

[0129] In one example, when the access pattern of the first operation is sequential and continuous access, the coordinate space lower bound is determined based on the minimum access coordinate of the subsequent access of the first operation.

[0130] In another example, when the access pattern of the first operation is regular access, the coordinate space lower bound is determined based on the regularity.

[0131] In yet another example, when the access pattern of the first operation cannot be determined, the coordinate space lower bound is determined based on a predetermined setting.

[0132] Likewise, those skilled in the art can understand that various rules can be adopted to determine the coordinate space lower bound to represent the second coordinate space range, as long as the sequential consistency between instructions that depend on each other can be ensured, and the disclosure is not limited to the above examples.

[0133] Finally, in step S440, the first operation is executed within the third coordinate space range determined by the intersection of the first coordinate space range and the second coordinate space range. This step can be performed by, for example, the execution unit 220 of the processing device 200. FIG. 4

[0134] The processing method performed by the processing device of the embodiment of the disclosure has been described above with reference to the flowchart. Those skilled in the art can understand that, since the operations performed in parallel are constrained based on the spatial range of the processed data, the parallel degree of the operations can be improved while ensuring the sequential consistency of the operation execution, thereby improving the processing efficiency. It should be noted that, for the foregoing method embodiments, in order to simply describe, they are all expressed as a series of action combinations, but those skilled in the art should know that the disclosure is not limited by the order of the described actions, because according to the disclosure, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should know that the embodiments described in the specification are all optional embodiments, and the actions and modules involved are not necessarily required by the disclosure.

[0135] It should be further noted that, although the steps in the flowchart of the processing device 200 are displayed in sequence according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified in this document, the execution of these steps is not strictly limited in order, and these steps can be executed in other orders. Moreover, FIG. 4 at least part of the steps in the flowchart of the processing device 200 can include multiple sub-steps or multiple stages, which are not necessarily executed at the same time, but can be executed at different times, and the execution order of these sub-steps or stages is not necessarily sequential, but can be executed in rotation or alternation with other steps or sub-steps or stages of at least part of other steps. FIG. 5

[0136] FIG. 5 is a structural diagram showing a combined processing device 500 according to an embodiment of the disclosure. As shown in FIG. 2 ​​As shown in FIG. 5, the combination processing device 500 includes a computing processing device 502, an interface device 504, other processing devices 506, and a storage device 508. Depending on different application scenarios, the computing processing device can include one or more computing devices 510, which can be configured to FIG. 4 The processing device 200 as shown in FIG. 2 is configured to perform the operations described herein in conjunction with the accompanying FIG. 6 The operations described.

[0137] In different embodiments, the computing processing device of the present disclosure can be configured to perform user-specified operations. In exemplary applications, the computing processing device can be implemented as a single-core artificial intelligence processor or a multi-core artificial intelligence processor. Similarly, one or more computing devices included in the computing processing device can be implemented as an artificial intelligence processor core or a partial hardware structure of an artificial intelligence processor core. When multiple computing devices are implemented as an artificial intelligence processor core or a partial hardware structure of an artificial intelligence processor core, the computing processing device of the present disclosure can be considered to have a single-core structure or a homogeneous multi-core structure.

[0138] In exemplary operations, the computing processing device of the present disclosure can interact with other processing devices through the interface device to jointly complete user-specified operations. Depending on different implementations, the other processing devices of the present disclosure can include one or more types of processors selected from general-purpose and / or special-purpose processors such as a central processing unit (CPU), a graphics processing unit (GPU), an artificial intelligence processor, etc. These processors can include but are not limited to a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic device, a discrete gate or transistor logic device, a discrete hardware component, etc., and the number thereof can be determined according to actual needs. As mentioned earlier, only in terms of the computing processing device of the present disclosure, it can be considered to have a single-core structure or a homogeneous multi-core structure. However, when the computing processing device and the other processing devices are considered together, they can be considered to form a heterogeneous multi-core structure.

[0139] In one or more embodiments, the other processing device can serve as an interface for the computing processing device (which can be embodied as a related processing device for artificial intelligence such as neural network operation) to external data and control, performing basic control including but not limited to data transfer, start and / or stop of the computing device, etc. In additional embodiments, the other processing device can also cooperate with the computing processing device to jointly complete the operation task.

[0140] In one or more embodiments, the interface device can be used to transfer data and control instructions between the computing processing device and the other processing device. For example, the computing processing device can obtain input data from the other processing device via the interface device and write to the storage device (or memory) on the computing processing device chip. Further, the computing processing device can obtain control instructions from the other processing device via the interface device and write to the control buffer on the computing processing device chip. Alternatively or optionally, the interface device can also read data from the storage device of the computing processing device and transmit to the other processing device.

[0141] Additionally or optionally, the combined processing device of the present disclosure can further include a storage device. As shown in the figure, the storage device is connected to the computing processing device and the other processing device, respectively. In one or more embodiments, the storage device can be used to save data of the computing processing device and / or the other processing device. For example, the data can be data that cannot be fully saved in the internal or on-chip storage device of the computing processing device or the other processing device.

[0142] In some embodiments, the present disclosure also discloses a chip (e.g. FIG. 5 shown in FIG. 6B). In an implementation, the chip is a system on chip (SoC) and integrates one or more combined processing devices as shown in FIG. 6 FIG. 6A. The chip can be connected to other related components through an external interface device (e.g. FIG. 6 shown in FIG. 6B). The related components can be, for example, a camera, a display, a mouse, a keyboard, a network card, or a wifi interface. In some application scenarios, other processing units (e.g. video codec) and / or interface modules (e.g. DRAM interface) can be integrated on the chip. In some embodiments, the present disclosure also discloses a chip package structure including the above chip. In some embodiments, the present disclosure also discloses a board card including the above chip package structure. The board card will be described in detail below. FIG. 6

[0143] FIG. 6 is a structural schematic diagram showing a board card 600 according to an embodiment of the present disclosure. As shown in​FIG. 5 As shown in FIG. 6B, the board card includes a storage device 604 for storing data, which includes one or more storage units 610. The storage device can be connected and transmit data with the control device 608 and the chip 602 described above by, for example, a bus. Further, the board card also includes an external interface device 606, which is configured for data relay or switching function between the chip (or the chip in the chip package structure) and an external equipment 612 (such as a server or a computer, etc.). For example, the data to be processed can be transmitted by the external equipment to the chip through the external interface device. For another example, the calculation result of the chip can be transmitted back to the external equipment via the external interface device. According to different application scenarios, the external interface device can have different interface forms, for example, it can adopt a standard PCIE interface, etc.

[0144] In one or more embodiments, the control device in the board card of the present disclosure can be configured for regulating the state of the chip. For this purpose, in one application scenario, the control device can include a micro controller unit (MCU) for regulating the working state of the chip.

[0145] According to the above description in combination with FIG. 6 and ​ , those skilled in the art can understand that the present disclosure also discloses an electronic equipment or device, which can include one or more of the above-mentioned board cards, one or more of the above-mentioned chips and / or one or more of the above-mentioned combined processing devices.

[0146] According to different application scenarios, the electronic device or apparatus of the present disclosure can include a server, a cloud server, a server cluster, a data processing apparatus, a robot, a computer, a printer, a scanner, a tablet computer, a smart terminal, a PC device, an Internet of Things terminal, a mobile terminal, a mobile phone, a vehicle record device, a navigator, a sensor, a camera, a camera, a video camera, a projector, a watch, a headset, a mobile storage, a wearable device, a visual terminal, an autonomous driving terminal, a vehicle, a household appliance, and / or a medical device. The vehicle includes an airplane, a ship, and / or a vehicle; the household appliance includes a television, an air conditioner, a microwave oven, a refrigerator, an electric rice cooker, a humidifier, a washing machine, an electric lamp, a gas stove, an oil smoke exhaust fan; the medical device includes a nuclear magnetic resonance instrument, a B-ultrasound instrument, and / or an electrocardiograph. The electronic device or apparatus of the present disclosure can also be applied to the fields of Internet, Internet of Things, data center, energy, transportation, public management, manufacturing, education, power grid, telecommunications, finance, retail, construction site, medical treatment, etc. Further, the electronic device or apparatus of the present disclosure can also be used in cloud, edge, terminal, etc. application scenarios related to artificial intelligence, big data, and / or cloud computing. In one or more embodiments, the electronic device or apparatus with high computing power according to the present disclosure can be applied to a cloud device (such as a cloud server), and the electronic device or apparatus with small power consumption can be applied to a terminal device and / or an edge device (such as a smart phone or a camera). In one or more embodiments, the hardware information of the cloud device and the hardware information of the terminal device and / or the edge device are compatible with each other, so that suitable hardware resources can be matched from the hardware resources of the cloud device according to the hardware information of the terminal device and / or the edge device to simulate the hardware resources of the terminal device and / or the edge device, so as to complete the unified management, scheduling and collaborative work of end-cloud integration or cloud-edge integration.

[0147] It should be noted that for the purpose of simplicity, the present disclosure describes some methods and embodiments thereof as a series of actions and combinations thereof, but those skilled in the art can understand that the schemes of the present disclosure are not limited by the order of the described actions. Therefore, those skilled in the art can understand that some steps can be executed in other orders or simultaneously according to the disclosure or teaching of the present disclosure. Further, those skilled in the art can understand that the described embodiments of the present disclosure can be regarded as optional embodiments, i.e. the actions or modules involved therein are not necessarily essential for the implementation of one or more schemes of the present disclosure. In addition, the description of some embodiments of the present disclosure also focuses on different schemes. Therefore, those skilled in the art can understand that the parts not described in detail in one embodiment of the present disclosure can also refer to the relevant description of other embodiments.

[0148] In terms of specific implementation, based on the disclosure and teachings of the present disclosure, those skilled in the art can understand that the several embodiments disclosed by the present disclosure can also be implemented in other manners not disclosed herein. For example, in terms of the units described in the aforementioned electronic device or apparatus embodiments, the units can be combined or integrated in another system, or some features or functions of the units can be selectively disabled. In terms of the connection relationship between the units or components, the connection between the units or components discussed in the foregoing can be direct or indirect coupling between units or components. In some scenarios, the aforementioned direct or indirect coupling refers to communication connection using an interface, where the communication interface can support electrical, optical, acoustic, magnetic or other forms of signal transmission.

[0149] In the present disclosure, the units described as separate components can or can not be physically separate, and the components shown as units can or can not be physical units. The aforementioned components or units can be located in the same position or distributed on a plurality of network units. In addition, according to actual needs, some or all of the units can be selected to achieve the purposes of the solutions described in the embodiments of the present disclosure. In addition, in some scenarios, multiple units in the embodiments of the present disclosure can be integrated in one unit or physically exist separately.

[0150] In some implementation scenarios, the integrated units described above can be implemented in the form of software program modules. If implemented in the form of software program modules and sold or used as independent products, the integrated units can be stored in a computer readable memory. Therefore, when the solutions of the present disclosure are embodied in the form of a software product (for example, a computer readable storage medium), the software product can be stored in a memory, which can include a plurality of instructions to make a computer device (for example, a personal computer, a server or a network device, etc.) execute some or all of the steps of the methods described in the embodiments of the present disclosure. The aforementioned memory can include, but is not limited to, a U disk, a flash disk, a read only memory (ROM), a random access memory (RAM), a mobile hard disk, a magnetic disk or an optical disk, and various media that can store program codes.

[0151] In some other implementation scenarios, the above-mentioned integrated units can also be implemented in the form of hardware, i.e., specific hardware circuits, which can include digital circuits and / or analog circuits, etc. The physical implementation of the hardware structure of the circuit can include but is not limited to physical devices, and the physical devices can include but are not limited to transistors or memristors, etc. In view of this, various apparatuses (e.g., computing apparatuses or other processing apparatuses) described herein can be implemented by appropriate hardware processors, such as CPUs, GPUs, FPGAs, DSPs, ASICs, etc. Further, the aforementioned storage units or storage devices can be any appropriate storage medium (including magnetic storage media or magneto-optical storage media, etc.), which can be, for example, resistive random access memory (RRAM), dynamic random access memory (DRAM), static random access memory (SRAM), enhanced dynamic random access memory (EDRAM), high bandwidth memory (HBM), hybrid memory cube (HMC), ROM, and RAM, etc.

[0152] The foregoing can be better understood in light of the following clauses:

[0153] Clause 1. A processing apparatus comprising a control unit and an execution unit, wherein:

[0154] The control unit is configured to:

[0155] obtain a first operation of an instruction; and

[0156] send, to the execution unit, the first operation and information indicating a first coordinate space range of tensor data allowed to be used by the first operation; and

[0157] The execution unit is configured to:

[0158] send, to the control unit, information indicating a second coordinate space range of the tensor data to be used when executing the first operation; and

[0159] execute the first operation in a third coordinate space range determined by an intersection of the first coordinate space range and the second coordinate space range.

[0160] Clause 2. The processing device according to clause 1, further comprising a storage unit, wherein the first coordinate space range and the second coordinate space range are respectively a part of a shape coordinate space of the tensor data, the shape coordinate space mapping to a data storage area of the tensor data on the storage unit.

[0161] Clause 3. The processing device according to clause 2, wherein the first coordinate space range and the second coordinate space range are dynamically determined based on execution of an operation.

[0162] Clause 4. The processing device according to clause 3, wherein the first coordinate space range is dynamically determined based on at least one of:

[0163] an order of operations;

[0164] operands involved in the operation;

[0165] information received from the execution unit indicating a second coordinate space range of a preceding operation; and

[0166] a pre-partition of a shape coordinate space of the tensor data.

[0167] Clause 5. The processing device according to any of clauses 3-4, wherein the second coordinate space range is dynamically determined based on at least one of:

[0168] an execution range of the operation;

[0169] an access pattern of the operation;

[0170] a current execution status of the operation; and

[0171] a pre-partition of a shape coordinate space of the tensor data.

[0172] Clause 6. The processing device according to any of clauses 4-5, wherein:

[0173] the information indicating the first coordinate space range comprises a coordinate space upper bound of one or more dimensions of the tensor data allowed to be used by the first operation; and / or

[0174] the information indicating the second coordinate space range comprises a coordinate space lower bound of one or more dimensions of the tensor data expected to be used by the first operation.

[0175] Clause 7. The processing device according to clause 6, wherein:

[0176] when the first operation is a read operation, the coordinate space upper bound is determined by a coordinate space lower bound of a preceding unended write operation to the tensor data; and / or

[0177] When the first operation is a write operation, the coordinate space upper bound is determined by a coordinate space lower bound of a previous write operation to the tensor data and a coordinate space lower bound of a read operation to the tensor data.

[0178] Clause 8. The processing device according to clause 7, wherein the coordinate space upper bound is determined by a minimum value in the related coordinate space lower bounds.

[0179] Clause 9. The processing device according to any one of clauses 6-8, wherein:

[0180] When the access pattern of the first operation is sequential and contiguous access, the coordinate space lower bound is determined based on a minimum access coordinate of a subsequent access of the first operation;

[0181] When the access pattern of the first operation is regular access, the coordinate space lower bound is determined based on the regularity; and / or

[0182] When the access pattern of the first operation cannot be determined, the coordinate space lower bound is determined based on a predetermined setting.

[0183] Clause 10. The processing device according to any one of clauses 4-9, wherein the pre-division of the shape coordinate space of the tensor data is performed based on at least one of:

[0184] processing capability of hardware;

[0185] pre-set parameters; and

[0186] size of the shape coordinate space of the tensor data.

[0187] Clause 11. The processing device according to any one of clauses 1-10, wherein the control unit is further configured to:

[0188] block the first operation when the third coordinate space range is empty.

[0189] Clause 12. The processing device according to any one of clauses 4-11, wherein:

[0190] the first operation and the previous operation are respectively operations in different instructions executed in parallel; or

[0191] the first operation and the previous operation are respectively different operations in a same instruction executed in parallel.

[0192] Clause 13. A chip, comprising the processing device according to any one of clauses 1-12.

[0193] Clause 14. A board card, comprising the chip of clause 13.

[0194] Clause 15. A processing method, comprising:

[0195] acquiring a first operation of an instruction;

[0196] determining a first coordinate space range of tensor data allowed to be used by the first operation;

[0197] determining a second coordinate space range of the tensor data to be used when performing the first operation; and

[0198] performing the first operation in a third coordinate space range determined by an intersection of the first coordinate space range and the second coordinate space range.

[0199] Clause 16. The processing method of clause 15, wherein the first coordinate space range and the second coordinate space range are respectively a part of a shape coordinate space of the tensor data, the shape coordinate space being mapped to a data storage area of the tensor data on a storage unit.

[0200] Clause 17. The processing method of clause 16, wherein the first coordinate space range and the second coordinate space range are dynamically determined based on execution of the operation.

[0201] Clause 18. The processing method of clause 17, wherein the first coordinate space range is dynamically determined based on at least one of:

[0202] an execution order of the operation;

[0203] an operand involved in the operation;

[0204] a second coordinate space range of a preceding operation; and

[0205] a pre-partition of a shape coordinate space of the tensor data.

[0206] Clause 19. The processing method of any of clauses 17-18, wherein the second coordinate space range is dynamically determined based on at least one of:

[0207] an execution range of the operation;

[0208] an access pattern of the operation;

[0209] a current execution state of the operation; and

[0210] a pre-partition of a shape coordinate space of the tensor data.

[0211] Clause 20. The processing method of any of clauses 17-19, wherein:

[0212] determining the first coordinate space range comprises determining a coordinate space upper bound of one or more dimensions of the tensor data allowed to be used by the first operation; and / or

[0213] determining the second coordinate space range comprises determining a coordinate space lower bound of one or more dimensions of the tensor data expected to be used by the first operation.

[0214] Clause 21. The processing method of clause 20, wherein:

[0215] when the first operation is a read operation, the coordinate space upper bound is determined according to a coordinate space lower bound of a write operation to the tensor data not ended before; and / or

[0216] when the first operation is a write operation, the coordinate space upper bound is determined according to a coordinate space lower bound of a write operation to the tensor data not ended before and a coordinate space lower bound of a read operation to the tensor data.

[0217] Clause 22. The processing method of clause 21, wherein the coordinate space upper bound is determined according to a minimum value of the related coordinate space lower bounds.

[0218] Clause 23. The processing method of any of clauses 20-22, wherein:

[0219] when the access pattern of the first operation is sequential and contiguous access, the coordinate space lower bound is determined based on a minimum access coordinate of a subsequent access of the first operation;

[0220] when the access pattern of the first operation is regular access, the coordinate space lower bound is determined based on the regularity; and / or

[0221] when the access pattern of the first operation cannot be determined, the coordinate space lower bound is determined based on a predetermined setting.

[0222] Clause 24. The processing method of any of clauses 18-23, wherein the pre-division of the shape coordinate space of the tensor data is performed based on at least one of:

[0223] processing capability of hardware;

[0224] pre-set parameter; and

[0225] size of the shape coordinate space of the tensor data.

[0226] Clause 25. The processing method of any of clauses 15-24, further comprising:

[0227] stall the first operation when the third coordinate space range is empty.

[0228] Clause 26. The processing method of any of clauses 18-25, wherein:

[0229] the first operation and the prior operation are respectively different operations in a same instruction executed in parallel; or

[0230] the first operation and the prior operation are respectively different operations in a same instruction executed in parallel.

Claims

1. A processing apparatus comprising a control unit and an execution unit, wherein: the control unit is configured to: obtain a first operation of an instruction; and send, to the execution unit, the first operation and information indicating a first coordinate space range of tensor data allowed to be used by the first operation; and the execution unit is configured to: send, to the control unit, information indicating a second coordinate space range of the tensor data to be used when executing the first operation; and execute the first operation within a third coordinate space range determined by an intersection of the first coordinate space range and the second coordinate space range, wherein the first coordinate space range and the second coordinate space range are dynamically determined based on execution of the operation.

2. The processing apparatus according to claim 1, further comprising a storage unit, wherein the first coordinate space range and the second coordinate space range are respectively a part of a shape coordinate space of the tensor data, the shape coordinate space being mapped to a data storage area of the tensor data on the storage unit.

3. The processing apparatus according to claim 1, wherein the first coordinate space range is dynamically determined based on at least one of: a sequential order of operations; an operand involved in the operation; information received from the execution unit indicating a second coordinate space range of a preceding operation; and a pre-partition of a shape coordinate space of the tensor data.

4. The processing apparatus according to claim 1, wherein the second coordinate space range is dynamically determined based on at least one of: an execution range of the operation; an access pattern of the operation; a current execution status of the operation; and a pre-partition of a shape coordinate space of the tensor data.

5. The processing apparatus according to claim 1, wherein: the information indicating the first coordinate space range comprises a coordinate space upper bound of one or more dimensions of the tensor data allowed to be used by the first operation; and / or the information indicating the second coordinate space range comprises a coordinate space lower bound of one or more dimensions of the tensor data expected to be used by the first operation.

6. The processing apparatus according to claim 5, wherein: when the first operation is a read operation, the coordinate space upper bound is determined by a coordinate space lower bound of a preceding unended write operation to the tensor data; and / or when the first operation is a write operation, the coordinate space upper bound is determined by a coordinate space lower bound of a preceding unended write operation to the tensor data and a coordinate space lower bound of a read operation to the tensor data.

7. The processing apparatus according to claim 6, wherein the coordinate space upper bound is determined by a minimum of the related coordinate space lower bounds.

8. The processing apparatus according to claim 5, wherein: when an access pattern of the first operation is sequential and contiguous access, the coordinate space lower bound is determined based on a minimum access coordinate of a subsequent access of the first operation; when the access pattern of the first operation is regular access, the coordinate space lower bound is determined based on the regularity; and / or ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ When the access pattern of the first operation cannot be determined, the lower bound of the coordinate space is determined based on a predetermined setting. 9.The processing apparatus of claim 3 or 4, wherein the pre-partitioning of the shape coordinate space of the tensor data is performed based on at least one of: a processing capability of hardware; a pre-set parameter; and a size of the shape coordinate space of the tensor data. 10.The processing apparatus of claim 1, wherein the control unit is further configured to: block the first operation when the third coordinate space range is empty. 11.The processing apparatus of claim 3, wherein: the first operation and the preceding operation are respectively operations in different instructions executed in parallel; or the first operation and the preceding operation are respectively different operations in a same instruction executed in parallel. The chip comprises the processing apparatus of any one of claims 1-11. The board card comprises the chip of claim 12. 14.A processing method, the method comprising: obtaining a first operation of an instruction; determining a first coordinate space range of tensor data allowed to be used by the first operation; determining a second coordinate space range of the tensor data to be used when the first operation is executed; and executing the first operation within a third coordinate space range determined by an intersection of the first coordinate space range and the second coordinate space range, wherein the first coordinate space range and the second coordinate space range are dynamically determined based on execution of the operation. 15.The processing method of claim 14, wherein the first coordinate space range and the second coordinate space range are respectively a part of a shape coordinate space of the tensor data, the shape coordinate space being mapped to a data storage area of the tensor data on a storage unit. 16.The processing method of claim 14, wherein the first coordinate space range is dynamically determined based on at least one of: an order of operations; an operand involved in the operation; a second coordinate space range of a preceding operation; and a pre-partitioning of the shape coordinate space of the tensor data. 17.The processing method of claim 14, wherein the second coordinate space range is dynamically determined based on at least one of: an execution range of the operation; an access pattern of the operation; a current execution state of the operation; and a pre-partitioning of the shape coordinate space of the tensor data.

12. A chip, characterized by 18.The processing method of claim 14, wherein: determining the first coordinate space range comprises determining an upper bound of a coordinate space of one or more dimensions of the tensor data allowed to be used by the first operation; and / or determining the second coordinate space range comprises determining a lower bound of a coordinate space of one or more dimensions of the tensor data expected to be used by the first operation.

13. A board card, characterized by 19.The processing method of claim 18, wherein: when the first operation is a read operation, the upper bound of the coordinate space is determined according to a lower bound of a coordinate space of the tensor data of a preceding write operation that has not ended; and / or ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ When the first operation is a write operation, the coordinate space upper bound is determined according to a coordinate space lower bound of a previous non-completed write operation to the tensor data and a coordinate space lower bound of a read operation to the tensor data. 20.The processing method of claim 19, wherein the coordinate space upper bound is determined according to a minimum value in the related coordinate space lower bounds. 21.The processing method of claim 18, wherein: when the access pattern of the first operation is sequential and contiguous access, the coordinate space lower bound is determined based on a minimum access coordinate of a subsequent access of the first operation; when the access pattern of the first operation is regular access, the coordinate space lower bound is determined based on the regularity; and / or when the access pattern of the first operation cannot be determined, the coordinate space lower bound is determined based on a predetermined setting. 22.The processing method of claim 16 or 17, wherein the pre-partitioning of the shape coordinate space of the tensor data is performed based on at least one of: processing capability of hardware; a pre-set parameter; and size of the shape coordinate space of the tensor data. 23.The processing method of claim 14, further comprising: blocking the first operation when the third coordinate space range is empty. 24.The processing method of claim 16, wherein: the first operation and the previous operation are respectively operations in different instructions executed in parallel; or the first operation and the previous operation are respectively different operations in a same instruction executed in parallel. ​

Images