Processing method, processing device and related products

Abstract

The present disclosure discloses a processing method, a processing device and related products. The processing device can be implemented as a computing device included in a combined processing device, which can further include an interface device and other processing devices. The computing device interacts with the other processing devices to jointly complete a user-specified computing operation. The combined processing device can further include a storage device connected to the computing device and the other processing devices respectively, for storing data of the computing device and the other processing devices. The scheme of the present disclosure provides a solution of instruction parallelism, which can improve the instruction parallelism and thus improve the processing efficiency of the machine.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to the field of processors, and in particular, to a processing method, a processing apparatus, a chip and a board card. BACKGROUND

[0002] Instruction system is the interface of computer software and hardware interaction, and is a very important part in computer system structure. With the continuous development of artificial intelligence technology, the amount and dimension of data to be processed are increasing. Therefore, how to reasonably and scientifically control the execution of instructions, especially to improve the degree of instruction parallelism and improve the performance of the machine, is an important problem in the field of processors. SUMMARY

[0003] In order to solve one or more technical problems as mentioned above, the present disclosure proposes a solution to enhance instruction parallelism in multiple aspects. Through the instruction system of the present disclosure, the degree of instruction parallelism can be improved, thereby improving the processing efficiency of the machine.

[0004] In a first aspect, the present disclosure provides a processing method, comprising: obtaining a first operation of an instruction, the first operation being an operation on tensor data, a shape coordinate space of the tensor data comprising at least one fine-grained region, the fine-grained region comprising one or more adjacent coordinate points of the shape coordinate space; determining whether there is a second operation on the tensor data being in progress; when the second operation exists, determining whether a first fine-grained region currently targeted by the first operation and a second fine-grained region currently targeted by the second operation overlap; and when the first fine-grained region and the second fine-grained region do not overlap, executing the first operation.

[0005] In a second aspect, the present disclosure provides a processing apparatus, comprising: an operation obtaining unit configured to obtain a first operation of an instruction, the first operation being an operation on tensor data, a shape coordinate space of the tensor data comprising at least one fine-grained region, the fine-grained region comprising one or more adjacent coordinate points of the shape coordinate space; a first determination unit configured to determine whether there is a second operation on the tensor data being in progress; a second determination unit configured to, when the second operation exists, determine whether a first fine-grained region currently targeted by the first operation and a second fine-grained region currently targeted by the second operation overlap; and an execution unit configured to, when the first fine-grained region and the second fine-grained region do not overlap, execute the first operation.

[0006] In a third aspect, the present disclosure provides a chip, comprising the processing apparatus of any one of the embodiments of the second aspect.

[0007] In a fourth aspect, the disclosure provides a board card comprising the chip of any one of the preceding third aspect.

[0008] By means of the processing apparatus, the processing method, the chip and the board card provided as above, in the operation execution process of the instruction, the parallelism of the operation is limited based on the fine-grained region of the shape coordinate space of the tensor data targeted by the operation, so that the parallel execution potential of the operation can be tapped. Therefore, according to the embodiments of the disclosure, in the parallel execution of the hardware, the consistency of the execution order can be guaranteed, and the parallel degree of the operation can be improved, thereby ensuring the accuracy and efficiency of the processing. BRIEF DESCRIPTION OF DRAWINGS

[0009] The above and other objects, features and advantages of the exemplary embodiments of the present disclosure will be more apparent from the following detailed description taken in conjunction with the accompanying drawings, in which:

[0010] Figure 1A A schematic diagram of a data storage space according to an embodiment of the disclosure is shown;

[0011] Figure 1B A schematic diagram of data blocking in a data storage space according to an embodiment of the disclosure is shown;

[0012] Figure 2 A schematic block diagram of a processing apparatus according to an embodiment of the disclosure is shown;

[0013] Figures 3A-3C A schematic flow chart of a processing method according to an embodiment of the disclosure is shown;

[0014] Figure 3D A schematic block diagram of a processing apparatus according to an embodiment of the disclosure is shown;

[0015] Figure 4 A schematic diagram of a coordinate space range according to an embodiment of the disclosure is shown;

[0016] Figure 5 A structural diagram of a combined processing apparatus according to an embodiment of the disclosure is shown; and

[0017] Figure 6 A structural schematic diagram of a board card according to an embodiment of the disclosure is shown. DETAILED DESCRIPTION

[0018] The technical solutions in the embodiments of the disclosure will be described clearly and completely in the embodiments of the disclosure combined with the drawings in the embodiments of the disclosure. Obviously, the described embodiments are part of the embodiments of the disclosure, rather than all the embodiments. Based on the embodiments of the disclosure, all other embodiments obtained by a person skilled in the art without creative work fall within the protection scope of the disclosure.

[0019] It should be understood that the terms "first", "second", "third", and "fourth" and the like used in the claims, the specification and the drawings of the disclosure are used to distinguish different objects, rather than to describe a particular order. The terms "include" and "contain" used in the specification and claims of the disclosure indicate the presence of described features, integers, steps, operations, elements and / or components, but do not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or sets thereof.

[0020] It should also be understood that the terms used in the specification of the disclosure herein are only for the purpose of describing specific embodiments, and are not intended to limit the disclosure. As used in the specification and claims of the disclosure, unless the context clearly indicates otherwise, the singular forms "a", "an" and "the" are intended to include the plural forms. It should be further understood that the term "and / or" used in the specification and claims of the disclosure means any combination of one or more of the associated listed items and all possible combinations, and includes these combinations.

[0021] As used in the specification and claims, the term "if" can be interpreted as "when" or "upon" or "in response to a determination" or "in response to detecting" depending on the context. Similarly, the phrase "if determined" or "if detected [the described condition or event]" can be interpreted to mean "upon being determined" or "in response to being determined" or "upon detecting [the described condition or event]" or "in response to detecting [the described condition or event]" depending on the context.

[0022] A computer processes various data by executing instructions. In order to indicate the source of data, the destination of operation results and the performed operation, an instruction usually contains the following information:

[0023] (1) Operation code (OP) is used to represent the operation to be completed by the instruction (e.g. addition, subtraction, multiplication, division, data transfer, etc.), which specifically describes the nature and function of the operation. A computer can have dozens to hundreds of instructions, each instruction has a corresponding operation code, and the computer completes different operations by identifying the operation code.

[0024] (2) an operand, which is used to describe the operation object of the instruction. The operand can involve the data type of the operation object, the memory address, the addressing mode, etc. The operand can directly give the operation object, or indicate the memory address or register address (i.e., the register name) of the operation object.

[0025] The instructions of the conventional processor are designed to be able to perform basic single data scalar operations. Here, the single data scalar operation refers to that each operand of the instruction is a scalar data. However, with the development of artificial intelligence technology, in tasks such as image processing and pattern recognition, the operation object tends to be a multi-dimensional vector (i.e., tensor data) data type, and only using scalar operations cannot make the hardware efficiently complete the operation task. Therefore, how to efficiently perform multi-dimensional tensor data processing is also a problem that needs to be solved in the current computing field.

[0026] In embodiments of the present disclosure, an instruction system is provided, wherein a descriptor is included in the operand of the instruction, through which information related to the tensor data can be obtained. Specifically, the descriptor can indicate at least one of the following information: shape information of the tensor data, spatial information of the tensor data. The shape information of the tensor data can be used to determine the data address of the tensor data corresponding to the operand in the data storage space. The spatial information of the tensor data can be used to determine the dependency relationship between instructions, and then the execution order of the instructions can be determined, for example. The spatial information of the tensor data can be indicated by a spatial ID. The spatial ID can also be referred to as a spatial alias, which refers to a spatial region for storing the corresponding tensor data. The spatial region can be a continuous space, or multiple spaces, and the present disclosure does not limit the specific form of the composition of the spatial region. Different spatial IDs represent that the spatial regions pointed to by the different spatial IDs do not have a dependency relationship. For example, the dependency relationship can be ensured by making the spatial regions pointed to by different spatial IDs not overlap with each other.

[0027] Various possible implementations of the shape information of the tensor data will be described in detail below in conjunction with the drawings.

[0028] The tensor can contain various forms of data composition. The tensor can be of different dimensions, such as a scalar can be regarded as a 0-dimensional tensor, a vector can be regarded as a 1-dimensional tensor, and a matrix can be a 2-dimensional or higher-dimensional tensor. The shape of the tensor includes information such as the dimensions of the tensor and the size of each dimension of the tensor. For example, for a three-dimensional tensor:

[0029] x3 = [[[1, 2, 3], [4, 5, 6]]; [[7, 8, 9], [10, 11, 12]]]

[0030] The shape or dimension of the tensor can be represented as X3 = (2, 2, 3), i.e. the tensor is a three-dimensional tensor represented by three parameters, and the size of the first dimension of the tensor is 2, the size of the second dimension is 2, and the size of the third dimension is 3. When storing tensor data in a memory, the shape of the tensor data cannot be determined according to its data address (or storage area), and thus the correlation between multiple tensor data and other related information cannot be determined, resulting in low efficiency of the processor in accessing the tensor data.

[0031] In a possible implementation, the shape of the N-dimensional tensor data can be indicated by a descriptor, where N is a positive integer, for example, N = 1, 2, or 3, or zero. The three-dimensional tensor in the above example can be represented by the descriptor as (2, 2, 3). It should be noted that the present disclosure does not limit the way the descriptor indicates the shape of the tensor.

[0032] In a possible implementation, the value of N can be determined according to the dimension (also referred to as the order) of the tensor data, or can be set according to the use requirement of the tensor data. For example, when the value of N is 3, the tensor data is three-dimensional tensor data, and the descriptor can be used to indicate the shape (such as offset, size, etc.) of the three-dimensional tensor data in three dimensions. It should be understood that the value of N can be set according to actual needs by those skilled in the art, and the present disclosure does not limit this.

[0033] Although the tensor data can be multi-dimensional, because the layout of the memory is always one-dimensional, there is a corresponding relationship between the tensor and the storage on the memory. The tensor data is usually allocated in a continuous storage space, i.e. the tensor data can be one-dimensionally unfolded (for example, in a row-major manner) and stored on the memory.

[0034] This relationship between the tensor and the underlying storage can be represented by the offset of the dimension, the size of the dimension, the stride of the dimension, etc. The offset of the dimension refers to the offset of the reference position in this dimension. The size of the dimension refers to the size of the dimension, i.e. the number of elements in this dimension. The stride of the dimension refers to the interval between adjacent elements in this dimension, for example, the stride of the above three-dimensional tensor is (6, 3, 1), i.e. the stride of the first dimension is 6, the stride of the second dimension is 3, and the stride of the third dimension is 1.

[0035] Figure 1A A schematic diagram of a data storage space according to an embodiment of the present disclosure is shown. As Figure 1AAs shown, the data storage space 21 stores a two-dimensional data in a row-major manner, which can be represented by (x, y) (wherein the X-axis is horizontally to the right, and the Y-axis is vertically downward). The size in the X-axis direction (the size of each row, or the total number of columns) is ori_x (not shown in the figure), the size in the Y-axis direction (the total number of rows) is ori_y (not shown in the figure), and the starting address PA_start (reference address) of the data storage space 21 is the physical address of the first data block 22. The data block 23 is part of the data in the data storage space 21, and the offset 25 in the X-axis direction is offset_x, the offset 24 in the Y-axis direction is offset_y, the size in the X-axis direction is size_x, and the size in the Y-axis direction is size_y.

[0036] In a possible implementation, when the data block 23 is defined using the descriptor, the data reference point of the descriptor can use the first data block of the data storage space 21, and it can be agreed that the reference address of the descriptor is the starting address PA_start of the data storage space 21. Then the content of the descriptor of the data block 23 can be determined in combination with the size ori_x of the data storage space 21 in the X-axis direction, the size ori_y in the Y-axis direction, and the offset offset_y of the data block 23 in the Y-axis direction, the offset offset_x in the X-axis direction, the size size_x in the X-axis direction, and the size size_y in the Y-axis direction.

[0037] In a possible implementation, the content of the descriptor can be represented by the following formula (1):

[0038]

[0039] It should be understood that, although in the above example, the content of the descriptor represents a two-dimensional space, a person skilled in the art can set the specific dimension represented by the content of the descriptor according to actual conditions, and the present disclosure does not limit this.

[0040] In a possible implementation, it can be agreed that the data reference point of the descriptor is at a reference address in the data storage space, and on the basis of the reference address, the content of the descriptor of the tensor data is determined according to the positions of at least two vertices at the diagonal positions in the N-dimensional directions relative to the data reference point.

[0041] For example, a data reference point of the descriptor can be agreed to have a reference address PA_base in the data storage space. For example, a data (e.g., the data at position (2, 2)) in the data storage space 21 can be selected as the data reference point, and a physical address of the data in the data storage space can be taken as the reference address PA_base. The content of the descriptor of the data block 23 can be determined according to positions of at least two vertices of the diagonal position relative to the data reference point, for example, positions of vertices of the diagonal position in the left-up to right-down direction relative to the data reference point, wherein a relative position of the left-up corner vertex is (x_min, y_min), and a relative position of the right-down corner vertex is (x_max, y_max). Then, the content of the descriptor of the data block 23 can be determined according to the reference address PA_base, the relative position (x_min, y_min) of the left-up corner vertex, and the relative position (x_max, y_max) of the right-down corner vertex. Figure 1A The content of the descriptor of the data block 23 can be determined according to positions of at least two vertices of the diagonal position relative to the data reference point. For example, the content of the descriptor of the data block 23 can be determined according to positions of vertices of the diagonal position in the left-up to right-down direction relative to the data reference point, wherein a relative position of the left-up corner vertex is (x_min, y_min), and a relative position of the right-down corner vertex is (x_max, y_max).

[0042] In a possible implementation, the content of the descriptor (the reference address is PA_base) can be represented by the following formula (2):

[0043]

[0044] It should be understood that, although the content of the descriptor is determined by using the left-up corner and the right-down corner in the above examples, a person skilled in the art can set specific vertices of at least two vertices of the diagonal position according to actual needs, and the disclosure does not limit this.

[0045] In a possible implementation, the content of the descriptor of the tensor data can be determined according to a reference address of a data reference point of the descriptor in the data storage space, and a mapping relationship between a data description position and a data address of the tensor data indicated by the descriptor. The mapping relationship between the data description position and the data address can be set according to actual needs. For example, when the tensor data indicated by the descriptor is three-dimensional space data, a function f(x, y, z) can be used to define the mapping relationship between the data description position and the data address.

[0046] In a possible implementation, the content of the descriptor can be represented by the following formula (3):

[0047]

[0048] In a possible implementation, the descriptor is also used to indicate an address of the N-dimensional tensor data, wherein the content of the descriptor further comprises at least one address parameter representing the address of the tensor data, for example, the content of the descriptor can be as follows (4):

[0049]

[0050] wherein PA is the address parameter. The address parameter can be a logical address or a physical address. When the descriptor is parsed, the PA can be any one of a vertex, a middle point or a preset point of a vector shape, and the corresponding data address can be obtained in combination with the shape parameters in the X direction and the Y direction.

[0051] In a possible implementation, the address parameter of the tensor data comprises a reference address of a data reference point of the descriptor in a data storage space of the tensor data, and the reference address comprises a start address of the data storage space.

[0052] In a possible implementation, the descriptor can further comprise at least one address parameter representing the address of the tensor data, for example, the content of the descriptor can be as follows (5):

[0053]

[0054] wherein PA_start is the reference address parameter, which will not be repeated.

[0055] It should be understood that a person skilled in the art can set the mapping relationship between the data description position and the data address according to actual conditions, and the present disclosure does not limit this.

[0056] In a possible implementation, a reference address can be set in a task, and the descriptor in the instruction of the task uses the reference address, and the shape parameter based on the reference address can be included in the content of the descriptor. The reference address can be determined by setting an environment parameter of the task. For details of the reference address and the use manner, refer to the above embodiments. In this implementation, the content of the descriptor can be mapped to the data address more quickly.

[0057] In a possible implementation, the reference address can be included in the content of each descriptor, and the reference addresses of the descriptors can be different. Compared with the manner of setting a common reference address by using an environment parameter, the data can be described more flexibly and a larger data address space can be used in this manner.

[0058] In a possible implementation, the data address of the data corresponding to the operand of the processing instruction in the data storage space can be determined according to the content of the descriptor. The calculation of the data address is automatically completed by hardware, and the calculation method of the data address is different when the representation of the content of the descriptor is different. The specific calculation method of the data address is not limited in the disclosure.

[0059] For example, the content of the descriptor in the operand is represented by formula (1), the offsets of the tensor data indicated by the descriptor in the data storage space are offset x and offset y respectively, and the size is size x * size y. The starting data address PA1 of the tensor data indicated by the descriptor in the data storage space is (x,y) The following formula (6) can be used to determine:

[0060] PA1 (x,y) = PA start + (offset y - 1) * ori x + offset x (6)

[0061] The data starting address PA1 determined according to the above formula (6) (x,y) , combined with the offsets offset x and offset y, and the size size x and size y of the storage area, the storage area of the tensor data indicated by the descriptor in the data storage space can be determined.

[0062] In a possible implementation, when the operand further includes a data description position for the descriptor, the data address of the data corresponding to the operand in the data storage space can be determined according to the content of the descriptor and the data description position. In this way, part of the data (for example, one or more data) in the tensor data indicated by the descriptor can be processed.

[0063] For example, the content of the descriptor in the operand is represented by formula (2), the offsets of the tensor data indicated by the descriptor in the data storage space are offset x and offset y respectively, the size is size x * size y, and the data description position for the descriptor included in the operand is (x q , y q ). The data address PA2 of the tensor data indicated by the descriptor in the data storage space is (x,y) The following formula (7) can be used to determine:

[0064] PA2 (x,y) = PA start + (offset y + y q - 1) * ori x + (offset x + x q ) (7)

[0065] In a possible implementation, the descriptor can indicate the data blocks. Data blocks can effectively speed up the operation and improve the processing efficiency in many applications. For example, in graphics processing, convolution operation often uses data blocks for fast operation processing.

[0066] Figure 1B A schematic diagram of data blocks in a data storage space according to an embodiment of the present disclosure is shown. As shown, the data storage space 26 also stores two-dimensional data in a row-major manner, which can be represented by (x, y) (where the X axis is horizontal to the right, and the Y axis is vertical downward). The size in the X axis direction (the size of each row, or the total number of columns) is ori_x (not shown in the figure), and the size in the Y axis direction (the total number of rows) is ori_y (not shown in the figure). Unlike the tensor data Figure 1B , Figure 1A the tensor data stored in the data storage space 26 includes a plurality of data blocks. Figure 1B In this case, the descriptor needs more parameters to represent these data blocks. Taking the X axis (X dimension) as an example, the following parameters can be involved: ori_x, x.tile.size (size 27 in the block), x.tile.stride (stride in the block, i.e. the distance between the first point of the first small block and the first point of the second small block), x.tile.num (the number of blocks, which is 3 in the example shown in

[0067] Figure 1B In a possible implementation, the descriptor can include an identifier of the descriptor and / or content of the descriptor. The identifier of the descriptor is used to distinguish the descriptor, for example, the identifier of the descriptor can be its number; the content of the descriptor can include at least one shape parameter representing the shape of the tensor data. For example, the tensor data is 3-dimensional data, and in the three dimensions of the tensor data, the shape parameters of two dimensions are fixed, and the content of the descriptor of the tensor data can include a shape parameter representing the shape of the other dimension of the tensor data.

[0068] In a possible implementation, the data address of the data storage space corresponding to each descriptor can be a fixed address. For example, a separate data storage space can be allocated for the tensor data, and the starting address of each tensor data in the data storage space corresponds to the descriptor one by one. In this case, the circuit or module responsible for parsing the computing instruction (for example, an entity outside the computing device of the present disclosure) can determine the data address of the data corresponding to the operand in the data storage space according to the descriptor.

[0069] In a possible implementation, the data address of the data storage space corresponding to each descriptor can be a fixed address. For example, a separate data storage space can be allocated for the tensor data, and the starting address of each tensor data in the data storage space corresponds to the descriptor one by one. In this case, the circuit or module responsible for parsing the computing instruction (for example, an entity outside the computing device of the present disclosure) can determine the data address of the data corresponding to the operand in the data storage space according to the descriptor. ​

[0070] In a possible implementation, when the data address of the data storage space corresponding to the descriptor is a variable address, the descriptor can also be used to indicate the address of the N-dimensional tensor data, wherein the content of the descriptor can further include at least one address parameter representing the address of the tensor data. For example, the tensor data is 3-dimensional data, and when the descriptor points to the address of the tensor data, the content of the descriptor can include one address parameter representing the address of the tensor data, for example, the starting physical address of the tensor data, or can include multiple address parameters of the address of the tensor data, for example, the starting address of the tensor data + address offset, or the address parameters of the tensor data based on each dimension. A person skilled in the art can set the address parameter according to actual needs, and the present disclosure does not limit this.

[0071] In a possible implementation, the address parameter of the tensor data can include the reference address of the data reference point of the descriptor in the data storage space of the tensor data. The reference address can be different according to the change of the data reference point. The present disclosure does not limit the selection of the data reference point.

[0072] In a possible implementation, the reference address can include the starting address of the data storage space. When the data reference point of the descriptor is the first data block of the data storage space, the reference address of the descriptor is the starting address of the data storage space. When the data reference point of the descriptor is other data in the data storage space except the first data block, the reference address of the descriptor is the address of the data block in the data storage space.

[0073] In a possible implementation, the shape parameter of the tensor data includes at least one of the following: the size of the data storage space in at least one direction of the N-dimensional direction, the size of the storage region in at least one direction of the N-dimensional direction, the offset of the storage region in at least one direction of the N-dimensional direction, the position of at least two vertices at the diagonal position of the N-dimensional direction relative to the data reference point, and the mapping relationship between the data description position of the tensor data indicated by the descriptor and the data address. The data description position is the mapping position of the point or region in the tensor data indicated by the descriptor. For example, when the tensor data is 3-dimensional data, the descriptor can use three-dimensional space coordinates (x, y, z) to represent the shape of the tensor data, and the data description position of the tensor data can be the position of the point or region in the three-dimensional space mapped by the tensor data represented by the three-dimensional space coordinates (x, y, z).

[0074] It should be understood that a person skilled in the art can select the shape parameter representing the tensor data according to actual conditions, and the present disclosure does not limit this. By using the descriptor in the data access process, the association between data can be established, thereby reducing the complexity of data access and improving the instruction processing efficiency.

[0075] Figure 2 A schematic block diagram of a processing device according to embodiments of the present disclosure is shown. As shown, the processing device 200 includes a control module 210, an operation module 220, and a storage module 230. Figure 2

[0076] The control module 210 can be configured to control the operation of the processing device 200, such as reading instructions from memory or externally incoming instructions, decoding the instructions, issuing micro-operation control signals to corresponding components, etc. Specifically, the control module 210 can be configured to control the execution unit 220 to perform corresponding processing according to received instructions. The instructions can include, but are not limited to, data access instructions, operation instructions, descriptor management instructions, and synchronization instructions, etc. The present disclosure does not limit the specific types of instructions and the specific ways of decoding.

[0077] The decoded instructions include an operation code and an operand. When the instructions involve processing of tensor data, at least one operand of the instructions can include at least one descriptor indicating at least one of the following information: shape information of the tensor data and spatial information of the tensor data.

[0078] The operation module 220 is configured to execute specific instructions or operations under the control of the control module 210. The operation module 220 can include, but is not limited to, an arithmetic and logic unit (ALU), a memory access unit (MAU), a neural functional unit (NFU), etc. The present disclosure does not limit the specific hardware types of the execution unit.

[0079] The storage module 230 can be configured to store various information, including but not limited to instructions, information associated with descriptors, tensor data, etc. The storage module 230 can include various storage resources, including but not limited to internal memory and external memory. The internal memory can include, for example, registers, on-chip SRAM, or other media cache. The external memory can include, for example, off-chip memory. The present disclosure does not limit the specific implementation of the storage module.

[0080] ​Optionally or additionally, the processing device 200 can further comprise a tensor interface unit (TIU) 240. The tensor interface unit 240 can be configured to implement operations associated with descriptors under the control of the control module 210. These operations can include, but are not limited to, registration, modification, deregistration, resolution of descriptors; read and write of descriptor contents, etc. The present disclosure does not limit the specific hardware type of the tensor interface unit. In this way, the operations associated with descriptors can be implemented by dedicated hardware, further improving the access efficiency of tensor data.

[0081] In some embodiments of the present disclosure, the tensor interface unit 240 can be configured to resolve a descriptor included in an operand of an instruction. For example, the tensor interface unit can resolve shape information of tensor data included in the descriptor to determine a data address of data corresponding to the operand in a data storage space.

[0082] Although the control module 210 and the tensor interface unit 240 are shown as two separate modules in Figure 2 , those skilled in the art can understand that the two modules / units can also be implemented as one module or more modules, and the present disclosure does not limit in this regard.

[0083] The data processing device 200 can be implemented by a general-purpose processor (such as a central processing unit CPU, a graphics processing unit GPU) and / or a special-purpose processor (such as an artificial intelligence processor, a scientific computing processor, or a digital signal processor, etc.), and the present disclosure does not limit the specific type of the data processing device.

[0084] When hardware executes instructions in parallel, if there is a dependency between the instructions executed in parallel, it can cause the execution result to be incorrect. For example, if two instructions executed in parallel access the same storage unit, and at least one of the two instructions is a write instruction to the storage unit, there is a dependency between the two instructions, such as a write-after-read dependency, a write-after-write dependency, or a read-after-write dependency. At this time, if the latter instruction is executed before the former instruction, it will cause the execution to be incorrect. Therefore, the order consistency of the execution of these instructions must be guaranteed, for example, by forcing sequential execution, that is, the latter instruction must wait for the former instruction to complete before it can be executed.

[0085] As can be known from the foregoing description of the tensor data, the tensor data is usually a multi-dimensional array and has a large amount of data, and thus the processing time of the instruction for the tensor data is usually longer than that of the scalar data. If the tensor data is still processed according to the previous order execution manner, the processing time is too long and the efficiency is low. In view of this, in the embodiments of the present disclosure, an operation level instruction parallel scheme is provided, in which the parallel execution of operations is limited based on the fine-grained region of the shape coordinate space of the tensor data to which the operations are directed, so that the parallel execution potential of the operations can be tapped. Therefore, according to the embodiments of the present disclosure, when the hardware is executed in parallel, the consistency of the execution order can be ensured, and the parallel degree of the operations can be improved, thereby ensuring the accuracy and efficiency of the processing.

[0086] Figure 3A An exemplary flowchart of a processing method 300 according to an embodiment of the present disclosure is shown. The processing method 300 may, for example, be implemented by the processing device 200 of Figure 2 .

[0087] As shown in Figure 3A , the method 300 starts at step S301, obtaining a first operation of an instruction. This step may, for example, be performed by the control module 210 of Figure 2 . In some embodiments, the first operation is an operation for tensor data, and the shape coordinate space of the tensor data includes at least one fine-grained region. In some embodiments, the fine-grained region can include one or more adjacent coordinate points of the shape coordinate space of the tensor data. One fine-grained region is the smallest unit of the operation.

[0088] It should be noted that the operation involved in the present disclosure can be a basic operation supported by the processor hardware, or a micro-instruction (such as a request signal, etc.) obtained by parsing the basic operation. The present disclosure does not limit the specific type of operation. The processing device of the present disclosure can execute two operations in parallel, or more than two operations in parallel, and the present disclosure does not limit the number of operations executed in parallel. The two operations executed in parallel can belong to the same instruction, or belong to different instructions, and the present disclosure does not limit in this regard.

[0089] When the hardware executes the instruction in parallel, the processor can execute multiple operations in parallel. In order to avoid memory conflict, when the multiple operations executed in parallel by the processor are all for the same data, the processor will only execute one of the multiple operations, while blocking the other operations, thereby reducing the efficiency of the processor. In the embodiments of the present disclosure, the shape coordinate space of the processed tensor data is further divided into multiple fine-grained regions, and whether the operations can be executed in parallel is determined based on the fine-grained regions, thereby greatly improving the efficiency of the processor.

[0090] In some embodiments, the shape, size and / or number of the fine-grained regions can be determined based at least in part on at least one of: a computing capability of the hardware; a bandwidth of the hardware; and a size of the shape coordinate space of the tensor data. The hardware computing capability can be an amount of data processed in parallel by the hardware in one computing cycle, and the hardware bandwidth can be a data transfer capability, such as an amount of data transferred per unit time.

[0091] For example, a processor applying the processing method of the embodiments of the present disclosure has a hardware computing capability of processing 100 bits of data in parallel in one computing cycle, and a hardware bandwidth of transferring 200 bits of data per unit time. For a two-dimensional tensor data with a size of 100*100 bits, the shape coordinate space of the tensor data can be divided into 100 fine-grained regions according to the hardware computing capability, wherein each fine-grained region includes 100 bits of data; or the shape coordinate space can be divided into 50 fine-grained regions according to the hardware bandwidth, wherein each fine-grained region includes 200 bits of data.

[0092] It should be understood that the hardware computing capability and the hardware bandwidth can be different according to different processor hardware, and the present disclosure does not limit the hardware computing capability and the hardware bandwidth. In this way, the size and / or number of the fine-grained regions can be determined according to the processing capability (hardware computing capability and / or hardware bandwidth) of the processor, so that the division result of the fine-grained regions is more in line with the needs of different hardware usage environments, and the operations performed according to the fine-grained regions are synchronized with the processing capability of the processor, so that the execution efficiency of the hardware can be as high as possible, thereby improving the processing efficiency of the processor.

[0093] It should be noted that the shapes and sizes of the plurality of fine-grained regions can be the same or different. For example, the first operation can carry a first fine-grained shape and size (e.g., the coordinate point positions and number of each fine-grained region), and can set the first fine-grained shape to a 8*8 = 64 coordinate point square (assuming a two-dimensional tensor). The second operation can carry a second fine-grained shape and size (e.g., the coordinate point positions and number of each fine-grained region), and can set the second fine-grained shape to a 16*16 = 256 coordinate point square. That is, when the first operation is executed, each 8*8 = 64 coordinate point square is taken as a fine-grained region, and when the second operation is executed, each 16*16 = 256 coordinate point square is taken as a fine-grained region. Similarly, the first operation can carry a first fine-grained number (e.g., set to 4), and the second operation can carry a second fine-grained number (e.g., set to 8). That is, when the first operation is executed, the shape coordinate space is divided into 4 fine-grained regions, and when the second operation is executed, the shape coordinate space is divided into 8 fine-grained regions. It can be understood that the operation can also simultaneously carry the shape, size, and number of the fine-grained regions. The shape, size, and / or number of each fine-grained region can be determined according to requirements, and the present disclosure does not limit this.

[0094] Continuing Figure 3A In step S302, it is determined whether there is an ongoing second operation on the tensor data.

[0095] As mentioned above, when the instruction involves processing of the tensor data, the descriptor is included in the operand, through which information related to the tensor data can be obtained. Therefore, in some embodiments, the spatial information (e.g., spatial identifier ID) of the tensor data can be included in the descriptor, and the dependency relationship between instructions can be determined according to the spatial information of the tensor data. Since different spatial IDs represent that there is no dependency relationship between the spatial regions pointed to, the dependency relationship between two instructions can be quickly determined according to whether the spatial IDs of the tensor data processed by the two instructions are the same, that is, whether the same tensor data is operated on.

[0096] In some embodiments, the processor can determine whether there is an ongoing second operation for the tensor data by querying the data storage region corresponding to the tensor data. For example, the processor can determine whether the data storage region corresponding to the tensor data is occupied by querying the occupancy state list. If the data storage region is occupied, the processor determines that there is an ongoing second operation for the tensor data. The occupancy state list can be pre-set and stored on the memory, or can be generated before the processor starts to perform a certain task and logged after the task is completed. When the occupancy state of each data storage region changes, the processor updates the content of the occupancy state list to record the occupancy state of each data storage region. The present disclosure does not limit the determination method of whether there is an ongoing second operation for one or more tensor data.

[0097] Next, in step S303, when there is such a second operation, it is determined whether the first fine-grained region currently targeted by the first operation overlaps with the second fine-grained region currently targeted by the second operation.

[0098] The first fine-grained region and the second fine-grained region can be any fine-grained region in the shape coordinate space of the tensor data. It can be understood that the operation for the tensor data is the operation for each fine-grained region in the shape coordinate space of the tensor data. For example, assuming that the tensor data A is a two-dimensional matrix of 8 rows and 16 columns, the shape coordinate space thereof is a two-dimensional space, and each 2 rows and 4 columns is a fine-grained region, the shape coordinate space of the tensor data includes 16 fine-grained regions. The write operation for the tensor data A can be regarded as the write operation for the 16 fine-grained regions. The execution process can be that the first fine-grained region (1st-2nd row and 1st-4th column) is written, then the second fine-grained region (1st-2nd row and 5th-8th column) is written after the first fine-grained region is written, the third fine-grained region (1st-2nd row and 9th-12th column) is written after the second fine-grained region is written, and so on, until the 16th fine-grained region (7th-8th row and 13th-16th column) is written, and the write operation for the tensor data A is completed. Those skilled in the art can understand that the operation can also be performed on multiple fine-grained regions at a time, for example, two or more fine-grained regions are written at a time, until the operation on all regions is completed.

[0099] When there is an operation on the tensor data, the state of the fine-grained region in the shape coordinate space of the tensor data can include an operation-completed state, an operation-in-progress state and an un-operated state as the operation is performed; or for cases where it is not necessary to record whether the operation has been completed, the state can include an occupied state and a usable state. The state of the fine-grained region currently targeted by the operation is the operation-in-progress state or the occupied state. Thus, when there is an operation on the tensor data, it can be considered that there is an operation on one fine-grained region in the shape coordinate space of the tensor data, and the fine-grained region being operated or occupied, i.e., the fine-grained region currently targeted by the operation.

[0100] In a possible implementation, the first fine-grained region currently targeted by the first operation can include a fine-grained region targeted by the first operation to be performed, for example, the first fine-grained region when the operation is initially performed, and when it is specified to be performed in a predetermined order. The first fine-grained region currently targeted by the first operation can also include a fine-grained region currently targeted by the first operation being performed, which can be any fine-grained region. The second fine-grained region currently targeted by the second operation can be a fine-grained region currently targeted by the second operation being performed, which can be any fine-grained region.

[0101] In a possible implementation, when it is determined whether there is a second operation in progress on the tensor data before the first operation performs the operation on the tensor data, the first fine-grained region currently targeted by the first operation is the fine-grained region on which the first operation is to be performed. For example, before the first operation performs the operation on the tensor data, the first fine-grained region currently targeted by the first operation is usually the first fine-grained region in the shape coordinate space of the tensor data. At this time, the first operation has not performed the operation on the first fine-grained region. The second fine-grained region currently targeted by the second operation in progress can be related to the execution process of the second operation. If the second operation also starts to be performed, the second fine-grained region can also be the first fine-grained region in the shape coordinate space of the tensor data. At this time, the first fine-grained region and the second fine-grained region overlap. If the second operation has completed the operation on the first fine-grained region, and the second fine-grained region currently targeted is the Pth fine-grained region (P is an integer greater than 1), the first fine-grained region and the second fine-grained region do not overlap.

[0102] In a possible implementation, when it is determined whether there is a second operation in progress on the tensor data during the operation of the first operation on the tensor data, the first fine-grained region can be determined according to the execution process of the first operation, the second fine-grained region can be determined according to the execution process of the second operation, and then it is determined whether the first fine-grained region and the second fine-grained region overlap.

[0103] In a possible implementation, if the beats of the operation execution processes are consistent, it can be determined whether there is a second operation being performed on the tensor data and whether the first fine-grained region and the second fine-grained region overlap only before the first operation performs the operation on the tensor data.

[0104] In a possible implementation, if the beats of the operation execution processes are inconsistent or it cannot be determined whether the beats are consistent, it can be determined whether there is a second operation being performed on the tensor data and whether the first fine-grained region and the second fine-grained region overlap after the first operation performs the operation on the first fine-grained region each time, to determine whether the first operation can continue to be performed.

[0105] In a possible implementation, whether the first fine-grained region and the second fine-grained region overlap can be determined according to a coordinate address, a pointer position, a fine-grained region identifier, and the like. For example, the coordinate addresses of the tensor data currently processed by the operations can be recorded, and the first fine-grained region currently processed by the first operation and the second fine-grained region currently processed by the second operation can be determined according to the coordinate address of the first operation and the coordinate address of the second operation and the correspondence between the coordinate addresses and the fine-grained regions, and then it can be determined whether the first fine-grained region and the second fine-grained region overlap. The coordinate address and the fine-grained region are both defined based on the shape coordinate space of the tensor data, and therefore, after the fine-grained division of the shape coordinate space is known, the corresponding fine-grained region can be directly determined from the coordinate address. For another example, a pointer can be set for each operation, and the pointer points to the fine-grained region currently processed by the operation. The first fine-grained region currently processed by the first operation and the second fine-grained region currently processed by the second operation can be determined according to the pointer position of the first operation and the pointer position of the second operation, and then it can be determined whether the first fine-grained region and the second fine-grained region overlap. For yet another example, an identifier can be set for each fine-grained region, and whether the first fine-grained region and the second fine-grained region overlap can be determined by recording the identifier of the fine-grained region currently processed by the operation. The identifier can include any combination of letters, numbers, or symbols. Whether the first fine-grained region and the second fine-grained region overlap can also be determined in other manners, and the disclosure does not limit the basis for determining whether the first fine-grained region and the second fine-grained region overlap.

[0106] Then, in step S304, when the first fine-grained region and the second fine-grained region do not overlap, the first operation is performed.

[0107] In a possible implementation, if the first fine-grained region currently targeted by the first operation does not overlap with the second fine-grained region currently targeted by the second operation, the first fine-grained region can be a fine-grained region that has been operated on by the second operation, or the first fine-grained region can be a fine-grained region that does not need to be operated on by the second operation, and in this case, performing the first operation does not affect the operation process and operation result of the second operation, and the first operation can be performed.

[0108] According to this embodiment, when the shape coordinate space of the tensor data targeted by the first operation includes at least one fine-grained region and there is a second operation being performed on the tensor data, it is determined whether the first fine-grained region currently targeted by the first operation overlaps with the second fine-grained region currently targeted by the second operation, and when the two do not overlap, the first operation is performed. In this way, the first operation and the second operation can be performed on the fine-grained regions that do not overlap, so that the first operation and the second operation can be performed on the same tensor data at the same time, and the processing efficiency of the processor is improved.

[0109] In a possible implementation, the processing method 300 can further include: blocking the first operation when the first fine-grained region overlaps with the second fine-grained region.

[0110] In a possible implementation, the overlap between the first fine-grained region and the second fine-grained region includes complete overlap or partial overlap. When the first fine-grained region and the second fine-grained region overlap, if the first operation is performed, the operation of the first operation on the overlapping region can affect the execution of the second operation, causing the operation result of the second operation to be inaccurate, or can affect the execution of the first operation, causing the operation result of the first operation to be inaccurate. In this case, the first operation can be blocked, that is, the execution of the first operation is paused, and the first operation can be performed after the second operation is completed on the second fine-grained region currently targeted by the second operation. That is, when the first fine-grained region and the second fine-grained region do not overlap, the first operation is performed.

[0111] In this embodiment, when the first fine-grained region and the second fine-grained region overlap, the first operation is blocked, which can avoid operation errors and inaccurate operation results caused by the overlap of the fine-grained regions of the operations, and ensures the correctness of the operations.

[0112] In some embodiments, at least one of the first operation and the second operation is a write operation. That is, when the operation on the target data is write-after-read (the second operation is a write operation and the first operation is a read operation), read-after-write (the second operation is a read operation and the first operation is a write operation), or write-after-write (both the second operation and the first operation are write operations), there is a dependency between the two operations, and the method in the embodiments of the present disclosure can be used. In these embodiments, by dividing the shape coordinate space of the target data into one or more fine-grained regions and performing the operations in units of the fine-grained regions, the read-after-write, write-after-read, write-after-write, and the like can be correctly executed to obtain accurate execution results, and the waiting time between operations can be reduced to improve the execution efficiency of the processor.

[0113] In the embodiments of the present disclosure, based on the fine-grained region division of the shape coordinate space of the tensor data, a processing method for determining the execution range of an operation based on the coordinate space range expressed by the fine-grained region is also provided.

[0114] Figure 3B An exemplary flowchart of the processing method according to the embodiments of the present disclosure is schematically shown. Similarly, Figure 3B The processing method can be implemented by, for example, Figure 2 The processing device 200.

[0115] As shown in Figure 3B , in step S311, a first coordinate space range of a number of tensors allowed to be used by the first operation is determined. This step can be performed by, for example, Figure 2 The control module 210. The first coordinate space range can be, for example, a part of the shape coordinate space of the tensor data involved in the first operation.

[0116] Then, in step S312, a second coordinate space range of the tensor data to be used when the first operation is executed is determined. This step can be performed by, for example, Figure 2 The execution unit 220. The second coordinate space range can be, for example, a part of the shape coordinate space of the tensor data involved in the first operation.

[0117] Finally, in step S313, the first operation is executed in a third coordinate space range determined by the intersection of the first coordinate space range and the second coordinate space range. This step can be performed by, for example, Figure 2 The execution unit 220.

[0118] The first coordinate space range, the second coordinate space range, and the third coordinate space range can all be expressed based on the fine-grained regions in the shape coordinate space of the tensor data, that is, the first, second, and third coordinate space ranges can be characterized in units of fine-grained regions.

[0119] In embodiments of the present disclosure, by limiting the coordinate space range of the tensor data that can be used when the operation is executed, for example, by limiting the operation to be executed within the third coordinate space range as described above, it can be guaranteed that the access of the instruction on each coordinate space range is sequential when the parallel execution of the instruction is performed, thereby ensuring the accuracy and efficiency of the processing. Further, since the programming on the software side usually uses the spatial coordinates to refer to the data points or data blocks in the tensor data, by restricting the parallel execution of the operation by the coordinate space range of the tensor data, the code programming on the software side can be simplified, and the execution of the instruction is more conducive.

[0120] In some embodiments, the above-described fine-grained region overlap determination can be performed only under certain conditions. Figure 3A and Figure 3B The overlap determination of the fine-grained region is described, thereby shortening the determination time and accelerating the instruction processing.

[0121] Figure 3C An exemplary flowchart of a processing method according to another embodiment of the present disclosure is schematically shown.

[0122] As Figure 3C shown, in step S321, a first operation of an instruction is obtained. In some embodiments, the first operation is an operation on a tensor data, and the descriptor of the tensor data can be included in the operand of the operation.

[0123] Then, in step S322, it is determined whether there is a second operation on the tensor data being performed. The operation is similar to the step S302 described above in conjunction with Figure 3A , which will not be repeated here.

[0124] If it is determined that there is no such second operation, the method can directly jump to step S326, i.e., the first operation is executed. This means that there is no second operation that can conflict with the first operation, so the first operation can be executed immediately. When there are other operations being executed, the first operation is executed in parallel with these other operations at this time.

[0125] If it is determined that there is such a second operation, i.e., a conflict can occur, the method can proceed to step S323, where it is further determined whether the data operation range of the first operation and the second operation overlap. It can be understood that since the tensor data usually has a large dimension, the data operation range of different operations can be different. When the data operation ranges of different operations do not overlap with each other, the first operation can be executed in parallel with the second operation in advance without conflict.

[0126] The data operation range of an operation can be determined in various manners. In some embodiments, the data operation range of an operation can be determined based on the spatial information and / or shape information of the tensor data to be operated on. The spatial information and shape information of a tensor data can be found in the foregoing detailed description and will not be repeated here. The shape information of a tensor data can be used to determine the access address of an operation, and thus determine whether there is overlap between the data operation ranges of two operations. The access address can be a coordinate space address of a tensor data, or a storage space address of a tensor data, and the present disclosure is not limited in this respect.

[0127] If it is determined in step S323 that the data operation ranges of the first operation and the second operation do not overlap, the method can proceed to step S326, i.e., the first operation is executed. This means that even if the first operation and the second operation access the same tensor data (determined in step S322), as long as the data operation ranges of the first operation and the second operation do not overlap, i.e., they access mutually non-overlapping parts of the same tensor data, the first operation can be executed in parallel with the second operation.

[0128] If it is determined in step S323 that the data operation ranges of the first operation and the second operation overlap, the method can proceed to step S324, where it is further determined whether the fine-grained regions currently targeted by the first operation and the second operation overlap. The specific determination method can refer to the description of the foregoing steps S221 and S222. Figure 3A and Figure 3B .

[0129] When it is determined in step S324 that the data operation ranges of the first operation and the second operation do not overlap, the method can proceed to step S326, i.e., the first operation is executed. In this way, based on the dynamic execution of an operation, it can be dynamically determined whether the fine-grained regions currently targeted overlap, so that the parallel execution of operations is realized at the level of fine-grained regions, and the parallel potential of operations is maximized.

[0130] If it is determined in step S324 that the data operation ranges of the two operations overlap, the first operation cannot be executed at this time, otherwise a conflict will occur. Therefore, in step S325, the first operation is blocked.

[0131] In the embodiments of Figure 3C , by first performing a static pre-determination based on the data operation range of an operation, and only performing a dynamic fine-grained region overlap determination under certain conditions (i.e., when the data operation ranges overlap), the determination time can be effectively shortened, and the instruction processing speed can be accelerated.

[0132] The present disclosure also provides exemplary processing apparatuses for implementing the processing methods of Figure 3A , Figure 3B and Figure 3C . Figure 3DA schematic functional block diagram of a processing apparatus according to embodiments of the present disclosure is shown.

[0133] As shown in Figure 3D Processing apparatus 30 comprises an operation obtaining unit 31, a first determining unit 32, a second determining unit 33 and an executing unit 34.

[0134] Operation obtaining unit 31 is configured to obtain a first operation of an instruction. The first operation is an operation on tensor data, a shape coordinate space of which comprises at least one fine-grained region, each fine-grained region comprising one or more adjacent coordinate points of the shape coordinate space. First determining unit 32 is configured to determine whether there is a second operation on the tensor data being in progress. Second determining unit 33 is configured to, when there is such a second operation, determine whether a first fine-grained region currently targeted by the first operation overlaps with a second fine-grained region currently targeted by the second operation. Executing unit 34 is configured to, when the first fine-grained region does not overlap with the second fine-grained region, execute the first operation.

[0135] In some embodiments, second determining unit 33 can comprise a first determining sub-unit 331 and a second determining sub-unit 332. First determining sub-unit 331 is configured to determine a first coordinate space range of the tensor data allowed to be used by the first operation. Second determining sub-unit 332 is configured to determine a second coordinate space range of the tensor data to be used when executing the first operation. In these embodiments, executing unit 34 can be configured to execute the first operation within a third coordinate space range determined by an intersection of the first coordinate space range and the second coordinate space range. In these embodiments, the first coordinate space range, the second coordinate space range and the third coordinate space range are characterized using fine-grained regions in the shape coordinate space of the tensor data.

[0136] In some embodiments, processing apparatus 30 can further comprise a blocking unit 35 and a third determining unit 36. Blocking unit 35 can be configured to, when it is determined that the first operation overlaps with the fine-grained region currently targeted by the second operation, block the first operation to prevent a conflict from occurring.

[0137] In some embodiments, third determining unit 36 can be configured to make a prior static judgment, i.e., determine whether the data operation ranges of the first operation and the second operation overlap. Only when the data operation ranges overlap, the judgment of second determining unit 33 is made. Executing unit 34 can execute the first operation according to the judgment results of the respective determining units.

[0138] It is understood by those skilled in the art that Figure 3DThe units shown are divided according to their functional implementation. This division is merely exemplary; in actual implementation, two or more functions may be implemented in the same hardware unit, and a single function may be implemented in two hardware units. For example, in one implementation, the operation acquisition unit 31, the first determination unit 32, and the optional third determination unit 36 ​​may be included in... Figure 2 The control module 210 of the processing device 200 shown contains the second determining unit 33 and the execution unit 34, while the arithmetic module 220 of the processing device 200 may include them. Alternatively, in another implementation, the operation acquisition unit 31, the first determining unit 32, the second determining unit 33, and the optional third determining unit 36 ​​may be included in... Figure 2 The control module 210 of the processing device 200 is included, while the execution unit 34 is included in the arithmetic module 220 of the processing device 200.

[0139] It should also be understood that the units included in the processing device 30 are related to the reference... Figure 3A , Figure 3B and Figure 3C The steps in the described method correspond to each other. Therefore, the operations and features described above for the method also apply to the processing device 30 and the units contained therein, and will not be repeated here.

[0140] Figure 4 This illustration schematically shows the division of the coordinate space range according to an embodiment of this disclosure. Figure 4 The illustration is based on two-dimensional data; however, those skilled in the art will understand that the same approach can be applied to three-dimensional or even more dimensional tensor data.

[0141] like Figure 4 As shown, the shape coordinate space 400 of the two-dimensional tensor data is divided into 12 fine-grained regions, namely 4001, 4002, ..., 4011 and 4012. Within each fine-grained region, access is guaranteed to be sequential. Any data element (e.g., a data point) in this tensor data can be represented by two-dimensional spatial coordinates (x, y) (where the X-axis is horizontal to the right and the Y-axis is vertically downward). Clearly, the coordinates of any data element in this tensor data will not exceed the maximum size of this shape coordinate space.

[0142] In some embodiments, the first coordinate space range can be defined as all fine-grained regions in the shape coordinate space of the tensor data that are not currently used by the prior operations associated with the first operation.

[0143] In these embodiments, for example when the preceding operation is in using the fine-grained regions 4004, 4008, 4009-4012, the spatial range (i.e. the first coordinate space range) that the first operation is allowed to use at this time can include the fine-grained regions 4001-4003 and the fine-grained regions 4005-4007, as shown by the hatched region.

[0144] Alternatively or additionally, in some embodiments, the fine-grained regions determined based on the coordinates of the tensor data to be accessed by the first operation are determined as the second coordinate space range.

[0145] In these embodiments, for example when it is predicted that the first operation will use fine-grained regions other than the fine-grained region 4001 and the fine-grained region 4002 (e.g. estimated according to the coordinates of the tensor data to be accessed), the spatial range (i.e. the second coordinate space range) that the first operation will use at this time can be determined as the fine-grained regions 4003-4012, as shown by the dotted region.

[0146] Then, according to embodiments of the present disclosure, the range that the first operation can operate in when actually executing, i.e. the third coordinate space range, is the intersection of the first coordinate space range and the second coordinate space range. As shown in the current example, the third coordinate space range is the region that is both hatched and dotted, i.e. the fine-grained regions 4003 and 4005-4007 in Figure 4 Figure 4

[0147] In some embodiments, the first coordinate space range, the second coordinate space range and the third coordinate space range can be directly characterized using the identifiers of the fine-grained regions included in each. For example, in the example shown in Figure 4

[0148] It is considered that in most cases, the access to the tensor data is usually in a certain dimension, and the access coordinates are gradually incremented, from front to back, to traverse the data units at each coordinate point in the tensor data.

[0149] ​​​Therefore, in other embodiments, the first coordinate space range is characterized using an upper bound on the coordinates of one or more dimensions of the tensor data for a fine-grained region that allows the first operation to be used; and / or the second coordinate space range is characterized using a lower bound on the coordinates of one or more dimensions of the tensor data for a fine-grained region that the first operation is expected to use. By leveraging this dimensional-order access characteristic of tensor data, the first or second coordinate space range can be characterized using only the upper or lower bounds, thereby simplifying control information and corresponding control methods.

[0150] Still with Figure 4 For example, as mentioned earlier, the first coordinate space extent can be characterized using fine-grained regions located at the upper bounds of one or more dimensions of the tensor data that allow the first operation to use. For instance, when the preceding operation uses the rightmost column and the bottom row, totaling six fine-grained regions, the spatial extent allowed for the first operation (i.e., the first coordinate space extent) could include the top two rows and three columns, totaling six fine-grained regions, as shown by the shaded area. In this case, in Figure 4 In this example, the first coordinate space range can be characterized by the fine-grained regions where the upper bound of the X axis 411 and the upper bound of the Y axis 421 are located. In this example, the first coordinate space range can be characterized by fine-grained regions 4003 and 4005, which indicate that the data coordinates accessed by the first operation cannot exceed the fine-grained region 4003 in the X dimension and cannot exceed the fine-grained region 4005 in the Y dimension.

[0151] Similarly, the second coordinate space extent can be characterized using fine-grained regions of coordinates that are expected to be used by the first operation and lie at the lower bound of one or more dimensions of the tensor data. For example, when it is determined, based on the coordinates of the tensor data that the first operation will access, that the first operation will use a fine-grained region other than the two fine-grained regions to the left of the first row, the second coordinate space extent can be determined to include the remaining 10 fine-grained regions, as shown in the dotted filling portion. At this point, in Figure 4 In this example, the second coordinate space range can be characterized by the fine-grained regions where the lower bound of the X-axis (412) and the lower bound of the Y-axis (422) are located. In this example, the second coordinate space range can be characterized by fine-grained regions 4002 and 4001, indicating that when performing the first operation, data in the tensor data whose X-dimensional dimension is lower than fine-grained region 4002 and whose Y-dimensional dimension is lower than fine-grained region 4001 will not be accessed.

[0152] The first operation, when actually executed, can operate within the third coordinate space, which is the intersection of the first and second coordinate spaces. For example... Figure 4 As shown, in the current example, the third coordinate space range is the area where both diagonal shading and point filling exist, that is...Figure 4 The "inverted L-shaped" region in the text.

[0153] There are several ways to determine the first coordinate space range and the second coordinate space range.

[0154] In some embodiments, at least one of the following may be considered to determine the first coordinate space range: the order of operations; the operands involved in the operations; and the second coordinate space range of the preceding operation. For example, in embodiments that use upper and lower bounds to characterize the coordinate space range, the lower bound of the coordinate space of the tensor data used by the preceding operation or instruction may be used as the upper bound of the tensor data used by the current new instruction.

[0155] In one example, when the first operation (i.e. the current operation) is a read operation, the upper bound of the coordinate space is the lower bound of the coordinate space of the most recent (i.e., the preceding or previous operation) write operation on the tensor data.

[0156] In another example, when the first operation is a write operation, the upper bound of the coordinate space is the minimum of the lower bound of the coordinate space for the most recent write operation on the tensor data and the lower bound of the coordinate space for all read operations on the tensor data between those two write operations. By selecting the minimum value, it can be ensured that the execution of the first operation does not affect the execution of any preceding operation.

[0157] Optionally or additionally, the second coordinate space range can be determined based on at least one of the following: the execution range of the operation; the access mode of the operation; and the current execution state of the operation. For example, in an embodiment that uses an upper and lower bound to characterize the coordinate space range, the above factors can be considered comprehensively to determine the second coordinate space range, ensuring that when accessing tensor data by dimension, the coordinates in the corresponding dimension are not less than the lower bound of the coordinate space. Furthermore, the maximum value of the lower bound of the coordinate space should be provided as much as possible, so that the accessible space range left for subsequent operations or instructions will be larger.

[0158] In one example, when the access pattern of the first operation is sequential and consecutive, the lower bound of the coordinate space can be determined based on the minimum access coordinates of the first operation. For example, the lower bound of the coordinate space can be determined as the fine-grained region where the minimum access coordinates are located. Figure 4 As shown, when the first operation accesses data according to the X dimension, assuming the smallest X coordinate of the accessed data is A, which is located in the second fine-grained region from the left, the lower bound of X can be determined as the second fine-grained region; when the first operation accesses data according to the Y dimension, assuming the smallest Y coordinate of the accessed data is B, which falls in the third fine-grained region from the top, the lower bound of Y can be determined as the third fine-grained region.

[0159] In another example, when the access pattern of the first operation is regular access, the coordinate space lower bound can be determined based on the regularity. For example, in a convolution operation, block access of tensor data can be required, and thus the coordinate space lower bound can be determined according to the block regularity of the convolution operation.

[0160] In yet another example, when the access pattern of the first operation cannot be determined, the coordinate space lower bound can be determined based on a predetermined setting. For example, the coordinate space lower bound can be a default value, such as 0 or 1 size of one or more fine-grained regions.

[0161] In some embodiments, the first and second coordinate space ranges can be determined based on a pre-partition of the shape coordinate space of the tensor data. Specifically, the shape coordinate space of the tensor data can be first partitioned into a number of spatial blocks, such as evenly or unevenly in each dimension, each spatial block including one or more fine-grained regions. For example, still referring to Figure 4 , the shape coordinate space of the tensor data is partitioned into 6 spatial blocks, each spatial block A, B, C, D, E and F including 2 fine-grained regions.

[0162] In these embodiments, the spatial blocks in which the previous operation has completed can be determined as the first coordinate space range in the shape coordinate space of the tensor data; and the spatial blocks to be accessed based on the coordinates of the tensor data to be accessed by the first operation can be determined as the second coordinate space range.

[0163] For example, when the previous operation has completed access to spatial blocks A, B, and spatial block C is being used, the spatial range allowed to be used by the first operation (i.e. the first coordinate space range) at this time can include spatial blocks A and B. Further, for example, when it is predicted that the first operation will use spatial blocks A and B (e.g. estimated according to the coordinates of the tensor data to be accessed), the spatial range to be used by the first operation (i.e. the second coordinate space range) at this time can be determined as spatial blocks A and B.

[0164] In turn, according to embodiments of the present disclosure, the range in which the first operation can actually operate when executed, i.e. the third coordinate space range, is the intersection of the first coordinate space range and the second coordinate space range. In the current example, the third coordinate space range is spatial blocks A and B.

[0165] Optionally or additionally, in some embodiments, within the third coordinate space range, the first operation can be executed based on at least one of the following orders: a predetermined spatial block order; and / or a predetermined fine-grained region order.

[0166] In some implementations, after the shape coordinate space of the tensor data to be operated on is pre-partitioned into spatial blocks, such as Figure 4For example, if there are two instructions with dependency relationship, and the operation objects or the used spaces of the two instructions are the whole tensor data, the instructions can be operated on the spatial blocks in the order. For example, assuming that the former instruction 1 is to write the tensor data, and the latter instruction 2 is to read the tensor data, the instruction 1 can first perform the write operation on the spatial block A, and then perform the write operation on the spatial block B. At this time, the instruction 2 can start to perform the read operation on the spatial block A. If the division of the spatial blocks makes the execution clock of the instruction 2 consistent with that of the instruction 1, in the subsequent time, when the instruction 1 starts to perform the write operation on the spatial block C, the instruction 2 has completed the read operation on the spatial block A and starts to perform the read operation on the spatial block B. In this way, it can be seen that the division of the spatial blocks is beneficial to the parallel execution of the instructions, and the agreement of the order of the spatial blocks is beneficial to simplifying the operation scheduling, shortening the processing time, and improving the processing efficiency.

[0167] Alternatively or additionally, in some implementations, when the first operation is performed in a single spatial block, the predetermined fine-grained region order can also be performed. When the operation range of the instructions executed in parallel is further controlled based on the fine-grained region of the current operation in a single spatial block, this way of performing in the predetermined fine-grained region order is beneficial to simplifying the operation scheduling, and the principle is similar to the principle of performing in the predetermined spatial block order described above, which will not be described here.

[0168] In yet some embodiments, the first and second coordinate space ranges can also be determined by combining the dynamic determination with the pre-division of the shape coordinate space of the tensor data. Specifically, the shape coordinate space of the tensor data can be first divided into a plurality of spatial blocks, for example, uniformly or non-uniformly divided in each dimension. Then, in each spatial block, the first and second coordinate space ranges can be dynamically determined based on the operation. The specific determination manner can refer to the description above, which will not be described here. In these implementations, when the exact position of the second coordinate space range in a spatial block cannot be determined, the range corresponding to the spatial block can be defaulted.

[0169] In some embodiments, the pre-partitioning of the shape coordinate space of the tensor data can be based on at least one of: a processing capability of the hardware; a pre-set parameter; and a size of the shape coordinate space of the tensor data. The processing capability of the hardware may, for example, include but is not limited to a data bit width that the hardware can process. Based on the data bit width that the hardware can process, the shape coordinate space of the tensor data can be partitioned to fully exploit the processing capability of the hardware and improve the parallel processing efficiency. The pre-set parameter may, for example, directly specify the number of space blocks to be partitioned, the size of each dimension of the space blocks, and the like. Based on the size / dimension of the shape coordinate space of the tensor data, the shape coordinate space of the tensor data can be partitioned. For example, when the tensor data is a two-dimensional matrix with a size of M rows * N columns (M and N are both positive integers), each row can be evenly partitioned into m parts and each column can be evenly partitioned into n parts, so that a total of m*n space blocks are obtained.

[0170] Although six space blocks are shown in Figure 4 which are evenly partitioned, various numbers of space blocks with unequal sizes can also be partitioned, and the present disclosure is not limited in the specific partitioning manner. The above describes a scheme for restricting the space range actually used by an operation when the hardware performs the operation in parallel, to ensure the sequential consistency of data processing while improving the parallel processing efficiency. Those skilled in the art can understand that the current operation (for example, the first operation described above) and the previous operation (or the preceding operation) can be operations in different instructions executed in parallel, respectively; and the current operation and the previous operation can also be different operations in the same instruction executed in parallel, respectively, and the present disclosure is not limited in this respect.

[0171] The processing method performed by the processing apparatus of the embodiments of the present disclosure has been described above with reference to the flowcharts. Those skilled in the art can understand that, since the operation performed in parallel is restricted based on the coordinate space range of the processed data, the parallel degree of the operation can be improved while ensuring the sequential consistency of the operation execution, thereby improving the processing efficiency. It should be noted that, for the above-mentioned method embodiments, in order to simply describe, they are all expressed as a combination of a series of actions, but those skilled in the art should know that the present disclosure is not limited by the order of the described actions, because according to the present disclosure, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should know that the embodiments described in the specification are all optional embodiments, and the actions and modules involved are not necessarily required by the present disclosure.

[0172] It is further noted that, although the steps of the method flowchart are shown in a sequence as indicated by arrows, the steps are not necessarily executed in the order as indicated by the arrows. Unless explicitly stated otherwise herein, the execution of the steps is not necessarily limited to the order as indicated by the arrows, and the steps can be executed in other orders. Moreover, at least some of the steps of the method flowchart can comprise multiple sub-steps or multiple stages, which are not necessarily executed at the same time, but can be executed at different times, and the execution of the sub-steps or stages is not necessarily executed in sequence, but can be executed in rotation or alternation with at least some of the other steps or sub-steps or stages of other steps.

[0173] Figure 5 is a structural diagram illustrating a combined processing device 500 according to an embodiment of the present disclosure. As shown in Figure 5 , the combined processing device 500 comprises a computing processing device 502, an interface device 504, other processing devices 506, and a storage device 508. According to different application scenarios, the computing processing device can comprise one or more computing devices 510, which can be configured to Figure 2 , the processing device 200 shown in Figure 4 , for performing the operations described herein in conjunction with the accompanying

[0174] In different embodiments, the computing processing device of the present disclosure can be configured to perform user-specified operations. In exemplary applications, the computing processing device can be implemented as a single-core artificial intelligence processor or a multi-core artificial intelligence processor. Similarly, the one or more computing devices included in the computing processing device can be implemented as an artificial intelligence processor core or a partial hardware structure of an artificial intelligence processor core. When multiple computing devices are implemented as an artificial intelligence processor core or a partial hardware structure of an artificial intelligence processor core, the computing processing device of the present disclosure can be considered to have a single-core structure or a homogeneous multi-core structure.

[0175] In exemplary operations, the computing processing device of the present disclosure can interact with other processing devices through the interface device to collectively accomplish user-specified operations. Depending on the implementation, the other processing devices of the present disclosure can include one or more types of processors, including general and / or special purpose processors such as central processing units (CPUs), graphics processing units (GPUs), artificial intelligence processors, etc. These processors can include, but are not limited to, digital signal processors (DSPs), application specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs) or other programmable logic devices, discrete gates or transistor logic, discrete hardware components, etc., and their number can be determined according to actual needs. As mentioned previously, only in terms of the computing processing device of the present disclosure, it can be considered as having a single-core structure or a homogeneous multi-core structure. However, when considering the computing processing device and the other processing devices collectively, both can be considered as forming a heterogeneous multi-core structure.

[0176] In one or more embodiments, the other processing devices can serve as the interface of external data and control for the computing processing device of the present disclosure (which can be embodied as the relevant operation device for artificial intelligence such as neural network operation), and perform basic controls including but not limited to data carrying, starting and / or stopping of the computing device, etc. In additional embodiments, the other processing devices can also cooperate with the computing processing device to collectively accomplish operation tasks.

[0177] In one or more embodiments, the interface device can be used to transmit data and control instructions between the computing processing device and the other processing devices. For example, the computing processing device can obtain input data from the other processing devices via the interface device and write into the storage device (or memory) on the computing processing device chip. Further, the computing processing device can obtain control instructions from the other processing devices via the interface device and write into the control cache on the computing processing device chip. Alternatively or optionally, the interface device can also read data in the storage device of the computing processing device and transmit to the other processing devices.

[0178] Additionally or alternatively, the combination processing apparatus of the present disclosure can further comprise a storage device. As shown in the figures, the storage device is connected with the computing processing apparatus and the other processing apparatus respectively. In one or more embodiments, the storage device can be used to save data of the computing processing apparatus and / or the other processing apparatus. For example, the data can be data that cannot be saved completely in the internal or on-chip storage device of the computing processing apparatus or the other processing apparatus.

[0179] In some embodiments, the present disclosure further discloses a chip (e.g. Figure 6 shown in FIG. 6B). In one implementation, the chip is a System on Chip (SoC), and integrates one or more combination processing apparatuses as shown in Figure 5 FIG. 6A. The chip can be connected with other related components through an external interface device (e.g. Figure 6 shown in FIG. 6B). The related components can be, for example, a camera, a display, a mouse, a keyboard, a network card or a wifi interface. In some application scenarios, other processing units (e.g. a video codec) and / or interface modules (e.g. a DRAM interface) can be integrated on the chip. In some embodiments, the present disclosure further discloses a chip package structure comprising the above chip. In some embodiments, the present disclosure further discloses a board card comprising the above chip package structure. The board card will be described in detail below. Figure 6

[0180] Figure 6 is a structural schematic diagram showing a board card 600 according to an embodiment of the present disclosure. As shown in Figure 6 FIG. 6A, the board card comprises a storage device 604 for storing data, which comprises one or more storage units 610. The storage device can be connected and perform data transmission with the controller device 608 and the chip 602 described above through, for example, a bus. Further, the board card further comprises an external interface device 606, which is configured to perform data relay or switching function between the chip (or the chip in the chip package structure) and an external device 612 (e.g. a server or a computer, etc.). For example, data to be processed can be transmitted from the external device to the chip through the external interface device. For another example, the computing result of the chip can be transmitted back to the external device via the external interface device. According to different application scenarios, the external interface device can have different interface forms, for example, it can adopt a standard PCIE interface, etc.

[0181] ​In one or more embodiments, the controller in the disclosed board can be configured to regulate the state of the chip. Therefore, in one application scenario, the controller may include a microcontroller (MCU) for regulating the operating state of the chip.

[0182] Based on the above combination Figure 5 and Figure 6 Based on the description, those skilled in the art will understand that this disclosure also discloses an electronic device or apparatus that may include one or more of the aforementioned boards, one or more of the aforementioned chips, and / or one or more of the aforementioned combined processing apparatus.

[0183] Depending on the application scenario, the electronic devices or apparatus disclosed herein may include servers, cloud servers, server clusters, data processing devices, robots, computers, printers, scanners, tablets, smart terminals, PC devices, IoT terminals, mobile terminals, mobile phones, dashcams, navigators, sensors, cameras, video cameras, projectors, watches, headphones, mobile storage, wearable devices, visual terminals, autonomous driving terminals, vehicles, home appliances, and / or medical devices. The vehicles include airplanes, ships, and / or vehicles; the home appliances include televisions, air conditioners, microwave ovens, refrigerators, rice cookers, humidifiers, washing machines, lights, gas stoves, and range hoods; the medical devices include MRI scanners, ultrasound machines, and / or electrocardiographs. The electronic devices or apparatus disclosed herein can also be applied in fields such as the Internet, IoT, data centers, energy, transportation, public management, manufacturing, education, power grids, telecommunications, finance, retail, construction sites, and healthcare. Furthermore, the electronic devices or apparatus disclosed herein can also be used in application scenarios related to artificial intelligence, big data, and / or cloud computing, such as cloud computing, edge computing, and terminal applications. In one or more embodiments, the high-computing-power electronic devices or apparatuses according to the present disclosure can be applied to cloud devices (e.g., cloud servers), while the low-power electronic devices or apparatuses can be applied to terminal devices and / or edge devices (e.g., smartphones or cameras). In one or more embodiments, the hardware information of the cloud devices and the hardware information of the terminal devices and / or edge devices are compatible with each other, so that suitable hardware resources can be matched from the hardware resources of the cloud devices to simulate the hardware resources of the terminal devices and / or edge devices based on the hardware information of the terminal devices and / or edge devices, so as to complete the unified management, scheduling and collaborative work of end-to-cloud or cloud-edge-end integration.

[0184] It should be noted that, for the purpose of clarity, the disclosure describes some methods and embodiments thereof as a series of acts and / or combinations thereof, but those skilled in the art will understand that the present disclosure is not limited to the order of the acts described. Those skilled in the art will understand and appreciate that some steps of the methods can be decided to be executed in other orders or at the same time with other steps. Further, those skilled in the art will understand and appreciate that some of the embodiments described in the disclosure can be considered optional, i.e., the acts or modules involved therein are not necessarily essential for the implementation of one or more of the aspects of the present disclosure. In addition, the disclosure describes some embodiments with different focuses according to different aspects. In view of this, those skilled in the art will understand that the parts not described in detail in some embodiments of the disclosure can also be seen from the relevant description of other embodiments.

[0185] In terms of specific implementation, based on the disclosure and teachings of the present disclosure, those skilled in the art can understand that the embodiments disclosed in the present disclosure can also be implemented in other ways not disclosed herein. For example, as for each unit in the electronic device or apparatus embodiments described above, the units are divided herein on the basis of logical functions, and other division manners can also be used in actual implementation. For another example, a plurality of units or components can be combined or integrated into another system, or some features or functions of the units or components can be selectively disabled. As for the connection relationship between different units or components, the connection discussed above in conjunction with the drawings can be a direct or indirect coupling between the units or components. In some scenarios, the aforementioned direct or indirect coupling involves a communication connection using an interface, wherein the communication interface can support electrical, optical, acoustic, magnetic or other forms of signal transmission.

[0186] In the present disclosure, the units described as separate components can or can not be physically separated, and the components shown as units can or can not be physical units. The aforementioned components or units can be located in the same place or distributed on a plurality of network units. In addition, according to actual needs, some or all of the units can be selected to achieve the purpose of the aspects described in the embodiments of the present disclosure. In addition, in some scenarios, a plurality of units in the embodiments of the present disclosure can be integrated into one unit or each unit physically exists separately.

[0187] In some implementation scenarios, the above-mentioned integrated units can be implemented in the form of software program modules. If implemented in the form of software program modules and sold or used as independent products, the integrated units can be stored in a computer-readable memory. Based on this, when the schemes of the present disclosure are embodied in the form of software products (for example, computer-readable storage media), the software products can be stored in the memory, which can include a number of instructions to make a computer device (for example, a personal computer, a server or a network device, etc.) execute part or all of the steps of the method described in the embodiments of the present disclosure. The aforementioned memory can include, but is not limited to, a variety of media capable of storing program codes, such as a U disk, a flash disk, a read-only memory (ROM), a random access memory (RAM), a mobile hard disk, a magnetic disk or an optical disk, etc.

[0188] In some implementation scenarios, the above-mentioned integrated units can be implemented in the form of software program modules. If implemented in the form of software program modules and sold or used as independent products, the integrated units can be stored in a computer-readable memory. Based on this, when the schemes of the present disclosure are embodied in the form of software products (for example, computer-readable storage media), the software products can be stored in the memory, which can include a number of instructions to make a computer device (for example, a personal computer, a server or a network device, etc.) execute part or all of the steps of the method described in the embodiments of the present disclosure. The aforementioned memory can include, but is not limited to, a variety of media capable of storing program codes, such as a U disk, a flash disk, a read-only memory (ROM), a random access memory (RAM), a mobile hard disk, a magnetic disk or an optical disk, etc.

[0189] The foregoing can be better understood in light of the following clauses:

[0190] Clause 1. A processing method, the method comprising:

[0191] a first operation of obtaining an instruction, the first operation being an operation on tensor data, a shape coordinate space of the tensor data including at least one fine-grained region, the fine-grained region including one or more adjacent coordinate points of the shape coordinate space;

[0192] determining whether there is an ongoing second operation on the tensor data;

[0193] when the second operation exists, determining whether a first fine-grained region currently targeted by the first operation overlaps with a second fine-grained region currently targeted by the second operation; and

[0194] when the first fine-grained region does not overlap with the second fine-grained region, executing the first operation.

[0195] Clause 2. The method of clause 1, wherein the method further comprises:

[0196] when the first fine-grained region overlaps with the second fine-grained region, blocking the first operation.

[0197] Clause 3. The method of any of clauses 1-2, wherein the method further comprises:

[0198] determining whether a data operation range of the first operation overlaps with a data operation range of the second operation;

[0199] when the data operation range overlaps, performing the determining whether a first fine-grained region currently targeted by the first operation overlaps with a second fine-grained region currently targeted by the second operation; and

[0200] when the data operation range does not overlap, executing the first operation.

[0201] Clause 4. The method of clause 3, wherein whether the data operation range of the first operation overlaps with the data operation range of the second operation is determined based on at least one of:

[0202] spatial information of the tensor data to be operated; and / or

[0203] shape information of the tensor data to be operated.

[0204] Clause 5. The method of any of clauses 1-4, wherein the method further comprises:

[0205] determining a first coordinate space range of the tensor data allowed to be used by the first operation;

[0206] determining a second coordinate space range of the tensor data to be used when the first operation is executed; and

[0207] performing the first operation within a third coordinate space range determined by an intersection of the first coordinate space range and the second coordinate space range;

[0208] wherein the first coordinate space range, the second coordinate space range, and the third coordinate space range are characterized using the fine-grained regions.

[0209] Clause 6. The method of clause 5, wherein the first coordinate space range is determined based on at least one of:

[0210] an order of operations;

[0211] operands involved in the operation;

[0212] a second coordinate space range of a preceding operation; and

[0213] a predetermined partitioning of a shape coordinate space of the tensor data.

[0214] Clause 7. The method of any of clauses 5-6, wherein the second coordinate space range is determined based on at least one of:

[0215] an execution range of the operation;

[0216] an access pattern of the operation;

[0217] a current execution state of the operation; and

[0218] a predetermined partitioning of a shape coordinate space of the tensor data.

[0219] Clause 8. The method of any of clauses 5-7, wherein:

[0220] determining the first coordinate space range comprises determining a coordinate space upper bound of one or more dimensions of the tensor data allowed for use by the first operation; and / or

[0221] determining the second coordinate space range comprises determining a coordinate space lower bound of one or more dimensions of the tensor data expected to be used by the first operation.

[0222] Clause 9. The method of any of clauses 1-8, wherein a size and / or a number of the fine-grained regions is determined based at least in part on at least one of:

[0223] a computational power of hardware;

[0224] a bandwidth of hardware; and

[0225] a size of a shape coordinate space of the tensor data.

[0226] Clause 10. The method of any of clauses 1-9, wherein at least one of the first operation and the second operation is a write operation.

[0227] Clause 11. The method of any of clauses 1-10, wherein:

[0228] the first operation and the second operation are operations in different instructions that are executed in parallel, respectively; or

[0229] the first operation and the second operation are different operations in a same instruction that are executed in parallel.

[0230] Clause 12. A processing device comprising:

[0231] an operation obtaining unit configured to obtain a first operation of an instruction, the first operation being an operation on tensor data, a shape coordinate space of the tensor data including at least one fine-grained region, the fine-grained region including one or more adjacent coordinate points of the shape coordinate space;

[0232] a first determination unit configured to determine whether there is a second operation on the tensor data that is in progress;

[0233] a second determination unit configured to, when there is the second operation, determine whether a first fine-grained region currently targeted by the first operation overlaps with a second fine-grained region currently targeted by the second operation; and

[0234] an execution unit configured to, when the first fine-grained region does not overlap with the second fine-grained region, execute the first operation.

[0235] Clause 13. The processing device of clause 12, wherein the processing device further comprises:

[0236] a blocking unit configured to, when the first fine-grained region overlaps with the second fine-grained region, block the first operation.

[0237] Clause 14. The processing device of any of clauses 12-13, wherein the processing device further comprises:

[0238] a third determination unit configured to determine whether a data operation range of the first operation overlaps with a data operation range of the second operation; and

[0239] the second determination unit is configured to, when the third determination unit determines that the data operation ranges overlap, perform the determination of whether the first fine-grained region currently targeted by the first operation overlaps with the second fine-grained region currently targeted by the second operation; and

[0240] The execution unit is configured to execute the first operation when the third determination unit determines that the data operation ranges do not overlap.

[0241] Clause 15. The processing device according to clause 14, wherein the third determination unit determines whether the data operation range of the first operation and the data operation range of the second operation overlap based on at least one of:

[0242] spatial information of the tensor data to be operated; and / or

[0243] shape information of the tensor data to be operated.

[0244] Clause 16. The processing device according to any of clauses 12-15, wherein the second determination unit further comprises:

[0245] a first determination sub-unit configured to determine a first coordinate space range of the tensor data allowed to be used by the first operation;

[0246] a second determination sub-unit configured to determine a second coordinate space range of the tensor data to be used when the first operation is executed; and

[0247] the execution unit is further configured to execute the first operation in a third coordinate space range determined by an intersection of the first coordinate space range and the second coordinate space range,

[0248] wherein the first coordinate space range, the second coordinate space range and the third coordinate space range are characterized using the fine-grained regions.

[0249] Clause 17. The processing device according to clause 16, wherein the first coordinate space range is determined based on at least one of:

[0250] an order of operations;

[0251] operands involved in the operation;

[0252] a second coordinate space range of a preceding operation; and

[0253] a predetermined partition of a shape coordinate space of the tensor data.

[0254] Clause 18. The processing device according to any of clauses 16-17, wherein the second coordinate space range is determined based on at least one of:

[0255] an execution range of the operation;

[0256] an access pattern of the operation;

[0257] a current execution state of the operation; and

[0258] a predetermined partition of a shape coordinate space of the tensor data.

[0259] Clause 19. The processing device according to any one of clauses 16-18, wherein:

[0260] the first determining subunit is further configured to determine a coordinate space upper bound of one or more dimensions of the tensor data allowed to be used by the first operation; and / or

[0261] the second determining subunit is further configured to determine a coordinate space lower bound of one or more dimensions of the tensor data expected to be used by the first operation.

[0262] Clause 20. The processing device according to any one of clauses 12-19, wherein a size and / or a number of the fine-grained regions are determined based at least in part on at least one of:

[0263] a computing capability of the hardware;

[0264] a bandwidth of the hardware; and

[0265] a size of a shape coordinate space of the tensor data.

[0266] Clause 21. The processing device according to any one of clauses 12-20, wherein at least one of the first operation and the second operation is a write operation.

[0267] Clause 22. The processing device according to any one of clauses 12-21, wherein:

[0268] the first operation and the second operation are operations in different instructions respectively executed in parallel; or

[0269] the first operation and the second operation are different operations in a same instruction respectively executed in parallel.

[0270] Clause 23. A chip, comprising the processing device according to any one of clauses 12-22.

[0271] Clause 24. A board card, comprising the chip according to clause 23.

Claims

1. A processing method, the method comprising: a first operation of obtaining an instruction, the first operation being an operation on tensor data, a shape coordinate space of the tensor data comprising at least one fine-grained region, the fine-grained region comprising one or more adjacent coordinate points of the shape coordinate space; determining whether there is an ongoing second operation on the tensor data; when there is the second operation, determining whether a first fine-grained region currently targeted by the first operation overlaps with a second fine-grained region currently targeted by the second operation; and when the first fine-grained region does not overlap with the second fine-grained region, executing the first operation; further comprising: determining a first coordinate space range of the tensor data allowed to be used by the first operation; determining a second coordinate space range of the tensor data to be used when executing the first operation; and executing the first operation within a third coordinate space range determined by an intersection of the first coordinate space range and the second coordinate space range; wherein the first, second, and third coordinate space ranges are characterized using the fine-grained regions.

2. The method of claim 1, wherein the method further comprises: when the first fine-grained region overlaps with the second fine-grained region, blocking the first operation.

3. The method of any of claims 1-2, wherein the method further comprises: determining whether a data operation range of the first operation overlaps with a data operation range of the second operation; when the data operation ranges overlap, performing the determining whether the first fine-grained region currently targeted by the first operation overlaps with the second fine-grained region currently targeted by the second operation; and when the data operation ranges do not overlap, executing the first operation.

4. The method of claim 3, wherein whether the data operation range of the first operation overlaps with the data operation range of the second operation is determined based on at least one of: spatial information of the tensor data to be operated on; and / or shape information of the tensor data to be operated on.

5. The method of claim 1, wherein the first coordinate space range is determined based on at least one of: an order of operations; an operand involved in the operation; a second coordinate space range of a preceding operation; and a predetermined partitioning of the shape coordinate space of the tensor data.

6. The method of claim 1, wherein the second coordinate space range is determined based on at least one of: an execution range of the operation; an access pattern of the operation; a current execution status of the operation; and a predetermined partitioning of the shape coordinate space of the tensor data.

7. The method of claim 1, wherein: determining the first coordinate space range comprises determining a coordinate space upper bound of one or more dimensions of the tensor data allowed to be used by the first operation; and / or determining the second coordinate space range comprises determining a coordinate space lower bound of one or more dimensions of the tensor data expected to be used by the first operation. ​ ​ 8. The method of claim 1, wherein, a size and / or a number of the fine-grained regions are determined based at least in part on at least one of: a computing capability of the hardware; a bandwidth of the hardware; and a size of a shape coordinate space of the tensor data.

9. The method of claim 1, wherein at least one of the first operation and the second operation is a write operation.

10. The method of claim 1, wherein: the first operation and the second operation are operations in different instructions executed in parallel, respectively; or the first operation and the second operation are different operations in a same instruction executed in parallel.

11. A processing apparatus comprising: an operation obtaining unit configured to obtain a first operation of an instruction, the first operation being an operation on tensor data, a shape coordinate space of the tensor data comprising at least one fine-grained region, the fine-grained region comprising one or more adjacent coordinate points of the shape coordinate space; a first determination unit configured to determine whether there is a second operation on the tensor data in progress; a second determination unit configured to, when there is the second operation, determine whether a first fine-grained region currently targeted by the first operation overlaps with a second fine-grained region currently targeted by the second operation; and an execution unit configured to, when the first fine-grained region does not overlap with the second fine-grained region, execute the first operation; wherein the second determination unit further comprises: a first determination sub-unit configured to determine a first coordinate space range of the tensor data allowed to be used by the first operation; a second determination sub-unit configured to determine a second coordinate space range of the tensor data to be used when the first operation is executed; and the execution unit is further configured to execute the first operation within a third coordinate space range determined by an intersection of the first coordinate space range and the second coordinate space range, wherein the first coordinate space range, the second coordinate space range and the third coordinate space range are characterized using the fine-grained regions.

12. The processing apparatus of claim 11, wherein the processing apparatus further comprises: a blocking unit configured to, when the first fine-grained region overlaps with the second fine-grained region, block the first operation.

13. The processing apparatus of any of claims 11-12, wherein the processing apparatus further comprises: a third determination unit configured to determine whether a data operation range of the first operation overlaps with a data operation range of the second operation; and the second determination unit is configured to, when the third determination unit determines that the data operation ranges overlap, perform the determination of whether the first fine-grained region currently targeted by the first operation overlaps with the second fine-grained region currently targeted by the second operation; and the execution unit is configured to, when the third determination unit determines that the data operation ranges do not overlap, execute the first operation.

14. The processing device of claim 13, wherein the third determining unit determines whether the data operation range of the first operation overlaps with the data operation range of the second operation based on at least one of the following: spatial information of the tensor data to be operated; and / or shape information of the tensor data to be operated.

15. The processing device of claim 11, wherein the first coordinate space range is determined based on at least one of the following: precedence order of operations; operands involved in the operations; second coordinate space range of a preceding operation; and predetermined partition of the shape coordinate space of the tensor data.

16. The processing device of claim 11, wherein the second coordinate space range is determined based on at least one of the following: execution range of the operation; access pattern of the operation; current execution status of the operation; and predetermined partition of the shape coordinate space of the tensor data.

17. The processing device of claim 11, wherein: the first determining subunit is further configured to determine an upper bound of the coordinate space of one or more dimensions of the tensor data allowed to be used by the first operation; and / or the second determining subunit is further configured to determine a lower bound of the coordinate space of one or more dimensions of the tensor data expected to be used by the first operation.

18. The processing device of claim 11, wherein, the size and / or number of the fine-grained regions are determined based at least in part on at least one of the following: computing power of the hardware; bandwidth of the hardware; and size of the shape coordinate space of the tensor data.

19. The processing device of claim 11, wherein at least one of the first operation and the second operation is a write operation.

20. The processing device of claim 11, wherein: the first operation and the second operation are operations in different instructions respectively executed in parallel; or the first operation and the second operation are different operations in a same instruction respectively executed in parallel.

21. A chip, characterized by the chip comprises the processing device of any one of claims 11-20.

22. A board card, characterized by the board card comprises the chip of claim 21.

Images