Data processing methods and apparatus, electronic devices and storage media

Abstract

This disclosure relates to a data processing method, apparatus, electronic device, and storage medium. The method includes: determining address information of a second tensor based on tensor information of a first tensor and access information of a second tensor to be accessed; and reading the second tensor from a global memory storing the first tensor to a local memory based on the address information of the second tensor; and triggering an interrupt in response to the local memory receiving the second tensor. Embodiments of this disclosure can improve the efficiency and flexibility of tensor storage, and each time the local memory receives a second tensor, an interrupt can be triggered, realizing asynchronous tensor reading.

Description

Technical Field

[0001] This disclosure relates to the field of computer technology, and in particular to a data processing method and apparatus, electronic device and storage medium. Background Technology

[0002] As the application areas of artificial intelligence (AI) technology continue to expand, the performance requirements for AI chips are also increasing. Storage performance is one of the bottlenecks in AI chip performance, and there is a desire to further improve the dataflow design of AI chips, enabling them to access data more efficiently and flexibly. Summary of the Invention

[0003] This disclosure proposes a data processing technology solution.

[0004] According to one aspect of this disclosure, a data processing method is provided, comprising: determining address information of a second tensor based on tensor information of a first tensor and access information of a second tensor to be accessed; reading the second tensor from a global memory storing the first tensor to a local memory based on the address information of the second tensor; and triggering an interrupt in response to the local memory receiving the second tensor.

[0005] In one possible implementation, when the second tensor to be accessed is a block of data of the first tensor, the access information includes: the position of the second tensor in the first tensor and the size of the second tensor. Determining the address information of the second tensor based on the tensor information of the first tensor and the access information of the second tensor to be accessed includes: determining the address information of the second tensor from the address of the first tensor based on the tensor information of the first tensor, the position of the second tensor in the first tensor, and the size of the second tensor.

[0006] In one possible implementation, the method further includes: performing a boundary check on the second tensor based on tensor information of a first tensor and access information of the second tensor to be accessed, to obtain a check result for the second tensor; if the check result indicates that a first part of the second tensor is located within the first tensor and a second part of the second tensor is located outside the first tensor, determining address information of the first part of the second tensor based on tensor information of the first tensor and access information of the second tensor; the tensor information of the first tensor includes a padding identifier; reading the second tensor from the global memory storing the first tensor to the local memory based on the address information of the second tensor includes: reading the first part of the second tensor from the global memory storing the first tensor to the local memory based on the address information of the first part of the second tensor; padding the second part of the second tensor based on the padding identifier to obtain the second part of the second tensor; and writing the second part of the second tensor to the local memory.

[0007] In one possible implementation, when the second tensor to be accessed is the coverage data corresponding to any one or more elements of the convolution kernel in the first tensor, the access information includes: the coordinates of the unfolded matrix and the coordinates of the convolution kernel elements. The unfolded matrix is ​​a matrix formed by sequentially converting the coverage data corresponding to the receptive field of the convolution kernel into corresponding column vectors or row vectors in the first tensor according to the description information, and arranging them in rows or columns. The description information includes convolution description information and deconvolution description information. Determining the address information of the second tensor based on the tensor information of the first tensor and the access information of the second tensor to be accessed includes: obtaining the mapping relationship between the unfolded matrix and the first tensor; and determining the address information of the second tensor from the address of the first tensor based on the tensor information of the first tensor, the mapping relationship, the coordinates of the unfolded matrix, and the coordinates of the convolution kernel elements.

[0008] In one possible implementation, determining the address information of the second tensor from the address of the first tensor based on the tensor information of the first tensor, the mapping relationship, the coordinates of the unfolded matrix, and the coordinates of the convolution kernel elements includes: determining the second tensor to be accessed from the unfolded matrix based on the coordinates of the unfolded matrix and the coordinates of the convolution kernel elements; determining multiple elements corresponding to the second tensor in the first tensor based on the mapping relationship between the unfolded matrix and the first tensor; and determining the address information of the multiple elements corresponding to the second tensor in the first tensor as the address information of the second tensor.

[0009] In one possible implementation, reading the second tensor from the global memory storing the first tensor to the local memory according to the address information of the second tensor includes: if the size of the second tensor is greater than the read bandwidth, splitting the first memory access request for reading the second tensor from the global memory to the local memory into multiple second memory access requests according to the address information of the second tensor, each second memory access request being used to read split data of different parts of the second tensor from the global memory to the local memory; and serially writing multiple split data constituting the second tensor from the global memory to the local memory according to the multiple second memory access requests.

[0010] In one possible implementation, the tensor information of the first tensor is stored in global memory, and the access information of the second tensor to be accessed is stored in a register.

[0011] In one possible implementation, the first tensor includes feature data from a deep learning task, wherein the feature data includes at least one of image feature data, speech feature data, and text feature data.

[0012] According to one aspect of this disclosure, a data processing apparatus is provided, comprising: a determining module, configured to determine address information of a second tensor based on tensor information of a first tensor and access information of a second tensor to be accessed; a reading module, configured to read the second tensor from a global memory storing the first tensor to a local memory based on the address information of the second tensor; and a triggering module, configured to trigger an interrupt in response to the local memory receiving the second tensor.

[0013] In one possible implementation, the determining module is configured to: determine the address information of the second tensor from the address of the first tensor, based on the tensor information of the first tensor, the position of the second tensor in the first tensor, and the size of the second tensor, when the second tensor to be accessed is a block of data of the first tensor, and the access information includes the position of the second tensor in the first tensor and the size of the second tensor.

[0014] In one possible implementation, the determining module is further configured to: perform a boundary check on the second tensor based on the tensor information of the first tensor and the access information of the second tensor to be accessed, to obtain a check result for the second tensor; if the check result indicates that a first part of the second tensor is located within the first tensor and a second part of the second tensor is located outside the first tensor, determine the address information of the first part of the second tensor based on the tensor information of the first tensor and the access information of the second tensor; the reading module is configured to: if the tensor information of the first tensor includes a padding identifier, read the first part of the second tensor from the global memory storing the first tensor to the local memory based on the address information of the first part of the second tensor; pad the second part of the second tensor based on the padding identifier to obtain the second part of the second tensor; and write the second part of the second tensor into the local memory.

[0015] In one possible implementation, the determining module is configured to: when the second tensor to be accessed is the coverage data corresponding to any one or more elements in the convolution kernel in the first tensor, and the access information includes the coordinates of the unfolded matrix and the coordinates of the convolution kernel elements, obtain the mapping relationship between the unfolded matrix and the first tensor; determine the address information of the second tensor from the address of the first tensor based on the tensor information of the first tensor, the mapping relationship, the coordinates of the unfolded matrix, and the coordinates of the convolution kernel elements; wherein, the unfolded matrix is ​​a matrix formed by sequentially converting the coverage data corresponding to the receptive field of the convolution kernel into corresponding column vectors or row vectors in the first tensor according to the description information, and arranging them in rows or columns, and the description information includes convolution description information and deconvolution description information.

[0016] In one possible implementation, determining the address information of the second tensor from the address of the first tensor based on the tensor information of the first tensor, the mapping relationship, the coordinates of the unfolded matrix, and the coordinates of the convolution kernel elements includes: determining the second tensor to be accessed from the unfolded matrix based on the coordinates of the unfolded matrix and the coordinates of the convolution kernel elements; determining multiple elements corresponding to the second tensor in the first tensor based on the mapping relationship between the unfolded matrix and the first tensor; and determining the address information of the multiple elements corresponding to the second tensor in the first tensor as the address information of the second tensor.

[0017] In one possible implementation, the reading module is configured to: when the size of the second tensor is greater than the reading bandwidth, split a first memory access request for reading the second tensor from the global memory to the local memory into multiple second memory access requests based on the address information of the second tensor, each second memory access request being used to read split data of different parts of the second tensor from the global memory to the local memory; and serially write multiple split data constituting the second tensor from the global memory into the local memory based on the multiple second memory access requests.

[0018] In one possible implementation, the tensor information of the first tensor is stored in global memory, and the access information of the second tensor to be accessed is stored in a register.

[0019] In one possible implementation, the first tensor includes feature data from a deep learning task, wherein the feature data includes at least one of image feature data, speech feature data, and text feature data.

[0020] According to one aspect of this disclosure, an electronic device is provided, comprising: a processor; a memory for storing processor-executable instructions; wherein the processor is configured to invoke the instructions stored in the memory to perform the method described above.

[0021] According to one aspect of this disclosure, a computer-readable storage medium is provided that stores computer program instructions thereon, which, when executed by a processor, implement the above-described method.

[0022] According to one aspect of this disclosure, a computer program product is provided, including computer-readable code or a computer-readable storage medium carrying the computer-readable code, wherein when the computer-readable code is run in a processor of an electronic device, the processor in the electronic device performs an action to implement the above-described method.

[0023] In this embodiment, the address information of the second tensor is determined based on the tensor information of the first tensor and the access information of the second tensor to be accessed. Then, based on the address information of the second tensor, the second tensor is read from the global memory storing the first tensor into the local memory. An interrupt is triggered in response to the local memory receiving the second tensor. In this way, the second tensor can be quickly read from the global memory storing the first tensor into the local memory based on the tensor information of the first tensor and the access information of the second tensor to be accessed, improving the efficiency and flexibility of tensor storage. Furthermore, each time the local memory receives the second tensor, an interrupt can be triggered, achieving asynchronous tensor reading.

[0024] It should be understood that the foregoing general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this disclosure. Other features and aspects of this disclosure will become clear from the following detailed description of exemplary embodiments with reference to the accompanying drawings. Attached Figure Description

[0025] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this disclosure and, together with the specification, serve to illustrate the technical solutions of this disclosure.

[0026] Figure 1 A flowchart illustrating a data processing method according to an embodiment of the present disclosure is shown.

[0027] Figure 2 A schematic diagram of a hardware architecture according to an embodiment of this disclosure is shown.

[0028] Figure 3 This diagram illustrates a method for determining address information of a second tensor according to an embodiment of the present disclosure.

[0029] Figure 4 This diagram illustrates another method for determining the address information of a second tensor according to an embodiment of the present disclosure.

[0030] Figure 5 A schematic diagram illustrating the mapping relationship between a first tensor and an expanded matrix according to an embodiment of the present disclosure is shown.

[0031] Figure 6 A block diagram of a data processing apparatus according to an embodiment of the present disclosure is shown.

[0032] Figure 7 A block diagram of an electronic device according to an embodiment of the present disclosure is shown. Detailed Implementation

[0033] Various exemplary embodiments, features, and aspects of this disclosure will now be described in detail with reference to the accompanying drawings. The same reference numerals in the drawings denote elements that have the same or similar functions. Although various aspects of the embodiments are shown in the drawings, they are not necessarily drawn to scale unless specifically indicated otherwise.

[0034] The term “exemplary” as used herein means “serving as an example, embodiment, or illustration.” Any embodiment illustrated herein as “exemplary” is not necessarily to be construed as superior to or better than other embodiments.

[0035] In this document, the term "and / or" is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent three cases: A alone, A and B simultaneously, and B alone. Furthermore, the term "at least one" in this document means any combination of at least two of any one or more elements. For example, including at least one of A, B, and C can mean including any one or more elements selected from the set consisting of A, B, and C.

[0036] Furthermore, to better illustrate this disclosure, numerous specific details are set forth in the following detailed description. Those skilled in the art will understand that this disclosure can be practiced without certain specific details. In some instances, methods, means, components, and circuits well known to those skilled in the art have not been described in detail in order to highlight the main points of this disclosure.

[0037] With the widespread application of deep learning and neural networks, data is often stored in the form of tensors, which can extend vectors and matrices to arbitrary dimensions. In related technologies, processors (such as general-purpose computing on graphics processing units (GPGPUs)) access the storage space containing tensors. This requires decomposing the storage space, with each thread of the processor accessing a portion of it. Furthermore, during the process of each thread accessing its assigned portion of the storage space, a large number of address calculation instructions and boundary check instructions are required. This method of accessing tensor storage space is inefficient. Additionally, because there is a space limit for accessing storage space with a single instruction, accessing tensors that occupy a large amount of storage space requires the coordinated use of multiple instructions, which is inflexible and also affects the improvement of memory access performance.

[0038] In view of this, in order to improve the efficiency and flexibility of tensor storage, embodiments of this disclosure provide a data processing method. Figure 1 A flowchart illustrating a data processing method according to an embodiment of this disclosure is shown, such as... Figure 1 As shown, the data processing method includes:

[0039] In step S11, the address information of the second tensor is determined based on the tensor information of the first tensor and the access information of the second tensor to be accessed;

[0040] In step S12, the second tensor is read from the global memory storing the first tensor to the local memory according to the address information of the second tensor;

[0041] In step S13, an interrupt is triggered in response to the local memory receiving the second tensor.

[0042] The data processing method of this disclosure can quickly read the second tensor from the global memory storing the first tensor to the local memory based on the tensor information of the first tensor and the access information of the second tensor to be accessed, thereby improving the efficiency and flexibility of tensor storage. Furthermore, each time the local memory receives the second tensor, an interrupt can be triggered to realize the asynchronous tensor reading process.

[0043] In one possible implementation, the data processing method of this disclosure embodiment can be executed by a tensor storage engine, which is a hardware module in a processor chip used to implement data transfer. The processor chip can be a newly designed chip or an improved version of an existing processor chip. The types of processor chips may include, but are not limited to, central processing units (CPUs), graphics processing units (GPUs), general-purpose graphics processing units (GPGPUs), digital signal processors (DSPs), application-specific integrated circuits (ASICs), tensor processing units (TPUs), field-programmable gate arrays (FPGAs), or other programmable logic devices. It may also include microprocessors or other conventional processors.

[0044] In one possible implementation, the processor chip may include one or more tensor memory engines, and this disclosure does not limit the number of tensor memory engines in the processor chip.

[0045] In the example, for a single-core, single-threaded processor chip, a tensor storage engine can be set up on the processor chip to perform data transfer processing on the first tensor stored in the global memory outside the processor chip, and store the second tensor determined based on the first tensor into the local memory inside the processor chip corresponding to the tensor storage engine.

[0046] In the example, for a multi-core, multi-threaded processor chip, multiple tensor storage engines can be configured on the processor chip. For instance, assuming the processor chip includes N (N≥2) computing cores, and each computing core can execute M (M≥2) threads in parallel, one tensor storage engine can be configured for each computing core. The M threads in each computing core can share one tensor storage engine and one local memory. This tensor storage engine is used to perform data transfer processing on the first tensor stored in the global memory shared by multiple computing cores, and store the second tensor determined based on the first tensor in the local memory within the computing core. Alternatively, M tensor storage engines can be configured for each computing core, and each thread in each computing core can correspond to one tensor storage engine and one local memory. This tensor storage engine is used to perform data transfer processing on the first tensor stored in the global memory shared by multiple computing cores, and store the second tensor determined based on the first tensor in the local memory corresponding to the tensor storage engine within the computing core.

[0047] In the example, for a multi-core, single-threaded processor chip, multiple tensor storage engines can be configured on the processor chip. For instance, assuming the processor chip includes N (N≥2) single-threaded computing cores, a tensor storage engine can be configured for each computing core. This tensor storage engine is used to perform data transfer processing on the first tensor stored in the global memory shared by multiple computing cores, and to store the second tensor determined based on the first tensor in the local memory within the computing core.

[0048] In the example, for a single-core, multi-threaded processor chip, multiple tensor storage engines can be configured on the processor chip. For instance, suppose the processor chip can execute M (M≥2) threads in parallel. Each thread can correspond to a tensor storage engine and a local memory. Each tensor storage engine can be used to perform data transfer processing on a first tensor stored in the global memory shared by multiple threads, and store a second tensor determined based on the first tensor into the local memory corresponding to that tensor storage engine.

[0049] In one possible implementation, Figure 2 A schematic diagram of a hardware architecture according to an embodiment of this disclosure is shown, such as... Figure 2 As shown, the tensor storage engine is connected to the global memory and the local memory respectively, and can be used to perform read and / or write operations on the global memory and the local memory. In addition, the tensor storage engine can also be used to receive tensor information of the first tensor and access information of the second tensor, so as to move the second tensor from the global memory that stores the first tensor to the local memory, and trigger an interrupt in response to the local memory receiving the second tensor.

[0050] The following is an exemplary description of the data processing method implemented by the tensor storage engine according to the embodiments of this disclosure.

[0051] In step S11, the address information of the second tensor is determined based on the tensor information of the first tensor and the access information of the second tensor to be accessed.

[0052] In one possible implementation, the first tensor is a multidimensional tensor, which can be viewed as a multidimensional matrix. For example, the first tensor can be a two-dimensional tensor, a three-dimensional tensor, a four-dimensional tensor, or other tensor data with more dimensions. The embodiments of this application do not limit the dimension of the first tensor.

[0053] In one possible implementation, the first tensor includes feature data from a deep learning task, wherein the feature data includes at least one of image feature data, speech feature data, and text feature data.

[0054] For example, in a scenario where a deep neural network is used to perform face recognition on a target object, the first tensor can be image feature data, and the image feature data of the target object (e.g., a face feature map) can be stored in global memory; in a scenario where a deep neural network is used to perform speech recognition on a target object, the first tensor can be speech feature data, and the speech feature data of the target object can be stored in global memory; in a scenario where a deep neural network is used to perform text recognition on a target document, the first tensor can be text feature data, and the text feature data of the target document can be stored in global memory; the embodiments of this application do not limit the type of the first tensor.

[0055] In one possible implementation, the tensor information of the first tensor may include the data structure information and the address information of the first tensor; wherein, the data structure information of the first tensor includes, for example, the dimensions of the first tensor, the size of the first tensor, the data type of the elements in the first tensor (e.g., integer type, single-precision floating-point type, double-precision floating-point type, character type, etc.) used to describe the first tensor; the address information of the first tensor includes, for example, the base address of the first tensor in global memory, the address space, and other address-related information.

[0056] Since the first tensor contains a lot of tensor information, it can be stored in the global memory.

[0057] In one possible implementation, when the second tensor to be accessed is a block of data of the first tensor, the access information includes: the position of the second tensor in the first tensor and the size of the second tensor. Step S11 may include: determining the address information of the second tensor from the address of the first tensor based on the tensor information of the first tensor, the position of the second tensor in the first tensor, and the size of the second tensor.

[0058] For example, Figure 3 This diagram illustrates a method for determining address information of a second tensor according to an embodiment of the present disclosure. Figure 3 As shown, the second tensor block to be accessed is a block of data from the first tensor tensor, and the second tensor block can be data from any local region within the first tensor tensor.

[0059] The first tensor is a two-dimensional tensor. The tensor information of the first tensor includes: the base address tensor_base_address used to determine the storage location of the first tensor in global memory, the size of the first tensor in the horizontal dimension tensor_dim[0], and the size of the first tensor in the vertical dimension tensor_dim[1]. Since the first tensor is address-contiguous in the lowest dimension (e.g., the horizontal or vertical dimension), the tensor storage engine can access each element in the first tensor based on the base address tensor_base_address and the position of each element in the first tensor.

[0060] like Figure 3 As shown, the access information of the second tensor block includes: the position of the second tensor block in the first tensor block block_pos, the size of the second tensor block in the horizontal dimension block_dim[0], and the size of the second tensor block in the vertical dimension block_dim[1].

[0061] The tensor storage engine can determine the base address of the second tensor block (block_base_address) based on the base address (tensor_base_address) in the tensor information of the first tensor and the position (block_pos) of the second tensor block in the first tensor.

[0062] The tensor storage engine can determine the size of the address space of the second tensor block as block_dim[0] × block_dim[1] based on the horizontal dimension size block_dim[0] and the vertical dimension size block_dim[1].

[0063] The tensor storage engine can determine the address information of the second tensor block to be accessed in the global memory based on the base address of the second tensor block (block_base_address) and the size of the address space of the second tensor block (block_dim[0]×block_dim[1]).

[0064] In this way, the tensor storage engine can quickly and accurately determine the address information of the second tensor (e.g., any block of data in the first tensor) from the address of the first tensor based on the tensor information of the first tensor and the access information of the second tensor to be accessed.

[0065] When the second tensor to be accessed is block data located at the edge of the first tensor, there may be a portion of the data in the second tensor that is not within the first tensor. In one possible implementation, the method further includes: firstly, performing a boundary check on the second tensor based on the tensor information of the first tensor and the access information of the second tensor to be accessed, obtaining a check result for the second tensor, the check result being used to determine whether the second tensor is located within the first tensor; if the check result indicates that a first part of the second tensor is located within the first tensor and a second part of the second tensor is located outside the first tensor, then determining the address information of the first part of the second tensor based on the tensor information of the first tensor and the access information of the second tensor to be accessed.

[0066] The tensor information of the first tensor also includes padding identifiers (such as the number 0, the number 1, etc.). The tensor storage engine can pad the second part of the second tensor according to the padding identifiers to obtain the second part of the second tensor. In subsequent step S12, in addition to reading the first part of the second tensor from the global memory that stores the first tensor to the local memory according to the address information of the first part of the second tensor, the engine will also synchronously write the second part of the second tensor to the local memory so that the local memory stores the complete second tensor.

[0067] Figure 4 This diagram illustrates a method for determining address information of a first portion of a second tensor according to an embodiment of the present disclosure. Figure 4As shown, the tensor information of the first tensor includes: the base address tensor_base_address used to determine the storage location of the first tensor in global memory, the horizontal dimension tensor_dim[0] of the first tensor, and the vertical dimension tensor_dim[1] of the first tensor. The access information of the second tensor block includes: the position block_pos of the second tensor block in the first tensor, the horizontal dimension block_dim[0] of the second tensor block, and the vertical dimension block_dim[1]. Among them, the position block_pos of the second tensor block in the first tensor includes: the horizontal dimension position block_pos[0] of the second tensor block in the first tensor, and the vertical dimension position block_pos[1] of the second tensor block in the first tensor.

[0068] Based on the tensor information of the first tensor and the access information of the second tensor block to be accessed, the check result of the second tensor block can be obtained. This check result is used to determine whether the second tensor block is located within the first tensor.

[0069] If the sum of the horizontal position block_pos[0] of the second tensor block in the first tensor and the horizontal dimension block_dim[0] of the second tensor block is less than or equal to the horizontal dimension tensor_dim[0] of the first tensor, i.e., (block_pos[0]+block_dim[0])≤tensor_dim[0], and the sum of the vertical position block_pos[1] of the second tensor block in the first tensor and the vertical dimension block_dim[1] of the second tensor block is less than or equal to the vertical dimension tensor_dim[1] of the first tensor, i.e., (block_pos[1]+block_dim[1])≤tensor_dim[1], it can be determined that the second tensor block is located within the first tensor. (Refer to the above text.) Figure 3 The address information of the second tensor block is determined in the manner shown, which will not be elaborated here.

[0070] If the sum of the horizontal position (block_pos[0]) of the second tensor block in the first tensor and its horizontal dimension (block_dim[0]) is greater than the horizontal dimension (tensor_dim[0]) of the first tensor, i.e., (block_pos[0] + block_dim[0]) > tensor_dim[0], and / or the sum of the vertical position (block_pos[1]) of the second tensor block in the first tensor and its vertical dimension (block_dim[1]) is greater than the vertical dimension (tensor_dim[1]) of the first tensor, i.e., (block_pos[1] + block_dim[1]) > tensor_dim[1], it can be determined that some data in the region containing the second tensor block is not within the first tensor. Figure 4 As shown, the first part of the second tensor block to be accessed is located inside the first tensor tensor, and the second part of the second tensor block is located outside the first tensor tensor.

[0071] When the first part of the second tensor block is located within the first tensor, and the second part of the second tensor block is located outside the first tensor, the tensor storage engine can determine the address information of the first part of the second tensor block based on the tensor information of the first tensor and the access information of the second tensor block to be accessed.

[0072] In the example, the tensor storage engine can determine the base address of the second tensor block, which is the base address of the first part of the second tensor block, based on the base address tensor_base_address in the tensor information of the first tensor and the position of the second tensor block in the first tensor block, block_pos.

[0073] Optionally, in the case that (block_pos[0]+block_dim[0])>tensor_dim[0] and (block_pos[1]+block_dim[1])>tensor_dim[1], the tensor storage engine can determine the size of the address space of the first part of the second tensor block based on the size of the first tensor in the horizontal dimension tensor_dim[0], the size of the first tensor in the vertical dimension tensor_dim[1], the position of the second tensor block in the horizontal dimension block_pos[0] in the first tensor, and the position of the second tensor block in the vertical dimension block_pos[1] in the first tensor, i.e.: (tensor_dim[0]-block_pos[0])×(tensor_dim[1]-block_pos[1]).

[0074] The tensor storage engine can determine the address information of the first part of the second tensor block to be accessed in global memory based on the base address of the second tensor block, block_base_address, and the size of the address space of the first part of the second tensor block (tensor_dim[0]-block_pos[0])×(tensor_dim[1]-block_pos[1]).

[0075] Optionally, in the case where (block_pos[0]+block_dim[0])>tensor_dim[0] and (block_pos[1]+block_dim[1])≤tensor_dim[1] (see [link to relevant documentation]) Figure 4 The tensor storage engine can determine the size of the address space of the first part of the second tensor block based on the size of the first tensor in the horizontal dimension tensor_dim[0], the position of the second tensor block in the horizontal dimension block_pos[0] in the first tensor, and the size of the second tensor block in the vertical dimension block_dim[1], that is: (tensor_dim[0]-block_pos[0])×block_dim[1].

[0076] The tensor storage engine can determine the address information of the first part of the second tensor block to be accessed in the global memory based on the base address of the second tensor block, block_base_address, and the size of the address space of the first part of the second tensor block (tensor_dim[0]-block_pos[0])×block_dim[1].

[0077] Optionally, if (block_pos[0]+block_dim[0])≤tensor_dim[0] and (block_pos[1]+block_dim[1])>tensor_dim[1], the tensor storage engine can determine the size of the address space of the first part of the second tensor block based on the vertical dimension size of the first tensor tensor tensor tensor tensor_dim[1], the vertical dimension position of the second tensor block in the first tensor tensor block block_pos[1], and the horizontal dimension size of the second tensor block block_dim[0], i.e.: block_dim[0]×(tensor_dim[1]-block_pos[1]).

[0078] The tensor storage engine can determine the address information of the first part of the second tensor block to be accessed in the global memory based on the base address of the second tensor block (block_base_address) and the size of the address space of the first part of the second tensor block (block_dim[0]×(tensor_dim[1]-block_pos[1])).

[0079] The tensor information of the first tensor also includes padding identifiers (such as the number 0, the number 1, etc.). The tensor storage engine can pad the second part of the second tensor block according to the padding identifiers to obtain the second part of the second tensor block. In subsequent step S12, in addition to reading the first part of the second tensor block from the global memory storing the first tensor to the local memory according to the address information of the first part of the second tensor block, the engine will also synchronously write the padded second part of the second tensor block to the local memory so that the local memory stores the complete second tensor block.

[0080] In this way, the address information of the second tensor can be determined from the address of the first tensor based on the tensor information of the first tensor, the position of the second tensor within the first tensor, and the size of the second tensor. The second tensor can be block data obtained by arbitrarily slicing the first tensor, which is beneficial for the subsequent tensor storage engine to access any block data in the first tensor that occupies a large space from the global memory with a single instruction, thus improving the applicability of the data processing method of this embodiment. Furthermore, the tensor storage engine can also realize automatic address calculation and boundary checking for block data, and can implement arbitrary constant value padding for out-of-bounds access.

[0081] In one possible implementation, the convolution kernel can slide across the entire first tensor. To facilitate convolution and / or deconvolution operations on the first tensor and further improve the applicability of the data processing method of this embodiment, when the second tensor to be accessed is the coverage data corresponding to any one or more elements of the convolution kernel in the first tensor, the access information includes: the coordinates of the unfolded matrix and the coordinates of the convolution kernel elements. The unfolded matrix is ​​a matrix formed by sequentially converting the coverage data corresponding to the receptive field of the convolution kernel into corresponding column vectors or row vectors in the first tensor according to the description information, and arranging them in rows or columns. The description information includes convolution description information and deconvolution description information. Step S11 may include: obtaining the mapping relationship between the unfolded matrix and the first tensor; determining the address information of the second tensor from the address of the first tensor according to the tensor information of the first tensor, the mapping relationship, the coordinates of the unfolded matrix, and the coordinates of the convolution kernel elements.

[0082] The tensor information of the first tensor may include its data structure information (e.g., the dimensions, size, and data types of its elements) and its address information (e.g., its base address and address space in global memory). The mapping relationship is used to determine the address information of each element in the expanded matrix in global memory; the expanded matrix coordinates and the convolution kernel element coordinates are used to locate the second tensor within the expanded matrix.

[0083] The descriptive information provides a method for expanding the original matrix into an expanded matrix suitable for the convolution kernel. This descriptive information includes the kernel size, stride, padding, and dilation rate. If the descriptive information is deconvolution descriptive information, and the stride in the deconvolution descriptive information is greater than 1, during the process of converting the coverage data corresponding to the receptive field of the convolution kernel into corresponding column or row vectors and arranging them into an expanded matrix by rows or columns, constant value padding can be performed between each element in the coverage data based on the padding identifiers (e.g., numbers 0 and 1) included in the tensor information of the first tensor.

[0084] Figure 5 A schematic diagram illustrating the mapping relationship between a first tensor and an expanded matrix according to an embodiment of this disclosure is shown. Figure 5 As shown, based on the tensor information of the first tensor obtained, the tensor storage engine knows that the first tensor is a four-dimensional tensor N×Cx×H×W (see...). Figure 5 The two cuboids on the top have dimensions of 2×2×5×4, where N represents the dimension of the batch, Cx represents the dimension of the channel, H represents the dimension of the height, and W represents the dimension of the width.

[0085] like Figure 5 As shown, the numerical labels 0 to 79 in the first tensor are used to distinguish the elements of the first tensor. Each element occupies 128 bytes (80HB in hexadecimal) of storage space in global memory. Thus, assuming the base address of the first tensor in global memory is address, the address information of element 0 in global memory is address to (address+007FH), the address information of element 1 in global memory is (address+0080H) to (address+00FFH), the address information of element 2 in global memory is (address+0100H) to (address+017FH), and so on. The address information of element 79 in global memory is (address+2780H) to (address+27FFH).

[0086] Figure 5 The two-dimensional matrix shown in the lower middle is the expansion matrix of the first tensor. This expansion matrix is ​​formed by converting the coverage data corresponding to the receptive field of the convolution kernel into corresponding row vectors in the first tensor according to the convolution description information (e.g., the convolution kernel size is 3×3, the stride of the convolution kernel is 1, and the padding is 0), and arranging them into a two-dimensional matrix by rows or columns.

[0087] like Figure 5 As shown, assume P represents the position where the convolution kernel slides in the height direction, Q represents the position where the convolution kernel slides in the width direction, Cx represents the position where it slides in the channel direction, and N represents whether the convolution kernel slides in the N1×Cx×H×W part of the first tensor or in the N2×Cx×H×W part of the first tensor.

[0088] like Figure 5 In the gray area, the first convolutional kernel slides to position Q0 in the width direction, to position P0 in the height direction, and to position Cx = 0 in the width direction. The receptive field of the convolutional kernel covers the N1×Cx×H×W part of the first tensor. This can be converted to [0 1 2 4 5 6 8 9 10], and the coverage data corresponding to the N2×Cx×H×W part of the first tensor is... This can be converted to [40 41 42 444546 48 49 50].

[0089] The second convolutional kernel slides to position Q1 in the width direction, remains at position P0 in the height direction, and slides to position Cx = 0 in the width direction. The receptive field of the convolutional kernel covers the N1×Cx×H×W part of the first tensor. This can be converted to [1 2 3 5 6 7 9 10 11], and the coverage data corresponding to the N2×Cx×H×W part of the first tensor of the convolution kernel is... This can be converted to [41 42 43 45 46 47 495051].

[0090] The third convolutional kernel slides to position Q0 in the width direction, remains at position P1 in the height direction, and slides to position Cx = 0 in the width direction. The receptive field of the convolutional kernel covers the N1×Cx×H×W part of the first tensor. This can be converted to [4 5 6 8 9 10 12 13 14], where the receptive field of the convolution kernel corresponds to the coverage data of the N2×Cx×H×W part of the first tensor. This can be converted to [44 45 46 48 49 505253 54].

[0091] This process continues until the 12th convolution kernel slides to position Q1 in the width direction, to position P2 in the height direction, and to position Cx = 1 in the width direction. The receptive field of the convolution kernel covers the N1×Cx×H×W portion of the first tensor. This can be converted to [29 30 31 33 34 35 37 38 39], where the receptive field of the convolution kernel corresponds to the coverage data of the N2×Cx×H×W part of the first tensor. It can be converted to [697071 73 74 75 77 78 79].

[0092] The row vectors obtained after transforming the 12 coverage data are arranged by row or column to obtain the following: Figure 5 The expansion matrix is ​​shown. The mapping relationship between the expansion matrix and the first tensor can be obtained based on the correspondence between the elements with the same numerical indices in the first tensor and the expansion matrix. In the example, the tensor storage engine can directly obtain the mapping relationship between the user-input expansion matrix and the first tensor; alternatively, the tensor storage engine can first perform the expansion processing described above on the first tensor read from global memory based on the tensor information and description information of the obtained first tensor to determine the expansion matrix, and then obtain the mapping relationship between the expansion matrix and the first tensor based on the correspondence between the first tensor and the expansion matrix.

[0093] It should be understood that Figure 5 The example shown illustrates the case where the description information is convolutional. If the description information is deconvolutional, please refer to the above description. In the case where the stride of the convolution kernel in the deconvolutional description information is greater than 1, constant value padding is performed between each element in the unfolded matrix. This will not be elaborated further here.

[0094] In one possible implementation, determining the address information of the second tensor from the address of the first tensor based on the tensor information of the first tensor, the mapping relationship, the coordinates of the unfolded matrix, and the coordinates of the convolution kernel elements includes: determining the second tensor to be accessed from the unfolded matrix based on the coordinates of the unfolded matrix and the coordinates of the convolution kernel elements; determining multiple elements corresponding to the second tensor in the first tensor based on the mapping relationship between the unfolded matrix and the first tensor; and determining the address information of the multiple elements corresponding to the second tensor in the first tensor as the address information of the second tensor.

[0095] In such Figure 5 In the expansion matrix shown, NPQ can be used as the coordinates of the expansion matrix in the vertical dimension to locate a certain row in the expansion matrix. For example, N1P0Q0 represents the first row, N1P0Q1 represents the second row, N1P1Q0 represents the third row, N1P1Q1 represents the fourth row, N1P2Q0 represents the fifth row, N1P2Q1 represents the sixth row, N2P0Q0 represents the seventh row, N2P0Q1 represents the eighth row, N2P1Q0 represents the ninth row, N2P1Q1 represents the tenth row, N2P2Q0 represents the eleventh row, and N2P2Q1 represents the twelfth row.

[0096] Cx can be used as the horizontal coordinate of the expanded matrix. Cx = 0 is used to locate the 8 columns on the left side of the expanded matrix, and Cx = 1 is used to locate the 8 columns on the right side of the expanded matrix.

[0097] RS represents the coordinates of the convolution kernel elements. R represents the vertical coordinates of the convolution kernel, used to locate a row of the convolution kernel. S represents the horizontal coordinates of the convolution kernel, used to locate a column of the convolution kernel. For example, R=0, S=0 represents the convolution kernel element in the first column of the first row; R=0, S=1 represents the convolution kernel element in the second column of the first row; R=0, S=2 represents the convolution kernel element in the third column of the first row; R=1, S=0 represents the convolution kernel element in the first column of the second row; R=1, S=1 represents the convolution kernel element in the second column of the first row; R=1, S=2 represents the convolution kernel element in the third column of the second row; R=2, S=0 represents the convolution kernel element in the first column of the third row; R=2, S=1 represents the convolution kernel element in the second column of the third row; and R=2, S=2 represents the convolution kernel element in the third column of the third row.

[0098] Based on the horizontal coordinates Cx of the unfolded matrix and the coordinates RS of the convolution kernel elements, any one or more columns of the second tensor can be located in the unfolded matrix, which is the covering data corresponding to any one or more elements of the convolution kernel in the first tensor.

[0099] For example, Cx = 0, R = 0, S = 0 represents the first column of the expanded matrix, which is the data covered by the first row and first column of the convolution kernel after sliding multiple times (e.g., 12 times) in the first tensor; Cx = 0, R = 0, S = 1 represents the second column of the expanded matrix, which is the data covered by the first row and second column of the convolution kernel after sliding multiple times (e.g., 12 times) in the first tensor; and so on, Cx = 1, R = 2, S = 2 represents the eighteenth column (last column) of the expanded matrix, which is the data covered by the third row and third column of the convolution kernel after sliding multiple times (e.g., 12 times) in the first tensor.

[0100] When a second tensor is located in any one or more columns of the expanded matrix, the corresponding elements of the second tensor in the first tensor can be determined according to the mapping relationship between the expanded matrix and the first tensor. The address information of the corresponding elements of the second tensor in the first tensor can then be used as the address information of the second tensor.

[0101] For example, suppose the second tensor is the first column of the expanded matrix [0 1 4 5 8 9 40 41 44 45 48 49]. The corresponding data of the first row and first column elements in the convolution kernel are covered by the first tensor. According to the mapping relationship between the expanded matrix and the first tensor, the elements with the same number as the elements in the first column of the expanded matrix can be obtained from the first tensor. The address of the element [0 1 4 5 8 9 40 41 44 45 48 49] in the first tensor is determined as the address information of the second tensor.

[0102] For example, suppose the second tensor is the second column of the expanded matrix [1 2 5 6 9 10 41 42 45 46 49 50]. The corresponding data of the first row and second column of the convolution kernel is covered by the first tensor. According to the mapping relationship between the expanded matrix and the first tensor, the elements with the same number as the elements in the second column of the expanded matrix can be obtained from the first tensor. The address of the element [1 2 5 6 9 10 41 42 45 46 49 50] in the first tensor is determined as the address information of the second tensor.

[0103] In practical applications, the above process can be encapsulated into a TensorMemoryEngine instruction, which can be used to access one or more columns in the unfolded matrix (corresponding to elements of the first tensor covered by one or more convolutional kernel elements). This allows the tensor storage engine to access the covered data corresponding to any one or more convolutional kernel elements in the first tensor, which occupies a large space, from global memory with a single instruction. The pseudocode is as follows:

[0104]

[0105] In this way, the tensor storage engine can avoid performing the process of expanding the first tensor into an expanded matrix, and it does not need to occupy a large amount of storage space to record the expanded matrix of the first tensor. In the case of not obtaining the expanded matrix, the tensor storage engine can calculate the coordinates of any element in the expanded matrix in the first tensor (e.g., the coordinates of the elements of the first tensor covered by one or more convolution kernel elements) based on the expanded matrix coordinates NPQCx and the convolution kernel element coordinates RS. This is beneficial for using less hardware resources to simplify the software implementation logic (e.g., software implementation of convolution and deconvolution operations).

[0106] In this way, the tensor storage engine can quickly and accurately determine the address information of the second tensor (e.g., the overlay data corresponding to any one or more elements in the convolution kernel in the first tensor) from the address of the first tensor based on the tensor information of the first tensor and the access information of the second tensor to be accessed (e.g., the coordinates of the unfolded matrix and the coordinates of the convolution kernel elements).

[0107] In the data processing method of this disclosure embodiment, the second tensor can be block data of the first tensor, or it can be the covering data corresponding to any one or more elements in the convolution kernel in the first tensor, or it can be data composed of covering data and constant value filling between each element in the covering data. It can support block loading of the first tensor, block loading of expanded data suitable for convolution operation of the first tensor, and block loading of expanded data suitable for deconvolution operation of the first tensor, unifying multiple loading modes and making it more user-friendly for programming.

[0108] In one possible implementation, the tensor information of the first tensor is stored in global memory, and the access information of the second tensor to be accessed is stored in a register.

[0109] Since the first tensor contains a large amount of information and its information is unlikely to change easily, it can be stored in global memory. However, since the second tensor to be accessed is either a block of data within any region of the first tensor, or it is the overlay data corresponding to any one or more elements of the convolution kernel in the first tensor, its access information is variable. Therefore, this access information can be stored in a register for easy modification during program execution.

[0110] This method of storing the unchanging tensor information of the first tensor in global memory and storing the variable access information of the second tensor in registers, according to whether the information is prone to change, puts various types of information in different places, which is beneficial to improving the flexibility and efficiency of data processing.

[0111] Compared to synchronous memory access methods in related technologies, which require loading data into a register file (e.g., for a General-Purpose Computing on Graphics Processing Unit (GPGPU) Single Instruction Multithreaded (SIMT) architecture, each thread's registers are isolated and cannot be shared, necessitating data transfer from registers to shared storage), the tensor storage engine in this disclosure is connected to a register storing second tensor access information. This allows the tensor storage engine to access the register at any time, enabling multiple threads corresponding to the tensor storage engine to share it. Furthermore, the processing method in this disclosure can also employ register passing, improving the flexibility of data processing.

[0112] In step S11, the address information of the second tensor is determined. In step S12, based on the address information of the second tensor, the second tensor is read from the global memory storing the first tensor and transferred to the local memory. The local memory is located closer to the tensor storage engine than the global memory, allowing the tensor storage engine to perform read and write operations on the local memory more efficiently. The tensor storage engine can perform data transfer processing on the first tensor stored in the global memory and store the second tensor, determined based on the first tensor, in the local memory corresponding to the tensor storage engine.

[0113] In one possible implementation, step S12 may include: if the size of the second tensor is greater than the read bandwidth, splitting a first memory access request for reading the second tensor from the global memory to the local memory into multiple second memory access requests based on the address information of the second tensor, each second memory access request being used to read split data of different parts of the second tensor from the global memory to the local memory; and serially writing multiple split data constituting the second tensor from the global memory into the local memory according to the multiple second memory access requests. Here, read bandwidth refers to the ability to read data from memory per unit time, and can be expressed as bits or bytes transferred per transmission.

[0114] For example, assuming the global memory read bandwidth is 128 bytes, and the first memory access request needs to access a 1024-bit second tensor, the corresponding address is 0, but the read bandwidth supports a maximum of 128-bit memory access requests, the first memory access request can be split into eight second memory access requests, namely: second memory access requests 0 to second memory access requests 7. Second memory access request 0 is used to read 128 bits of split data 0 from address 0 in global memory to local memory; second memory access request 1 is used to read 128 bits of split data 1 from address 128 in global memory to local memory; second memory access request 2 is used to read 128 bits of split data 1 from address 256 in global memory... The process involves several memory access requests: 1) Reading 128-bit fragment 2 of the second tensor into local memory; 2) Reading 128-bit fragment 3 of the second tensor from address 384 in global memory into local memory; 3) Reading 128-bit fragment 4 of the second tensor from address 512 in global memory into local memory; 4) Reading 128-bit fragment 5 of the second tensor from address 640 in global memory into local memory; 5) Reading 128-bit fragment 6 of the second tensor from address 768 in global memory into local memory; and 6) Reading 128-bit fragment 7 of the second tensor from address 896 in global memory into local memory. Then, based on memory access requests 0 through 7, the fragments 0 through 7 constituting the second tensor in global memory can be serially written into local memory. It should be understood that this disclosure does not impose specific restrictions on the size of the second tensor or the size of the read bandwidth, which can be set according to the actual application scenario.

[0115] In this way, if the size of the second tensor in the global memory exceeds the maximum granularity supported by the storage system, it can be split and processed serially, improving the applicability of the data processing method in this disclosure. Furthermore, in the embodiments of this disclosure, the read bandwidth of the global memory is less than the read bandwidth of the local memory. Data (e.g., the second tensor) is read from the global memory to the local memory, and the local memory can be reused multiple times, reducing the bandwidth pressure on the global memory.

[0116] In step S12, the second tensor is read from the global memory storing the first tensor to the local memory. In step S13, an interrupt is triggered in response to the local memory receiving the second tensor. The process of moving the second tensor from global memory to local memory in step S12 may take a relatively long time. Before waiting for the local memory to receive the second tensor and trigger an interrupt, the tensor storage engine can asynchronously execute other instructions to further improve processing efficiency.

[0117] In this way, by setting up an interrupt wake-up mechanism, it is not necessary to follow a pipeline approach (for example, the next instruction can only be executed after the data has been moved to the destination). Instead, other instructions can be executed after the current instruction is issued, and the current instruction can be waited for to complete and send a signal to the processor core to trigger an interrupt, thus waking up the current instruction. This is more efficient.

[0118] The data processing method of this disclosure can quickly read the second tensor from the global memory storing the first tensor to the local memory based on the tensor information of the first tensor and the access information of the second tensor to be accessed, thereby improving the efficiency and flexibility of tensor storage. Furthermore, each time the local memory receives the second tensor, an interrupt can be triggered to realize the asynchronous tensor reading process.

[0119] In this embodiment, the second tensor can be block data of the first tensor, or it can be the covering data corresponding to any one or more elements in the convolution kernel in the first tensor. It can also be data composed of covering data and constant value filling between each element in the covering data. It can support block loading of the first tensor, block loading of expanded data suitable for convolution operation of the first tensor, and block loading of expanded data suitable for deconvolution operation of the first tensor. It unifies multiple loading modes and is more user-friendly for programming.

[0120] The processing method of this embodiment can store the tensor information of the unchanging first tensor in the global memory and store the access information of the changing second tensor in the register. This method of placing various types of information in different places according to whether the information is prone to change is beneficial to improving the flexibility and efficiency of data processing.

[0121] It is understood that the various method embodiments mentioned above in this disclosure can be combined with each other to form combined embodiments without violating the principle and logic. Due to space limitations, this disclosure will not elaborate further. Those skilled in the art will understand that in the above methods of specific implementation, the specific execution order of each step should be determined by its function and possible internal logic.

[0122] In addition, this disclosure also provides a data processing apparatus, an electronic device, a computer-readable storage medium, and a program, all of which can be used to implement any of the data processing methods provided in this disclosure. The corresponding technical solutions and descriptions are described in the corresponding records in the method section and will not be repeated here.

[0123] Figure 6 A block diagram of a data processing apparatus according to an embodiment of the present disclosure is shown, such as Figure 6 As shown, the device includes:

[0124] The determination module 61 is used to determine the address information of the second tensor based on the tensor information of the first tensor and the access information of the second tensor to be accessed;

[0125] The reading module 62 is used to read the second tensor from the global memory storing the first tensor to the local memory according to the address information of the second tensor;

[0126] Trigger module 63 is used to trigger an interrupt in response to the local memory receiving the second tensor.

[0127] In one possible implementation, the determining module 61 is configured to: determine the address information of the second tensor from the address of the first tensor, based on the tensor information of the first tensor, the position of the second tensor in the first tensor, and the size of the second tensor, when the second tensor to be accessed is a block of data of the first tensor, and the access information includes the position of the second tensor in the first tensor and the size of the second tensor.

[0128] In one possible implementation, the determining module 61 is further configured to: perform a boundary check on the second tensor based on the tensor information of the first tensor and the access information of the second tensor to be accessed, and obtain a check result for the second tensor; if the check result indicates that the first part of the second tensor is located within the first tensor and the second part of the second tensor is located outside the first tensor, determine the address information of the first part of the second tensor based on the tensor information of the first tensor and the access information of the second tensor; the reading module 62 is configured to: if the tensor information of the first tensor includes a padding identifier, read the first part of the second tensor from the global memory storing the first tensor to the local memory based on the address information of the first part of the second tensor, pad the second part of the second tensor according to the padding identifier, and obtain the second part of the second tensor; and write the second part of the second tensor into the local memory.

[0129] In one possible implementation, the determining module 61 is configured to: when the second tensor to be accessed is the coverage data corresponding to any one or more elements in the convolution kernel in the first tensor, and the access information includes the coordinates of the unfolded matrix and the coordinates of the convolution kernel elements, obtain the mapping relationship between the unfolded matrix and the first tensor; determine the address information of the second tensor from the address of the first tensor according to the tensor information of the first tensor, the mapping relationship, the coordinates of the unfolded matrix, and the coordinates of the convolution kernel elements; wherein, the unfolded matrix is ​​a matrix formed by sequentially converting the coverage data corresponding to the receptive field of the convolution kernel into corresponding column vectors or row vectors in the first tensor according to the description information, and arranging them in rows or columns, and the description information includes convolution description information and deconvolution description information.

[0130] In one possible implementation, determining the address information of the second tensor from the address of the first tensor based on the tensor information of the first tensor, the mapping relationship, the coordinates of the unfolded matrix, and the coordinates of the convolution kernel elements includes: determining the second tensor to be accessed from the unfolded matrix based on the coordinates of the unfolded matrix and the coordinates of the convolution kernel elements; determining multiple elements corresponding to the second tensor in the first tensor based on the mapping relationship between the unfolded matrix and the first tensor; and determining the address information of the multiple elements corresponding to the second tensor in the first tensor as the address information of the second tensor.

[0131] In one possible implementation, the reading module 62 is configured to: when the size of the second tensor is greater than the reading bandwidth, split a first memory access request for reading the second tensor from the global memory to the local memory into multiple second memory access requests based on the address information of the second tensor, each second memory access request being used to read split data of different parts of the second tensor from the global memory to the local memory; and serially write multiple split data constituting the second tensor from the global memory into the local memory based on the multiple second memory access requests.

[0132] In one possible implementation, the tensor information of the first tensor is stored in global memory, and the access information of the second tensor to be accessed is stored in a register.

[0133] In one possible implementation, the first tensor includes feature data from a deep learning task, wherein the feature data includes at least one of image feature data, speech feature data, and text feature data.

[0134] This method is specifically technically related to the internal structure of computer systems and can solve technical problems of how to improve hardware computing efficiency or execution performance (including reducing data storage, reducing data transmission, and increasing hardware processing speed), thereby achieving technical effects that improve the internal performance of computer systems in accordance with natural laws.

[0135] In some embodiments, the functions or modules of the apparatus provided in this disclosure can be used to perform the methods described in the above method embodiments. The specific implementation can be referred to the description of the above method embodiments, and for the sake of brevity, it will not be repeated here.

[0136] This disclosure also proposes a computer-readable storage medium storing computer program instructions that, when executed by a processor, implement the above-described method. The computer-readable storage medium can be volatile or non-volatile.

[0137] This disclosure also proposes an electronic device, including: a processor; and a memory for storing processor-executable instructions; wherein the processor is configured to invoke the instructions stored in the memory to execute the above-described method.

[0138] This disclosure also provides a computer program product, including computer-readable code, or a non-volatile computer-readable storage medium carrying computer-readable code, wherein when the computer-readable code is run in a processor of an electronic device, the processor in the electronic device performs the above-described method.

[0139] The aforementioned computer program product can be implemented through hardware, software, or a combination thereof. In one optional embodiment, the computer program product is specifically embodied in a computer storage medium; in another optional embodiment, the computer program product is specifically embodied in a software product, such as a software development kit (SDK), etc.

[0140] This disclosure also provides an electronic device, which can be provided as a terminal, server, or other type of device. Examples include user equipment (UE), mobile device, user terminal, terminal, cellular phone, cordless phone, personal digital assistant (PDA), handheld device, computing device, in-vehicle device, wearable device, etc.

[0141] Figure 7 A block diagram of an electronic device 1900 according to an embodiment of the present disclosure is shown. For example, the electronic device 1900 may be provided as a server or a terminal device. (Refer to...) Figure 7The electronic device 1900 includes a processing component 1922, which further includes one or more processors, and memory resources represented by memory 1932 for storing instructions, such as application programs, that can be executed by the processing component 1922. The application programs stored in memory 1932 may include one or more modules, each corresponding to a set of instructions. Furthermore, the processing component 1922 is configured to execute instructions to perform the methods described above.

[0142] Electronic device 1900 may also include a power supply component 1926 configured to perform power management of electronic device 1900, a wired or wireless network interface 1950 configured to connect electronic device 1900 to a network, and an input / output interface 1958. Electronic device 1900 can operate on an operating system stored in memory 1932, such as Microsoft Server operating system (Windows Server). TM Apple's graphical user interface-based operating system (Mac OS X) TM ), a multi-user, multi-process computer operating system (Unix) TM Linux is a free and open-source Unix-like operating system. TM ), the open-source Unix-like operating system (FreeBSD) TM (or similar.)

[0143] In an exemplary embodiment, a non-volatile computer-readable storage medium is also provided, such as a memory 1932 including computer program instructions that can be executed by a processing component 1922 of an electronic device 1900 to perform the above-described method.

[0144] This disclosure can be a system, method, and / or computer program product. A computer program product may include a computer-readable storage medium having computer-readable program instructions loaded thereon for causing a processor to implement various aspects of this disclosure.

[0145] Computer-readable storage media can be tangible devices capable of holding and storing instructions for use by an instruction execution device. Computer-readable storage media can be, for example, (but not limited to) electrical storage devices, magnetic storage devices, optical storage devices, electromagnetic storage devices, semiconductor storage devices, or any suitable combination of the foregoing. More specific examples (a non-exhaustive list) of computer-readable storage media include: portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), static random access memory (SRAM), portable compact disc read-only memory (CD-ROM), digital multifunction disc (DVD), memory sticks, floppy disks, mechanical encoding devices, such as punch cards or recessed protrusions storing instructions thereon, and any suitable combination of the foregoing. The computer-readable storage media used herein are not to be construed as transient signals themselves, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through waveguides or other transmission media (e.g., light pulses through fiber optic cables), or electrical signals transmitted through wires.

[0146] The computer-readable program instructions described herein can be downloaded from computer-readable storage media to various computing / processing devices, or downloaded via a network, such as the Internet, local area network, wide area network, and / or wireless network, to an external computer or external storage device. The network may include copper transmission cables, fiber optic transmission, wireless transmission, routers, firewalls, switches, gateway computers, and / or edge servers. A network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards them to the computer-readable storage media in the respective computing / processing device.

[0147] Computer program instructions used to perform the operations of this disclosure may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, status setting data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Smalltalk, C++, etc., and conventional procedural programming languages ​​such as the "C" language or similar programming languages. The computer-readable program instructions may execute entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or may be connected to an external computer (e.g., via the Internet using an Internet service provider). In some embodiments, electronic circuitry, such as programmable logic circuitry, field-programmable gate arrays (FPGAs), or programmable logic arrays (PLAs), is personalized by utilizing the status information of the computer-readable program instructions to implement various aspects of this disclosure.

[0148] Various aspects of this disclosure are described herein with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this disclosure. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer-readable program instructions.

[0149] These computer-readable program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus to produce a machine such that, when executed by the processor of the computer or other programmable data processing apparatus, they create means for implementing the functions / actions specified in one or more blocks of the flowchart and / or block diagram. These computer-readable program instructions can also be stored in a computer-readable storage medium that causes a computer, programmable data processing apparatus, and / or other device to operate in a particular manner; thus, the computer-readable medium storing the instructions comprises an article of manufacture that includes instructions for implementing aspects of the functions / actions specified in one or more blocks of the flowchart and / or block diagram.

[0150] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions executed on the computer, other programmable data processing apparatus, or other device to perform the functions / actions specified in one or more boxes of a flowchart and / or block diagram.

[0151] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of an instruction containing one or more executable instructions for implementing a specified logical function. In some alternative implementations, the functions marked in the blocks may occur in a different order than those shown in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, may be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions.

[0152] The computer program product can be implemented specifically through hardware, software, or a combination thereof. In one alternative embodiment, the computer program product is specifically embodied in a computer storage medium; in another alternative embodiment, the computer program product is specifically embodied in a software product, such as a software development kit (SDK), etc.

[0153] The description of the various embodiments above tends to emphasize the differences between the various embodiments. The similarities or similarities between them can be referred to, and for the sake of brevity, they will not be repeated here.

[0154] Those skilled in the art will understand that, in the above-described method of the specific implementation, the order in which each step is written does not imply a strict execution order and does not constitute any limitation on the implementation process. The specific execution order of each step should be determined by its function and possible internal logic.

[0155] The various embodiments of this disclosure have been described above. These descriptions are exemplary and not exhaustive, nor are they limited to the disclosed embodiments. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described embodiments. The terminology used herein is chosen to best explain the principles, practical application, or improvement of the technology in the market, or to enable others skilled in the art to understand the embodiments disclosed herein.

Claims

1. A data processing method, characterized by, The method comprises: determining address information of the second tensor according to tensor information of the first tensor and access information of the second tensor to be accessed; wherein the second tensor comprises any one of the following: patch data of the first tensor, covering data corresponding to any one or more elements of a convolution kernel in the first tensor, and data formed by the covering data and constant value padding between elements in the covering data; the tensor information of the first tensor comprises data structure information of the first tensor and address information of the first tensor; reading the second tensor from a global memory storing the first tensor to a local memory according to the address information of the second tensor; triggering an interrupt in response to the local memory receiving the second tensor.

2. The method of claim 1, wherein, in a case where the second tensor to be accessed is patch data of the first tensor, the access information comprises a position of the second tensor in the first tensor and a size of the second tensor, determining address information of the second tensor according to tensor information of the first tensor and access information of the second tensor to be accessed, comprising: determining the address information of the second tensor from the address of the first tensor according to the tensor information of the first tensor, the position of the second tensor in the first tensor, and the size of the second tensor.

3. The method of claim 2, wherein, The method further comprises: performing boundary checking on the second tensor according to the tensor information of the first tensor and the access information of the second tensor to be accessed to obtain a checking result of the second tensor; in a case where the checking result indicates that a first part of the second tensor is located within the first tensor and a second part of the second tensor is located outside the first tensor, determining address information of the first part of the second tensor according to the tensor information of the first tensor and the access information of the second tensor; the tensor information of the first tensor comprises a padding identifier, and reading the second tensor from a global memory storing the first tensor to a local memory according to the address information of the second tensor, comprising: reading the first part of the second tensor from the global memory storing the first tensor to the local memory according to the address information of the first part of the second tensor, padding the second part of the second tensor according to the padding identifier to obtain the second part of the second tensor; writing the second part of the second tensor into the local memory.

4. The method of claim 1, wherein, in a case where the second tensor to be accessed is covering data corresponding to any one or more elements of a convolution kernel in the first tensor, the access information comprises unfolded matrix coordinates and convolution kernel element coordinates, the unfolded matrix is a matrix arranged by rows or columns in the first tensor by sequentially converting covering data corresponding to a receptive field of the convolution kernel into corresponding column vectors or row vectors according to description information, and the description information comprises convolution description information and deconvolution description information, determining address information of the second tensor according to tensor information of the first tensor and access information of the second tensor to be accessed, comprising: obtaining a mapping relationship between the unfolded matrix and the first tensor; According to the tensor information of the first tensor, the mapping relationship, the unfolded matrix coordinate, and the convolution kernel element coordinate, address information of the second tensor is determined from an address of the first tensor.

5. The method of claim 4, wherein, According to the tensor information of the first tensor, the mapping relationship, the unfolded matrix coordinate, and the convolution kernel element coordinate, address information of the second tensor is determined from an address of the first tensor, including: According to the unfolded matrix coordinate and the convolution kernel element coordinate, the second tensor to be accessed is determined from the unfolded matrix; According to the mapping relationship between the unfolded matrix and the first tensor, a plurality of elements corresponding to the second tensor in the first tensor are determined; Address information of the plurality of elements corresponding to the second tensor in the first tensor is determined as address information of the second tensor.

6. The method according to any one of claims 1-5, characterized in that, According to the address information of the second tensor, the second tensor is read from a global memory storing the first tensor to a local memory, including: In a case where a size of the second tensor is greater than a reading bandwidth, according to the address information of the second tensor, a first memory access request for reading the second tensor from the global memory to the local memory is split into a plurality of second memory access requests, each of which is used for reading split data of different parts of the second tensor from the global memory to the local memory; According to the plurality of second memory access requests, a plurality of split data constituting the second tensor in the global memory are written to the local memory in series.

7. The method according to any one of claims 1-5, characterized in that, The tensor information of the first tensor is stored in a global memory, and the access information of the second tensor to be accessed is stored in a register.

8. The method according to any one of claims 1-5, characterized in that, The first tensor includes feature data in a deep learning task, and the feature data includes at least one of image feature data, speech feature data, and text feature data.

9. A data processing apparatus, characterized by, Including: A determination module is configured to determine address information of a second tensor according to tensor information of a first tensor and access information of the second tensor to be accessed, wherein the second tensor includes any one of block data of the first tensor, covering data corresponding to any one or more elements in a convolution kernel in the first tensor, and data formed by the covering data and constant value padding between elements in the covering data; the tensor information of the first tensor includes data structure information of the first tensor and address information of the first tensor; A reading module is configured to read the second tensor from a global memory storing the first tensor to a local memory according to the address information of the second tensor. A triggering module is configured to trigger an interrupt in response to the local memory receiving the second tensor.

10. An electronic device, comprising: Including: A memory and a processor; The memory stores computer readable instructions, and the computer readable instructions are executed by the processor to perform the method in any one of claims 1 to 8.

11. A computer-readable storage medium having stored thereon computer program instructions, wherein, The computer program instructions are executed by the processor to implement the method in any one of claims 1 to 8.

12. A computer program product comprising computer readable code, or a computer readable storage medium carrying computer readable code, which when run in a processor of an electronic device, the processor in the electronic device performs the method of any one of claims 1 to 8.

Images