A data access method, apparatus and readable storage medium

Abstract

This application discloses a data access method, device, and readable storage medium, relating to the field of computer technology. This application broadcasts settings for the input of a target operator. Then, based on the tensor memory layout characteristics of the target operator, the dimension types corresponding to the input and output are determined. Thus, based on the dimension types, the memory corresponding to the input and output can be merged and rearranged, and then the memory space can be assembled in segments. Therefore, upon receiving a data access instruction, the memory segment to be accessed can be determined using the data access instruction, and the tensor data of the target operator can be accessed using a data access method adapted to the memory segment, thereby accelerating the acquisition of tensor data. This application transforms the tensor memory space of the target operator from a scattered and disorganized state to a segmented and assembled memory space, resulting in a regular memory arrangement for storing the same dimension type. When accessing data, a data access method adapted to the memory segment is used, enabling rapid access to the corresponding tensor data.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular to a data access method, apparatus and readable storage medium. Background Technology

[0002] In deep learning tasks, data typically requires various preprocessing operations, such as normalization and standardization. These operations often involve performing specific mathematical operations on each element of the dataset. Elementwise binary broadcast operators allow for element-wise operations on data structures of varying shapes.

[0003] The existence of broadcast dimensions leads to discontinuous memory access, impacting data retrieval efficiency. Currently, explicit data copying for broadcasting increases data copying time and memory usage, offering little improvement in data retrieval efficiency. While calculating the data's offset in memory based on the tensor's stride and retrieving the data using this offset avoids unnecessary data copying, this method still suffers from low data retrieval efficiency due to the uniformly treating data as discontinuous.

[0004] Therefore, how to improve the speed of data retrieval from memory during operator computation is a technical problem that urgently needs to be solved by those skilled in the art. Summary of the Invention

[0005] This application provides a data access method, device, and readable storage medium that can accelerate data access from memory during operator computation, thereby increasing computational efficiency.

[0006] This application provides a data access method, including:

[0007] Obtain the input of the target operator, and if the input satisfies the broadcast condition, broadcast the setting of the input;

[0008] Obtain the tensor memory layout features of the target operator, and use the tensor memory layout features to determine the dimension types corresponding to the input and output respectively;

[0009] After performing continuous dimension merging, step dimension sorting, and broadcast dimension merging on the memory spaces corresponding to the input and output respectively, based on the dimension type, the memory spaces are reorganized in segments.

[0010] The memory segment to be accessed is determined using data access instructions, and the tensor data of the target operator is accessed using a data access method adapted to the memory segment.

[0011] This application also provides a data access device, including:

[0012] The broadcast setting module is used to obtain the input of the target operator and, if the input satisfies the broadcast requirement, to broadcast the setting of the input.

[0013] The memory layout analysis module is used to obtain the tensor memory layout features of the target operator and use the tensor memory layout features to determine the dimension types corresponding to the input and output respectively.

[0014] The memory layout merging and rearranging module is used to combine the dimension type to perform continuous dimension merging, step dimension sorting and broadcast dimension merging on the memory spaces corresponding to the input and the output respectively, and then reorganize the memory space in segments.

[0015] The operator computation module is used to determine the memory segment to be accessed using data access instructions, and to access the tensor data of the target operator using a data access method adapted to the memory segment.

[0016] This application also provides an electronic device, including: a memory for storing a computer program; and a processor for implementing the steps of any of the above data access methods when executing the computer program.

[0017] This application also provides a computer-readable storage medium storing a computer program, wherein the computer program, when executed by a processor, implements the steps of any of the above-described data access methods.

[0018] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the steps of any of the above-described data access methods.

[0019] This application first performs broadcast settings for the input of the target operator. Then, based on the tensor memory layout characteristics of the target operator, the dimension types corresponding to the input and output are determined. Thus, the memory corresponding to the input and output can be merged and rearranged based on the dimension types. After the memory merging and rearrangement are completed, the memory space is assembled in segments. Therefore, upon receiving a data access instruction, the memory segment to be accessed can be determined using the data access instruction, and the tensor data of the target operator can be accessed using a data access method adapted to the memory segment, thereby accelerating the acquisition of tensor data.

[0020] This application has the technical effect of transforming the tensor memory space of the target operator from a scattered and disorganized state to a segmented and assembled memory space, so that the memory arrangement of storing the same dimension type is regular. When accessing data, a data access method adapted to the memory segment is adopted, which can quickly access the corresponding tensor data and further accelerate the operator's computation efficiency. Attached Figure Description

[0021] To more clearly illustrate the embodiments of this application, the accompanying drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0022] Figure 1 A flowchart illustrating a data access method provided in an embodiment of this application;

[0023] Figure 2 This application provides a schematic diagram of a broadcast setup process.

[0024] Figure 3 This is a schematic diagram illustrating a process for determining a dimension type, as provided in an embodiment of this application.

[0025] Figure 4 This is a schematic diagram of memory merging and reorganization provided in an embodiment of this application;

[0026] Figure 5 This is a schematic diagram of the structure of a data access device provided in an embodiment of this application;

[0027] Figure 6 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application;

[0028] Figure 7 This is a schematic diagram of the specific structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0029] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the protection scope of this application.

[0030] It should be noted that, in the description of this application, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. The terms "first," "second," etc., in this application are used to distinguish similar objects and are not used to describe a specific order or sequence.

[0031] To enable those skilled in the art to better understand the present application, the present application will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0032] For ease of understanding, the technical terms used in this application will be explained below.

[0033] Element-wise operators are a class of operations that perform independent operations on each element at a corresponding position in a tensor (or data structure such as arrays or matrices).

[0034] Tensor broadcasting is an important concept in deep learning and scientific computing. It allows tensors of different shapes to automatically expand into a compatible shape when performing arithmetic operations without explicitly copying the data. This mechanism can significantly simplify code and improve computational efficiency.

[0035] The basic rules of broadcasting are as follows: When two tensors are operated on, frameworks such as NumPy (a basic library for scientific computing), PyTorch (a dynamic computing framework), and TensorFlow (an industrial application framework) will automatically broadcast according to the following rules: Right alignment of dimensions: The shapes of the two tensors are compared starting from the rightmost dimension; Dimension compatibility conditions: The two dimensions are equal; One of the dimensions is 1; Automatic expansion: The axis with dimension 1 is copied along that dimension to match the corresponding dimension of the other tensor.

[0036] A tensor is a mathematical concept that is a generalization of the concepts of vectors and matrices.

[0037] The way layout data is stored in memory and the order in which it is calculated.

[0038] CUDA is a parallel computing platform and programming model, or it can be considered a programming language.

[0039] Kernel: In CUDA (Compute Unified Device Architecture), kernel refers to parallel computing functions that run on NVIDIA GPUs (a type of graphics processing unit).

[0040] CUDA vectorization: CUDA vectorization refers to the process of improving performance by utilizing CUDA's vector data types and instructions. The main purpose of vectorization is to reduce memory accesses and increase instruction throughput.

[0041] Shape, the shape of a tensor is an integer tuple that represents the size of the tensor in each dimension, which is referred to as shape in this paper.

[0042] The stride of a tensor is an integer tuple of the same length as the shape, representing the number of elements to skip when traversing a single dimension in memory. In other words, the stride indicates how to calculate the position of the next element from its memory address when accessing a specific dimension of the tensor. This paper refers to the stride as the step size.

[0043] Ndim, the dimension of a tensor.

[0044] In CUDA programming, grid refers to the thread grid, which is a two-dimensional or three-dimensional layout composed of multiple thread blocks.

[0045] blockIdx is also a variable of type dim3, representing the index of the current thread block in the entire thread grid.

[0046] threadIdx is a variable of type dim3, representing the index of the current thread within its own thread block.

[0047] vect_num: The number of data items processed in the vectorized batch.

[0048] Please refer to Figure 1 , Figure 1 This is a flowchart of a data access method according to an embodiment of this application, which includes the following steps.

[0049] S101. Obtain the input of the target operator, and broadcast the input if the input satisfies the broadcast condition.

[0050] The target operator can be either an operator that requires broadcasting or an operator that does not require broadcasting. This target operator can be used in scenarios that actually require corresponding operator computation, such as speech recognition, natural language processing, semantic recognition, and face recognition.

[0051] When the target operator satisfies broadcast conditions, the input can be broadcast-configured. Broadcast configuration involves expanding the operator's input according to the rules of broadcast computation.

[0052] In one specific embodiment of this application, when the input satisfies broadcasting, broadcasting settings for the input are performed, including: if the input includes a first input and a second input, determining whether the first input and the second input satisfy broadcasting; if so, setting the stride[i] of the broadcasting dimension dim_broadcast to 0. That is, determining whether broadcasting is satisfied based on the shape of the input, and if broadcasting is satisfied, setting the stride[i] of the broadcasting dimension dim_broadcast to 0.

[0053] For the two inputs of the target operator, i.e., the two tensors, broadcasting is possible when the following conditions are met:

[0054] 1. Shape alignment: Starting from the last dimension (rightmost), compare each dimension backwards. Each dimension must satisfy: the dimensions are equal, or one of the tensors has a dimension size of 1, or one of the tensors does not have that dimension (i.e., fewer dimensions).

[0055] 2. At least one dimension of the tensor needs to be expanded: if the shapes are completely identical, there is no need for broadcasting.

[0056] Specifically, such as Figure 2 As shown, the first step is to determine the shapes of the two inputs. If the input shapes are not equal, and one of them has a shape value of 1, then the stride of that dimension is set to 0. If the lengths of the two shapes are not equal, then the shorter shape is extended to match the longer shape, and the extra shape is set to 1. For example, if the first input corresponds to shape1=[2,3,4] and the second input corresponds to shape2=[4], then the length of shape2 is extended to match the length of shape1, and the extra shape is set to 1, so shape2 becomes [1,1,4].

[0057] S102. Obtain the tensor memory layout features of the target operator, and use the tensor memory layout features to determine the dimension types corresponding to the input and output respectively.

[0058] The tensor of the target operator includes input and output.

[0059] By obtaining the tensor memory layout features of the target operator, the dimension types corresponding to the input and output can be determined using these features.

[0060] Specifically, the memory layout features of the input and output tensors can be analyzed separately, starting from the innermost dimension and working outwards to analyze continuity and determine the dimension type.

[0061] In one specific embodiment of this application, the dimension types corresponding to the input and output are determined using tensor memory layout features, including: determining the memory continuity of the input using tensor memory layout features; and determining the dimension type of each dimension in the input using memory continuity.

[0062] Among them, memory continuity corresponds to step size. The dimension type of each dimension in the input is determined by memory continuity, including: if the step size of the current dimension is 0, the dimension type of the current dimension is determined to be a broadcast dimension; if the step size of the current dimension is 1, the dimension type of the current dimension is determined to be a continuous dimension; if the step size of the current dimension is equal to the product of the size of the next dimension and the step size, the dimension type of the current dimension is determined to be a continuous dimension; if the step size of the current dimension is neither 0 nor 1 and is not the product of the size of the next dimension and the step size, the dimension type of the current dimension is determined to be a strided dimension.

[0063] In one specific embodiment of this application, different stride determination methods are used for different tensors. Specifically, for contiguous memory tensors, the stride can be calculated according to the following rules: starting from the innermost dimension (the last dimension), the initial stride is 1. Calculate layer by layer outwards: s[i] = s[i+1] × shape[i+1]. For non-contiguous memory tensors, if the tensor has undergone transpose, slicing, or other operations, the stride will be rearranged and may no longer be contiguous. The stride determination method can be obtained by executing the following code:

[0064] Python;

[0065] copy;

[0066] C = torch.rand(3, 4).transpose(0, 1) # shape (4, 3);

[0067] print(C.stride()) # Output (1, 4).

[0068] After obtaining the step size, the memory layout features of the input and output tensors are analyzed separately. The continuity is analyzed from the innermost dimension outwards. The rules for determining the dimension type are as follows: if the step size is 0, it corresponds to a broadcast dimension (B); if the step size is 1, it corresponds to a continuous dimension (C); if the step size is equal to the size of the next dimension × the step size, it corresponds to a continuous dimension (C); if the step size corresponds to other cases, then the corresponding step dimension (S) is determined.

[0069] Please refer to Figure 3 The continuity of tensor memory layout features is analyzed from the innermost dimension outwards. Regarding the explanation of inner and outer layers, the following uses a stride of [20, 5, 1] ​​as an example to explain how to determine the dimension type.

[0070] The innermost step size is strides[2] = 1; the middle step size is strides[1] = strides[2] * shape[2] = 5; the outermost step size is strides[0] = strides[1] * shape[1] = 20. Starting from the highest dimension (i.e. the innermost layer), if strides[i] = 0, the dimension type dim_type is set to B (Broacast); if strides[i] = 1, the dimension type dim_type is set to C (Contiguous); if the step size is equal to the next dimension size × step size, i.e., strides[i] == shape[i+1] * strides[i+1], the dimension type is set to dim_type set to C (Contiguous); other cases correspond to stride (S), and the dimension type is set to dim_type set to S (Stride).

[0071] S103. After combining the dimension types, the memory spaces corresponding to the input and output are merged into continuous dimensions, sorted by step dimension, and merged by broadcast dimension, and then the memory spaces are reorganized in segments.

[0072] After clarifying the dimension types of the input and output, the memory spaces corresponding to the input and output can be merged and sorted according to the corresponding dimension types. After merging and sorting, the memory space is reorganized into segments. This allows the fragmented memory spaces to be merged into larger memory segments, storing approximate vectors in the same memory for easier data access.

[0073] In one specific embodiment of this application, the memory spaces corresponding to the input and output are merged according to the dimension type, including continuous dimension merging, step dimension sorting, and broadcast dimension merging. If the dimension type is continuous and the merging and rearrangement do not change the correspondence between the input and output, then the memory spaces corresponding to the input and output are merged. If the dimension type is step dimension, then the memory spaces corresponding to the input and output are arranged in ascending order according to the original step size. If the dimension type is broadcast dimension and the merging and rearrangement do not change the correspondence between the input and output, then the memory spaces corresponding to the input and output are merged.

[0074] For details, please refer to Figure 4 The memory layout after memory analysis will have dimension types. Based on the dimension types, the memory of all input and output data is merged and rearranged from the outside in. Specific processing operations include the following cases.

[0075] If the dimension type is C, perform continuous dimension merging. This combines physically continuous dimensions into a larger computational block.

[0076] If the dimension type is S, sort by step size. Sort by step size from smallest to largest to improve memory access locality.

[0077] If the dimension type is B, broadcast dimensions are merged. Merging adjacent broadcast dimensions reduces the need for conditional checks.

[0078] It is important to note that during the merge and rearrangement process, in order to ensure that the correspondence between inputs and outputs is not changed after the merge and rearrangement, the correspondence between inputs and outputs must be checked. If the correspondence has changed, the merge and rearrangement will not be performed.

[0079] For example: First, the dimensions can be grouped according to the dimension type dim_type, and each group has a corresponding dim index.

[0080] For example, vect_C stores the index of the dimension accessed continuously in memory, vect_S stores the index of the dimension accessed non-contiguously, and vect_B stores the index of the dimension with a shape size of 1 and a stride of 0. For example, if shape=[5,4,1], stride=[4,1,0], and dim_type is [C,C,B], then the value of vect_C is [0,1], vect_S is empty, and vect_B is [2].

[0081] For merging consecutive dimensions, traverse vect_C from the inside out. The merging condition is: adjacent consecutive dimensions satisfy stride[i] == stride[i+1] * size[i+1]. To ensure that the correspondence between input and output is not changed after merging and rearranging, the three input and output vectors must simultaneously satisfy stride[i] == stride[i+1] * size[i+1] before merging will proceed. For example, in the case of a+b=c, a has shape=[3,4,5] and stride=[20,5,1], b has shape=[3,4,1] and stride=[12,1,0], c has shape=[3,4,5] and stride=[20,5,1], a has a vect_C value of [0,1], b has a vect_C value of [0,1], and c has a vect_C value of [0,1]. In this case, a, b, and c satisfy stride[0] == stride[1] * size[1], and the dimensions of a, b, and c can be merged. However, if b has shape=[3,4,1], stride=[20,1,0], and a vect_C value of [1], then stride[i] == stride[i+1] * size[i+1] cannot be merged.

[0082] After merging, update the shape to the product of the shapes corresponding to the subscripts of vect_C, the stride to the minimum value of the stride corresponding to the subscript of vect_C, and update the subscript of vect_C to the minimum value of the corresponding subscript.

[0083] The merged dimensions are: shape=[12,5], stride=[5,1] for a, shape=[12,1], stride=[1,0] for b, and shape=[12,5], stride=[5,1] for c.

[0084] For step-dimensional sorting, traverse vect_S from the inside out. It can be sorted in ascending order of the original step size (smaller step size first) to optimize locality.

[0085] For broadcast dimension merging, iterate through vect_B from the inside out. Merge all broadcast dimensions. To ensure that the correspondence between input and output is not changed after merging and rearranging, the relationship between the three input and output vectors must also be checked. For example, for a+b=c, a's shape=[3,4,5,6], stride=[120,30,6,1], b's shape=[3,4,1,1], stride=[12,1,0,0], c's shape=[3,4,5,6], stride=[120,30,6,1], a's vect_C value is empty, b's vect_B value is [2,3], and c's vect_C value is []. At this time, a, b, and c satisfy stride[2] == stride[3] * size[2], so the dimensions of a, b, and c can be merged. After merging, update the shape of vect_C to the product of the shapes corresponding to the indices of vect_C, and the stride to the minimum stride value corresponding to the indices of vect_C. Update the indices of vect_C to the minimum value of their corresponding indices. Update the shape of vect_B to the product of the shapes corresponding to the indices of vect_B, multiply the broadcast dimensions by their sizes (size * = next_size), keep the stride at 0, and update the indices of vect_B to the minimum value of their corresponding indices. The merged dimensions are: shape=[3,4,30], stride=[120,30,1] for a, shape=[3,4,1], stride=[4,1,0] for b, and shape=[3,4,30], stride=[120,30,1].

[0086] In one specific embodiment of this application, the segmented reorganization of memory space includes: sorting the memory segments corresponding to the merged continuous dimension, the sorted stride dimension, and the merged broadcast dimension according to priority; and assembling the memory segments after sorting. The final dimension assembly order can be continuous → stride → broadcast, and the merged and rearranged dimension type becomes (C, S, B), and the corresponding merged shape and stride are obtained.

[0087] In other words, the dimensions are assembled in order and arranged according to priority, resulting in the following new memory space: [merged contiguous dimensions] → [sorted straddle dimensions] → [merged broadcast dimensions].

[0088] S104. Use data access instructions to determine the memory segment to be accessed, and use the data access method adapted to the memory segment to access the tensor data of the target operator.

[0089] In this application, after operations such as reorganization and sorting, the memory space exists in segments such as [merged continuous dimension], [sorted stride dimension], and [merged broadcast dimension]. Therefore, the optimal data access method can be set for different memory segments.

[0090] That is, when a target operator needs to be invoked or computed, it is considered that a data access instruction has been received. This data access instruction can determine the memory segment to be accessed. By using a data access method adapted to that memory segment, the corresponding tensor data can be accessed quickly. Accordingly, data access instructions can be divided into data storage instructions and data read instructions, with different instructions corresponding to the writing or reading of data.

[0091] In one specific embodiment of this application, tensor data of the target operator is accessed using a data access method adapted to the memory segment, including: if the memory segment is contiguous memory, tensor data is accessed in batches from the memory segment in a vectorized manner; if the memory segment is spanned memory, tensor data is accessed centrally from the memory segment in an offset manner; if the memory segment is broadcast memory, tensor data is reused in a data reuse manner.

[0092] The process of batch accessing tensor data from memory segments in a vectorized manner includes: calling a vectorization function to batch access tensor data from memory segments.

[0093] Specifically, segmented kernel processing is adopted. Based on the rearranged input and output memory, the operators are calculated. When the dimension type is C, data is stored in a vectorized manner. When the dimension type is S, data is stored by calculating the offset. When the dimension type is B, the data is reused.

[0094] That is, for the memory segment containing the vectors corresponding to the dimensions of vect_C, both input and output are vectorized for computation. For the memory segment containing the vectors corresponding to the dimensions of vect_S, both input and output are stored and retrieved by calculating offsets, and computation is performed on the data. For the memory segment containing the vectors corresponding to the dimensions of vect_B, the non-empty input of vect_B is reused, and the other input and output are vectorized.

[0095] Applying the method provided in this application's embodiments, firstly, a broadcast setting is performed for the input of the target operator. Then, the dimension types corresponding to the input and output are determined based on the tensor memory layout characteristics of the target operator. Thus, the memory corresponding to the input and output can be merged and rearranged based on the dimension types. After the memory merging and rearrangement are completed, the memory space is assembled in segments. Therefore, upon receiving a data access instruction, the memory segment to be accessed can be determined using the data access instruction, and the tensor data of the target operator can be accessed using a data access method adapted to the memory segment, thereby accelerating the acquisition of tensor data.

[0096] This application has the technical effect of transforming the tensor memory space of the target operator from a scattered and disorganized state to a segmented and assembled memory space, so that the memory arrangement of storing the same dimension type is regular. When accessing data, a data access method adapted to the memory segment is adopted, which can quickly access the corresponding tensor data and further accelerate the operator's computation efficiency.

[0097] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods according to the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method.

[0098] Please refer to Figure 5 Embodiments of this application also provide a data access device, the device comprising:

[0099] The broadcast setting module 101 is used to obtain the input of the target operator and, if the input satisfies the broadcast requirements, to broadcast the settings of the input.

[0100] The memory layout analysis module 102 is used to obtain the tensor memory layout features of the target operator and use the tensor memory layout features to determine the dimension types corresponding to the input and output respectively.

[0101] The memory layout merging and rearranging module 103 is used to combine the dimension type to perform continuous dimension merging, step dimension sorting and broadcast dimension merging on the memory spaces corresponding to the input and output respectively, and then reorganize the memory space in segments.

[0102] The operator calculation module 104 is used to determine the memory segment to be accessed using data access instructions, and to access the tensor data of the target operator using a data access method adapted to the memory segment.

[0103] Using the apparatus of this application, the input of the target operator is first broadcast and configured. Then, the dimension types corresponding to the input and output are determined based on the tensor memory layout characteristics of the target operator. This allows for merging and rearranging of the memory corresponding to the input and output based on the dimension types. After merging and rearranging the memory, the memory space is assembled in segments. Thus, upon receiving a data access instruction, the memory segment to be accessed can be determined using the instruction, and the tensor data of the target operator can be accessed using a data access method adapted to that segment, thereby accelerating the acquisition of tensor data.

[0104] This application has the technical effect of transforming the tensor memory space of the target operator from a scattered and disorganized state to a segmented and assembled memory space, so that the memory arrangement of storing the same dimension type is regular. When accessing data, a data access method adapted to the memory segment is adopted, which can quickly access the corresponding tensor data and further accelerate the operator's computation efficiency.

[0105] In one specific embodiment of this application, the operator calculation module is specifically used to: if the memory segment is contiguous memory, then retrieve tensor data in batches from the memory segment in a vectorized manner; if the memory segment is spanned memory, then retrieve tensor data centrally from the memory segment in an offset manner; if the memory segment is broadcast memory, then reuse tensor data in a data reuse manner.

[0106] In one specific embodiment of this application, the operator computation module is specifically used to call a vectorization function to batch access tensor data from a memory segment.

[0107] In one specific embodiment of this application, the broadcast setting module is specifically used to determine whether the first input and the second input satisfy broadcast if the input includes a first input and a second input; if so, the step size of the dimension corresponding to the broadcast of the first input or the second input is set to 0.

[0108] In one specific embodiment of this application, the memory layout analysis module is specifically used to determine the memory continuity of the input using tensor memory layout features; and to determine the dimension type of each dimension in the input using memory continuity.

[0109] In one specific embodiment of this application, memory continuity corresponds to step size. The memory layout analysis module is specifically used to determine the dimension type of the current dimension as a broadcast dimension if the step size of the current dimension is 0; determine the dimension type of the current dimension as a continuous dimension if the step size of the current dimension is 1; determine the dimension type of the current dimension as a continuous dimension if the step size of the current dimension is equal to the product of the size of the next dimension and the step size; and determine the dimension type of the current dimension as a straddle dimension if the step size of the current dimension is neither 0 nor 1 and is not the product of the size of the next dimension and the step size.

[0110] In one specific embodiment of this application, the memory layout merging and rearranging module is specifically used to sort the memory segments corresponding to the merged continuous dimension, the sorted stride dimension, and the merged broadcast dimension according to priority; after the memory segments are sorted, the memory segments are assembled.

[0111] In one specific embodiment of this application, the memory layout merging and rearranging module is specifically used to merge the memory spaces corresponding to the input and output if the dimension type is a continuous dimension and the merging and rearranging does not change the correspondence between the input and output; if the dimension type is a step dimension, the memory spaces corresponding to the input and output are arranged in ascending order according to the original step size; if the dimension type is a broadcast dimension and the merging and rearranging does not change the correspondence between the input and output, the memory spaces corresponding to the input and output are merged.

[0112] For a description of the features in the embodiment corresponding to the data access device, please refer to the relevant description in the embodiment corresponding to the data access method, which will not be repeated here.

[0113] To facilitate those skilled in the art in implementing the data access method and apparatus provided in the embodiments of this application, the data access method and apparatus will be described in detail below with reference to the pseudocode required for practical applications.

[0114] In practical applications, corresponding pseudocode can be set for each module to achieve the corresponding function.

[0115] Taking the addition operator a + b = c as an example, where a and b are inputs and c is the output. First, `broadcast_setup` (the function name corresponding to the broadcast setup module) is called to set the inputs for broadcasting, setting the stride of the broadcast in a and b to 0. Then, `analyze_layout` (the function name corresponding to the memory layout analysis module) is called to analyze the memory distribution of a, b, and c, marking the dimension type: continuous types are marked as C, straddle types as S, and broadcast types as B. Next, `reorganize_dims` (the function name corresponding to the memory layout merging and rearranging module) is called to rearrange the memory of a, b, and c, while maintaining the correspondence between elements in a, b, and c. The final memory arrangement is [continuous, straddle, broadcast]. Finally, `vectorized_func` (the function name corresponding to the operator calculation module) is called to calculate a, b, and c, performing calculations in segments according to the memory layout.

[0116] Specifically, if the segment is contiguous memory, then a vectorized approach is used to read and write data in batches, calling the vectorized function `cal_func_vectorized` (another function name corresponding to the operator calculation module) to calculate the data. If the segment is spanned memory, then the offset of each element is calculated, and the values ​​of `a` and `b` are obtained based on the offset, and the value of `a+b` is written to `c` based on the offset. If the segment is broadcast memory, assuming that the memory segment of `b` is broadcast memory, then the elements of `b` are reused, and the elements of `a` and `c` are read and written using a vectorized approach, calling the vectorized function `cal_func_vectorized` to calculate the data.

[0117] Corresponding to the above method embodiments, this application also provides an electronic device. The electronic device described below and the data access method described above can be referred to each other.

[0118] See Figure 6 As shown, the electronic device includes:

[0119] Memory 332 is used to store computer programs;

[0120] The processor 322 is used to implement the steps of the data access method in the above method embodiment when executing a computer program.

[0121] For details, please refer to Figure 7 , Figure 7This is a schematic diagram of the specific structure of an electronic device provided in this embodiment. The electronic device can vary significantly due to differences in configuration or performance. It may include one or more central processing units (CPUs) (e.g., one or more processors) and a memory 332. The memory 332 stores one or more computer programs 342 or data 344. The memory 332 can be temporary or permanent storage. The program stored in the memory 332 may include one or more modules (not shown in the diagram), each module may include a series of instruction operations on the data processing device. Furthermore, the processor 322 may be configured to communicate with the memory 332 and execute the series of instruction operations stored in the memory 332 on the electronic device 301.

[0122] Electronic device 301 may also include one or more power supplies 326, one or more wired or wireless network interfaces 350, one or more input / output interfaces 358, and / or one or more operating systems 341.

[0123] The steps in the data access method described above can be implemented by the structure of an electronic device.

[0124] Corresponding to the above method embodiments, this application also provides a readable storage medium. The readable storage medium described below corresponds to the data access method described above. This application also provides a computer-readable storage medium storing a computer program, wherein the computer program is configured to execute the steps in any of the above data access method embodiments at runtime.

[0125] In one exemplary embodiment, the aforementioned computer-readable storage medium may include, but is not limited to, various media capable of storing computer programs, such as a USB flash drive, read-only memory (ROM), random access memory (RAM), portable hard disk, magnetic disk, or optical disk.

[0126] Embodiments of this application also provide a computer program product, which includes a computer program that, when executed by a processor, implements the steps in any of the above-described data access method embodiments.

[0127] Embodiments of this application also provide another computer program product, including a non-volatile computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps in any of the above-described data access method embodiments.

[0128] Those skilled in the art will further recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0129] This document uses specific examples to illustrate the principles and implementation methods of this application. The descriptions of the embodiments above are only for the purpose of helping to understand the methods and core ideas of this application. It should be noted that those skilled in the art can make several improvements and modifications to this application without departing from the principles of this application, and these improvements and modifications also fall within the protection scope of this application.

Claims

1. A data access method, characterized in that, include: Obtain the input of the target operator, and if the input satisfies the broadcast condition, broadcast the setting of the input; Obtain the tensor memory layout features of the target operator, and use the tensor memory layout features to determine the dimension types corresponding to the input and output respectively; After performing continuous dimension merging, step dimension sorting, and broadcast dimension merging on the memory spaces corresponding to the input and output respectively, based on the dimension type, the memory spaces are reorganized in segments. The memory segment to be accessed is determined using data access instructions, and the tensor data of the target operator is accessed using a data access method adapted to the memory segment; Specifically, determining the dimension types corresponding to the input and output using tensor memory layout features includes: The memory continuity of the input is determined using the tensor memory layout features; The memory continuity is used to determine the dimension type of each dimension in the input; Wherein, the memory continuity corresponds to the step size, and the memory continuity is used to determine the dimension type of each dimension in the input, including: If the step size of the current dimension is 0, then the dimension type of the current dimension is determined to be a broadcast dimension; If the step size of the current dimension is 1, then the dimension type of the current dimension is determined to be a continuous dimension. If the step size of the current dimension is equal to the product of the size of the next dimension and the step size, then the dimension type of the current dimension is determined to be a continuous dimension. If the step size of the current dimension is neither 0 nor 1, and is not the product of the size of the next dimension and the step size, then the dimension type of the current dimension is determined to be a step dimension.

2. The method according to claim 1, characterized in that, Accessing the tensor data of the target operator using a data access method adapted to the memory segment includes: If the memory segment is contiguous, the tensor data is accessed in batches from the memory segment in a vectorized manner; If the memory segment is a spanned memory segment, the tensor data is centrally accessed from the memory segment according to the offset method; If the memory segment is broadcast memory, the tensor data is reused in a data reuse manner.

3. The method according to claim 2, characterized in that, Batch access of the tensor data from the memory segment in a vectorized manner includes: The vectorization function is invoked to retrieve the tensor data in batches from the memory segment.

4. The method according to claim 1, characterized in that, If the input satisfies the broadcast requirement, broadcast settings are applied to the input, including: If the input includes a first input and a second input, then determine whether the first input and the second input satisfy the broadcast condition; If so, the step size of the dimension corresponding to the first or second input broadcast is set to 0.

5. The method according to claim 1, characterized in that, The memory space is reorganized into segments, including: Sort the memory segments corresponding to the merged continuous dimension, the sorted step dimension, and the merged broadcast dimension according to priority. After sorting the memory segments, the memory segments are assembled.

6. The method according to any one of claims 1 to 5, characterized in that, Combining the dimension type, the memory spaces corresponding to the input and output are subjected to continuous dimension merging, step dimension sorting, and broadcast dimension merging, including: If the dimension type is a continuous dimension, and the correspondence between input and output is not changed after merging and rearranging, then the memory spaces corresponding to the input and the output are merged. If the dimension type is a step dimension, then the memory spaces corresponding to the input and the output are arranged in ascending order according to the original step size; If the dimension type is a broadcast dimension, and the input-output correspondence is not changed after merging and reorganizing, then the memory spaces corresponding to the input and the output are merged.

7. An electronic device, characterized in that, include: Memory, used to store computer programs; A processor, configured to implement the steps of the data access method as described in any one of claims 1 to 6 when executing the computer program.

8. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program, wherein the computer program, when executed by a processor, implements the steps of the data access method as described in any one of claims 1 to 6.

Images