Indexing method and device in heterogeneous computer, equipment, medium and product

Abstract

The invention relates to the technical field of heterogeneous computing, and discloses an indexing method and device in a heterogeneous computer, equipment, a medium and a product. Determining a target index tensor list based on the original index, wherein the dimensions of a plurality of tensors in the target index tensor list are the same; and calculating the original tensor based on a preset kernel function, and determining a target output tensor. According to the method, the indexes in different forms are converted into the same tensor expression format, so that the computational logic of the back-end kernel is simplified, the need of writing various back-end kernels for different index forms is avoided, the computational efficiency is further improved, unnecessary repeated computation and memory access are avoided, and the computational overhead is reduced.

Description

Technical Field

[0001] The present invention relates to the field of heterogeneous computing technology, and in particular to indexing methods, devices, equipment, media and products in heterogeneous computers. Background Art

[0002] Tensor indexing functions are indispensable tools in deep learning frameworks, allowing users to efficiently access and manipulate elements or sub-tensors in multidimensional arrays. As the basic data structure of deep learning, tensors cover arrays from scalars to vectors, matrices, and even higher dimensions. In the inference and training process of deep learning models, efficient implementation of tensor indexing functions is crucial to improving computational efficiency and ensuring computational accuracy. There are many existing indexing methods. For different indexing methods, different kernel functions are usually used for targeted calculations based on the characteristics of the calculations. Therefore, in order to implement the tensor indexing function, it is necessary to write multiple sets of calculation logic for different indexes, resulting in low efficiency. Summary of the invention

[0003] In view of this, the present invention provides an indexing method, apparatus, device, medium and product in a heterogeneous computer to solve the problem of low tensor indexing efficiency.

[0004] In a first aspect, the present invention provides an indexing method in a heterogeneous computer, the method comprising:

[0005] Get the original tensor and the original index;

[0006] Determine a target index tensor list based on the original index, wherein dimensions of multiple tensors in the target index tensor list are the same;

[0007] The original tensor is calculated based on a preset kernel function to determine a target output tensor.

[0008] In an optional implementation, determining a target index tensor list based on the original index includes:

[0009] Determine whether the original index is in the form of a preset index, and convert the original index into a first index tensor list based on the determination result;

[0010] Perform dimension alignment processing on the tensors in the first index tensor list to obtain a target index tensor list.

[0011] In an optional implementation, converting the original index into a first index tensor list based on the judgment result includes:

[0012] If the original index is not in the format of the preset index, performing type conversion on the original index;

[0013] When the type of the original index is the target type, the type-converted original index is converted into a target index tensor list.

[0014] In an optional implementation, performing dimension alignment processing on the tensors in the first index tensor list to obtain a target index tensor list includes:

[0015] Determine a first tensor with the largest dimension in the first index tensor list, and compare the dimensions of other tensors in the first index tensor list with the dimensions of the first tensor;

[0016] If there is a second tensor with a smaller dimension than the first tensor, stretch the second tensor so that the dimension of the stretched second tensor is the same as the dimension of the first tensor;

[0017] A target index tensor list is determined based on the stretched second tensor and the first tensor.

[0018] In an optional implementation, calculating the original tensor based on a preset kernel function to determine the target output tensor includes:

[0019] Collecting parameter information of the original tensor, the parameter information at least including dimension information and stride information of the parameter information;

[0020] Calculation is performed based on a preset kernel function, parameter information of the original tensor, a target index tensor list, and the original tensor to determine the target output tensor based on the original tensor.

[0021] In an optional implementation, the calculating based on a preset kernel function, parameter information of the original tensor, a target index tensor list, and the original tensor to determine the target output tensor based on the original tensor includes:

[0022] Calculate the target offset of the target element in the original tensor based on the parameter information of the original tensor;

[0023] A target element is extracted from the original tensor based on the target offset to determine the target output tensor according to the target element.

[0024] The indexing method in a heterogeneous computer provided in this embodiment includes obtaining an original tensor and an original index; determining a target index tensor list based on the original index, wherein the dimensions of multiple tensors in the target index tensor list are the same; and calculating the original tensor based on a preset kernel function to determine a target output tensor. This method converts indexes of different forms into the same tensor expression format, simplifies the calculation logic of the backend kernel, avoids the need to write multiple backend kernels for different index forms, thereby improving calculation efficiency, avoiding unnecessary repeated calculations and memory accesses, and reducing calculation overhead.

[0025] In a second aspect, the present invention provides an indexing device in a heterogeneous computer, the device comprising:

[0026] The original data acquisition module is used to obtain the original tensor and original index;

[0027] A target tensor list determination module, used to determine a target index tensor list based on the original index, wherein the dimensions of multiple tensors in the target index tensor list are the same;

[0028] The data output module is used to calculate the original tensor based on a preset kernel function to determine a target output tensor.

[0029] In a third aspect, the present invention provides a computer device, comprising: a memory and a processor, the memory and the processor being communicatively connected to each other, the memory storing computer instructions, and the processor executing the indexing method in a heterogeneous computer of the first aspect or any corresponding embodiment thereof by executing the computer instructions.

[0030] In a fourth aspect, the present invention provides a computer-readable storage medium having computer instructions stored thereon, the computer instructions being used to enable a computer to execute the indexing method in a heterogeneous computer of the first aspect or any corresponding embodiment thereof.

[0031] In a fifth aspect, the present invention provides a computer program product, comprising computer instructions for causing a computer to execute the indexing method in a heterogeneous computer according to the first aspect or any corresponding embodiment thereof. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] In order to more clearly illustrate the specific implementation methods of the present invention or the technical solutions in the prior art, the drawings required for use in the specific implementation methods or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some implementation methods of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0033] Figure 1is a flow chart of an indexing method in a heterogeneous computer according to an embodiment of the present invention;

[0034] Figure 2 is a schematic diagram of original index conversion according to an embodiment of the present invention;

[0035] Figure 3 is a schematic diagram of tensor list values ​​according to an embodiment of the present invention;

[0036] Figure 4 is a schematic diagram of the relationship between the stride and actual storage according to an embodiment of the present invention;

[0037] Figure 5 is a structural block diagram of an indexing device in a heterogeneous computer according to an embodiment of the present invention;

[0038] Figure 6 It is a schematic diagram of the hardware structure of a computer device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0039] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present invention.

[0040] Tensor indexing functions are an indispensable tool in deep learning frameworks, which enable users to efficiently access and manipulate elements or sub-tensors in multidimensional arrays (i.e., tensors). As the basic data structure of deep learning, tensors cover arrays ranging from scalars (0 dimensions) to vectors (1 dimensions), matrices (2 dimensions), and even higher dimensions. In the inference and training process of deep learning models, efficient implementation of tensor indexing functions is crucial to improving computational efficiency and ensuring computational accuracy. There are various indexing methods, mainly including integer indexing and Boolean indexing, and also support the combination of integer and Boolean indexing. In specific applications, the indexing format is also flexible and varied, such as using a single tensor for indexing, or indexing multiple tensors in a list, and using slicing and other methods for indexing. In heterogeneous computer systems, different processors have their own advantages and applicable scenarios. For operations such as tensor indexing, traditional implementation methods often require writing multiple backend kernel codes according to different indexing forms. For example, when encountering a Boolean index, a Boolean kernel function is called for calculation; when encountering an integer index, an integer kernel function is called for calculation. Therefore, it is necessary to write multiple sets of kernel functions for different index types, which not only increases the development complexity, but also may lead to low computing efficiency. Based on this, an embodiment of the present invention provides an indexing method in a heterogeneous computer.

[0041] According to an embodiment of the present invention, an embodiment of an indexing method in a heterogeneous computer is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0042] In this embodiment, an indexing method in a heterogeneous computer is provided, which can be used in terminals such as computers and tablet computers. Figure 1 is a flow chart of an indexing method in a heterogeneous computer according to an embodiment of the present invention. Figure 1 As shown, the process includes the following steps:

[0043] Step S101, obtaining the original tensor and the original index.

[0044] A raw tensor represents a multidimensional array that can store different types of data, such as scalars (0-dimensional tensors), vectors (1-dimensional tensors), matrices (2-dimensional tensors), or higher-dimensional arrays. When indexing is required, the raw tensor is obtained from the data source. The raw tensor can be image data, voice signals, text data, or any other form of multidimensional data.

[0045] A raw index can be one or more tensors containing information pointing to specific data locations in the original tensor. A raw index can be a simple integer index (for selecting a single element or a group of contiguous elements) or a more complex indexing structure (such as a Boolean mask, a slice object, or an advanced index array) for selecting non-contiguous or irregular subsets of data.

[0046] Step S102, determining a target index tensor list based on the original index.

[0047] Among them, the dimensions of multiple tensors in the target index tensor list are the same. The target index tensor list is generated according to the original index, and the tensors in the target index tensor list have the same dimension. The target index tensor list has a specific form, which is pre-set. First, it is necessary to determine whether the original index is in a specific form. If the original index is in a specific form, only the data in the original index needs to be processed to ensure that the dimensions of all processed tensors are consistent. If the original index is not in a specific form, the original index needs to be converted to a type. For example, if the original index is a Boolean index, it first needs to be converted to an integer index, and then the integer index is further converted to a specific index list format, and then the data therein is processed to ensure that the dimensions of all tensors are consistent to obtain the target index tensor list.

[0048] When manipulating the dimensions of the data to ensure that all tensors have consistent dimensions, this can be done by adding additional dimensions (e.g., using broadcasting) or re-arranging existing dimensions when necessary.

[0049] Step S103, calculating the original tensor based on a preset kernel function to determine a target output tensor.

[0050] A preset kernel function is a predefined function that accepts input tensors and index tensors as parameters and returns an output tensor. The preset kernel function can be implemented on a high-performance computing platform. Call the preset kernel function and pass the original tensor and the target index tensor list as input to the preset kernel function. The preset kernel function extracts data from the original tensor according to the position specified in the target index tensor list and stores the data in the target output tensor. Optionally, after extracting the data, you can perform the specified calculation first according to the requirements, and then store the calculation results in the target output tensor.

[0051] The target output tensor is the result of the kernel function calculation. It can be a new tensor with the same dimensions as the original tensor, or a tensor with different dimensions or data types. The target output tensor can be used for subsequent data processing, analysis, or visualization tasks.

[0052] In some optional embodiments, taking feature extraction in image processing as an actual application scenario as an example, the indexing method in a heterogeneous computer provided by the present method includes: obtaining the original tensor and the original index; determining a target index tensor list based on the original index; calculating the original tensor based on a preset kernel function to determine the target output tensor.

[0053] In image processing, a raw tensor can be a matrix of pixel values ​​of one or more images. For example, if a grayscale image is being processed, the raw tensor is a two-dimensional matrix in which each element represents the grayscale value of a pixel. If a color image (such as an RGB image) is being processed, the raw tensor is a three-dimensional matrix in which the third dimension represents the color channel (red, green, and blue). The raw index can represent the location information of a region of interest (ROI) in the image. For example, for an image of size M×N, the raw index can be the upper left and lower right coordinates of one or more two-dimensional coordinate points or rectangular regions, which are used to specify the image portion that requires special attention. The target index tensor list is determined based on the raw index, and the raw index needs to be converted in form and type to obtain the target index tensor list. In image processing, a preset kernel function is used to perform specific image processing operations, such as filtering, edge detection, blurring, etc. The image is processed by the preset kernel function to obtain a series of processed tensors, namely the target output tensor, which can be used to characterize the results of the entire image or image set after specific processing.

[0054] The indexing method in a heterogeneous computer provided in this embodiment includes obtaining an original tensor and an original index; determining a target index tensor list based on the original index, wherein the dimensions of multiple tensors in the target index tensor list are the same; and calculating the original tensor based on a preset kernel function to determine a target output tensor. This method converts indexes of different forms into the same tensor expression format, simplifies the calculation logic of the backend kernel, avoids the need to write multiple backend kernels for different index forms, thereby improving calculation efficiency, avoiding unnecessary repeated calculations and memory accesses, and reducing calculation overhead.

[0055] In this embodiment, an indexing method in a heterogeneous computer is provided, which can be used in terminals such as computers and tablet computers. The method includes the following steps:

[0056] Step S201, obtaining the original tensor and the original index.

[0057] For details, please see Figure 1 Step S101 of the illustrated embodiment will not be described in detail here.

[0058] Step S202, determining a target index tensor list based on the original index.

[0059] Specifically, step S202 includes:

[0060] Step S2021, determine whether the original index is in the form of a preset index, and convert the original index into a first index tensor list based on the determination result.

[0061] The original index may exist in various forms, specifically, it may be a Boolean index, an integer index, an array index, a slice index, etc. It is determined whether the original index conforms to a preset index form, and the preset form may be a specific data structure or format.

[0062] If the original index conforms to the preset format, no additional conversion is required. If the original index does not conform to the preset format, it needs to be converted to a series of index tensors, which constitute the first index tensor list. If the original index contains multiple types, each type can be processed separately and converted to the corresponding first index tensor list.

[0063] In some optional embodiments, step S2021 includes: if the original index is not in the form of a preset index, performing type conversion on the original index; when the type of the original index is the target type, converting the original index after type conversion into a first index tensor list.

[0064] If the original index is not in the form of a preset index, you need to perform type conversion on the original index. Type conversion involves converting the original index from one data type to another data type.

[0065] The original index can contain complex structures or multiple types of indexes, in which case type conversion may involve parsing the original index, creating a new data structure to store the converted index, and handling any possible exceptions or errors.

[0066] After the original index is type-converted, check whether the converted index meets the requirements of the target type, which is a specific tensor data type. If the converted original index meets the requirements of the target type, it is converted to the first index tensor list, which can contain multiple index tensors.

[0067] For example, Figure 2 As shown, assume that the original tensor a is a 2x3 tensor: tensor([[8, 6, 3], [6, 0, 9]]). If the original index is of Boolean type, it is necessary to first convert the Boolean type to an integer index, and then further convert the index after type conversion to a first index tensor list, where the first index tensor list has a preset index format.

[0068] For example, if the Boolean index is:

[0069] tensor([[True, True, False], [True, False, True]]),

[0070] Converting it to integer form, we get:

[0071] tensor([[0,0],[0,1],[1,0],[1,2]]),

[0072] Further convert it into the form of preset indexes to get the first index tensor list:

[0073] tensor([0, 0, 1, 1]), tensor([0, 1, 0, 2]).

[0074] Step S2022, perform dimension alignment processing on the tensors in the first index tensor list to obtain a target index tensor list.

[0075] Since the shapes of multiple tensors in the first index tensor list may be different, it is necessary to perform dimension alignment on the tensors therein to obtain the target index tensor list. The dimensions of the tensors in the target index tensor list are consistent, and the dimensions of each tensor can be adjusted through broadcasting.

[0076] In some optional embodiments, step S2022 includes: determining the first tensor with the largest dimension in the first index tensor list, and comparing the dimensions of other tensors in the first index tensor list with the dimensions of the first tensor; if there is a second tensor with a smaller dimension than the first tensor, stretching the second tensor, and the dimension of the stretched second tensor is the same as the dimension of the first tensor; determining the target index tensor list based on the stretched second tensor and the first tensor.

[0077] In this embodiment, the dimension alignment process is performed in accordance with the largest shape size, specifically starting from the rightmost dimension (i.e., the last dimension) for the dimensions of the two tensors. If a dimension has the same size in the two tensors, continue to align the next dimension. If a dimension has a size of 1 in one tensor, the tensor is stretched in that dimension to match the size of the other tensor. After the index tensor list is broadcasted, the number of index tensors is padded according to the number of dimensions of the original tensor a, so that the number of index tensors is aligned with the number of dimensions of the original tensor.

[0078] For example, the three indexes in the first index tensor list are:

[0079] [0, 1, 0, 1], [1], [1, 0, 1, 0]

[0080] Among them, the first tensor with the largest dimension is [0, 1, 0, 1] and [1, 0, 1, 0], and the second tensor is [1]. It can be seen that the shape of the second tensor is different from the other two tensors. Therefore, broadcast processing is performed to stretch the second tensor. After stretching, the second tensor is: [1, 1, 1, 1].

[0081] After processing, reset the stride information of the original tensor, set all strides on the indexed dimension to 0, and set the length of this dimension to the same as the output. This step is to ignore the values ​​of these dimensions when calculating the actual bias in the backend, and then calculate the bias of the index based on the value in the index tensor. The main purpose is to simplify the subsequent kernel code.

[0082] Step S203, calculating the original tensor based on a preset kernel function to determine a target output tensor.

[0083] Specifically, step S203 includes:

[0084] Step S2031, collecting parameter information of the original tensor.

[0085] The parameter information at least includes dimension information and stride information of the parameter information.

[0086] In order to simplify the calculation logic of the kernel code, the parameter information of the original tensor can be modified during preprocessing, which may include dimension information, stride information, etc. Therefore, it is necessary to store these parameter information in a separate storage unit in advance for use in subsequent calculations. Before the dimension (i.e., shape) and stride information of the original tensor are modified, the collected parameter information is stored, specifically in a preset container, which is used to store the original parameter information. Among them, the dimension information defines the size of the tensor, and the stride information determines the interval between elements when traversing the tensor.

[0087] Step S2032, performing calculation based on a preset kernel function, parameter information of the original tensor, a target index tensor list, and the original tensor to determine a target output tensor based on the original tensor.

[0088] First, the preset kernel function, parameter information of the original tensor, target index tensor list, and original tensor data are passed to the backend to call the preset kernel function of the asynchronous accelerator for calculation. During the calculation process, each (or certain specific) position of the original tensor is traversed, the kernel function is applied to the local data of the position, and the value of the corresponding position of the output tensor is calculated according to the kernel function and stride information. The final output result is the target output tensor processed by the preset kernel function, which is obtained after data extraction of the original tensor.

[0089] In some optional embodiments, step S2032 includes: calculating a target offset of the target element in the original tensor based on parameter information of the original tensor; and extracting the target element from the original tensor based on the target offset to determine a target output tensor according to the target element.

[0090] The offset refers to the relative distance from the starting position of the original tensor to the position of the target element. The target element is the data to be extracted. First, the size of each element of the input data x can be calculated. This step is used to understand the overall layout of the tensor and the number of elements in each dimension. The offset of the output y and input x is calculated according to the parameter information. The logical index is converted to the actual offset in the memory using the dimension information and stride information. The target offset is obtained by iterating each dimension of the tensor and using modulo operations and division to calculate the offset in each dimension.

[0091] For tensors of the same shape, the actual storage location in memory is related to the stride. For example, for a tensor of shape 2x3x4, if the stride is (12, 4, 1), the tensor is stored continuously, and the logical index number and the offset in memory are the same. If the stride is (24, 8, 1), the tensor is stored discontinuously, and the logical index number and the offset in memory are different.

[0092] Based on the calculated target offset, the value is assigned from the original tensor to the target output tensor. First, the target index tensor list can be looped through to fill the arrays corresponding to the dimension, stride, and data pointer. Calculate the data pointer out_ptr of the output tensor and the data pointer in_ptr of the input tensor. The data pointer of the output tensor points to the memory location where the output data is stored. The data pointer of the input tensor points to the memory location where the input data is stored. By comparing the dimension sizes of the first index tensor in the target index tensor list and the output tensor, the stride of the index tensor is adjusted (if the dimension sizes do not match, the corresponding stride is set to 0). Each element is processed in parallel through auxiliary functions (for example, auxiliary functions used to start CUDA kernel functions). CUDA kernel functions are functions used for parallel execution on graphics processing units. Inside the kernel function: calculate the output offset y_offset and the input tensor offset x_offset, and adjust the input tensor offset x_offset according to the value of the index tensor. The offset of the output tensor is used to determine where the current thread should write the output tensor, and the offset of the input tensor is used to determine where the current thread should read data from the input tensor. Finally, the preset function f is called to process the data. The preset function f can be provided by the user, and the target output tensor is finally calculated.

[0093] like Figure 3 As shown in the figure, taking the original tensor as tensor([[8, 6, 3], [6, 0, 9]]), the target index tensor list as tensor([0, 0, 1, 1]), tensor([0, 1, 0, 2]) as an example, the target output tensor is tensor([8, 6, 6, 9]).

[0094] For example, suppose the shape (dimension) of the original tensor a is: (2, 3, 4), stride (step): (12, 4, 1), see Figure 4 Schematic diagram of the relationship between stride and actual storage.

[0095] tensor([[[0.3259,-1.8271,0.0630,-0.1644],

[0096] [0.0358,-2.1713,0.1783,0.5325],

[0097] [0.3867,0.5847,0.5775,-0.1356]],

[0098] [[1.0928,-0.6469,1.3403,0.3939],

[0099] [1.0481,-0.9301,-0.3197,1.2006],

[0100] [-0.0222,-0.6422,1.1927,-0.6101]]])

[0101] The index tensor list index: [tensor([0,1,0,1])], indexed_sizes[2], indexed_strides

[12] . a[index] is taken like this, because index has only one dimension, so it is taken from the 0th dimension, and the output shape is [4,3,4]. In the dimension that is not indexed, its shape corresponds to the original tensor one-to-one. In this example, the first and second dimensions are 3 and 4 respectively.

[0102] The output has 4*3*4=48 elements. Taking the 20th element as an example, its index in the output is [1,2,0]. When calculating the actual position of an element in memory, the bias calculation method described below should be used.

[0103] When calculating the memory position offset in the 1st and 2nd dimensions, since no indexing is performed, the calculation can be done directly according to the stride of the original tensor in the 1st and 2nd dimensions. Since the indexing is performed on the 0th dimension, the value of the 0th dimension is calculated according to the index value * the stride of the 0th dimension. Therefore, the shape and stride of the 0th dimension are saved in indexed_sizes and indexed_strides, and then the stride of the 0th dimension of the original tensor is set to 0. The specific algorithm is as follows:

[0104] int offset=0

[0105] for(int i=dim-1; i>=0; --i){

[0106] offset+=shape[i]*stride[i]

[0107] }

[0108] This will calculate the biases except for the 0th dimension first.

[0109] Then the bias on the index dimension is calculated.

[0110] for(int i=0; i <num_indices;++i){

[0111] offset+=index*indexed_strides[i]

[0112] }

[0113] As an example, when the original tensor is (2, 3, 4), the indices on the three dimensions in the original index are [0, 1, 0, 1], [1], [1, 0, 1, 0], and the shape of the second index [1] is different from the other two, so it is broadcasted to [1, 1, 1, 1], and the target index tensor list is [0, 1, 0, 1], [1, 1, 1, 1], [1, 0, 1, 0]. The values ​​​​taken from the original tensor are as follows:

[0114] a[0, 1, 1]: -2.1713,

[0115] a[1, 1, 0]: 1.0481,

[0116] a[0, 1, 1]: -2.1713,

[0117] a[1, 1, 0]:1.0481.

[0118] That is, the target output tensor is: -2.1713, 1.0481, -2.1713, 1.0481.

[0119] The indexing method in a heterogeneous computer provided by an embodiment of the present invention converts different forms of indexes (including Boolean indexes, integer indexes, array indexes, slice indexes, etc.) into a unified tensor expression format. This conversion method enables the subsequent calculation process to uniformly process different types of indexes, avoiding the need to write multiple backend kernels for different index forms. Broadcast processing is performed on multiple indexes in the index tensor list so that they have the same shape, and the number of index tensors is completed to ensure that the number of index tensors is aligned with the number of dimensions of the original tensor. At the same time, the shape and stride information of the original tensor are modified, and the length and stride information on the indexed dimension are saved in advance for use in subsequent calculations. This method simplifies the calculation logic of the subsequent kernel code and improves the calculation efficiency. Before the shape and stride information of the original tensor are modified, the length and stride information on the indexed dimension are collected and stored in a separate storage unit. This module provides the necessary information for subsequent calculations and ensures the accuracy of the calculation. Implementation of heterogeneous accelerator kernel operators: After preprocessing is completed, the preprocessed data is passed to the backend, and the kernel function of the asynchronous accelerator is called for calculation. This kernel function adjusts the offset of the input tensor according to the value of the index tensor and calls the function provided by the user to process the data. This implementation fully utilizes the computing power of heterogeneous accelerators and improves data processing efficiency. The overall solution realizes efficient calculation of advanced indexing operators on heterogeneous computer devices, with significant advantages of improved computing efficiency and code simplicity. This solution not only reduces development complexity, but also improves the flexibility and efficiency of tensor indexing operations in deep learning frameworks.

[0120] In this embodiment, an indexing device in a heterogeneous computer is also provided, which is used to implement the above-mentioned embodiments and preferred implementation modes, and the descriptions that have been made will not be repeated. As used below, the term "module" can be a combination of software and / or hardware that implements a predetermined function. Although the devices described in the following embodiments are preferably implemented in software, the implementation of hardware, or a combination of software and hardware, is also possible and conceivable.

[0121] This embodiment provides an indexing device in a heterogeneous computer, such as Figure 5 As shown, including:

[0122] The original data acquisition module 301 is used to obtain the original tensor and the original index;

[0123] A target tensor list determining module 302 is used to determine a target index tensor list based on the original index, wherein the dimensions of multiple tensors in the target index tensor list are the same;

[0124] The data output module 303 is used to calculate the original tensor based on a preset kernel function to determine a target output tensor.

[0125] In some optional implementations, the target tensor list determination module 302 includes:

[0126] A tensor list conversion unit, configured to determine whether the original index is in the form of a preset index, and convert the original index into a first index tensor list based on the determination result;

[0127] The target tensor list determining unit is used to perform dimension alignment processing on the tensors in the first index tensor list to obtain a target index tensor list.

[0128] In some optional implementations, the tensor list conversion unit includes:

[0129] a type conversion subunit, configured to perform type conversion on the original index if the original index is not in the form of the preset index;

[0130] The index conversion subunit is used to convert the original index after type conversion into a target index tensor list when the type of the original index is the target type.

[0131] In some optional implementations, the target tensor list determining unit includes:

[0132] A tensor dimension comparison subunit, used to determine a first tensor with the largest dimension in the first index tensor list, and compare the dimensions of other tensors in the first index tensor list with the dimensions of the first tensor;

[0133] A tensor stretching subunit, configured to stretch a second tensor if there is a second tensor with a smaller dimension than the first tensor, so that the dimension of the stretched second tensor is the same as the dimension of the first tensor;

[0134] The tensor list determination subunit is used to determine a target index tensor list based on the stretched second tensor and the first tensor.

[0135] In some optional implementations, the data output module 303 includes:

[0136] A parameter information acquisition unit, used to acquire parameter information of the original tensor, wherein the parameter information at least includes dimension information and stride information of the parameter information;

[0137] A tensor output unit is used to perform calculations based on a preset kernel function, parameter information of the original tensor, a target index tensor list and the original tensor to determine the target output tensor based on the original tensor.

[0138] In some optional embodiments, the tensor output unit includes:

[0139] A target offset calculation subunit, used to calculate a target offset of a target element in the original tensor based on parameter information of the original tensor;

[0140] The target output tensor determination subunit is used to extract a target element from the original tensor based on the target offset to determine the target output tensor according to the target element.

[0141] The further functional description of each of the above modules and units is the same as that of the above corresponding embodiments and will not be repeated here.

[0142] The indexing device in the heterogeneous computer in this embodiment is presented in the form of a functional unit, where the unit refers to an ASIC (Application Specific Integrated Circuit) circuit, a processor and memory that executes one or more software or fixed programs, and / or other devices that can provide the above functions.

[0143] An embodiment of the present invention further provides a computer device having an indexing device in a heterogeneous computer.

[0144] See also Figure 6 , Figure 6 is a schematic diagram of the structure of a computer device provided by an optional embodiment of the present invention, such as Figure 6 As shown, the computer device includes: one or more processors 10, a memory 20, and interfaces for connecting various components, including high-speed interfaces and low-speed interfaces. Various components are connected to each other using different buses for communication, and can be installed on a common mainboard or installed in other ways as needed. The processor can process the instructions executed in the computer device, including instructions stored in or on the memory to display the graphical information of the GUI on an external input / output device (such as, a display device coupled to the interface). In some optional embodiments, if necessary, multiple processors and / or multiple buses can be used together with multiple memories and multiple memories. Similarly, multiple computer devices can be connected, and each device provides some necessary operations (for example, as a server array, a group of blade servers, or a multi-processor system). Figure 6 A processor 10 is taken as an example.

[0145] The processor 10 may be a central processing unit, a network processor or a combination thereof. The processor 10 may further include a hardware chip. The hardware chip may be a dedicated integrated circuit, a programmable logic device or a combination thereof. The programmable logic device may be a complex programmable logic device, a field programmable gate array, a general purpose array logic or any combination thereof.

[0146] The memory 20 stores instructions executable by at least one processor 10, so that the at least one processor 10 executes the method shown in the above embodiment.

[0147] The memory 20 may include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function; the data storage area may store data created according to the use of the computer device, etc. In addition, the memory 20 may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some optional embodiments, the memory 20 may optionally include a memory remotely arranged relative to the processor 10, and these remote memories may be connected to the computer device via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0148] The memory 20 may include a volatile memory, such as a random access memory; the memory may also include a non-volatile memory, such as a flash memory, a hard disk or a solid state drive; the memory 20 may also include a combination of the above types of memory.

[0149] The computer device further comprises a communication interface 30 for the computer device to communicate with other devices or a communication network.

[0150] The embodiment of the present invention also provides a computer-readable storage medium. The method according to the embodiment of the present invention can be implemented in hardware, firmware, or can be implemented as a computer code that can be recorded in a storage medium, or can be implemented as a computer code that is originally stored in a remote storage medium or a non-temporary machine-readable storage medium and will be stored in a local storage medium through a network download, so that the method described herein can be stored in such software processing on a storage medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware. Among them, the storage medium can be a magnetic disk, an optical disk, a read-only storage memory, a random access memory, a flash memory, a hard disk or a solid-state hard disk, etc.; further, the storage medium can also include a combination of the above types of memories. It can be understood that a computer, a processor, a microprocessor controller, or programmable hardware includes a storage component that can store or receive software or computer code. When the software or computer code is accessed and executed by a computer, a processor, or hardware, the method shown in the above embodiment is implemented.

[0151] A part of the present invention may be applied as a computer program product, such as a computer program instruction, which, when executed by a computer, can call or provide the method and / or technical solution according to the present invention through the operation of the computer. Those skilled in the art should understand that the existence of the computer program instruction in a computer-readable medium includes, but is not limited to, a source file, an executable file, an installation package file, etc., and accordingly, the way in which the computer program instruction is executed by the computer includes, but is not limited to: the computer directly executes the instruction, or the computer compiles the instruction and then executes the corresponding compiled program, or the computer reads and executes the instruction, or the computer reads and installs the instruction and then executes the corresponding installed program. Here, the computer-readable medium may be any available computer-readable storage medium or communication medium accessible to the computer.

[0152] Although the embodiments of the present invention have been described in conjunction with the accompanying drawings, those skilled in the art may make various modifications and variations without departing from the spirit and scope of the present invention. Such modifications and variations are all within the scope defined by the present invention.

Claims

1. An indexing method in a heterogeneous computer, characterized in that: The method comprises: Get the original tensor and the original index; Determine a target index tensor list based on the original index, wherein dimensions of multiple tensors in the target index tensor list are the same; The original tensor is calculated based on a preset kernel function to determine a target output tensor.

2. The method according to claim 1, characterized in that The determining a target index tensor list based on the original index includes: Determine whether the original index is in the form of a preset index, and convert the original index into a first index tensor list based on the determination result; Perform dimension alignment processing on the tensors in the first index tensor list to obtain a target index tensor list.

3. The method according to claim 2, characterized in that The converting the original index into a first index tensor list based on the judgment result includes: If the original index is not in the format of the preset index, performing type conversion on the original index; When the type of the original index is the target type, the type-converted original index is converted into a target index tensor list.

4. The method according to claim 2, characterized in that: The step of performing dimension alignment processing on the tensors in the first index tensor list to obtain a target index tensor list includes: Determine a first tensor with the largest dimension in the first index tensor list, and compare the dimensions of other tensors in the first index tensor list with the dimensions of the first tensor; If there is a second tensor with a smaller dimension than the first tensor, stretch the second tensor so that the dimension of the stretched second tensor is the same as the dimension of the first tensor; A target index tensor list is determined based on the stretched second tensor and the first tensor.

5. The method according to claim 1, characterized in that The calculating the original tensor based on the preset kernel function to determine the target output tensor includes: Collecting parameter information of the original tensor, the parameter information at least including dimension information and stride information of the parameter information; Calculation is performed based on a preset kernel function, parameter information of the original tensor, a target index tensor list, and the original tensor to determine the target output tensor based on the original tensor.

6. The method according to claim 5, characterized in that The calculating based on the preset kernel function, the parameter information of the original tensor, the target index tensor list and the original tensor to determine the target output tensor based on the original tensor includes: Calculate the target offset of the target element in the original tensor based on the parameter information of the original tensor; A target element is extracted from the original tensor based on the target offset to determine the target output tensor according to the target element.

7. An indexing device in a heterogeneous computer, characterized in that: The device comprises: The original data acquisition module is used to obtain the original tensor and original index; A target tensor list determination module, used to determine a target index tensor list based on the original index, wherein the dimensions of multiple tensors in the target index tensor list are the same; The data output module is used to calculate the original tensor based on a preset kernel function to determine a target output tensor.

8. A computer device, characterized in that: include: A memory and a processor, wherein the memory and the processor are communicatively connected to each other, the memory stores computer instructions, and the processor executes the indexing method in a heterogeneous computer according to any one of claims 1 to 6 by executing the computer instructions.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a computer to execute the indexing method in a heterogeneous computer according to any one of claims 1 to 6.

10. A computer program product, characterized in that The method comprises computer instructions, wherein the computer instructions are used to cause a computer to execute the indexing method in a heterogeneous computer according to any one of claims 1 to 6.

Images