Index tensor vectorization calculation method and related product

Abstract

The invention discloses an index tensor vectorization calculation method and a related product. The method comprises the following steps: acquiring the length of a first index tensor of a first sparse tensor and the length of a second index tensor of a second sparse tensor; based on the length of the first index tensor, obtaining at least one first target instruction corresponding to the first index tensor from an instruction set; obtaining at least one second target instruction corresponding to the second index tensor from the instruction set based on the length of the second index tensor; and executing the at least one first target instruction and the at least one second target instruction to perform vectorization calculation on the first index tensor and the second index tensor to obtain a third index tensor. According to the invention, the computing resources of the computing units can be saved and the splicing efficiency of the sparse tensor can be improved.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence technology, and in particular to an index tensor vectorization calculation method and related products. Background Art

[0002] The Sparse Concat operator is one of the common operators in sparse models. In the embedding table lookup fusion optimization, the Sparse Concat operator needs to concatenate hundreds or thousands of sparse tensors from the upstream. It is an important part of converting discrete features into dense matrices and affects the reasoning performance of the entire model.

[0003] The main function of the Sparse Concat operator is to concatenate a set of Sparse Tensors. In the embedding fusion query optimization, Sparse Concat is usually used to concatenate different discrete feature Sparse Tensors in the first dimension, that is, to concatenate Sparse Tensors of different shapes in the batch size direction to increase the number of features in one embedding query. However, when concatenating Sparse Tensors of different shapes, the computing unit performs bit-by-bit addition operations on the index tensor of the Sparse Tensor. This method has a relatively low computational efficiency and wastes the computing unit's computing resources. Summary of the invention

[0004] The embodiments of the present application provide an index tensor vectorized calculation method and related products. Through vectorized calculation, the index tensors of sparse tensors are spliced, which reduces the number of operations, reduces the waste of hardware computing resources, and improves the splicing efficiency.

[0005] In a first aspect, an embodiment of the present application provides an index tensor vectorization calculation method, including:

[0006] Get the length of the first index tensor of the first sparse tensor and the length of the second index tensor of the second sparse tensor;

[0007] Based on the length of the first index tensor, obtaining at least one first target instruction corresponding to the first index tensor from an instruction set;

[0008] Based on the length of the second index tensor, obtaining at least one second target instruction corresponding to the second index tensor from the instruction set;

[0009] Execute the at least one first target instruction and the at least one second target instruction to perform vectorized calculations on the first index tensor and the second index tensor to obtain a third index tensor.

[0010] In a second aspect, an embodiment of the present application provides a computing device, including: an acquisition module and a processing module;

[0011] The acquisition module is used to acquire a first index tensor of the first sparse tensor and a second index tensor of the second sparse tensor;

[0012] The processing module is used to obtain the length of the first index tensor of the first sparse tensor and the length of the second index tensor of the second sparse tensor;

[0013] Based on the length of the first index tensor, obtaining at least one first target instruction corresponding to the first index tensor from an instruction set;

[0014] Based on the length of the second index tensor, obtaining at least one second target instruction corresponding to the second index tensor from the instruction set;

[0015] Execute the at least one first target instruction and the at least one second target instruction to perform vectorized calculations on the first index tensor and the second index tensor to obtain a third index tensor.

[0016] In a third aspect, an embodiment of the present application provides an electronic device, comprising: a processor and a memory, wherein the processor is connected to the memory, the memory is used to store a computer program, and the processor is used to execute the computer program stored in the memory, so that the electronic device performs the method described in the first aspect.

[0017] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and the computer program enables a computer to execute the method described in the first aspect.

[0018] In a fifth aspect, an embodiment of the present application provides a computer program product, which includes a non-transitory computer-readable storage medium storing a computer program, and the computer is operable to cause the computer to execute the method described in the first aspect.

[0019] Implementing the embodiments of the present application has the following beneficial effects:

[0020] It can be seen that in an embodiment of the present application, when the index tensors of different sparse tensors are spliced, the instructions corresponding to the index tensor of each sparse tensor are respectively obtained, and then the instructions corresponding to the index tensor of each sparse tensor are used to perform vectorized calculations on the index tensor of each sparse tensor, so that multiple elements in the index tensor can be added in parallel at the same time, thereby reducing the number of addition operations in the process of splicing the index tensors, thereby reducing the demand for computing resources of the computing unit, and improving the splicing efficiency of the index tensors, thereby improving the splicing efficiency of the sparse tensors, reducing the waiting time of threads that need to use the spliced ​​sparse tensors, and improving the efficiency of subsequent model reasoning. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0022] Figure 1 A schematic diagram of a sparse tensor provided in an embodiment of the present application;

[0023] Figure 2 A schematic diagram of splicing sparse tensors provided in an embodiment of the present application;

[0024] Figure 3 A schematic diagram of performing an addition operation on an index tensor provided in an embodiment of the present application;

[0025] Figure 4 A schematic diagram of a computing device provided in an embodiment of the present application;

[0026] Figure 5 A schematic diagram of a flow chart of an index tensor vectorization calculation method provided in an embodiment of the present application;

[0027] Figure 6 A schematic diagram of adjusting the storage type of an index tensor provided in an embodiment of the present application;

[0028] Figure 7 A schematic diagram of performing an addition operation on an index tensor based on a vector provided in an embodiment of the present application;

[0029] Figure 8 A schematic diagram of a computing device provided in an embodiment of the present application;

[0030] Fig. 9 A schematic diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0031] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0032] The terms "first", "second", "third" and "fourth" etc. in the specification and claims of the present application and the drawings are used to distinguish different objects, rather than to describe a specific order. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device that includes a series of steps or units is not limited to the listed steps or units, but optionally includes steps or units that are not listed, or optionally includes other steps or units inherent to these processes, methods, products or devices.

[0033] Reference to "embodiments" herein means that a particular feature, result, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present application. The appearance of the phrase in various locations in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0034] In order to facilitate understanding of the technical solution of the present application, the relevant technologies designed in the present application are first explained and illustrated.

[0035] Tensor: A multidimensional array of uniform type, the purpose of which is to push vectors and matrices to higher dimensions. A 0-dimensional tensor can be regarded as a scalar, a 1-dimensional tensor can be regarded as a vector, and a 2-dimensional tensor can be regarded as a matrix.

[0036] Sparse Tensor: Also known as sparse vector or discrete vector, a Sparse Tensor can be represented by three sub-Tensors, which are called index tensor (Indices), value tensor (Values) and shape tensor (Dense_Shape). Among them, Indices represents the index of the position of all non-zero elements in the Sparse Tensor, that is, it is used to indicate the rows and columns of non-zero elements in the Sparse Tensor; Values ​​represents the values ​​of all non-zero elements in the Sparse Tensor; Dense_Shape is used to represent the shape of the Sparse Tensor.

[0037] Sparse Concat operator: The Sparse Concat operator is mainly used to implement the Sparse Concat operation. The Sparse Concat operation concatenates multiple Sparse Tensors together to form a larger Sparse Tensor.

[0038] Single Instruction Multiple Data (SIMD): One instruction is used to perform the same operation on each piece of data in a group of data at the same time, thereby achieving spatial parallel computing.

[0039] Single Instruction Single Data (SISD): Executes a single instruction and performs operations on a single data at a time.

[0040] The following will illustrate the Sparse Tensor and Sparse Concat operations with examples in conjunction with the accompanying drawings.

[0041] like Figure 1 The Sparse Tensor B shown is Then Sparse Tensor B can be obtained through Indices B, that is Values ​​B, i.e. [1 3 1 7], and Dense_Shape B, i.e. [4 4] are represented.

[0042] Furthermore, in deep learning systems, the discrete features or non-numerical features of samples need to be converted into corresponding embedding vectors. However, there will be a large amount of zero-value data in the samples under discrete features. Therefore, a Sparse Tensor can be used to represent the behavior of multiple samples under multiple discrete features, that is, multiple samples in a batch size can be represented by a Sparse Tensor under multiple discrete features.

[0043] Exemplarily, each row in the Sparse Tensor represents a discrete feature of a sample, and the elements in each row are used to characterize the behavior of the sample for each discrete feature. Therefore, when using the Sparse Tensor to characterize the behavior of the sample under discrete features, the first dimension of the Dense_Shape of the Sparse Tensor is the number of samples contained in the Sparse Tensor, that is, the batchsize corresponding to the Sparse Tensor, and the second dimension of the Dense_Shape of the Sparse Tensor characterizes the number of discrete features. Therefore, in addition to characterizing the index of the position of the non-zero element, the first dimension of each coordinate in Indices also characterizes the index of the sample in the batchsize, and the second dimension characterizes the position index index of the discrete feature in the embedding table, and the non-zero value corresponding to each coordinate in Indices characterizes the behavior of the sample under the discrete feature corresponding to the second dimension of the coordinate.

[0044] It should be noted that since this application is mainly aimed at discrete features in the recommendation system, the samples involved in this application can be understood as users, and the discrete features of the samples can be understood as discrete features of the users. The behavior of the samples on the discrete features is the behavior of the user under the discrete features and the interaction behavior between the user and the discrete features.

[0045] For example, if the discrete feature is a video, and the user's interactive behavior under the discrete feature is the number of times the user views the video, Figure 1 Taking the Sparse Tensor B shown as an example, the first dimension of Dense_Shape B represents four users, and the second dimension represents four videos; the first coordinate in Indices B represents that the element in the first row and second column of SparseTensor B is not zero. Since the first row also indicates the first user among the four users, and the second column indicates the second video, it can be seen from the first coordinate in Indices B that the first user has browsed the second video, and from the first value of Values ​​B, it can be seen that the number of times the first user browsed the second video is 1; similarly, based on Indices B and Values ​​B, it can be seen that the first user has also browsed the second video, and the number of views is 2, the second user has browsed the first video, and the number of views is 1, the third user has not browsed the four videos, and the fourth user has browsed the third video, and the number of views is 7, so that the number of views of the four videos by the four users can be represented by one Sparse Tensor.

[0046] Furthermore, for each discrete feature, an embedding table corresponding to the discrete feature is constructed in advance. Each row in the embedding table is an embedding vector of a discrete feature. In this way, when using Sparse Tensor to represent the user's discrete features, the user's behavior on each discrete feature can be quickly obtained based on Sparse Tensor. Then, for discrete features with data, the embedding vector of the discrete feature can be queried from the embedding table based on the second dimension in Indices. In this way, model inference can be performed based on the discrete feature vector and the user's behavior on the discrete feature.

[0047] Furthermore, in order to speed up the reasoning speed of the model, the Sparse Concat operator is generally used to concatenate multiple Sparse Tensors together to form a larger Sparse Tensor. In this way, when optimizing the query merge of the embedding table, the number of feature vectors queried at a time can be increased, thereby improving the reasoning speed of the model.

[0048] Still Figure 1 For example, if the model reasoning is to recommend videos to users, then Figure 1 After obtaining the discrete features that interact with each user from the SparseTensor shown, the feature vector corresponding to the discrete feature that interacts with each user is queried from the embedding table based on the position index of the discrete feature that interacts with each user, and the number of interactions between the user and the discrete feature is used as a weight to perform operations on the feature vector of the discrete feature to obtain the user's preference, so that videos can be recommended to the user. After splicing multiple Sparse Tensors, the number of interactions between a large number of users and discrete features can be obtained at one time through the spliced ​​Sparse Tensors, and more feature vectors of discrete features that interact can be queried from the embedding table at one time, thereby increasing the number of feature vectors queried at one time, so that the preferences of multiple users can be calculated at one time, thereby improving the reasoning speed of the model and the efficiency of recommendation to users.

[0049] However, when the Sparse Concat operator is used to concatenate multiple Sparse Tensors, the computational effort is mainly concentrated on concatenating the Indices of different Sparse Tensors. Therefore, this application mainly introduces how to concatenate the Indices of Sparse Tensors, and uses the following example: Figure 2The Sparse Tensor A and Sparse Tensor B are shown as examples to illustrate the process of concatenating Indices A of Sparse Tensor A and Indices B of Sparse Tensor B.

[0050] For example, Sparse Tensor A and Sparse Tensor B are concatenated on the first dimension of Indices, that is, Sparse Tensor A and Sparse Tensor B are concatenated on the batchsize dimension, and Sparse Tensor B is concatenated to the bottom of Sparse Tensor A, so as to obtain Figure 2 The Sparse Tensor C shown, and the Indices C of Sparse Tensor C are obtained.

[0051] Specifically, based on the concatenation direction of Sparse Tensor A and Sparse Tensor B, it can be determined that after Sparse Tensor C is concatenated, the rows and columns of the data in Sparse Tensor A do not change in Sparse Tensor C, so the self-increment part of Indices A is 0. The rows of the data in Sparse Tensor B change, but the columns do not change. Therefore, the first dimension self-increment part of Indices B is the batchsize (4) corresponding to Sparse Tensor A, and the second dimension self-increment part is 0. Then, after determining the self-increment part of Indices A and Indices B, you can follow Figure 3 The arrows shown in the figure perform an addition operation on each element in Indices A and Indices B in turn, and we can get Figure 3 Indices C is shown. It can be seen that a total of 14 addition operations are performed in the process of concatenating Indices A and Indices B.

[0052] It should be understood that although the data in Indices A and Indices B are coordinates, writing the coordinates into memory in sequence will expand the coordinates into Figure 3The one-dimensional data shown is written into the memory, and the one-dimensional data can be called the index vector of SparseTensor. When splicing different Sparse Tensors, the Indices used in this application are all index vectors of Sparse Tensor for operation. The explanation will not be repeated later. Then, when splicing IndicesA and Indices B, each element in Indices A and each element in IndicesB can be read from the memory in sequence, and then an addition operation is performed on each element read to obtain Indices C.

[0053] Therefore, in the prior art, when splicing different Sparse Tensors, an addition operation is performed on each element in the Indices of different Sparse Tensors in turn, which wastes the computing resources of the computing unit (such as CPU or GPU), and the efficiency of the computing unit in implementing the entire splicing process is relatively low. Therefore, in the process of splicing Indices, how to reduce the utilization rate of the computing resources of the computing unit and improve the splicing efficiency of the computing unit for Indices is a problem that needs to be solved urgently.

[0054] See also Figure 4 , Figure 4 A schematic diagram of a computing device provided in an embodiment of the present application. Figure 4 As shown, the computing device includes a storage unit 10 , a computing unit 20 , a general register 30 , and a special register 40 .

[0055] Optionally, the general register 30 is used to store data in a common addition operation process, wherein the common addition operation is to perform an addition operation on one element at a time, that is, Figure 3 The addition operation process shown; optionally, the number of general registers 30 can be multiple, so as to store the input data, operands and output results in the ordinary addition operation process;

[0056] Optionally, the dedicated register 40 is used to store data in the vectorized computing process corresponding to the SIMD instruction, so as to realize the vectorized computing corresponding to the SIMD instruction. Therefore, the dedicated register can also be called SIMD register; optionally, the number of the dedicated register 40 can be multiple, so as to store input data, operands and output results in the vectorized computing process.

[0057] Optionally, the storage unit 10 is used to store an index tensor of the sparse tensor;

[0058] Optionally, the computing unit 20 may be a CPU or a GPU. The present application does not limit the type of the computing unit 20, but the present application mainly illustrates the computing unit 20 as a CPU.

[0059] Therefore, based on Figure 4 When the computing device needs to concatenate the first index tensor of the first sparse tensor and the second index tensor of the second sparse tensor, the computing unit 20 first obtains the length of the first index tensor and the length of the second index tensor; then, the computing unit 20 obtains at least one first target instruction corresponding to the first index tensor from the instruction set based on the length of the first index tensor, and obtains at least one second target instruction corresponding to the second index tensor from the instruction set based on the length of the second index tensor. Finally, the computing unit 20 uses each first target instruction to perform vectorized calculations on the first index tensor based on the execution order of at least one first target instruction, and uses each second target instruction to perform vectorized calculations on the second index tensor based on the execution order of at least one second target instruction, so as to concatenate the first index tensor and the second index tensor to obtain a third index tensor.

[0060] Specifically, for any first target instruction, when using the first target instruction, the computing unit 20 will read the data corresponding to each first target instruction from the storage unit 10 based on the storage address corresponding to the first index tensor, and store the data in a special register 40, and obtain the self-increment data corresponding to the data, and store the self-increment data in another special register 40, and then the computing unit 20 executes the first target instruction, thereby adding the data at the corresponding positions in the two special registers 40 at the same time, and storing the addition result in another special register 40, and finally the computing unit 20 loads the addition result from the register to the storage unit 10. Similarly, for each second target instruction, similar operations as each first target instruction are performed, so that after executing the above-mentioned at least one first target instruction and at least one second target instruction, and loading the addition results into the storage unit 10 in sequence, the third index tensor can be obtained, and the splicing of the first index tensor and the second index tensor is completed.

[0061] It can be seen that in an embodiment of the present application, when the index tensors of different sparse tensors are spliced, the instructions corresponding to the index tensor of each sparse tensor are respectively obtained, and then the instructions corresponding to the index tensor of each sparse tensor are used to perform vectorized calculations on the index tensor of each sparse tensor, so that multiple elements in the index tensor can be added in parallel at the same time, thereby reducing the number of addition operations in the process of splicing the index tensors, thereby reducing the demand for computing resources of the computing unit, and improving the splicing efficiency of the index tensors, thereby improving the splicing efficiency of the sparse tensors, reducing the waiting time of threads that need to use the spliced ​​sparse tensors, and improving the efficiency of subsequent model reasoning.

[0062] It should be noted that different SIMD instructions process different data lengths. For different SIMD instructions, dedicated memories with different bits can be configured, or shared registers can be designed. For example, for AVX (Advanced Vector Extensions) 256 instructions, which can process 256-bit data, 256-bit registers can be configured for AVX256 instructions. For SSE (Streaming SIMD Extensions) 128 instructions, which can process 128-bit data, 128-bit registers can be configured for SSE128. Of course, AVX256 instructions and SSE12 instructions can also be configured to share a 256-bit register, so that when executing SSE128 instructions, only the lower 128 bits of the 256-bit register need to be used.

[0063] See also Figure 5 , Figure 5 This is a flow chart of a method for vectorizing index tensor calculation according to an embodiment of the present application. The method is applied to the above-mentioned computing device. The method includes but is not limited to the following steps:

[0064] 501: Get the length of the first index tensor of the first sparse tensor and the length of the second index tensor of the second sparse tensor.

[0065] Exemplarily, after the computing device obtains the concatenation instruction for the first sparse tensor and the second sparse tensor, the concatenation instruction can be parsed to obtain the length of the first index tensor and the length of the second index tensor. Specifically, the computing unit further includes a control unit, and the control unit can parse the concatenation instruction to obtain the length of the first index tensor and the length of the second index tensor.

[0066] Optionally, the length of the first index tensor is the number of elements in the first index tensor, and the length of the second index tensor is the number of elements in the second index tensor.

[0067] It should be noted that when splicing the first sparse tensor and the second sparse tensor, the first sparse tensor and the second sparse tensor are mainly spliced ​​in the first dimension, where the first dimension is also called the batchsize dimension. In addition, when splicing the first sparse tensor and the second sparse tensor in the first dimension, the second sparse tensor can be spliced ​​at the bottom of the first sparse tensor, or the first tensor can be spliced ​​at the bottom of the second sparse tensor, and this application does not limit this. For the sake of clarity, this application is mainly explained by taking the splicing of the second sparse tensor at the bottom of the first sparse tensor as an example.

[0068] It should be noted that the splicing instruction also includes the storage address of the first index tensor in the storage unit and the storage address of the second index tensor in the storage unit, so as to facilitate the subsequent loading of the first index tensor and the second index tensor from the storage unit.

[0069] 502: Based on the length of the first index tensor, obtain at least one first target instruction corresponding to the first index tensor from an instruction set.

[0070] The instruction set includes: SISD instructions and SIMD instructions.

[0071] Optionally, SISD instructions include but are not limited to: addition instructions, multiplication instructions, subtraction instructions, and the like.

[0072] Optionally, since this application uses the CPU to splice index tensors as an example, SISD instructions that are compatible with the CPU include but are not limited to SSE128 instructions, AVX256 instructions, and AVX512 instructions. Among them, SSE128 instructions can process 128 bits (16 bytes) at a time, AVX256 instructions can process 256 bits (32 bytes) at a time, and AVX512 instructions can process 512 bits (64 bytes) at a time.

[0073] Exemplarily, the computing device obtains the length of data processed by each instruction in the instruction set SIMD, wherein the length of data processed by each instruction is the number of elements in the index tensor that each instruction can process at one time.

[0074] Specifically, the computing device determines the length of the data processed by each instruction based on the storage type of the index tensor (including the first index tensor and the second index tensor) and the byte length that each instruction can process at one time. Specifically, the computing device determines the byte length of each element in the index tensor based on the storage type of the index tensor, and then determines the length of the data processed by each instruction based on the byte length that each instruction can process at one time and the byte length of each element in the index tensor, wherein the length of the data processed by each instruction is B1 / B2, wherein B1 is the byte length that each instruction can process at one time, and B2 is the byte length of each element in the index tensor.

[0075] It should be noted that since the first dimension of Indices indicates the index of the sample in batchsize, and the second dimension indicates the position index of the discrete feature in the embedding table, the value in Indices generally does not exceed the value in Dense_Shape. Due to the limitation of the number of discrete features and the number of samples in each batchsize, the value in the Dense_Shape tensor is generally in the range of 0 to 2147483647 (that is, the positive value range of int32), so the value in Indices is also in the range of 0 to 2147483647, but in order to adapt to the future needs of hardware devices, the int64 data type is currently used to store the Indices tensor. Therefore, the storage type of the first index tensor and the second index tensor is int64.

[0076] Therefore, when the storage type of the index tensor is int64, each element in Indices occupies 8 bytes, so the AVX512 instruction can process 8 elements in the index tensor at one time, that is, the data length processed by the AVX256 instruction is 8, the AVX256 instruction can process 4 elements in the index tensor at one time, that is, the data length processed by the AVX256 instruction is 4, and the SSE128 instruction can process 2 elements in the index tensor at one time, that is, the data length processed by the SSE128 instruction is 2.

[0077] In one embodiment of the present application, in order to speed up the reading efficiency of the index tensor and increase the length of data that can be processed by the instruction at one time, the present application first converts the storage type of the index tensor.

[0078] For example, Figure 6As shown, the storage type of the first index tensor is converted from a first storage type to a second storage type and / or the storage type of the second index tensor is converted from a first storage type to a second storage type, wherein the first storage type occupies more bytes than the second storage type, and optionally, the first storage type is int64 and the second storage type is int32.

[0079] It should be noted that when converting the storage type, one or both of the first index tensor and the second index tensor can be converted, and this application does not limit this. For example, the data type of the elements in the first index tensor is not converted and remains int64, and only the data type of the elements in the second index tensor is converted. However, for the convenience of description and subsequent calculation process, this application adjusts the storage type of the first index tensor and the second index tensor to the second storage type, that is, this application takes the storage type of the first index tensor and the second index tensor as int32 as an example for explanation.

[0080] Therefore, when the storage type of the index tensor is int32, each element in Indices occupies 4 bytes, so the AVX512 instruction can process 16 elements in the index tensor at one time, that is, the data length processed by the AVX256 instruction is 16, the AVX256 instruction can process 8 elements in the index tensor at one time, that is, the data length processed by the AVX256 instruction is 8, and the SSE128 instruction can process 4 elements in the index tensor at one time, that is, the data length processed by the SSE128 instruction is 4.

[0081] It can be understood that, for the SISD instruction, no matter what the storage type of the index tensor is, the SISD instruction can only process one element in the index tensor at a time, so the data length processed by the SISD instruction is 1.

[0082] The following takes the first index tensor as an example to introduce how to obtain at least one first target instruction corresponding to the first index tensor.

[0083] Exemplarily, based on the length of data processed by each instruction and the length of the first index tensor, at least one first target instruction corresponding to the first index tensor is obtained from the instruction set.

[0084] Exemplarily, whether there is an instruction in the instruction set whose processed data length is the same as the length of the first index tensor, and optionally, the instruction is used as the at least one first target instruction. For example, if the length of the first index tensor is 8, and the length of the data processed by the AVX256 instruction is also 8, the AVX256 instruction is directly used as the at least one first target instruction. Optionally, if it does not exist, at least one first target instruction is determined based on the length of the data processed by each instruction in the instruction set.

[0085] Specifically, firstly, an instruction whose processing data length is less than the length of the first index tensor is selected from the instruction set, and then the instruction whose difference with the length of the first index tensor is the smallest is obtained from the instruction, and the instruction is used as a first target instruction; then, the difference between the length of the first index tensor and the length of the data processed by the instruction is used as the remaining length of the first index tensor. Then, based on the remaining length, the instruction whose processing data length is less than or equal to the remaining length of the first index tensor and whose difference with the remaining length is the smallest is continued to be selected from the instruction set, and the instruction is also used as a first target instruction, until the remaining length of the first index tensor is 0, after all the selected instructions with the smallest difference are used as the first target instruction, the above-mentioned at least one first target instruction can be obtained.

[0086] Take the first sparse tensor as Figure 1 Taking Spare Tensor B shown as an example, the length of the first index tensor is 6, then there is no instruction in the instruction set that can process 6 elements at a time, and the data length processed by the SSE128 instruction is less than 6, and the difference in the length of the first index tensor is the smallest, then the SSE128 instruction can be used as a first target instruction; then, the remaining length of the first index tensor is 2, and the SISD instruction is continuously selected from the instruction set as a first target instruction; in this way, the remaining length of the first index tensor becomes 1, and the SISD instruction is continuously selected from the instruction set as a first target instruction. At this time, the remaining length of the first index tensor is 0, and at least one first target instruction is obtained, namely one SSE128 instruction and two SISD instructions.

[0087] It can be seen that in the embodiment of the present application, if an instruction can process the index tensor, the instruction is selected to process the index tensor. If no instruction can just process the index tensor, the instruction closest to the length of the index tensor is selected. In this way, as few instructions as possible can be selected from the instruction set, and instructions with longer lengths for processing data can be used as much as possible to perform vectorized calculations on the index tensor, thereby improving the efficiency of splicing the index tensor as much as possible.

[0088] 503: Based on the length of the second index tensor, obtain at least one second target instruction corresponding to the second index tensor from the instruction set.

[0089] Among them, the method of obtaining at least one second target instruction of the second index tensor from the instruction set is similar to the above-mentioned method of obtaining at least one first target instruction of the first index tensor, and will not be repeated.

[0090] 504: Execute the at least one first target instruction and the at least one second target instruction to perform vectorized calculation on the first index tensor and the second index tensor to obtain a third index tensor.

[0091] Exemplarily, based on the execution order, the at least one first target instruction and the at least one second target instruction are executed in sequence to perform vectorized calculations on the first index tensor and the second index tensor respectively, complete the concatenation of the first index tensor and the second index tensor, and obtain a third index tensor.

[0092] It should be noted that the present application mainly completes the splicing of the first index tensor and the second index tensor through one thread. Therefore, the present application will execute the above-mentioned at least one first target instruction and at least one second target instruction in sequence. Since the present application is explained by splicing the second sparse tensor to the bottom of the first sparse tensor as an example, the computing device will first execute at least one first target instruction in sequence before executing the at least one second target instruction in sequence. In other words, the execution time of any first target instruction is earlier than the above-mentioned at least one second target instruction. Therefore, the focus of the present application is on how to determine the execution order between at least one first target instruction and the execution order between at least one second target instruction.

[0093] In one embodiment of the present application, the execution order of the at least one first target instruction is randomly generated, that is, after selecting at least one first target instruction, the at least one first target instruction can be randomly sorted to obtain the execution order of the at least one first target instruction.

[0094] In one embodiment of the present application, the execution order of the at least one first target instruction is determined based on the order in which each first target instruction is selected from the instruction set. Exemplarily, the order of selection from front to back or from back to front can be used as the execution order of the at least one first target instruction. For example, for the first index tensor, one SSE128 instruction and two SISD instructions are selected in sequence, and the execution order of the SSE128 instruction and the two SISD instructions can be determined as follows: execute the SSE128 instruction first, then execute a SISD instruction, and finally execute another SISD instruction.

[0095] In one embodiment of the present application, the execution order of the at least one first target instruction is determined based on the state of the register. Exemplarily, for each first target instruction, the current occupancy rate of the register corresponding to each first target instruction is obtained; then, the order of the usage rate from low to high is used as the execution order of the at least one first target instruction. In this way, instructions with lower register occupancy rates are executed first, so that during the execution of the instruction, all registers of the instruction adaptation type will not be occupied. Instructions with higher register occupancy rates will be executed later. In this way, when the instruction is executed, some registers corresponding to the instruction may have been released, so that there are enough register resources to use.

[0096] It should be noted that the method of determining the execution order of at least one second target instruction is similar to the above-mentioned method of determining the execution order of at least one first target instruction, and will not be described again.

[0097] In one embodiment of the present application, after determining the execution order of at least one first target instruction and the execution order of at least one second target instruction, at least one first target instruction and at least one second target instruction can be cached in an instruction queue according to the execution order, so that the computing device can take out each instruction from the instruction queue in turn, and then execute the vectorized calculation process corresponding to the instruction to complete the splicing of the index tensor.

[0098] Exemplarily, when the at least one first target instruction is executed in sequence, the first sub-tensor corresponding to each first target instruction is obtained from the first index tensor in sequence. Specifically, according to the execution order of at least one first target instruction, the computing unit obtains each first target instruction in sequence, and then obtains the first sub-tensor corresponding to each first target instruction from the memory, that is, corresponding to the execution order, the elements in the first index tensor are read from the memory in sequence in the direction of the first dimension to obtain the first sub-tensor corresponding to each first target instruction, wherein the length of the first sub-tensor corresponding to each first target instruction is the same as the length of the data processed by each first target instruction, that is, in the direction of the first dimension, the elements with the same length as the length of the first target instruction are read from the memory to obtain the first sub-tensor.

[0099] For example, at least one target instruction is an SSE128 instruction and two SISD instructions, and the execution order is to first execute the SSE128 instruction, and then execute the two SISD instructions in sequence. Figure 7 As shown, when executing the SSE128 instruction, the first sub-tensor read from the memory is [0 6 1 2], then when executing the first SISD instruction, the first sub-tensor read from the memory is [2], and when executing the second SISD instruction, the first sub-tensor read from the memory is [4].

[0100] Further, the computing unit performs vectorized calculations on elements in the first sub-tensor corresponding to the first target instructions to obtain a second sub-tensor corresponding to each first target instruction.

[0101] Exemplarily, based on the splicing direction of the first sparse tensor and the second sparse tensor, the operand corresponding to each element in the first sub-tensor corresponding to each first target instruction is determined; each element in the first sub-tensor corresponding to each first target instruction and the operand corresponding to each element are added in parallel to obtain the second sub-tensor corresponding to each first target instruction.

[0102] Exemplarily, when the second sparse tensor is spliced ​​to the bottom of the first sparse tensor, the operand corresponding to each element is 0; when the first sparse tensor is spliced ​​to the bottom of the second sparse tensor, if the element in the first sub-tensor is used to represent the row position of the non-zero element in the first sparse tensor in the first sparse tensor, the operand corresponding to the element is the number of samples in the second sparse tensor, and if the element is used to represent the column position of the non-zero element in the first sparse tensor in the first sparse tensor, the operand corresponding to the element is 0.

[0103] Specifically, the computing unit loads the first sub-tensor corresponding to each first target instruction into the register corresponding to the first target instruction, wherein if the target instruction is a SIMD instruction, the register is a dedicated register, and if the target instruction is a SISD instruction, the register is a general register, and the computing unit also loads the operand corresponding to each element in the first sub-tensor corresponding to each first target instruction into the register corresponding to the first target instruction. Finally, the computing unit uses the register corresponding to the first target instruction to simultaneously perform addition operations on each element in the first sub-tensor stored in the register and the operand corresponding to each element, and obtains the second sub-tensor corresponding to each first target instruction.

[0104] It can be understood that after obtaining the second sub-tensor corresponding to each first target instruction, the computing unit first writes the second sub-tensor corresponding to each first target instruction into the register corresponding to the first target instruction; then, the second sub-tensor is loaded from the register to the memory, so that after executing at least one first target instruction, the addition operation of the first index tensor can be completed. Furthermore, after executing at least one second target instruction, the addition operation of the second index tensor can be completed. Finally, after loading the second sub-tensor corresponding to each first target instruction and the second sub-tensor corresponding to each second target instruction into the memory in sequence, the third index tensor can be obtained.

[0105] For example, if Figure 7 As shown, when the first index tensor Indices A and the second index tensor Indices B are concatenated, it is determined that the operand corresponding to each element in the first sparse tensor Indices A is 0, the operand corresponding to the element used to represent the row information of the non-zero element in Indices B is 4, and the operand corresponding to the element used to represent the column information is 0. After determining the operands in the index tensor, the SSE128 instruction can be executed first to load the four elements in the first sub-tensor [0 6 1 2] into a special register, and then the four operands corresponding to the four elements, that is, four 0s, are also loaded into a special register, and then the two special registers are used to perform addition operations on the four elements in parallel, that is, perform an addition operation process, and obtain the second sub-tensor corresponding to the SSE128 instruction, that is, [0 6 1 2]. Finally, [0 6 1 2] is loaded into the memory in sequence. Then execute the SISD instruction again, load the first sub-tensor [2] into a general register, and load the operand corresponding to the first sub-tensor [2], that is, 0, into a general register, and use these two general registers to perform an addition operation on the first sub-tensor [2] to obtain the second sub-tensor [2] corresponding to the SISD instruction, and similarly load the second sub-tensor [2] in sequence. Then, execute the SISD instruction again, and obtain the second sub-tensor [4] corresponding to the SISD instruction, and similarly load the second sub-tensor [4] in sequence. Then, the second target instruction, that is, the AVX256 instruction, is executed to load the first sub-tensor corresponding to the AVX256 instruction, that is, the 8 elements of Indices B, into a special register at one time. Then, the 8 operation data corresponding to these 8 elements, that is, [4 0 4 0 4 0 4 0], are recorded in another special register. Similarly, these two registers are used to perform addition operations on these 8 elements in parallel, that is, an addition operation process is performed to obtain the second sub-tensor corresponding to the AVX256 instruction, that is, [4 1 4 2 5 0 7 2], and the second sub-tensor is written to the memory in sequence, so that the third index tensor Indices C can be obtained, and then the splicing of Indices A and Indices B is completed.

[0106] It can be seen that Figure 7 In the vectorized calculation, the index tensor is concatenated, and Figure 3 Compared with the splicing shown, the number of addition operations is reduced from 14 to 4, and 10 addition operations are reduced. Therefore, in the process of splicing the index tensor, the computing resources of the computing device are saved, and the splicing efficiency of the index tensor is significantly improved.

[0107] It can be seen that in an embodiment of the present application, when the index tensors of different sparse tensors are spliced, the instructions corresponding to the index tensor of each sparse tensor are respectively obtained, and then the instructions corresponding to the index tensor of each sparse tensor are used to perform vectorized calculations on the index tensor of each sparse tensor, so that multiple elements in the index tensor can be added in parallel at the same time, thereby reducing the number of addition operations in the process of splicing the index tensors, thereby reducing the demand for computing resources of the computing unit, and improving the splicing efficiency of the index tensors, thereby improving the splicing efficiency of the sparse tensors, reducing the waiting time of threads that need to use the spliced ​​sparse tensors, and improving the efficiency of subsequent model reasoning.

[0108] Furthermore, in order to improve the splicing efficiency, multiple threads are currently created to execute multiple addition processes in parallel. However, creating multiple threads requires consuming multiple thread resources, and computing resources need to be allocated for each thread resource. Moreover, there is a complex logic of data merging after the operation of multiple threads. Although the splicing efficiency is improved, there is a large amount of computing resources wasted on the computing device, and the processing logic is relatively complex. Compared with creating multiple threads to improve the splicing efficiency, the present application executes each instruction sequentially through a single thread, so the execution logic is relatively simple, thereby achieving low computational complexity while improving the splicing efficiency, and a single thread only occupies one thread resource, saving computing resources of the computing device.

[0109] The following is an example of the application scenario of this application.

[0110] Scenario 1: Recommended scenario

[0111] In the recommendation scenario, the user's discrete features include the user's personal features and browsing features, where personal features include but are not limited to: the user's gender, age, occupation, hobbies, geographic location, etc.; browsing features include but are not limited to: the user browses, likes, reposts, exposure, downloads, favorites, etc. For each discrete feature, first establish an embedding table for each discrete feature. For example, the first row in the embedding table represents the embedding vector of the male gender, the second row represents the embedding vector of the female gender, the fifth row represents the feature vector of the service industry occupation, and so on.

[0112] In this way, the user data of each user is obtained, and based on the data of each user, the data of each user under each discrete feature is determined, so as to construct a sparse tensor corresponding to each user. For example, if a user is male and his occupation is in the service industry, etc., the sparse vector of the user can be constructed as [1 0...1...], that is, the user has data under the discrete feature "male" and data under the discrete feature service industry (no data in other industries). Then, taking multiple users as a sample batch (batchsize), the sparse vectors of the sample batch can be combined into a larger Spare Tensor.

[0113] Furthermore, in order to improve the efficiency of querying the embedding table and increase the number of users processed, the vectorized calculation method of the present application can be used to splice multiple Spare Tensors to form a larger SpareTensor. For ease of distinction, the present application refers to this larger Spare Tensor as the target Spare Tensor. However, when using the recommendation model for reasoning, the computing device obtains the target Indices, target Values ​​and target Dense_Shape of the target Spare Tensor. Therefore, the computing device can restore the target Spare Tensor based on the target Indices, target Values ​​and target Dense_Shape, and then based on the target Spare Tensor, it can determine the data of each user under each discrete feature, and query the feature vector of the discrete feature where the data exists from the embedding table; then, the recommendation model uses the data of each user under each discrete feature and the feature vector of the discrete feature for reasoning, calculates the preference of each user, obtains the preference of each user, and then recommends to the user based on the preference of each user.

[0114] It should be noted that the above recommendations include but are not limited to: video recommendations, news recommendations, consulting recommendations, merchant recommendations, etc.

[0115] Scenario 2: Search scenario

[0116] In the search scenario, the user's discrete features include personal features and search features, where personal features include but are not limited to: the user's gender, age, occupation, hobbies, geographic location, etc.; search features include but are not limited to: search terms, search time, type of search device, etc. Therefore, for each discrete feature of the user, first establish an embedding table for each discrete feature. For example, the first row in the embedding table represents the embedding vector of male gender, the second row represents the embedding vector of female gender, the 100th row represents the feature vector of the search time in the morning, the 101st row represents the feature vector of the search time in the afternoon, and so on.

[0117] Therefore, similar to the above recommendation scenario, the discrete features of multiple users in a batchsize can be combined into a Spare Tensor, and the vectorized calculation method of this application can be used to splice the Spare Tensors of multiple batchsizes into a larger Spare Tensor. This application refers to the Spare Tensor as the target Spare Tensor. However, when using the search model for processing, the computing device obtains the target Indices, target Values, and target Dense_Shape of the target Spare Tensor. Therefore, the computing device can restore the target Spare Tensor based on the target Indices, target Values, and target Dense_Shape, and then based on the target Spare Tensor, it can determine the data of each user under each discrete feature, and query the feature vector of the discrete feature where the data exists from the embedding table; then, the search model uses the data of each user under each discrete feature and the feature vector of the discrete feature to perform reasoning, identify the intent of each user, obtain the search intent of each user, and then search for the user based on the search intent of each user.

[0118] It should be noted that the above-mentioned searches include but are not limited to: text search, video search, news search, business search, etc.

[0119] See also Figure 8 , Figure 8 A schematic diagram of a computing device provided in an embodiment of the present application. The computing device 800 includes: an acquisition module 801 and a processing module 802, wherein:

[0120] An acquisition module 801 is used to acquire a first sparse tensor and a second sparse tensor;

[0121] Processing module 802 is used to obtain at least one first target instruction corresponding to the first index tensor from an instruction set based on the length of the first index tensor; based on the length of the second index tensor, obtain at least one second target instruction corresponding to the second index tensor from the instruction set; execute the at least one first target instruction and the at least one second target instruction to perform vectorized calculations on the first index tensor and the second index tensor to obtain a third index tensor.

[0122] In one embodiment of the present application, in terms of obtaining at least one first target instruction corresponding to the first index tensor from an instruction set based on the length of the first index tensor, the processing module 802 is specifically configured to:

[0123] Obtaining the length of data processed by each instruction in the instruction set;

[0124] If there is an instruction in the instruction set whose processed data has the same length as the first index tensor, use the instruction as the at least one first target instruction;

[0125] If there is no instruction in the instruction set that processes data with the same length as the first index tensor, the at least one first target instruction is determined based on the length of the data processed by each instruction in the instruction set.

[0126] In one embodiment of the present application, in determining the at least one first target instruction based on the length of data processed by each instruction in the instruction set, the processing module 802 is specifically configured to:

[0127] Selecting, from the instruction set, an instruction whose length of processed data is less than the length of the first index tensor and whose difference with the length of the first index tensor is the smallest as a first target instruction;

[0128] Continue to select from the instruction set an instruction whose length of processing data is less than or equal to the remaining length of the first index tensor and whose difference with the remaining length is the smallest, as a first target instruction, until the remaining length of the first index tensor is zero, and obtain the at least one first target instruction.

[0129] In one embodiment of the present application, in terms of performing vectorized calculations on the first index tensor and the second index tensor based on the at least one first target instruction and the at least one second target instruction to obtain a third index tensor, the processing module 802 is specifically configured to:

[0130] When executing the at least one first target instruction in sequence, sequentially obtaining a first sub-tensor corresponding to each first target instruction from the first index tensor, and performing vectorized calculations on the elements in the first sub-tensor corresponding to each first target instruction to obtain a second sub-tensor corresponding to each first target instruction, wherein the length of the first sub-tensor corresponding to each first target instruction is the same as the length of the data processed by each first target instruction;

[0131] When executing the at least one second target instruction in sequence, sequentially obtaining a first sub-tensor corresponding to each second target instruction from the second index tensor, and performing vectorized calculations on elements in the first sub-tensor corresponding to each second target instruction to obtain a second sub-tensor corresponding to each second target instruction, wherein the length of the first sub-tensor corresponding to each second target instruction is the same as the length of data processed by each second target instruction;

[0132] The third index tensor is obtained based on the second sub-tensor corresponding to each first target instruction and the second sub-tensor corresponding to each second target instruction.

[0133] In one embodiment of the present application, in performing vectorized calculations on elements in the first sub-tensor corresponding to each first target instruction to obtain a second sub-tensor corresponding to each first target instruction, the processing module 802 is specifically configured to:

[0134] Determine, based on a concatenation direction of the first sparse tensor and the second sparse tensor, an operand corresponding to each element in the first sub-tensor corresponding to each first target instruction;

[0135] An addition operation is performed in parallel on each element in the first sub-tensor corresponding to each first target instruction and the operand corresponding to each element to obtain a second sub-tensor corresponding to each first target instruction.

[0136] In one embodiment of the present application, when the second sparse tensor is spliced ​​to the bottom of the first sparse tensor, the operand corresponding to each element is 0; when the first sparse tensor is spliced ​​to the bottom of the second sparse tensor, if the element in the first sub-tensor is used to represent the row position of the non-zero element in the first sparse tensor in the first sparse tensor, the operand corresponding to the element is the number of samples in the second sparse tensor, and if the element is used to represent the column position of the non-zero element in the first sparse tensor in the first sparse tensor, the operand corresponding to the element is 0.

[0137] In one embodiment of the present application, based on the at least one first target instruction and the at least one second target instruction, the first index tensor and the second index tensor are respectively vectorized and calculated, and before obtaining the third index tensor, the processing module 802 is further used to:

[0138] Convert the storage type of the first index tensor from a first storage type to a second storage type; and / or convert the storage type of the second index tensor from a first storage type to the second storage type; wherein the first storage type occupies more bytes than the second storage type.

[0139] In one embodiment of the present application, the instruction set includes at least: single instruction single data stream SISD instructions and single instruction multiple data stream SIMD instructions, wherein the SIMD instructions include at least SSE128 instructions, AVX256 instructions and AVX512 instructions.

[0140] In one embodiment of the present application, each row in the first sparse tensor and the second sparse tensor corresponds to a sample, and the value of each element in each row is used to characterize the behavior of the discrete feature indicated by the sample to the element.

[0141] See also Fig. 9 , Fig. 9 A schematic diagram of an electronic device provided in an embodiment of the present application. Fig. 9 As shown, the electronic device 900 includes a transceiver 901, a processor 902 and a memory 903. They are connected via a bus 904. The memory 903 is used to store computer programs and data, and can transmit the data stored in the memory 903 to the processor 902. The electronic device 900 can be the computing device 800 mentioned above.

[0142] The processor 902 is used to read the computer program in the memory 903 and perform the following operations:

[0143] Get the length of the first index tensor of the first sparse tensor and the length of the second index tensor of the second sparse tensor;

[0144] Based on the length of the first index tensor, obtaining at least one first target instruction corresponding to the first index tensor from an instruction set;

[0145] Based on the length of the second index tensor, obtaining at least one second target instruction corresponding to the second index tensor from the instruction set;

[0146] Execute the at least one first target instruction and the at least one second target instruction to perform vectorized calculations on the first index tensor and the second index tensor to obtain a third index tensor.

[0147] Specifically, the transceiver 901 may be Figure 8 The acquisition module 801 of the computing device 800 of the embodiment described above, the processor 902 may be Figure 8 The processing module 802 of the computing device 800 of the illustrated embodiment.

[0148] The computing device server or node of the present application, wherein the server or node can be a cloud computing server, a content delivery network (Content Delivery Network, CDN) server, a network time protocol (Network Time Protocol, NTP), a domain name resolution system (Domain Name System, DNS) server, and other various types of servers. The above servers are only examples, not exhaustive, including but not limited to the above servers.

[0149] An embodiment of the present application also provides a computer-readable storage medium, which stores a computer program, and the computer program is executed by a processor to implement part or all of the steps of any index tensor vectorization calculation method recorded in the above method embodiments.

[0150] An embodiment of the present application also provides a computer program product, which includes a non-transitory computer-readable storage medium storing a computer program, and the computer program is operable to cause a computer to execute part or all of the steps of any index tensor vectorization calculation method recorded in the above method embodiments.

[0151] It should be noted that, for the aforementioned method embodiments, for the sake of simplicity, they are all expressed as a series of action combinations, but those skilled in the art should be aware that the present application is not limited by the described order of actions, because according to the present application, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all optional embodiments, and the actions and modules involved are not necessarily required by the present application.

[0152] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0153] In the several embodiments provided in the present application, it should be understood that the disclosed device can be implemented in other ways. For example, the device embodiments described above are only schematic, such as the division of the units, which is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, and the indirect coupling or communication connection of the device or unit can be electrical or other forms.

[0154] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0155] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above integrated unit may be implemented in the form of hardware or in the form of a software program module.

[0156] If the integrated unit is implemented in the form of a software program module and sold or used as an independent product, it can be stored in a computer-readable memory. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a memory, including a number of instructions to enable a computer device (which can be a personal computer, server or network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned memory includes: U disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), mobile hard disk, disk or CD-ROM and other media that can store program codes.

[0157] A person skilled in the art can understand that all or part of the steps in the various methods of the above embodiments can be completed by instructing related hardware through a program, and the program can be stored in a computer-readable memory, and the memory can include: a flash drive, a read-only memory (English: Read-Only Memory, abbreviated as: ROM), a random access memory (English: Random Access Memory, abbreviated as: RAM), a magnetic disk or an optical disk, etc.

[0158] The embodiments of the present application are introduced in detail above. Specific examples are used in this article to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method and core idea of ​​the present application. At the same time, for general technical personnel in this field, according to the idea of ​​the present application, there will be changes in the specific implementation method and application scope. In summary, the content of this specification should not be understood as a limitation on the present application.

Claims

1. A method for vectorizing index tensors, characterized in that: include: Get the length of the first index tensor of the first sparse tensor and the length of the second index tensor of the second sparse tensor; Based on the length of the first index tensor, obtaining at least one first target instruction corresponding to the first index tensor from an instruction set; Based on the length of the second index tensor, obtaining at least one second target instruction corresponding to the second index tensor from the instruction set; Execute the at least one first target instruction and the at least one second target instruction to perform vectorized calculations on the first index tensor and the second index tensor to obtain a third index tensor.

2. The method according to claim 1, characterized in that The acquiring, based on the length of the first index tensor, at least one first target instruction corresponding to the first index tensor from an instruction set includes: Obtaining the length of data processed by each instruction in the instruction set; If there is an instruction in the instruction set whose processed data has the same length as the first index tensor, use the instruction as the at least one first target instruction; If there is no instruction in the instruction set that processes data with the same length as the first index tensor, the at least one first target instruction is determined based on the length of the data processed by each instruction in the instruction set.

3. The method according to claim 2, characterized in that The determining the at least one first target instruction based on the length of data processed by each instruction in the instruction set comprises: Selecting, from the instruction set, an instruction whose length of processed data is less than the length of the first index tensor and whose difference with the length of the first index tensor is the smallest, as a first target instruction; Continue to select from the instruction set an instruction whose length of processing data is less than or equal to the remaining length of the first index tensor and whose difference with the remaining length is the smallest, as a first target instruction, until the remaining length of the first index tensor is zero, and obtain the at least one first target instruction.

4. The method according to any one of claims 1 to 3, characterized in that: The performing vectorized calculation on the first index tensor and the second index tensor based on the at least one first target instruction and the at least one second target instruction respectively to obtain a third index tensor includes: When executing the at least one first target instruction in sequence, sequentially obtaining a first sub-tensor corresponding to each first target instruction from the first index tensor, and performing vectorized calculations on the elements in the first sub-tensor corresponding to each first target instruction to obtain a second sub-tensor corresponding to each first target instruction, wherein the length of the first sub-tensor corresponding to each first target instruction is the same as the length of the data processed by each first target instruction; When executing the at least one second target instruction in sequence, sequentially obtaining a first sub-tensor corresponding to each second target instruction from the second index tensor, and performing vectorized calculations on elements in the first sub-tensor corresponding to each second target instruction to obtain a second sub-tensor corresponding to each second target instruction, wherein the length of the first sub-tensor corresponding to each second target instruction is the same as the length of data processed by each second target instruction; The third index tensor is obtained based on the second sub-tensor corresponding to each first target instruction and the second sub-tensor corresponding to each second target instruction.

5. The method according to claim 4, characterized in that The performing vectorized calculation on the elements in the first sub-tensor corresponding to each first target instruction to obtain the second sub-tensor corresponding to each first target instruction includes: Determine, based on a concatenation direction of the first sparse tensor and the second sparse tensor, an operand corresponding to each element in the first sub-tensor corresponding to each first target instruction; An addition operation is performed in parallel on each element in the first sub-tensor corresponding to each first target instruction and the operand corresponding to each element to obtain a second sub-tensor corresponding to each first target instruction.

6. The method according to claim 5, characterized in that In the case of splicing the second sparse tensor to the bottom of the first sparse tensor, the operands corresponding to each element are 0; When the first sparse tensor is spliced ​​to the bottom of the second sparse tensor, if the element in the first sub-tensor is used to represent the row position of the non-zero element in the first sparse tensor in the first sparse tensor, the operand corresponding to the element is the number of samples in the second sparse tensor; if the element is used to represent the column position of the non-zero element in the first sparse tensor in the first sparse tensor, the operand corresponding to the element is 0.

7. The method according to any one of claims 1 to 6, characterized in that: Based on the at least one first target instruction and the at least one second target instruction, before respectively performing vectorized calculations on the first index tensor and the second index tensor to obtain a third index tensor, the method further includes: Convert the storage type of the first index tensor from a first storage type to a second storage type; and / or, Converting the storage type of the second index tensor from the first storage type to the second storage type; The first storage type occupies more bytes than the second storage type.

8. The method according to any one of claims 1 to 7, characterized in that: The instruction set includes at least single instruction single data stream SISD instructions and single instruction multiple data stream SIMD instructions, wherein the SIMD instructions include at least SSE128 instructions, AVX256 instructions and AVX512 instructions.

9. The method according to any one of claims 1 to 8, characterized in that: Each row in the first sparse tensor and the second sparse tensor corresponds to a sample, and the value of each element in each row is used to characterize the behavior of the sample with respect to the discrete feature indicated by the element.

10. A computing device, characterized in that: include: Acquisition module and processing module; The acquisition module is used to acquire a first index tensor of the first sparse tensor and a second index tensor of the second sparse tensor; The processing module is used to obtain the length of the first index tensor of the first sparse tensor and the length of the second index tensor of the second sparse tensor; Based on the length of the first index tensor, obtaining at least one first target instruction corresponding to the first index tensor from an instruction set; Based on the length of the second index tensor, obtaining at least one second target instruction corresponding to the second index tensor from the instruction set; Execute the at least one first target instruction and the at least one second target instruction to perform vectorized calculations on the first index tensor and the second index tensor to obtain a third index tensor.

11. An electronic device, characterized in that: include: A processor and a memory, wherein the processor is connected to the memory, the memory is used to store a computer program, and the processor is used to execute the computer program stored in the memory, so that the electronic device executes the method as described in any one of claims 1 to 9.

12. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and the computer program is executed by a processor to implement the method according to any one of claims 1 to 9.

Images