Reverse calculation method, device, equipment and storage medium based on embedded operator

Abstract

The present disclosure provides a reverse calculation method, apparatus, device and storage medium based on an embedded operator, which relates to the field of artificial intelligence technology, and in particular to the field of deep learning. The specific implementation scheme is: obtaining an index tensor and a gradient table to be calculated, wherein the index tensor includes multiple elements, each element corresponds to an index number, and the gradient table to be calculated stores data to be calculated; according to the index number, the data to be calculated is output according to a preset rule to obtain a vocabulary gradient table, wherein the preset rule is related to the data type of the data to be calculated. The present application can improve the utilization rate of the graphics processing unit (GPU) and effectively improve computing performance.

Description

Technical Field

[0001] The present disclosure relates to the field of artificial intelligence technology, specifically to the field of deep learning, and more particularly to a reverse computing method, apparatus, device, storage medium, and computer program product based on an embedded operator. Background Art

[0002] The embedding layer represents text symbols such as letters and words as word vectors that can be processed by computers and serve as semantic representations. Embedding operators are the most commonly used operators in many deep learning models in the field of natural language processing (NLP). In recent years, graphics processing units (GPUs), with their powerful computing throughput and bandwidth capabilities, have become the most commonly used hardware for deep learning model training and deployment. Summary of the Invention

[0003] The present disclosure provides a reverse calculation method, apparatus, device, storage medium, and computer program product based on embedded operators, which improve the utilization rate of the GPU and effectively alleviate the write conflict problem.

[0004] According to one aspect of the present disclosure, a reverse calculation method based on an embedding operator is provided, comprising: obtaining an index tensor and a gradient table to be calculated, wherein the index tensor includes multiple elements, each element corresponds to an index number, and the gradient table to be calculated stores data to be calculated; outputting the data to be calculated according to a preset rule according to the index number to obtain a vocabulary gradient table, wherein the preset rule is related to the data type of the data to be calculated

[0005] According to another aspect of the present disclosure, a reverse computing device based on an embedding operator is provided, comprising: an acquisition unit configured to acquire an index tensor and a gradient table to be calculated, wherein the index tensor comprises a plurality of elements, each element corresponds to an index number, and the gradient table to be calculated stores data to be calculated; a reverse computing unit configured to output the data to be calculated according to a preset rule based on the index number to obtain a vocabulary gradient table, wherein the preset rule is related to the data type of the data to be calculated.

[0006] According to another aspect of the present disclosure, an electronic device is provided, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can perform the above-mentioned reverse calculation method based on the embedded operator.

[0007] According to another aspect of the present disclosure, a non-transitory computer-readable storage medium storing computer instructions is provided, wherein the computer instructions are used to enable the computer to execute the above-mentioned inverse calculation method based on the embedded operator.

[0008] According to another aspect of the present disclosure, a computer program product is provided, including a computer program, which implements the above-mentioned reverse calculation method based on embedded operators when executed by a processor.

[0009] It should be understood that the contents described in this section are not intended to identify the key or important features of the embodiments of the present disclosure, nor are they intended to limit the scope of the present disclosure. Other features of the present disclosure will become readily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] The accompanying drawings are provided to facilitate a better understanding of the present invention and do not constitute a limitation of the present disclosure.

[0011] Figure 1 is a schematic diagram of a reverse calculation process according to an embodiment of the present disclosure;

[0012] Figure 2 is a flowchart of a reverse calculation method based on an embedded operator according to the first embodiment of the present disclosure;

[0013] Figure 3 is a schematic diagram of a forward calculation process according to an embodiment of the present disclosure;

[0014] Figure 4 is a flowchart of a reverse calculation method based on an embedded operator according to a second embodiment of the present disclosure;

[0015] Figure 5 is a flowchart of a reverse calculation method based on an embedded operator according to a third embodiment of the present disclosure;

[0016] Figure 6 2 is a schematic diagram for verifying and comparing reverse calculation of single-precision fp32 type according to an embodiment of the present disclosure;

[0017] Figure 7 2 is a schematic diagram of verification and comparison of reverse calculation of half-precision fp16 type according to an embodiment of the present disclosure;

[0018] Figure 8 is a block diagram of a reverse computing device based on an embedded operator according to another embodiment of the present disclosure;

[0019] Figure 9 3 is a block diagram of an electronic device for implementing the reverse calculation method based on embedded operators according to an embodiment of the present disclosure. DETAILED DESCRIPTION

[0020] The following description of exemplary embodiments of the present disclosure is made in conjunction with the accompanying drawings, including various details of the embodiments of the present disclosure to facilitate understanding. These details should be considered as merely exemplary. Therefore, those skilled in the art will recognize that various changes and modifications may be made to the embodiments described herein without departing from the scope and spirit of the present disclosure. Similarly, for the sake of clarity and conciseness, descriptions of well-known functions and structures are omitted in the following description.

[0021] The Embedding operator is actually a table lookup operation, which includes forward and reverse calculations. It is the most commonly used operator in many deep learning models in the field of natural language processing (NLP). Figure 1 As shown in the figure, during reverse calculation, each row vector of the output gradient out_grad is accumulated to the corresponding row of the output vocabulary gradient table_grad according to the index number in the index tensor index. Since the index tensor index may contain duplicate index numbers, when different threads process the same index number, a write conflict may occur. In the prior art, reverse calculation is performed using the Embedding operator in the Pytorch deep learning framework, the PaddlePaddle deep learning framework, and the Tensorflow deep learning framework. The embedding operator handles intra-block write conflicts by adopting shared memory or atomic instructions. However, the computing performance is poor for certain types of data.

[0022] To this end, this paper proposes a reverse calculation method based on embedded operators based on the shortcomings of existing solutions. Figure 2 , which shows a process 200 of an embodiment of a reverse calculation method based on an embedded operator according to the present disclosure. The method comprises the following steps:

[0023] Step 201: Obtain an index tensor and a table of gradients to be calculated, wherein the index tensor includes multiple elements, each element corresponds to an index number, and the table of gradients to be calculated stores data to be calculated.

[0024] To deepen the understanding of reverse calculation, this embodiment first explains the process of reverse calculation in conjunction with the process of forward calculation. Figure 3The calculation process of the forward calculation is shown. Its input is the index tensor index and the vocabulary vector table, and the output is out. Among them, the dimension of index is the number of queries [num_indices]. It should be noted that the index tensor can also be multi-dimensional, but it can be expanded into one dimension for calculation during the underlying implementation. Therefore, this embodiment takes one dimension as an example. The index tensor includes multiple elements, each element corresponds to an index number; table is the vocabulary vector, and its dimension is [vocab_size, embed_dim], where vocab_size is the vocabulary length and embed_dim is the embedding length. The vocabulary vector table stores the word vector corresponding to each index. During the forward calculation, the vector in the table corresponding to each index number in index is copied to the output out, and the dimension of the final output out is [num_indices, embed_dim].

[0025] Step 202: Output the data to be calculated according to the index number according to the preset rules to obtain a vocabulary gradient table, wherein the preset rules are related to the data type of the data to be calculated.

[0026] In this embodiment, the backward computation uses the gradient out_grad of the output out of the forward computation and the index tensor index as input. Each row vector of the gradient out_grad of the output out is accumulated to the corresponding row of the output table_grad according to the index number in index. table_grad is recorded as the vocabulary gradient table; the gradient out_grad of the output out of the forward computation is recorded as the to-be-computed gradient table, which stores the data to be calculated.

[0027] The reverse calculation method based on the embedded operator provided in this embodiment first obtains the index tensor and the gradient table to be calculated. Secondly, the data to be calculated is output according to the preset rules based on the index number to obtain the vocabulary gradient table, wherein the preset rules are related to the data type of the data to be calculated, that is, different atomic plus instructions are called according to different data types, which can alleviate the write conflict problem during reverse calculation.

[0028] See further Figure 4 , which shows a process 400 of another embodiment of the reverse calculation method based on the embedded operator according to the present disclosure. The reverse calculation method includes the following steps:

[0029] Step 401: Obtain an index tensor and a table of gradients to be calculated, wherein the index tensor includes multiple elements, each element corresponds to an index number, and the table of gradients to be calculated stores data to be calculated.

[0030] In this embodiment, the specific operation of step 401 has been Figure 2Step 201 is described in detail in the illustrated embodiment and will not be repeated here.

[0031] Step 402: Determine the data type of the data to be calculated.

[0032] In this embodiment, the data types of the data to be calculated include half-precision fp16, single-precision fp32, and double-precision fp64.

[0033] Step 403: Call the corresponding atomic instruction according to the data type, calculate the data to be calculated, and obtain the vocabulary gradient table.

[0034] In this embodiment, to increase the speed of atomic calculations, the GPU has added more and more support at the hardware level and provides corresponding atomic instruction interfaces for users to use. In this embodiment, atomic instructions include double-precision atomic addition instructions atomicAdd(double*), single-precision atomic addition instructions atomicAdd(float*), half-precision atomic addition instructions atomicAdd(half_*), and vector half-precision atomic addition instructions. In this embodiment, based on the data type of the data to be calculated, the corresponding atomic instructions are called for calculation, which can alleviate the write conflict problem.

[0035] It should be noted that the above-mentioned atomic instructions are exemplary and do not constitute a specific limitation on atomic instructions. Those skilled in the art can call corresponding atomic instructions to perform calculations on different data types to be calculated based on actual conditions.

[0036] See further Figure 5 , which shows a process 500 of another embodiment of the reverse calculation method based on the embedded operator according to the present disclosure. The reverse calculation method includes the following steps:

[0037] Step 501: Obtain an index tensor and a table of gradients to be calculated, wherein the index tensor includes multiple elements, each element corresponds to an index number, and the table of gradients to be calculated stores data to be calculated.

[0038] In this embodiment, the specific operation of step 501 has been Figure 2 Step 201 is described in detail in the illustrated embodiment and will not be repeated here.

[0039] Step 502: Determine the data type of the data to be calculated.

[0040] In this embodiment, the specific operation of step 502 is already in Figure 4 Step 402 is described in detail in the illustrated embodiment and will not be repeated here.

[0041] Step 503: In response to the data type being non-fp16, an atomic instruction consistent with the data type is called to perform an atomic add operation to obtain a vocabulary gradient table. When the data type is fp32, a single-precision atomic add instruction is called; when the data type is fp64, a double-precision atomic add instruction is called.

[0042] In this embodiment, the data type of the data to be calculated is determined. If the current data type is not fp16, that is, the data type is fp32 or fp64, the single-precision atomic add instruction and the double-precision atomic add instruction have good performance and can alleviate the write conflict problem in the reverse calculation. Therefore, in this embodiment, when the data type is fp32, the single-precision atomic add instruction is called; when the data type is fp64, the double-precision atomic add instruction is called.

[0043] Step 504: In response to the data type being fp16 and the first address of the current row where the data to be calculated is located being 32-byte aligned, two adjacent half-precision elements in the data to be calculated are packed to obtain a vector half-precision element; a vector half-precision atomic add instruction is called to perform an atomic add operation on the vector half-precision elements to obtain a vocabulary gradient table.

[0044] The poor performance of fp16 data is mainly due to the poor performance of half-precision atomic add instructions, while vector half-precision atomic add instructions perform better. Based on this discovery, this embodiment optimizes the reverse calculation by converting the half-precision atomic add instruction atomicAdd(half*) into the vector half-precision atomic add instruction atomicAdd(half_2*).

[0045] Among them, when the data type is fp16 and the first address of the current row where the data is located is 32-byte aligned, that is, when the first address can be divided by 32, it is 32-byte aligned. By packing two adjacent half-precision elements in the calculation data, the vector half-precision element is obtained, and the vector half-precision atomic add instruction with better performance is called to obtain the vocabulary gradient table, which effectively alleviates the write conflict problem and optimizes the performance of reverse calculation.

[0046] Step 505: In response to the data type being fp16 and the first address of the current row where the data to be calculated is located being 32-byte misaligned, zero-fill the half-precision elements in the data to be calculated to obtain vector half-precision elements; call a vector half-precision atomic add instruction to perform an atomic add operation on the vector half-precision elements to obtain a vocabulary gradient table, wherein the vector half-precision elements include half-precision elements and 0.

[0047] In this embodiment, since the data type is fp16, the reverse calculation is optimized by converting the half-precision atomic addition instruction atomicAdd(half*) into the vector half-precision atomic addition instruction atomicAdd(half_2*).

[0048] Those skilled in the art should understand that when the first address of the current row is not 32-byte aligned, that is, when the first address cannot be divided by 32, if it is directly packed, the performance of calling atomicAdd(half_2*) will be poor due to the misalignment. In this case, the present application takes each element of the row as a unit and directly expands each half-precision half element into a vector half-precision half_2 element, where the other element in half_2 is padded with zeros, and then calls the atomicAdd(half_2*) instruction for calculation.

[0049] It should be noted that in practice, embed_dim in the index tensor usually takes an even value, such as 768, 1024, and 4096. In other words, if the output first address is 32-byte aligned, the first address of each row can also be guaranteed to be 32-byte aligned. Therefore, in practical applications, the above method of directly packing two adjacent half-precision elements is the most common.

[0050] This application converts the poor-performance atomicAdd(half*) instruction into the good-performance atomicAdd(half_2*) atomic instruction, thereby achieving good computing performance when the data type is fp16 and alleviating the write conflict problem.

[0051] In addition, the reverse calculation method based on embedded operators proposed in this application is executed on a graphics processing unit (GPU). Those skilled in the art will understand that a GPU (Graphics Processing Unit) is a heterogeneous acceleration processor and the primary hardware platform for current deep learning training. It is typically composed of a streaming multiprocessor (SM), global memory, shared memory, and a bus interface.

[0052] Among them, the streaming multiprocessor (SM) is the main computing resource on the GPU. An SM is usually composed of multiple stream processors (SP). At the software level, to facilitate parallel programming, hardware vendors provide some parallel programming models. Taking NVIDIA CUDA as an example, CUDA implements multi-level abstraction of hardware parallelism, providing concepts such as grids and thread blocks. The grid corresponds to the kernel function running on the GPU. A grid consists of multiple thread blocks, and a thread block has multiple threads. Among them, the thread block corresponds to the hardware SM. An SM can process multiple thread blocks simultaneously. At the bottom level, each thread block is scheduled to a specific SM and will not exit until execution is completed.

[0053] In this embodiment, the GPU includes multiple thread blocks, each of which contains multiple threads. For example, during parallel programming, users can control the total number of threads executing in parallel by configuring the thread block size and the number of threads within the block. CUDA allows configuration in up to three dimensions. For example, setting the thread block size to (8,8,8) means 8 threads in each of the x, y, and z directions, for a total of 512 threads.

[0054] In this embodiment, the task division strategy adopted is: let one thread block process the table lookup of an element in the index; multiple threads in the same thread block collaborate to complete the operation of a row of data in the table, thereby improving the parallel performance during reverse calculation, that is, the thread block is used to process multiple elements in the index tensor and the data to be calculated; the data in the vocabulary gradient table is obtained through multiple threads in the thread block to obtain the vocabulary gradient table.

[0055] To improve GPU utilization, this application reconfigures the number of thread blocks, defining the number of streaming multiprocessors (SMs) as M, and the number of thread blocks equal to the number of elements as N, with the number of streaming multiprocessors and the number of thread blocks satisfying N = 2M. For example, when the number of streaming multiprocessors (SMs) is 40, the GPU can be configured with 80 thread blocks; when the number of streaming multiprocessors (SMs) is 50, the GPU can be configured with 100 thread blocks.

[0056] It should be noted that the above-mentioned number of thread blocks and streaming multiprocessors is exemplary. Those skilled in the art can set a reasonable configuration method based on the computing resources in the GPU. Compared with the previous fixed number of thread blocks, it can ensure that the number of thread blocks can be flexibly set under architectures with different numbers of streaming multiprocessors SM, thereby reducing the number of idle SMs and improving GPU utilization.

[0057] The following section verifies the proposed reverse computation method based on the embedding operator. Specifically, we use the PaddlePaddle framework and design two sets of experiments to demonstrate the optimization effect of the embedding operator itself and the performance improvement in the application. The experimental test environment is an NVIDIA A100 GPU with 80GB of video memory and CUDA version CUDA 11.6.

[0058] (1) The first set of experiments: Optimization effect of the Embedding operator

[0059] In this set of experiments, embed_dim and vocab_size were fixed at 1024 and 30522, respectively, and the number of queries for varying indexes ranged from [56, 28672]. The performance of reverse computations for fp32 and fp16 data types was verified.

[0060] like Figure 6 As shown, when the input data is of type fp32, compared to PaddlePaddle's existing (PaddlePaddle-before) Embedding implementation, performance is improved by 0.99-1.51x. This performance improvement is mainly due to the reconfiguration of the number of thread blocks, which improves GPU utilization. Compared to Tensorflow (Tensorflow), the performance gain is relatively large, exceeding 50 times in many cases, and up to over 60 times. This is mainly due to insufficient parallelism caused by improper task division. Compared to Pytorch Embedding (Pytorch), a 0.94-1.97x improvement is achieved. This performance gain is mainly due to the different methods used to handle write conflicts. It is also observed that due to the increasing support of GPU hardware for fp32 atomic instructions, methods using atomic instructions have the potential to achieve better performance than sorting-based methods in most cases.

[0061] like Figure 7 As shown in the figure, when the input data is of fp16 type, the optimization in this paper also brings significant performance improvement compared with existing work: 1.67-37.3x acceleration compared to PaddlePaddle before optimization, and 5.72-19.2x acceleration compared to Pytorch Embedding implementation; the improvement compared to Tensorlow Embedding implementation is even greater, exceeding 50 times in many cases and up to more than 100 times, which reflects the advantage of fully utilizing efficient atomic instructions implemented by hardware over software simulation implementation.

[0062] (2) The second set of experiments: Verifying the performance effect in the bert-large model

[0063] Next, we use the PaddlePaddle framework to verify the performance improvement of the Embedding operator in actual applications in the Bert-large model. The dataset from Wikipedia-2020 is used for training tasks. In the Bert-large model, the batch size (batch_size), sequence length (seq_len), and embed_dim are set to 56, 512, and 1024, respectively. There are three Embedding layers in the model. The three layers have the same number of retrievals n_indices (the value is batch_size*seq_len), but the vocabulary lengths of the three layers are different, vocab_size, which are set to 30522, 512, and 2, respectively. In addition, mixed precision mode is used for training, that is, all operators except sum and mean are calculated using fp16.

[0064] Table 1 shows the final performance results. Before using the reverse computation optimization proposed in this application, the Embedding operator consumed approximately 9.1% of the total model time, severely impacting the model's training throughput. After the optimization, the Embedding operator's time consumption decreased from 9.1% to 0.5%, resulting in an approximately 10.9% improvement in model throughput.

[0065] Table 1

[0066] Embedding operator time consumption ratio Throughput (seq / s) Before optimization 9.1％ 169.6 After optimization 0.5％ 298.9 Performance improvements -8.6％ +10.9％

[0067] The above verification further illustrates that the reverse calculation method based on the embedding operator of the present application can improve the performance of the parallel implementation of the reverse calculation of the Embedding operator on the GPU and improve the training throughput of the NLP model to a certain extent.

[0068] Further references Figure 8 As an implementation of the methods shown in the above figures, the present disclosure provides an embodiment of a reverse calculation device based on an embedded operator. Figure 2 Corresponding to the method embodiment shown, the device can be specifically applied to various electronic devices.

[0069] like Figure 8 As shown, the reverse computing device 600 of this embodiment may include an acquisition unit 601 and a reverse computing unit 602. The acquisition unit is configured to acquire an index tensor and a gradient table to be calculated, wherein the index tensor includes multiple elements, each element corresponds to an index number, and the gradient table to be calculated stores data to be calculated; the reverse computing unit is configured to output the calculated data according to a preset rule based on the index number to obtain a vocabulary gradient table, wherein the preset rule is related to the data type of the data to be calculated.

[0070] In this embodiment, in the reverse calculation device 600, the specific processing of the acquisition unit 601 and the reverse calculation unit 602 and the technical effects thereof can be referred to in Figure 2 The relevant descriptions of steps 200 - 202 in the corresponding embodiment are not repeated here.

[0071] In some optional implementations of this embodiment, the reverse calculation unit 602 includes: a data type determination module, configured to determine the data type of the data to be calculated; a calling module, configured to call the corresponding atomic instruction according to the data type, calculate the data to be calculated, and obtain a vocabulary gradient table.

[0072] In some optional implementations of this embodiment, the data types include half-precision fp16, single-precision fp32, and double-precision fp64; the atomic instructions include single-precision atomic add instructions, double-precision atomic add instructions, half-precision atomic add instructions, and vector half-precision atomic add instructions.

[0073] In some optional implementations of this embodiment, the calling module includes: a first calling submodule configured to, in response to the data type being non-fp16, call an atomic instruction consistent with the data type to perform an atomic add operation, and obtain a vocabulary gradient table, wherein:

[0074] When the data type is fp32, the single-precision atomic addition instruction is called;

[0075] When the data type is fp64, calls the double-precision atomic add instruction.

[0076] In some optional implementations of this embodiment, the calling module includes: a second calling submodule configured to, in response to the data type being fp16 and the first address of the current row where the data to be calculated is located being 32-byte aligned,

[0077] Pack two adjacent half-precision elements in the data to be calculated to obtain a vector half-precision element;

[0078] Call the vector half-precision atomic add instruction to perform an atomic add operation on the vector half-precision elements to obtain the vocabulary gradient table.

[0079] In some optional implementations of this embodiment, the third calling submodule is configured to, in response to the data type being fp16 and the first address of the current row where the data to be calculated is located being 32-byte misaligned,

[0080] Zero-fill the half-precision elements in the data to be calculated to obtain the half-precision elements of the vector;

[0081] A vector half-precision atomic add instruction is called to perform an atomic add operation on the vector half-precision elements to obtain a vocabulary gradient table, wherein the vector half-precision elements include half-precision elements and 0.

[0082] In some optional implementations of this embodiment, the reverse computing device further includes a graphics processing unit (GPU), wherein the GPU includes a plurality of thread blocks, each thread block includes a plurality of threads, wherein,

[0083] The thread block is configured to process multiple elements in the index tensor and the data to be calculated;

[0084] Multiple threads in the thread block are configured to obtain data in the vocabulary gradient table to obtain the vocabulary gradient table.

[0085] In some optional implementations of this embodiment, the GPU also includes a streaming multiprocessor, and the thread blocks are scheduled to the streaming multiprocessor, where the number of streaming multiprocessors is M, the number of thread blocks is the same as the number of elements, which is N, and the number of streaming multiprocessors and the number of thread blocks satisfy N=2M.

[0086] In the technical solutions disclosed herein, the acquisition, storage, and application of user personal information involved comply with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0087] According to an embodiment of the present disclosure, the present disclosure also provides an electronic device, a readable storage medium, and a computer program product.

[0088] Figure 9 A schematic block diagram of an example electronic device 800 that can be used to implement embodiments of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital assistants, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are provided as examples only and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0089] like Figure 9 As shown, the device 800 includes a computing unit 801, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 802 or a computer program loaded from a storage unit 808 into a random access memory (RAM) 803. Various programs and data required for the operation of the device 800 can also be stored in the RAM 803. The computing unit 801, the ROM 802, and the RAM 803 are connected to each other via a bus 804. An input / output (I / O) interface 805 is also connected to the bus 804.

[0090] Various components in device 800 are connected to I / O interface 805, including an input unit 806, such as a keyboard, mouse, etc.; an output unit 807, such as various types of displays, speakers, etc.; a storage unit 808, such as a magnetic disk, optical disk, etc.; and a communication unit 809, such as a network card, modem, wireless communication transceiver, etc. The communication unit 809 allows device 800 to exchange information / data with other devices via a computer network such as the Internet and / or various telecommunication networks.

[0091] The computing unit 801 can be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the computing unit 801 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units that run machine learning model algorithms, digital signal processors (DSPs), and any appropriate processors, controllers, microcontrollers, etc. The computing unit 801 performs the various methods and processes described above, such as the reverse calculation method based on the embedded operator. For example, in some embodiments, the reverse calculation method based on the embedded operator can be implemented as a computer software program, which is tangibly contained in a machine-readable medium, such as a storage unit 808. In some embodiments, part or all of the computer program can be loaded and / or installed on the device 800 via the ROM 802 and / or the communication unit 809. When the computer program is loaded into the RAM 803 and executed by the computing unit 801, one or more steps of the reverse calculation method based on the embedded operator described above can be performed. Alternatively, in other embodiments, the computing unit 801 may be configured to execute the inverse computing method based on the embedded operator in any other appropriate manner (for example, by means of firmware).

[0092] Various embodiments of the systems and techniques described herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), system-on-chip systems (SOCs), programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments can include being implemented in one or more computer programs that are executable and / or interpreted on a programmable system that includes at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0093] The program code for implementing the method of the present disclosure can be written in any combination of one or more programming languages. These program codes can be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device so that when the program code is executed by the processor or controller, the functions / operations specified in the flow chart and / or block diagram are implemented. The program code can be executed entirely on the machine, partially on the machine, as a stand-alone software package, partially on the machine and partially on a remote machine, or entirely on a remote machine or server.

[0094] In the context of the present disclosure, a machine-readable medium can be a tangible medium that can contain or store a program for use by or in conjunction with an instruction execution system, device or equipment. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium can include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0095] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0096] The systems and techniques described herein can be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer having a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network (LAN), a wide area network (WAN), and the Internet.

[0097] A computer system may include a client and a server. The client and server are generally remote from each other and typically interact through a communication network. The client-server relationship arises through computer programs running on the respective computers and having a client-server relationship with each other. The server may be a cloud server, a server in a distributed system, or a server integrated with a blockchain.

[0098] It should be understood that the various forms of the processes shown above can be used to reorder, add, or delete steps. For example, the steps described in this disclosure can be performed in parallel, sequentially, or in a different order, as long as the desired results of the technical solutions disclosed in this disclosure can be achieved. This is not a limitation herein.

[0099] The above specific embodiments do not constitute a limitation on the scope of protection of this disclosure. Those skilled in the art will appreciate that various modifications, combinations, sub-combinations, and substitutions may be made based on design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this disclosure shall be included within the scope of protection of this disclosure.

Claims

1. A reverse calculation method based on an embedded operator, comprising: Obtain an index tensor and a table of gradients to be calculated, wherein the index tensor includes multiple elements, each element corresponds to an index number, and there are repeated index numbers in the index tensor, and each row of the table of gradients to be calculated stores data to be calculated; Determine the data type of each row of data to be calculated based on the index number, where the data type includes half precision; Calling a corresponding atomic instruction according to the data type of each row of data to be calculated, calculating the data to be calculated in the to-be-calculated gradient table to obtain a vocabulary gradient table, including: in response to the data type being half-precision and the first address of the current row where the data to be calculated is located being 32-byte misaligned, zero-padding the half-precision elements in the data to be calculated to obtain vector half-precision elements, wherein one element of the vector half-precision elements is the half-precision element and the other element is 0; calling a vector half-precision atomic add instruction in the atomic instruction to perform an atomic add operation on the vector half-precision elements to obtain the vocabulary gradient table; In response to the data type being half-precision and the first address of the current row where the data to be calculated is located being 32-byte aligned, two adjacent half-precision elements in the data to be calculated are packed to obtain a vector half-precision element; and the vector half-precision atomic add instruction is called to perform an atomic add operation on the vector half-precision elements to obtain the vocabulary gradient table.

2. The method according to claim 1, wherein The data types further include single precision and double precision; the atomic instructions further include single precision atomic add instructions, double precision atomic add instructions and half precision atomic add instructions.

3. The method according to claim 2, wherein: The calling of the corresponding atomic instruction according to the data type further comprises: In response to the data type being non-half precision, an atomic instruction consistent with the data type is called to perform an atomic addition operation to obtain the vocabulary gradient table, wherein: When the data type is single precision, the single precision atomic addition instruction is called; When the data type is double precision, the double precision atomic add instruction is called.

4. The method according to any one of claims 1 to 3, executed on a graphics processing unit (GPU), wherein: The GPU includes multiple thread blocks, each of which includes multiple threads. The data to be calculated is output according to a preset rule based on the index number, and the vocabulary gradient table obtained includes: Processing multiple elements in the index tensor and the data to be calculated by the thread block; The data in the vocabulary gradient table is obtained by using multiple threads in the thread block to obtain a vocabulary gradient table.

5. The method according to claim 4, wherein The GPU also includes a streaming multiprocessor, and the thread blocks are scheduled to the streaming multiprocessor, wherein the number of the streaming multiprocessors is M, the number of the thread blocks is the same as the number of elements, which is N, and the number of the streaming multiprocessors and the number of the thread blocks satisfy N=2M.

6. A reverse computing device based on an embedded operator, comprising: an acquisition unit configured to acquire an index tensor and a table of gradients to be calculated, wherein the index tensor includes a plurality of elements, each element corresponds to an index number, and there are repeated index numbers in the index tensor, and each row of the table of gradients to be calculated stores data to be calculated; Reverse calculation unit, including: a data type determination module, configured to determine the data type of each row of data to be calculated based on the index number, wherein the data type includes half precision; a calling module configured to call a corresponding atomic instruction according to the data type of each row of data to be calculated, calculate the data to be calculated in the to-be-calculated gradient table, and obtain a vocabulary gradient table; Wherein, the calling module includes: a third calling submodule, configured to, in response to the data type being half-precision and the first address of the current row where the data to be calculated is located being 32-byte misaligned, zero-fill the half-precision elements in the data to be calculated to obtain vector half-precision elements, one element of the vector half-precision elements being the half-precision element and the other element being 0; and calling a vector half-precision atomic add instruction in the atomic instruction to perform an atomic add operation on the vector half-precision elements to obtain the vocabulary gradient table; The calling module also includes: The second calling submodule is configured to respond to the data type being half precision and the first address of the current row where the data to be calculated is located being 32-byte aligned, Packing two adjacent half-precision elements in the data to be calculated to obtain a vector half-precision element; calling the vector half-precision atomic add instruction to perform an atomic add operation on the vector half-precision elements to obtain the vocabulary gradient table.

7. The device according to claim 6, wherein The data types further include single precision and double precision; the atomic instructions further include single precision atomic add instructions, double precision atomic add instructions and half precision atomic add instructions.

8. The device according to claim 7, wherein The calling module also includes: The first calling submodule is configured to, in response to the data type being non-half-precision, call an atomic instruction consistent with the data type to perform an atomic addition operation to obtain the vocabulary gradient table, wherein: When the data type is single precision, the single precision atomic addition instruction is called; When the data type is double precision, the double precision atomic add instruction is called.

9. The apparatus according to any one of claims 6 to 8, further comprising a graphics processor (GPU), wherein: The GPU includes a plurality of thread blocks, each of which includes a plurality of threads, wherein: The thread block is configured to process multiple elements in the index tensor and the data to be calculated; The plurality of threads in the thread block are configured to obtain data in the vocabulary gradient table to obtain a vocabulary gradient table.

10. The device according to claim 9, wherein The GPU also includes a streaming multiprocessor, and the thread blocks are scheduled to the streaming multiprocessor, wherein the number of the streaming multiprocessors is M, the number of the thread blocks is the same as the number of elements, which is N, and the number of the streaming multiprocessors and the number of the thread blocks satisfy N=2M.

11. An electronic device comprising: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 5.

12. A non-transitory computer-readable storage medium storing computer instructions, wherein: The computer instructions are used to cause the computer to execute the method according to any one of claims 1 to 5.

13. A computer program product comprising a computer program, which, when executed by a processor, implements the method according to any one of claims 1 to 5.

Images