A data processing method, device and storage medium
By blocking and storage optimization of data during GPGPU operator execution, the data overflow problem caused by insufficient TLR capacity is solved, the operator performance and data processing efficiency are improved, and more efficient data interaction and storage utilization are achieved.
Patent Information
- Application Number
- CN202410296982.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-03-15
- Publication Date
- 2024-11-01
- Estimated Expiration
- 2044-03-15
AI Technical Summary
In the development of GPGPU operators, due to the limited capacity of thread local registers (TLRs), temporary data overflows to high-speed video memory (HBM), affecting the performance and flexibility of the operator. Especially in operations such as matrix multiplication and convolution, TLR is insufficient. Causes performance degradation.
By blocking the data used during operator execution, the data is divided into multiple data blocks according to the size and processing order of different storage units, and the corresponding storage unit (such as register unit) is called according to the storage location of the data block. , the first cache unit and the second cache unit) execute operators to optimize the use of TLR and data interaction.
Improves the temporary available capacity of TLR during the calculation process of GPGPU operator algorithm, reduces the situation of data overflow to HBM, improves the operator performance and utilization of high-speed storage space, enhances the flexibility of data processing, and improves the overall data processing efficiency.
Smart Images

Figure CN118012628B_ABST
Abstract
Description
Technical Field
[0001] Embodiments of the present disclosure relate to a data processing method, a data processing device, and a storage medium, and more particularly to large-scale data processing of a general-purpose graphics processor in the field of artificial intelligence. Background Art
[0002] General-purpose computing on graphics processing units (GPGPU) is a graphics processor that uses graphics processing tasks to calculate general computing tasks that were originally handled by the central processing unit. Due to the powerful parallel processing capabilities and programmable pipelines of modern graphics processors, the processor can process non-graphic data. Especially in the field of artificial intelligence, when faced with single instruction stream multiple data streams (SIMD) and the amount of data processing is far greater than the needs of data scheduling and transmission, general-purpose graphics processors have greatly surpassed traditional central processing unit applications in performance. Summary of the invention
[0003] At least one embodiment of the present disclosure provides a data processing method, comprising: processing data used in an operator execution process in blocks to obtain multiple data blocks; determining storage units of the multiple data blocks according to a processing order of the multiple data blocks; and a computing unit calling the multiple data blocks to execute the operator according to the storage locations of the multiple data blocks.
[0004] For example, in the data processing method provided in at least one embodiment of the present disclosure, the data used in the execution of the operator is processed in blocks to obtain multiple data blocks, including: processing the data in blocks according to the size of each storage unit and the processing order of the data used in the execution of the operator.
[0005] For example, in the data processing method provided by at least one embodiment of the present disclosure, the storage unit includes a register unit, a first cache unit, and a second cache unit, and the data is processed in blocks according to the size of each storage unit and the processing order of the data used in the execution of the operator, including: when the data amount of the data is greater than the storage capacity of the register unit and the first cache unit, the data is divided into a first data block, a second data block, and a third data block; when the data amount of the data is greater than the storage capacity of the register unit and less than the storage capacity of the first cache unit, the data is divided into a first data block and a second data block.
[0006] For example, in the data processing method provided by at least one embodiment of the present disclosure, the first data block is stored in the register unit, the second data block is stored in the first cache unit, and the third data block is stored in the second cache unit.
[0007] For example, in the data processing method provided by at least one embodiment of the present disclosure, the processing order of the first data block is earlier than the processing order of the second data block, the processing order of the second data block is earlier than the processing order of the third data block, the distance between the register unit and the computing unit is smaller than the distance between the first cache unit and the computing unit, and the distance between the first cache unit and the computing unit is smaller than the distance between the second cache unit and the computing unit.
[0008] For example, in the data processing method provided by at least one embodiment of the present disclosure, when the operator is a single operator, the data processing method also includes: dividing the single operator into multiple sub-parts, and performing block processing on the data used by each sub-part to obtain multiple data blocks; determining the storage units of the multiple data blocks according to the processing order of the data of each sub-part of the operator and the capacity of the storage unit; and the computing unit calling the multiple data blocks to execute the sub-parts of the single operator according to the storage positions of the multiple data blocks.
[0009] For example, in the data processing method provided by at least one embodiment of the present disclosure, the multiple sub-parts include N sub-parts, the processing order of the nth sub-part is earlier than the processing order of the n+1th sub-part, and the computing unit calls the multiple data blocks according to the storage positions of the multiple data blocks to execute the sub-parts of the single operator, including: the computing unit uses the first data block in the register unit to execute the calculation of the nth sub-part and obtain the calculation result; according to the use time of the calculation result of the nth sub-part, the calculation result is stored in the first cache unit as the second data block or stored in the second cache unit as the third data block; according to the execution order of the single operator, the second data block is read from the first cache unit or the third data block is read from the second cache unit to the register unit for the computing unit to execute the calculation of the n+1th sub-part; wherein N is an integer greater than 1, and n is an integer greater than 0 and less than or equal to N.
[0010] For example, in the data processing method provided by at least one embodiment of the present disclosure, when the operator is a fusion operator, the data processing method also includes: splitting the fusion operator into multiple single operators, and performing block processing on the data used by each single operator to obtain multiple data blocks; determining the storage unit of the multiple data blocks based on the execution order of the single operators, the processing order of the data of the single operators and the capacity of the storage unit; the computing unit calls the multiple data blocks to execute the fusion operator according to the storage positions of the multiple data blocks.
[0011] For example, in the data processing method provided by at least one embodiment of the present disclosure, the computing unit calls the multiple data blocks to execute the operator according to the storage positions of the multiple data blocks, including: executing partial calculation of the operator based on the first data block stored in the register unit to obtain temporary data; storing the temporary data in the register unit, the first cache unit or the second cache unit based on the processing order of the temporary data and the capacity of the storage unit; reading the second data block stored in the first cache unit or the third data block stored in the second cache unit to the register unit for use by the computing unit to execute the operator.
[0012] At least one embodiment of the present disclosure further provides a data processing device, comprising: a block division unit, configured to perform block processing on data used in the execution process of an operator to obtain multiple data blocks; a determination unit, configured to determine the storage units of the multiple data blocks according to the processing order of the multiple data blocks; and a calculation unit, configured to call the multiple data blocks to execute the operator according to the storage locations of the multiple data blocks.
[0013] For example, in the data processing device provided in at least one embodiment of the present disclosure, the blocking unit is further configured to: block the data according to the size of each storage unit and the processing order of the data used in the execution process of the operator.
[0014] For example, in the data processing device provided by at least one embodiment of the present disclosure, the storage unit includes a register unit, a first cache unit and a second cache unit, and the block division unit is further configured to: when the amount of the data is greater than the storage capacity of the register unit and the first cache unit, divide the data into a first data block, a second data block and a third data block; when the amount of the data is greater than the storage capacity of the register unit and less than the first cache unit, divide the data into a first data block and a second data block.
[0015] For example, in the data processing device provided by at least one embodiment of the present disclosure, the first data block is stored in the register unit, the second data block is stored in the first cache unit, and the third data block is stored in the second cache unit.
[0016] For example, in the data processing device provided by at least one embodiment of the present disclosure, the processing order of the first data block is earlier than the processing order of the second data block, the processing order of the second data block is earlier than the processing order of the third data block, the distance between the register unit and the computing unit is smaller than the distance between the first cache unit and the computing unit, and the distance between the first cache unit and the computing unit is smaller than the distance between the second cache unit and the computing unit.
[0017] At least one embodiment of the present disclosure further provides a data processing device, comprising: a processor; a memory; and one or more computer program modules, wherein the one or more computer program modules are stored in the memory and configured to be executed by the processor, and the one or more computer program modules include instructions for executing the data processing method provided by any embodiment of the present disclosure.
[0018] At least one embodiment of the present disclosure further provides a storage medium that non-temporarily stores computer-readable instructions, and when the computer-readable instructions are executed by a computer, the data processing method provided by any embodiment of the present disclosure is executed. BRIEF DESCRIPTION OF THE DRAWINGS
[0019] In order to more clearly illustrate the technical solutions of the embodiments of the present disclosure, the drawings of the embodiments will be briefly introduced below. Obviously, the drawings in the following description only relate to some embodiments of the present disclosure, but are not intended to limit the present disclosure.
[0020] Figure 1 A schematic diagram of a data processing method provided by at least one embodiment of the present disclosure is shown.
[0021] Figure 2 A flow chart of a data processing method provided by at least one embodiment of the present disclosure is shown.
[0022] Figure 3A A flowchart of a data processing method for a single operator example provided by at least one embodiment of the present disclosure is shown.
[0023] Figure 3B A flow chart of a data processing method when the single operator is a matrix multiplication operator is shown.
[0024] Figure 4 A flowchart of a data processing method for a fusion operator example provided by at least one embodiment of the present disclosure is shown.
[0025] Figure 5 A schematic block diagram of a data processing device provided for at least one embodiment of the present disclosure.
[0026] Figure 6 A schematic block diagram of another data processing device provided for at least one embodiment of the present disclosure.
[0027] Figure 7 A schematic block diagram of an electronic device provided for at least one embodiment of the present disclosure.
[0028] Figure 8 A schematic diagram of a storage medium provided for at least one embodiment of the present disclosure. DETAILED DESCRIPTION
[0029] In order to make the purpose, technical solution and advantages of the embodiments of the present disclosure clearer, the technical solution of the embodiments of the present disclosure will be clearly and completely described below in conjunction with the drawings of the embodiments of the present disclosure. Obviously, the described embodiments are part of the embodiments of the present disclosure, not all of the embodiments. Based on the described embodiments of the present disclosure, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present disclosure.
[0030] Unless otherwise defined, the technical terms or scientific terms used in the present disclosure should be understood by people with ordinary skills in the field to which the present disclosure belongs. The "first", "second" and similar words used in the present disclosure do not indicate any order, quantity or importance, but are only used to distinguish different components. Similarly, similar words such as "one", "one" or "the" do not indicate quantity restrictions, but indicate that there is at least one. Similar words such as "include" or "comprise" mean that the elements or objects appearing before the word cover the elements or objects listed after the word and their equivalents, without excluding other elements or objects. Similar words such as "connect" or "connected" are not limited to physical or mechanical connections, but can include electrical connections, whether direct or indirect. "Up", "down", "left", "right" and the like are only used to indicate relative positional relationships. When the absolute position of the described object changes, the relative positional relationship may also change accordingly.
[0031] In the current GPGPU operator development process, three types of storage spaces are usually used: Global-Shared-Memory (GSM), Extended Thread-Local-Register (TLR), and High-Bandwidth-Memory (HBM). Among them, TLR is closest to the computing unit and has the fastest speed. It is usually used to store temporary data for operator calculations. GSM is the second fastest and is usually used to temporarily store data that needs to be used repeatedly. HBM is the outer memory of GPGPU, which is far away from the computing unit and has the slowest speed. It is usually used to store the input and output of operator calculations and all other data involved. In the current operators, especially matrix multiplication, convolution, and other operator implementations that require the interaction between the tensor computing unit (Tcore) and the vector computing unit (Vcore), TLR is usually used to store temporary interaction data between the two for performance reasons.
[0032] The inventors noticed that due to the limitation of chip area, the number of TLRs is usually very limited. In the prior art, if the temporary space required by the operator algorithm is large, especially when the interactive data between Tcore and Vcore is large, the TLR capacity may be insufficient, and the excess data may overflow and be stored on HBM, resulting in poor operator performance. When designing and implementing high-performance operators, the algorithm is strictly limited by the total number of TLRs, and the flexibility is reduced.
[0033] At least one embodiment of the present disclosure provides a data processing method, which processes data used in an operator execution process in blocks to obtain multiple data blocks; determines storage units of the multiple data blocks according to the processing order of the multiple data blocks; and a computing unit calls the multiple data blocks to execute operators according to the storage locations of the multiple data blocks.
[0034] At least one embodiment of the present disclosure also provides a data processing device and a storage medium.
[0035] The data processing method provided by at least one embodiment of the present disclosure can use a smaller data transmission overhead to increase the capacity of the temporarily available TLR during the calculation process of the GPGPU operator algorithm, reduce the situation where data overflows to the HBM due to insufficient TLR, and thus improve the overall performance of the operator; at the same time, it also improves the utilization rate of the high-speed storage space, avoids the failure of the operator algorithm, and improves the flexibility of data processing, thereby greatly improving the efficiency of data processing.
[0036] The embodiments of the present disclosure and some examples thereof are described in detail below with reference to the accompanying drawings.
[0037] Figure 1 FIG. 1 is a schematic diagram showing a data processing method provided by at least one embodiment of the present disclosure. Figure 1 As shown, the data processing method includes steps S110 to S130.
[0038] Step S110: block processing is performed on the data used in the execution of the operator to obtain multiple data blocks.
[0039] Step S120: Determine storage units of the plurality of data blocks according to the processing order of the plurality of data blocks.
[0040] Step S130: the computing unit calls multiple data block execution operators according to the storage locations of the multiple data blocks.
[0041] For step S110, for example, in some examples, the storage unit includes a register unit, a first cache unit, and a second cache unit. For example, the distance between the register unit and the computing unit is smaller than the distance between the first cache unit and the computing unit, and the distance between the first cache unit and the computing unit is smaller than the distance between the second cache unit and the computing unit.
[0042] For example, the register unit may be a TLR, which is closest to the computing unit and has a smaller capacity; the first cache unit may be a GSM, which is closer to the computing unit but farther from the computing unit than the TLR and has a smaller capacity; the second cache unit may be a vector operation unit storage (GEMM-Main-Buffer, referred to as GMB), which is closer to the computing unit but farther from the computing unit than the GSM and has a larger capacity. The following is an introduction using the register unit being a TLR, the first cache unit being a GSM, and the second cache unit being a GMB as an example, and the embodiments of the present disclosure are not limited to this.
[0043] For example, in some examples, data is processed in blocks according to the size of each storage unit and the processing order of the data used in the operator execution process to achieve the best use effect of each storage unit. For example, when the amount of data is greater than the storage capacity of the register unit and the first cache unit, the data can be divided into a first data block, a second data block, and a third data block; when the amount of data is greater than the storage capacity of the register unit and less than the storage capacity of the first cache unit, the data can be divided into a first data block and a second data block.
[0044] For example, when the amount of data is greater than the storage capacity of the register unit and the first cache unit, the first data block is stored in the register unit, the second data block is stored in the first cache unit, and the third data block is stored in the second cache unit; when the amount of data is greater than the storage capacity of the register unit but less than the storage capacity of the first cache unit, the first data block is stored in the register unit and the second data block is stored in the first cache unit; when the amount of data is less than the storage capacity of the register unit, the data can be directly stored in the register unit without being divided into blocks, thereby ensuring flexible data storage in the effective use space of each storage unit to improve the data interaction speed and improve the efficiency of data processing.
[0045] For example, the data used in the present disclosure may include initial input data of operator calculation, temporary data generated during the execution of the operator, and final output data.
[0046] For step S120, for example, the processing order of the first data block is earlier than the processing order of the second data block (i.e., the use time of the first data block is earlier than the use time of the second data block), and the processing order of the second data block is earlier than the processing order of the third data block (i.e., the use time of the second data block is earlier than the use time of the third data block).
[0047] For example, Figure 2As shown in the flowchart of , when designing the operator implementation algorithm, the use time of the operator data is first analyzed, and according to the use time (i.e., processing time) and data volume of the operator data and the generated temporary data, the data to be used in the operator algorithm (e.g., the first data block mentioned above) is stored in the TLR for direct calculation, and the data that cannot be processed in the short term and the temporary data (e.g., Figure 2 TLR data that is not used in recent calculations and TLR data that is not used in the next calculation) are placed in GSM and GMB for hierarchical temporary storage. For example, recently used temporary small data (for example, the second data block mentioned above) is stored in GSM, and larger or long-unused temporary data or temporary / large data (for example, the third data block) is stored in GMB, and the TLR space that should be occupied is released, thereby realizing the hierarchical expansion of the limited capacity of TLR using the high-speed storage space of GSM and GMB. When used subsequently, this part of the data is read from GSM and GMB to TLR for use by the computing unit, thereby reducing the demand for TLR capacity of the operator algorithm to a certain extent, and improving the performance and flexibility of the operator.
[0048] For example, in the embodiments of the present disclosure, the operator may be a single operator or a fusion operator, and the embodiments of the present disclosure are not limited to this.
[0049] For example, when the operator is a single operator, the data processing method also includes: dividing the single operator into multiple sub-parts, and performing block processing on the data used by each sub-part to obtain multiple data blocks; determining the storage units of the multiple data blocks according to the processing order of the data of each sub-part of the single operator and the capacity of the storage unit; and the computing unit calling the multiple data blocks to execute the sub-parts of the single operator according to the storage positions of the multiple data blocks.
[0050] For example, in some examples, the multiple sub-parts include N sub-parts, the processing order of the nth sub-part is earlier than the processing order of the n+1th sub-part, and the computing unit calls the multiple data blocks according to the storage positions of the multiple data blocks to execute the sub-parts of the single operator, including: the computing unit uses the first data block in the register unit to execute the calculation of the nth sub-part and obtain the calculation result; according to the use time of the calculation result of the nth sub-part, the calculation result is stored in the first cache unit as the second data block or in the second cache unit as the third data block; according to the execution order of the single operator, the second data block is read from the first cache unit or the third data block is read from the second cache unit to the register unit for the computing unit to execute the calculation of the n+1th sub-part; N is an integer greater than 1, and n is an integer greater than 0 and less than or equal to N.
[0051] Figure 3A A flowchart of a data processing method for a single operator example provided by at least one embodiment of the present disclosure is shown. Figure 3AAs shown, for a single operator, the calculation process of the operator can usually be divided into multiple steps (i.e., multiple sub-parts), each of which has its own data dependency and data output. For example, the following takes N=3 as an example, that is, the operator includes a first sub-part 1 (e.g., corresponding to the nth sub-part), a second sub-part 2 (e.g., corresponding to the n+1th sub-part), and a third sub-part. It can also include more or fewer sub-parts, depending on the actual situation, and the embodiments of the present disclosure do not limit this. For example, Figure 3A As shown, the first sub-part 1 of the operator calculates some temporary data (i.e., the calculation result of the first sub-part 1), which needs to be used by the third sub-part. Therefore, this part of the temporary data (i.e., the calculation result of the first sub-part 1) can be temporarily stored from the TLR to GSM or GMB according to its use time and capacity (big data or data not used recently is temporarily stored in GMB, and vice versa stored in GSM), and then the TLR resources are released for the second sub-part 2 to calculate and use. After the second sub-part is calculated, this part of the data (i.e., the calculation result of the first sub-part 1) is taken out to the TLR according to its use time for subsequent use. This makes it so that this part of the data will not occupy the TLR for a long time, avoiding algorithm failure or data overflow caused by insufficient TLR. It should be noted that the second sub-part 2 and the third sub-part can be different positions of a parallel processing algorithm. In other words, a calculation that can be parallelized and requires a large TLR space can be divided into blocks, occupying less TLR resources each time, and completed in multiple times.
[0052] Figure 3B The flowchart of the data processing method when the single operator is a matrix multiplication operator is shown below. Figure 3B , and a detailed introduction is given by taking the single operator as the matrix multiplication (MMA) operator as an example.
[0053] It should be noted that the single operator is not limited to the matrix multiplication operator, but may be other operators, and the embodiments of the present disclosure do not impose any restrictions on this.
[0054] In the matrix multiplication operator, it is usually necessary to split the entire matrix multiplication operator into the multiplication and addition calculations of multiple matrix blocks. The tensor calculation unit (Tcore) calculates the partial sum of the matrix blocks, and then transmits the partial sum of each matrix block to multiple vector calculation units (Vcore), and completes the accumulation in the TLR therein. When the Vcore receives the partial sum data, for certain scales of matrix multiplication blocks, it requires more temporary partial sum data, which may result in insufficient number of TLRs, making the algorithm of this scale of blocks unusable, affecting the flexibility and performance of the operator.
[0055] For this situation, in the data processing method provided in the embodiment of the present disclosure, for example, Figure 3BAs shown, in the calculation process of the matrix multiplication operator, it is assumed that the matrix is divided into 8 blocks [1, 8] for accumulation, and accumulation is required for multiple times, but the number of TLRs can only store the data of the next 4 blocks [1, 4] (for example, corresponding to the first data block mentioned above). At this time, the operator first temporarily stores the temporary data of the second half [5, 8] (for example, the second data block mentioned above) in the corresponding GSM (if the GSM capacity is still insufficient, the excess part (for example, the third data block mentioned above) is temporarily stored in the GMB), and uses the TLR to complete the accumulation of the first half [1, 4] data (for example, the first data block mentioned above) to obtain the calculation result. After the data of the first half [1, 4] completes one accumulation (for example, corresponding to the calculation of the nth sub-part), its calculation result is temporarily stored in the GSM or GMB, and the data of the second half [5, 8] is read from the GSM or GMB to the TLR for one accumulation (for example, corresponding to the calculation of the n+1th sub-part). This cycle continues, and the front and rear parts of the data are accumulated alternately using TLR resources to obtain the final calculation result and output it.
[0056] It should be noted that since GSM is physically distributed in each Vcore, it has a shorter transmission path and less overhead than GMB which is physically present in a stream processor cluster (SPC, including multiple vector computing units and one tensor computing unit). Therefore, GSM will be used for storage in the data processing method provided in the embodiment of the present disclosure.
[0057] It should be noted that the above "4 blocks" and "8 blocks" are for the purpose of helping to illustrate the above embodiments of the present disclosure. The actual number can be flexibly configured according to the characteristics of the operator, the capacity of TLR, GSM and GMB, and the embodiments of the present disclosure do not limit this.
[0058] For example, in other examples, when the operator is a fusion operator, the data processing method also includes: splitting the fusion operator into multiple single operators, and processing the data used by each single operator in blocks to obtain multiple data blocks; determining the storage units of the multiple data blocks based on the execution order of each single operator, the processing order of the data of each single operator and the capacity of the storage unit; the computing unit calls the multiple data blocks to execute the fusion operator according to the storage positions of the multiple data blocks.
[0059] Figure 4 A flowchart of a data processing method for a fusion operator example provided by at least one embodiment of the present disclosure is shown. Figure 4 , the data processing method of the fusion operator is introduced in detail.
[0060] For the fusion operator, the overall calculation process can be split into multiple single operators (for example, Figure 4 The operator 1 and operator 2 shown in the figure may include more operators, which are not limited in the embodiments of the present disclosure. Each operator has its own data dependency and data output. For example, the calculation result of operator 1 needs to be used by operator 2. Under normal circumstances, this part of the data is stored in the TLR. When the TLR capacity is insufficient, it overflows to the HBM. Since the HBM is far away from the computing unit, the performance is poor.
[0061] For example, Figure 4 As shown, in the embodiment of the present disclosure, GSM and GMB can be hierarchically mapped and logically regarded as additional TLRs. That is, the use time of the operator data is analyzed. When the TLR capacity is insufficient to store the interactive data of the fusion operator, the excess part (i.e., the second data block) is distributedly stored in the GSM with higher internal bandwidth of each computing unit (Compute Unit, referred to as CU, the function of which is to perform vector calculations and cache related data). At this time, if the GSM capacity is still insufficient, or the GSM capacity has been used during the calculation process of operator 1, the excess temporary data (i.e., the third data block) is stored in the GMB of the SPC where it is located, thereby expanding the capacity of the TLR in terms of use logic to achieve a fusion operator with a larger data interaction granularity.
[0062] For step S130, the computing unit calls multiple data blocks to execute operators according to the storage locations of the multiple data blocks, including: executing partial calculations of the operator based on the first data block stored in the register unit to obtain temporary data; storing the temporary data in the register unit, the first cache unit or the second cache unit based on the processing order of the temporary data and the capacity of the storage unit; reading the second data block stored in the first cache unit or the third data block stored in the second cache unit to the register unit for use by the computing unit to execute the operator.
[0063] For example, see above Figure 2-Figure 4 According to the introduction, the data to be used in the operator algorithm (for example, the first data block mentioned above) is stored in the TLR for direct calculation to obtain temporary data. According to the use time of the temporary data and the capacity of the TLR, GSM and GMB (i.e., storage unit), it can be determined whether to store it in the TLR, GSM or GMB. When it is used subsequently, the stored data is read from the GSM and GMB to the TLR again for use by the calculation unit to obtain the calculation result, thereby improving the utilization rate of the high-speed storage space.
[0064] It should be noted that, in the embodiments of the present disclosure, the process of the data processing method provided by the above-mentioned various embodiments of the present disclosure may include more or fewer operations, and these operations may be performed sequentially or in parallel. Although the process of the data processing method described above includes multiple operations that appear in a specific order, it should be clearly understood that the order of multiple operations is not limited. The data processing method described above can be executed once or multiple times according to predetermined conditions.
[0065] The above-mentioned data processing method provided by the present disclosure can use a smaller data transmission overhead to increase the capacity of the temporarily available TLR during the calculation process of the GPGPU operator algorithm, reduce the situation where data overflows to the HBM due to insufficient TLR, thereby improving the overall performance of the operator; at the same time, it also improves the utilization rate of the high-speed storage space, avoids the failure of the operator algorithm, and improves the flexibility of data processing, thereby greatly improving the efficiency of data processing.
[0066] Figure 7 A schematic block diagram of a data processing device provided in at least one embodiment of the present disclosure. Figure 7 In the example shown, the data processing device 100 includes a block unit 110, a determination unit 120, and a calculation unit 130. For example, these units can be implemented by hardware (e.g., circuit) modules or software modules, etc. The following embodiments are the same and will not be repeated. For example, these units can be implemented by a central processing unit (CPU), a general-purpose graphics processing unit (GPGPU), an image processing unit (GPU), a tensor processing unit (TPU), a field programmable gate array (FPGA), or other forms of processing units with data processing capabilities and / or instruction execution capabilities and corresponding computer instructions. For example, in an embodiment of the present disclosure, these units can be implemented by a general-purpose graphics processing unit (GPGPU) in the above-mentioned embodiments.
[0067] The block unit 110 is configured to process the data used in the operator execution process in blocks to obtain multiple data blocks. For example, in some examples, the block unit 110 is further configured to process the data in blocks according to the size of each storage unit and the processing order of the data used in the operator execution process.
[0068] For example, the storage unit includes a register unit, a first cache unit, and a second cache unit, and the block unit 110 is also configured to: when the amount of data is greater than the storage capacity of the register unit and the first cache unit, divide the data into a first data block, a second data block, and a third data block; when the amount of data is greater than the storage capacity of the register unit and less than the first cache unit, divide the data into a first data block and a second data block.
[0069] For example, the first data block is stored in the register unit, the second data block is stored in the first cache unit, and the third data block is stored in the second cache unit.
[0070] For example, the processing order of the first data block is earlier than the processing order of the second data block, the processing order of the second data block is earlier than the processing order of the third data block, the distance between the register unit and the computing unit is smaller than the distance between the first cache unit and the computing unit, and the distance between the first cache unit and the computing unit is smaller than the distance between the second cache unit and the computing unit.
[0071] For example, the block unit 110 may implement step S110. The specific implementation method may refer to the relevant description of step S110, which will not be repeated here.
[0072] The determination unit 120 is configured to determine the storage units of the multiple data blocks according to the processing order of the multiple data blocks. For example, the determination unit 120 can implement step S120, and its specific implementation method can refer to the relevant description of step S120, which will not be repeated here.
[0073] The calculation unit 130 is configured to call multiple data block execution operators according to the storage locations of the multiple data blocks. For example, the calculation unit 130 can implement step S130, and its specific implementation method can refer to the relevant description of step S130, which will not be repeated here.
[0074] Figure 6 A schematic block diagram of another data processing device provided by at least one embodiment of the present disclosure. Figure 6 As shown, the data processing device 200 includes a processor 210 , a memory 220 , and one or more computer program modules 221 .
[0075] For example, the processor 210 is connected to the memory 220 via a bus system 230. For example, one or more computer program modules 221 are stored in the memory 220. For example, one or more computer program modules 221 include instructions for executing the data processing method provided by any embodiment of the present disclosure. For example, the instructions in one or more computer program modules 221 can be executed by the processor 210. For example, the bus system 230 can be a commonly used serial or parallel communication bus, etc., and the embodiments of the present disclosure are not limited to this.
[0076] For example, the processor 210 may be a central processing unit (CPU), a digital signal processor (DSP), a graphics processor (GPU), a general purpose graphics processor (GPGPU), or other forms of processing units with data processing capabilities and / or instruction execution capabilities, and may be a general purpose processor or a dedicated processor, and may control other components in the data processing device 100 to perform desired functions. The embodiments of the present disclosure are described by taking the processor as a general purpose graphics processor (GPGPU) as an example.
[0077] The memory 220 may include one or more computer program products, which may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. The volatile memory may include, for example, a random access memory (RAM) and / or a cache memory (cache), etc. The non-volatile memory may include, for example, a read-only memory (ROM), a hard disk, a flash memory, etc. One or more computer program instructions may be stored on a computer-readable storage medium, and the processor 210 may run the program instructions to implement the functions (implemented by the processor 210) in the embodiment of the present disclosure and / or other desired functions, such as a data processing method, etc. Various applications and various data, such as a first data block, a second data block, and a third data block, as well as various data used and / or generated by an application, etc., may also be stored in the computer-readable storage medium.
[0078] It should be noted that, for the sake of clarity and brevity, the embodiments of the present disclosure do not provide all the components of the data processing device 200. To implement the necessary functions of the data processing device 200, those skilled in the art may provide and set other components not shown according to specific needs, and the embodiments of the present disclosure are not limited to this.
[0079] The data processing method or device according to the embodiment of the present disclosure can also be used as follows Figure 7 The exemplary electronic device 3000 shown in FIG. Figure 7As shown, the electronic device 3000 may include a bus 3010, one or more central processing units (CPUs) or graphics processing units (GPUs) or GPGPUs 3020, a read-only memory (ROM) 3030, a random access memory (RAM) 3040, a communication port 3050 connected to a network, an input / output component 3060, a hard disk 3070, and the like. The storage device in the electronic device 3000, such as the ROM 3030, the hard disk 3070, or the RAM memory (i.e., video memory) inside the GPGPU itself, may store various data or files required for the processing and / or communication of the method provided in the present disclosure, as well as program instructions executed by the CPU or GPU or GPGPU. The electronic device 3000 may also include a user interface 3080. Of course, Figure 7 The architecture shown is only exemplary and can be omitted according to actual needs when implementing different devices. Figure 7 One or more components of an electronic device are shown.
[0080] At least one embodiment of the present disclosure further provides a storage medium. Figure 8 A schematic diagram of a storage medium provided by at least one embodiment of the present disclosure. Figure 8 As shown, the storage medium 400 non-temporarily stores computer-readable instructions 401 , and when the non-temporary computer-readable instructions are executed by a computer (including a processor), the data processing method provided by any embodiment of the present disclosure can be executed.
[0081] For example, the storage medium may be any combination of one or more computer-readable storage media, such as a computer-readable storage medium containing a computer-readable program code for processing the data used in the execution of the operator in blocks to obtain a plurality of data blocks, another computer-readable storage medium containing a computer-readable program code for determining the storage unit of the plurality of data blocks according to the processing order of the plurality of data blocks, and another computer-readable storage medium containing a computer-readable program code for a computing unit to call the plurality of data blocks to execute the operator according to the storage locations of the plurality of data blocks. For example, when the program code is read by a computer, the computer may execute the program code stored in the computer storage medium, and execute, for example, the data processing method provided in any embodiment of the present disclosure.
[0082] For example, the storage medium may include a memory card of a smart phone, a storage component of a tablet computer, a hard disk of a personal computer, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM), a portable compact disk read-only memory (CD-ROM), flash memory, or any combination of the above storage media, or other applicable storage media.
[0083] Regarding the above public content, there are a few points that need to be explained:
[0084] (1) The drawings of the embodiments of the present disclosure only relate to the structures involved in the embodiments of the present disclosure. Other structures may refer to the general design.
[0085] (2) In the absence of conflict, the embodiments of the present disclosure and the features therein may be combined with each other to obtain new embodiments.
[0086] The above description is only a specific implementation of the present disclosure, but the protection scope of the present disclosure is not limited thereto. The protection scope of the present disclosure shall be based on the protection scope of the claims.
Claims
1. A data processing method, comprising: The data used in the execution of the operator is processed in blocks to obtain multiple data blocks; Determine storage units of the multiple data blocks according to a processing order of the multiple data blocks, wherein the storage unit storing the data blocks with an earlier processing order has a smaller distance from the computing unit; wherein the multiple data blocks are executed serially according to the processing order, the storage unit comprises a register unit and at least one cache unit, the multiple data blocks comprise a first data block and a second data block, wherein the first data block is stored in the register unit, the second data block is stored in the at least one cache unit, the processing order of the first data block is earlier than the processing order of the second data block, and the distance between the register unit and the computing unit is smaller than the distance between the at least one cache unit and the computing unit; The computing unit calls the multiple data blocks to execute the operator according to the storage locations of the multiple data blocks, wherein after completing the processing of the first data block, the second data block stored in the at least one cache unit is read to the register unit for use by the computing unit to execute the operator.
2. The data processing method according to claim 1, wherein: The data used in the operator execution process is divided into blocks to obtain multiple data blocks, including: The data is processed in blocks according to the sizes of the storage units corresponding to the multiple data blocks and the processing order of the data used in the execution process of the operator.
3. The data processing method according to claim 2, wherein: The at least one cache unit includes a first cache unit and a second cache unit, The data is processed in blocks according to the sizes of the storage units corresponding to the multiple data blocks and the processing order of the data used in the execution process of the operator, including: When the amount of the data is greater than the storage capacity of the register unit and the first cache unit, dividing the data into the first data block, the second data block and the third data block; When the amount of the data is greater than the storage capacity of the register unit and less than the storage capacity of the first cache unit, the data is divided into the first data block and the second data block.
4. The data processing method according to claim 3, wherein: The second data block is stored in the first cache unit, and the third data block is stored in the second cache unit.
5. The data processing method according to claim 3, wherein: The processing order of the second data block is earlier than the processing order of the third data block, the distance between the register unit and the computing unit is smaller than the distance between the first cache unit and the computing unit, and the distance between the first cache unit and the computing unit is smaller than the distance between the second cache unit and the computing unit.
6. The data processing method according to claim 3, wherein: When the operator is a single operator, the data processing method further includes: Dividing the single operator into a plurality of sub-parts, and performing block processing on the data used by each sub-part to obtain a plurality of data blocks; Determining storage units of the plurality of data blocks according to a processing order of data of each sub-part of the single operator and a capacity of the storage unit; The computing unit calls the multiple data blocks according to the storage locations of the multiple data blocks to execute the respective sub-parts of the single operator.
7. The data processing method according to claim 6, wherein: The plurality of sub-parts include N sub-parts, the processing order of the nth sub-part is earlier than the processing order of the n+1th sub-part, The computing unit calls the multiple data blocks to execute the sub-parts of the single operator according to the storage locations of the multiple data blocks, including: The calculation unit uses the first data block in the register unit to perform calculation of the nth sub-part and obtain a calculation result; According to the use time of the calculation result of the nth sub-part, storing the calculation result in the first cache unit as the second data block or storing in the second cache unit as the third data block; According to the execution order of the single operator, read the second data block from the first cache unit or read the third data block from the second cache unit to the register unit for the calculation unit to perform the calculation of the n+1th sub-part; Wherein, N is an integer greater than 1, and n is an integer greater than 0 and less than or equal to N.
8. The data processing method according to claim 1, wherein: When the operator is a fusion operator, the data processing method further includes: Splitting the fusion operator into multiple single operators, and performing block processing on the data used by each single operator to obtain multiple data blocks; Determining storage units of the plurality of data blocks based on an execution order of the single operators, a processing order of data of the single operators, and a capacity of the storage unit; The computing unit calls the multiple data blocks to execute the fusion operator according to the storage locations of the multiple data blocks.
9. The data processing method according to claim 3, wherein: The computing unit calls the multiple data blocks to execute the operator according to the storage locations of the multiple data blocks, including: Performing partial calculation of the operator based on the first data block stored in the register unit to obtain temporary data; storing the temporary data in the register unit, the first cache unit or the second cache unit based on the processing order of the temporary data and the capacity of the storage unit; The second data block stored in the first cache unit or the third data block stored in the second cache unit is read to the register unit for use by the computing unit to execute the operator.
10. A data processing device, comprising: A block division unit configured to perform block processing on the data used in the execution process of the operator to obtain a plurality of data blocks; A determination unit is configured to determine storage units of the multiple data blocks according to a processing order of the multiple data blocks, wherein the storage unit storing the data blocks with an earlier processing order has a smaller distance from the computing unit; wherein the multiple data blocks are executed serially according to the processing order, the storage unit comprises a register unit and at least one cache unit, the multiple data blocks comprise a first data block and a second data block, wherein the first data block is stored in the register unit, the second data block is stored in the at least one cache unit, the processing order of the first data block is earlier than the processing order of the second data block, and the distance between the register unit and the computing unit is smaller than the distance between the at least one cache unit and the computing unit; The computing unit is configured to call the multiple data blocks to execute the operator according to the storage locations of the multiple data blocks, wherein after completing the processing of the first data block, the second data block stored in the at least one cache unit is read to the register unit for use by the computing unit to execute the operator.
11. The data processing device according to claim 10, wherein: The block division unit is further configured as: The data is processed in blocks according to the sizes of the storage units corresponding to the multiple data blocks and the processing order of the data used in the execution process of the operator.
12. The data processing device according to claim 11, wherein: The at least one cache unit includes a first cache unit and a second cache unit, The block division unit is further configured as: When the amount of the data is greater than the storage capacity of the register unit and the first cache unit, dividing the data into the first data block, the second data block and the third data block; When the amount of the data is greater than the storage capacity of the register unit and less than the storage capacity of the first cache unit, the data is divided into the first data block and the second data block. 13 . The data processing device according to claim 12 , wherein the second data block is stored in the first cache unit, and the third data block is stored in the second cache unit.
14. The data processing apparatus according to claim 12, wherein: The processing order of the second data block is earlier than the processing order of the third data block, the distance between the register unit and the computing unit is smaller than the distance between the first cache unit and the computing unit, and the distance between the first cache unit and the computing unit is smaller than the distance between the second cache unit and the computing unit.
15. A data processing device, comprising: processor; Memory; One or more computer program modules, wherein the one or more computer program modules are stored in the memory and configured to be executed by the processor, and the one or more computer program modules include instructions for executing the data processing method described in any one of claims 1-9.
16. A storage medium that non-temporarily stores computer-readable instructions, which, when executed by a computer, executes the data processing method according to any one of claims 1 to 9.
