Tensor data processing method and device, equipment and medium
By introducing input cache into the tensor computing core, the problem of bit width limitation of register heap data is solved, and a larger-scale tensor data processing is achieved and computational efficiency is improved.
Patent Information
- Application Number
- CN202411173408.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-26
- Publication Date
- 2024-12-17
AI Technical Summary
The tensor computing core cannot process tensor data on a large scale due to the limited bit width of the register heap data, which limits its single operation scale and efficiency.
Introduce input cache in the tensor computing core to store the pending data. When obtaining tensor data processing tasks and calculation instructions, data is stored in the input cache, and only when all data is stored in the cache Processing, data processing is performed using multiplier array and accumulator.
Through the input cache mechanism, the single-time operation data scale of the tensor computing core is significantly increased, avoiding the register bit width limitation, and greatly improving operation efficiency.
Smart Images
Figure CN119148977A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of data processing technology, and in particular to a tensor data processing method, device, equipment and medium. Background Art
[0002] The tensor computing core is a hardware unit in the general purpose graphics processing unit (GPGPU), which is designed to accelerate tensor computing in deep learning and machine learning. It consists of multiple multiplication and accumulation units, can efficiently perform operations such as matrix multiplication and convolution, and provides high computing density, low power consumption and high performance. It is widely used in deep learning, machine learning and scientific computing.
[0003] At present, the tensor data processing process based on GPGPU mainly reads instructions from the register file for convolution calculation. However, due to the limited data bit width of the register file in GPGPU, it is impossible to store a large amount of data, resulting in the limitation of the single operation scale of the tensor computing core.
[0004] In view of the above problems, how to solve the problem that the current single operation scale of the tensor computing core is limited by the data bit width of the register stack and cannot calculate and process tensor data on a large scale is an urgent problem that needs to be solved by technicians in this field. Summary of the invention
[0005] The purpose of this application is to provide a tensor data processing method, device, equipment and medium to solve the problem that the current single operation scale of the tensor computing core is limited by the data bit width of the register stack and cannot calculate and process tensor data on a large scale.
[0006] In order to solve the above technical problems, the present application provides a tensor data processing method, which is applied to a tensor computing core including a controller, a register file, an accumulator, an input cache and a multiplier array; the method comprises:
[0007] Obtaining a tensor data processing task and obtaining a computing instruction corresponding to the tensor data processing task; wherein the tensor data processing task at least includes the number of threads, the number of thread bundles, the number of registers, and the number of instructions required for this tensor data processing;
[0008] Acquire data to be processed, and transfer the data to be processed to the input buffer;
[0009] Determining whether all the data to be processed are stored in the input buffer according to the calculation instruction;
[0010] If not, return to the step of obtaining the data to be processed and transferring the data to be processed to the input buffer;
[0011] If so, the data to be processed is processed by the multiplier array and the accumulator to obtain a tensor data processing result.
[0012] On the one hand, the obtaining of the tensor data processing task and obtaining the computing instruction corresponding to the tensor data processing task includes:
[0013] Receiving the tensor data processing task sent by the host end;
[0014] Determining whether a tensor data processing condition is currently met according to the number of threads, the number of thread warps, and the number of registers required for the current tensor data processing in the tensor data processing task;
[0015] If it is confirmed that there is no condition for tensor data processing at present, then end;
[0016] If it is confirmed that the tensor data processing conditions are currently met, the calculation instructions corresponding to the tensor data processing task are obtained from the memory.
[0017] On the other hand, judging whether the tensor data processing condition is currently met according to the number of threads, the number of thread warps, and the number of registers required for the tensor data processing in the tensor data processing task includes:
[0018] Determine whether the number of currently idle thread warps is not less than the number of thread warps required for the current tensor data processing in the tensor data processing task;
[0019] If it is confirmed that the number of currently idle thread warps is less than the number of thread warps required for the current tensor data processing in the tensor data processing task, it is confirmed that the tensor data processing condition is not met at present;
[0020] If it is confirmed that the number of currently idle thread warps is not less than the number of thread warps required for the current tensor data processing in the tensor data processing task, then determining whether the number of idle threads in the current thread warp is not less than the number of threads required for the current tensor data processing in the tensor data processing task;
[0021] If it is confirmed that the number of idle threads in the current thread bundle is less than the number of threads required for the current tensor data processing in the tensor data processing task, it is confirmed that the tensor data processing condition is not met at present;
[0022] If it is confirmed that the number of idle threads in the current thread bundle is not less than the number of threads required for the current tensor data processing in the tensor data processing task, then determining whether the number of registers in the current register file is not less than the number of registers required for the current tensor data processing in the tensor data processing task;
[0023] If it is confirmed that the number of registers in the current register file is less than the number of registers required for the current tensor data processing in the tensor data processing task, it is confirmed that the tensor data processing condition is not met at present;
[0024] If it is confirmed that the number of registers in the current register stack is not less than the number of registers required for the current tensor data processing in the tensor data processing task, it is confirmed that the tensor data processing conditions are currently met.
[0025] On the other hand, the obtaining of the data to be processed and transferring the data to be processed to the input buffer includes:
[0026] Decoding the computing instruction to generate a control signal; wherein the control signal includes target register information of the register where the data to be processed is located;
[0027] Selecting a corresponding vector register in the register file according to the target register information;
[0028] Obtaining feature data, weight data and bias data in the vector register;
[0029] The feature data, weight data and bias data are respectively transmitted to the input buffer.
[0030] On the other hand, judging whether all the to-be-processed data are stored in the input buffer according to the calculation instruction includes:
[0031] Acquire the total amount of all the data to be processed contained in the control signal;
[0032] Determining whether the amount of the to-be-processed data in the input buffer is equal to the total amount of data;
[0033] If not, confirming that all the data to be processed are not stored in the input buffer;
[0034] If so, it is confirmed that all the data to be processed are stored in the input buffer.
[0035] On the other hand, the processing of the data to be processed by the multiplier array and the accumulator to obtain a tensor data processing result includes:
[0036] Performing convolution calculation on the feature data and the weight data through the multiplier-adder array to obtain a convolution calculation result;
[0037] The convolution calculation result and the bias data are added by the accumulator to obtain the tensor data processing result.
[0038] On the other hand, the control signal further includes register information of a target vector register for storing the tensor data processing result; correspondingly, after obtaining the tensor data processing result, it also includes:
[0039] Determine the target vector register in the register file according to the register information in the control signal;
[0040] The tensor data processing result is stored in the target vector register.
[0041] In order to solve the above technical problems, the present application also provides a tensor data processing device, which is applied to a tensor computing core including a controller, a register file, an accumulator, an input cache and a multiplier array; the device includes:
[0042] An acquisition module, used to acquire a tensor data processing task and acquire a computing instruction corresponding to the tensor data processing task; wherein the tensor data processing task at least includes the number of threads, the number of thread bundles, the number of registers, and the number of instructions required for this tensor data processing;
[0043] A transmission module, used for acquiring data to be processed and transmitting the data to be processed to the input buffer;
[0044] A judgment module, used for judging whether all the data to be processed are stored in the input buffer according to the calculation instruction; if not, triggering the transmission module; if yes, triggering the processing module;
[0045] The processing module is used to process the data to be processed through the multiplier array and the accumulator to obtain a tensor data processing result.
[0046] In order to solve the above technical problems, the present application also provides a tensor data processing device, including:
[0047] Memory for storing computer programs;
[0048] A processor is used to implement the steps of the above-mentioned tensor data processing method when executing the computer program.
[0049] In order to solve the above technical problems, the present application also provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the above tensor data processing method are implemented.
[0050] The tensor data processing method provided in the present application is applied to a tensor computing core including a controller, a register stack, an accumulator, an input cache, and a multiplier array. The method specifically obtains a tensor data processing task and obtains the computing instructions corresponding to the tensor data processing task; wherein the tensor data processing task at least includes the number of threads, the number of thread bundles, the number of registers, and the number of instructions required for this tensor data processing; obtains the data to be processed, and transfers the data to be processed to the input cache; determines whether all the data to be processed are stored in the input cache according to the computing instructions; if not, returns to the step of obtaining the data to be processed and transferring the data to be processed to the input cache; if so, processes the data to be processed through the multiplier array and the accumulator to obtain the tensor data processing result. It can be seen that this scheme adds an input cache in the tensor computing core to store the data to be processed in tensor data processing. When a tensor data processing task and its corresponding computing instructions are obtained and the tensor data calculation is ready to be executed, the data to be processed is stored in the input cache; and only when all the data to be processed are stored in the input cache, the data to be processed will be sent to the multiplier array and accumulator to execute the tensor data processing process, so that the single operation data of the tensor computing core is significantly increased, and it will not be limited by the register bit width, which greatly improves the operating efficiency of the tensor computing core.
[0051] In addition, the present application also provides a tensor data processing device, equipment and medium, with the same effects as above. BRIEF DESCRIPTION OF THE DRAWINGS
[0052] In order to more clearly illustrate the embodiments of the present application, the following is a brief introduction to the drawings required for use in the embodiments. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.
[0053] Figure 1 An architecture diagram of a tensor computing core provided in an embodiment of the present application;
[0054] Figure 2 A flowchart of a tensor data processing method provided in an embodiment of the present application;
[0055] Figure 3 A schematic diagram of a tensor data processing device provided in an embodiment of the present application;
[0056] Figure 4 A schematic diagram of a tensor data processing device provided in an embodiment of the present application. DETAILED DESCRIPTION
[0057] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.
[0058] The core of this application is to provide a tensor data processing method, device, equipment and medium to solve the problem that the current single operation scale of the tensor computing core is limited by the data bit width of the register stack and cannot calculate and process tensor data on a large scale.
[0059] In order to enable those skilled in the art to better understand the present application, the present application is further described in detail below in conjunction with the accompanying drawings and specific implementation methods.
[0060] At present, the tensor data processing process based on GPGPU mainly reads instructions from the register file for convolution calculation. However, due to the limited data bit width of the register file in GPGPU, it is impossible to store a large amount of data to be processed, resulting in a small single operation scale of the tensor computing core, which reduces the computing efficiency of GPGPU. This problem limits the scale and performance of tensor computing. In order to solve the above problems, the present application provides a tensor data processing method. It is applied to an improved tensor computing core.
[0061] Figure 1 This is an architecture diagram of a tensor computing core provided in an embodiment of the present application. Figure 1 As shown, the tensor computing core includes a controller, a register stack, an accumulator, an input cache, and a multiplier array. The controller is used to determine the number of times the register stack needs to send data in this calculation according to the amount of data to be processed and the amount of data that the register can store at a time; the register stack is used to store the data to be processed, mainly including feature data, weight data, and bias data, and read out the corresponding data according to the decoding information of the calculation instruction; the input cache is used to cache the data to be processed, especially the data that needs to be reused for multiple calculations; the multiplier array is used to perform convolution operations on the feature data and the weight data, and the result is then added to the bias data through the accumulator to obtain the final output. The following is a specific description of the tensor data processing method that can be implemented by the tensor computing core in combination with the specific process.
[0062] Figure 2 A flowchart of a tensor data processing method provided in an embodiment of the present application. Figure 2 As shown, the method includes:
[0063] S10: Obtain a tensor data processing task, and obtain a computing instruction corresponding to the tensor data processing task.
[0064] The tensor data processing task at least includes the number of threads, the number of thread bundles, the number of registers, and the number of instructions required for this tensor data processing.
[0065] In a specific implementation, the GPGPU system is powered on and reset, and the tensor computing core therein is in an initial state. At this time, the tensor data processing task is obtained, and the computing instructions corresponding to the tensor data processing task are obtained. It should be noted that the tensor data processing task at least includes the number of threads, the number of thread bundles, the number of registers, and the number of instructions required for this tensor data processing. When the tensor data processing is subsequently executed, it needs to be executed based on the various information in the tensor data processing task. In this embodiment, there is no restriction on the method of obtaining the tensor data processing task and its corresponding computing instructions, which depends on the specific implementation situation.
[0066] S11: Obtain the data to be processed and transfer the data to be processed to the input buffer.
[0067] Further, the data to be processed in this tensor data processing is obtained, and the data to be processed is transferred to the input buffer. It can be understood that the data to be processed mainly includes feature data, weight data and bias data. In this embodiment, there is no restriction on the method of obtaining the data to be processed.
[0068] S12: Determine whether all the data to be processed are stored in the input buffer according to the calculation instruction; if not, return to step S11; if yes, proceed to step S13.
[0069] S13: Process the data to be processed through the multiplier array and the accumulator to obtain a tensor data processing result.
[0070] In order to process multiple data to be processed at the same time and improve the operating efficiency of the tensor computing core, it is also necessary to determine whether all the data to be processed are stored in the input cache according to the calculation instructions.
[0071] If some of the data to be processed is not stored in the input cache, it is necessary to return to step S11 to reacquire the data to be processed and transfer the data to be processed to the input cache. If all the data to be processed are stored in the input cache, it is considered that the calculation can be performed at present, and the data to be processed is processed by the multiplier array and the accumulator to obtain the tensor data processing result. In this embodiment, there is no restriction on the calculation process of the tensor data processing result, which depends on the specific implementation situation.
[0072] In this embodiment, an input cache is newly added to the tensor computing core to store the data to be processed in tensor data processing. When the tensor data processing task and its corresponding computing instructions are obtained and the tensor data calculation is ready to be executed, the data to be processed is stored in the input cache; and only when all the data to be processed are stored in the input cache, the data to be processed will be sent to the multiplier array and accumulator to execute the tensor data processing process, so that the single operation data of the tensor computing core is significantly increased, and it will not be limited by the register bit width, which greatly improves the operation efficiency of the tensor computing core.
[0073] Based on the above embodiment, at runtime, obtaining a tensor data processing task and obtaining a computing instruction corresponding to the tensor data processing task include:
[0074] S101: receiving a tensor data processing task sent by a host;
[0075] S102: judging whether the current tensor data processing conditions are met according to the number of threads, the number of thread bundles and the number of registers required for the current tensor data processing in the tensor data processing task; if not, ending; if yes, proceeding to step S103;
[0076] S103: Obtain computing instructions corresponding to the tensor data processing task from the memory.
[0077] In order to obtain tensor data processing tasks and their corresponding computing instructions, in this embodiment, the tensor data processing tasks sent by the host are specifically received. It should be noted that in the process of processing tensor data, the host is responsible for dispatching tensor data processing tasks to the slave, and the slave GPGPU processes the tensor data processing tasks.
[0078] Furthermore, according to the number of threads, the number of thread bundles and the number of registers required for the current tensor data processing in the tensor data processing task, it is determined whether the current tensor data processing conditions are met. If it is determined that the current tensor data processing conditions are not met, the current tensor data processing process is terminated; if it is determined that the current tensor data processing conditions are met, the calculation instructions corresponding to the tensor data processing task are obtained from the memory.
[0079] It should be noted that the specific process of determining whether the conditions for tensor data processing are currently met is not limited in this embodiment and depends on the specific implementation situation.
[0080] In this embodiment, by receiving the tensor data processing task sent by the host side, and judging whether the current tensor computing core has the tensor data processing conditions based on the information in the tensor data processing task, the corresponding computing instructions are obtained only when the tensor data processing conditions are met, thereby ensuring the normal execution of tensor data processing.
[0081] Based on the above embodiments, in some embodiments, judging whether the current tensor data processing conditions exist according to the number of threads, the number of warps, and the number of registers required for the current tensor data processing in the tensor data processing task includes:
[0082] S104: Determine whether the number of currently idle thread warps is not less than the number of thread warps required for the current tensor data processing in the tensor data processing task; if not, proceed to step S107; if yes, proceed to step S105;
[0083] S105: Determine whether the number of idle threads in the current thread bundle is not less than the number of threads required for the current tensor data processing in the tensor data processing task; if not, proceed to step S107; if yes, proceed to step S106;
[0084] S106: Determine whether the number of registers in the current register stack is not less than the number of registers required for the current tensor data processing in the tensor data processing task; if not, proceed to step S107; if yes, proceed to step S108;
[0085] S107: confirming that there is no tensor data processing condition currently;
[0086] S108: Confirm that the current tensor data processing conditions exist.
[0087] In a specific implementation, in order to determine whether the current tensor computing core has the tensor data processing conditions, it is specifically determined whether the number of currently idle thread bundles is not less than the number of thread bundles required for the current tensor data processing in the tensor data processing task. If it is confirmed that the number of currently idle thread bundles is less than the number of thread bundles required for the current tensor data processing in the tensor data processing task, it is considered that the number of current thread bundles is insufficient, and it is confirmed that the current tensor data processing conditions are not met; if it is confirmed that the number of currently idle thread bundles is not less than the number of thread bundles required for the current tensor data processing in the tensor data processing task, it is considered that the number of current thread bundles is sufficient, and it can be further determined whether the number of idle threads in the current thread bundle is not less than the number of threads required for the current tensor data processing in the tensor data processing task.
[0088] If it is confirmed that the number of idle threads in the current thread bundle is less than the number of threads required for the current tensor data processing in the tensor data processing task, it is considered that the number of current threads is insufficient, and it is confirmed that the current conditions for tensor data processing are not met; if it is confirmed that the number of idle threads in the current thread bundle is not less than the number of threads required for the current tensor data processing in the tensor data processing task, it is considered that the number of current threads is sufficient, and it can be further determined whether the number of registers in the current register stack is not less than the number of registers required for the current tensor data processing in the tensor data processing task.
[0089] If it is confirmed that the number of registers in the current register stack is less than the number of registers required for the current tensor data processing in the tensor data processing task, it is considered that the current number of registers is insufficient, and it is confirmed that the current conditions for tensor data processing are not met; if it is confirmed that the number of registers in the current register stack is not less than the number of registers required for the current tensor data processing in the tensor data processing task, it is considered that the current number of registers is sufficient, and it is confirmed that the current conditions for tensor data processing are met.
[0090] In this way, it is possible to judge whether the current tensor computing core has the conditions for tensor data processing.
[0091] Based on the above embodiments, in some embodiments, obtaining the data to be processed and transferring the data to be processed to the input buffer includes:
[0092] S110: Decoding the calculation instruction to generate a control signal;
[0093] Wherein, the control signal includes target register information of the register where the data to be processed is located;
[0094] S111: Select a corresponding vector register in the register file according to the target register information;
[0095] S112: Acquire feature data, weight data, and bias data in the vector register;
[0096] S113: transmitting the feature data, weight data and bias data to the input buffer respectively.
[0097] In order to obtain the data to be processed, in this embodiment, the calculation instruction is specifically decoded to generate a control signal. It should be noted that the control signal contains the target register information of the register where the data to be processed is located. The corresponding vector register is further selected in the register stack according to the target register information. For example, assuming that the target register information specifies three vector registers, the corresponding three vector registers can be selected in the register stack according to the target register information, and the feature data, weight data and bias data required by the tensor calculation core in the vector register are further obtained. Finally, the feature data, weight data and bias data are respectively transmitted to the input cache, thereby realizing the storage of the data to be processed by the input cache.
[0098] Based on the above embodiments, in some embodiments, judging whether all the data to be processed are stored in the input buffer according to the calculation instruction includes:
[0099] S121: Obtaining the total amount of all data to be processed contained in the control signal;
[0100] S122: Determine whether the amount of data to be processed in the input buffer is equal to the total amount of data; if not, proceed to step S123; if yes, proceed to step S124;
[0101] S123: confirming that all the data to be processed are not stored in the input buffer;
[0102] S124: Confirm that all data to be processed are stored in the input buffer.
[0103] In a specific implementation, the control signal includes not only the target register information of the register where the data to be processed is located, but also the total amount of data of all the data to be processed. Therefore, it is determined whether the amount of data to be processed in the input cache is equal to the total amount of data. If it is confirmed that the amount of data to be processed in the input cache is not equal to the total amount of data, it is confirmed that all the data to be processed are not stored in the input cache; if it is confirmed that the amount of data to be processed in the input cache is equal to the total amount of data, it is confirmed that all the data to be processed are stored in the input cache.
[0104] In this way, it is possible to determine whether all the data to be processed are stored in the input cache, so as to facilitate the subsequent processing of the complete data to be processed and improve the operating efficiency of the tensor computing core.
[0105] Based on the above embodiments, in some embodiments, the data to be processed is processed by a multiplier array and an accumulator to obtain a tensor data processing result, including:
[0106] S131: performing convolution calculation on the feature data and the weight data through a multiplier-adder array to obtain a convolution calculation result;
[0107] S132: Add the convolution calculation result and the bias data through an accumulator to obtain a tensor data processing result.
[0108] In order to calculate the tensor data processing result, the multiplier array in this embodiment performs convolution calculation on the feature data and the weight data to obtain the convolution calculation result. The convolution calculation result and the bias data are further added by the accumulator to obtain the tensor data processing result.
[0109] It is worth noting that the above process completes the execution of a calculation instruction for tensor data processing. When a new calculation instruction is received again, the above process can be repeated until the corresponding tensor data processing result is obtained. It should be noted that the input cache will store the data to be processed received during the previous tensor data processing process, and some of the data to be processed can be reused. Therefore, in the subsequent process of acquiring the data to be processed, the reused data does not need to be acquired again, and can be called directly from the input cache, thereby saving data loading time when performing multiple tensor data calculations and improving the efficiency of the calculation.
[0110] On the basis of the above embodiments, in some embodiments, the control signal further includes register information of a target vector register for storing the tensor data processing result; correspondingly, after obtaining the tensor data processing result, it also includes:
[0111] S14: Determine the target vector register in the register file according to the register information in the control signal;
[0112] S15: Store the tensor data processing result into the target vector register.
[0113] In order to better store the calculated tensor data processing results, in a specific implementation, the control signal obtained by decoding the calculation instruction also includes register information of the target vector register used to store the tensor data processing results.
[0114] Therefore, after obtaining the tensor data processing result, the target vector register in the register file can be further determined according to the register information in the control signal. Finally, the tensor data processing result is stored in the target vector register, thereby achieving better storage of the tensor data processing result.
[0115] In the above embodiments, the tensor data processing method is described in detail, and the present application also provides corresponding embodiments of the tensor data processing device.
[0116] Figure 3 A schematic diagram of a tensor data processing device provided in an embodiment of the present application. The device is applied to a tensor computing core including a controller, a register file, an accumulator, an input buffer, and a multiplier array; Figure 3 As shown, the device comprises:
[0117] The acquisition module 10 is used to acquire a tensor data processing task and a computing instruction corresponding to the tensor data processing task; wherein the tensor data processing task at least includes the number of threads, the number of thread bundles, the number of registers and the number of instructions required for this tensor data processing.
[0118] The transmission module 11 is used to obtain the data to be processed and transmit the data to be processed to the input buffer.
[0119] The judgment module 12 is used to judge whether all the data to be processed are stored in the input buffer according to the calculation instruction; if not, the transmission module 11 is triggered; if yes, the processing module 13 is triggered.
[0120] The processing module 13 is used to process the data to be processed through a multiplier array and an accumulator to obtain a tensor data processing result.
[0121] Since the embodiments of the apparatus part correspond to the embodiments of the method part, please refer to the description of the embodiments of the method part for the embodiments of the apparatus part, which will not be repeated here.
[0122] Figure 4 A schematic diagram of a tensor data processing device provided in an embodiment of the present application. Figure 4 As shown, the tensor data processing device includes:
[0123] A memory 20, used for storing computer programs;
[0124] The processor 21 is used to implement the steps of the tensor data processing method mentioned in the above embodiment when executing the computer program.
[0125] The tensor data processing device provided in this embodiment may include but is not limited to a smart phone, a tablet computer, a laptop computer, or a desktop computer.
[0126] Among them, the processor 21 may include one or more processing cores, such as a 4-core processor, an 8-core processor, etc. The processor 21 can be implemented in at least one hardware form of a digital signal processor (DSP), a field programmable gate array (FPGA), and a programmable logic array (PLA). The processor 21 may also include a main processor and a coprocessor. The main processor is a processor for processing data in an awake state, also known as a central processing unit (CPU); the coprocessor is a low-power processor for processing data in a standby state. In some embodiments, the processor 21 may be integrated with a graphics processing unit (GPU), and the GPU is responsible for rendering and drawing the content to be displayed on the display screen. In some embodiments, the processor 21 may also include an artificial intelligence (AI) processor, which is used to process computing operations related to machine learning.
[0127] The memory 20 may include one or more computer-readable storage media, which may be non-transitory. The memory 20 may also include a high-speed random access memory, and a non-volatile memory, such as one or more disk storage devices, flash memory storage devices. In this embodiment, the memory 20 is at least used to store the following computer program 201, wherein the computer program, after being loaded and executed by the processor 21, can implement the relevant steps of the tensor data processing method disclosed in any of the aforementioned embodiments. In addition, the resources stored in the memory 20 may also include an operating system 202 and data 203, etc., and the storage method may be temporary storage or permanent storage. Among them, the operating system 202 may include Windows, Unix, Linux, etc. Data 203 may include but is not limited to data involved in the tensor data processing method.
[0128] In some embodiments, the tensor data processing device may further include a display screen 22 , an input / output interface 23 , a communication interface 24 , a power supply 25 , and a communication bus 26 .
[0129] Those skilled in the art will understand that Figure 4 The structure shown in the figure does not constitute a limitation on the tensor data processing device, and may include more or less components than shown in the figure.
[0130] Finally, the present application also provides an embodiment corresponding to a computer-readable storage medium. The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps recorded in the above method embodiment are implemented.
[0131] It is understandable that if the method in the above embodiment is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), disk or optical disk and other media that can store program code.
[0132] The above is a detailed introduction to a tensor data processing method, device, equipment and medium provided by the present application. The various embodiments in the specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same and similar parts between the embodiments can refer to each other. For the device disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant parts can be referred to the method part description. It should be pointed out that for ordinary technicians in this technical field, without departing from the principles of the present application, several improvements and modifications can be made to the present application, and these improvements and modifications also fall within the scope of protection of the present application.
[0133] It should also be noted that, in this specification, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "comprise", "include" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the statement "comprises a ..." does not exclude the presence of other identical elements in the process, method, article or device including the element.
Claims
1. A tensor data processing method, characterized in that: Applied to a tensor computing core including a controller, a register file, an accumulator, an input buffer, and a multiplier-adder array; the method comprises: Obtaining a tensor data processing task and obtaining a computing instruction corresponding to the tensor data processing task; wherein the tensor data processing task at least includes the number of threads, the number of thread bundles, the number of registers, and the number of instructions required for this tensor data processing; Acquire data to be processed, and transfer the data to be processed to the input buffer; Determining whether all the data to be processed are stored in the input buffer according to the calculation instruction; If not, return to the step of obtaining the data to be processed and transferring the data to be processed to the input buffer; If so, the data to be processed is processed by the multiplier array and the accumulator to obtain a tensor data processing result.
2. The tensor data processing method according to claim 1, characterized in that: The obtaining of the tensor data processing task and obtaining the computing instruction corresponding to the tensor data processing task includes: Receiving the tensor data processing task sent by the host end; Determining whether a tensor data processing condition is currently met according to the number of threads, the number of thread warps, and the number of registers required for the current tensor data processing in the tensor data processing task; If it is confirmed that there is no condition for tensor data processing at present, then end; If it is confirmed that the tensor data processing conditions are currently met, the calculation instructions corresponding to the tensor data processing task are obtained from the memory.
3. The tensor data processing method according to claim 2, characterized in that: The determining whether the tensor data processing condition is currently met according to the number of threads, the number of thread warps, and the number of registers required for the tensor data processing in the tensor data processing task includes: Determine whether the number of currently idle thread warps is not less than the number of thread warps required for the current tensor data processing in the tensor data processing task; If it is confirmed that the number of currently idle thread warps is less than the number of thread warps required for the current tensor data processing in the tensor data processing task, it is confirmed that the tensor data processing condition is not met at present; If it is confirmed that the number of currently idle thread warps is not less than the number of thread warps required for the current tensor data processing in the tensor data processing task, then determining whether the number of idle threads in the current thread warp is not less than the number of threads required for the current tensor data processing in the tensor data processing task; If it is confirmed that the number of idle threads in the current thread bundle is less than the number of threads required for the current tensor data processing in the tensor data processing task, it is confirmed that the tensor data processing condition is not met at present; If it is confirmed that the number of idle threads in the current thread bundle is not less than the number of threads required for the current tensor data processing in the tensor data processing task, then determining whether the number of registers in the current register file is not less than the number of registers required for the current tensor data processing in the tensor data processing task; If it is confirmed that the number of registers in the current register file is less than the number of registers required for the current tensor data processing in the tensor data processing task, it is confirmed that the tensor data processing condition is not met at present; If it is confirmed that the number of registers in the current register stack is not less than the number of registers required for the current tensor data processing in the tensor data processing task, it is confirmed that the tensor data processing conditions are currently met.
4. The tensor data processing method according to claim 1, characterized in that: The obtaining of the data to be processed and transmitting the data to be processed to the input buffer includes: Decoding the computing instruction to generate a control signal; wherein the control signal includes target register information of the register where the data to be processed is located; Selecting a corresponding vector register in the register file according to the target register information; Obtaining feature data, weight data and bias data in the vector register; The feature data, weight data and bias data are respectively transmitted to the input buffer.
5. The tensor data processing method according to claim 4, characterized in that: The step of determining whether all the to-be-processed data are stored in the input buffer according to the calculation instruction includes: Acquire the total amount of all the data to be processed contained in the control signal; Determining whether the amount of the to-be-processed data in the input buffer is equal to the total amount of data; If not, confirming that all the data to be processed are not stored in the input buffer; If so, it is confirmed that all the data to be processed are stored in the input buffer.
6. The tensor data processing method according to claim 4, characterized in that: The processing of the data to be processed by the multiplier array and the accumulator to obtain a tensor data processing result includes: Performing convolution calculation on the feature data and the weight data through the multiplier-adder array to obtain a convolution calculation result; The convolution calculation result and the bias data are added by the accumulator to obtain the tensor data processing result.
7. The tensor data processing method according to claim 5, characterized in that: The control signal also includes register information of a target vector register for storing the tensor data processing result; correspondingly, after obtaining the tensor data processing result, it also includes: Determine the target vector register in the register file according to the register information in the control signal; The tensor data processing result is stored in the target vector register.
8. A tensor data processing device, characterized in that: Applicable to a tensor computing core including a controller, a register file, an accumulator, an input buffer and a multiplier-adder array; the device includes: An acquisition module, used to acquire a tensor data processing task and acquire a computing instruction corresponding to the tensor data processing task; wherein the tensor data processing task at least includes the number of threads, the number of thread bundles, the number of registers, and the number of instructions required for this tensor data processing; A transmission module, used for acquiring data to be processed and transmitting the data to be processed to the input buffer; A judgment module, used for judging whether all the data to be processed are stored in the input buffer according to the calculation instruction; if not, triggering the transmission module; if yes, triggering the processing module; The processing module is used to process the data to be processed through the multiplier array and the accumulator to obtain a tensor data processing result.
9. A tensor data processing device, characterized in that: include: Memory for storing computer programs; A processor, configured to implement the steps of the tensor data processing method as claimed in any one of claims 1 to 7 when executing the computer program.
10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the tensor data processing method according to any one of claims 1 to 7 are implemented.