GPU (Graphics Processing Unit) write storage system and method based on Stream write instruction

Abstract

The invention provides a GPU (Graphics Processing Unit) write storage system based on a Stream write instruction and a method thereof, and belongs to the technical field of calculation. According to the method, the special Stream Write instruction is provided, so that the operation of flexibly writing a small amount of data into the memory by a single instruction is realized, the method can be used for carrying out hardware acceleration on a data writing function of a common artificial intelligence software framework, a software compiler can be simplified, the instruction execution efficiency can be improved, and the support of a GPU on high-performance calculation is realized.

Description

Technical Field

[0001] The present invention relates to the field of computing technology, and in particular to a GPU write storage system based on a Stream write instruction and a method thereof. Background Art

[0002] In the era of big data, computing power has become the driving force for the development of the digital economy. As the demand for data computing increases, GPUs have become an important part of computing infrastructure. Especially for artificial intelligence applications, training and reasoning based on massive data sets are required. Traditional control-oriented CPUs are difficult to meet such high-performance computing needs. GPUs for parallel computing have a large number of small processing cores and high-bandwidth memory systems. They can execute multiple thread instructions at the same time and achieve highly parallel computing. They significantly improve the computing speed when processing large-scale data and complex computing tasks, and can quickly read and write data. They are suitable for tasks that require frequent and flexible access to data, making them better hardware accelerators.

[0003] When it comes to the operation of flexibly writing small amounts of data into storage, current common artificial intelligence software frameworks, including Google Tensorflow and open source PyTorch, provide dedicated methods. For example, in the Google Tensorflow framework, the tf.data API is used to call the write() method to complete memory access and write operations; in the open source PyTorch framework, it is necessary to use the corresponding write operation functions such as write() in combination with the repository according to specific needs to write data to the storage.

[0004] By extension, in other software fields, there is also a general need to flexibly write small amounts of data to the memory, such as parameter storage, model storage, configuration files, log records, cache data, and temporary storage of user interaction data. Its feature is that the user can calculate the write address and write data in advance, and flexibly write the carried data to the correct destination address. In the following text, this type of data writing function is uniformly referred to as the write() function.

[0005] The traditional implementation method requires compiling the write() function into multiple memory access instructions, generating a write data block, issuing a DMA COPY (direct memory access copy) instruction, the GPU reading the data, and the GPU writing the data to the destination address. This increases the complexity of compiling instructions and the overhead of the write operation, and reduces the efficiency of the write operation. Summary of the invention

[0006] In view of the high complexity and low efficiency of traditional memory writing of small amounts of data, the present invention adopts a GPU instruction management system based on Stream Write and its method to achieve simplified and efficient writing of small amounts of data.

[0007] A first aspect of the present invention provides a GPU write storage system based on a Stream write instruction, comprising an instruction read and write module, a Stream Write state control module, and a storage module; the input end of the Stream Write state control module is connected to the output end of the instruction read and write module, and the output end of the Stream Write state control module is connected to the input end of the storage module.

[0008] A second aspect of the present invention provides a GPU storage writing method based on a Stream write instruction, comprising the following steps: S1, the user constructs a Stream Write instruction and writes it into the buffer of the instruction read / write module; S2, the instruction reading and writing module reads the Stream Write instruction from the buffer and sends it to the Stream Write state control module; S3, the Stream Write state control module performs instruction parsing on the Stream Write instruction to obtain information carried by the Stream Write instruction; S4, the Stream Write state control module sends a write request to the storage module based on the information; S5, after receiving the write request, the storage module writes the data into the corresponding address space according to the write address.

[0009] Furthermore, the Stream Write instruction includes the following information: Stream Write instruction operation code, write address, write data, secure storage identifier, and data location exchange configuration.

[0010] Furthermore, the information in step S3 includes a write address, a write data amount, write data, a secure storage identifier, and a data location exchange configuration.

[0011] Furthermore, the step S4 includes: during the write operation, the Stream Write state control module automatically processes the correspondence between the write address and the write data, automatically accumulates and calculates the write address, and ensures that the data is written to the correct address.

[0012] Furthermore, the write request in step S4 includes a secure storage access identifier. When the secure storage access identifier is 1, it represents that the secure storage area designated by the system can be successfully accessed, i.e., secure access; when the secure storage identifier is 0, it represents non-secure access.

[0013] Furthermore, in step S4, the information of the Stream Write state control module initiates a write request to the storage module, including: The Stream Write state control module calculates the write address, performs data exchange according to the data position exchange configuration, initiates a write request, and constructs a secure or non-secure access according to the secure storage identifier.

[0014] The beneficial effects of the present invention are: 1) Simplify compiler design. The Stream Write instruction is directly open to users and compilers. The software's write() function can be directly compiled into a Stream Write instruction.

[0015] 2) High efficiency, dedicated Stream Write instruction execution state machine, a single instruction completes the flexible write operation of small amounts of data of variable length configured by the user.

[0016] 3) High flexibility, supporting user-defined, flexible, and data location exchange of different granularities.

[0017] 4) High security, user-controlled secure or non-secure storage access. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required for use in the embodiments are briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present invention and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without creative work.

[0019] Figure 1 It is a structural schematic diagram of a GPU write storage system based on Stream write instructions in the present invention.

[0020] Figure 2 It is a schematic diagram of the instruction reading and writing module executing instruction writing and reading according to an embodiment of the present invention.

[0021] Figure 3 It is a schematic diagram of the workflow of the Stream write state control module in an embodiment of the present invention.

[0022] Figure 4It is a schematic diagram of writing data and reading data of a storage module according to an embodiment of the present invention. DETAILED DESCRIPTION

[0023] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0024] The words "first, second, third, etc." or module A, module B, module C and the like in the specification and claims are used only to distinguish similar objects and do not represent a specific ordering of the objects. It is understandable that the specific order or sequence can be interchanged where permitted so that the embodiments of the invention described herein can be implemented in an order other than that illustrated or described herein.

[0025] In the following description, the numbers representing the steps, such as S110, S120, etc., do not necessarily mean that the steps must be executed in this manner. If permitted, the order of the previous and next steps can be interchanged, or they can be executed simultaneously.

[0026] The term "comprising" as used in the description and claims should not be interpreted as being limited to what is listed thereafter; it does not exclude other elements or steps. Therefore, it should be interpreted as specifying the presence of the features, integers, steps or components mentioned, but does not exclude the presence or addition of one or more other features, integers, steps or components and groups thereof. Therefore, the expression "a device comprising means A and B" should not be limited to a device consisting of components A and B only.

[0027] References to "one embodiment" or "an embodiment" in this specification mean that a particular feature, structure, or characteristic described in conjunction with the embodiment is included in at least one embodiment of the present invention. Therefore, the phrases "in one embodiment" or "in an embodiment" appearing in various places in this specification do not necessarily refer to the same embodiment, but may refer to the same embodiment. In addition, in one or more embodiments, the particular features, structures, or characteristics can be combined in any appropriate manner, as would be apparent to one of ordinary skill in the art from this disclosure.

[0028] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art of the present invention. In the event of any inconsistency, the meanings described in this specification or derived from the contents recorded in this specification shall prevail. In addition, the terms used herein are only for the purpose of describing the embodiments of the present invention and are not intended to limit the present invention.

[0029] See also Figure 1 An embodiment of the present invention discloses a GPU write storage system based on a Stream write instruction, including an instruction read-write module, a Stream Write state control module, and a storage module; the input end of the Stream Write state control module is connected to the output end of the instruction read-write module, and the output end of the Stream Write state control module is connected to the input end of the storage module.

[0030] Among them, the instruction read and write module is mainly responsible for writing, caching and reading instructions; the Stream Write state control module is responsible for parsing instructions, calculating write addresses, exchanging data according to data location exchange configuration, initiating write requests, and constructing secure or non-secure access according to secure storage identifiers; the storage module is mainly responsible for storing, reading and writing variables.

[0031] The embodiment of the present invention discloses a GPU storage writing method based on a Stream write instruction, comprising the following steps: S1, the user constructs a Stream Write instruction and writes it into the buffer of the instruction read / write module; S2, the instruction reading and writing module reads the Stream Write instruction from the buffer and sends it to the Stream Write state control module; Reference Figure 2 As shown, the instruction read and write module is mainly responsible for writing, caching and reading instructions.

[0032] The latest instruction will be written to the next address in the buffer. The instruction buffer is a first-in-first-out cache with a depth of M bytes and an address range of 0 to M-1 bytes. When the instruction is written to the last address M-1 in the buffer, it will start again from 0. The instruction written first will be read out first.

[0033] When the Host buffer is not empty, it starts reading instructions. The instruction reading and writing module actively sends a section of instructions to the Stream Write state control module in sequence. The length of each transmission is L, the minimum value of L is 1 Byte, and the maximum value is the instruction length.

[0034] S3, the Stream Write state control module performs instruction parsing on the Stream Write instruction to obtain information carried by the Stream Write instruction; S4, the Stream Write state control module sends a write request to the storage module based on the information; S5, after receiving the write request, the storage module writes the data into the corresponding address space according to the write address.

[0035] Furthermore, the Stream Write instruction includes the following information: Stream Write instruction operation code, write address, write data, secure storage identifier, and data location exchange configuration.

[0036] The Stream Write command contains the following information: 1) Stream Write instruction operation code (OP CODE), used to identify this command.

[0037] 2) Write address, the starting destination address of this write fill operation.

[0038] 3) Write data. The data written this time is flexibly configured by the user, with unfixed length and content.

[0039] 4) Secure storage identifier, used to identify whether the destination address of this filling is a secure storage area or a non-secure storage area.

[0040] 5) Data location exchange configuration, including the following types: no exchange, 16-bit exchange, 32-bit exchange, and 64-bit exchange.

[0041] Data location exchange instructions: 1) No swap: Stream Write writes data as is, 0xaabb->0xaabb.

[0042] 2) 16-bit swap: The data position is swapped with 16-bit granularity, 0xaabb -> 0xbbaa.

[0043] 3) 32-bit swap: Data positions are swapped at 32-bit granularity, 0xaabbccdd -> 0xddccbbaa.

[0044] 4) 64-bit swap: The data position is swapped with 64-bit granularity, 0xaabbccddeeff0011->0x1100ffeeddccbbaa.

[0045] Furthermore, the information in step S3 includes a write address, a write data amount, write data, a secure storage identifier, and a data location exchange configuration.

[0046] Furthermore, the step S4 includes: during the write operation, the Stream Write state control module automatically processes the correspondence between the write address and the write data, automatically accumulates and calculates the write address, and ensures that the data is written to the correct address.

[0047] Furthermore, the write request in step S4 includes a secure storage access identifier. When the secure storage access identifier is 1, it represents that the secure storage area designated by the system can be successfully accessed, i.e., secure access; when the secure storage identifier is 0, it represents non-secure access.

[0048] Furthermore, in step S4, the information of the Stream Write state control module initiates a write request to the storage module, including: The Stream Write state control module calculates the write address, performs data exchange according to the data position exchange configuration, initiates a write request, and constructs a secure or non-secure access according to the secure storage identifier.

[0049] In view of the high complexity and low efficiency of traditional memory writing small amounts of data, the present invention provides a dedicated Stream Write instruction, which corresponds to the write() function of the software framework and supports multiple parameters of the function, including: the address of data writing, the amount of data writing, the value of the written data, etc. In addition, the Stream Write instruction is exposed to the compiler, and the compiler can compile the data writing function into one instruction at one time, and the data movement and writing operations are hidden from the software, reducing the intervention of the software in the whole process. The scheme of the present invention also supports a flexible data location exchange (SWAP) mechanism, which can provide flexible 16-bit, 32-bit, and 64-bit granularity data location exchange functions according to the system size requirements and software data format requirements, without the need for additional data exchange at the software level. The scheme of the present invention also supports secure storage access control, realizes the division of application security levels, and improves the security of the system.

[0050] Continue to refer to Figure 3 As shown, the workflow of the Stream Write state control module in the present invention is explained as follows: First, in the initial state, the Stream Write state control module is in an idle state; when the StreamWrite state control module receives the Stream Write instruction from the instruction read and write module, it enters the instruction parsing state, and parses the write address, write data amount, write data, secure storage identifier, data location exchange indication and other information carried by the Stream Write instruction.

[0051] Then, after the analysis is completed, it enters the write preparation state, calculates the write address, performs data exchange according to the data location exchange instruction, and performs security level control according to the secure storage identifier. Since a single write request to the memory has a maximum burst limit, such as 64B, 256B, etc., the write operation of the Stream Write instruction requires multiple write requests.

[0052] Next, enter the write operation state, send a write request to the memory, and wait for a response.

[0053] After that, after the write operation succeeds, the Stream Write state control module calculates whether the last address has been reached. If so, it enters the write completion state; otherwise, it returns to the write preparation state and continues the next write request.

[0054] Finally, it returns to the idle state and waits for the next Stream Write instruction.

[0055] Reference Figure 4 As shown, the storage module in the present invention can be a register, SRAM, DDR, HBM, etc., and the present invention does not limit it. The range of the storage space is the maximum value that can be represented by the address bit width in the Stream Write instruction. When the memory receives the write operation sent by the Stream Write state control module, the storage module will write the new variable data into the storage unit of the corresponding address according to the write address and write data. Moreover, during the execution of the Stream Write instruction, no read operation on the memory is involved.

[0056] Based on the working principle of the present invention, the following two embodiments are provided: Example 1) The user wants to write 32 bytes of data to the starting address 0x16, and the data is the 32 digits after the decimal point of pi. The Stream Write instruction information prepared by the user includes: Stream Write identifier 0x2 (OP CODE, custom), write address 0x16, write data length 32, write data 1415 9265 3589 7932 3846 2643 3832 7950, secure memory identifier 0, data position swap indication 0 (no swap). The StreamWrite instruction length is 48 bytes, including a 16-byte instruction header and 32-byte write data. The 16-byte instruction header, which contains the aforementioned control information, is usually defined as 4B alignment to simplify software address management.

[0057] In the initial state, there is no Stream Write instruction in the instruction buffer, the address is 0, and the buffer depth is 1MByte. Therefore, the latest Stream Write instruction will be written to the buffer address 0-47B.

[0058] When the command buffer is not empty, it indicates that a command has been written into the Host buffer. At this time, the command reading and writing module will actively send a section of command to the status control module. The length of each command sent is 16B, and it is sent three times.

[0059] The Stream Write state control module will parse the Stream Write instruction and obtain the write address, write data amount, write data, secure storage identifier, data location exchange indication and other information of the Stream Write instruction.

[0060] The data position swap indicator is 0, and the data is not swapped. The secure storage indicator is 0, and in the issued write request, it is marked as non-secure access. If the memory can support a single 32B access, only one write request needs to be sent to complete the write operation.

[0061] After receiving the write request, the storage module writes the data to a continuous 32-byte storage body starting at address 0x16.

[0062] Example 2) The user wants to write 128B of data to the starting address 0x100, and the data is 128 digits after the decimal point of pi. Therefore, the Stream Write instruction information prepared by the user includes: Stream Write identifier 0x2 (OP CODE, custom), write address 0x100, write data length 128, write data bit 14159 26535 89793 23846 26433 8327950288 41971 69399 37510 5820974944 59230 78164 06286 20899 86280 34825 3421170679 82148 08651 32823 06647 09384 460, secure memory identifier 1, and data position swap indication 32b swap. The length of the Stream Write instruction is 144 bytes, including a 16-byte instruction header and 128 bytes of write data.

[0063] There are other instructions in the instruction buffer, the address has been written to 0x500, and the buffer depth is 1MByte. Therefore, the latest Stream Write instruction will be written to the buffer address 0x500-0x58F.

[0064] When the command buffer is not empty, it indicates that a command has been written into the Host buffer. At this time, the command reading and writing module will actively send a section of command to the status control module. The length of each command sent is 16B, and it is sent 9 times.

[0065] The Stream Write state control module will parse the Stream Write instruction and obtain the write address, write data amount, write data, secure storage identifier, data location exchange indication and other information of the Stream Write instruction.

[0066] The data position exchange indication is 16b exchange, and the exchange is performed according to the aforementioned "data position exchange" rule. The secure storage flag is 1, and in the issued write request, it is marked as secure access. If the memory supports a single maximum access of 32B, four write requests need to be sent, and the write addresses are 0x100, 0x120, 0x140, and 0x160 respectively, to complete the write operation.

[0067] After receiving the write operation, the storage module writes the write data to the continuous 128B storage body starting at address 0x100.

[0068] Note that the above are only preferred embodiments of the present invention and the technical principles used. Those skilled in the art will understand that the present invention is not limited to the specific embodiments described herein, and that various obvious changes, readjustments and substitutions can be made by those skilled in the art without departing from the scope of protection of the present invention. Therefore, although the present invention has been described in more detail through the above embodiments, the present invention is not limited to the above embodiments, and may also include more other equivalent embodiments without departing from the concept of the present invention, all of which belong to the scope of protection of the present invention.

Claims

1. A GPU write storage system based on Stream write instruction, characterized in that: It includes an instruction read-write module, a Stream Write state control module, and a storage module; the input end of the Stream Write state control module is connected to the output end of the instruction read-write module, and the output end of the Stream Write state control module is connected to the input end of the storage module.

2. According to claim 1, a GPU write storage method of a GPU write storage system based on a Stream write instruction comprises the following steps: S1, the user constructs a Stream Write instruction and writes it into the buffer of the instruction read / write module; S2, the instruction reading and writing module reads the Stream Write instruction from the buffer and sends it to the Stream Write state control module; S3, the Stream Write state control module performs instruction parsing on the Stream Write instruction to obtain information carried by the Stream Write instruction; S4, the Stream Write state control module sends a write request to the storage module based on the information; S5, after receiving the write request, the storage module writes the data into the corresponding address space according to the write address.

3. The method according to claim 2, characterized in that: The Stream Write instruction includes the following information: Stream Write instruction operation code, write address, write data, secure storage identifier, and data location exchange configuration.

4. The method according to claim 2, characterized in that: The information in step S3 includes a write address, a write data amount, write data, a secure storage identifier, and a data location exchange configuration.

5. The method according to claim 2, characterized in that: The step S4 includes: during the write operation, the Stream Write state control module automatically processes the correspondence between the write address and the write data, automatically accumulates and calculates the write address, and ensures that the data is written to the correct address.

6. The method according to claim 2, characterized in that: The write request in step S4 includes a secure storage access flag. When the secure storage access flag is 1, it represents that the secure storage area designated by the system can be successfully accessed, i.e., secure access; when the secure storage flag is 0, it represents non-secure access.

7. The method according to claim 2, characterized in that: In step S4, the information of the Stream Write state control module initiates a write request to the storage module, including: The Stream Write state control module calculates the write address, performs data exchange according to the data position exchange configuration, initiates a write request, and constructs a secure or non-secure access according to the secure storage identifier.

Images