Efficient hardware mapping method and system for generative large-scale artificial intelligence model

Abstract

Provided are an efficient hardware mapping method and system for a generative large-scale artificial intelligence model. The hardware mapping method, according to one embodiment, may comprise the steps of: receiving model software; and, by means of at least one processor, sequentially performing a source code level simulation, an instruction level simulation, and a register transfer level simulation with respect to the model software.

Description

Method and system for efficient hardware mapping of generative giant artificial intelligence models.

[0001] Embodiments of the present invention relate to a method and system for efficient hardware mapping of a generative giant artificial intelligence model, and more particularly, to a method and system for more quickly and efficiently mapping various generative giant artificial intelligence models that are being rapidly distributed to dedicated hardware structures such as FPGAs (Field Programmable Gate Arrays) / ASICs (Application Specific Integrated Circuits).

[0002] Recently, the software community has been rapidly deploying various generative AI models. Due to the discrepancy between software and hardware development speeds, mapping these models to dedicated hardware architectures is challenging. Furthermore, unlike models like the Convolutional Neural Network (CNN) of the past, model sizes in the billions further slow down development.

[0003] A method and system for efficiently mapping a generative large-scale artificial intelligence model to hardware can be provided.

[0004] The technical problems of the present invention are not limited to the technical problems mentioned above, and other technical problems not mentioned will be clearly understood by those skilled in the art from the description below.

[0005] A hardware mapping method for a computer device including at least one processor, comprising: a step of receiving model software by the at least one processor; and a step of sequentially performing a source code level simulation, an instruction level simulation, and a register transfer level simulation for the model software by the at least one processor.

[0006] According to one aspect, the performing step may be characterized by including a step of cross-verifying instructions written through the source code level simulation through the instruction level simulation.

[0007] According to another aspect, the hardware mapping method may further include a step of receiving more information about the existing hardware structure, and the performing step may include a step of determining whether the model software can be implemented with the existing hardware structure; a step of adding a hardware module to the existing hardware structure if the implementation is impossible; and a step of re-performing the instruction level simulation.

[0008] According to another aspect, the performing step may be characterized by including: a step of writing a module-level test case for each instruction cross-verified through the source code level simulation and the instruction level simulation; and a step of verifying at least one of the written test cases through the register transfer level simulation.

[0009] According to another aspect, the verifying step may be characterized by cross-verifying through the register transfer level simulation and the instruction level simulation.

[0010] According to another aspect, the performing step may be characterized by including a step of processing cross-validation through the source code level simulation and the instruction level simulation using a GPU (Graphics Processing Unit); and a step of processing multi-threaded loading of a file of the model software for the register transfer level simulation.

[0011] According to another aspect, the step of processing the multi-thread loading may be characterized by dividing the file of the model software by channel of the memory and loading it into the memory using multi-threading.

[0012] A computer program stored on a computer-readable recording medium is provided for executing the above method on a computer device in combination with a computer device.

[0013] A computer-readable recording medium having recorded thereon a program for executing the above method on a computer device is provided.

[0014] A computer device is provided, comprising at least one processor implemented to execute instructions readable from a computer device, characterized in that model software is input by the at least one processor and source code level simulation, instruction level simulation, and register transfer level simulation for the model software are sequentially performed.

[0015] Specific details of other embodiments are included in the detailed description and drawings.

[0016] A method and system for efficiently mapping a generative large-scale artificial intelligence model to hardware can be provided.

[0017] The effects of the present invention are not limited to the effects mentioned above, and other effects not mentioned will be clearly understood by those skilled in the art from the description of the claims.

[0018] FIG. 1 is a diagram illustrating an example of a verification framework according to one embodiment of the present invention.

[0019] FIG. 2 is a diagram illustrating an example of a process for verifying multiple models in one embodiment of the present invention.

[0020] FIG. 3 is a flowchart illustrating an example of a hardware mapping method according to one embodiment of the present invention.

[0021] FIG. 4 is a diagram illustrating an example of a process for processing an artificial intelligence model with large data using a register transfer level simulator according to one embodiment of the present invention.

[0022] FIG. 5 is a block diagram illustrating an example of a computer device according to one embodiment of the present invention.

[0023] The advantages and features of the present invention, and the methods for achieving them, will become clearer with reference to the embodiments described in detail below together with the accompanying drawings. However, the present invention is not limited to the embodiments disclosed below, but may be implemented in various different forms. These embodiments are provided only to ensure that the disclosure of the present invention is complete and to fully inform those skilled in the art of the scope of the invention, and the present invention is defined only by the scope of the claims. Like reference numerals designate like elements throughout the specification.

[0024] When one component is referred to as being "connected to" or "coupled to" another component, it includes both cases where it is directly connected or coupled to the other component, or cases where there is another component intervening therebetween. Conversely, when one component is referred to as being "directly connected to" or "directly coupled to" another component, it indicates that there is no other component intervening therebetween. "And / or" includes each and any combination of one or more of the mentioned items.

[0025] The terminology used herein is for the purpose of describing embodiments only and is not intended to limit the present invention. In this specification, the singular also includes the plural unless specifically stated otherwise. As used herein, the terms "comprises" and / or "comprising" do not exclude the presence or addition of one or more other components, steps, operations, and / or elements.

[0026] Although terms like "first" and "second" are used to describe various components, these components are not limited by these terms. These terms are merely used to distinguish one component from another. Therefore, it should be understood that a "first" component referred to below may also be a "second" component within the technical scope of the present invention.

[0027] Unless otherwise defined, all terms (including technical and scientific terms) used herein may be used in their common sense to those of ordinary skill in the art to which the present invention pertains. Furthermore, terms defined in commonly used dictionaries are not to be interpreted ideally or excessively unless explicitly and specifically defined otherwise.

[0028] The hardware development process proceeds through RTL (Register Transfer Level) simulation debugging of the written Verilog code. However, RTL simulation relies on register-level verification, making it extremely slow for application to large-scale AI models. Therefore, embodiments of the present invention can handle sequential verification by adding code-level and instruction-level simulation.

[0029] Additionally, since code-level simulation and instruction-level simulation added to the development tool can process simulations using NVIDA's GPU (Graphics Processing Unit), it can reduce the burden on simulators that must process large data.

[0030] Additionally, thanks to code-level simulation and instruction-level simulation, RTL simulation can be configured to perform local verification only at the instruction level or module level.

[0031] FIG. 1 is a diagram illustrating an example of a verification framework according to an embodiment of the present invention. The hardware mapping system according to the present embodiment may include a verification framework (100). The verification framework (100) may include a source code level simulator (110), an instruction level simulator (120), and a register transfer level simulator (130).

[0032] The verification framework (100) can receive a giant artificial intelligence model (140) and an existing hardware structure (150) as inputs and output model instructions (160) and a hardware structure (170) through verification using a source code level simulator (110), an instruction level simulator (120), and a register transfer level simulator (130). Here, the hardware structure (170) can include information on the structure of hardware to be mapped to the giant artificial intelligence model (140).

[0033] FIG. 2 is a diagram illustrating an example of a process for verifying multiple models in one embodiment of the present invention. FIG. 2 shows that deployment time can be shortened by verifying models 1, 2, and 3 in a pipeline manner through a source code level simulator (110), an instruction level simulator (120), and a register transfer level simulator (130). The source code level simulator (110) can provide a function to view the intermediate value of a desired location at the source code level by setting a GPU prober at a desired location at the source code level, thereby identifying which instructions are required and which hardware structure is required. The instruction level simulator (120) is a simulator that creates and verifies instructions for a model. When verifying, the information identified at the source code level becomes correct, so that the source code level simulator (110) and the instruction level simulator (120) can identify the entire instruction and hardware structure from the beginning to the end for the model through upper-level debugging. On the other hand, the register transfer level simulator (130) is too slow to be viewed end-to-end, so it can be divided into modules to write test cases and process verification for the written test cases. When one model is sequentially verified through the source code level simulator (110), the instruction level simulator (120), and the register transfer level simulator (130), the hardware can be deployed, and as described above, it becomes possible to quickly verify multiple models that are developed relatively quickly compared to hardware through the pipeline method.

[0034] FIG. 3 is a flowchart illustrating an example of a hardware mapping method according to an embodiment of the present invention. The hardware mapping method according to the present embodiment may be performed by at least one computer device. In this case, the processor of the computer device may be implemented to execute control instructions based on the code of an operating system included in the memory or the code of at least one computer program. Here, the processor may control the computer device so that the computer device performs steps (310 to 380) included in the method of FIG. 3 according to the control instructions provided by the code stored in the computer device. An example of the structure of the computer device will be described in more detail later with reference to FIG. 5.

[0035] At step (310), the computer device may receive model software. For example, the large-scale artificial intelligence model (140) described above with reference to FIG. 1 may be received. Depending on the embodiment, the computer device may receive additional information about the existing hardware structure at step (310), or before or after step (310).

[0036] Thereafter, the computer device can sequentially perform source code level simulation, instruction level simulation, and register transfer level simulation for the input model software.

[0037] In step (320), the computer device can set up a source code-level simulation and write instructions. This step (320) may be a process in which a source code-level simulation is performed using a source code-level simulator (110). At this time, the source code-level simulator (110) can check whether the formula can be easily applied to hardware and whether the model can be modified in a hardware-friendly manner through quantization.

[0038] At step (330), the computer device can process instruction level simulation. The instruction level simulation can be processed through an instruction level simulator (120).

[0039] Here, high-level debugging may be a process in which the source code-level simulator (110) and the instruction-level simulator (120) identify the entire instruction and hardware structure of the model from start to end, using information identified at the source code level as previously described. In other words, the computer device can cross-validate the instructions created through the source code-level simulation in step (320) through the instruction-level simulation in step (330).

[0040] In step (340), the computer device can determine whether the model software can be implemented using the existing hardware structure. At this time, the existing hardware structure can be obtained through the existing hardware structure (150) described above with reference to FIG. 1. If it is determined that the model software cannot be implemented using the existing hardware structure, the computer device can perform step (350). If it is determined that the model software can be implemented using the existing hardware structure, the computer device can perform step (360).

[0041] In step (350), the computer device can add a hardware module. For example, the computer device can add a hardware module to the existing hardware structure. Thereafter, the computer device can perform steps (330) and (340) again based on the existing hardware structure to which the hardware module has been added. These steps (330) to (350) can be repeated until it is determined in step (340) that the corresponding model software can be implemented with the existing hardware structure. In other words, if the computer device determines through source code level simulation and instruction level simulation that implementation with the existing hardware structure is possible, the computer device can move on to lower-level debugging, and if implementation with the existing hardware structure is not possible, the computer device can add a hardware module and re-verify the instruction.

[0042] At step (360), the computer device can create modular test cases for each instruction. Here, each instruction may refer to an instruction cross-verified through source code-level simulation and instruction-level simulation. As previously described, because the register transfer level simulator (130) is too slow, test cases can be created by dividing the code into modules and verifying the created test cases. Step (360) may be an example of a process for creating these test cases in modular units.

[0043] In step (370), the computer device can perform register transfer level simulation verification for the main test cases. This register transfer level simulation can be performed through a register transfer level simulator (130).

[0044] At this time, sub-debugging may mean performing verification at the instruction level and register transfer level for major test cases. In other words, in step (370), the computer device may perform cross-validation of major test cases by performing verification using the instruction level simulator (120) in addition to verification using the register transfer level simulator (130) for major test cases.

[0045] At step (380), the computer device can deploy model hardware. Here, the model hardware may include a hardware structure whose lower-level debugging has been completed by adding hardware modules to the existing hardware structure at step (350). This hardware structure may correspond to the hardware structure (170) described above with reference to FIG. 1. In other words, after verification is completed, the computer device can provide model instructions (160) and the hardware structure (170) as output.

[0046] Additionally, for high-level debugging, GPUs can be utilized to accelerate simulation processing. For example, the computer can handle cross-validation through source code-level simulation and instruction-level simulation using the GPU. Meanwhile, for low-level debugging, while GPUs cannot be utilized due to the nature of register-transfer-level simulation, multithreading can be used to accelerate data loading during the LLM (Large Language Model) data loading process into the simulator. For example, the computer can handle multithreaded loading of model software files for register-transfer-level simulation.

[0047] FIG. 4 is a diagram illustrating an example of a process for processing an AI model with large data using a register transfer level simulator according to an embodiment of the present invention. While loading a large model, such as a generative giant AI model, would normally be impossible due to memory shortages, embodiments of the present invention utilize the C / C++ language to utilize the DRAM (Dynamic Random Access Memory) of a server system to load the model. Furthermore, rather than loading a single large file, files are separated by memory channel, enabling rapid loading into memory through multithreading.

[0048] In this way, according to embodiments of the present invention, a method and system for efficiently mapping a generative large-scale artificial intelligence model to hardware can be provided.

[0049] FIG. 5 is a block diagram illustrating an example of a computer device according to an embodiment of the present invention. As illustrated in FIG. 5, the computer device (Computer device) 500 may include a memory (Memory) 510, a processor (Processor) 520, a communication interface (Communication interface) 530, and an input / output interface (I / O interface) 540. The memory (510) is a computer-readable recording medium and may include a random access memory (RAM), a read only memory (ROM), and a permanent mass storage device such as a disk drive. Here, the permanent mass storage device such as the ROM and the disk drive may be included in the computer device (500) as a separate permanent storage device distinct from the memory (510). In addition, the memory (510) may store an operating system and at least one program code. These software components may be loaded into the memory (510) from a computer-readable recording medium separate from the memory (510). This separate computer-readable recording medium may include a computer-readable recording medium such as a floppy drive, a disk, a tape, a DVD / CD-ROM drive, a memory card, etc. In another embodiment, the software components may be loaded into the memory (510) via a communication interface (530) other than a computer-readable recording medium. For example, the software components may be loaded into the memory (510) of the computer device (500) based on a computer program installed by files received via a network (Network, 560).

[0050] The processor (520) may be configured to process instructions of a computer program by performing basic arithmetic, logic, and input / output operations. Instructions may be provided to the processor (520) via the memory (510) or the communication interface (530). For example, the processor (520) may be configured to execute instructions received according to program code stored in a storage device such as the memory (510).

[0051] The communication interface (530) may provide a function for the computer device (500) to communicate with other devices via a network (560). For example, requests, commands, data, files, etc. generated by the processor (520) of the computer device (500) according to program codes stored in a recording device such as a memory (510) may be transmitted to other devices via the network (560) under the control of the communication interface (530). Conversely, signals, commands, data, files, etc. from other devices may be received by the computer device (500) via the communication interface (530) of the computer device (500) via the network (560). Signals, commands, data, etc. received via the communication interface (530) may be transmitted to the processor (520) or the memory (510), and files, etc. may be stored in a storage medium (the aforementioned permanent storage device) that the computer device (500) may further include.

[0052] The input / output interface (540) may be a means for interfacing with an input / output device (I / O device, 550). For example, the input device may include a device such as a microphone, a keyboard, or a mouse, and the output device may include a device such as a display or a speaker. As another example, the input / output interface (540) may be a means for interfacing with a device that integrates input and output functions, such as a touchscreen. The input / output device (550) may also be configured as a single device with the computer device (500).

[0053] Additionally, in other embodiments, the computer device (500) may include fewer or more components than those illustrated in FIG. 5 . However, it is not necessary to explicitly illustrate most conventional components. For example, the computer device (500) may be implemented to include at least some of the input / output devices (550) described above, or may further include other components such as a transceiver, a database, and the like.

[0054] Although the embodiments of the present invention have been described with reference to the attached drawings, those skilled in the art will appreciate that the present invention can be implemented in other specific forms without altering the technical concept or essential features thereof. Therefore, the embodiments described above should be understood to be illustrative in all respects and not restrictive.

Claims

1. A hardware mapping method of a computer device including at least one processor, A step of receiving model software by at least one processor; and A step of sequentially performing source code level simulation, instruction level simulation, and register transfer level simulation for the model software by at least one processor. Including, The steps performed above are: A step of cross-verifying the instructions written through the above source code level simulation through the above instruction level simulation. A hardware mapping method comprising:

2. A hardware mapping method of a computer device including at least one processor, A step of receiving model software by at least one processor; A step of further inputting information about an existing hardware structure by at least one processor; and A step of sequentially performing source code level simulation, instruction level simulation, and register transfer level simulation for the model software by at least one processor. Including, The steps performed above are: A step of determining whether the model software can be implemented with the existing hardware structure; If the above implementation is impossible, a step of adding a hardware module to the existing hardware structure; and Step of re-performing the above instruction level simulation A hardware mapping method comprising:

3. A hardware mapping method of a computer device including at least one processor, A step of receiving model software by at least one processor; and A step of sequentially performing source code level simulation, instruction level simulation, and register transfer level simulation for the model software by at least one processor. Including, The steps performed above are: A step of creating a module-level test case for each instruction cross-verified through the source code level simulation and the instruction level simulation; and A step of verifying at least one of the above-described test cases through the register transfer level simulation. A hardware mapping method comprising:

4. In paragraph 3, The above verification steps are: A hardware mapping method characterized by cross-verifying through the register transfer level simulation and the instruction level simulation.

5. A hardware mapping method of a computer device including at least one processor, A step of receiving model software by at least one processor; and A step of sequentially performing source code level simulation, instruction level simulation, and register transfer level simulation for the model software by at least one processor. Including, The steps performed above are: A step of processing cross-validation through the source code level simulation and the instruction level simulation using a GPU (Graphic Processing Unit); and A step for handling multi-threaded loading of files of the model software for the above register transfer level simulation. A hardware mapping method comprising:

6. In paragraph 5, The steps for handling the above multi-thread loading are: A hardware mapping method characterized in that the file of the above model software is separated by channel of memory and loaded into the memory using multi-threading.

7. A computer program stored on a computer-readable recording medium for executing the method of any one of claims 1 to 6 on a computer device by being combined with a computer device.

8. A computer-readable recording medium having recorded thereon a program for executing the method of any one of clauses 1 to 6 on a computer device.

9. At least one processor implemented to execute instructions readable by a computer device. Including, By at least one processor, Input the model software, Sequentially perform source code level simulation, instruction level simulation, and register transfer level simulation for the above model software, Cross-verifying the instructions written through the above source code level simulation through the above instruction level simulation. A computer device characterized by:

10. At least one processor implemented to execute instructions readable by a computer device. Including, By at least one processor, Input the model software, Get more information about your existing hardware structure, Sequentially perform source code level simulation, instruction level simulation, and register transfer level simulation for the above model software, Determine whether the above model software can be implemented with the above existing hardware structure, If the above implementation is not possible, add a hardware module to the existing hardware structure, Re-performing the above instruction level simulation A computer device characterized by:

11. At least one processor implemented to execute instructions readable by a computer device. Including, By at least one processor, Input the model software, Sequentially perform source code level simulation, instruction level simulation, and register transfer level simulation for the above model software, In the above sequentially performed process, by at least one processor, For each instruction cross-verified through the above source code level simulation and the above instruction level simulation, a module-level test case is created, Verifying at least one of the above-described test cases through the register transfer level simulation. A computer device characterized by:

12. In paragraph 11, To verify at least one of the above-described test cases, by at least one processor, Cross-verification through the above register transfer level simulation and the above instruction level simulation A computer device characterized by:

13. At least one processor implemented to execute instructions readable by a computer device. Including, By at least one processor, Input the model software, Sequentially perform source code level simulation, instruction level simulation, and register transfer level simulation for the above model software, Processing cross-verification through the source code level simulation and the instruction level simulation by the GPU (Graphic Processing Unit) included in the at least one processor, Processing multi-threaded loading of the file of the model software for the register transfer level simulation by the CPU (Central Processing Unit) included in the at least one processor A computer device characterized by:

Images