Method and system for verifying operation and data precision of hardware for artificial intelligence

Abstract

Provided are a method and system for verifying the operation and data precision of large generative artificial intelligence hardware. The verification method according to an embodiment may comprise the steps of: receiving target device information, model instructions, and model parameters for an artificial intelligence model; configuring a simulator corresponding to actual hardware on the basis of the target device information; processing operations between the model instructions and the model parameters via the configured simulator; and storing the processing results of the operations in a memory module included in the simulator.

Description

Method and system for verifying the operation and data accuracy of hardware for artificial intelligence

[0001] Embodiments of the present invention relate to a method and system for verifying the operation and data accuracy of generative large-scale artificial intelligence hardware, and to a hardware simulator framework optimized for large-scale artificial intelligence models.

[0002] Recently deployed large-scale AI models possess data volumes significantly larger than those of conventional convolutional neural networks (CNNs). For example, with the release of large AI models measuring tens of gigabytes (GB), hardware acceleration is also becoming increasingly necessary. However, rapid hardware development for these large AI models requires simulators that emulate the hardware.

[0003] Furthermore, the sheer volume of data in large AI models makes hardware simulators burdened with loading and processing models. Therefore, to perform large-scale comparisons, simulator acceleration is essential.

[0004] Furthermore, AI models frequently reduce the size of their computational units by reducing data types. However, this can impact model accuracy, so it's crucial to determine the extent to which computational units can be reduced. However, because computational units are ultimately used within the CPU (Central Processing Unit), it's difficult to determine the extent to which computational units can be reduced unless implemented in software. Furthermore, the computational load in large AI models is significantly larger, making this even more problematic.

[0005] A method and system for verifying the operation and data precision of generative giant artificial intelligence hardware can be provided.

[0006] The technical problems of the present invention are not limited to the technical problems mentioned above, and other technical problems not mentioned will be clearly understood by those skilled in the art from the description below.

[0007] A verification method for a computer device including at least one processor, comprising: a step of receiving, by the at least one processor, target device information, model instructions, and model parameters for an artificial intelligence model; a step of configuring, by the at least one processor, a simulator corresponding to actual hardware based on the target device information; a step of processing, by the at least one processor, an operation between the model instructions and the model parameters through the configured simulator; and a step of storing, by the at least one processor, a processing result of the operation in a memory module included in the simulator, wherein the at least one processor includes a CPU (Central Processing Unit) and a GPU (Graphics Processing Unit), and the step of configuring the simulator includes: configuring a first simulator using the CPU when a high-precision mode is selected, and configuring a second simulator using both the CPU and the GPU when a low-latency mode is selected.

[0008] According to one aspect, the step of configuring the simulator may include: determining a mode of the simulator as one of the high-precision mode and the low-latency mode; configuring a first simulator that implements at least one module for processing the operation and the memory module using the CPU in response to the mode of the simulator being determined to be the high-precision mode; and configuring a second simulator that implements the module for processing the operation using the GPU and the memory module using the CPU in response to the mode of the simulator being determined to be the low-latency mode.

[0009] According to another aspect, the step of configuring the first simulator may be characterized by allocating a module for processing the operation and the memory module to the CPU.

[0010] According to another aspect, the step of configuring the second simulator may be characterized by allocating a module for processing the operation to a GPU and allocating the memory module to the CPU.

[0011] According to another aspect, the determining step may be characterized in that the mode of the simulator is determined as one of the high-precision mode and the low-latency mode according to at least one of the length of the layer included in the artificial intelligence model, the number of the layers, the length of the token for the artificial intelligence model, and the number of the tokens.

[0012] According to another aspect, in the high-precision mode, it is possible to confirm the intermediate value of the operation in each module of the first simulator, and in the low-delay mode, it is possible to confirm the final value of the operation in the memory module of the second simulator.

[0013] According to another aspect, the second simulator may be characterized in that it reverse-operates the weight matrix whose value positions are adjusted when mapping the weight matrix of the artificial intelligence model to return it to the original weight matrix.

[0014] According to another aspect, the second simulator may be characterized by processing the operation through a CUDA kernel of the GPU.

[0015] A computer program stored on a computer-readable recording medium is provided for executing the above method on a computer device in combination with a computer device.

[0016] A computer-readable recording medium having recorded thereon a program for executing the above method on a computer device is provided.

[0017] A computer device is provided, comprising at least one processor implemented to execute instructions readable from a computer device, characterized in that model software is input by the at least one processor and source code level simulation, instruction level simulation, and register transfer level simulation for the model software are sequentially performed.

[0018] Specific details of other embodiments are included in the detailed description and drawings.

[0019] A method and system for verifying the operation and data precision of generative giant artificial intelligence hardware can be provided.

[0020] The effects of the present invention are not limited to the effects mentioned above, and other effects not mentioned will be clearly understood by those skilled in the art from the description of the claims.

[0021] FIG. 1 is a diagram illustrating an example of the structure of a latency processing unit according to an embodiment of the present invention.

[0022] FIGS. 2 to 5 are drawings illustrating examples of implementation models of LPUs according to embodiments of the present invention.

[0023] FIG. 6 is a diagram for explaining weight matrix data mapping of a high-bandwidth memory for matrix multiplication operation of a latency processing unit according to an embodiment of the present invention.

[0024] FIG. 7 is a diagram for explaining a high-bandwidth memory interface included in a latency processing unit according to one embodiment of the present invention.

[0025] FIG. 8 is a diagram illustrating a reconfigurable multi-function operation unit included in a latency processing unit according to one embodiment of the present invention.

[0026] FIG. 9 is a diagram for explaining the configuration of an address-based non-sequential multi-unit scheduler included in a latency processing unit according to one embodiment of the present invention.

[0027] FIG. 10 is a drawing showing an example of the structure of a simulator according to one embodiment of the present invention.

[0028] FIG. 11 is a drawing for explaining a high-precision mode and a low-delay mode in one embodiment of the present invention.

[0029] FIG. 12 is a diagram illustrating an example of assigning a module to a GPU in one embodiment of the present invention.

[0030] Figure 13 is a flowchart illustrating an example of a verification method according to one embodiment of the present invention.

[0031] FIG. 14 is a block diagram illustrating an example of a computer device according to one embodiment of the present invention.

[0032] The advantages and features of the present invention, and the methods for achieving them, will become clearer with reference to the embodiments described in detail below together with the accompanying drawings. However, the present invention is not limited to the embodiments disclosed below, but may be implemented in various different forms. These embodiments are provided only to ensure that the disclosure of the present invention is complete and to fully inform those skilled in the art of the scope of the invention, and the present invention is defined only by the scope of the claims. Like reference numerals designate like elements throughout the specification.

[0033] When one component is referred to as being "connected to" or "coupled to" another component, it includes both cases where it is directly connected or coupled to the other component, or cases where there is another component intervening therebetween. Conversely, when one component is referred to as being "directly connected to" or "directly coupled to" another component, it indicates that there is no other component intervening therebetween. "And / or" includes each and any combination of one or more of the mentioned items.

[0034] The terminology used herein is for the purpose of describing embodiments only and is not intended to limit the present invention. In this specification, the singular also includes the plural unless specifically stated otherwise. As used herein, the terms "comprises" and / or "comprising" do not exclude the presence or addition of one or more other components, steps, operations, and / or elements.

[0035] Although terms like "first" and "second" are used to describe various components, these components are not limited by these terms. These terms are merely used to distinguish one component from another. Therefore, it should be understood that a "first" component referred to below may also be a "second" component within the technical scope of the present invention.

[0036] Unless otherwise defined, all terms (including technical and scientific terms) used herein may be used in their common sense to those of ordinary skill in the art to which the present invention pertains. Furthermore, terms defined in commonly used dictionaries are not to be interpreted ideally or excessively unless explicitly and specifically defined otherwise.

[0037] First, Figures 1 to 9 describe examples of hardware structures.

[0038] FIG. 1 is a diagram illustrating an example of the structure of a latency processing unit according to an embodiment of the present invention.

[0039] Referring to FIG. 1, an LPU (Latency Processing Unit) 100 according to an embodiment of the present invention may include an SMA (Streamlined Memory Access) 110, an OIU (Operand Issue Unit) 120, an SXE (Streamlined eXecution Engine) 130, a VXE (Vector eXecution Engine) 140, an LMU (Local Memory Unit) 150, an ISU (Instruction Scheduling Unit) 160, a PCIe (Peripheral Component Interconnect express) Interface 170, and a P2P (Peer to Peer) Interface 180.

[0040] The SMA (110) may be a specialized DMA (Direct Memory Access). For example, the SMA (110) may connect all channels (e.g., 32) of the HBM (200) to the execution engine (e.g., the SEE (130)) to transfer half-precision floating point (FP16) data at maximum bandwidth. The SMA (110) may be designed with a deep FIFO (First In First Out) to transfer sequential memory requests based on preloaded memory (MEM) instructions. Hardware-based memory mapping may reduce latency by eliminating matrix modification or transposition operations. Therefore, the SMA (110) may stream data received in a maximum burst size to the execution engine with minimal latency. The SMA (110) may also efficiently perform matrix transposes using a strobe signal. Streaming data may contain parameters for vector matrix operations (e.g., weights, biases) and other vector-related operations (e.g., gamma / beta, embedding).

[0041] The OIU (120) can coordinate data streamed from the SMA (110) (e.g., a first operand) with input from on-chip memory (e.g., a second operand) before issuing it to the execution engine. Based on the execute (EXE) instruction, the OIU (120) can generate microcode that configures the execution engine and determines the target engine of the operand. In addition, the OIU (120) can be provided with a reusable buffer to eliminate read latency of static operands (e.g., input vectors) and an asymmetric buffer to hold vectorized data used as scalars (e.g., biases). Therefore, appropriate operands are almost always prefetched and ready to be issued to the execution engine immediately.

[0042] The SXE (130) is the main computing hardware of the LPU (100), and can be designed to maximize the incoming bandwidth to execute vector-matrix multiplication (V·M) such as attention, 1D convolution, and feedforward network. The SXE (130) can include a number of MAC (multiply-and-accumulation) trees (MAC Trees, 131) that can match the reception bandwidth and the computation bandwidth of the HBM (200). For example, if 1024 elements are received from the HBM (200) per cycle, the reception bandwidth and the computation bandwidth can be matched through 16 MAC trees (131) each having 64 input vectors. In addition, the MAC tree (131) with 64 input vectors can be composed of 64 multipliers and 63 adders.

[0043] A plurality of MAC trees (131) can perform matrix multiplication operations and can be connected by channel through the HBM (200), which is an external memory and a high-bandwidth memory, and the SMA (110). Specifically, one of the plurality of MAC trees (131) is connected to the HBM (200) through one channel, thereby maximizing the transmission bandwidth between the LPU (100) and the HBM (200), and performing the matrix multiplication operations required for the ultra-large artificial intelligence model without a bottleneck. Therefore, the number of the plurality of MAC trees (131) and the number of memory channels of the HBM (200) can be configured to be the same.

[0044] The matrix multiplication operation results of multiple MAC trees (131) can be provided to the VXE (140). The VXE (140) can be implemented using a custom low-latency ALU (Arithmetic Logic Unit) and can execute vector operations such as token embedding, softmax, normalization, and residual operations. Since these vector operations occur relatively infrequently, the fan-in from the OIU (120) to this path can be adjusted to reduce hardware resources with negligible performance loss. The VXE (140) can receive the operation results of multiple MAC trees (131) and receive activation values ​​from the LMU (150) to perform subsequent operations. The VXE (140) can be configured to include various combinations of operators by including multiple multi-function operation datapaths.

[0045] The LMU (150) can transmit activation values ​​to multiple MAC trees (131) and VXEs (140). At this time, the LMU (150) can copy and transmit the activation values ​​in order to transmit the same activation values ​​to multiple MAC trees (131). In addition, the LMU (150) can store the result values ​​calculated by the multiple MAC trees (131) and VXEs (140). In other words, the LMU (150) can function as an internal buffer corresponding to the HBM (200) within the LPU (100). At this time, the LPU (100) can store activation values ​​or model parameters with a high reuse rate in the LMU (150) and store weights with a low reuse rate in the HBM (200) in the matrix multiplication operation. The LMU (150) can be implemented as a 4MB multi-bank register file with scalar vector separation for fast, high-bandwidth access to input, output, and intermediate data. Additionally, the LMU (150) may be multi-ported to support simultaneous reads and writes in the write-back phase of the OIU (120) and the execution engine.

[0046] The ISU (160) can control the entire execution flow of the LPU (100). The ISU (160) can utilize the PIC (Parallel Instruction Chaining) method, and by using instruction chaining, dependent instructions can be executed sequentially. The PIC separates instructions requiring independent hardware into groups of dependent instructions (e.g., memory (MEM) instructions, execution (EXE) instructions, network (NET) instructions), so that all instructions can be executed in parallel with the instruction chain of each group, thereby achieving low control overhead and latency savings. The ISU (160) can also update control registers (e.g., token and layer number) for engine execution. The internal scheduler supports out-of-order execution of the SXE (130) and VXE (140) to maximize hardware utilization, and a powerful scoreboard can be designed to handle data risks. For example, the ISU (160) can schedule multiple MAC trees (131) and VXEs (140) to perform operations simultaneously. In addition, the ISU (160) can improve operation throughput and delay time by pre-executing instructions without dependencies to maximize parallel operations, thereby minimizing the idle time of each operation unit and memory access unit.

[0047] The LPU (100) can be connected to a host computer via a PCIe interface (170), and can receive commands required for the operation of the LPU (100) and input values ​​and weights of a large-scale artificial intelligence model from the host computer, perform calculations, and then transmit the results to the host computer.

[0048] The LPU (100) can be scaled out into a cluster of multiple LPUs connected via a P2P interface (180). The expanded cluster structure can further enhance the acceleration of computation of ultra-large artificial intelligence models.

[0049] FIGS. 2 to 5 are diagrams illustrating examples of implementation models of LPUs according to embodiments of the present invention. The embodiment of FIG. 1 previously described an example of an implementation model using external memory of HBM (200). Instead of HBM (200), DDR (Double Data Rate) may be used as the external memory. In this case, since it is difficult to store a large model in a single device, it may be divided into multiple partitions and stored in external memories for multiple devices (multiple LPUs) for each partition. In this case, synchronization between multiple devices may be required for inference of the large model.

[0050] The embodiment of FIG. 2 illustrates a plurality of external memories (320) storing a plurality of partitions (310) of a large model, similar to the embodiment described above in FIG. 1, and a plurality of LPUs (330) connected in parallel to the plurality of external memories (320). One LPU can be implemented in one FPGA (Field Programmable Gate Array), and one partition can be connected in parallel to one FPGA. The transformer structure includes multi-head attention, layer normalization, feed forward, etc. in the decoder layer, and the multi-head attention and feed forward can be model-parallelized. In this case, when the multi-head attention is terminated, one embedding vector can be output as a result. Since one device only has a portion of the embedding vector, synchronization may be required because multiple devices need to share each embedding vector in order to proceed to the next operation. At this time, considering scalability, a single LPU may be implemented with multiple external memories (e.g., two or four). For example, the embodiment of FIG. 1 illustrates an example in which two HBMs (200), each storing one partition, are used.

[0051] The embodiment of Fig. 3 illustrates an example of a PIM (Processing-in-Memory) model, in which one LPU is implemented as a PIM chip, and both the partition and the LPU computation unit are integrated into a single chip. The embodiment of Fig. 3 illustrates a plurality of LPUs (410), a plurality of partitions (310), and a plurality of LPU computation units (420), each of which can be implemented as a PIM chip. In this case, each of the plurality of LPUs (410) can include one partition and one LPU computation unit.

[0052] The embodiment of Fig. 4 illustrates an example of a Processing-near-Memory (PNM) model. It may be difficult to include a configuration for processing all LPU operations within a single PIM chip. The embodiment of Fig. 4 illustrates a model in which a plurality of partitions (310) are stored within a plurality of memory chips (510), and an LPU operation unit (521) for LPU operations is included within a buffer chip (520) such as a PNM chip.

[0053] The embodiment of Fig. 5 illustrates an example of a model combining PIM and PNM. For example, multiple partitions (310) may be stored in multiple memory chips (610). In addition, PIM-style LPU operation units (611) may be implemented as accumulation units, such as MAC trees, in each of the multiple memory chips (610). In this case, the LPU operation units (621) for the remaining high-level operations of the LPU may be implemented in the PNM manner in the buffer chip (620).

[0054] FIG. 6 is a diagram for explaining weight matrix data mapping of a high-bandwidth memory for matrix multiplication operation of a latency processing unit according to an embodiment of the present invention.

[0055] Referring to FIG. 6, the LPU (100) according to the present embodiment can store weight matrix data mapped to a high-bandwidth memory (610) such as HBM (200) so that the weight data can be retrieved without accessing another memory channel when performing matrix multiplication operation on each MAC tree, since the number of MAC trees constituting multiple MAC trees (131) and the number of memory channels of SMA (110) are the same.

[0056] Specifically, weight matrix data may be stored in the high-bandwidth memory (610) so that it is mapped to each channel (620-n) in the column direction (D1) of the weight matrix in the number of multiple MAC trees (131). Since the column direction in the weight matrix allows matrix multiplication operations to be performed in parallel, the multiple MAC trees (131) can read column direction data from their respective allocated memory channels (620-n) and perform matrix multiplication operations.

[0057] Next, multiple MAC trees (131) can map the weight matrix data so that they can accumulate in the row direction (D2) of the weight matrix to complete the final operation result. The number of row data mapped at once can be determined by the bandwidth of the high-bandwidth memory (610), and this can be determined by the size of the tiles that multiple MAC trees (131) can process at once.

[0058] FIG. 7 is a diagram for explaining a high-bandwidth memory interface included in a latency processing unit according to one embodiment of the present invention.

[0059] Referring to FIG. 7, the SMA (110) can connect the LMU (150), multiple MAC trees (131), and high-bandwidth memory (610). The SMA (110) is not connected to other computational units of the LPU (100), and thus, the high-bandwidth memory interface can be minimized in terms of hardware resources.

[0060] Multiple MAC trees (131) and memory channels (620-n) can be connected in a one-to-one correspondence, respectively. That is, multiple MAC trees (131) do not need to access channels other than the directly assigned channels, and thus can perform matrix multiplication operations without using complex interfaces such as cross-bars, which consume a lot of resources and have a high delay time.

[0061] SMA (110) can configure only a read interface for loading weight matrix data stored in high-bandwidth memory (610) of multiple MAC trees (131). In other words, as described later, since the result of the operation is stored in the high-bandwidth memory (610) through LMU (150), a write interface for the high-bandwidth memory (610) of multiple MAC trees (131) is not configured, and hardware resources can be reduced accordingly.

[0062] In contrast, the SMA (110) can only form a write interface between the LMU (150) and the high-bandwidth memory (610). The computational results stored in the LMU (150) as an internal buffer through the SMA (110) can be transmitted to be written to the high-bandwidth memory (610), and the memory channel of the writing target can be selected using the demultiplexer (710).

[0063] FIG. 8 is a diagram illustrating a reconfigurable multi-function operation unit included in a latency processing unit according to one embodiment of the present invention.

[0064] Referring to FIG. 8, the VXE (140) may include a plurality of multi-function operation data paths (810, 820), and the plurality of multi-function operation data paths (810, 820) may be connected to an operator / result value chain network (830) to form various combinations of operators.

[0065] As illustrated in FIG. 8, a plurality of multi-functional operation datapaths (810, 820) may include various operation units required for, for example, a lookup table-based nonlinear activation function, a masking operation, etc. However, the configuration of the operation units of the reconfigurable multi-functional operation datapaths (810, 820) illustrated in FIG. 8 is exemplary, and it goes without saying that additional operation units required for large-scale model operation may be included in the multi-functional operation datapaths (810, 820). The result calculated by the VXE (140) may be transmitted to the LMU (150).

[0066] FIG. 9 is a diagram for explaining the configuration of an address-based non-sequential multi-unit scheduler included in a latency processing unit according to one embodiment of the present invention.

[0067] Referring to FIG. 9, the ISU (160) as an address-based out-of-order multi-unit scheduler included in the LPU (100) according to the present embodiment may include an address-based instruction dependency determination and scheduling controller (910), a plurality of instruction issue controllers (921, 922, 923, 924), a multi-bank buffer address status table (950), an instruction buffer (960) and result address status update logic (970), and a multi-unit instruction dispatcher (980).

[0068] The ISU (160) can operate each operation unit and data movement unit simultaneously through the address-based instruction dependency judgment and scheduling controller (910) and multiple instruction issue controllers (921, 922, 923, 924). At this time, the ISU (160) can change the status of the operator address and result address of the multi-bank buffer address status table (950) for the instruction performed in each operation unit to 1.

[0069] The multi-bank buffer address status table (950) can change the status of the result address of a completed instruction to 0 through the result address status update logic (970).

[0070] The address-based instruction dependency determination and scheduling controller (910) can determine the dependency between the instruction to be executed and the instruction being executed, as well as the dependency between the instructions to be executed, by referencing the states of the addresses through the multi-bank buffer address state table (970). This enables preprocessing of instructions without dependencies, thereby minimizing the idle time of each operation unit and data movement unit.

[0071] The address-based instruction dependency determination and scheduling controller (910) included in the ISU (160) can load and process instructions from the instruction buffer (960). At this time, the address-based instruction dependency determination and scheduling controller (910) performs a loop instruction, and decodes other instructions to distinguish the instructions and transmit them to the device-to-device instruction issue controller (921), the direct memory access instruction issue controller (922), the MAC tree instruction issue controller (923), and the reconfigurable multi-function operation unit instruction issue controller (924) through the multi-unit instruction dispatcher (980).

[0072] The ISU (160) receives and stores commands of the LPU (100) from the host computer through the PCIe interface (170), and can store the current status of the LPU (100) in a status register. The host computer can check the status register through the PCIe interface (170).

[0073] Embodiments of the present invention provide a structure that utilizes a GPU (Graphics Processing Unit) device in a simulator that mimics the hardware architecture, enabling rapid processing of even large models. Furthermore, the GPU's CUDA (Compute Unified Device Architecture) kernel is utilized to emulate data types commonly used in machine learning down to the computational level, ensuring the accuracy of the final result without compromising speed.

[0074] According to one embodiment, a simulator can be implemented to utilize not only the Central Processing Unit (CPU) but also the GPU. By accelerating the most computationally intensive modules of the simulator on the GPU, computations can be processed more quickly. However, this poses the challenge of extracting intermediate values ​​during simulator processing. This problem can be addressed by supporting both CPU and GPU modes. For small scopes, the CPU can handle all processing, enabling detailed debugging. For larger code, the GPU can be actively utilized for rapid debugging. Furthermore, when using the GPU, CUDA kernels are used for computations, allowing access to all CUDA-supported operators. This allows for high-precision and low-precision judgments, even on large AI models, by modifying data types at the operator level.

[0075] FIG. 10 is a diagram illustrating an example of the structure of a simulator according to an embodiment of the present invention. The simulator top module (1000) according to the present embodiment is an example of a simulator and may include various modules to operate in the same manner as actual hardware. The modules included in the simulator top module (100) may vary depending on the hardware to be supported for simulation. In the embodiment of FIG. 10, the simulator top module (1000) implemented for simulation of the LPU (100) described above as hardware is shown. In the present embodiment, the simulator top module (1000) may include an HBM module (1010), a DMA module (1020), an operator module (1030), a register file module (1040), a DDR module (1050), and a controller module (1060). Here, the HBM module (1010) can correspond to the HBM (200), the DMA module (1020) can correspond to the SMA (110), the operator module (1030) can correspond to the SXE (130), the register file module (1040) can correspond to the VXE (140), and the controller module (1060) can correspond to the ISU (160).

[0076] Most operations can be processed in the calculator module (1030) and the DMA module (1020). The simulator top module (1000) can receive target device information (1071), model instructions (1072), and model parameters (1073) as input values. Here, the target device information (1071) can include, for example, information such as the size and number of HBM (High Bandwidth Memory), and the simulator top module (1000) can dynamically configure internal modules based on the target device information (1071). The model instructions (1072) can be processed by the controller module (1060) and transferred to the calculator module (1030), and the model parameters (1073) can be stored in the HBM module (1010) and then transferred to the calculator module (1030) via the DMA module (1020). For example, the calculation module (1030) can process a matrix multiplication operation between an input transmitted through the controller module (1060) and a weight matrix read from the HBM module (1010) through the DMA module (1020).

[0077] In this way, the simulator top module (100) can operate instructions in the same manner as actual hardware and store the final result in the off-chip DDR (Double Data Rate, 1081) and the off-chip HBM (1082). In addition, the simulator top module (100) can dump the stored final result in HEX format so that it can be confirmed. Here, the HEX format may refer to a file format that transmits binary information in ASCII text format. In addition, the simulator top module (100) can store the intermediate value of the operation transmitted to the register file module (1040) in the on-chip register (1083), thereby enabling confirmation of the intermediate value.

[0078] FIG. 11 is a diagram illustrating a high-precision mode and a low-latency mode according to an embodiment of the present invention. In the high-precision mode, all operations of the simulator can be processed on the CPU. Since it operates like a typical simulator, intermediate values ​​can be checked anywhere. On the other hand, in the low-latency mode, all modules where operations occur (for example, the DMA module (1020), the operator module (1030), the register file module (1040), and the controller module (1060)) can be allocated to the GPU. Since all modules where operations occur are allocated to the GPU, checking intermediate values ​​may result in a decrease in speed, making it impossible to check intermediate values. However, since the DDR module (1150) and the HBM module (1010), where the final values ​​are stored, are located on the CPU side, it is possible to check the final values.

[0079] High-precision mode is useful for low-computation scenarios, such as those with a small number of layers or tokens, while low-latency mode is useful for high-computation scenarios, such as those with long layers and long token lengths. Therefore, for example, debugging can proceed in the following order: Find bugs in high-precision mode and then identify edge cases in low-latency mode.

[0080] FIG. 12 is a diagram illustrating an example of assigning a module to a GPU in one embodiment of the present invention.

[0081] The order of values ​​in the weight matrix of an AI model can change. For example, the rotary embedding operation in a transformer model requires performing a matrix multiplication operation followed by an operation on values ​​located far apart. This congests hardware routing, so the order of values ​​in the weight matrix can be changed during memory mapping, moving the values ​​used for the operation to adjacent locations.

[0082] In this case, the DMA module (1020) can reverse the model mapping to return it to the original weight matrix. In this case, when using a GPU, the model mapping can be reversed using a linear library function familiar to the GPU without using a programming language statement such as a 'for loop' familiar to the CPU. Therefore, the reverse operation for the model mapping can be efficiently processed on both the CPU and the GPU. Since the model mapping is reversed to the original matrix in the DMA module (1020), the operator module (1030) can utilize linear library operations such as matrix multiplication (matmul). These are also operations familiar to both the CPU and the GPU.

[0083] Additionally, when processing linear library operations on the GPU, they can be computed through CUDA kernels, in which case the GPU's operators can be utilized. This means that the data precision can be mimicked similarly to the actual RTL. For example, when processing 16-bit floating point (float16) on the CPU, since the CPU does not have a float16 operator, float16 can be converted to float32 and the float32 operator can be utilized. This results in more precise calculations than the actual RTL simulation, making it difficult to accurately mimic the RTL simulation. On the other hand, the GPU has operators for various data types that are frequently used in the artificial intelligence inference process, making it possible to mimic the RTL simulation more accurately.

[0084] FIG. 13 is a flowchart illustrating an example of a verification method according to an embodiment of the present invention. The verification method according to the present embodiment may be performed by at least one computer device. At least one processor of the computer device may be implemented to execute control instructions based on the code of an operating system included in a memory or the code of at least one computer program. Here, at least one processor may control the computer device so that the computer device performs steps (1310 to 1340) included in the method of FIG. 13 according to the control instructions provided by the code stored in the computer device. An example of the structure of the computer device will be described in more detail later with reference to FIG. 14.

[0085] In step (1310), the computer device may receive target device information, model instructions, and model parameters for an artificial intelligence model. Here, the target device information, model instructions, and model parameters may correspond to the target device information (1071), model instructions (1072), and model parameters (1073) described with reference to FIG. 10, respectively.

[0086] In step (1320), the computer device may configure a simulator corresponding to actual hardware based on target device information. At this time, at least one processor included in the computer device may include a CPU (Central Processing Unit) and a GPU (Graphics Processing Unit). In this case, the computer device may configure a first simulator utilizing the CPU when the high-precision mode is selected in step (1320), and a second simulator utilizing both the CPU and GPU when the low-latency mode is selected. More specifically, step (1320) may include steps (1321) to (1324).

[0087] In step (1321), the computer device may determine the mode of the simulator as either a high-precision mode or a low-latency mode. For example, the computer device may determine the mode of the simulator as either a high-precision mode or a low-latency mode based on at least one of the length of layers included in the artificial intelligence model, the number of layers, the length of tokens for the artificial intelligence model, and the number of tokens.

[0088] In step (1322), the computer device can determine whether the simulator mode is determined to be high-precision mode. If the simulator mode is determined to be high-precision mode, the computer device can perform step (1323). If the simulator mode is not determined to be high-precision mode, i.e., if the simulator mode is determined to be low-latency mode, the computer device can perform step (1324).

[0089] In step (1323), the computer device may configure a first simulator that implements at least one module for processing operations and a memory module using a CPU. For example, the computer device may configure the first simulator by allocating a module for processing operations and the memory module to a CPU.

[0090] In step (1324), the computer device may configure a second simulator that implements a module for processing operations using a GPU and a memory module using a CPU. For example, the computer device may configure the second simulator by assigning the module for processing operations to a GPU and the memory module to a CPU. At this time, the second simulator may perform an inverse operation on the weight matrix whose value positions have been adjusted when mapping the weight matrix of the artificial intelligence model to return it to the original weight matrix.

[0091] At step (1330), the computer device can process operations between model instructions and model parameters through the configured simulators. For example, the first simulator can process operations through the CPU, and the second simulator can process operations through the CUDA kernel of the GPU.

[0092] In step (1340), the computer device can store the processing results of the operation in a memory module included in the simulator. In high-precision mode, the intermediate values ​​of the operation can be confirmed in each module of the first simulator. Furthermore, in low-latency mode, the final values ​​of the operation can be confirmed in the memory module of the second simulator.

[0093] In this way, according to embodiments of the present invention, a method and system for verifying the operation and data precision of generative large-scale artificial intelligence hardware can be provided.

[0094] FIG. 14 is a block diagram illustrating an example of a computer device according to an embodiment of the present invention. The computer device (500) may include a memory (510), a processor (520), a communication interface (530), and an input / output interface (I / O interface) 540, as illustrated in FIG. 5. The memory (510) is a computer-readable recording medium and may include a random access memory (RAM), a read only memory (ROM), and a permanent mass storage device such as a disk drive. Here, the permanent mass storage device such as the ROM and the disk drive may be included in the computer device (500) as a separate permanent storage device distinct from the memory (510). In addition, the memory (510) may store an operating system and at least one program code. These software components may be loaded into the memory (510) from a computer-readable recording medium separate from the memory (510). This separate computer-readable recording medium may include a computer-readable recording medium such as a floppy drive, a disk, a tape, a DVD / CD-ROM drive, a memory card, etc. In another embodiment, the software components may be loaded into the memory (510) via a communication interface (530) other than a computer-readable recording medium. For example, the software components may be loaded into the memory (510) of the computer device (500) based on a computer program installed by files received via a network (Network, 560).

[0095] The processor (520) may be configured to process instructions of a computer program by performing basic arithmetic, logic, and input / output operations. Instructions may be provided to the processor (520) via the memory (510) or the communication interface (530). For example, the processor (520) may be configured to execute instructions received according to program code stored in a storage device such as the memory (510).

[0096] The communication interface (530) may provide a function for the computer device (500) to communicate with other devices via a network (560). For example, requests, commands, data, files, etc. generated by the processor (520) of the computer device (500) according to program codes stored in a recording device such as a memory (510) may be transmitted to other devices via the network (560) under the control of the communication interface (530). Conversely, signals, commands, data, files, etc. from other devices may be received by the computer device (500) via the communication interface (530) of the computer device (500) via the network (560). Signals, commands, data, etc. received via the communication interface (530) may be transmitted to the processor (520) or the memory (510), and files, etc. may be stored in a storage medium (the aforementioned permanent storage device) that the computer device (500) may further include.

[0097] The input / output interface (540) may be a means for interfacing with an input / output device (I / O device, 550). For example, the input device may include a device such as a microphone, a keyboard, or a mouse, and the output device may include a device such as a display or a speaker. As another example, the input / output interface (540) may be a means for interfacing with a device that integrates input and output functions, such as a touchscreen. The input / output device (550) may also be configured as a single device with the computer device (500).

[0098] Additionally, in other embodiments, the computer device (500) may include fewer or more components than those illustrated in FIG. 5 . However, it is not necessary to explicitly illustrate most conventional components. For example, the computer device (500) may be implemented to include at least some of the input / output devices (550) described above, or may further include other components such as a transceiver, a database, and the like.

[0099] Although the embodiments of the present invention have been described with reference to the attached drawings, those skilled in the art will appreciate that the present invention can be implemented in other specific forms without altering the technical concept or essential features thereof. Therefore, the embodiments described above should be understood to be illustrative in all respects and not restrictive.

Claims

1. A method for verifying a computer device including at least one processor, A step of receiving target device information, model instructions and model parameters for an artificial intelligence model by at least one processor; A step of configuring a simulator corresponding to actual hardware based on the target device information by at least one processor; A step of processing operations between the model instructions and the model parameters through the configured simulator by at least one processor; and A step of storing the processing result of the operation in a memory module included in the simulator by at least one processor. Including, The at least one processor includes a CPU (Central Processing Unit) and a GPU (Graphics Processing Unit), The steps for configuring the above simulator are: A step of determining the mode of the above simulator as one of a high-precision mode and a low-latency mode; In response to the mode of the simulator being determined to be the high-precision mode, a step of configuring a first simulator in which at least one module for processing the operation in the simulator and the memory module are allocated to the CPU; and In response to the mode of the above simulator being determined to be the low-latency mode, a step of configuring a second simulator in which at least one module for processing the operation is allocated to the GPU and the memory module is allocated to the CPU. A verification method characterized by including:

2. In paragraph 1, The above decision step is, A verification method characterized in that the mode of the simulator is determined as one of the high-precision mode and the low-latency mode according to at least one of the length of the layer included in the artificial intelligence model, the number of the layers, the length of the token for the artificial intelligence model, and the number of the tokens.

3. In paragraph 1, In the above high-precision mode, it is possible to check the intermediate value of the operation in each module of the first simulator, In the above low-latency mode, it is possible to check the final value of the operation in the memory module of the second simulator. A verification method characterized by .

4. In paragraph 1, In order to process the above operation, the position of the value of the weight matrix of the artificial intelligence model is adjusted during the memory mapping process, The second simulator uses the GPU-based linear library function to reverse the weight matrix whose value positions have been adjusted to return it to the original weight matrix. A verification method characterized by .

5. In paragraph 1, A verification method characterized in that the second simulator processes the operation through the CUDA kernel of the GPU.

6. A computer program stored on a computer-readable recording medium for executing the method of any one of claims 1 to 5 on a computer device in combination with a computer device.

7. A computer-readable recording medium having recorded thereon a program for executing the method of any one of clauses 1 to 5 on a computer device.

8. At least one processor implemented to execute instructions readable by a computer device. Including, By at least one processor, Input target device information, model instructions, and model parameters for the artificial intelligence model, Based on the above target device information, a simulator corresponding to actual hardware is configured, Processing operations between the model instructions and the model parameters through the above-described simulator, The processing result of the above operation is stored in the memory module included in the simulator, The at least one processor includes a CPU (Central Processing Unit) and a GPU (Graphics Processing Unit), To configure the above simulator, by at least one processor, When the high-precision mode is selected, a first simulator utilizing the CPU is configured, and when the low-latency mode is selected, a second simulator utilizing both the CPU and the GPU is configured. To configure the above simulator, by at least one processor, Determine the mode of the above simulator as either high-precision mode or low-latency mode, In response to the mode of the above simulator being determined to be the high-precision mode, at least one module for processing the operation in the above simulator and the memory module constitute a first simulator allocated to the CPU, In response to the mode of the above simulator being determined to be the low-latency mode, a second simulator is configured in which at least one module for processing the operation is allocated to the GPU and the memory module is allocated to the CPU. A computer device characterized by:

9. In paragraph 8, To determine the mode of the above simulator, by at least one processor, Determining the mode of the simulator as one of the high-precision mode and the low-latency mode according to at least one of the length of the layer included in the artificial intelligence model, the number of the layers, the length of the token for the artificial intelligence model, and the number of the tokens. A computer device characterized by:

10. In paragraph 8, In the above high-precision mode, it is possible to check the intermediate value of the operation in each module of the first simulator, In the above low-latency mode, it is possible to check the final value of the operation in the memory module of the second simulator. A computer device characterized by:

11. In paragraph 8, In order to process the above operation, the position of the value of the weight matrix of the artificial intelligence model is adjusted during the memory mapping process, The second simulator uses the GPU-based linear library function to reverse the weight matrix whose value positions have been adjusted to return it to the original weight matrix. A computer device characterized by:

Images