Integrated streaming control method and device supporting regularization and activation functions for artificial intelligence acceleration
The Tensor Streaming Core method addresses the inefficiencies in AI computation by employing delayed statistics-based normalization and conditional polynomial-based approximation, enhancing the speed and efficiency of normalization and non-linear activation functions.
Patent Information
- Application Number
- KR1020250056116
- Authority / Receiving Office
- KR · KR
- Patent Type
- Patents
- Current Assignee / Owner
- Priority Date
- 2025-02-14
- Filing Date
- 2025-04-29
- Publication Date
- 2026-11-10
- Estimated Expiration
- 2045-04-29
Smart Images

Figure 112025048537558-PAT00002_ABST
Abstract
Description
Technology Field
[0001] The present invention relates to an integrated streaming control method and apparatus that supports normalization and activation functions for artificial intelligence acceleration, and more specifically, to a specific method and apparatus capable of further accelerating artificial intelligence computation in an artificial intelligence computation process. Background Technology
[0002] Artificial intelligence requires not only major operations that account for the largest amount of computation, such as convolution and matrix multiplication, but also post-processing such as normalization and non-linear activation functions. As AI models have evolved, the forms of these normalization and non-linear activation functions have also changed significantly from their early days. For example, regarding normalization, batch normalization was primarily used in the early days, but recently, various normalization methods such as layer normalization and group normalization are mainly used instead of batch normalization. As for non-linear activation functions, unlike early AI models that mainly used ReLU and PReLU, recent AI models primarily use activation functions such as SiLU, GELU, and Sigmoid.
[0003] Therefore, the present invention intends to provide an empirical discussion regarding the normalization and activation function processing in artificial intelligence computation to further secure artificial intelligence acceleration and efficiency.
[0004] The aforementioned background technology is technical information that the inventor possessed for the derivation of the present invention or acquired during the process of deriving the present invention, and it cannot be considered technology that was known to the general public prior to the filing of the present invention. The problem to be solved
[0005] The problem to be solved through the disclosure of the present invention is to provide a specific method and apparatus capable of further accelerating artificial intelligence computation in the artificial intelligence computation process. means of solving the problem
[0006] The problem to be solved by the disclosure of the present invention is a method performed by an artificial intelligence acceleration device comprising: a step of loading data from at least one user terminal; a step of preprocessing the loaded data; a step of performing a first operation process for an artificial intelligence model on the preprocessed data; a step of performing a second operation process for an artificial intelligence model on the preprocessed data; a step of postprocessing the output data after the first operation and the second operation process; and a step of calculating the operation efficiency with respect to the first operation, the second operation, and the postprocessing process, wherein the first operation is a normalization operation and the second operation may correspond to a non-linear activation function operation.
[0007] In one embodiment, the post-processing process may be a process that utilizes a Tensor Streaming Core to receive and process the main computation results immediately without storing them in output memory.
[0008] In one embodiment, the first operation may be characterized by the delayed statistics-based normalization method remembering the most recently calculated mean and standard deviation values, reusing those values for linear transformation, and performing parallel operations on new statistics in the meantime.
[0009] In one embodiment, the second operation may be a conditional second-order polynomial-based approximation method.
[0010] In one embodiment, the second operation may be a method of approximating a function of the form of a sigmoid.
[0011] In one embodiment, the second operation is to apply a second polynomial according to the magnitude of the parameter value of the second polynomial in a conditional second polynomial-based approximation method, and the operation may be implemented using only a multiplier and an adder. Effects of the invention
[0012] According to the means for solving the problem of the present invention described above, a specific method and apparatus capable of further accelerating artificial intelligence computation can be provided for the artificial intelligence computation process. Brief explanation of the drawing
[0013] FIG. 1 illustrates an exemplary environment in which an artificial intelligence acceleration device according to some embodiments of the present disclosure may be applied. FIG. 2 is a flowchart relating to normalization and activation operations for accelerating a deep learning algorithm that can be performed in an artificial intelligence accelerator according to some embodiments of the present disclosure. FIGS. 3 through 8 are drawings relating to exemplary architectures and performance of an artificial intelligence acceleration device according to some embodiments of the present disclosure. FIG. 9 is a drawing of an exemplary computing device capable of implementing a device and / or system according to various embodiments of the present disclosure. Specific details for implementing the invention
[0014] Hereinafter, preferred embodiments of the present disclosure will be described in detail with reference to the accompanying drawings. The advantages and features of the present disclosure and the methods for achieving them will become clear by referring to the embodiments described below in detail together with the accompanying drawings. However, the technical concept of the present disclosure is not limited to the following embodiments but can be implemented in various different forms. The following embodiments are provided merely to complete the technical concept of the present disclosure and to fully inform those skilled in the art of the scope of the present disclosure, and the technical concept of the present disclosure is defined only by the scope of the claims.
[0015] It should be noted that when assigning reference numerals to the components of each drawing, the same components are given the same reference numeral whenever possible, even if they are shown in different drawings. Furthermore, in describing the present disclosure, if it is determined that a detailed description of related known components or functions could obscure the essence of the present disclosure, such detailed description is omitted.
[0016] Unless otherwise defined, all terms used herein (including technical and scientific terms) may be used in a meaning commonly understood by those skilled in the art to which this disclosure pertains. Additionally, terms defined in commonly used dictionaries are not to be interpreted ideally or excessively unless explicitly and specifically defined otherwise. The terms used herein are for describing the embodiments and are not intended to limit this disclosure. In this specification, the singular form includes the plural form unless specifically stated otherwise in the text.
[0017] Additionally, terms such as first, second, A, B, (a), (b), etc., may be used to describe the components of the present disclosure. These terms are intended only to distinguish the components from other components and do not limit the nature, order, or sequence of the components. Where it is stated that a component is "connected," "coupled," or "joined" to another component, it should be understood that the component may be directly connected or joined to the other component, but that another component may also be "connected," "coupled," or "joined" between each component.
[0018] As used in the specification, "comprises" and / or "comprising" do not exclude the presence or addition of one or more other components, steps, actions, and / or elements to the mentioned components, steps, actions, and / or elements.
[0019] Hereinafter, various embodiments of the present disclosure will be described in detail with reference to the attached drawings.
[0020] In addition, terms such as first, second, A, B, (a), (b), etc. may be used when describing the components of the present invention. These terms are intended merely to distinguish the components from other components, and the essence, order, or sequence of the components is not limited by these terms. Throughout the specification, when a part is described as 'comprising' or 'equipped' with a certain component, unless specifically stated otherwise, this means that it may include additional components rather than excluding other components. Furthermore, terms such as 'part' or 'module' described in the specification refer to a unit that processes at least one function or operation, and this may be implemented in hardware, software, or a combination of hardware and software.
[0022] FIG. 1 illustrates an exemplary environment in which an artificial intelligence acceleration device according to some embodiments of the present disclosure may be applied. Through a system including a user terminal (100) and an artificial intelligence acceleration device (200) as illustrated in FIG. 1, operations supporting normalization and non-linear activation functions for artificial intelligence acceleration can be performed.
[0023] Below, we will explain in more detail the components shown in FIG. 1 related to the artificial intelligence model acceleration operation by utilizing the user terminal (100) and the artificial intelligence acceleration device (200) through the system described above.
[0024] FIG. 1 illustrates an example in which a user terminal (100) and an artificial intelligence accelerator (200) are connected via a network, but this is merely for convenience of understanding, and the number of devices that can be connected to the network can vary.
[0025] Meanwhile, FIG. 1 merely illustrates a preferred embodiment for achieving the purpose of the present disclosure, and some components may be added or deleted as needed. Below, the components illustrated in FIG. 1 will be described in more detail.
[0026] The artificial intelligence accelerator (200) can collect and analyze various information generated from the user terminal (100). The various information may include all data generated from the user terminal (100).
[0027] The user terminal (100) illustrated in FIG. 1 may be implemented with one or more computing devices. For example, all functions of the user terminal (100) may be implemented in a single computing device. As another example, the first function of the user terminal (100) may be implemented in a first computing device, and the second function may be implemented in a second computing device. Here, the computing device may be a notebook, a desktop, a laptop, etc., but is not limited thereto and may include all types of devices equipped with computing functions. The user terminal (100) may be connected to the artificial intelligence acceleration device (200) of the present invention via a network to transmit and receive data to and from each other.
[0028] Meanwhile, the artificial intelligence acceleration device (200) may also be implemented as one or more computing devices. For example, all functions of the artificial intelligence acceleration device (200) may be implemented in a single computing device. As another example, the first function of the artificial intelligence acceleration device (200) may be implemented in the first computing device, and the second function may be implemented in the second computing device. Here, the computing device may be a notebook, desktop, laptop, etc., but is not limited thereto and may include all types of devices equipped with computing functions. However, it may be preferable for the artificial intelligence acceleration device (200) to be implemented as a high-performance server-class computing device. An example of a computing device will be described with reference to FIG. 9.
[0029] Additionally, the functions that can be implemented in the artificial intelligence acceleration device (200) may also be implemented by utilizing an electronic device installed in the user terminal (100). Therefore, although the artificial intelligence acceleration device (200) and the user terminal (100) are depicted separately in FIG. 1, it is obvious that according to one embodiment, the artificial intelligence acceleration device (200) is installed in the user terminal (100) and the corresponding device within the user terminal (100) can implement the first function, the second function, etc. Accordingly, it should be noted that this is not limited to an embodiment in which the user terminal (100) and the artificial intelligence acceleration device (200) are externally separated as shown in FIG. 1.
[0030] For convenience of explanation, the present specification describes a situation in which the user terminal (100) and the artificial intelligence acceleration device (200) are separated to implement functions.
[0031] In some embodiments, components included in an environment to which the artificial intelligence accelerator (200) is applied can communicate through a network. The network can be implemented as any type of wired or wireless network, such as a Local Area Network (LAN), a Wide Area Network (WAN), a mobile radio communication network, or Wibro (Wireless Broadband Internet).
[0032] Meanwhile, although the environment illustrated in FIG. 1 is shown as being connected via a network through a user terminal (100) and an artificial intelligence acceleration device (200), it should be noted that the scope of the present disclosure is not limited thereto and that the user terminal (100) may be connected to the artificial intelligence acceleration device (200) via P2P (Peer to Peer).
[0033] Up to now, with reference to FIG. 1, exemplary environments in which a corresponding device (200) according to some embodiments of the present disclosure may be applied have been described. Hereinafter, with reference to FIG. 2 and subsequent drawings, methods according to various embodiments of the present disclosure will be described in detail.
[0034] Each step of the methods described below may be performed by a computing device. In other words, each step of the methods may be implemented by one or more instructions executed by a processor of the computing device. All steps included in these methods may be performed by a single physical computing device, but the first steps of the methods may be performed by a first computing device and the second steps of the methods may be performed by a second computing device.
[0035] In FIG. 2 below, the explanation will continue assuming that each step of the methods is performed by the artificial intelligence acceleration device (200) exemplified in FIG. 1. However, for the convenience of explanation, the subject of the operation of each step included in the methods may be omitted.
[0037] FIG. 2 is a flowchart relating to normalization and activation operations for accelerating a deep learning algorithm that can be performed in an artificial intelligence accelerator according to some embodiments of the present disclosure.
[0038] In step S100, the artificial intelligence acceleration device (200) may load data from at least one user terminal. The loaded data may be data processed by an artificial intelligence model for performing calculations in an artificial intelligence model included in the artificial intelligence acceleration device (200).
[0039] In step S200, the artificial intelligence accelerator (200) can preprocess the loaded data. The preprocessing process may refer to a process of generating data directly for computation in order to perform artificial intelligence computation in the artificial intelligence model included in the artificial intelligence accelerator (200).
[0040] In step S300, the artificial intelligence accelerator (200) can perform a first operation process on the artificial intelligence model on the preprocessed data. In step S400, the artificial intelligence accelerator (200) can perform a second operation process on the artificial intelligence model on the preprocessed data. In step S500, the artificial intelligence accelerator (200) can perform post-processing on the output data after the first and second operation processes.
[0041] The above first operation process may refer to a normalization operation, which is a core element in the operation of an artificial intelligence model, and the above second operation process may refer to an operation process utilizing a non-linear activation function.
[0042] Since normalization and non-linear activation functions do not constitute the majority of AI operations like convolution and matrix multiplication, existing hardware has only proposed methods to accelerate these operations at high speed and high efficiency. However, as normalization and non-linear activation functions are essential elements that must be supported, it is crucial to implement them in hardware with minimal footprint and low power consumption. Furthermore, unlike batch normalization, which can be merged with convolution or matrix multiplication, layer normalization or group normalization cannot skip the process of calculating the mean and standard deviation. Additionally, compared to the ReLU non-linear activation function, which only requires data masking through simple comparison with zero, Sigmoid, SiLU, and GELU require complex and large computational units similar to exponential functions. Therefore, methods to efficiently accelerate these operations and hardware capable of supporting them are absolutely necessary.
[0043] To achieve the above-mentioned objective, the present invention proposes a delayed statistics-based normalization operation method to efficiently support the latest normalization method and a conditional second-order polynomial-based approximation method to efficiently support the latest non-linear activation function operation, and implements them together in a Tensor Streaming Core architecture, which is a post-processing core architecture capable of minimizing internal / external memory access. Finally, the operator performing the tensor post-processing is designed to integrately support both the normalization operation and the non-linear activation function operation, and may include an integrated Tensor Streaming Core architecture.
[0044] In step S600, the artificial intelligence accelerator (200) can calculate computational efficiency regarding the first, second, and post-processing processes. The above process may be a process of calculating efficiency regarding the computation process after the artificial intelligence model included in the artificial intelligence accelerator (200) has performed computation, and verifying the efficiency to continuously check whether the computation performed by the current artificial intelligence model has been performed efficiently.
[0045] Below, the computational efficiency process performed by the artificial intelligence acceleration device (200) will be described in more detail using FIGS. 3 to 8.
[0047] FIGS. 3 through 8 are drawings relating to exemplary architectures and performance of an artificial intelligence acceleration device according to some embodiments of the present disclosure.
[0048] Convolution and matrix multiplication are widely known as the major operations in artificial intelligence, and while the importance of parallel processing hardware such as GPUs and NPUs to accelerate these operations at high speed and efficiency is becoming increasingly prominent, the implementation of AI must undergo post-processing at every layer in addition to convolution and matrix multiplication. Among these post-processing steps, the two most commonly used are normalization and non-linear activation functions. However, as AI models have evolved, the forms of normalization and non-linear activation functions have changed significantly from their early stages.
[0049] For example, regarding normalization, as shown in Figure 3, Batch Normalization was primarily used in the early stages. Batch Normalization is a method that normalizes data by utilizing the mean and standard deviation of multiple batches, based on the distribution characteristics of the output values of Convolution or Matrix Multiplication layers during the training of an AI model. During training, Batch Normalization calculates the mean and standard deviation for each output value and modifies the values by learning and applying hyperparameters called Gamma and Beta. However, the overhead of applying this Batch Normalization during inference is not significant because the mean and standard deviation values used during training are remembered and reused during inference without recalculating these statistics. In this case, the remembered mean and standard deviation are integrated with the preceding Convolution or Matrix Multiplication layers and the operations to modify values using Gamma and Beta, allowing the Batch Normalization effect to be applied without an increase in additional computational load.
[0050] However, recently, various normalization methods such as Layer Normalization and Group Normalization are primarily used instead of Batch Normalization. The biggest difference between these newly emerging normalization methods is that statistical values such as the mean and standard deviation cannot be reused and must be recalculated every time a new input is received. Therefore, to support these recent normalization methods, new hardware blocks capable of efficiently calculating and applying the mean and standard deviation are essential.
[0051] Non-linear activation functions, in addition to normalization, are post-processing operations that must be included to implement artificial intelligence. As shown in Figure 2, unlike early AI models that primarily used ReLU and PReLU, recent AI models mainly use activation functions that are difficult to implement in hardware, such as SiLU, GELU, and Sigmoid. The ReLU-family non-linear activation functions mainly used in early models did not apply any post-processing if the input tensor was greater than '0', and performed simple operations such as masking values smaller than '0' to '0' or multiplying by a simple scale factor. However, Sigmoid and its derivatives SiLU and GELU all require more complex operations, such as exponential operations and division.
[0052] In order to accelerate the normalization and non-linear activation function operations, which are widely used recently, at high speed and high efficiency, the present invention proposes a method for approximating each of the normalization and non-linear activation functions, and a hardware core architecture capable of accelerating this on the same core architecture.
[0053] Figures 3 through 8 illustrate a delayed statistics-based normalization method proposed by the present invention to efficiently accelerate normalization operations. In conventional methods, normalization operations are implemented by first performing major operations such as convolution or matrix multiplication, calculating the mean and variance of the operation results, and then performing a linear transformation (Scaling & Biasing) on the mean and variance values obtained. Since the Scale and Bias values for the linear transformation can only be finally determined after the statistical calculations are completed, this conventional process makes parallel processing difficult and limits operations to sequential execution. The proposed delayed statistics-based normalization method remembers the most recently calculated mean and standard deviation values, reuses them for the linear transformation, and performs parallel processing of new statistics in the meantime. In other words, the mean and standard deviation values calculated from the current input are used when performing the linear transformation in the next inference. For example, in the case of a CNN (Convolutional Neural Network) or a Transformer-based non-diffusion model, the mean and standard deviation values calculated in the user's most recent inference operation are remembered and used as is in the next inference. Of course, in the next inference, the statistics for the subsequent inference are calculated together. In the case of a diffusion model that completes inference through multiple iterations, as can be seen in the right figure of Figure 3, the mean and standard deviation values calculated in the previous iteration are reused in the next iteration. This lazy statistics-based normalization method can increase the speed of computation acceleration because statistical calculations and linear transformation operations can be processed in parallel without the need for operations to be performed sequentially as in the conventional normalization process.
[0054] Figures 3 through 8 illustrate a conditional second-order polynomial-based approximation method proposed by the present invention to efficiently accelerate non-linear activation function operations. First, recently developed non-linear activation functions such as Sigmoid, SiLU, and GELU all start with a Sigmoid-type function operation and involve multiplying the original input value or multiplying it after a linear transformation. In other words, the part with high hardware implementation overhead compared to existing ReLU-based non-linear activation functions is the process of computing the Sigmoid-type function, which requires exponential functions and division; the method proposed by the present invention proposes a method for approximating such a Sigmoid-type function. First, the Sigmoid-type function [is used] with an Inlier Threshold (Th i ) and Outlier Threshold (Th o It is distinguished based on ). Inputs with an absolute value greater than Tho are treated as constants, and those with an absolute value smaller than Tho, but Th i For inputs larger than , a second-order polynomial is applied using hyperparameters a, b, and c to the concentrated approximation of the outlier tensor, and for other input absolute values, Th i For smaller inputs, a second-order polynomial with hyperparameters a, b, and c is applied by concentrating the inlier tensor. Since a second-order polynomial is applied based on the magnitude of the value, operations can be implemented using only multipliers and adders, unlike hardware that had to support exponential functions and division.
[0055] As such, the conditional second-order polynomial-based nonlinear activation function approximation method, which applies different second-order polynomials depending on the magnitude of the value, requires a, b, c, and Th to minimize accuracy loss i , Th o must be determined carefully. The process is summarized in Fig. 5. First, depending on where the sigmoid-type function converges, the inlier b (bi ) and outlier b (b o The user determines ) and sets the Outlier Threshold to be assumed as a constant. Subsequently, the range of values for a and c to be found is determined through Grid Search, and based on this, the a for which the two quadratic polynomials can have an intersection is i , a o , c i , c o Find the input coordinates where the contact point occurs. The Inlier Threshold (Th i Decide as ). a found in this way i , a o , c i , c o Based on this, the Error value is calculated to determine how much it differs from the actual output of the original function. In this way, several a i , a o , c i , c o After testing, the a with the lowest error i , a o , c i , c o Adopt as the final hyperparameter.
[0056] So far, we have explained methods to effectively approximate normalization and non-linear activation function operations, which are essential post-processing steps in artificial intelligence algorithms.
[0057] Continuing to look at Figures 3 through 8, the present invention adopts a Tensor Streaming Core architecture to effectively perform normalization and non-linear activation function operations. Generally, tensor post-processing involves storing the results of major operations, such as convolution or matrix multiplication, in output memory and then reading those results back to perform the operations for post-processing. In this process, memory write / read operations are necessarily performed once each; however, the Tensor Streaming Core receives the results of major operations immediately without storing them in output memory and performs post-processing. Since the post-processing results of the major operations are stored directly in output memory, memory write / read operations are reduced, thereby minimizing unnecessary energy consumption caused by memory access.
[0058] This is a structural diagram of the final hardware core architecture proposed in the present invention, named the Unified Tensor Streaming Core (UTSC). Since the post-processing core follows the Tensor Streaming Core architecture described above, it directly receives the results of major operations such as convolution or matrix multiplication and performs normalization and non-linear activation function post-processing operations. At this time, normalization is accelerated using a delayed statistics-based normalization approximation method, and non-linear activation function operations are accelerated using a conditional second-order polynomial-based approximation method.
[0059] To implement the delayed statistics-based normalization approximation method, UTSC includes ΣX, which is the accumulated current input value, and ΣX, which is the accumulated square of the input value. 2 Statistic Register storing, ΣX and ΣX of the previous input value 2The core blocks are a Delayed Statistic buffer that stores the mean and variance of previous input values calculated based on [the data], and a Scaling Pre-Calculator that pre-calculates parameter values (Gamma, Beta) for a new linear transformation by considering the weights Gamma and Beta learned during the normalization process along with the mean and variance. However, if the statistics used in the previous inference results are utilized as is, the loss of accuracy may be greater as the difference in distribution between the current input and the previous input increases. To overcome this problem, the proposed hardware includes a Normalization Recovery Circuit, as shown in Fig. 8. Conventional delayed statistics-based normalization could minimize the delay time required for tensor post-processing by simultaneously performing a linear transformation using delayed statistics and statistical calculations for the current input. However, if the mean and standard deviation values used for the linear transformation differ significantly from the actual newly calculated mean and standard deviation values, the Normalization Recovery Circuit detects the error, ignores the tensor with the completed linear transformation, and performs a new linear transformation using the newly calculated mean and standard deviation values for the current input.
[0060] To implement non-linear activation function operations using conditional second-order polynomial-based approximation, the UTSC is composed of a Non-linear Activation Unit (NLAU) register containing hyperparameters of inlier and outlier second-order polynomials, an NLAU controller to distinguish whether a tensor is inlier or outlier and to support simple activation functions such as ReLU, and a 2-step NLAU controller to support activation functions that produce a final output value by multiplying the original input value after calculating a sigmoid-type function such as SiLU or GELU. Here, the present invention selects different hyperparameters of the second-order polynomial depending on the standard deviation of the input distribution to further improve the accuracy of the second-order polynomial-based approximation. Generally, since non-linear activation functions appear after the normalization layer, the additional computational overhead for calculating the standard deviation is not significant, and it is acceptable to calculate it using lag statistics. Two different hyperparameters selected based on the standard deviation select the hyperparameter optimized for when the distribution is narrow or wide, respectively. When the standard deviation is smaller than the threshold value, a second-order polynomial with a smaller error in a narrow distribution is selected, and when the standard deviation is larger than the threshold value, a second-order polynomial with a smaller error in a wide distribution is selected.
[0061] Since both normalization and non-linear activation functions can be implemented with multipliers and adders and the two post-processing operations never proceed simultaneously, UTSC is accelerated by sharing the same Processing Element (PE).
[0062] To explain the method of sharing arithmetic units to accelerate normalization and non-linear activation function operations, the area required for UTSC implementation is minimized by sharing arithmetic units across both post-processing steps. When supporting non-linear activation functions, one adder remains, which is used when supporting residual paths.
[0063] Below, an exemplary computing device in which an artificial intelligence acceleration device (200) can be implemented will be described in detail through FIG. 9.
[0065] FIG. 9 is a drawing of an exemplary computing device capable of implementing a device and / or system according to various embodiments of the present disclosure.
[0066] A computing device (1500) may include one or more processors (1510), a bus (1550), a communication interface (1570), a memory (1530) for loading a computer program (1591) executed by the processor (1510), and a storage (1590) for storing the computer program (1591). However, only components related to the embodiments of the present disclosure are illustrated in FIG. 9. Therefore, a person skilled in the art to which the present disclosure pertains will understand that other general-purpose components may be included in addition to the components illustrated in FIG. 9.
[0067] The processor (1510) controls the overall operation of each component of the computing device (1500). The processor (1510) may be configured to include a CPU (Central Processing Unit), an MPU (Micro Processor Unit), an MCU (Micro Controller Unit), a GPU (Graphic Processing Unit), or any form of processor well known in the art of the present disclosure. Additionally, the processor (1510) may perform operations for at least one application or program for executing the method according to the embodiments of the present disclosure. The computing device (1500) may have one or more processors.
[0068] The memory (1530) stores various data, commands and / or information. The memory (1530) may load one or more programs (1591) from storage (1590) to execute a method according to embodiments of the present disclosure. The memory (1530) may be implemented as volatile memory such as RAM, but the technical scope of the present disclosure is not limited thereto.
[0069] The bus (1550) provides communication functions between components of the computing device (1500). The bus (1550) can be implemented as various types of buses, such as an address bus, a data bus, and a control bus.
[0070] The communication interface (1570) supports wired and wireless internet communication of the computing device (1500). Additionally, the communication interface (1570) may support various communication methods other than internet communication. To this end, the communication interface (1570) may be configured to include a communication module well known in the art of the present disclosure.
[0071] According to some embodiments, the communication interface (1570) may be omitted.
[0072] Storage (1590) can store one or more of the above programs (1591) and various data non-temporarily.
[0073] Storage (1590) may be configured to include non-volatile memory such as ROM (Read Only Memory), EPROM (Erasable Programmable ROM), EEPROM (Electrically Erasable Programmable ROM), flash memory, a hard disk, a removable disk, or any form of computer-readable recording medium well known in the art to which this disclosure belongs.
[0074] A computer program (1591) may include one or more instructions that cause a processor (1510) to perform a method / operation according to various embodiments of the present disclosure when loaded into memory (1530). That is, the processor (1510) may perform a method / operation according to various embodiments of the present disclosure by executing the one or more instructions.
[0075] Various embodiments of the present disclosure and effects according to those embodiments have been described with reference to FIGS. 1 through 9. The effects according to the technical concept of the present disclosure are not limited to those described above, and other unmentioned effects will be clearly understood by a person skilled in the art from the description in the specification.
[0076] The technical concept of the present disclosure described so far with reference to FIGS. 1 through 9 may be implemented as computer-readable code on a computer-readable medium. The computer-readable recording medium may be, for example, a removable recording medium (CD, DVD, Blu-ray disc, USB storage device, removable hard disk) or a fixed recording medium (ROM, RAM, computer-equipped hard disk). The computer program recorded on the computer-readable recording medium may be transmitted to another computing device via a network such as the Internet and installed on the other computing device, thereby being used on the other computing device.
[0077] In the foregoing, although all components constituting the embodiments of the present disclosure have been described as being combined or operating together, the technical concept of the present disclosure is not necessarily limited to such embodiments. That is, within the scope of the purpose of the present disclosure, all components may be selectively combined and operated in one or more ways.
[0078] Although operations are depicted in a specific order in the drawings, it should not be understood that the operations must be executed in the specific order depicted or in a sequential order, or that all depicted operations must be executed to obtain the desired result. In certain situations, multitasking and parallel processing may be advantageous. Furthermore, the separation of the various configurations in the embodiments described above should not be understood as a necessary separation, and it should be understood that the described program components and systems can generally be integrated together into a single software product or packaged into multiple software products.
[0079] Although embodiments of the present disclosure have been described above with reference to the attached drawings, those skilled in the art will understand that the present disclosure may be practiced in other specific forms without changing the technical concept or essential features thereof. Therefore, the embodiments described above should be understood as illustrative in all respects and not restrictive. The scope of protection of the present disclosure shall be interpreted by the claims below, and all technical concepts within the equivalent scope shall be interpreted as being included within the scope of rights of the technical concept defined by the present disclosure.
Claims
Claim 1 A method performed by an artificial intelligence acceleration device comprises: loading data from at least one user terminal; preprocessing the loaded data; performing a first operation process for an artificial intelligence model on the preprocessed data; performing a second operation process for an artificial intelligence model on the preprocessed data; postprocessing the output data after the first operation and the second operation process; and calculating the computational efficiency with respect to the first operation, the second operation, and the postprocessing process. It includes, wherein the first operation is a normalization operation, the second operation corresponds to a non-linear activation function operation, the second operation is a conditional second-order polynomial-based approximation method, the second operation is a method for approximating a sigmoid-type function, and in the conditional second-order polynomial-based approximation method, a second-order polynomial is applied according to the magnitude of the parameter values of the second-order polynomial, and the operation can be implemented using only multipliers and adders, and the artificial intelligence model is composed of a Unified Tensor Streaming Core (UTSC), and the Unified Tensor Streaming Core is composed of a Non-linear Activation Unit register containing hyperparameters of inlier and outlier second-order polynomials, an NLAU controller for distinguishing inlier / outlier status and supporting simple activation functions such as ReLU, and a 2-step NLAU controller for supporting an activation function that calculates a sigmoid-type function and multiplies it by the original input value to create a final output value, and also, during the second operation, input An integrated streaming control method for artificial intelligence acceleration that supports normalization and activation functions, characterized by selecting different hyperparameters of a conditional second-order polynomial depending on the standard deviation of the distribution. Claim 2 In claim 1, the post-processing process is a process of receiving and processing key computation results directly without storing them in output memory using a Tensor Streaming Core, an integrated streaming control method supporting normalization and activation functions for artificial intelligence acceleration. Claim 3 In claim 1, the first operation is characterized by the delayed statistics-based normalization method remembering the most recently calculated mean and standard deviation values, reusing those values for linear transformation, and performing parallel computation on new statistics in the meantime, in an integrated streaming control method supporting normalization and activation functions for artificial intelligence acceleration. Claim 4 delete Claim 5 delete Claim 6 delete Claim 7 A processor; a network interface; a memory; and a computer program loaded into the memory and executed by the processor, wherein the processor comprises: an instruction for loading data from at least one user terminal; an instruction for preprocessing the loaded data; an instruction for performing a first operation process for an artificial intelligence model on the preprocessed data; an instruction for performing a second operation process for an artificial intelligence model on the preprocessed data; an instruction for postprocessing the output data after the first operation and the second operation process; and an instruction for calculating the computational efficiency with respect to the first operation, the second operation, and the postprocessing process.It is performed including, wherein the first operation is a normalization operation, the second operation corresponds to a non-linear activation function operation, the second operation is a conditional second-order polynomial-based approximation method, the second operation is a method for approximating a sigmoid-type function, and in the conditional second-order polynomial-based approximation method, a second-order polynomial is applied according to the magnitude of the parameter values of the second-order polynomial, and the operation can be implemented using only multipliers and adders, and the artificial intelligence model is composed of a Unified Tensor Streaming Core (UTSC), and the Unified Tensor Streaming Core is composed of a Non-linear Activation Unit register containing hyperparameters of inlier and outlier second-order polynomials, an NLAU controller for distinguishing inlier / outlier status and supporting simple activation functions such as ReLU, and a 2-step NLAU controller for supporting an activation function that calculates a sigmoid-type function and multiplies it by the original input value to create a final output value, and also, during the second operation, An AI acceleration device that supports normalization and activation functions for AI acceleration and controls integrated streaming, characterized by selecting different hyperparameters of a conditional second-order polynomial based on the standard deviation of the input distribution. Claim 8 In claim 7, the above post-processing process is a process that utilizes a Tensor Streaming Core to receive and process the main computation results immediately without storing them in output memory, and is an artificial intelligence acceleration device that supports normalization and activation functions for artificial intelligence acceleration and integrates streaming control. Claim 9 An AI acceleration device that supports normalization and activation functions for AI acceleration and integrates streaming control, wherein the first operation is characterized by the delayed statistics-based normalization method remembering the most recently calculated mean and standard deviation values and reusing them for linear transformation, while performing parallel computation on new statistics. Claim 10 delete Claim 11 delete Claim 12 delete
Citation Information
Patent Citations
Method and apparatus for dynamic normalization and relay in a neural network
US20220207359A1
Memory-efficient streaming convolutions in neural network processor
US20230368008A1
