Apparatus and method for processing quantization artificial intelligence learning using accuracy information and similarity information

Abstract

The present invention relates to an apparatus and method for processing quantization artificial intelligence learning using accuracy information and similarity information, and the apparatus for processing quantization artificial intelligence learning, according to an embodiment of the present invention, may comprise: a learning processing unit that outputs learning data by learning input data by using an artificial intelligence model; a quantization processing unit that outputs quantized data by quantizing the learning data; and a relearning processing unit that relearns the quantized data by considering a ratio between accuracy information and distribution information, which are based on a loss function, with respect to the learning data and the quantized data.

Description

Quantization artificial intelligence learning processing device and method using accuracy information and similarity information

[0001] The present invention relates to a quantized artificial intelligence learning processing device and method using accuracy information and similarity information, and more specifically, to a technology for increasing saliency of quantized artificial intelligence learning results by adjusting the ratio between accuracy and similarity by learning accuracy information on loss and similarity information on distribution between learning data and quantized data together when re-learning artificial intelligence learning data after quantizing it.

[0002] This invention is the result of research conducted with the support of the National Research Foundation of Korea (No. RS-2023-00213548) and the National IT Industry Promotion Agency (NIPA) (No. RS-2022-00155966, Artificial Intelligence Convergence Innovation Talent Development (Ewha Womans University), National Project Number: 1711179344, No. 2021-0-02068, Artificial Intelligence Innovation Hub Research and Development) funded by the government (Ministry of Science and ICT) in 2024.

[0003] Artificial Intelligence (AI) technologies, with their unique ability to learn from data and make predictions or decisions without explicit programming, offer a promising solution to the problem of increasing data complexity and volume.

[0004] In the field of data analysis, LSTM (Long Short-Term Memory) is a type of artificial intelligence learning model, a type of recurrent neural network (RNN) technique, and is attracting attention because it is adept at processing time-series data.

[0005] Time series data has strong temporal correlation and can contain long-term patterns or trends, and indoor environment data is a representative example.

[0006] For example, indoor environmental data is data related to energy consumed within a building, and can be composed by considering various variable data such as temperature, humidity, occupancy level, and energy usage.

[0007] Additionally, indoor environmental data is related to the control of HVAC (Heating, Ventilation and Air Conditioning).

[0008] Unlike existing methods that often struggle with temporal dependencies, LSTMs are designed to learn effectively from data sequences by retaining information in memory over long periods of time.

[0009] However, LSTMs have several drawbacks that limit their practicality in real-world applications: their computational complexity and resource-intensive nature limit their computational power and memory.

[0010] To alleviate these problems, quantization is considered and used, but model performance often deteriorates due to the introduction of approximation errors.

[0011] In other words, conventional quantization involves lowering the precision of the numbers used in the model, which leads to information loss and ultimately lowers accuracy.

[0012] Finding a trade-off between model efficiency and accuracy for artificial intelligence learning processing models can be seen as an important task when utilizing LSTM networks in energy data analysis.

[0013] The present invention aims to increase saliency of quantized artificial intelligence learning results by adjusting the ratio between accuracy and similarity by learning accuracy information and distribution similarity information for loss between learning data and quantized data together when relearning artificial intelligence learning data after quantization.

[0014] The purpose of the present invention is to identify a trade-off between the efficiency and accuracy of a model in artificial intelligence learning processing, effectively identify temporal dependence and contextual relevance, and derive artificial intelligence learning results.

[0015] The present invention aims to secure efficiency and accuracy of artificial intelligence learning results by minimizing the mismatch between the two distributions by effectively maintaining a close approximation using divergence between the distribution of artificial intelligence learning results learned based on LSTM and an attention mechanism and the distribution after quantization processing of the artificial intelligence learning results.

[0016] The present invention aims to provide a quantized artificial intelligence learning processing device and method that can be applied to various applications in various technical fields that require prediction based on time series data, while ensuring efficiency and accuracy of artificial intelligence learning results and processing complex multivariate prediction tasks without reducing speed or accuracy.

[0017] A quantization artificial intelligence learning processing device according to one embodiment of the present invention may include a learning processing unit that learns input data using an artificial intelligence model and outputs learning data, a quantization processing unit that quantizes the learning data and outputs quantized data, and a relearning processing unit that relearns the quantized data by considering a ratio between accuracy information and distribution information based on a loss function for the learning data and the quantized data.

[0018] The above relearning processing unit may calculate an absolute weight for the accuracy information and the distribution information based on the difference value of the skewness and kurtosis of the learning data and the quantized data, and may calculate a relative ratio for the accuracy information and the distribution information based on the calculated absolute weight.

[0019] The above relearning processing unit may calculate a parameter (x) related to similarity based on a ratio between a difference value between a skewness value of the learning data and a skewness value of the quantized data for a skewness value of the quantized data, calculate a parameter (m') for the absolute proportion based on the calculated parameter (x), and calculate a parameter (m) for the relative proportion based on the calculated parameter (m').

[0020] The above relearning processing unit may calculate a parameter (y) related to accuracy based on a ratio between a difference value between a skewness value of the learning data and a kurtosis value of the quantized data for the kurtosis value of the quantized data, calculate a parameter (n') for the absolute proportion based on the calculated parameter (y), and calculate a parameter (n) for the relative proportion based on the calculated parameter (n').

[0021] The above relearning processing unit may configure the loss function as a mean square error (MSE) loss operation and a relative entropy operation, and may relearn the quantized data according to the operation result of applying a parameter (n) related to a relative proportion of the accuracy information to the mean square error loss operation and applying a parameter (m) related to a relative proportion of the distribution information to the relative entropy operation.

[0022] The above relearning processing unit can reduce the distribution mismatch between the learning data and the quantized data by relearning the quantized data so that the sum of the parameter (m) and the parameter (n) approaches “1”.

[0023] The above relearning processing unit can relearn the quantized data by increasing the weight on similarity as the parameter (m) increases in the loss function and by increasing the weight on accuracy as the parameter (n) increases.

[0024] The above learning processing unit can use the artificial intelligence model as a combined model of LSTM (Long Short Term Memory), which is one of RNN (Recurrent Neural Network), and an attention mechanism.

[0025] According to one embodiment of the present invention, the method may include a step of learning input data using an artificial intelligence model in a learning processing unit and outputting learning data, a step of quantizing the learning data in a quantization processing unit and outputting quantized data, and a step of relearning the quantized data in a relearning processing unit by considering a ratio between accuracy information and distribution information based on a loss function for the learning data and the quantized data.

[0026] The step of retraining the quantized data may include the steps of calculating a parameter (x) related to similarity based on a ratio between a difference value between a skewness value of the quantized data and a skewness value of the quantized data based on a difference value between the skewness value of the quantized data and the skewness value of the learning data based on a difference value between the skewness value of the quantized data and the skewness value of the learning data, and the kurtosis value of the learning data and the quantized data, calculating a parameter (m') related to similarity based on the calculated parameter (x), calculating a parameter (m) related to absolute proportion based on the calculated parameter (x), and calculating a parameter (m) related to relative proportion based on the calculated parameter (m'), and calculating a parameter (y) related to accuracy based on a ratio between a difference value between a skewness value of the learning data and a kurtosis value of the quantized data and a kurtosis value of the quantized data, calculating a parameter (n') related to absolute proportion based on the calculated parameter (y), and calculating a parameter (n) related to relative proportion based on the calculated parameter (n').

[0027] The present invention can increase saliency of quantized artificial intelligence learning results by adjusting the ratio between accuracy and similarity by learning accuracy information and distribution similarity information for loss between learning data and quantized data together when relearning artificial intelligence learning data after quantization.

[0028] The present invention can effectively identify temporal dependence and contextual relevance by identifying a trade-off between model efficiency and accuracy in artificial intelligence learning processing, thereby deriving artificial intelligence learning results.

[0029] The present invention effectively maintains a close approximation by using divergence for the distribution of artificial intelligence learning results learned based on LSTM and attention mechanisms and the distribution after quantization processing of the artificial intelligence learning results, thereby minimizing the mismatch between the two distributions and ensuring efficiency and accuracy of the artificial intelligence learning results.

[0030] The present invention can provide a quantized artificial intelligence learning processing device and method that can be applied to various applications in various technical fields that require prediction based on time series data, by processing complex multivariate prediction tasks without reducing speed or accuracy while securing efficiency and accuracy for artificial intelligence learning results.

[0031] FIG. 1 is a drawing illustrating a quantization artificial intelligence learning processing device according to one embodiment of the present invention.

[0032] FIG. 2 is a drawing illustrating a learning result of a quantization artificial intelligence learning processing device according to an embodiment of the present invention.

[0033] FIG. 3 and FIG. 4 are diagrams illustrating the learning performance of a quantization artificial intelligence learning processing device according to one embodiment of the present invention.

[0034] FIG. 5 is a diagram illustrating a quantization artificial intelligence learning processing method according to an embodiment of the present invention.

[0035] Below, various embodiments of this document are described with reference to the attached drawings.

[0036] The examples and terms used herein are not intended to limit the technology described in this document to a particular embodiment, but should be understood to encompass various modifications, equivalents, and / or alternatives of the embodiments.

[0037] In the following description of various embodiments, if it is determined that a detailed description of a related known function or configuration may unnecessarily obscure the gist of the invention, the detailed description will be omitted.

[0038] The terms described below are defined based on their functions in various embodiments, and may vary depending on the intent or custom of the user or operator. Therefore, their definitions should be based on the contents of this specification.

[0039] In connection with the description of the drawings, similar reference numerals may be used for similar components.

[0040] A singular expression may include a plural expression unless the context clearly indicates otherwise.

[0041] In this document, expressions such as "A or B" or "at least one of A and / or B" may include all possible combinations of the items listed together.

[0042] Expressions such as "first," "second," "first," or "second," may modify the components without regard to order or importance, and are used only to distinguish one component from another, but do not limit the components.

[0043] When it is said that a component (e.g., a first component) is “(functionally or communicatively) connected” or “connected” to another component (e.g., a second component), the component may be directly connected to the other component, or may be connected via another component (e.g., a third component).

[0044] In this specification, “configured to” may be used interchangeably with “suitable for,” “capable of,” “modified to,” “made to,” “capable of,” or “designed to,” depending on the context, for example, in terms of hardware or software.

[0045] In some contexts, the expression "a device configured to" may mean that the device is "capable of" doing something in conjunction with other devices or components.

[0046] For example, the phrase "a processor configured (or set) to perform A, B, and C" may mean a dedicated processor (e.g., an embedded processor) for performing those operations, or a general-purpose processor (e.g., a CPU or application processor) that can perform those operations by executing one or more software programs stored in a memory device.

[0047] Also, the term 'or' means 'inclusive or' rather than 'exclusive or'.

[0048] That is, unless otherwise stated or clear from context, the expression 'x utilizes a or b' means any one of the natural inclusive permutations.

[0049] The terms '..bu', '..gi', etc. used below mean a unit that processes at least one function or operation, and this can be implemented by hardware, software, or a combination of hardware and software.

[0050] FIG. 1 is a drawing illustrating a quantization artificial intelligence learning processing device according to one embodiment of the present invention.

[0051] FIG. 1 illustrates components of a quantized artificial intelligence learning processing device according to an embodiment of the present invention, and the quantized artificial intelligence learning processing device according to an embodiment of the present invention implements a technology for increasing saliency for quantized artificial intelligence learning results by adjusting the ratio between accuracy and similarity by learning accuracy information and distribution information of a loss function together when relearning artificial intelligence learning data after quantizing it.

[0052] Referring to FIG. 1, a quantization artificial intelligence learning processing device (100) according to one embodiment of the present invention includes a learning processing unit (110), a quantization processing unit (120), and a re-learning processing unit (130).

[0053] For example, the learning processing unit (110) learns input data using an artificial intelligence model and outputs learning data.

[0054] For example, the learning processing unit (110) uses an artificial intelligence model as a combined model of LSTM (Long Short Term Memory), one of RNN (Recurrent Neural Network), and an attention mechanism.

[0055] The learning processing unit (110) learns input data using an artificial intelligence model that accurately learns complex expressions within long sequence data.

[0056] For example, input data may consist of time series data that is used for predictive learning of data that requires future prediction, such as energy usage prediction.

[0057] A quantization processing unit (120) according to one embodiment of the present invention quantizes learning data and outputs quantized data.

[0058] The quantization processing unit (120) quantizes learning data to output quantized data with lowered numerical value precision to reduce computational demands and model size.

[0059] Here, quantized data can promote distribution efficiency on resource-constrained devices.

[0060] However, quantized data has some problems in that accuracy decreases as subtle but important differences in the data are removed.

[0061] According to one embodiment of the present invention, the relearning processing unit (130) relearns quantized data by considering the ratio between accuracy information and distribution information based on a loss function for the learning data and the quantized data.

[0062] For example, the relearning processing unit (130) can calculate the absolute weight of accuracy information and distribution information based on the difference values ​​of skewness and kurtosis of learning data and quantized data, and can calculate the relative ratio of the accuracy information and the distribution information based on the calculated absolute weight.

[0063] Skewness can indicate asymmetry as a distribution's symmetry, and kurtosis indicates a flat distribution.

[0064] According to one embodiment of the present invention, the relearning processing unit (130) can calculate a similarity-related parameter (x) and an accuracy-related parameter (y) based on the following mathematical expression 1.

[0065] [Mathematical Formula 1]

[0066]

[0067] In mathematical expression 1, x can represent a similarity-related parameter, y can represent an accuracy-related parameter, α can represent the skewness of the learning data, α' can represent the skewness of the quantized data, and e can represent a natural constant.

[0068] Training data can be data before quantization, and quantized data can be data after quantization.

[0069] According to one embodiment of the present invention, the relearning processing unit (130) may calculate a parameter (x) related to similarity based on a ratio between a difference value between a skewness value of learning data and a skewness value of the quantized data for a skewness value of the quantized data, calculate a parameter (m') for an absolute proportion based on the calculated parameter (x), and calculate a parameter (m) for a relative proportion based on the calculated parameter (m').

[0070] For example, the relearning processing unit (130) can calculate a parameter (y) related to accuracy based on the ratio between the difference value between the skewness value of the learning data and the kurtosis value of the quantized data with respect to the kurtosis value of the quantized data, calculate a parameter (n') for absolute proportion based on the calculated parameter (y), and calculate a parameter (n) for relative proportion based on the calculated parameter (n').

[0071] According to one embodiment of the present invention, the relearning processing unit (130) can calculate the parameter (n') and the parameter (n) using the similarity-related parameter (x) and the accuracy-related parameter (y) based on the following mathematical expression 2.

[0072] [Equation 2]

[0073]

[0074] In mathematical expression 2, m' can represent a parameter for the absolute proportion for parameter (x), m can represent a parameter for the relative proportion based on parameter (m'), n' can represent a parameter for the absolute proportion for parameter (y), and n can represent a parameter for the relative proportion based on parameter (n').

[0075] According to one embodiment of the present invention, the relearning processing unit (130) configures a loss function as a mean square error (MSE) loss operation and a relative entropy operation, applies a parameter (n) related to a relative ratio for the accuracy information to the mean square error loss operation, and applies a parameter (m) related to a relative ratio for the distribution information to the relative entropy operation, and can relearn quantized data according to the operation result.

[0076] For example, the relearning processing unit (130) can relearn quantized data using the following mathematical expression 3.

[0077] [Equation 3]

[0078]

[0079] Mathematical expression 3 can represent the operation of λ, which is a hyperparameter that determines the compromise between MSE and KL divergence operations by applying the parameter (m) to the KL divergence operation, which is the relative entropy operation, and the operation result of N corresponding to the total number of samples and the parameter (n) based on Mathematical expression 2, which is the quantized output quantized data Q(xi) and the existing output learning data yi in relation to the part that calculates the mean square error (MSE) loss.

[0080] Additionally, in the KL divergence operation, p(x) can represent the distribution of the corresponding training data before quantization, and q(x) can represent the distribution of the corresponding quantized data after quantization.

[0081] A relearning processing unit (130) according to one embodiment of the present invention can reduce distribution mismatch between learning data and quantized data by relearning quantized data so that the sum of parameters (m) and parameters (n) approaches “1”.

[0082] For example, the relearning processing unit (130) can relearn quantized data by increasing the weight on similarity as the parameter (m) in the loss function increases and by increasing the weight on accuracy as the parameter (n) increases.

[0083] In other words, the relearning processing unit (130) derives the relearning result, which is the result of the operation, so that when the parameter (n) in Equation 3 increases, more weight is placed on accuracy, and when the parameter (m) increases, more weight is placed on similarity, so that the relearning result, which is the result of the operation, is derived. The parameter (m) and the parameter (n) are used as values ​​for adjusting the ratio between accuracy and similarity.

[0084] A quantization artificial intelligence learning processing device (100) according to one embodiment of the present invention allows the model to recognize salient features of data well during the quantization process by incorporating KL divergence into the loss function while maintaining the distribution of quantized data for initial learning.

[0085] The combined model of LSTM and attention mechanisms can be referred to as Latte, and the model that applies quantization and accuracy information and distribution information can be referred to as Q-Latte.

[0086] Accordingly, the present invention can increase saliency of quantized artificial intelligence learning results by adjusting the ratio between accuracy and similarity by learning accuracy information and distribution similarity information for loss between learning data and quantized data together when re-learning artificial intelligence learning data after quantization.

[0087] FIG. 2 is a drawing illustrating a learning result of a quantization artificial intelligence learning processing device according to an embodiment of the present invention.

[0088] FIG. 2 illustrates a learning result according to a re-learning process in an artificial intelligence learning processing process along with a learning result of a quantization artificial intelligence learning processing device according to an embodiment of the present invention.

[0089] Referring to FIG. 2, an artificial intelligence learning processing device according to one embodiment of the present invention learns input data and outputs learning data (200) as a learning result.

[0090] For example, an artificial intelligence learning processing device performs quantization processing on learning data (200) and outputs quantized data (201).

[0091] There is a difference in skewness and kurtosis between the learning data (200) and the quantized data (201).

[0092] The artificial intelligence learning processing device derives parameters that can be used for relearning quantized data using the difference values ​​between skewness and kurtosis using mathematical equations 1 to 3.

[0093] An artificial intelligence learning processing device according to one embodiment of the present invention outputs relearning data (202) by relearning quantized data (201) using parameters calculated using mathematical expressions 1 to 3.

[0094] The difference in the values ​​of skewness and kurtosis between the training data (200) and the retraining data (202) is reduced.

[0095] Therefore, the present invention can effectively identify temporal dependence and contextual relevance by identifying a trade-off between the efficiency and accuracy of a model in artificial intelligence learning processing, thereby deriving artificial intelligence learning results.

[0096] FIG. 3 and FIG. 4 are diagrams illustrating the learning performance of a quantization artificial intelligence learning processing device according to one embodiment of the present invention.

[0097] FIG. 3 shows that, in relation to the learning performance of a quantized artificial intelligence learning processing device according to an embodiment of the present invention, the relearning result includes a difference portion that can be removed through quantization, thereby preventing a degradation in model performance.

[0098] Referring to FIG. 3, graph (300) represents learning data before quantization, graph (310) represents quantized data after quantization, and graph (320) represents relearning data after relearning of quantized data.

[0099] When comparing graph (310) and graph (320) with respect to graph (300), it can be confirmed that graph (310) contains data that has been lost from graph (300).

[0100] Comparing graph (300) and graph (310) shows that quantization can distort the distribution, resulting in loss of important information.

[0101] Although LSTM models are more efficient for deployment on resource-constrained devices, they must be retrained as in graph (320) to consider the trade-off between efficiency and accuracy for effective application in real-world applications.

[0102] On the other hand, it can be seen that the graph (320) has less data lost from the graph (300) compared to the graph (310).

[0103] FIG. 4 illustrates learning results depending on whether re-learning is applied in relation to the learning performance of a quantization artificial intelligence learning processing device according to an embodiment of the present invention.

[0104] Referring to FIG. 4, the results of not applying relearning processing in relation to the learning performance of a quantization artificial intelligence learning processing device according to one embodiment of the present invention are represented as a graph (400), and the results of applying relearning processing are represented as a graph (410).

[0105] The graph (400) can represent first quantized data (402) to which quantization is applied to learning data (401) and second quantized data (403) to which quantization is applied.

[0106] The graph (410) can represent first quantized data (412) to which quantization is applied to learning data (411) and second quantized data (413) to which quantization is applied.

[0107] When comparing graph (400) and graph (410), it can be seen that the difference in distribution information is relatively reduced.

[0108] That is, the quantization artificial intelligence learning processing device according to one embodiment of the present invention reduces data loss and differences due to quantization by considering distribution information.

[0109] For example, a quantized artificial intelligence learning processing unit improves performance by developing efficient and accurate deep learning models.

[0110] A quantization artificial intelligence learning processing device according to an embodiment of the present invention can provide a highly efficient solution for time series prediction tasks by providing excellent performance while significantly reducing inference costs.

[0111] The present invention can provide a versatile solution that can be applied to various types of prediction tasks, from long-term building energy prediction to simultaneous prediction of multiple energy data within a building, based on a quantized artificial intelligence learning processing device.

[0112] The present invention can be implemented as a quantized artificial intelligence learning processing device in accordance with the need for a high-performance and efficient prediction model capable of handling complex multivariate prediction tasks without compromising speed or accuracy.

[0113] The quantization artificial intelligence learning processing device of the present invention presents a new standard for time series prediction and can be applied to various practical applications in various fields where powerful prediction functions are essential.

[0114] Therefore, the present invention effectively maintains a close approximation by using divergence for the distribution of the artificial intelligence learning result learned based on LSTM and attention mechanism and the distribution after quantization processing of the artificial intelligence learning result, thereby minimizing the mismatch between the two distributions, thereby ensuring efficiency and accuracy of the artificial intelligence learning result.

[0115] FIG. 5 is a diagram illustrating a quantization artificial intelligence learning processing method according to an embodiment of the present invention.

[0116] FIG. 5 illustrates a procedure for increasing saliency for quantized artificial intelligence learning results by learning accuracy information and distribution information of a loss function together when re-learning artificial intelligence learning data after quantizing it according to an embodiment of the present invention, thereby adjusting the ratio between accuracy and similarity.

[0117] Referring to FIG. 5, in step (S501), a quantization artificial intelligence learning processing method according to an embodiment of the present invention learns input data.

[0118] That is, the quantization artificial intelligence learning processing method according to one embodiment of the present invention can learn input data using an artificial intelligence model and output learning data.

[0119] In step (S502), a quantization artificial intelligence learning processing method according to an embodiment of the present invention performs quantization processing of learning data.

[0120] That is, the quantization artificial intelligence learning processing method according to one embodiment of the present invention can output quantized data by quantizing learning data.

[0121] In step (S503), a quantization artificial intelligence learning processing method according to an embodiment of the present invention re-learns quantization data by taking into account the ratio between accuracy information and distribution information.

[0122] That is, the quantization artificial intelligence learning processing method according to one embodiment of the present invention can relearn the quantized data by considering the ratio between the accuracy information and the distribution information based on the loss function for the learning data and the quantized data.

[0123] Accordingly, the present invention can provide a quantized artificial intelligence learning processing device and method that can be applied to various applications in various technical fields that require prediction based on time series data, while ensuring efficiency and accuracy of artificial intelligence learning results and processing complex multivariate prediction tasks without reducing speed or accuracy.

[0124] The devices described above may be implemented as hardware components, software components, and / or a combination of hardware components and software components. For example, the devices and components described in the embodiments may be implemented using one or more general-purpose computers or special-purpose computers, such as, for example, a processor, a controller, an arithmetic logic unit (ALU), a digital signal processor, a microcomputer, a field programmable array (FPA), a programmable logic unit (PLU), a microprocessor, or any other device capable of executing instructions and responding to them. The processing device may execute an operating system (OS) and one or more software applications running on the operating system. The processing device may also access, store, manipulate, process, and generate data in response to the execution of the software. For ease of understanding, the processing device is sometimes described as being used alone; however, one of ordinary skill in the art will recognize that the processing device may include multiple processing elements and / or multiple types of processing elements. For example, a processing unit may include multiple processors, or a processor and a controller. Other processing configurations, such as parallel processors, are also possible.

[0125] Software may include a computer program, code, instructions, or a combination of one or more of these, and may configure a processing device to perform a desired operation or, independently or collectively, command the processing device. The software and / or data may be permanently or temporarily embodied in any type of machine, component, physical device, virtual equipment, computer storage medium or device, or transmitted signal wave, for interpretation by the processing device or for providing instructions or data to the processing device. The software may be distributed on network-connected computer systems and stored or executed in a distributed manner. The software and data may be stored on one or more computer-readable recording media.

[0126] Although the embodiments described above have been described with limited drawings, those skilled in the art will recognize that various modifications and variations can be made based on the above description. For example, appropriate results can still be achieved even if the described techniques are performed in a different order than described, and / or components of the described systems, structures, devices, circuits, etc. are combined or combined in a different manner than described, or are replaced or substituted with other components or equivalents.

[0127] Therefore, other implementations, other embodiments, and equivalents to the claims also fall within the scope of the claims described below.

Claims

1. A learning processing unit that learns input data using an artificial intelligence model and outputs learning data; A quantization processing unit that quantizes the above learning data and outputs quantized data; and It is characterized by including a re-learning processing unit that re-learns the quantized data by considering the ratio between the accuracy information and the distribution information based on the loss function for the above learning data and the quantized data. Quantized artificial intelligence learning processing unit.

2. In paragraph 1, The above relearning processing unit calculates the absolute weight of the accuracy information and the distribution information based on the difference value of the skewness and kurtosis of the learning data and the quantized data, and calculates the relative ratio of the accuracy information and the distribution information based on the calculated absolute weight. Quantized artificial intelligence learning processing unit.

3. In paragraph 2, The above relearning processing unit calculates a parameter (x) related to the similarity based on the ratio between the difference value of the skewness value of the learning data and the skewness value of the quantized data for the skewness value of the quantized data, calculates a parameter (m') for the absolute proportion based on the calculated parameter (x), and calculates a parameter (m) for the relative proportion based on the calculated parameter (m'). Quantized artificial intelligence learning processing unit.

4. In paragraph 2, The above relearning processing unit calculates a parameter (y) related to accuracy based on a ratio between a difference value between a skewness value of the learning data and a kurtosis value of the quantized data for a kurtosis value of the quantized data, calculates a parameter (n') for the absolute proportion based on the calculated parameter (y), and calculates a parameter (n) for the relative proportion based on the calculated parameter (n'). Quantized artificial intelligence learning processing unit.

5. In paragraph 2, The above relearning processing unit is characterized in that it configures the loss function with a mean square error (MSE) loss operation and a relative entropy operation, applies a parameter (n) related to the relative ratio of the accuracy information to the mean square error loss operation, and relearns the quantized data according to the operation result of applying a parameter (m) related to the relative ratio of the distribution information to the relative entropy operation. Quantized artificial intelligence learning processing unit.

6. In paragraph 5, The above relearning processing unit is characterized in that it reduces the distribution mismatch between the learning data and the quantized data by relearning the quantized data so that the sum of the parameter (m) and the parameter (n) approaches “1”. Quantized artificial intelligence learning processing unit.

7. In paragraph 5, The above relearning processing unit is characterized in that it relearns the quantized data by increasing the weight on the similarity according to the increase in the parameter (m) in the loss function and increasing the weight on the accuracy according to the increase in the parameter (n). Quantized artificial intelligence learning processing unit.

8. In paragraph 1, The above learning processing unit is characterized by using the artificial intelligence model as a combined model of LSTM (Long Short Term Memory), one of RNN (Recurrent Neural Network), and Attention Mechanism. Quantized artificial intelligence learning processing unit.

9. In the learning processing unit, a step of learning input data using an artificial intelligence model and outputting learning data; In the quantization processing unit, a step of quantizing the learning data and outputting quantized data; and In the re-learning processing unit, it is characterized by including a step of re-learning the quantized data by considering the ratio between the accuracy information and the distribution information based on the loss function for the learning data and the quantized data. Quantization artificial intelligence learning processing method.

10. In paragraph 9, The step of retraining the above quantized data is: A step of calculating a parameter (x) related to similarity based on a ratio between the difference value between the skewness value of the learning data and the skewness value of the quantized data and the skewness value of the quantized data based on the difference value of the skewness and kurtosis of the learning data and the quantized data, calculating a parameter (m') for absolute proportion based on the calculated parameter (x), and calculating a parameter (m) for relative proportion based on the calculated parameter (m'); and It is characterized by including a step of calculating a parameter (y) related to accuracy based on a ratio between a difference value between a skewness value of the learning data and a kurtosis value of the quantized data for the kurtosis value of the quantized data, calculating a parameter (n') for the absolute proportion based on the calculated parameter (y), and calculating a parameter (n) for the relative proportion based on the calculated parameter (n'). Quantization artificial intelligence learning processing method.

Images