Electronic device for data compression and decompression and compression method thereof

Abstract

A data compression method and a data decompression method are provided. The method includes: pruning original data including a plurality of weight parameters; confirming at least one first weight parameter whose first value is not changed by pruning among the plurality of weight parameters included in the pruned original data; obtaining first index data including position information of at least one first weight parameter whose first value is not changed; confirming at least one second weight parameter whose second value is changed due to pruning from among the plurality of weight parameters included in the pruned original data; and replacing at least one second weight parameter whose second value is changed with an irrelevant parameter.

Description

Technical Field

[0001] The present disclosure relates to a method for compressing and decompressing data. More particularly, the present disclosure relates to an electronic device capable of performing compression in parallel.

[0002] In addition, the present disclosure relates to an artificial intelligence (AI) system and its applications that use machine learning algorithms to imitate functions such as cognition and decision-making of the human brain. Background Art

[0003] Unlike previous rule-based intelligent systems, AI systems are computer systems that achieve human-level intelligence, and are systems in which machines learn and make decisions on their own, thereby becoming intelligent. As AI systems are increasingly used, the recognition rate is improved, and therefore, the user's taste is understood more accurately. Therefore, previous rule-based intelligent systems are gradually being replaced by deep learning AI systems.

[0004] AI technology includes machine learning (deep learning) and element technologies that use machine learning.

[0005] Machine learning is an algorithmic technology that classifies and learns the features of input data by itself. Element technology is a technology that uses machine learning algorithms such as deep learning, which includes technology fields including semantic understanding, visual understanding, reasoning / prediction, knowledge representation, motion control, etc.

[0006] Various fields where AI technology is applied are as follows. Semantic understanding is a technology that recognizes human language and features and applies and processes them, including natural language processing, machine translation, dialogue systems, question and answer, speech recognition and synthesis, etc. Visual understanding is a technology that recognizes and processes objects like human vision, including object recognition, object tracking, image search, human recognition, scene understanding, spatial understanding, image improvement, etc. Reasoning prediction is a technology that determines information and performs logical reasoning and prediction, including knowledge / probability-based reasoning, optimization prediction, preference-based planning, recommendation, etc. Knowledge expression is a technology that uses knowledge data to perform automated processing on human experience information, including knowledge construction (data generation / classification), knowledge management (data use), etc. Motion control is a technology that controls the autonomous driving of vehicles and the movement of robots, including motion control (navigation, collision, driving), manipulation control (behavior control), etc. Summary of the invention

[0007] Technical issues

[0008] In recent years, deep learning technology has been discussed as a field of artificial intelligence. Generally, in the case of a deep learning model whose storage scale is large, there is an aspect that it is difficult to use the deep learning model in a general electronic device such as a smartphone due to the limitation of the storage scale.

[0009] In addition, in the case of applying previous compression methods to deep learning suitable for parallel operations, there are many problems in the utilization of memory bandwidth and the power consumption is also large.

[0010] Solution to the problem

[0011] Provided are an electronic device and a data compression method for compressing weight parameters of an artificial intelligence model.

[0012] An electronic device and a decompression method for decompressing compressed data are also provided.

[0013] Additional aspects will be set forth in part in the description which follows and, in part, will be obvious from the description, or may be learned by practice of the illustrated embodiments.

[0014] According to one aspect of the present disclosure, there is provided a data compression method for compressing weight parameters of an artificial intelligence model, the method being performed by an electronic device, and the method comprising: pruning original data including a plurality of weight parameters; confirming at least one first weight parameter whose first value has not changed according to pruning from among the plurality of weight parameters included in the pruned original data; and obtaining first index data including the position information of at least one first weight parameter whose first value has not changed. The method further comprises: confirming at least one second weight parameter whose second value has changed due to pruning from among the plurality of weight parameters included in the pruned original data; replacing at least one second weight parameter whose second value has changed with an irrelevant parameter (don'tcare parameter); and quantizing the first data to n bits, the first data comprising at least one first weight parameter whose first value has not changed and an irrelevant parameter replacing at least one second weight parameter. The method further comprises: obtaining n second data based on the first data quantized to n bits; and obtaining n compressed data by applying each of the obtained n second data to the Viterbi algorithm.

[0015] The first data may be in a matrix form, and quantizing the first data may include quantizing each of at least one first weight parameter whose at least one first value is unchanged and an irrelevant parameter replacing at least one second weight parameter in the matrix form.

[0016] The n second data may include: 2-1 data, which includes a first bit value of at least one first weight parameter quantized into n bits and a first bit of an irrelevant parameter quantized into n bits; and 2-2 data, which includes a second bit value of at least one first weight parameter quantized into n bits and a second bit of an irrelevant parameter quantized into n bits.

[0017] Obtaining n compressed data may include: comparing output data output by inputting at least one compression candidate data into at least one Viterbi decompressor with the obtained n second data; and, based on the obtained n second data and the output data matching at a predetermined ratio or above, obtaining at least one compression candidate data as the n compressed data.

[0018] The comparison may include comparing data corresponding to only the at least one first weight parameter whose at least one first value has not changed among the n second data with the plurality of pieces of data.

[0019] The data compression method may further include obtaining compressed index data by applying the first index data to a Viterbi algorithm.

[0020] Obtaining compressed index data may include: comparing output data output by inputting at least one compression candidate data into a Viterbi decompressor with first index data; and obtaining at least one compression candidate data as compressed index data based on that the first index data and the output data match at a predetermined ratio or above.

[0021] The data compression method may further include storing the compression index data, the n compressed data, and the table data for quantization.

[0022] According to one aspect of the present disclosure, an electronic device for compressing weight parameters of an artificial intelligence model is provided, the device comprising a memory storing instructions and a processor, the processor being configured to: prune original data including a plurality of weight parameters; confirm at least one first weight parameter whose first value has not changed according to pruning from among the plurality of weight parameters included in the pruned original data; and obtain first index data including the position information of at least one first weight parameter whose first value has not changed. The processor is also configured to execute the stored instructions to: confirm at least one second weight parameter whose second value has changed due to pruning from among the plurality of weight parameters included in the pruned original data; replace at least one second weight parameter whose second value has changed with an irrelevant parameter; and quantize the first data to n bits, the first data comprising at least one first weight parameter whose first value has not changed and an irrelevant parameter replacing at least one second weight parameter. The processor is also configured to execute the stored program to: obtain n second data based on the first data quantized to n bits; and obtain n compressed data by applying each of the obtained n second data to the Viterbi algorithm.

[0023] The first data may be in matrix form, and the processor is further configured to execute the stored instructions to: quantize each of at least one first weight parameter whose at least one first value is not changed and an irrelevant parameter replacing at least one second weight parameter in matrix form.

[0024] The n second data may include: 2-1 data, which includes a first bit value of at least one first weight parameter quantized into n bits and a first bit of an irrelevant parameter quantized into n bits; and 2-2 data, which includes a second bit value of at least one first weight parameter quantized into n bits and a second bit of an irrelevant parameter quantized into n bits.

[0025] The processor is also configured to execute stored instructions to: compare output data output by inputting n at least one compression candidate data into at least one Viterbi decompressor with n second data obtained; and, based on the obtained n second data and the output data matching at a predetermined ratio, obtain at least one compression candidate data as n compressed data.

[0026] The processor is also configured to execute the stored instructions to compare data corresponding to only the at least one first weight parameter whose at least one first value has not changed among the n second data with the output data.

[0027] The processor is also configured to execute the stored instructions to obtain compressed index data by applying the first index data to a Viterbi algorithm.

[0028] The processor is also configured to execute stored instructions to: compare output data output by inputting at least one compression candidate data into a Viterbi decompressor with first index data; and obtain index data for at least one compression candidate data as compressed index data based on that the first index data and the output data match at a predetermined ratio.

[0029] The memory may also be configured to store the compressed index data, the n compressed data, and table data for quantizing the first data.

[0030] According to one aspect of the present disclosure, a decompression method for decompressing compressed data is provided, the method being performed by an electronic device, and the method comprising: receiving compressed index data and n compressed data; obtaining first index data obtained by applying the compressed index data to a first Viterbi decompressor; and obtaining n first data by applying the n compressed data to a plurality of Viterbi decompressors, respectively. The method further comprises obtaining pruned original data based on the first index data and the n first data.

[0031] The decompression method may also include: inputting the pruned original data into an artificial intelligence model; retraining the artificial intelligence model that has been input with the pruned original data; and obtaining multiple weight parameters applied to the artificial intelligence model based on the retraining results.

[0032] According to one aspect of the present disclosure, an electronic device for decompressing compressed data is provided, the electronic device comprising a memory storing instructions and a processor, the processor being configured to execute the stored instructions to: receive compressed index data and n compressed data; obtain first index data obtained by applying the compressed index data to a first Viterbi decompressor; and obtain n first data by applying the n compressed data to a plurality of Viterbi decompressors, respectively. The processor being configured to execute the stored instructions to: obtain pruned original data based on the first index data and the n first data.

[0033] The processor is configured to execute stored instructions to: input the pruned original data into the artificial intelligence model; retrain the artificial intelligence model that has been input with the pruned original data; and obtain multiple weight parameters applied to the artificial intelligence model based on the results of the retrained artificial intelligence model.

[0034] According to one aspect of the present disclosure, there is provided a data compression method performed by an electronic device, the method comprising: pruning original data including multiple weight parameters of an artificial intelligence model; obtaining first index data including position information of at least one first weight parameter whose first value has not changed according to pruning from the multiple weight parameters included in the pruned original data; obtaining first data by replacing at least one second weight parameter whose second value among the multiple weight parameters included in the pruned original data has changed due to pruning with an irrelevant parameter; and quantizing at least one first weight parameter whose first value has not changed and the irrelevant parameter replacing the at least one second weight parameter to n bits. The method also comprises: obtaining second data based on the first data quantized to n bits; and obtaining compressed index data and compressed data by applying each of the first index data and the second data to the Viterbi algorithm.

[0035] The second data may include: 2-1 data, which includes a first bit value of at least one first weight parameter quantized into n bits and a first bit of an irrelevant parameter quantized into n bits; and 2-2 data, which includes a second bit value of at least one first weight parameter quantized into n bits and a second bit of an irrelevant parameter quantized into n bits. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] The above and other aspects and advantages of the embodiments of the present disclosure will become more apparent from the following description in conjunction with the accompanying drawings, in which:

[0037] Figure 1 is a diagram showing an example compression method according to the prior art;

[0038] Figure 2 is a block diagram of an electronic device according to an embodiment;

[0039] Figure 3 is a diagram illustrating an example method for compressing index data according to an embodiment;

[0040] Figure 4 is a diagram illustrating an example method for compressing non-zero data according to an embodiment;

[0041] Figure 5 is a view for explaining a data compression method according to an embodiment;

[0042] Figure 6 and Figure 7 is a view for explaining a method for decompressing raw data according to an embodiment;

[0043] Figure 8 is a flowchart of a method for compressing data according to an embodiment; and

[0044] Fig. 9 is a flowchart for explaining a data decompression method according to an embodiment. DETAILED DESCRIPTION

[0045] Embodiments of the present disclosure will be described in detail with reference to the accompanying drawings. However, it will be appreciated that the present disclosure is not limited to the following description of the embodiments, but may include various modifications, equivalents and / or alternative forms of the embodiments of the present disclosure. With respect to the description of the accompanying drawings, similar reference numerals may be used for similar constituent elements.

[0046] Hereinafter, terms used in the embodiments will be briefly explained, and the embodiments will be described in more detail with reference to the accompanying drawings.

[0047] The terms used in this disclosure are general terms that are currently widely used and selected in consideration of the functions of the present disclosure. However, these terms may vary depending on the intentions of those skilled in the art, precedents, or the emergence of new technologies. In addition, in certain cases, the terms may be selected arbitrarily. In this case, the meaning of the terms will be described in the corresponding description. Therefore, the terms used in the specification need not be simply interpreted as the names of the terms, but are defined based on the meaning of the terms and the overall content of the present disclosure.

[0048] The embodiments may vary and may be provided in different embodiments. The embodiments will be described with reference to the accompanying drawings. However, this does not necessarily limit the scope of the embodiments to the embodiment forms. As an alternative, modifications, equivalent forms and replacements within the disclosed concepts and technical scopes of this specification may be adopted. When describing the embodiments, if it is determined that the description of the known technology obscures the gist of the present disclosure, the description is omitted.

[0049] In the present disclosure, relational terms such as first and second, etc., may be used to distinguish one entity from another entity without necessarily implying any actual relationship or order between such entities. In embodiments of the present disclosure, relational terms such as first and second, etc., may be used to distinguish one entity from another entity without necessarily implying any actual relationship or order between such entities.

[0050] It should be understood that the singular forms of "a", "an" and "the" also include multiple referents unless the context clearly indicates otherwise. The terms "including", "comprising", "configured to", etc. in the specification are used to indicate the presence of features, numbers, steps, operations, elements, parts, or a combination of the above items, and they should not exclude the possibility of combining or adding one or more features, numbers, steps, operations, elements, parts, or a combination of the above items. In the present disclosure, a 'module' or 'unit' performs at least one function or operation and can be implemented as hardware, software, or a combination of hardware and software. In addition, multiple 'modules' or multiple 'units' can be integrated into at least one module, and can also be implemented as at least one processor in addition to the 'modules' or 'units' that should be implemented in specific hardware.

[0051] The above and other aspects of the present disclosure will become more apparent by referring to the embodiments described in detail with reference to the accompanying drawings. However, the embodiments can be implemented in various different configurations and are not limited to the description provided herein. In addition, the contents irrelevant to the description are omitted to more clearly describe the embodiments, and similar reference numerals are used for similar elements throughout the description.

[0052] Figure 1 is a view showing an example compression method according to the related art.

[0053] like Figure 1 As shown, the electronic device 100 can reduce the size of the original data by pruning the original data. The data to be compressed can be data related to the connection strength between neurons of the learned neural network. For example, the data can be the connection strength between neurons included in the data recognition model, such as a deep neural network (DNN), a recurrent neural network (RNN), and a bidirectional recurrent deep neural network (BRDNN). In the following, the data compressed by the electronic device 100 is referred to as a weight parameter. The original data may include multiple weight parameters.

[0054] Pruning is one of the compression methods for removing unimportant weight parameters or weight parameters that do not significantly affect the output value from the weight parameters of the artificial intelligence model. For example, in the case where the connection strength between the learned neurons is weak or does not significantly affect the output value, the electronic device 100 can delete the connection strength between the learned neurons by pruning.

[0055] The electronic device 100 may prune data whose size is less than or equal to a size based on the value of the size of each weight parameter, and retrain the deep learning model by using the remaining data, and restore the accuracy of the deep learning model.

[0056] For example, we can assume that the pruned original data is in the form of a matrix as shown below. The original data after pruned and retrained has many zero values.

[0057]

[0058] That is, the electronic device 100 can go through the pruning process and obtain multiple zero values ​​and some non-zero values. The electronic device 100 can reduce the size of the matrix by eliminating the number of zero values ​​according to the compressed sparse row (CSR) format. For example, the electronic device can reduce the size of the matrix by transforming the matrix shown above into three matrices as shown below.

[0059] A=[5 8 3 6]

[0060] IA=[0 0 2 3 4]

[0061] JA=[0 1 2 1]

[0062] The matrix A may be a matrix corresponding to non-zero values ​​among the matrices corresponding to the pruned original data. The matrix IA may be a matrix in which the number of non-zero data corresponding to rows in the matrix corresponding to the pruned original data is accumulated. The matrix JA may be a matrix capable of storing the values ​​of the column index corresponding to each non-zero value. The electronic device 100 may obtain a matrix corresponding to the pruned original data by using the matrix A, the matrix IA, and the matrix JA.

[0063] However, in the case of a compression method using the CSR format, the size of the index is always larger than the size of the non-zero data corresponding to the matrix A. Therefore, there is a problem that the compression rate of the index may need to be increased to increase the overall compression rate. In addition, in the case of a compression method using the CSR format, there is a problem of not being suitable for parallel operations. Therefore, in the case of using parallel operations to process compressed deep learning models, there are disadvantages of reduced memory bandwidth usage and high power consumption. Therefore, a compression method that can perform compression in parallel while maintaining the previous pruning rate is needed.

[0064] Figure 2 is a block diagram of a brief configuration of an electronic device according to an embodiment.

[0065] In this case, the electronic device 100 may be various devices. The electronic device 100 may be implemented as a smart phone, but is not limited thereto. The electronic device 100 may include a smart phone, a tablet personal computer (PC), a mobile phone, a video phone, an e-book reader, a desktop personal computer (PC), a laptop personal computer (PC), a netbook computer, a workstation, a server, a personal digital assistant (PDA), a portable multimedia player (PMP), and an MP3 player.

[0066] In another embodiment, the electronic device 100 may be a household appliance. The household appliance may include, for example, a television, a digital video disk (DVD) player, a stereo, a refrigerator, an air conditioner, a cleaner, an oven, a microwave, a washing machine, an air purifier, a set-top box, a home automation control panel, a security control panel, a TV box (e.g., Samsung HomeSync TM 、AppleTV TM or Google TV TM ), game consoles (e.g., Xbox TM and PlayStation TM ), electronic dictionaries, electronic keys, portable cameras, and electronic photo frames. Alternatively, the electronic device 100 may be a system itself in which a cloud computing environment is constructed.

[0067] The electronic device 100 according to the embodiment may include a memory 110 and a processor 120. However, various elements such as a display, a communication interface, an input interface, an output interface, a sensor, etc. may be added or omitted according to the needs of the above-mentioned various electronic devices.

[0068] The memory 110 may store various software and / or programs. The program may include, for example, a kernel, middleware, an application programming interface (API), and / or an application program (or "application"). At least a portion of the kernel, middleware, or API may be referred to as an operating system (OS). The kernel may, for example, control or manage system resources for performing operations or functions implemented in other programs. In addition, the kernel may provide an interface to control or manage system resources by accessing various elements of the electronic device 100 in the middleware, API, or application program.

[0069] For example, middleware can act as an intermediary for an API or application to communicate with a kernel and exchange data. In addition, middleware can process one or more job requests received from an application based on priority. For example, middleware can prioritize any one or any combination of applications to use the system resources of the electronic device 100, and can process one or more job requests. API is an interface for an application to control functions provided in a kernel or middleware, and can include, for example, at least one interface or function (e.g., instruction) for file control, window control, image processing, or character control.

[0070] In addition, the memory 130 may include one or both of a built-in memory and an external memory. For example, the built-in memory may include a volatile memory (e.g., DRAM, SRAM, or SDRAM, etc.) and a non-volatile memory (e.g., one-time programmable ROM (OTPROM), PROM, EPROM, EEPROM, mask ROM, flash ROM, flash memory, hard disk drive, or solid-state drive (SSD)), such as a non-temporary computer-readable memory. The external memory may include a flash drive such as a compact flash (CF) memory, a secure digital (SD) memory, a micro SD, a mini SD, an extreme digital (xD), a multimedia (MMC) card, or a memory stick. The external memory may be functionally or physically connected to the electronic device 100 through various interfaces.

[0071] In an embodiment, the memory 110 may store various artificial intelligence models. In addition, the memory 110 may store raw data and compressed data. In addition, the memory 110 may store error data generated in the process of compressing and decompressing raw data. In addition, the memory 110 may also store compressed intermediate data, compressed index data, and table data for quantization used to compress raw data.

[0072] The processor 120 is a configuration for controlling the electronic device 100. In detail, the processor 120 may prune original data including a plurality of weight parameters, determine a weight parameter whose value is not changed according to the pruning from among the weight parameters included in the pruned original data, and obtain index data of position information of the weight parameter whose value is not changed according to the pruning. The processor 120 may determine a weight parameter whose value is changed according to the pruning from among the weight parameters included in the pruned original data, and replace the weight parameter whose value is changed according to the pruning with an irrelevant parameter.

[0073] Quantization is one of the compression methods for reducing the size of the weight parameter. That is, the electronic device 100 can change the size of the weight parameter to a predetermined number of bits to reduce the size of the weight parameter. For example, in the case where each weight parameter has a size of 32 bits, the electronic device 100 can quantize the weight parameter to a predetermined size of 3 bits to reduce the size of the weight parameter. According to the embodiment, various quantization methods can be applied, but the technical idea of ​​the present disclosure is not limited to the quantization method described in the present disclosure.

[0074] A don't care value refers to a value that is not defined by a value. For example, in the case where true / false is determined by comparing a value (0 or 1) with a don't care value, the don't care value has a true value for either 0 or 1. As will be described later, by replacing the weight parameter that has become 0 according to pruning, when comparing data output by inputting the compressed candidate data as an input value to the Viterbi decompressor with the obtained n second data, regardless of the parameter value of the output data corresponding to the don't care value included in the obtained n second data, only the value of the weight parameter that has not been changed can be compared with the parameter value of the output data.

[0075] The processor 120 may obtain n second data based on the quantized data. The processor 120 may obtain n compressed data by applying each of the obtained n second data to a Viterbi algorithm.

[0076] In detail, the processor 120 may compress each of the obtained n second data by using an existing test data compression method.

[0077] The raw data including the weight parameters can be represented in a matrix form. For example, the raw data can be in a matrix form as shown below.

[0078]

[0079] However, in addition to the 4*4 form shown above, the matrix representation of the original data can also be represented as various m*m matrices. However, hereinafter, for the sake of convenience of explanation, it will be described that the original data is in a matrix form with a size of 4*4.

[0080] In the case where pruning is performed on the original data, the pruned original data can be represented as follows.

[0081]

[0082] The processor 120 can determine the weight parameters whose values ​​have been changed according to the pruning from the weight parameters included in the pruned original data, and replace the weight parameters whose values ​​have been changed according to the pruning with irrelevant parameters. The irrelevant value replacement means that the corresponding value can be any value between 0 and 1. For ease of explanation, the data processed as irrelevant parameters are represented as x. The matrix processed as irrelevant parameters may be as follows.

[0083]

[0084] The processor 120 may quantize the first data into n bits, the first data including the weight parameter and the irrelevant parameter whose value is not changed according to the pruning. For example, the result of quantizing the first data into 3 bits may be as follows.

[0085]

[0086] The processor 120 may obtain n second data based on the data quantized into n bits. The n second data may include: 2-1 data, the 2-1 data including the first bit value quantized into n bits of the weight parameter whose value does not change according to the pruning and the first bit value quantized into n bits of the irrelevant parameter; and 2-2 data, the 2-2 data including the second bit value quantized into n bits of the weight parameter whose value does not change according to the pruning and the second bit value quantized into n bits of the irrelevant parameter. In the same way, the processor 120 may obtain 2-3 data to 2-n data.

[0087] For example, in the case where quantization is performed into 3 bits as described above, the processor 120 may obtain first to third data as shown below.

[0088]

[0089] By obtaining data for each corresponding bit, the processor 120 can obtain a matrix for independent data. Since each parameter included in the data is independent of each other (one parameter does not affect the determination of the value of another parameter), the data can be compressed by using a test data compression method as will be described later.

[0090] The processor 120 may compare the data outputted by inputting at least one compression candidate data as an input value to the Viterbi decompressor with the obtained n second data, and if the obtained n second data and the output data match at a predetermined ratio or more, obtain the input value of the output data as the compressed data. For this, n compressed data may be generated.

[0091] The processor 120 may compare only the data corresponding to the weight parameter whose value is not changed according to the pruning among the data included in the corresponding n second data with the output data. That is, in the case where the value output by inputting the compressed candidate data to the Viterbi decompressor is compared with the first data among the n second data, the processor 120 may not compare the data corresponding to the irrelevant parameter.

[0092] For example, if the values ​​output by inputting the compressed candidate data to the Viterbi decompressor are as shown in the left matrix below, and the first data are as shown in the right matrix below, the processor 120 can obtain the compressed candidate data input to the Viterbi decompressor as the first compressed data, because the values ​​of the remaining parameters of the first data except the irrelevant parameters are the same as the values ​​output by inputting the compressed candidate data to the Viterbi decompressor.

[0093]

[0094] The processor 120 may obtain index data from the pruned original data. The index data may be data for searching for a position of a weight parameter whose value is not changed according to the pruning.

[0095] For example, assume that the index data looks like the following.

[0096]

[0097] The processor 120 may compress the index data by using the Viterbi algorithm. The compressed index data may be, for example, [0 1 1 0].

[0098] In detail, the compressed index data may be obtained by the method for obtaining n compressed data as described above. For example, the processor 120 may compare the data output by inputting at least one compressed candidate data as an input value into a Viterbi decompressor with the index data, and if the index data and the output data match at least a predetermined ratio, the input value of the output data is obtained as the compressed index data.

[0099] By the above method, the processor 120 can compress the original data and obtain n compressed data and compressed index data. The processor 120 can decompress the compressed data. In detail, the processor 120 can decompress the n compressed data and compressed index data and restore the original data. The processor 120 can replace the irrelevant parameters with preset values ​​(e.g., 0). That is, the processor 120 can replace the weight value that has changed according to the pruning with the preset value.

[0100] When there is an error between the original data after decompression and the original data before decompression, the processor 120 may apply n second data obtained based on the data quantized into n bits to the artificial intelligence model as an input value and retrain the error.

[0101] In the above-mentioned embodiment, it is described that n data and index data are compressed using a Viterbi decompressor, but the example is not limited thereto. That is, the processor 120 may compress the original data by using various decoders capable of performing parallel operations other than the Viterbi decompressor.

[0102] The processor 120 may also be implemented as any one or any combination of an application specific integrated circuit (ASIC), an embedded processor, a microprocessor, hardware control logic, a hardware finite state machine (FSM), a digital signal processor (DSP). The processor 120 may also include an interface for communicating with each configuration, such as a bus.

[0103] The processor 120 may control a plurality of hardware or software elements connected to the processor 120, for example, by driving an operating system or an application program, and perform various data processing and calculations. For example, the processor 120 may be implemented as a system on a chip (SoC). According to an embodiment, the processor 120 may also include a graphics processing unit (GPU) and / or an image signal processor. The processor 120 may load and process instructions or data received from any one or any combination of other elements (e.g., a non-volatile memory such as a non-transitory computer-readable storage device) to a volatile memory, and store the resulting data in the non-volatile memory.

[0104] In the above embodiment, the process of compressing the original data and the process of decompressing the compressed data are performed in one electronic device 100, but the example is not limited to this. That is, the electronic device that compresses the original data and the electronic device that decompresses the compressed data may be different from each other. For example, data compression may be performed in a server, and data decompression may be performed in a user terminal.

[0105] Figure 3 is a view showing an example method for compressing index data according to an embodiment.

[0106] like Figure 3 As shown, compression of index data can be performed using Viterbi decompressor 301. As described above, when the pruned original data is in the form of a matrix as shown on the left below, the index data used to decompress non-zero data into the pruned original data can be in the form of a matrix on the right below.

[0107]

[0108] The processor 120 may obtain an input value of the Viterbi decompressor 301, wherein Figure 3 The output values ​​of the Viterbi decompressor 301 shown change from the top to (0 0 0 0), (1 1 0 0), (0 0 1 0), and (0 1 0 0). For example, when (0 1 1 0) is input to the Viterbi decompressor 301, the output values ​​may be (0 0 0 0), (1 1 00), (0 0 1 0), and (0 1 0 0) from the top. The processor 120 may recognize the input value (0 1 1 0) as compressed index data.

[0109] That is, the processor 120 may obtain the original index data by using (0 1 1 0) and the Viterbi decompressor 301. Therefore, the processor 120 may recognize (0 1 0 0) as compressed index data. In addition, in the case of using the Viterbi decompressor 301, the processor 120 may compress the index data in parallel.

[0110] Figure 4 is a view showing an example method for compressing non-zero data according to an embodiment.

[0111] For effective compression, the processor 120 may quantize the pruned raw data and reduce the size of the data. Various quantization techniques may be applied to the method of quantizing the pruned data. In one embodiment, the processor 120 may quantize the raw data by presetting bits.

[0112] For the sake of convenience, it is assumed that the pruned original data is quantized to 3 bits. In this regard, the data quantized to 3 bits can be shown as follows.

[0113]

[0114] As described above, the processor 120 may obtain three data based on the data quantized to 3 bits. In the case where quantization is performed to n bits, n second data may be obtained.

[0115]

[0116] The processor 120 may compress the obtained n second data. In detail, the processor 120 may compress the n data by using an existing test data compression method.

[0117] In detail, in the existing test data compression method, in the case of target data (or target fault) to be confirmed, test data (or test cube) including the target data and irrelevant parameters other than the target data can be generated. The test equipment can decompress the target data by inputting a ratio corresponding to the target data at each clock cycle.

[0118] That is, the test data in the test data compression method may have a data structure in which most of the parameters are irrelevant, and the test device periodically inputs as much data as a given ratio in each clock cycle. Therefore, it is advantageous to use a parallel decoding method.

[0119] In detail, Figure 4As shown, the generated test data obtained from the random filling may, for example, include 95-99% of irrelevant parameters and the remaining parameters that are not irrelevant parameters. The test data may be compressed using a decoder that is capable of receiving data of the size of data that is not irrelevant parameters from the test data. For example, where the size of the test data is 1000 bits and the irrelevant parameters are 900 bits, the test device may apply a predetermined ratio of bits as input values ​​for 100 bits that are not irrelevant parameters (e.g., 10 bits are input 10 times per clock as input values, or 1 bit is input 100 times per clock), and accurately fill in the data that is not irrelevant parameters. That is, the test device may fill in the irrelevant parameters in the test data with arbitrary values, but may accurately fill in the values ​​that are not irrelevant parameters. In an embodiment, multiple data may be compressed by using a test device that meets the above-mentioned characteristics.

[0120] In summary, the processor 120 may change the weight parameter whose value is changed according to the pruning to an irrelevant non-zero parameter and perform compression using the test data compression method, thereby improving compression efficiency.

[0121] In an embodiment, the decoder used in the test apparatus is as follows Figure 3 In the case of the Viterbi decompressor 301 shown, a data compression device with a four-fold compression efficiency can be realized.

[0122] The electronic device 100 can compress the intermediate data by using the above-mentioned test data compression method. For example, in the case where the electronic device 100 obtains 3 data as described above, the obtained 3 data may include 36 irrelevant parameters and 12 parameters that are not irrelevant parameters. According to the above method, the electronic device 100 can apply a predetermined ratio of bits as 12 bits as an input value. For example, the electronic device 100 can input a 1-bit input value 12 times, a 2-bit input value 6 times, or a 3-bit input value 4 times.

[0123] In summary, when any one of the n second data is output with respect to the input value applied at each clock, the electronic device 100 can obtain the applied input value as any one of the n compressed second data.

[0124] In summary, the processor 120 can confirm the position of the pruned original data through the index data (or compressed index data or mask bit data), and therefore, it is not necessary to compress the n second data so that the value of the pruned original data satisfies zero (i.e., irrelevant parameter). Therefore, it does not matter which of 0 or 1 is input to the position of the pruned original data (i.e., by replacing the weight parameter whose value has changed according to the pruning with the irrelevant parameter), and thus the compression efficiency can be improved.

[0125] As described above, the Viterbi decompressor can be used to compress the original data, so that compressed data with a fixed compression rate in all cycles can be generated. Therefore, there is an advantage that the utilization rate of the storage bandwidth can be improved and parallel operations can be performed in the compression process. In addition, the Viterbi decompressor can be implemented using only a trigger and an XOR gate, so that the hardware of the electronic device 100 can have a simple configuration.

[0126] Even in the case of compressing the original data using the above method, errors may occur. In detail, the compression performed by the above test data compression method can have an accuracy of about 96% to 98%. This probability can be called coding efficiency, and the compression rate in this case can correspond to 1 / (1-pruning rate). The electronic device 100 can improve the accuracy of the errors that occur through a retraining process of deep learning, or can store data in which errors have occurred in the memory 110 and use the stored data to decompress the original data.

[0127] Figure 5 is a view for explaining an exemplary data compression method according to an embodiment.

[0128] like Figure 5 As shown, the electronic device 100 may prune the original data 510. The electronic device 100 may obtain index data 530 based on the pruned original data. In detail, the electronic device 100 may index the data 530, which is used to find the position of the weight parameter whose value has not changed according to the pruning from the pruned original data 520. The electronic device 100 may compress the index data 530 by using a Viterbi decompressor.

[0129] The electronic device 100 may quantize the pruned original data 520 to 3 bits. The electronic device 100 may also quantize the pruned original data 520 to n bits. The weight parameter whose value has changed according to the pruning may be replaced with an irrelevant parameter. In detail, the weight parameter whose value has changed according to the pruning may be replaced before performing quantization to 3 bits. However, the example is not limited thereto, and the weight parameter whose value has changed according to the pruning may be replaced after being quantized to 3 bits.

[0130] The electronic device 100 can obtain 3 data based on the data 550 quantized into 3 bits. In detail, the electronic device 100 can obtain: first data 560-1, which includes the first bit of each parameter including 3 bits contained in the data 550 quantized into 3 bits; second data 560-2, which includes the second bit of each parameter including 3 bits contained in the data 550 quantized into 3 bits; and third data 560-3, which includes the third bit of each parameter including 3 bits contained in the data 550 quantized into 3 bits.

[0131] The electronic device 100 may compress each of the obtained first to third data 560 - 1 to 560 - 3 by using a Viterbi decompressor.

[0132] In summary, the electronic device 100 can obtain the compressed index data 540, the compressed first data 570-1, the compressed second data 570-2, and the compressed third data 570-3 through the above-mentioned compression method.

[0133] Figure 6 and Figure 7 is a view for explaining a method for decompressing raw data according to an embodiment.

[0134] In detail, the processor 120 may obtain a plurality of compressed data from the memory 100 (eg, DRAM 701), and convert the n compressed second data (eg, Figure 7 The compressed quantization bit 1, compressed quantization bit 2, and compressed quantization bit 3 in the quantization bit 1 are input to a Viterbi decompressor (VD) (e.g., Figure 7 The processor 120 may be configured to decode the data output by the Viterbi decompressor and the index data (eg, Figure 7 The index data (or mask bit data) may be data generated by compressed index data. For example, the index data may be a result output by inputting the compressed index data to a Viterbi decompressor. The index data may be data for distinguishing between weight parameters whose values ​​have changed due to pruning and weight parameters whose values ​​have not changed according to pruning. That is, the electronic device 100 (for example, Figure 7 The mask 702 and the binary code-based calculation engine 703) can replace the value of the weight parameter, which has been changed according to the pruning, with zero by using the index data. In detail, the electronic device 100 can operate the output value and the index data and decompress the data.

[0135] In summary, the compressed data may be first decompressed by a Viterbi decompressor, and weight parameters whose values ​​in the data decompressed by the Viterbi decompressor are changed according to pruning may be replaced with 0 values ​​using mask bits, and data identical to the original data may be decompressed.

[0136] Figure 6 The circuit structure shown is a view showing an example method for decompressing compressed data in the case where quantization is performed to 1 bit in the compression process. That is, the circuit structure for decompressing original data may be different depending on the number of bits to be quantized in the compression process. For example, in the case where the original data is quantized to 3 bits, it may be as follows Figure 7 The processor 120 for decompressing the raw data is implemented as in FIG.

[0137] In the above-mentioned embodiment, the compression and decompression of data are performed by one electronic device, but it is not limited thereto. That is, the electronic device 100 may be any one of an electronic device for compressing original data and an electronic device for decompressing compressed data.

[0138] Figure 8 is a flowchart of a method for compressing data according to an embodiment.

[0139] First, in operation S810, the electronic device 100 may prune raw data including a plurality of weight parameters. The raw data may be data including weight parameters for deep learning. In detail, the raw data may be data for values ​​of magnitudes of the weight parameters.

[0140] In operation S820, the electronic device 100 may obtain index data including position information of weight parameters whose values ​​are not changed according to the pruning from the pruned original data. The electronic device 100 may compress the obtained index data by using a Viterbi algorithm.

[0141] In operation S830, the electronic device 100 may replace the weight parameters whose values ​​are changed according to the pruning among the pruned original data with irrelevant parameters.

[0142] In operation S840, the electronic device 100 may quantize the replaced data into n bits, and obtain n second data based on the data quantized into n bits.

[0143] In operation S850, the electronic device 100 may apply each of the obtained n second data to the Viterbi algorithm and obtain n compressed data. In detail, the electronic device 100 may confirm the input value capable of decompressing the weight parameter whose value is not changed according to pruning as compressed data by using the test data compression method.

[0144] The electronic device 100 may input the compressed data into a Viterbi decompressor and decompress the original data. In the case where there is an error between the decompressed data and the original data, the error may be corrected by relearning based on deep learning. The electronic device 100 may requantize the relearned data and perform the above-mentioned compression process.

[0145] Fig. 9 is a flow chart illustrating a method for decompressing data according to an embodiment.

[0146] In operation S910, the electronic device 100 may receive a signal according to a reference signal. Figure 8 The embodiment described includes n compressed second data and compressed index data.

[0147] In operation S920, the electronic device 100 may apply each of the n compressed second data to a Viterbi decompressor and obtain index data for decompressing the original data.

[0148] In operation S930, the electronic device 100 may apply each of the n compressed data to a Viterbi decompressor and obtain n second data for decompressing the original data.

[0149] In operation S940, the electronic device 100 may calculate index data for decompressing original data and n compressed data for decompressing original data, and obtain pruned original data.

[0150] Figure 8 and Fig. 9 The example is limited to the case where the electronic device compresses data and the electronic device decompresses data independently of each other. In this case, the electronic device for data compression may be a server with a large storage space and computing power, and the electronic device for data decompression may be a user terminal with a relatively small storage space and computing power compared to the server. However, data compression and decompression may be performed in one electronic device.

[0151] The present disclosure is not necessarily limited to these embodiments, because all elements constituting the embodiments of the present disclosure are described as being combined or operated in a kind of operation. That is, within the scope of the present disclosure, all elements can be selectively coupled to one or more of them. In addition, although all components can be implemented as an independent hardware, some or all components can be selectively combined and implemented as a computer program with a program module to perform a part or all of the functions in one or more hardware.

[0152] At least some of the apparatuses (e.g., modules or functions thereof) or methods (e.g., operations) according to the embodiments may be implemented as a program module format of instructions stored in a temporary or non-temporary computer-readable recording medium. When the instructions are executed by a processor (e.g., processor 120), the processor may perform a function corresponding to the instructions.

[0153] Here, the program may be stored in a non-transitory recording medium readable by a computer, and may be read and executed by a computer, thereby realizing the embodiments of the present disclosure.

[0154] Here, the non-transitory readable recording medium refers to a medium that stores data semi-permanently and can be read by a device, and includes a register, a cache, a buffer, etc., but does not include a transmission medium such as a signal, a current, etc.

[0155] In detail, the above-mentioned program can be stored and provided in a non-transitory computer-readable medium, such as a CD, a digital versatile disk (DVD), a hard disk, a Blu-ray disc, a USB, an internal memory, a memory card, a ROM, a RAM, etc.

[0156] Furthermore, the method according to the above-described embodiments may be provided as a computer program product.

[0157] A computer program product may include a software program, a computer-readable storage medium having a software program stored therein, or an item traded between a seller and a consumer.

[0158] For example, a computer program product may include an electronic device or a product (e.g., a downloadable application) in the form of a software program that is electronically distributed by a manufacturer of the electronic device or an electronic store (e.g., Google Play Store and App Store). For electronic distribution, at least some of the software programs may be stored in a storage medium or may be temporarily generated. In this case, the storage medium may be a server of the manufacturer, a server of the electronic market, or a relay server that temporarily stores the software programs.

[0159] While the present disclosure has been shown and described with reference to example embodiments of the inventive concept, it will be understood by those skilled in the art that various changes in form and details may be made therein without departing from the spirit and scope of the present disclosure as defined in the appended claims.

Claims

1. A data compression method of an electronic device for compressing weight parameters of an artificial intelligence model, the data compression method comprising: Pruning raw data including multiple weight parameters; identifying, from among the weight parameters included in the pruned original data, weight parameters whose values ​​have not changed according to the pruned data; Obtaining index data including position information of weight parameters whose values ​​have not changed; identifying, from among the weight parameters included in the pruned original data, weight parameters whose values ​​have been changed due to the pruned data; Replace weight parameters whose values ​​have changed with irrelevant parameters; quantizing the first data including the weight parameter whose value has not changed and the irrelevant parameter into n bits; Based on the quantized data quantized into n bits, obtaining n second data; Applying each of the obtained n second data to a Viterbi algorithm; as well as Get n compressed data.

2. The data compression method according to claim 1, wherein: The first data is in matrix form, and The quantized data quantized into n bits is data generated by quantizing the weight parameters whose values ​​are not changed and the irrelevant parameters included in the matrix.

3. The data compression method according to claim 2, wherein: The n second data include: 2-1 data, the 2-1 data including the first bit value of the weight parameter whose value has not changed, quantized into n bits, and the first bit of the irrelevant parameter, quantized into n bits; and 2-2 data, the 2-2 data including the second bit value of the weight parameter whose value has not changed and quantized into n bits and the second bit of the irrelevant parameter and quantized into n bits.

4. The data compression method according to claim 1, wherein: Obtaining the compressed data includes: comparing data output by inputting at least one compression candidate data as an input value to a Viterbi decompressor with the obtained n second data; and Based on the obtained n second data and the output data matching at a predetermined ratio or more, an input value for the output data is obtained as compressed data.

5. The data compression method according to claim 4, wherein: The comparing includes comparing data corresponding to only the weight parameter whose value has not changed among data included in each of the n second data with the output data.

6. The data compression method according to claim 1, wherein: Obtaining the index data includes: applying the index data to a Viterbi algorithm, and obtaining compressed index data.

7. The data compression method according to claim 6, wherein: Acquiring the index data includes: comparing data output by inputting at least one compression candidate data as an input value to a Viterbi decompressor with the index data; and Based on the index data and the output data matching at a predetermined ratio or more, an input value for the output data is obtained as compressed index data.

8. The data compression method according to claim 6, further comprising: The compressed index data, the n compressed data and the table data required for quantization are stored.

9. An electronic device for compressing weight parameters of an artificial intelligence model, the electronic device comprising: Memory; as well as A processor, the processor being configured to: Pruning raw data including multiple weight parameters; identifying, from among the weight parameters included in the pruned original data, weight parameters whose values ​​have not changed according to the pruned data; Obtaining index data including position information of weight parameters whose values ​​have not changed; identifying, from among the weight parameters included in the pruned original data, weight parameters whose values ​​have been changed due to the pruned data; Replace weight parameters whose values ​​have changed with irrelevant parameters; quantizing the first data including the weight parameter whose value has not changed and the irrelevant parameter into n bits; Based on the quantized data quantized into n bits, obtaining n second data; Applying each of the obtained n second data to a Viterbi algorithm; as well as Get n compressed data.

10. The electronic device according to claim 9, wherein: The first data is in matrix form, and The quantized data quantized into n bits is data generated by quantizing the weight parameters whose values ​​are not changed and the irrelevant parameters included in the matrix.

11. The electronic device according to claim 10, wherein: The n second data include: 2-1 data, the 2-1 data including the first bit value of the weight parameter whose value has not changed, quantized into n bits, and the first bit of the irrelevant parameter, quantized into n bits; and 2-2 data, the 2-2 data including the second bit value of the weight parameter whose value has not changed and quantized into n bits and the second bit of the irrelevant parameter and quantized into n bits.

12. The electronic device according to claim 9, wherein: The processor is configured to: comparing data output by inputting at least one compression candidate data as an input value to a Viterbi decompressor with the obtained n second data; as well as Based on the obtained n second data and the output data matching at a predetermined ratio or more, an input value for the output data is obtained as compressed data.

13. The electronic device according to claim 12, wherein: The processor is configured to compare, among the data included in each of the n second data, only data corresponding to the weight parameter whose value has not changed, with the output data.

14. The electronic device according to claim 9, wherein: The processor is configured to apply the index data to a Viterbi algorithm and obtain compressed index data.

15. The electronic device according to claim 14, wherein: The processor is configured to: comparing data output by inputting at least one compression candidate data as an input value to a Viterbi decompressor with the index data; as well as Based on the index data and the output data matching at a predetermined ratio or more, an input value for the output data is obtained as compressed index data.

Images