Method and apparatus for lightweighting autoencoder based on mutual information

Abstract

A method for lightweighting an autoencoder based on mutual information includes: a step in which a lightweighting device receives an autoencoder; a step in which the lightweighting device selects a layer to be pruned among the layers of the autoencoder; a step in which the lightweighting device calculates the importance of a link connected to the layer to be pruned based on mutual information; a step in which the lightweighting device prunes by removing a link with low importance among the links connected to the layer to be pruned; and a step in which the lightweighting device fine-tunes the pruned autoencoder.

Description

Technology Field

[0001] The technology described below relates to a method for lightweighting an autoencoder based on mutual information. Background Technology

[0002] Deep learning models with a Deep Neural Network (DNN) structure possess higher accuracy compared to models that perform only linear operations. An autoencoder is one of the deep learning models. An autoencoder is a type of unsupervised learning-based model. Autoencoders are trained to compress input data and then restore it. Autoencoders are utilized for tasks such as compressing images, removing noise, reconstructing images, and detecting anomalies. Prior art literature

[0003] Published Patent 10-2022-0099875 The problem to be solved

[0004] Deep learning models, including autoencoders, require significant resources. Consequently, deep learning models face difficulties in being deployed on IoT (Internet of Things) devices or mobile devices where computational resources, storage space, and available energy are limited. To address this, methods to lighten deep learning models are being developed. Methods for lightening deep learning models include quantizing the model or pruning, which removes unnecessary weights.

[0005] The technology described below aims to provide a method for lightweighting deep learning models based on mutual information. In particular, it aims to provide a lightweighting method specialized for autoencoders among deep learning models. means of solving the problem

[0006] The technology described below discloses a method for lightweighting a mutual information-based autoencoder to solve the aforementioned problem.

[0007] A method for lightweighting an autoencoder based on mutual information includes: a step in which a lightweighting device receives an autoencoder; a step in which the lightweighting device selects a layer to be pruned among the layers of the autoencoder; a step in which the lightweighting device calculates the importance of a link connected to the layer to be pruned based on mutual information; a step in which the lightweighting device prunes by removing a link with low importance among the links connected to the layer to be pruned; and a step in which the lightweighting device fine-tunes the pruned autoencoder.

[0008] Selecting the above pruning target layer includes selecting the layer furthest from the autoencoder's latent vector among the remaining layers, excluding the layer previously selected as the pruning target layer, as the pruning target layer. Effects of the invention

[0009] The technology described below can be used to lighten the autoencoder. In particular, the autoencoder can be efficiently lightened through a lightweighting method specialized for autoencoders among deep learning models. This enables the autoencoder to operate even on devices with limited computational resources, storage space, and available energy.

[0010] By utilizing the technology described below, a lightweight autoencoder can be used. Specifically, anomalies can be detected using a lightweight autoencoder.

[0011] The technology described below can detect anomalies in network traffic. Accordingly, by operating a lightweight autoencoder even on IoT devices or mobile devices that require low computational power, anomalies with excellent performance can be detected with low detection complexity. Brief explanation of the drawing

[0012] Figure 1 shows the overall process of a lightweighting device (100) lightweighting an autoencoder. FIG. 2 is a flowchart (200) of one embodiment of an autoencoder lightweighting method. FIG. 3 is one of the embodiments of an autoencoder. FIGS. 4 and FIGS. 5 show one of the embodiments of a predetermined sequence. FIG. 6 is a flowchart (300) of one embodiment of a method for detecting anomalies using a lightweight autoencoder. Figure 7 shows the anomaly score histograms of an autoencoder that is not lightweight and an autoencoder that is 50% lightweight. Figure 8 shows the results of evaluating the anomaly detection performance of the autoencoder according to the pruning ratio. Figure 9 shows the results of evaluating the performance after lightweighting the autoencoder by applying various pruning methods. FIG. 10 is a configuration of one of the embodiments of a lightweight device. Specific details for implementing the invention

[0013] The technology described below may be subject to various modifications and may have various embodiments. Specific embodiments of the technology described below may be described in the drawings of the specification. However, this is for the purpose of explaining the technology described below and is not intended to limit the technology described below to specific embodiments. Accordingly, it should be understood that all modifications, equivalents, and substitutions that fall within the spirit and scope of the technology described below are included in the technology described below.

[0014] In the terms used below, singular expressions should be understood to include plural expressions unless the context clearly indicates otherwise, and terms such as "includes" should be understood to mean that the described features, number, steps, actions, components, parts, or combinations thereof exist, and not to exclude the existence or addition of one or more other features, numbers, steps, actions, components, parts, or combinations thereof.

[0015] Before providing a detailed description of the drawings, it is to clarify that the classification of components in this specification is merely based on the primary function each component is responsible for. That is, two or more components described below may be combined into a single component, or a single component may be divided into two or more components based on more subdivided functions. Furthermore, each component described below may additionally perform some or all of the functions of other components in addition to its own primary function, and it is obvious that some of the primary functions of each component may be exclusively performed by other components.

[0016] Furthermore, in performing the method or operation method, each process constituting the method may occur differently from the specified order unless a specific order is clearly indicated in the context. That is, each process may occur in the same order as specified, may be performed substantially simultaneously, or may be performed in the reverse order.

[0018] The following describes the overall process of the autoencoder lightweighting device (hereinafter, lightweighting device) lightweighting the autoencoder.

[0019] Figure 1 shows the overall process of a lightweighting device (100) lightweighting an autoencoder.

[0020] The lightweighting device (100) can receive an autoencoder. The lightweighting device (100) can select a layer to be pruned among the layers of the autoencoder. The lightweighting device (100) can calculate the importance of the links connected to the layer to be pruned based on mutual information. The lightweighting device (100) can prune by removing links with low importance among the links connected to the layer to be pruned. The lightweighting device (100) can fine-tune the pruned autoencoder.

[0021] The lightweighting device (100) can determine whether all layers of the fine-tuned autoencoder have been selected once as pruning target layers. If it is determined that all layers of the fine-tuned autoencoder have not been selected once as pruning target layers, the lightweighting device (100) can perform the selection of pruning target layers, the calculation of the importance of the links, the pruning, and the fine-tuning again for the fine-tuned autoencoder.

[0022] Selecting the pruning target layer may include selecting the layer furthest from the autoencoder's latent vector among the remaining layers, excluding the layer previously selected as the pruning target layer.

[0024] The following explains in detail the method for lightweighting the autoencoder.

[0025] FIG. 2 is a flowchart (200) of one embodiment of an autoencoder lightweighting method.

[0026] The lightweight device can receive an autoencoder input (210).

[0027] One example of an autoencoder is shown in FIG. 3.

[0028] An autoencoder may include an input layer, a hidden layer, and an output layer. The input layer is input data ( It is the layer that receives ). A hidden layer can be a layer that connects the input layer and the output layer. The output layer is the output data ( It may be a layer that outputs ). The autoencoder of FIG. 3 has L layers of encoder and L layers of decoder, so it has a total of 2L layers.

[0029] Each layer of an autoencoder contains nodes. Nodes can be connected to nodes in the previous layer and nodes in the next layer. Links can connect nodes. Links have different weights ( Can have )

[0030] An autoencoder can be an unsupervised learning-based model. An autoencoder can be trained to output a value that is as similar as possible to the input value. Therefore, if the dimension of the input layer is N, the dimension of the output layer is also N.

[0031] An autoencoder can compress input values ​​and then restore them to be as similar as possible to the input values ​​based on the compressed values ​​to produce output values. Accordingly, the hidden layer can have a lower dimension than the input and output layers.

[0032] An autoencoder can consist of an encoder and a decoder. The encoder takes the input values ​​as a latent vector, It can be compressed into ). The decoder can restore the latent vector to the output value.

[0033] Latent vectors can contain information in which the features of the input data are compressed. Autoencoders aim to obtain latent vectors in which the features of the input data are well compressed.

[0034] The lightweighting device can select a pruning target layer among the layers of the autoencoder (220).

[0035] The lightweighting device performs pruning on only one layer at a time and then fine-tunes it. In other words, the layer targeted for pruning refers to the layer that is pruned when the lightweighting device performs a single pruning operation. Since pruning is performed on only one layer at a time, the pruning process must be repeated to prune all layers of the autoencoder.

[0036] The process of selecting pruning targets may include a step of selecting the layer furthest from the autoencoder's latent vector among the remaining layers, excluding the layer previously selected as a pruning target. The latent vector is a compressed version of the autoencoder's input data, in which the features of the input data are most compressed. Therefore, by performing pruning starting from the layer furthest from the latent vector, pruning can be carried out while preserving as much of the information contained in the latent vector—which includes important data—as possible.

[0037] When selecting the layer furthest from the autoencoder's latent vector as the layer to be pruned, the layer to be pruned can be selected according to a predetermined order.

[0038] FIGS. 4 and FIGS. 5 show one of the embodiments of a predetermined sequence.

[0039] In one embodiment, selecting the layer furthest from the latent vector of the autoencoder as the layer to be pruned may proceed in the order of first selecting all layers belonging to the encoder, and then selecting layers belonging to the decoder. That is, as shown in (a) of FIG. 4, the layer furthest from the latent vector among the layers belonging to the encoder can be selected in order to perform lightweighting first. Afterward, once all layers belonging to the encoder have been selected, the layer furthest from the latent vector among the layers belonging to the decoder can be selected in order to perform lightweighting.

[0040] In one embodiment, selecting the layer furthest from the potential vector of the autoencoder as the layer to be pruned may proceed in the order of first selecting all layers belonging to the decoder, and then selecting layers belonging to the encoder. That is, as shown in (b) of FIG. 4, the layer furthest from the potential vector among the layers belonging to the decoder can be selected in order to perform lightweighting first. Afterward, once all layers belonging to the decoder have been selected, the layer furthest from the potential vector among the layers belonging to the encoder can be selected in order to perform lightweighting.

[0041] In one embodiment, selecting the layer furthest from the latent vector of the autoencoder as the layer to be pruned may proceed in a sequence of alternately selecting layers belonging to the encoder and layers belonging to the decoder. That is, as in FIG. 5 (a), the encoder and the decoder can be alternately selected by first selecting layers belonging to the encoder and then selecting layers belonging to the decoder, repeating this process. Alternatively, as in FIG. 5 (b), the encoder and the decoder can be alternately selected by first selecting layers belonging to the decoder and then selecting layers belonging to the encoder, repeating this process.

[0042] The lightweight device can calculate the importance of the link connected to the pruning target layer based on mutual information (MI) (230).

[0043] Mutual information can be a concept used in general information theory. It can be used to measure the interdependence between two random variables. In other words, mutual information can indicate how correlated two variables are and how much they change together.

[0044] In an autoencoder, mutual information can be a value that quantifies the influence that nodes in the previous layer have on nodes in the next layer as a signal propagates through each layer. In other words, the higher the mutual information, the greater the importance of the weights connecting nodes between layers.

[0045] Mathematical Equation 1 is the formula used to calculate mutual information.

[0046]

[0047] In mathematical formula 1 Is In the nth layer It means the nth node. Is [ It can be expressed as ]. In Mathematical Formula 1 represents the total number of data. In Mathematical Formula 1 ( ; )Is The nth floor the nth node and The nth floor the nth node It refers to the mutual information between them.

[0048] The lightweighting device can calculate the importance of links only for the pruning target layer. In other words, after a single fine-tuning is completed, when calculating the importance of links again, the importance of links is not calculated for layers previously selected as pruning targets.

[0049] The lightweighting device can prune by removing links of low importance among the links connected to the pruning target layer (240).

[0050] Pruning can mean replacing the link's weight value with 0. This can disable the propagation of the signal along that path.

[0051] The lightweighting device can remove low-importance links based on the pruning ratio. The pruning ratio is a ratio used to determine how much to prune links. The pruning ratio can be set arbitrarily depending on the target accuracy. The pruning ratio can have a value between 0 and 1. The pruning ratio has a value of 1 if all links are pruned, and a value of 0 if no links are pruned. For example, if the pruning ratio is 0.7, links with an importance of 70% or less of the calculated importance can be removed. Therefore, based on the pruning ratio, only links with weights that have a large mutual information value can be activated.

[0052] The lightweighting device can select the link to be pruned based on whether the pruning target layer belongs to an encoder or a decoder.

[0053] In one embodiment, if the layer to be pruned belongs to an encoder, the link connecting the layer to be pruned and the subsequent layer may be the target of pruning. Alternatively, if the layer to be pruned belongs to a decoder, the link connecting the layer to be pruned and the previous layer may be the target of pruning.

[0054] The lightweight device can fine-tune the above-mentioned pruned autoencoder (250).

[0055] Fine-tuning can mean optimizing the parameters of an autoencoder while keeping the weights of the pruned parts at zero. Specifically, fine-tuning means keeping the parameters of the parts that were pruned in the previous step and have zero weights at zero, while adjusting the parameters of the remaining parts that were not pruned and therefore have non-zero weights using training data. For example, if the links connected to node 2 of the second layer and node 3 of the third layer were pruned in the previous step, the weights of those links can be kept at zero while adjusting the remaining parameters of the autoencoder using training data.

[0056] The lightweight device can determine whether all layers of the finely tuned autoencoder have been selected as pruning target layers at least once (260).

[0057] As mentioned above, the lightweighting device selects one layer to be pruned at a time to perform pruning and fine-tuning. Therefore, the above process must be repeated to ensure that all layers of the autoencoder are pruned. Consequently, the lightweighting device can determine whether every layer of the fine-tuned autoencoder has been selected as a layer to be pruned at least once. In this case, the device may not determine whether a layer associated with a latent vector has been selected as a layer to be pruned, because the link connected to the latent vector has been pruned previously.

[0058] If it is determined that all layers of the fine-tuned autoencoder have not been selected as pruning targets at least once, the lightweight device may calculate the importance of the link connected to the layer that has never been selected as a pruning target for the fine-tuned autoencoder, select the pruning target layer, prun, and fine-tune again (270).

[0059] That is, the aforementioned steps (220 to 250) can be performed again based on the finely tuned autoencoder. At this time, when selecting the pruning target layer (230) as described above, the remaining layers excluding the layers previously selected as pruning target layers can be selected as pruning target layers. That is, once a layer is selected as a pruning target layer, it is not selected as a pruning target layer again.

[0061] By using the aforementioned method, the autoencoder can be made lightweight. The lightweight autoencoder can be utilized in various fields such as data classification, data compression, data generation, data conversion, noise removal, and anomaly detection.

[0062] Below, we will examine in detail a method for detecting anomalies as one example of using a lightweight autoencoder.

[0063] FIG. 6 is a flowchart (300) of one embodiment of a method for detecting anomalies using a lightweight autoencoder.

[0064] The anomaly detection device can train an autoencoder using normal data as training data (310).

[0065] Normal data can be data measured in an anomaly-free state. In other words, an autoencoder can be trained based solely on data measured in an anomaly-free state.

[0066] The anomaly detection device can lighten the trained autoencoder (320).

[0067] To lighten the autoencoder, the autoencoder lightening method described above through FIGS. 1 to 5 can be used.

[0068] The anomaly detection device can input data to be analyzed into a lightweight autoencoder (330).

[0069] The data subject to analysis may refer to data measured from a target intended to detect abnormalities. In one embodiment, to detect abnormalities in a network, network traffic data may be measured and used as the data subject to analysis.

[0070] An anomaly detection device can detect whether there is an anomaly based on the output value of the autoencoder (340).

[0071] Since autoencoders are trained based solely on normal data, they fail to reconstruct abnormal data effectively when input. Therefore, if the difference between the input value and the output value is small, the data can be identified as normal; if the difference is large, the data can be identified as abnormal.

[0072] In one embodiment, an anomaly detection device can detect whether there is an anomaly by calculating an anomaly score based on the output value of an autoencoder. The anomaly score may include the degree of difference between the input value and the output value. The anomaly score may be a value calculated based on the Mean Absolute Error (MAE), Mean Squared Error (MSE), or cross entropy.

[0073] Mathematical Formula 2 is the outlier score based on Mean Absolute Error (MAE). This is the formula used when calculating.

[0074]

[0075] In mathematical formula 2 Is The nth input data It refers to the value of the i-th dimension. In mathematical formula 2 Is When the i-th input data is input into the autoencoder, the output value It refers to the value of the i-th dimension. In mathematical formula 2 represents the number of dimensions of the input data. In mathematical formula 2 Is It refers to the outlier score when the i-th input data is entered.

[0076] In mathematical formula 2 If is greater than the threshold, the data under analysis can be classified as abnormal data. Conversely, in Equation 2 If it is smaller than the threshold, the data under analysis can be classified as normal data.

[0078] The researchers lightweighted the autoencoder by applying the aforementioned mutual information-based autoencoder lightweighting method. The researchers conducted experiments to evaluate the performance of the lightweighted autoencoder. The encoder and decoder of the autoencoder used in the experiment each have three layers. The experimental results are examined below.

[0080] Figure 7 shows anomaly score histograms for an autoencoder that is not lightweight and an autoencoder that is 50% lightweight. Figure 7(a) shows the autoencoder that is not lightweight. Figure 7(b) shows the autoencoder that is 50% lightweight.

[0081] A non-lightweight autoencoder utilizes all information from the input data to construct the output data. Therefore, it can be confirmed that the outlier scores of normal data and abnormal data are easily distinguished.

[0082] In the case of the 50% lighter autoencoder, it can be seen that the distribution of normal data and the distribution of abnormal data are still easily distinguishable despite some loss of information.

[0083] In other words, it can be confirmed that there is no difficulty in detecting data anomalies even when applying the aforementioned autoencoder lightweighting method. This is because high-importance connections between nodes in different layers have been preserved while being pruned. In other words, this means that information unnecessary for anomaly detection has been excluded, while key information for distinguishing between normal and abnormal data has been retained.

[0085] Figure 8 shows the results of evaluating the anomaly detection performance of the autoencoder according to the pruning ratio.

[0086] It can be confirmed that the model (Proposed) pruned using the aforementioned mutual information-based autoencoder lightweighting method does not experience a significant decrease in accuracy or F1-score even as the rruning ratio increases. On the contrary, it can be observed that the F1-score is higher compared to the model before pruning (Dense autoencoder).

[0087] On the other hand, in the aforementioned mutual information-based autoencoder lightweighting method, it can be observed that the accuracy and F1-score values ​​of the model without fine-tuning (Pruned autoencoder without fine-tuning) decrease compared to the model before pruning (Dense autoencoder).

[0088] Through this, it can be confirmed that when the autoencoder is lightweighted using the aforementioned mutual information-based autoencoder lightweighting method, anomaly detection is possible with excellent performance while preserving the expressiveness of the weights.

[0090] Figure 9 shows the results of evaluating the performance after lightweighting the autoencoder by applying various pruning methods.

[0091] Figure 9 (a) shows the experimental results when pruning sequentially from the input layer to the output layer.

[0092] Figure 9(b) shows the experimental results when selecting the pruning target layer, where the layer belonging to the decoder is selected first, followed by the layer belonging to the encoder. In other words, it is the experimental result when the method of Figure 4(b) is applied.

[0093] Figure 9 (c) shows the experimental results when alternating between selecting layers belonging to the encoder and layers belonging to the decoder as pruning targets. In other words, it is the experimental result when the method of Figure 5 (a) is applied.

[0094] Figure 9 (d) shows the experimental result of pruning the entire layer at once without separate fine-tuning.

[0095] Comparing the experimental results of Fig. 9(a) with those of Figs. 9(b) and 9(c), it can be confirmed that the degree of performance reduction in the cases of Figs. 9(b) and 9(c) is relatively weak even when the pruning ratio increases.

[0096] Looking at the experimental results in Fig. 9(d), it can be seen that in the absence of a separate fine-tuning process, the performance is significantly worse than that of a non-lightweight autoencoder.

[0098] The following describes the lightweight device.

[0099] FIG. 10 is a configuration of one of the embodiments of a lightweight device.

[0100] The lightweighting device (400) may correspond to the lightweighting device (100) described in FIG. 1. The lightweighting device (400) may be a device that performs the aforementioned mutual information-based autoencoder lightweighting method. Furthermore, the lightweighting device (400) may perform the role of an anomaly detection device. That is, the lightweighting device (400) can perform an anomaly detection method using the lightweighted autoencoder after lightweighting the autoencoder. In other words, the description of the device below can be applied as is to the anomaly detection device.

[0101] The lightweight device (400) can be physically implemented in various forms. For example, the lightweight device (400) can take the form of a PC, laptop, smart device, server, or a chipset dedicated to data processing.

[0102] The lightweight device (400) may include an input device (410), a storage device (420), a computing device (430), an output device (440), an interface device (450), and a communication device (460).

[0103] The input device (410) may include an interface device (keyboard, mouse, touchscreen, etc.) for receiving certain commands or data. The input device (410) may include a configuration for receiving information through a separate storage device (USB, CD, hard disk, etc.). The input device (410) may receive input data through a separate measuring device or through a separate DB. The input device (410) may receive data via wired or wireless communication.

[0104] The input device (410) can receive information and models necessary for the mutual information-based autoencoder lightweighting method. The input device (410) can receive an autoencoder. The input device (410) can receive data to be analyzed and training data.

[0105] The storage device (420) can store information received through the input device (410). The storage device (420) can store information generated during the process of the computation device (430) performing calculations. That is, the storage device (420) may include memory. The storage device (420) can store the result calculated by the computation device (430).

[0106] The storage device (420) can store information and models required for the mutual information-based autoencoder lightweighting method. The storage device (420) can store the autoencoder.

[0107] The computing device (430) may be a device such as a processor, AP, or a chip with a program embedded therein that processes data and performs certain operations. The computing device (430) may generate a control signal that controls the lightweight device (400).

[0108] The computing device (430) can perform calculations required for the mutual information-based autoencoder lightweighting method.

[0109] The computing device (430) can select a layer to be pruned among the layers of the autoencoder. The computing device (430) can calculate the importance of the link connected to the layer to be pruned based on mutual information. The computing device (430) can prune by removing links with low importance among the links connected to the layer to be pruned. The computing device (430) can fine-tune the pruned autoencoder. The computing device (430) can determine whether all layers of the fine-tuned autoencoder have been selected once as a layer to be pruned. If it is determined that not all layers of the fine-tuned autoencoder have been selected once as a layer to be pruned, the computing device (430) can perform the calculation of the importance of the link, the selection of the layer to be pruned, the pruning, and the fine-tuning again for the fine-tuned autoencoder. The computing device (430) can input the data to be analyzed into a lightweight autoencoder and then detect whether there is an abnormality based on the output value of the autoencoder.

[0110] The output device (440) may be a device that outputs certain information. The output device (440) may output an interface required for a data process, input data, analysis results, etc. The output device (440) may be implemented in various physical forms, such as a display, a document output device, etc.

[0111] The interface device (450) may be a device that receives certain commands and data from the outside. The interface device (450) may receive information and models necessary for the mutual information-based autoencoder lightweighting method from a physically connected input device or an external storage device. The interface device (450) may receive control signals for controlling the lightweighting device (400). The interface device (450) may output the results analyzed by the lightweighting device (400).

[0112] The communication device (460) may refer to a configuration that receives and transmits certain information via a wired or wireless network. The communication device (460) may receive control signals necessary to control the lightweight device (400). The communication device (460) may transmit the results analyzed by the lightweight device (400).

[0114] The aforementioned method for lightweighting an autoencoder based on mutual information or the method for detecting anomalies using a lightweight autoencoder can be implemented as a program (or application) including an executable algorithm that can be executed on a computer.

[0115] The above program may be provided by storing it on a transitory or non-transitory computer-readable medium.

[0116] A non-transient readable medium refers to a medium that stores data semi-permanently and can be read by a device, rather than a medium that stores data for a short moment, such as a register, cache, or memory. Specifically, the various applications or programs described above may be stored and provided on a non-transient readable medium such as a CD, DVD, hard disk, Blu-ray disc, USB, memory card, ROM (Read-Only Memory), PROM (Programmable Read Only Memory), EPROM (Erasable PROM), EEPROM (Electrically EPROM), or flash memory.

[0117] Transient readable media refers to various types of RAM such as Static RAM (SRAM), Dynamic RAM (DRAM), Synchronous DRAM (SDRAM), Double Data Rate SDRAM (DDR SDRAM), Enhanced SDRAM (ESDRAM), Synclink DRAM (SLDRAM), and Direct Rambus RAM (DRRAM).

[0118] The embodiments and drawings attached to this specification merely clearly illustrate a part of the technical ideas included in the aforementioned technology, and it is self-evident that variations and specific embodiments that can be easily inferred by a person skilled in the art within the scope of the technical ideas included in the specification and drawings of the aforementioned technology are all included within the scope of the rights of the aforementioned technology.

Claims

Claim 1 A method for lightweighting an autoencoder based on mutual information, comprising: a step in which a lightweighting device receives an autoencoder; a step in which the lightweighting device selects a layer to be pruned among the layers of the autoencoder; a step in which the lightweighting device calculates the importance of a link connected to the layer to be pruned based on mutual information; a step in which the lightweighting device prunes by removing a link with low importance among the links connected to the layer to be pruned; and a step in which the lightweighting device fine-tunes the pruned autoencoder; wherein the selection of the layer to be pruned includes selecting the layer furthest from the latent vector of the autoencoder among the remaining layers excluding the layer previously selected as the layer to be pruned as the layer to be pruned. Claim 2 delete Claim 3 A mutual information-based autoencoder lightweighting method according to claim 1, further comprising: a step of determining whether all layers of the fine-tuned autoencoder have been selected once as pruning target layers; and, if it is determined that all layers of the fine-tuned autoencoder have not been selected once as pruning target layers, a step of the lightweighting device selecting the pruning target layers for the fine-tuned autoencoder, calculating the importance of the link, pruning, and fine-tuning again. Claim 4 A mutual information-based autoencoder lightweighting method according to claim 1, wherein the mutual information is a value calculated through the following mathematical formula 1. [Mathematical Formula 1] In the above mathematical formula 1 Is In the nth layer It refers to the nth node. The above Is [ It can be expressed as ]. In the above mathematical formula 1 represents the total number of data. In the above mathematical formula 1 ( ; )Is The nth floor the nth node and The nth floor the nth node It refers to the mutual information between them. Claim 5 A mutual information-based autoencoder lightweighting method according to claim 1, wherein the layer furthest from the latent vector of the autoencoder is selected as the layer to be pruned, in the order of first selecting all layers belonging to the encoder and then selecting layers belonging to the decoder. Claim 6 A mutual information-based autoencoder lightweighting method according to claim 1, wherein the layer furthest from the latent vector of the autoencoder is selected as the layer to be pruned, in the order of first selecting all layers belonging to the decoder and then selecting layers belonging to the encoder. Claim 7 A mutual information-based autoencoder lightweighting method according to claim 1, wherein the layer furthest from the latent vector of the autoencoder is selected as the layer to be pruned, which proceeds in the order of alternately selecting layers belonging to the encoder and layers belonging to the decoder. Claim 8 A mutual information-based autoencoder lightweighting method according to claim 1, wherein the pruning is to remove links of low importance according to the pruning ratio. Claim 9 An input device for receiving an autoencoder; a computing device for selecting a pruning target layer among the layers of the autoencoder, calculating the importance of a link connected to the pruning target layer based on mutual information, pruning by removing a link with low importance among the links connected to the pruning target layer, and fine-tuning the pruned autoencoder; and a storage device for storing the autoencoder; wherein the selection of the pruning target layer includes selecting the layer furthest from the autoencoder's latent vector among the remaining layers excluding the layer previously selected as the pruning target layer as the pruning target layer. Claim 10 delete Claim 11 In claim 9, the computing device determines whether all layers of the fine-tuned autoencoder have been selected once as pruning target layers, and if it determines that all layers of the fine-tuned autoencoder have not been selected once as pruning target layers, the lightweighting device performs again the selection of pruning target layers, calculation of the importance of the link, pruning, and fine-tuning with respect to the fine-tuned autoencoder. Claim 12 A mutual information-based autoencoder lightweighting device according to claim 9, wherein the mutual information is a value calculated through the following mathematical formula 1. [Mathematical Formula 1] In the above mathematical formula 1 Is In the nth layer It refers to the nth node. The above Is [ It can be expressed as ]. In the above mathematical formula 1 represents the total number of data. In the above mathematical formula 1 ( ; )Is The nth floor the nth node and The nth floor the nth node It refers to the mutual information between them. Claim 13 A mutual information-based autoencoder lightweighting device according to claim 9, wherein the layer furthest from the potential vector of the autoencoder is selected as the layer to be pruned, in the order of first selecting all layers belonging to the encoder and then selecting layers belonging to the decoder. Claim 14 A mutual information-based autoencoder lightweighting device according to claim 9, wherein the layer furthest from the potential vector of the autoencoder is selected as the layer to be pruned, in the order of first selecting all layers belonging to the decoder and then selecting layers belonging to the encoder. Claim 15 A mutual information-based autoencoder lightweighting device according to claim 9, wherein selecting the layer furthest from the potential vector of the autoencoder as the pruning target layer proceeds in the order of alternately selecting layers belonging to the encoder and layers belonging to the decoder. Claim 16 A mutual information-based autoencoder lightweighting device according to claim 9, wherein the pruning described above removes links of low importance according to the pruning ratio. Claim 17 A computer-readable recording medium storing a program for executing the mutual information-based autoencoder lightweighting method described in paragraph 1. Claim 18 A method for detecting an anomaly using a lightweight autoencoder, comprising: a step of inputting data to be analyzed into a lightweight autoencoder; and a step of the anomaly detection device detecting whether there is an anomaly based on the output value of the lightweight autoencoder, wherein the lightweight autoencoder is trained using only normal data as training data, and is a lightweight autoencoder using the mutual information-based autoencoder lightweighting method described in claim 1. Claim 19 In claim 18, a method for detecting anomalies using a lightweight autoencoder, wherein detecting whether an anomaly exists based on the output value of the autoencoder involves calculating an anomaly score and determining that an anomaly exists if the anomaly score is greater than or equal to a preset threshold. Claim 20 A method for compressing data, classifying data, generating data, converting data, or removing noise using an autoencoder that has been lightweighted through the mutual information-based autoencoder lightweighting method described in claim 1.

Images