Method and apparatus for lightening autoencoder on basis of mutual information

Abstract

This method for lightening an autoencoder on the basis of mutual information amount comprises steps in which a lightening apparatus: receives an autoencoder; selects, from among layers of the autoencoder, a layer to be pruned; calculates, on the basis of a mutual information amount, the importance of links connected to the layer to be pruned; removes a link having low importance from among the links connected to the layer to be pruned, and prunes same; and finely adjusts the pruned autoencoder.

Description

Method and device for lightweighting an autoencoder based on mutual information

[0001] The technique described below is a method for making an autoencoder lightweight based on mutual information.

[0002] Deep learning models with a deep neural network (DNN) structure exhibit higher accuracy than models that perform only linear operations. The autoencoder is one type of deep learning model. An unsupervised learning model, the autoencoder is trained to compress input data and then decompress it. Autoencoders are used in various applications, such as image compression, noise removal, image reconstruction, and anomaly detection.

[0003] [Prior Art Literature]

[0004] [Patent Document]

[0005] Publication Patent No. 10-2022-0099875

[0006] Deep learning models, including autoencoders, require significant resources. Consequently, they present challenges for deployment on Internet of Things (IoT) devices or mobile devices, which often have limited computational resources, storage space, and available energy. To address this, methods for reducing the weight of deep learning models are being developed. These methods include quantizing the model and pruning, which removes unnecessary weights.

[0007] The technique described below aims to provide a method for lightweighting deep learning models based on mutual information. Specifically, this method aims to provide a lightweighting method specifically tailored to autoencoders, a type of deep learning model.

[0008] The technology described below discloses a method for lightweighting an autoencoder based on mutual information to solve the aforementioned problem.

[0009] A method for reducing weight of an autoencoder based on mutual information includes: a step in which a weight reduction device receives an autoencoder as input; a step in which the weight reduction device selects a pruning target layer from among layers of the autoencoder; a step in which the weight reduction device calculates the importance of a link connected to the pruning target layer based on mutual information; a step in which the weight reduction device prunes by removing a link with a low importance from among the links connected to the pruning target layer; and a step in which the weight reduction device fine-tunes the pruned autoencoder. The selecting of the pruning target layer includes selecting a layer that is farthest from a latent vector of the autoencoder from among the remaining layers excluding a layer previously selected as a pruning target layer as the pruning target layer.

[0010] The above mutual information-based autoencoder weight reduction method may further include a step in which the weight reduction device determines whether all layers of the fine-tuned autoencoder have been selected as pruning target layers at least once; and a step in which the weight reduction device selects the pruning target layers for the fine-tuned autoencoder, calculates the importance of the link, performs the pruning, and performs the fine-tuning again when it determines that all layers of the fine-tuned autoencoder have not been selected as the pruning target layers at least once.

[0011] The techniques described below can be used to make autoencoders lighter. Specifically, a lightweight method specifically tailored to autoencoders among deep learning models can be used to efficiently reduce the weight of autoencoders. This allows autoencoders to operate on devices with limited computational resources, storage space, and available energy.

[0012] The techniques described below allow the use of lightweight autoencoders. Specifically, these lightweight autoencoders can be used to detect anomalies.

[0013] The technology described below can detect network traffic anomalies. By operating a lightweight autoencoder on IoT devices and mobile devices requiring minimal computation, it is possible to detect anomalies with high performance and low detection complexity.

[0014] Figure 1 shows the overall process of a lightweight device (100) to lighten an autoencoder.

[0015] Figure 2 is a flowchart (200) of one embodiment of an autoencoder lightweighting method.

[0016] Figure 3 is one of the embodiments of an autoencoder.

[0017] Figures 4 and 5 show one embodiment of a predetermined order.

[0018] Figure 6 is a flowchart (300) of one embodiment of a method for detecting anomalies using a lightweight autoencoder.

[0019] Figure 7 shows the anomaly score histograms of the non-lightweight autoencoder and the 50% lightweight autoencoder.

[0020] Figure 8 shows the results of evaluating the anomaly detection performance of an autoencoder according to the pruning ratio.

[0021] Figure 9 shows the results of evaluating the performance of an autoencoder after applying various pruning methods to make it lighter.

[0022] Fig. 10 is a configuration of one embodiment of a lightweight device.

[0023] The technology described below is susceptible to various modifications and embodiments. Specific embodiments of the technology described below may be illustrated in the drawings of the specification. However, these are intended to illustrate the technology described below and are not intended to limit the technology described below to any specific embodiments. Therefore, it should be understood that all modifications, equivalents, or alternatives that fall within the spirit and scope of the technology described below are encompassed by the technology described below.

[0024] In the terms used hereinafter, singular expressions should be understood to include plural expressions unless the context clearly dictates otherwise, and terms such as "comprises" should be understood to mean the presence of a described feature, number, step, operation, component, part, or combination thereof, but not to exclude the possibility of the presence or addition of one or more other features, numbers, steps, operations, components, parts, or combinations thereof.

[0025] Before going into a detailed description of the drawings, it should be made clear that the division of components in this specification is merely a division based on the main function of each component. That is, two or more components described below may be combined into one component, or one component may be further divided into two or more components with more detailed functions. In addition to its own main function, each component described below may additionally perform some or all of the functions of other components, and of course, some of the main functions of each component may be exclusively performed by other components.

[0026] Additionally, in performing a method or method of operation, each process constituting the method may occur in a different order than the stated order, unless the context clearly indicates a specific order. That is, each process may occur in the same order as the stated order, may be performed substantially simultaneously, or may be performed in the opposite order.

[0027]

[0028] The following describes the overall process of making an autoencoder lightweight by using an autoencoder lightweight device (hereinafter, lightweight device).

[0029] Figure 1 shows the overall process of a lightweight device (100) to lighten an autoencoder.

[0030] The lightweight device (100) can receive an autoencoder as input. The lightweight device (100) can select a pruning target layer from among the layers of the autoencoder. The lightweight device (100) can calculate the importance of a link connected to the pruning target layer based on mutual information. The lightweight device (100) can prune by removing links with low importance from among the links connected to the pruning target layer. The lightweight device (100) can fine-tune the pruned autoencoder.

[0031] The lightweight device (100) can determine whether all layers of the fine-tuned autoencoder have been selected as pruning target layers at least once. If it is determined that all layers of the fine-tuned autoencoder have not been selected as pruning target layers at least once, the lightweight device (100) can select the pruning target layers for the fine-tuned autoencoder, calculate the importance of the links, perform the pruning, and perform the fine-tuning again.

[0032] Selecting a pruning target layer may include selecting a layer that is farthest from the latent vector of the autoencoder among the remaining layers, excluding the layer previously selected as the pruning target layer, as the pruning target layer.

[0033] Below, we describe in detail the method for reducing the weight of the autoencoder.

[0034] Figure 2 is a flowchart (200) of one embodiment of an autoencoder lightweighting method.

[0035] The lightweight device can receive an autoencoder as input (210).

[0036] One embodiment of an autoencoder is shown in Fig. 3.

[0037] An autoencoder may include an input layer, a hidden layer, and an output layer. The input layer is a layer that receives input data (X). The hidden layer may be a layer that connects the input layer and the output layer. The output layer may output output data ( ) may be a layer that outputs the output. The autoencoder of Fig. 3 has L layers of encoders and L layers of decoders, for a total of 2L layers.

[0038] Each layer of an autoencoder contains nodes. Nodes can be connected to nodes in the previous layer and nodes in the subsequent layer. Links can connect nodes. Links have different weights ( ) can have

[0039] An autoencoder can be an unsupervised learning-based model. It can be trained to produce output values ​​that are as similar as possible to the input values. Therefore, if the input layer has dimension N, the output layer also has dimension N.

[0040] An autoencoder compresses an input value, then restores the compressed value to create an output value that is as similar as possible to the input value. Therefore, the hidden layer can have a lower dimensionality than the input and output layers.

[0041] An autoencoder can be composed of an encoder and a decoder. The encoder converts the input value into a latent vector (Z (L) ) can be compressed. The decoder can restore the latent vector as an output value.

[0042] A latent vector can contain compressed information about the features of the input data. The goal of an autoencoder is to obtain a latent vector that well-compresses the features of the input data.

[0043] The lightweight device can select a pruning target layer among the layers of the above autoencoder (220).

[0044] The pruning device performs pruning on only one layer at a time, followed by fine-tuning. In other words, the pruning target layer can be defined as the layer that is pruned when the pruning device performs a single pruning operation. Since pruning is performed on only one layer at a time, pruning must be repeated to cover all layers of the autoencoder.

[0045] Selecting a pruning target may involve selecting the layer farthest from the autoencoder's latent vector among the remaining layers, excluding the layer previously selected as the pruning target. The latent vector is a compressed representation of the autoencoder's input data, and contains the most compressed features. Therefore, by performing pruning starting from the layer furthest from the latent vector, the information contained in the latent vector, which contains important information, can be preserved as much as possible.

[0046] When selecting the layer furthest from the latent vector of the autoencoder as the pruning target layer, the pruning target layer can be selected in a predetermined order.

[0047] Figures 4 and 5 show one embodiment of a predetermined order.

[0048] In one embodiment, selecting the layer furthest from the latent vector of the autoencoder as the pruning target layer can be performed in the following order: first, selecting all layers belonging to the encoder, and then selecting layers belonging to the decoder. That is, as in Fig. 4 (a), the layer belonging to the encoder that is furthest from the latent vector can be sequentially selected to perform weight reduction first. After all layers belonging to the encoder are selected, the layer belonging to the decoder that is furthest from the latent vector can be sequentially selected to perform weight reduction.

[0049] In one embodiment, selecting the layer furthest from the latent vector of the autoencoder as the pruning target layer can be performed in the following order: first, selecting all layers belonging to the decoder, and then selecting layers belonging to the encoder. That is, as in Fig. 4 (b), the layer belonging to the decoder that is furthest from the latent vector can be sequentially selected to perform weight reduction first. After all layers belonging to the decoder are selected, the layer belonging to the encoder that is furthest from the latent vector can be sequentially selected to perform weight reduction.

[0050] In one embodiment, selecting the layer furthest from the latent vector of the autoencoder as the pruning target layer may be performed by alternately selecting layers belonging to the encoder and layers belonging to the decoder. That is, as in (a) of Fig. 5, the layer belonging to the encoder may be selected first, and then the layer belonging to the decoder may be repeatedly selected, and the encoder and decoder may be alternately selected. Alternatively, as in (b) of Fig. 5, the layer belonging to the decoder may be selected first, and then the layer belonging to the encoder may be repeatedly selected, and the encoder and decoder may be alternately selected.

[0051] The lightweight device can calculate the importance of the link connected to the pruning target layer based on mutual information (MI) (230).

[0052] Mutual information can be a concept used in general information theory. It can be used to measure the interdependence between two random variables. In other words, it can indicate the degree to which two variables are correlated and how much they change together.

[0053] In an autoencoder, mutual information can be a value that quantifies the influence that nodes in a previous layer have on nodes in the next layer as a signal propagates through each layer. In other words, the greater the mutual information, the more important the weights connecting nodes between layers may be.

[0054] Mathematical formula 1 is the formula used to calculate mutual information.

[0055] [Mathematical Formula 1]

[0056]

[0057] In mathematical equation 1 means the i-th node in the l-th layer. Is It can be expressed as . In mathematical formula 1, d means the total number of data. In mathematical formula 1, is the i-th node of the (l-1)th layer. The jth node of the lth layer It means the mutual information between them.

[0058] The lightweight device can only calculate link importance for the pruning target layer. That is, after a single fine-tuning operation, when calculating link importance, etc. again, the link importance is not calculated for layers previously selected as pruning target layers.

[0059] The lightweight device can prune by removing low-importance links among the links connected to the pruning target layer (240).

[0060] Pruning can mean replacing a link's weight value with 0, thereby disabling signal propagation along that path.

[0061] The pruning device can remove low-importance links based on a pruning ratio (η). The pruning ratio can be used to determine how much of a link to prune. The pruning ratio can be arbitrarily set depending on the target accuracy. The pruning ratio can have a value between 0 and 1. The pruning ratio has a value of 1 if all links are pruned, and a value of 0 if no links are pruned. For example, a pruning ratio of 0.7 can remove links with an importance less than 70% of the calculated importance. Therefore, only links with a weight with a large mutual information value can be activated, depending on the pruning ratio.

[0062] The lightweight device can select the link to be pruned depending on whether the pruning target layer belongs to the encoder or decoder.

[0063] In one embodiment, if the pruning target layer belongs to an encoder, the link connecting the pruning target layer and the subsequent layer may be the target of pruning. Alternatively, if the pruning target layer belongs to a decoder, the link connecting the pruning target layer and the previous layer may be the target of pruning.

[0064] The lightweight device can fine-tune the pruned autoencoder (250).

[0065] Fine-tuning can mean optimizing the parameters of an autoencoder while keeping the weights of the pruned portions as 0. Specifically, fine-tuning can mean keeping the parameters of the portions that were pruned in the previous step and had weights of 0 as 0, but adjusting the parameters of the remaining portions that were not pruned and had weights other than 0 using training data. For example, if the link connecting node 2 of the 2nd layer and node 3 of the 3rd layer was pruned in the previous step, the weights of the corresponding links can be kept as 0, while adjusting the remaining parameters of the autoencoder using training data.

[0066] The lightweight device can determine whether all layers of the fine-tuned autoencoder have been selected as pruning target layers at least once (260).

[0067] As described above, the pruning unit selects one target layer at a time and performs pruning and fine-tuning. Therefore, the above process must be repeated for all layers of the autoencoder to be pruned. Therefore, the pruning unit can determine whether all layers of the fine-tuned autoencoder have been selected as pruning targets. At this time, the layer associated with the latent vector may not be determined as a target layer for pruning. This is because the link connected to the latent vector has been previously pruned.

[0068] If it is determined that not all layers of the fine-tuned autoencoder have been selected as pruning target layers at least once, the lightweight device can calculate the importance of the link connected to the layer that has not been previously selected as a pruning target for the fine-tuned autoencoder, select the pruning target layer, perform the pruning, and perform the fine-tuning again (270).

[0069] That is, based on the fine-tuned autoencoder, the aforementioned steps (220 to 250) can be performed again. At this time, when selecting a pruning target layer as described above (230), the remaining layers, excluding the layers previously selected as pruning target layers, can be selected as pruning target layers. In other words, once a layer has been selected as a pruning target layer, it will not be selected as a pruning target layer again.

[0070] The aforementioned method can be used to make autoencoders more lightweight. These lightweight autoencoders can be used in a variety of fields, including data classification, data compression, data generation, data transformation, noise removal, and anomaly detection.

[0071] Below, we will look specifically at an example of how to detect anomalies using a lightweight autoencoder.

[0072] Figure 6 is a flowchart (300) of one embodiment of a method for detecting anomalies using a lightweight autoencoder.

[0073] The anomaly detection device can train an autoencoder using normal data as training data (310).

[0074] Normal data can be data measured under conditions without anomalies. In other words, an autoencoder can be trained only based on data measured under conditions without anomalies.

[0075] The anomaly detection device can lighten the autoencoder after training is completed (320).

[0076] In order to make the autoencoder lighter, the autoencoder lightweighting method described above through Figures 1 to 5 can be used.

[0077] The anomaly detection device can input analysis target data into a lightweight autoencoder (330).

[0078] The data to be analyzed can refer to data measured from a target for which anomalies are to be detected. For example, to detect network anomalies, network traffic data can be measured and used as the data to be analyzed.

[0079] The anomaly detection device can detect whether there is an anomaly based on the output value of the autoencoder (340).

[0080] Because autoencoders are trained solely on normal data, they struggle to reconstruct abnormal data when it's input. Therefore, if the difference between the input and output values ​​is small, the data is considered normal. If the difference is large, the data is considered abnormal.

[0081] In one embodiment, an anomaly detection device can detect anomalies by calculating an anomaly score based on the output of an autoencoder. The anomaly score may include the degree of difference between the input and output values. The anomaly score may be calculated based on the mean absolute error (MAE), mean squared error (MSE), or cross entropy.

[0082] Equation 2 is the mean absolute error (MAE) based outlier score e(d k ) is the formula used to calculate it.

[0083] [Mathematical Formula 2]

[0084]

[0085] In mathematical equation 2 means the value of the lth dimension of the kth input data. In mathematical expression 2, means the value of the lth dimension of the output value when the kth input data is input to the autoencoder. In Equation 2, n means the dimensionality of the input data. In Equation 2, e(d k ) means the outlier score when the kth input data is entered.

[0086] In mathematical expression 2, e(d k ) is greater than the threshold value, the data to be analyzed can be classified as abnormal data. Conversely, in Equation 2, e(d k ) is less than the threshold value, the data to be analyzed can be classified as normal data.

[0087] The researchers applied the aforementioned mutual information-based autoencoder weight reduction method to lighten the autoencoder. They conducted an experiment to evaluate the performance of the lightweight autoencoder. The encoder and decoder of the autoencoder used in the experiment each had three layers. The experimental results are discussed below.

[0088] Figure 7 shows the anomaly score histograms for a non-lightweight autoencoder and a 50% lightweight autoencoder. Figure 7(a) is for the non-lightweight autoencoder. Figure 7(b) is for the 50% lightweight autoencoder.

[0089] Non-lightweight autoencoders utilize all information in the input data to construct the output data. Therefore, it can be confirmed that the outlier scores of normal and abnormal data are easily distinguishable.

[0090] For the 50% lighter autoencoder, we can see that the distribution of normal data and abnormal data can still be easily distinguished despite some loss of information.

[0091] In other words, even when applying the aforementioned autoencoder lightweight method, we can confirm that there is no difficulty in detecting data anomalies. This is because high-importance connections between nodes across layers are preserved and pruned. This means that information unnecessary for anomaly detection is removed, while the core information that distinguishes normal and anomalous data is maintained.

[0092] Figure 8 shows the results of evaluating the anomaly detection performance of an autoencoder according to the pruning ratio.

[0093] Using the aforementioned mutual information-based autoencoder weighting method, the pruned model (Proposed) demonstrates that accuracy and F1-socre do not significantly decrease even as the pruning ratio increases. In fact, F1-socre is significantly higher than that of the pre-pruned model (Dense autoencoder).

[0094] On the other hand, in the aforementioned mutual information-based autoencoder lightweighting method, it can be confirmed that the accuracy and F1-socre values ​​of the model that does not perform fine-tuning (Pruned autoencoder without fine-tuning) are lower than those of the model before pruning (Dense autoencoder).

[0095] Through this, it can be confirmed that when the autoencoder is made lightweight through the mutual information-based autoencoder lightweighting method described above, it is possible to detect anomalies with excellent performance while preserving the expressive power of the weights.

[0096] Figure 9 shows the results of evaluating the performance of an autoencoder after applying various pruning methods to make it lighter.

[0097] Figure 9 (a) shows the experimental results when pruning is performed sequentially from the input layer to the output layer.

[0098] Figure 9 (b) shows the experimental results when, when selecting a pruning target layer, the layer belonging to the decoder is selected first, and then the layer belonging to the encoder is selected. In other words, this is the experimental result when the method (b) of Figure 4 is applied.

[0099] Figure 9 (c) shows the experimental results when selecting layers belonging to the encoder and layers belonging to the decoder alternately when selecting the pruning target layers. In other words, this is the experimental result when method (a) of Figure 5 is applied.

[0100] Figure 9 (d) shows the experimental results when the entire layer is pruned at once without any separate fine adjustment.

[0101] Comparing the experimental results of Fig. 9(a) with those of Figs. 9(b) and 9(c), it can be seen that the degree of performance decrease is relatively small in the cases of Figs. 9(b) and 9(c) even when the pruning ratio increases.

[0102] Looking at the experimental results of Fig. 9(d), we can see that the performance is significantly worse than that of a non-lightweight autoencoder without a separate fine-tuning process.

[0103] Below is a description of the lightweight device.

[0104] Fig. 10 is a configuration of one embodiment of a lightweight device.

[0105] The lightweight device (400) may correspond to the lightweight device (100) described in FIG. 1. The lightweight device (400) may be a device that performs the aforementioned mutual information-based autoencoder lightweighting method. Furthermore, the lightweight device (400) may function as an anomaly detection device. That is, the lightweight device (400) may lightweight an autoencoder and then perform an anomaly detection method using the lightweight autoencoder. In other words, the description of the device below may be directly applied to the anomaly detection device.

[0106] The lightweight device (400) can be physically implemented in various forms. For example, the lightweight device (400) can take the form of a PC, a laptop, a smart device, a server, or a data processing-dedicated chipset.

[0107] The lightweight device (400) may include an input device (410), a storage device (420), an operation device (430), an output device (440), an interface device (450), and a communication device (460).

[0108] The input device (410) may include an interface device (keyboard, mouse, touch screen, etc.) that receives a specific command or data. The input device (410) may also include a configuration that receives information through a separate storage device (USB, CD, hard disk, etc.). The input device (410) may receive the input data through a separate measuring device or a separate database. The input device (410) may receive data through wired or wireless communication.

[0109] The input device (410) can receive information and models required for a mutual information-based autoencoder lightweighting method. The input device (410) can receive an autoencoder. The input device (410) can receive analysis target data and training data.

[0110] The storage device (420) can store information input through the input device (410). The storage device (420) can store information generated during the calculation process of the calculation device (430). That is, the storage device (420) can include a memory. The storage device (420) can store the results calculated by the calculation device (430).

[0111] The storage device (420) can store information and models required for a mutual information-based autoencoder lightweighting method. The storage device (420) can store an autoencoder.

[0112] The computing device (430) may be a device such as a processor, AP, or chip embedded with a program that processes data and performs certain operations. The computing device (430) may generate a control signal that controls the lightweight device (400).

[0113] The calculation device (430) can perform calculations required for a mutual information-based autoencoder lightweighting method.

[0114] The computational unit (430) can select a pruning target layer among the layers of the autoencoder. The computational unit (430) can calculate the importance of a link connected to the pruning target layer based on the mutual information. The computational unit (430) can prune by removing links with low importance among the links connected to the pruning target layer. The computational unit (430) can fine-tune the pruned autoencoder. The computational unit (430) can determine whether all layers of the fine-tuned autoencoder have been selected as pruning target layers at least once. If it is determined that all layers of the fine-tuned autoencoder have not been selected as pruning target layers at least once, the computational unit (430) can re-perform the steps of calculating the importance of the link, selecting the pruning target layer, pruning, and fine-tuning for the fine-tuned autoencoder. The calculation device (430) can input the analysis target data into a lightweight autoencoder and then detect whether there is an abnormality based on the output value of the autoencoder.

[0115] The output device (440) may be a device that outputs certain information. The output device (440) may output interfaces required for data processing, input data, analysis results, etc. The output device (440) may be physically implemented in various forms, such as a display, a device that outputs documents, etc.

[0116] The interface device (450) may be a device that receives certain commands and data from an external source. The interface device (450) may receive information and models required for a mutual information-based autoencoder lightweighting method from a physically connected input device or an external storage device. The interface device (450) may receive control signals for controlling the lightweighting device (400). The interface device (450) may output the results analyzed by the lightweighting device (400).

[0117] The communication device (460) may refer to a configuration that receives and transmits certain information via a wired or wireless network. The communication device (460) may receive control signals necessary to control the lightweight device (400). The communication device (460) may transmit the results analyzed by the lightweight device (400).

[0118] The aforementioned mutual information-based autoencoder lightweighting method or the method of detecting anomalies using a lightweight autoencoder can be implemented as a program (or application) including an executable algorithm that can be executed by a computer.

[0119] The above program may be provided stored on a non-transitory computer readable medium.

[0120] A non-transitory readable medium refers to a medium that stores data semi-permanently and can be read by a device, rather than a medium that stores data for a short period of time, such as a register, cache, or memory. Specifically, the various applications or programs described above may be stored and provided on a non-transitory readable medium, such as a CD, DVD, hard disk, Blu-ray disk, USB, memory card, ROM (Read-Only Memory), PROM (Programmable Read Only Memory), EPROM (Erasable PROM), EEPROM (Electrically EPROM), or flash memory.

[0121] Temporarily readable media refers to various types of RAM, such as Static RAM (SRAM), Dynamic RAM (DRAM), Synchronous DRAM (SDRAM), Double Data Rate SDRAM (DDR SDRAM), Enhanced SDRAM (ESDRAM), Synchronous DRAM (Synclink DRAM, SLDRAM), and Direct Rambus RAM (DRRAM).

[0122] The present embodiment and the drawings attached to the present specification only clearly illustrate a part of the technical idea included in the above-described technology, and it will be obvious that all modified examples and specific embodiments that can be easily inferred by a person skilled in the art within the scope of the technical idea included in the specification and drawings of the above-described technology are included in the scope of the rights of the above-described technology.

Claims

1. A step in which a lightweight device receives an autoencoder as input; A step in which the lightweight device selects a layer to be pruned among the layers of the autoencoder; A step of calculating the importance of a link connected to the pruning target layer based on the mutual information amount by the lightweight device; A step of pruning by removing low-importance links from among the links connected to the pruning target layer by the lightweight device; and A method for reducing weight of an autoencoder based on mutual information, comprising: a step of fine-tuning the pruned autoencoder by the above-mentioned weight-reducing device; 2. In paragraph 1, A method for reducing weight of an autoencoder based on mutual information, wherein selecting the pruning target layer includes selecting the layer farthest from the latent vector of the autoencoder among the remaining layers excluding the layer previously selected as the pruning target layer as the pruning target layer.

3. In paragraph 1, A step of determining whether all layers of the fine-tuned autoencoder have been selected as pruning target layers once by the lightweight device; and A method for reducing weight of an autoencoder based on mutual information, further comprising: a step of the lightweight device selecting a layer to be reduced for pruning from the fine-tuned autoencoder, calculating the importance of the link, performing the pruning, and performing the fine-tuning again when it is determined that not all layers of the fine-tuned autoencoder have been selected as layers to be reduced for pruning at least once; 4. In paragraph 1, A method for reducing the weight of an autoencoder based on mutual information, wherein the mutual information is a value calculated using the following mathematical formula 1. [Mathematical Formula 1] In the above mathematical formula 1 means the i-th node in the l-th layer. Is can be expressed as . In the above mathematical expression 1, d means the total number of data. In the above mathematical expression 1, MI( ; ) is the i-th node of the (l-1)th layer. And the jth node of the lth layer It means the mutual information between them.

5. In paragraph 2, A method for reducing the weight of an autoencoder based on mutual information, wherein the layer most distant from the latent vector of the above autoencoder is selected as the pruning target layer by first selecting all layers belonging to the encoder and then selecting layers belonging to the decoder.

6. In paragraph 2, A method for reducing the weight of an autoencoder based on mutual information, wherein the layer most distant from the latent vector of the above autoencoder is selected as the pruning target layer by first selecting all layers belonging to the decoder and then selecting layers belonging to the encoder.

7. In paragraph 2, A method for reducing the weight of an autoencoder based on mutual information, wherein the layer most distant from the latent vector of the above autoencoder is selected as the pruning target layer by alternately selecting layers belonging to the encoder and layers belonging to the decoder.

8. In paragraph 1, A method for lightweighting an autoencoder based on mutual information, wherein the above pruning removes links with low importance according to a pruning ratio.

9. Input device that receives autoencoder input; A computational device that selects a pruning target layer from among the layers of an autoencoder, calculates the importance of a link connected to the pruning target layer based on mutual information, prunes by removing low-importance links from among the links connected to the pruning target layer, and fine-tunes the pruned autoencoder; and A lightweight autoencoder device based on mutual information, comprising a storage device for storing the above autoencoder.

10. In paragraph 9, A lightweight autoencoder based on mutual information, wherein selecting the pruning target layer includes selecting the layer farthest from the latent vector of the autoencoder among the remaining layers excluding the layer previously selected as the pruning target layer as the pruning target layer.

11. In paragraph 9, The above-mentioned calculating device determines whether all layers of the fine-tuned autoencoder have been selected as pruning target layers at least once, and if it determines that all layers of the fine-tuned autoencoder have not been selected as pruning target layers at least once, the lightweight device selects the pruning target layers for the fine-tuned autoencoder, calculates the importance of the link, performs the pruning, and performs the fine-tuning again. A mutual information-based autoencoder lightweight device.

12. In paragraph 9, A lightweight autoencoder device based on mutual information, wherein the mutual information is a value calculated using the following mathematical formula 1. [Mathematical Formula 1] In the above mathematical formula 1 means the i-th node in the l-th layer. Is It can be expressed as . In the above mathematical expression 1, d means the total number of data. In the above mathematical expression 1, is the i-th node of the (l-1)th layer. And the jth node of the lth layer It means the mutual information between them.

13. In paragraph 10, A lightweight autoencoder device based on mutual information, wherein selecting the layer farthest from the latent vector of the above autoencoder as the pruning target layer is performed in the order of first selecting all layers belonging to the encoder and then selecting layers belonging to the decoder.

14. In paragraph 10, A lightweight autoencoder device based on mutual information, wherein selecting the layer farthest from the latent vector of the above autoencoder as the pruning target layer is performed in the order of first selecting all layers belonging to the decoder and then selecting layers belonging to the encoder.

15. In paragraph 10, A lightweight autoencoder device based on mutual information, wherein selecting the layer furthest from the latent vector of the above autoencoder as the pruning target layer is performed in an order in which layers belonging to the encoder and layers belonging to the decoder are selected alternately.

16. In paragraph 9, A lightweight autoencoder based on mutual information, wherein the above pruning removes links with low importance according to a pruning ratio.

17. A computer-readable recording medium recording a program for executing the method for reducing the weight of an autoencoder based on mutual information as described in Article 1.

18. A step of inputting the analysis target data into a lightweight autoencoder with an anomaly detection device; and A step of detecting whether there is an abnormality based on the output value of the lightweight autoencoder by the above-mentioned abnormality detection device; The above lightweight autoencoder is trained using only normal data as training data, and then a method for detecting anomalies using a lightweight autoencoder, which is a lightweight autoencoder using the mutual information-based autoencoder lightweighting method described in the first paragraph.

19. In paragraph 18, A method for detecting anomalies using a lightweight autoencoder, wherein detecting anomalies based on the output values ​​of the above autoencoder involves calculating an anomaly score and determining that an anomaly exists if the anomaly score is greater than a preset threshold.

20. A method for compressing data, classifying data, generating data, converting data, or removing noise using a lightweight autoencoder using the mutual information-based autoencoder lightweighting method described in Article 1.

Images