Method and electronic device for light-weighting deep learning model through channel pruning

Abstract

Disclosed is a method for light-weighting a deep learning model by repeating the steps of: obtaining a training data set including a hard negative sample; training the deep learning model by using the training data set as an input; determining the importance of a plurality of channels corresponding to respective layers of the trained deep learning model; and removing selected channels of the plurality of channels on the basis of the determined importance.

Description

Method for lightweighting a deep learning model through channel pruning and electronic device

[0001] This relates to a method for lightweighting a deep learning model through channel pruning and an electronic device.

[0002] Model pruning is a technique that makes artificial intelligence models smaller and more efficient by removing unimportant or unnecessary parameters, such as channels and weights. Through model pruning, model size can be reduced and inference speed improved. A lightweight model can then be utilized in a variety of environments.

[0003] According to one embodiment of the present disclosure, a method for lightweighting a deep learning model is provided. The method for lightweighting a deep learning model includes the step of (a) acquiring a training data set containing hard negative samples. The method for lightweighting a deep learning model includes the step of (b) training a deep learning model using the training data set as input. The method for lightweighting a deep learning model includes the step of (c) determining the importance of a plurality of channels corresponding to each layer of the trained deep learning model. The method for lightweighting a deep learning model includes the step of (d) removing a selected channel among the plurality of channels based on the determined importance. The method for lightweighting a deep learning model includes the step of repeating steps (a) through (d) until a predetermined criterion is satisfied.

[0004] According to one embodiment of the present disclosure, a computer-readable recording medium is provided on which a program for executing the above-described method is recorded.

[0005] According to one embodiment of the present disclosure, an electronic device for lightweighting a deep learning model is provided. The electronic device for lightweighting a deep learning model includes at least one processor comprising a memory including one or more storage media for storing instructions and a processing circuit. By executing one or more instructions individually or jointly, the electronic device obtains a training data set including hard negative samples, trains a deep learning model using the training data set as input, determines the importance of a plurality of channels corresponding to each layer of the trained deep learning model, and repeats the process of removing a selected channel among the plurality of channels based on the determined importance until a predetermined criterion is satisfied.

[0006] FIG. 1 is a drawing for explaining an electronic device that lightweights a deep learning model according to one embodiment of the present disclosure and a user device that executes the deep learning model to provide a service.

[0007] FIG. 2 is a drawing for explaining the configuration and operation of an electronic device and a user device according to one embodiment of the present disclosure.

[0008] FIG. 3 is a diagram illustrating the process of a processor of an electronic device according to one embodiment of the present disclosure performing a channel pruning process to lighten a deep learning model.

[0009] FIG. 4 is a drawing for illustrating a channel pruning process according to one embodiment of the present disclosure.

[0010] FIG. 5 is a diagram illustrating a method for estimating channel importance according to one embodiment of the present disclosure.

[0011] FIG. 6 is a diagram illustrating a process of repeatedly performing a channel pruning process according to one embodiment of the present disclosure.

[0012] FIG. 7 is a diagram illustrating a step of restoring a channel removed in a channel pruning process according to one embodiment of the present disclosure.

[0013] FIG. 8 is a diagram illustrating a channel pruning process using different training data sets corresponding to different hard negative samples based on different anchors according to one embodiment of the present disclosure.

[0014] FIG. 9 is a flowchart illustrating a method for lightweighting a deep learning model according to one embodiment of the present disclosure.

[0015] The terms used in this specification will be briefly explained, and the present disclosure will be described in detail. In the present disclosure, the expression "at least one of a, b, or c" may refer to "a," "b," "c," "a and b," "a and c," "b and c," "all of a, b, and c," or variations thereof.

[0016] The terms used in this disclosure have been selected to be as widely used and general as possible, taking into account their functions within this disclosure; however, these terms may vary depending on the intent of those skilled in the art, case law, the emergence of new technologies, etc. Additionally, in specific cases, terms have been selected at the applicant's discretion, and in such cases, their meanings will be described in detail in the relevant explanatory sections. Therefore, terms used in this disclosure should be defined not merely by their names, but based on their meanings and the overall content of this disclosure.

[0017] Singular expressions may include plural expressions unless the context clearly indicates otherwise. Terms used herein, including technical or scientific terms, may have the same meaning as generally understood by those skilled in the art as described in this specification. Additionally, terms including ordinal numbers, such as "first" or "second," used in this specification may be used to describe various components, but said components should not be limited by said terms. Such terms are used solely for the purpose of distinguishing one component from another.

[0018] When a part of a specification is described as "comprising" a certain component, this means that, unless specifically stated otherwise, it does not exclude other components but may include additional components. Furthermore, terms such as "part" or "module" as used in the specification refer to a unit that processes at least one function or operation, and this may be implemented in hardware or software, or as a combination of hardware and software.

[0019] Functions related to artificial intelligence according to the present disclosure are operated through a processor and memory. The processor may be composed of one or more processors. In this case, the one or more processors may be general-purpose processors such as CPUs, APs, and DSPs (Digital Signal Processors), graphics-dedicated processors such as GPUs and VPUs (Vision Processing Units), or artificial intelligence-dedicated processors such as NPUs. The one or more processors control the processing of input data according to predefined operation rules or artificial intelligence models stored in memory. Alternatively, if the one or more processors are artificial intelligence-dedicated processors, the artificial intelligence-dedicated processors may be designed with a hardware structure specialized for processing a specific artificial intelligence model.

[0020] The predefined rules of operation or artificial intelligence models are characterized by being created through learning. Here, being created through learning means that a predefined rules of operation or artificial intelligence models configured to perform desired characteristics (or objectives) are created by a basic artificial intelligence model being trained using multiple learning data by a learning algorithm. Such learning may be performed on the device itself where the artificial intelligence according to the present disclosure is executed, or it may be performed through a separate server and / or system. Examples of learning algorithms include supervised learning, unsupervised learning, semi-supervised learning, or reinforcement learning, but are not limited to the examples described above.

[0021] An artificial intelligence model includes a deep learning model composed of multiple neural network layers. Each of the multiple neural network layers has multiple weight values ​​and performs neural network operations through operations between the results of previous layers and the multiple weights. The multiple weights possessed by the multiple neural network layers can be optimized based on the learning results of the artificial intelligence model. For example, the multiple weights can be updated so that the loss value or cost value obtained from the artificial intelligence model during the learning process is reduced or minimized.

[0022] Below, with reference to the attached drawings, embodiments of the present disclosure are described in detail so that those skilled in the art can easily implement the present invention. However, the present disclosure may be embodied in various different forms and is not limited to the embodiments described herein.

[0023] The present disclosure will be described in detail below with reference to the attached drawings.

[0024] FIG. 1 is a drawing for explaining an electronic device (100) that lightens a deep learning model according to one embodiment of the present disclosure and a user device (200) that executes the deep learning model and provides a service.

[0025] The electronic device (100) (hereinafter, electronic device (100)) for lightweighting a deep learning model may be a server as shown in FIG. 1, but is not limited thereto. The electronic device (100) is a device having computing power and resources of higher performance than the user device (200), and may be an edge device or PC including a high-performance processor as well as a server. Depending on the situation, a mobile device such as a smartphone may also be the electronic device (100) for lightweighting a deep learning model according to one embodiment of the present disclosure.

[0026] A user device (200) is a device that provides a specific service through computational processing using a deep learning model. As illustrated in FIG. 1, the user device (200) may be a TV, refrigerator, microwave oven, door lock, air purifier, robot vacuum cleaner, washing machine, etc. A user device (200) that provides AI services using a deep learning model may provide recommendation services, assistant services, QA (Question Answering) services, etc. by using a deep learning model that takes user data as input. For example, if there is an action being performed by the user, the user device (200) may recommend content related to the action being performed through a recommendation service, or provide information related to relevant past experiences or future events through an assistant service. The user device (200) may provide user-customized answers based on a personalized database to the user's questions through a QA service.

[0027] The user device (200) may be equipped with a neural network model related to the service. For example, the user device may be equipped with at least one deep learning model such as a Convolutional Neural Network (CNN), Graph Convolutional Neural Network (GCN), Graph Neural Network (GNN), Deep Neural Network (DNN), Recurrent Neural Network (RNN), and Bidirectional Recurrent Deep Neural Network (BRDNN), and may use them in combination.

[0028] The user device (200) in an on-device environment may be equipped with a mobile / embedded processor with low computational power and minimize unnecessary power consumption through a low-power hardware design. Additionally, due to limitations in memory or storage space, it is desirable for the user device (200) to be equipped with a model optimized for an environment where the model size, computational amount, and memory usage are all limited.

[0029] In the limited environment of the user device (200), it is difficult to perform the task of lightweighting (or optimizing) the deep learning model on the user device (200). The task of lightweighting the deep learning model may include a process of evaluating the prediction performance of the model using training data and evaluating the importance of each channel to remove some channels. In addition, to compensate for the degradation of the model's performance, it may include fine-tuning using training data to adjust the remaining channels to have optimal weights.

[0030] As illustrated in FIG. 1, a user device (200) can transmit training data to an electronic device (100). The training data may be training data samples in the form of triples (anchor, positive sample, negative sample). An anchor is data that serves as a comparison standard, positive samples refer to data that belongs to the same class as the anchor or is semantically similar, and negative samples refer to data that belongs to a different class from the anchor or is semantically unrelated. Among negative samples, samples that actually belong to a different class from the anchor but appear similar to the anchor in the embedding space are called hard negative samples. While hard negative samples are difficult for the model to distinguish, they can be samples that induce the most discriminative learning. Therefore, by selectively utilizing hard negative samples, the deep learning model can be induced to effectively learn subtle differences between classes. Training of a deep learning model using hard negative samples can contribute to improving the performance of the deep learning model in various application fields such as speech recognition, image search, object detection, and recommendation systems.

[0031] For example, if the user device (200) has a face recognition function, it can form a triple with a face photo registered by the user (anchor), another face photo of the user (positive sample), and another person's face photo (negative sample). If the user device (200) has a voice recognition function, it can form a triple with a voice registered by the user to the user device (200) (anchor), another voice of the user (positive sample), and another person's voice (negative sample). The user device (200) may also transmit a pair consisting of an anchor and a negative sample, or just the anchor, to the electronic device (100) as training data. Based on the received anchor, the electronic device (100) can obtain positive samples and negative samples using a training data database or a user database stored in the electronic device (100).

[0032] The electronic device (100) can perform a task for lightweighting a model using training data received from a user device (200). After performing the task for lightweighting a model, the electronic device (100) can transmit the lightweighted model or information regarding the lightweighting of the lightweighted model, for example, identification information of channels removed through channel pruning, to the user device (200). Below, a method for lightweighting a deep learning model running on the user device (100) using the electronic device (100) will be described in detail.

[0033] FIG. 2 is a drawing for explaining the configuration and operation of an electronic device (100) and a user device (200) according to one embodiment of the present disclosure.

[0034] Referring to FIG. 2, the electronic device (100) may include a memory (110), a processor (120), a communication unit (130), and a storage (140), but is not limited thereto, and additional general configurations may be added.

[0035] A memory (110) of an electronic device (100) according to one embodiment may store a program for processing and controlling a processor (120), and may store data and information that is input to or generated from the electronic device (100). The memory (110) may store instructions, data structures, and program code that can be read by the processor (120). Operations performed by the processor (120) may be implemented by executing instructions or program code stored in the memory (110).

[0036] The memory (110) may include a flash memory type, ROM (Read-Only Memory), EEPROM (Electrically Erasable Programmable Read-Only Memory), PROM (Programmable Read-Only Memory), RAM (Random Access Memory), or SRAM (Static Random Access Memory), etc.

[0037] The memory (110) can store one or more instructions and / or programs that control the electronic device (100) to train a deep learning model or to lighten a deep learning model.

[0038] A processor (120) according to one embodiment can control the operation or function of an electronic device (100) so that it can perform a task by executing instructions or programmed software modules stored in memory (110). The processor (120) may be composed of hardware components that perform arithmetic, logic, and input / output operations and signal processing. The processor (120) can control the overall operation of the electronic device (100) by executing one or more instructions stored in memory (110). The processor (120) can control the communication unit (130) and storage (140) by executing programs stored in memory (110).

[0039] The processor (120) may include various processing circuits and / or multiple processors. For example, the term “processor” as used herein, including in the claim, may include various processing circuits including at least one processor. “At least one processor” may be configured to perform the various functions described herein, individually and / or collectively. As used herein, “processor,” “at least one processor,” and “one or more processors” may be configured to perform various functions. However, these terms cover, without limitation, situations where one processor performs some of the functions and other processor(s) perform other parts of the functions, and situations where a single processor can perform all functions. Additionally, “at least one processor” may include a combination of processors performing various functions of the disclosed functions in a distributed manner. “At least one processor” may execute program instructions to achieve or perform various functions.

[0040] A processor (120) according to one embodiment may be composed of at least one of, for example, a Central Processing Unit, a microprocessor, a Graphic Processing Unit, ASICs (Application Specific Integrated Circuits), DSPs (Digital Signal Processors), DSPDs (Digital Signal Processing Devices), PLDs (Programmable Logic Devices), FPGAs (Field Programmable Gate Arrays), an Application Processor, a Neural Processing Unit, or an AI-dedicated processor designed with a hardware structure specialized for processing AI models, but is not limited thereto. Each processor constituting the processor (120) may be a dedicated processor for performing a predetermined function.

[0041] One or more processors (120) may be implemented in the form of an integrated system-on-chip (SoC) including one or more electronic components. Each processor (120) may be implemented as separate hardware (H / W). A processor (120) may be implemented as a single-core processor or as a multi-core processor.

[0042] A processor (120) according to one embodiment can perform computation and control for processing a task set to perform lightweighting of a deep learning model. The processor (120) may be manufactured in the form of a dedicated hardware chip for artificial intelligence (AI), or may be manufactured as part of a general-purpose processor (e.g., CPU or application processor) or a graphics-dedicated processor (e.g., GPU) and mounted on an electronic device (100).

[0043] The communication unit (130) can perform data exchange between various devices within the electronic device (100) (internal communication) and data exchange with the outside (external communication). The communication unit (130) supports various communication protocols and can perform communication through wired or wireless networks. The communication unit (130) may include short-range wireless communicators, mobile communication devices, etc., but is not limited thereto.

[0044] The communication unit (130) of the electronic device (100) can perform data transmission and reception with the communication unit (230) of the user device (200) according to a predetermined protocol. For example, the electronic device (100) can communicate with the user device (200) through a wireless network such as LTE, 5G, WiFi, or Bluetooth.

[0045] Storage (140) can store a training database or a user database. The processor (120) can obtain a training data set from the training database that can be used for lightweighting a deep learning model. The processor (120) can obtain user information used in the electronic device (100) from the user database.

[0046] The user device (200) may include a memory (210), a processor (220), a communication unit (230), a sensor unit (240), and an input / output unit (250), but is not limited thereto, and additional general configurations may be added.

[0047] A memory (210) of a user device (200) according to one embodiment may store a program for processing and controlling a processor (220), and may store data and information that is input to or generated from the user device (200). The memory (210) may store instructions, data structures, and program code that can be read by the processor (220). Operations performed by the processor (220) may be implemented by executing instructions or program code stored in the memory (210).

[0048] A memory (210) according to one embodiment may include a flash memory type, a hard disk type, a multimedia card micro type, a card type memory (e.g., SD or XD memory, etc.), a non-volatile memory including at least one of ROM (Read-Only Memory), EEPROM (Electrically Erasable Programmable Read-Only Memory), PROM (Programmable Read-Only Memory), magnetic memory, a magnetic disk, and an optical disk, and a volatile memory such as RAM (Random Access Memory) or SRAM (Static Random Access Memory).

[0049] A memory (210) according to one embodiment may store one or more instructions and / or programs that control a user device (200) to use a deep learning model.

[0050] A processor (220) according to one embodiment can control operations or functions so that a user device (200) can perform tasks by executing instructions or programmed software modules stored in memory (210). The processor (220) may be composed of hardware components that perform arithmetic, logic, and input / output operations and signal processing. The processor (220) can control the overall operations of the user device (200) by executing one or more instructions stored in memory (210). The processor (220) can control a sensor unit (230) including at least one sensor, a communication unit (240), and an input / output device (250) by executing programs stored in memory (210).

[0051] The communication unit (230) can perform data exchange between various electronic devices inside the user device (200) (internal communication) and data exchange with the outside (external communication). The communication unit (230) can perform internal communication through various wired networks such as CAN (Controller Area Network), LIN (Local Interconnect Network), FlexRay, Ethernet, and MOST (Media Oriented System Transport). The communication unit (230) can perform external communication through wireless networks such as LTE, 5G, DSRC (Dedicated Short-Range Communications), WiFi, and Bluetooth.

[0052] The sensor unit (240) can detect the state of the user device (200) and the surrounding environment in real time. The sensor unit (240) may include various types of sensors to support the function or convenience of the user device (200). For example, the sensor unit (240) may include an image sensor such as a camera, a fingerprint recognition sensor, a temperature / humidity sensor, a pressure sensor, a position sensor, a speed sensor, an accelerometer / gyroscope sensor, etc. The sensor unit (240) may include an infrared sensor, an ultrasonic sensor, a vibration sensor, etc. as a motion / motion sensor capable of detecting predefined movements or motions.

[0053] The input / output device (250) is a device for interacting with the user of the user device (200). For example, the input / output device (250) may include an input device that receives various types of input, such as the user's voice, touch, or button operation, and an output device that provides content and information. The input / output device (250) may include an input interface (e.g., a touch screen, keyboard, touchpad, physical button, microphone, etc.) for receiving commands or information from the user. The input / output device (250) may include an output interface (e.g., a display panel, speaker, etc.) for outputting the result of an operation according to the user's command, or status information, operation information, warning messages, etc. of the user device (200).

[0054] In order for the user device (200) to provide a specific function, if it requires the user to register their face or voice, it can take a picture of the user's face with a camera or acquire the user's voice through a microphone via the sensor unit (240) and the input / output device (250).

[0055] When there is a request from a user device (200) to lighten a deep learning model, the following operations may be performed in the electronic device (100) by the configurations described above.

[0056] The electronic device (100) may receive information regarding a deep learning model to be lightweighted from the user device (200). The electronic device (100) may store a deep learning model identical to the deep learning model running on the user device (200) or receive a deep learning model from the user device (200).

[0057] The electronic device (100) can acquire a training data set based on a training data sample received from a user device (200) and train a deep learning model using the training data set as input. The electronic device (100) can determine the importance of multiple channels corresponding to each layer of the trained deep learning model. Based on the determined importance, the electronic device (100) can remove selected channels among the multiple channels. The electronic device (100) can repeat this channel pruning process until a predetermined criterion is satisfied.

[0058] An electronic device (100) according to an embodiment of the present disclosure may use a training data set containing 'hard negative samples' in a channel pruning process. In the present disclosure, a hard negative sample refers to a sample that is of a different class from the correct answer but appears very similar to the correct answer from the perspective of a deep learning model. For example, in contrastive learning or embedding learning, a sample that is of a different class from the anchor but has very similar features is a hard negative sample. The electronic device (100) may lighten the deep learning model by repeatedly performing a channel pruning process using a training data set containing 'hard negative samples' and terminating when the target compression rate of the deep learning model is reached.

[0059] The electronic device (100) can transmit a lightweight deep learning model to a user device (200). The user device (200) can provide AI services by updating an existing model with the lightweight deep learning model and then running a deep learning model optimized for an on-device environment.

[0060] FIG. 3 is a diagram illustrating the process of a processor (120) of an electronic device (100) according to one embodiment of the present disclosure performing a channel pruning process to lighten a deep learning model.

[0061] The processor (120) of the electronic device (100) can load and execute a program module stored in memory (110). Referring to FIG. 3, the processor (120) can execute a data mining module (121), a learning module (123), an importance estimation module (125), a channel removal module (127), and a model optimization judgment module (129).

[0062] The data mining module (121) can obtain a training data set containing hard negative samples. The data mining module (121) can perform hard negative mining by selecting hard negative samples from a training database and including them in a training data set for channel pruning. Hard negative samples are negative samples in which the deep learning model cannot easily distinguish between correct and incorrect answers and is highly likely to make incorrect predictions. For example, in face recognition, a face photo of a person who looks very similar to the user in the face photo corresponding to the anchor (reference sample) but is actually a different person can be a hard negative sample. If such hard negative samples are actively utilized in the training of the deep learning model, the deep learning model can distinguish similarities and differences more precisely, thereby improving embedding quality, classification performance, and search accuracy.

[0063] The data mining module (121) can redefine and obtain a training data set containing hard negative samples each time the channel pruning process is repeated, because the embedding space of the deep learning model changes during channel pruning and fine-tuning in the channel pruning process.

[0064] The data mining module (121) can obtain other samples similar to the reference sample as hard negative samples based on the angular distance based on the sample-specific representative value of the embedding space in the training database. If the reference metric is cosine angular distance, the data mining module (1210) can obtain negative samples as hard negative samples that correspond to a medoid whose cosine similarity with the reference sample satisfies a predetermined criterion.

[0065] For example, the degree to which another speaker's voice is similar to User 'A's voice can be determined based on the cosine similarity value between the mediaoid of User 'A's voice samples in the embedding space and the mediaoid of other speakers' voice samples. In this case, voice samples of other speakers whose cosine similarity value falls within the top 20% (determined as the threshold showing the lowest Equal Error Rate based on experimental results) can be hard negative samples. If there are a total of 10 other speakers, from User 'B' to User 'K', and the cosine similarity values ​​between the mediaoid of each speaker's voice sample and the mediaoid of the reference sample, User 'A's voice sample, are as shown in Table 1 below, and the top 20% is used as the criterion for hard negative samples, then the voice samples of User 'B' and User 'C' can be hard negative samples. On the other hand, the voice samples of Users 'D' to User 'K' can be normal samples.

[0066] Cosine Similarity Top 20% User B 0.95 Applicable User C 0.85 Applicable User D 0.75 Unapplicable User E 0.65 Unapplicable User F 0.55 Unapplicable User G 0.45 Unapplicable User H 0.35 Unapplicable User I 0.25 Unapplicable User J 0.15 Unapplicable User K 0.05 Unapplicable

[0067] The data mining module (121) can construct a training data set in which the ratio of hard negative samples and normal samples follows a predetermined standard. For example, the data mining module (121) can generate a training data set of a mixed batch in which hard negative samples make up 40% and normal samples make up 60%. The data mining module (121) can generate a training data set of a dedicated batch in which hard negative samples make up 100%. The data mining module (121) can construct a training data set including a mixed batch containing hard negative samples and normal samples and a dedicated batch containing only hard negative samples, so that the training module (123) can alternately input the mixed batch and the dedicated batch to train a deep learning model. The data mining module (121) can obtain different training data sets corresponding to different hard negative samples based on different anchors depending on the situation.

[0068] The training module (123) can train a deep learning model using a training data set obtained from the data mining module (121) as input. The training module (123) can train the deep learning model for a predetermined number of epochs using a training data set that includes a mixed batch and a dedicated batch. The training module (123) can alternately input the mixed batch and the dedicated batch as inputs to the deep learning model.

[0069] Training deep learning models using hard negative samples enables embedding learning that better distinguishes similar samples, refines decision boundaries, and structurally improves the embedding space. Additionally, it can reduce the error rate during the actual application of deep learning models. However, if there are too many hard negative samples, the training of the deep learning module may become unstable. Therefore, maintaining an appropriate balance between hard negative samples and normal samples when training a deep learning model can play an important role in lightweighting and maximizing the performance of the deep learning model.

[0070] The importance estimation module (125) can determine the importance of multiple channels corresponding to each layer of the learned deep learning model. Multiple different importance estimation methods may be used to estimate the importance of each channel. For example, importance estimation methods may include sensitivity-based methods, attention-based methods, embedding perturbation-based methods, activation-based methods, weight-based methods, etc. The importance estimation module (125) can determine the importance of each of the multiple channels based on the sum of values ​​obtained by normalizing each of the multiple indicators according to different importance estimation methods and applying weights. This will be described in detail later in the description of FIG. 5.

[0071] The channel removal module (127) can remove selected channels among multiple channels based on determined importance. The channel removal module (127) can select lower channels among multiple channels whose determined importance is below a predetermined standard and remove the selected channels from the multiple channels. For example, if there are channels from the first channel to the 20th channel corresponding to any layer of a deep learning model, the channel removal module (127) can select channels among the first to 20 channels whose channel importance is in the bottom 10% and remove them from the multiple channels. Accordingly, whenever the channel pruning process is performed once, the channel removal module (127) can remove channels among the current channels whose channel importance is in the bottom 10%. By removing less important channels, the deep learning model can be made lighter, and computational load, memory usage, and response latency can be reduced.

[0072] However, problems may arise in the performance of the deep learning model after removing channels according to the channel pruning process. In such cases, the channel removal module (127) can adjust the channel removal rate or restore previously removed channels. For example, if the error for hard negative samples increases more rapidly than a certain standard after removing the channels compared to before removing the channels, the channel removal module (127) can set the channel removal rate lower than before. That is, previously, if a channel was removed when its importance was in the bottom 10%, the channel can be changed to be removed when it is in the bottom 5%. The channel removal module (127) checks the amount of change in the output of the deep learning model before and after removing the channels, and if the amount of change in the output of the deep learning model changes by more than a predetermined standard, the removed channels can be restored. That is, if the performance of the deep learning model drops significantly or does not operate correctly after removing the channel, the channel removal module (127) can restore the removed channel and exclude the restored channel from the channel importance evaluation so that it is not removed in the subsequent channel pruning process.

[0073] The model optimization judgment module (129) can restore and optimize performance by retraining the deep learning model with a modified structure using training data through retraining or fine-tuning, because if some channels are removed by the channel pruning process, the embedding space may change from the existing one and information loss or distortion may occur. The model optimization judgment module (129) can control the channel pruning process to be repeated until a predetermined criterion is satisfied. For example, the model optimization judgment module (129) can control the channel pruning process to be repeated until the size of the deep learning model on which the channel pruning process was performed becomes less than or equal to a target value, or until the target compression rate of the deep learning model is reached.

[0074] FIG. 4 is a drawing for illustrating a channel pruning process according to one embodiment of the present disclosure.

[0075] The channel pruning process may include a data mining step for obtaining a training data set for training a deep learning model by collecting, preprocessing, and analyzing training data; a training step in which the importance of each channel is reflected by training a deep learning model using the prepared training data set; an importance evaluation step for estimating the importance of each channel in the deep learning model after training is complete; and a channel removal step for lightweighting the deep learning model by ranking and listing channels according to the importance ranking of the channels, removing channels selected based on importance, and removing associated weights and connection structures together.

[0076] Referring to FIG. 4, the electronic device (100) may obtain a reference sample corresponding to an anchor from a user device (200) or a user database within the electronic device (100). The reference sample may be a face photo of a user registered to use a specific service on the user device (200) or a voice of a user, but is not limited thereto. The electronic device (100) may obtain other samples similar to the reference sample as hard negative samples. A face photo of another person whose facial features are very similar to the user's, or a voice of another person whose voice is very similar to the user's voice, may be hard negative samples that the deep learning model cannot easily distinguish. The electronic device (100) may obtain a training data set containing hard negative samples and train a deep learning model.

[0077] Referring to FIG. 4, the deep learning model is depicted as having a total of five convolution layers, from the first convolution layer to the fifth convolution layer, but this is for convenience of explanation only and is not limited thereto. The channels corresponding to each convolution layer are also depicted as having a total of five channels, from the first channel to the fifth channel, but this is for convenience of explanation only and is not limited thereto.

[0078] The electronic device (100) can determine the importance of multiple channels corresponding to each convolution layer of a learned deep learning model by using multiple importance estimation methods. This will be explained in detail below with reference to FIG. 5.

[0079] FIG. 5 is a diagram illustrating a method for estimating channel importance according to one embodiment of the present disclosure.

[0080] There may be various methods for estimating the importance of each channel of a deep learning model. For example, there may be sensitivity-based methods, attention-based methods, embedding perturbation-based methods, activation-based methods, weight-based methods, etc., but are not limited to these.

[0081] Sensitivity-based methods calculate importance by measuring changes in model performance after temporarily removing a channel. For example, gradient-based methods evaluate the importance of a channel by approximating the change in the loss function when the channel is removed using the gradient. Empirical sensitivity analysis calculates importance by measuring the decrease in model accuracy after temporarily removing a channel.

[0082] Attention-based methods are approaches in which the network directly learns and dynamically adjusts the importance of each channel in a deep learning model, allowing the network to independently determine how important each channel is based on the characteristics of the input data. For example, the Squeeze and Excaitation block method compresses the spatial information of the input feature map using methods such as global average pooling to create a single representative value for each channel. It then passes the compressed channel vector through a small neural network to calculate an attention weight representing the importance of each channel, which is then multiplied by the feature map of each channel to emphasize the information of the important channels.

[0083] The embedding perturbation-based method is a method that quantitatively evaluates the importance of a channel by artificially perturbing each channel of a deep learning model one by one and measuring the change in the embedding. For example, it is a method in which each channel of a deep learning model is temporarily removed one by one, and the greater the distance between the embeddings before and after perturbation (e.g., cosine similarity, L2 norm, etc.), the higher the importance of that channel is determined.

[0084] The activation-based method is a method that uses training data to measure activation values ​​for each channel and evaluates channels with low activation values ​​as having low importance. For example, the Mean Activation or APoZ (Average Percentage of Zeros) can be obtained, and the importance of a channel can be determined based on the obtained values.

[0085] Weight-based methods are methods that obtain the sum or sum of squares of the absolute weights of each filter and evaluate channels with small values ​​as having low importance. For example, L1 / L2 norm-based methods can determine channels with small values ​​as having low importance by calculating the sum (L1 norm) or sum of squares (L2 norm) of the absolute weights corresponding to each channel.

[0086] According to one embodiment of the present disclosure, an electronic device (100) can determine the importance of each of the plurality of channels based on a value obtained by normalizing each of the plurality of indicators according to different importance estimation methods and then summing them with weights. Referring to FIG. 5, the electronic device (100) can determine the importance of the plurality of channels corresponding to each layer of a deep learning model according to each of the first importance estimation method (e.g., gradient-based method), the second importance estimation method (e.g., Squeeze and Excaitation block method), and the third importance estimation method (e.g., embedding perturbation method). For each of the plurality of channels corresponding to each layer of the deep learning model, the electronic device (100) can obtain a first indicator according to the first importance estimation method and perform normalization. For each of the plurality of channels corresponding to each layer of the deep learning model, the electronic device (100) can obtain a second indicator according to the second importance estimation method and perform normalization. The electronic device (100) can obtain a third indicator according to a third importance estimation method and normalize each of the multiple channels corresponding to each layer of a deep learning model. The electronic device (100) can obtain a final indicator by calculating a weighted sum for the normalized first indicator, second indicator, and third indicator, and can determine the importance of each of the multiple channels based on the final indicator. By using importance estimation methods from different perspectives, the electronic device (100) can determine the importance of each of the multiple channels corresponding to each layer of the deep learning model from an unbiased and comprehensive perspective.

[0087] Referring again to FIG. 4, it was estimated that the importance of the fifth channel among the first to fifth channels corresponding to the first convolution layer was the lowest, and the importance of the fourth channel among the first to fifth channels corresponding to the second convolution layer was estimated to be the lowest. It was estimated that the importance of the fifth channel among the first to fifth channels corresponding to the third convolution layer was the lowest, the importance of the third channel among the first to fifth channels corresponding to the fourth convolution layer was estimated to be the lowest, and the importance of the fourth channel among the first to fifth channels corresponding to the fifth convolution layer was estimated to be the lowest.

[0088] If the electronic device (100) is programmed to remove channels of the lowest importance first in order to lighten the deep learning model, the fifth channel corresponding to the first convolution layer, the fourth channel corresponding to the second convolution layer, the fifth channel corresponding to the third convolution layer, the third channel corresponding to the fourth convolution layer, and the fourth channel corresponding to the fifth convolution layer may be removed. As a result, the electronic device (100) can reduce the size of the deep learning model by 20% through a single channel pruning process.

[0089] If the goal is to reduce the size of the deep learning model to 50% or less of the size of the existing model, the electronic device (100) can remove not only the least important channel but also the second least important and third least important channels at once. However, if many channels are removed at once through a single channel pruning process like this, the precise boundary that the deep learning model uses to distinguish hard negative samples is no longer maintained, and the performance of the deep learning model may decrease rapidly.

[0090] FIG. 6 is a diagram illustrating a process of repeatedly performing a channel pruning process according to one embodiment of the present disclosure.

[0091] In order to prevent the deterioration of the deep learning model's discrimination characteristics and model performance regarding hard negative samples due to channel pruning, a gradual reduction of channels corresponding to each layer of the deep learning model is required. Accordingly, the electronic device (100) can limit the removal rate of channels that can be removed through a single channel pruning and can perform the channel pruning process repeatedly. Referring to FIG. 6, it can be seen that whenever the channel pruning process is performed repeatedly, the channels with the lowest importance are selected and removed one by one. Meanwhile, if the error regarding hard negative samples increases rapidly after removing channels compared to before removing channels, the electronic device (100) can set the channel removal rate to be lower than before.

[0092] However, whenever the channel pruning process is performed, if some channels are removed by the channel pruning process, the embedding space changes from the original, which may result in information loss or distortion.

[0093] The electronic device (100) can restore and optimize performance by retraining a deep learning model with a modified structure through retraining or fine-tuning using training data. For example, the electronic device (100) can perform embedding perturbation correction by additional training or adjustment to minimize or restore perturbations appearing in the embedding space of the deep learning model after channel pruning. Nevertheless, since the embedding space changes with each channel pruning and retraining, the electronic device (100) can redefine and acquire a training data set containing hard negative samples at each iteration.

[0094] The electronic device (100) can repeat the channel pruning process until a predetermined standard is satisfied. For example, the electronic device (100) can obtain a lightweight model by repeating the channel pruning process until the size of the deep learning model on which the channel pruning process was performed becomes less than or equal to a target value or until the target compression rate of the deep learning model is reached.

[0095] FIG. 7 is a diagram illustrating a step of restoring a channel removed in a channel pruning process according to one embodiment of the present disclosure.

[0096] The electronic device (100) can check the amount of change in the output of the deep learning model before and after the removal of the channel. The electronic device (100) can restore the removed channel when the amount of change in the output of the deep learning model changes by more than a predetermined standard.

[0097] Referring to FIG. 7, the electronic device (100) can further determine whether restoration of the removed channels is necessary after removing the channels through a channel pruning process. For example, if there is no change in the performance of the deep learning model beyond a threshold after removing some channels, it can be determined that restoration of the removed channels is not necessary. Accordingly, the electronic device (100) can perform a second channel pruning process.

[0098] However, if there is a change in the performance of the deep learning model that exceeds a threshold after removing some channels, that is, if the performance of the deep learning model drops significantly or does not operate correctly, the electronic device (100) can restore the removed channels through the initial channel pruning process. The restored channels can be excluded from the channel importance re-evaluation target so that they are not removed in a subsequent channel pruning process. Accordingly, the electronic device (100) can remove the channel with the lowest importance among the other channels, excluding the restored channels. Accordingly, the electronic device (100) can perform a second channel pruning process.

[0099] FIG. 8 is a diagram illustrating a channel pruning process using different training data sets corresponding to different hard negative samples based on different anchors according to one embodiment of the present disclosure.

[0100] Referring to FIG. 8, the electronic device (100) can acquire different training data sets corresponding to different hard negative samples based on different anchors. The electronic device (100) can train a deep learning model for each different training data set and determine the importance of the channel.

[0101] The electronic device (100) can determine the importance of a channel of a deep learning model by comprehensively considering the importance of the channels of a deep learning model trained for different training data sets. Alternatively, the electronic device (100) can determine the importance of a channel of a deep learning model by selecting one deep learning model according to the situation and considering the importance of the channels of the deep learning model. The electronic device (100) can remove the selected channel based on the determined importance of the channel.

[0102] FIG. 9 is a flowchart illustrating a method for lightweighting a deep learning model according to one embodiment of the present disclosure.

[0103] In the case where a method for lightweighting a deep learning model according to one embodiment of the present disclosure includes a plurality of operations, the plurality of operations may be performed by a single processor or by a plurality of processors. For example, when a first operation, a second operation, and a third operation are performed by a method according to one embodiment, the first operation, the second operation, and the third operation may all be performed by a first processor, or the first operation and the second operation may be performed by a first processor (e.g., a general-purpose processor) and the third operation may be performed by a second processor (e.g., an artificial intelligence dedicated processor). Each operation of the method for lightweighting a deep learning model according to one embodiment of the present disclosure may be performed by at least one processor (110) mounted on an electronic device (100), but is not limited thereto.

[0104] The electronic device (100) can acquire a training data set containing hard negative samples (S 910). The electronic device (100) can construct a training data set for channel pruning by selecting hard negative samples from a training database. The electronic device (100) can acquire other samples similar to a reference sample as hard negative samples based on the angular distance based on the sample-specific representative value of the embedding space in the training database. If the reference metric is cosine angular distance, the electronic device (100) can acquire a negative sample as a hard negative sample that corresponds to a medoid whose cosine similarity with the reference sample satisfies a predetermined criterion. The electronic device (100) can redefine and acquire a training data set containing hard negative samples each time the channel pruning process is repeated.

[0105] The electronic device (100) can configure a training data set in which the ratio of hard negative samples and normal samples follows a predetermined standard. The electronic device (100) can configure a training data set including a mixed batch and a dedicated batch so that a deep learning model can be trained by alternately inputting a mixed batch containing hard negative samples and normal samples and a dedicated batch containing only hard negative samples.

[0106] The electronic device (100) can acquire different training data sets corresponding to different hard negative samples based on different anchors. For the same sample, different anchors may exist depending on the situation, or different types of anchors may exist.

[0107] The electronic device (100) can train a deep learning model by taking a training data set as input. (S920) The electronic device (100) can train a deep learning model by taking an acquired training data set as input. The electronic device (100) can train a deep learning model for a predetermined number of epochs using a training data set that includes a mixed batch and a dedicated batch. The electronic device (100) can alternately input a mixed batch and a dedicated batch as input to the deep learning model.

[0108] The electronic device (100) can determine the importance of multiple channels corresponding to each layer of a learned deep learning model. (S 930) The electronic device (100) may use different multiple importance estimation methods to estimate the importance of each channel. The electronic device (100) can determine the importance of each of the multiple channels based on a value obtained by normalizing each of the multiple indicators according to different importance estimation methods, giving them weights, and summing them.

[0109] The electronic device (100) can remove a selected channel among a plurality of channels based on the importance of a plurality of channels corresponding to each layer of a deep learning model. (S 940) The electronic device (100) can remove a selected channel among a plurality of channels based on the importance of a plurality of channels. The electronic device (100) can select a lower channel among a plurality of channels whose importance is below a predetermined standard and remove the selected channel from the plurality of channels. The electronic device (100) can remove a channel among the current channels whose importance is lower each time the channel pruning process is performed.

[0110] The electronic device (100) can adjust the channel removal rate or restore previously removed channels. For example, the electronic device (100) can set the channel removal rate lower than before if the error for hard negative samples increases more rapidly than a certain standard after removing the channel compared to before removing the channel. The electronic device (100) checks the amount of change in the output of the deep learning model before and after removing the channel, and if the amount of change in the output of the deep learning model changes more than a predetermined standard, it can restore the removed channel. The electronic device (100) can restore the removed channel and exclude the restored channel from the channel importance evaluation target so that it is not removed in the subsequent channel pruning process.

[0111] The electronic device (100) may repeat steps S 910 to S 940 until a predetermined criterion is satisfied (S 950). Since some channels are removed by the channel pruning process, the embedding space may differ from the existing one, causing information loss or distortion, the deep learning model with the modified structure can be retrained with training data through retraining or fine-tuning to recover and optimize performance. The electronic device (100) may repeat the channel pruning process until a predetermined criterion is satisfied. The electronic device (100) may repeat the channel pruning process until the size of the deep learning model on which the channel pruning process was performed becomes less than or equal to a target value, or until the target compression rate of the deep learning model is reached.

[0112] Meanwhile, embodiments of the present disclosure may also be implemented in the form of a recording medium containing computer-executable instructions, such as program modules executed by a computer. A computer-readable medium may be any available medium accessible by a computer and includes both volatile and non-volatile media, and both removable and non-removable media. Additionally, a computer-readable medium may include computer storage media and communication media. Computer storage media include both volatile and non-volatile, removable and non-removable media implemented by any method or technique for storing information, such as computer-readable instructions, data structures, program modules, or other data. Communication media may typically include other data of modulated data signals, such as computer-readable instructions, data structures, or program modules.

[0113] Additionally, computer-readable storage media may be provided in the form of non-transitory storage media. Here, 'non-transitory storage media' simply means that it is a tangible device and does not contain a signal (e.g., electromagnetic waves), and this term does not distinguish between cases where data is stored semi-permanently and cases where it is stored temporarily. For example, 'non-transitory storage media' may include a buffer in which data is stored temporarily.

[0114] According to one embodiment, the method according to one embodiment of the present disclosure may be provided by being included in a computer program product. The computer program product may be traded between a seller and a buyer as a product. The computer program product may be distributed in the form of a device-readable storage medium (e.g., compact disc read-only memory (CD-ROM)), or distributed online (e.g., download or upload) through an application store or directly between two user devices (e.g., smartphones). In the case of online distribution, at least a portion of the computer program product (e.g., downloadable app) may be temporarily stored or temporarily created on a device-readable storage medium, such as the memory of a manufacturer's server, an application store's server, or a relay server.

[0115] According to one embodiment of the present disclosure, a method for lightweighting a deep learning model is provided. The method for lightweighting a deep learning model may include (a) a step of acquiring a training data set containing hard negative samples (S910). Additionally, the method for lightweighting a deep learning model may include (b) a step of training a deep learning model using the training data set as input (S920). Additionally, the method for lightweighting a deep learning model may include (c) a step of determining the importance of a plurality of channels corresponding to each layer of the trained deep learning model (S930). Additionally, the method for lightweighting a deep learning model may include (d) a step of removing a selected channel among the plurality of channels based on the determined importance (S940). Additionally, the method for lightweighting a deep learning model may include a step of repeating steps (a) through (d) until a predetermined criterion is satisfied (S950).

[0116] Additionally, according to one embodiment of the present disclosure, the step of acquiring a training data set (S910) may include the step of acquiring another sample similar to a reference sample as the hard negative sample based on the angular distance based on the sample-specific representative value of the embedding space in the training database. Additionally, the step of acquiring a training data set (S910) may include the step of configuring a training data set in which the ratio of hard negative samples to normal samples follows a predetermined criterion.

[0117] Additionally, according to one embodiment of the present disclosure, the step of constructing a training data set may include a mixed batch containing hard negative samples and normal samples and a dedicated batch containing only hard negative samples.

[0118] Additionally, according to one embodiment of the present disclosure, the step (S920) of training a deep learning model may train the deep learning model for a predetermined number of epochs using a training data set including a mixed batch and a dedicated batch.

[0119] Additionally, according to one embodiment of the present disclosure, the step (S930) of determining the importance of a plurality of channels can determine the importance of each of the plurality of channels based on a sum of values ​​obtained by normalizing each of the plurality of indicators according to different importance estimation methods and then applying weights.

[0120] Additionally, according to one embodiment of the present disclosure, the step of removing a selected channel (S940) may include the step of selecting a sub-channel among a plurality of channels whose determined importance is below a predetermined standard. Additionally, the step of removing a selected channel (S940) may include the step of removing a selected channel from a plurality of channels.

[0121] Additionally, according to one embodiment of the present disclosure, the step of removing a selected channel (S940) may include a step of checking the amount of change in the output of the deep learning model before and after the removal of the selected channel. Additionally, the step of removing a selected channel (S940) may include a step of restoring the removed channel when the amount of change in the output of the deep learning model changes by more than a predetermined standard.

[0122] Additionally, according to one embodiment of the present disclosure, the step of acquiring a training data set (S910) may acquire different training data sets corresponding to different hard negative samples based on different anchors.

[0123] Additionally, according to one embodiment of the present disclosure, the repeating step (S950) may repeat steps (a) through (d) until the size of the deep learning model becomes less than or equal to the target value.

[0124] According to one embodiment of the present disclosure, a computer-readable recording medium may be provided on which a program for executing a method to lighten a deep learning model is recorded.

[0125] According to one embodiment of the present disclosure, an electronic device (100) for lightweighting a deep learning model may be provided. The electronic device (100) for lightweighting a deep learning model may include a memory (110) comprising one or more storage media for storing instructions and at least one processor (120) comprising a processing circuit. By having at least one processor (120) execute one or more instructions individually or jointly, the electronic device (100) may acquire a training data set including hard negative samples. Additionally, by having at least one processor (120) execute one or more instructions individually or jointly, the electronic device (100) may train a deep learning model using the training data set as input. Additionally, by having at least one processor (120) execute one or more instructions individually or jointly, the electronic device (100) may determine the importance of a plurality of channels corresponding to each layer of the trained deep learning model. Additionally, by having at least one processor (120) execute one or more instructions individually or jointly, the electronic device (100) can remove a selected channel among a plurality of channels based on a determined importance. Additionally, by having at least one processor (120) execute one or more instructions individually or jointly, the electronic device (100) can repeat the channel pruning process until a predetermined criterion is satisfied.

[0126] Additionally, according to one embodiment of the present disclosure, by having at least one processor (120) execute one or more instructions individually or jointly, the electronic device (100) can acquire other samples similar to a reference sample as hard negative samples based on the angular distance of the sample-specific representative value of the embedding space in the learning database, and can construct the learning data set in which the ratio of hard negative samples to normal samples follows a predetermined standard.

[0127] Additionally, according to one embodiment of the present disclosure, by having at least one processor (120) execute one or more instructions individually or jointly, the electronic device (100) can construct a training data set comprising a mixed batch containing hard negative samples and normal samples and a dedicated batch containing only hard negative samples.

[0128] Additionally, according to one embodiment of the present disclosure, by having at least one processor (120) execute one or more instructions individually or jointly, the electronic device (100) can train a deep learning model for a predetermined number of epochs using a training data set including a mixed batch and a dedicated batch.

[0129] Additionally, according to one embodiment of the present disclosure, by having at least one processor (120) execute one or more instructions individually or jointly, the electronic device (100) can determine the importance of each of the plurality of channels based on a value obtained by normalizing each of the plurality of indicators according to different importance estimation methods and then adding weights.

[0130] Additionally, according to one embodiment of the present disclosure, by having at least one processor (120) execute one or more instructions individually or jointly, the electronic device (100) can select a lower channel among a plurality of channels whose determined importance is below a predetermined standard and remove the selected channel from the plurality of channels.

[0131] Additionally, according to one embodiment of the present disclosure, by having at least one processor (120) execute one or more instructions individually or jointly, the electronic device (100) can check the amount of change in the output of a deep learning model before and after the removal of a selected channel, and when the amount of change in the output of the deep learning model changes by more than a predetermined standard, the removed channel can be restored.

[0132] Additionally, according to one embodiment of the present disclosure, by having at least one processor (120) execute one or more instructions individually or jointly, the electronic device (100) can obtain different learning data sets corresponding to different hard negative samples based on different anchors.

[0133] Additionally, according to one embodiment of the present disclosure, by having at least one processor (120) execute one or more instructions individually or jointly, the electronic device (100) can repeat the channel pruning process until the size of the deep learning model becomes less than or equal to a target value.

[0134] Additionally, according to one embodiment of the present disclosure, the electronic device (100) for lightweighting a deep learning model may further include a communication unit (130). Additionally, by having at least one processor (120) execute one or more instructions individually or jointly, the electronic device (100) may transmit information regarding the lightweighting of a deep learning model or the lightweighting of a deep learning model to a user device through the communication unit (130).

[0135] The foregoing description of the present disclosure is for illustrative purposes only, and those skilled in the art will understand that other specific forms can be easily modified without altering the technical spirit or essential features of the present disclosure. Therefore, the embodiments described above should be understood as illustrative in all respects and not restrictive. For example, each component described as a single unit may be implemented in a distributed manner, and components described as distributed may likewise be implemented in a combined form.

[0136] The scope of the present disclosure is defined by the claims set forth below rather than by the detailed description above, and all modifications or variations derived from the meaning and scope of the claims and equivalent concepts thereof should be interpreted as being included within the scope of the present disclosure.

Claims

1. (a) A step of acquiring a training data set including hard negative samples (S910); (b) A step of training a deep learning model using the above training data set as input (S920); (c) A step of determining the importance of a plurality of channels corresponding to each layer of the above-mentioned learned deep learning model (S930); (d) a step of removing a selected channel among the plurality of channels based on the importance determined above (S940); and Step (S950) of repeating steps (a) through (d) until a predetermined standard is satisfied; A method for lightweighting a deep learning model including 2. In Paragraph 1, The step of acquiring the above training data set (S910) is, A step of obtaining another sample similar to a reference sample as the hard negative sample based on the angular distance based on the sample-specific representative value of the embedding space in the training database; and A step of constructing the training data set in which the ratio of the hard negative samples and normal samples follows a predetermined standard; A method including 3. In Paragraph 1 or 2, The step of constructing the above training data set is, A method for constructing a training dataset comprising a mixed batch containing the hard negative samples and the normal samples, and a dedicated batch containing only the hard negative samples.

4. In any one of paragraphs 1 to 3, The step (S920) of training the above deep learning model is, A method for training a deep learning model for a predetermined number of epochs using a training dataset including the mixed batch and the dedicated batch.

5. In any one of paragraphs 1 through 4, The step (S930) of determining the importance of the plurality of channels above is, A method for determining the importance of each of the plurality of channels based on a sum of values ​​obtained by normalizing each of the plurality of indicators according to different importance estimation methods and then applying weights.

6. In any one of paragraphs 1 through 5, The step (S940) of removing the selected channel above is, A step of selecting a sub-channel among the plurality of channels above in which the determined importance is below a predetermined standard; and A step of removing the selected channel from the plurality of channels; A method including 7. In any one of paragraphs 1 through 6, The step (S940) of removing the selected channel above is, A step of verifying the amount of change in the output of the deep learning model before and after the removal of the selected channel; and A method further comprising the step of restoring the removed channel when the amount of change in the output of the deep learning model is changed beyond a predetermined standard.

8. In any one of paragraphs 1 through 7, The step of acquiring the above training data set (S910) is, A method for obtaining different training data sets corresponding to different hard negative samples based on different anchors.

9. In any one of paragraphs 1 through 8, The above repeating step (S950) is, A method of repeating steps (a) through (d) until the size of the deep learning model becomes less than or equal to a target value.

10. A computer-readable recording medium having a program recorded thereon for executing the method of any one of paragraphs 1 through 9.

11. Memory (110) comprising one or more storage media for storing instructions; and It includes at least one processor (120) including a processing circuit, and By having at least one processor (120) execute one or more instructions individually or jointly, the electronic device (100) An electronic device (100) for lightweighting a deep learning model, which acquires a training data set including hard negative samples, trains a deep learning model using the training data set as input, determines the importance of a plurality of channels corresponding to each layer of the trained deep learning model, and repeats the process of removing a selected channel among the plurality of channels based on the determined importance until a predetermined criterion is satisfied.

12. In Paragraph 11, By having at least one processor (120) execute one or more instructions individually or jointly, the electronic device (100) Based on the angular distance based on sample-specific representative values ​​of the embedding space in the training database, other samples similar to the reference sample are obtained as the hard negative samples, and An electronic device (100) that forms a training data set in which the ratio of the hard negative samples and normal samples follows a predetermined standard.

13. In any one of paragraphs 11 to 12, By having at least one processor (120) execute one or more instructions individually or jointly, the electronic device (100) An electronic device (100) that determines the importance of each of the plurality of channels based on a sum of values ​​obtained by normalizing each of the plurality of indicators according to different importance estimation methods and then giving them weights.

14. In any one of paragraphs 11 through 13, By having at least one processor (120) execute one or more instructions individually or jointly, the electronic device (100) Among the above plurality of channels, select a sub-channel whose determined importance is below a predetermined standard, and An electronic device (100) for removing the selected channel from the plurality of channels.

15. In any one of paragraphs 11 through 14, By having at least one processor (120) execute one or more instructions individually or jointly, the electronic device (100) An electronic device (100) that acquires different training data sets corresponding to different hard negative samples based on different anchors.

Images