Learning difficulty level-based continuous training method for mobile deep learning model

Abstract

Provided is a learning difficulty level-based continuous training method for a mobile deep learning model. The continuous training method according to an embodiment of the present invention includes obtaining datasets to be used for continuous training of a previously-trained deep learning model, calculating learning difficulty levels of the datasets, determining a training order of the datasets according to the calculated learning difficulty levels, and continuously training the deep learning model with the datasets according to the determined training order. Accordingly, it is possible to maximize the performance of the continuous training within restricted resources and model environment by determining the training order according to the learning difficulty levels of the datasets and continuously training the deep learning model.

Description

A learning difficulty-based continuous learning method for mobile deep learning models

[0001] The present invention relates to deep learning model training, and more particularly, to a method for continuously training a deep learning model in a limited mobile environment for object classification and location information data in computer vision.

[0002] Conventional CNN continuous learning models are mostly models that focus on maintaining previous learning performance based on environments that learn large amounts of data information. However, they have the problem of being unsuitable for learning on a very small number of image data or for small learning models that are implemented in limited environments such as mobile devices.

[0003] Therefore, there are additional issues that cannot be considered in the environment of a large-scale model that can sufficiently learn based on sufficient image data, and there is a problem that performance fluctuations occur when conventional technologies that do not have measures in place to address these issues are applied to limited environments.

[0004] The present invention has been devised to solve the above problems, and the purpose of the present invention is to provide a method for continuously training a deep learning model by determining the learning order according to the learning difficulty of datasets, as a means for maximizing the performance of continuous learning within limited resources and model environments.

[0005] A continuous learning method according to one embodiment of the present invention for achieving the above object includes the steps of: acquiring datasets to be used for continuous learning for a pre-trained deep learning model; calculating learning difficulties of the datasets; determining a learning order of the datasets according to the calculated learning difficulties; and continuously training a deep learning model with the datasets according to the determined learning order.

[0006] The decision step can be such that the higher the learning difficulty, the faster the learning order is decided, and the lower the learning difficulty, the slower the learning order is decided.

[0007] Deep learning models may have a reduced number of learnable parameters as continuous learning progresses.

[0008] As deep learning models progress through continuous learning, fixed parameters may increase and pruned parameters may decrease.

[0009] The dataset can be an image dataset.

[0010] The calculation step may be to calculate the learning difficulty of the datasets based on the number of channels whose output value level is higher than a standard when the image dataset is input to a pre-trained deep learning model.

[0011] The calculation step may be to calculate the learning difficulty as lower the number of channels with output value levels higher than the standard, and to calculate the learning difficulty as higher the number of channels with output value levels higher than the standard.

[0012] Deep learning models may not scale well during continuous learning.

[0013] Deep learning models can be deployed on mobile devices with limited resources.

[0014] According to another aspect of the present invention, a deep learning computing device is provided, characterized by including: a communication interface for acquiring datasets to be used for continuous learning for a pre-learned deep learning model; a computing device for calculating learning difficulties of the datasets, determining a learning order of the datasets according to the calculated learning difficulties, and continuously learning the deep learning model with the datasets according to the determined learning order.

[0015] According to another aspect of the present invention, a continuous learning method is provided, characterized by including: a step of determining a learning order of datasets to be used for continuous learning for a pre-trained deep learning model according to the learning difficulty of the datasets; and a step of continuously training a deep learning model with the datasets according to the determined learning order.

[0016] According to another aspect of the present invention, a deep learning computing device is provided, characterized by including: a computing device that determines the learning order of datasets to be used for continuous learning for a pre-learned deep learning model according to the learning difficulty of the datasets, and continuously trains the deep learning model with the datasets according to the determined learning order; and a memory that provides storage space required for the computing device.

[0017] As described above, according to embodiments of the present invention, by determining the learning order according to the learning difficulty of datasets and continuously learning a deep learning model, the performance of continuous learning can be maximized within limited resources and model environments.

[0018] Figure 1. Mobile device environment with limited model and dataset size.

[0019] Figure 2. Example of the influence of subsequent learning on the difficulty of the learning dataset.

[0020] Figure 3-4. Comparison of channel value distributions between face images with low learning difficulty (top) and dog images with high learning difficulty (bottom).

[0021] Figure 5-6. Comparison of model performances achieved through continuous learning in different order based on learning difficulty.

[0022] Figure 7. Learning difficulty-based continuous learning method for mobile deep learning models.

[0023] Figure 8. Mobile deep learning computing device

[0024] Hereinafter, the present invention will be described in more detail with reference to the drawings.

[0025] In real-time object recognition through cameras, attention is being paid not only to models that simply perform recognition based on learned models, but also to the design of neural network models capable of continuous learning on image datasets coming in in real time.

[0026] Furthermore, as the design of neural networks capable of learning in limited environments such as mobile devices for application in these cases emerges, deep learning models are no longer simply required to perform well, but also to develop technologies that can learn with limited image data information in much smaller models compared to existing large-scale models.

[0027] However, as shown in Fig. 1, there are several limitations to designing a continuous learning model structure suitable for hardware, and in particular, it is very difficult to develop a small-scale continuous learning model applicable to hardware structures without additional model expansion as new image data is added.

[0028] Most continuous learning methods assume a learning environment where large, multi-class image datasets are trained based on models large enough for cumulative learning. Therefore, existing models struggle to address the challenges associated with training small-scale neural network models suitable for mobile devices.

[0029] The basic continuous learning model we intend to develop is based on PackNet, an existing continuous learning method. This model enables continuous learning while minimizing the need for additional expansion of existing neural network models to learn new image datasets. This model compresses and fixes only the core information through post-training image dataset pruning, thereby creating space for additional dataset learning.

[0030] However, in this process, a problem arises in that the learning performance of additional image datasets falls short of general learning or transfer learning due to a decrease in the number of learnable parameters, as shown in Fig. 2.

[0031] To address these issues, models are extended using additional neural networks. However, this approach is difficult to apply in environments where a fixed model structure must be maintained for hardware design purposes. In particular, for small-scale learning models intended for use on mobile devices, the reduced number of learnable parameters significantly reduces performance.

[0032] As a solution to this problem, an embodiment of the present invention proposes a method of judging the learning suitability of a model of a continuous dataset to be learned in advance before learning, and proceeding with learning in the learning order most suitable for the model.

[0033] The upper portions of Figures 3 and 4 compare the channel value distributions for face images, which have a low learning difficulty, with the lower portions comparing the channel value distributions for dog images, which have a high learning difficulty. The output values ​​for each channel are listed in descending order by simply passing each image dataset through a deep learning model with pre-trained parameters as initial values.

[0034] Pre-learned parameters are generally information learned from large-scale image datasets, and transfer learning, which learns other image datasets based on this information, is commonly used because the parameters have the versatility to enable adaptive learning for various types of images.

[0035] Therefore, when passing an image dataset through a model based on pre-learned parameters, the fact that there are many channels that output high values ​​overall (top) can be used as an indicator that the model is likely to be relatively easy to learn compared to cases where there are few channels that output high values ​​(bottom).

[0036] Image datasets with fewer channels that output relatively high values ​​in a similarly trained neural network model have a higher learning difficulty, which means a relatively larger number of parameters are required for training. Conversely, image datasets with more channels that output relatively high values ​​in a similarly trained neural network model have a lower learning difficulty, which means a relatively smaller number of parameters are required for training.

[0037] Accordingly, in the deep learning model to be used in the embodiment of the present invention, as the number of additional learning image datasets accumulates, the number of learnable parameters in the model decreases, so that datasets with high learning difficulty are trained first, which is advantageous for improving overall performance.

[0038] Figure 5 shows the cumulative performance of a model that first learned an image dataset with relatively low learning difficulty based on a learning difficulty index, and Figure 6 shows the cumulative performance of a model that first learned a dataset with relatively high learning difficulty.

[0039] Even though the same dataset was trained, a model that used more parameters for a dataset with a lower learning difficulty, that is, a model that trained from a dataset with a lower learning difficulty, could not properly train a dataset with a higher learning difficulty in subsequent continuous learning processes.

[0040] On the other hand, a model that uses more parameters for datasets with high learning difficulty first, that is, when learning from datasets with high learning difficulty, not only achieves much smoother learning and higher performance than the former model, but can also properly learn datasets with relatively low learning difficulty even when the number of learnable parameters is small.

[0041] Figure 7 is a flowchart illustrating a learning difficulty-based continuous learning method for a mobile deep learning model. The continuous learning method according to an embodiment of the present invention achieves optimal performance based on a small dataset and limited resources without using model expansion through additional neural networks.

[0042] In the continuous learning method according to an embodiment of the present invention, the number of learnable parameters of a deep learning model decreases as continuous learning progresses. That is, as continuous learning progresses, fixed parameters increase and pruned parameters decrease.

[0043] In order to continuously learn a deep learning model installed on a mobile device, first, image datasets to be used for continuous learning for a pre-trained mobile deep learning model are acquired (S110).

[0044] The learning difficulty of the following acquired image datasets is calculated (S120), and the calculated learning difficulty is stored in memory together with the image datasets (S130).

[0045] Learning difficulty is calculated based on the number of channels whose output value levels (e.g., sum or average) exceed a threshold when an image dataset is input to a pre-trained deep learning model. More specifically, the more channels with output values ​​above the threshold, the lower the learning difficulty is calculated. The fewer channels with output values ​​above the threshold, the higher the learning difficulty is calculated.

[0046] In other words, when an image dataset is input to a pre-trained deep learning model, the learning difficulty of the dataset can be calculated based on the number of channels whose output value level is above the standard.

[0047] The learning difficulty of each image dataset is compared (S140), and the learning order of the datasets is determined / changed based on the learning difficulty (S150). Specifically, the higher the learning difficulty, the faster the learning order is determined, while the lower the learning difficulty, the slower the learning order is determined.

[0048] Afterwards, the deep learning model is continuously trained using image datasets according to the learning order determined in step S150 (S160).

[0049] Continuous learning thus provides performance advantages in limited environments such as mobile devices, and also has advantages in hardware design by inducing maximum learning efficiency within a fixed model.

[0050] FIG. 8 is a diagram illustrating the configuration of a mobile deep learning computing device according to another embodiment of the present invention. As illustrated, the mobile deep learning computing device according to an embodiment of the present invention is configured to include a communication interface (210), a deep learning computing unit (220), and a memory (230).

[0051] The communication interface (210) communicates with an external host system to receive an image dataset and a pre-trained deep learning model. The deep learning operator (220) loads the received deep learning model and performs continuous learning according to the method presented in FIG. 7 described above. The memory (230) provides the storage space required for the deep learning operator (220) to perform calculations.

[0052] So far, we have described in detail a preferred embodiment of a learning difficulty-based continuous learning method for a mobile deep learning model.

[0053] In the above embodiment, a method of continuously training a deep learning model by determining the learning order according to the learning difficulty of the datasets is presented as a general-purpose performance improvement method that can be applied to most mobile deep learning models and enables learning with maximum performance within a limited memory and model environment.

[0054] Meanwhile, it goes without saying that the technical idea of ​​the present invention can also be applied to a computer-readable recording medium containing a computer program that performs the functions of the device and method according to the present embodiment. In addition, the technical idea according to various embodiments of the present invention can be implemented in the form of computer-readable code recorded on a computer-readable recording medium. The computer-readable recording medium can be any data storage device that can be read by a computer and store data. For example, the computer-readable recording medium can be a ROM, a RAM, a CD-ROM, a magnetic tape, a floppy disk, an optical disk, a hard disk drive, etc. In addition, the computer-readable code or program stored on the computer-readable recording medium can be transmitted through a network connected between computers.

[0055] In addition, although the preferred embodiments of the present invention have been illustrated and described above, the present invention is not limited to the specific embodiments described above, and various modifications can be made by a person having ordinary skill in the art to which the present invention pertains without departing from the gist of the present invention as claimed in the claims, and such modifications should not be understood individually from the technical idea or prospect of the present invention.

Claims

1. A step for acquiring datasets to be used for continuous learning for a pre-trained deep learning model; Step of calculating the learning difficulty of datasets; A step of determining the learning order of datasets based on the calculated learning difficulty; A continuous learning method, characterized by including a step of continuously learning a deep learning model using datasets according to a determined learning order.

2. In claim 1, The decision stage is, The higher the learning difficulty, the faster the learning order is determined, A continuous learning method characterized by determining the learning order slowly as the learning difficulty level decreases.

3. In claim 2, The deep learning model is, A continuous learning method characterized in that the number of learnable parameters decreases as continuous learning progresses.

4. In claim 3, The deep learning model is, A continuous learning method characterized in that as continuous learning progresses, fixed parameters increase and pruned parameters decrease.

5. In claim 1, A continuous learning method characterized in that the dataset is an image dataset.

6. In claim 5, The calculation steps are: A continuous learning method characterized by calculating the learning difficulty of datasets based on the number of channels whose output value level is higher than a standard when an image dataset is input to a pre-trained deep learning model.

7. In claim 6, The calculation steps are The more channels there are with output value levels above the standard, the lower the learning difficulty is calculated. A continuous learning method characterized in that the learning difficulty is calculated to be higher as the number of channels with output value levels higher than a standard decreases.

8. In claim 1, The deep learning model is, A continuous learning method characterized by not being expanded during the continuous learning process.

9. In claim 8, The deep learning model is, A continuous learning method characterized by being installed on a mobile device with limited resources.

10. A communication interface for acquiring datasets to be used for continuous learning for a pre-trained deep learning model; A deep learning computing device characterized by including a computing unit that calculates the learning difficulty of datasets, determines the learning order of datasets according to the calculated learning difficulty, and continuously trains a deep learning model with datasets according to the determined learning order.

11. A step of determining the learning order of datasets based on the learning difficulty of the datasets to be used for continuous learning for a pre-trained deep learning model; and A continuous learning method, characterized by including a step of continuously learning a deep learning model using datasets according to a determined learning order.

12. A computation unit that determines the learning order of datasets to be used for continuous learning for a pre-trained deep learning model based on the learning difficulty of the datasets, and continuously trains the deep learning model with the datasets according to the determined learning order; and A deep learning computing device characterized by including a memory that provides storage space required for the computing device.

Images