A method and apparatus for optimizing a neural network model on a device using a sub-kernel search module

Abstract

Disclosed is a method for optimizing a neural network model on a device using a sub-kernel search module, which includes: a learning device (a) training a large neural network model having a maximum capacity capable of performing a target task using a maximum computing capacity allowed by an edge device on which the neural network model is installed into a state of generating a first inference result for input data, causing a sub-kernel search module to obtain architecture information of a specific sub-kernel suitable for performing a target task on learning data by identifying constraints and state vectors corresponding to the learning data; (b) optimizing the large neural network model with reference to the architecture information of the specific sub-kernel to generate a specific small neural network model for generating a second inference result for the learning data; and (c) training the sub-kernel search module with reference to the first inference result and the second inference result.

Description

Technical Field

[0001] This invention relates to a method and apparatus for optimizing an on-device neural network model, and more specifically to a method for optimizing an on-device neural network model using a Sub-Kernel Searching Module (SSM), as well as a learning apparatus and optimization apparatus using the SSM, which trains the Sub-Kernel Searching Module for optimizing the on-device neural network model. Background Technology

[0002] Due to various factors, such as the brightness, jitter, and noise of the input image, devices using image classification may have difficulty identifying at least a portion of the input image.

[0003] Therefore, the accuracy of image classification can be improved by applying deep learning to continuously learn from various images.

[0004] However, in order to improve the accuracy of image classification, the neural network model needs to be continuously optimized. Accordingly, various techniques have been developed for effectively updating the neural network model.

[0005] Therefore, in existing technologies, people directly design deep learning network models, but recently methods such as Neural Architect Search (NAS) are being developed, which directly design neural network models for devices using artificial intelligence neural networks.

[0006] As an example, see Figure 1Existing NAS (Neural Network Architecture) designs child neural network models by generating configuration strings containing architectural information about the neural network model to be applied or to a device using recurrent neural networks (RNNs). The performance of the child neural network model is then measured using a validation set, and the resulting accuracy R is used as a reward for reinforcement learning to update the parameters of the child neural network model. Furthermore, techniques such as gradient descent optimization and evolutionary search are being applied to achieve optimal network performance.

[0007] Furthermore, refer to Figure 2 Another existing neural network optimization technique includes methods such as online learning. In these online learning methods, after performing online inference on an existing neural network model, the existing neural network model is updated or relearned based on the results.

[0008] Thus, using neural networks to optimize neural network model architectures is useful for edge devices installed in driving devices that require deep learning technologies, such as autonomous vehicles, unmanned aerial vehicles, and robots. Existing methods using neural network models that perform specific functions on edge devices have the following problems.

[0009] First, neural network models installed on edge devices have limited model capacity because the amount of computation and the number of parameters allowed are relatively small.

[0010] Furthermore, the model capacity required for detecting and recognizing images varies depending on the difficulty of multiple images, but existing neural network models installed on edge devices apply the same amount of computation and parameters to all images, thus the bias in detection and recognition capabilities increases with the difficulty of multiple images.

[0011] In addition, since neural network models installed on edge devices have difficulty finding hard samples without human intervention, improving the performance of neural network models by learning from hard samples requires a lot of manpower and cost.

[0012] Therefore, an improved solution is needed to address the aforementioned problem. Summary of the Invention

[0013] Technical issues

[0014] The purpose of this invention is to solve all the above-mentioned problems.

[0015] Another objective of this invention is to optimize neural network models installed on edge devices without the need for external servers.

[0016] Furthermore, another objective of the present invention is to generate a sub-architecture of the neural network model based on the computing power of the edge device and the surrounding environment in order to optimize the neural network model installed on the edge device.

[0017] Another objective of this invention is to reduce the size of the weights and the number of computations (mac / flops) read during inference by applying computational load optimized according to the difficulty of the input data.

[0018] Furthermore, another object of the present invention is to improve the performance of each sub-architecture of the neural network model by continuously collecting multiple hard samples based on each computational cost of each sub-architecture.

[0019] Technical solution

[0020] In order to achieve the above-mentioned objectives of the present invention and to realize the characteristic effects of the present invention described later, the characteristic structure of the present invention is as follows.

[0021] According to one aspect of the present invention, a method for optimizing an on-device neural network model using a Sub-Kernel Searching Module (SSM) is disclosed, comprising: (a) training a Big Neural Network Model (B-NNModel) with a maximum capacity capable of performing a target task using the maximum computing power allowed by the edge device on which the neural network model is installed, to generate a first inference result for input data; and, upon obtaining learning data, the learning device performing or supporting the following processing: inputting the learning data to the Sub-Kernel Searching Module, causing the Sub-Kernel Searching Module to (i) identify at least one constraint corresponding to the learning data, wherein the constraint includes at least one of computational load, parameters, latency, accuracy, and floating-point operations; and (ii) generating a state vector corresponding to the learning data. (iii) Generate architecture information of a specific sub-kernel consisting of the minimum kernel size and minimum number of channels capable of performing the target task on the learning data having the state vector under the constraints, wherein the specific sub-kernel is a subset of the super kernel corresponding to the maximum capacity of the large neural network model, consisting of a kernel size and a number of channels less than or equal to the super kernel; (b) The learning device performs or supports the following processing: (i) Optimize the large neural network model with reference to the architecture information of the specific sub-kernel to generate a specific small neural network model (S-NN Model) having the architecture information of the specific sub-kernel from the large neural network model; (ii) Cause the specific small neural network model to generate a second inference result for the learning data. (c) The learning device performs or supports the following processing: generating at least one first loss with reference to the first inference result and the second inference result, and training the subkernel search module using the first loss.

[0022] As an example, in (a), the learning device performs or supports performing the following processes: causing the sub-kernel search module (i) to generate multiple scene features corresponding to the learning data through the Scene Overviewing Module (SSM) and to generate the state vector corresponding to the multiple scene features, and (ii) to generate the architecture information of the specific sub-kernel consisting of the minimum kernel size and the minimum number of channels that are capable of performing the target task on the learning data having the state vector under the constraints through the Sub-kernel Network Generator (SNG).

[0023] As an example, the learning device performs or supports the following processing: (i) training the scene overview module and the sub-kernel network generator to minimize the first loss through back-propagation of the first loss under the given constraints; (ii) generating at least one second loss with reference to the state vector and the corresponding ground truth, and further training the scene overview module to minimize the second loss.

[0024] As an example, the scene overview module is composed of a convolutional neural network (CNN), the sub-kernel network generator is composed of a recurrent neural network (RNN), and the learning device performs or supports the following processing: training the scene overview module and the sub-kernel network generator separately or simultaneously.

[0025] As an example, the learning device performs or supports the following processing: applying at least one convolution operation, at least one pooling operation, and at least one fully-connected operation to the learning data through the convolutional neural network of the scene overview module, thereby generating multiple scene features corresponding to the learning data.

[0026] As an example, in (b), the learning device performs or supports the following process: adjusting at least one of the kernel height, kernel width, and number of channels of the superkernel of the large neural network model according to the architecture information of the specific subkernel to optimize the large neural network model, so as to generate the specific small neural network model with the architecture information of the specific subkernel.

[0027] As an example, the learning device performs or supports the following processing: referring to the first inference result and the second inference result, when the difference between the first inference result and the second inference result exceeds a preset benchmark value, the learning data and the architecture information of the specific small neural network model are transmitted to the server, so that the server tags the learning data as a hard sample in the specific small neural network model and adds it to the learning database.

[0028] According to another aspect of the present invention, a method for optimizing an on-device neural network model using a Sub-Kernel Searching Module (SSM) is disclosed, comprising: (a) when learning data is obtained, in a state where a learning device (i) has trained a Big Neural Network Model (B-NN Model) with a maximum capacity capable of performing a target task using the maximum computing power allowed by the edge device on which the neural network model is installed to generate a first inference result for input data, the learning data is input to the Sub-Kernel Searching Module, causing the Sub-Kernel Searching Module (i-1) to identify at least one learning constraint corresponding to the learning data, wherein the learning constraint includes at least one of the constraints applied to computational load, parameters, latency, accuracy, and floating-point operations; and (i-2) generating a learning state vector corresponding to the learning data. (i-3) Generate learning architecture information consisting of the minimum kernel size and minimum number of channels for the learning data with the learning state vector under the learning constraints, wherein the learning architecture information consists of the minimum kernel size and minimum number of channels for the learning data with the learning state vector under the learning constraints, wherein the learning architecture information consists of a subset of the superkernel with the maximum capacity of the large neural network model, consisting of a kernel size and a number of channels less than or equal to the superkernel; (ii-1) Optimize the large neural network model with reference to the learning architecture information of the learning architecture information of the learning architecture information of the learning architecture information of the learning architecture information of the learning architecture information of the learning architecture information of the learning architecture information of the learning architecture information of the learning architecture information of the learning architecture information of the learning architecture information of the learning architecture information of the learning architecture; (ii-2) Make the learning architecture information of the learning architecture information of the learning architecture information of the learning architecture information of the learning architecture information of the learning architecture information of the learning architecture information of the learning architecture information of the learning architecture information of the learning architecture;And (iii) with reference to the first inference result and the second inference result, at least one first loss is generated, and the sub-kernel search module is trained using the first loss, the optimization device performs or supports the following processing: when test data is obtained, the test data is input to the sub-kernel search module so that the sub-kernel search module (i) confirms at least one test constraint corresponding to the test data, wherein the test constraint includes at least one constraint applicable to computational complexity, parameters, latency, accuracy, and floating-point arithmetic; (ii) generates a test state vector corresponding to the test data, wherein the test state vector includes at least one information regarding the complexity and clarity of the test data; and (iii) generates a system capable of processing the test data under the test constraint. The test architecture information comprises a test-specific sub-kernel consisting of the minimum kernel size and minimum number of channels for executing the target task with the test data having the test state vector, wherein the test-specific sub-kernel is a subset of the superkernel corresponding to the maximum capacity of the large neural network model, consisting of a kernel size and a number of channels less than or equal to the superkernel; and (b) the optimization device performs or supports performing the following processes: (i) optimizing the large neural network model with reference to the test architecture information of the test-specific sub-kernel to generate a test-specific small neural network model with the test architecture information of the test-specific sub-kernel from the large neural network model, and (ii) causing the test-specific small neural network model to generate a third inference result for the test data.

[0029] As an example, in (a), the optimization device performs or supports performing the following processes: causing the sub-kernel search module (i) to generate multiple test scene features corresponding to the test data through the Scene Overviewing Module (SSM) and to generate test state vectors corresponding to the multiple test scene features; and (ii) to generate test architecture information for a test-specific sub-kernel consisting of the minimum kernel size and minimum number of channels capable of performing the target task on the learning data having the test state vector under the test constraints through the Sub-kernel Network Generator (SNG).

[0030] As an example, the scene overview module is composed of a convolutional neural network (CNN), and the sub-kernel network generator is composed of a recurrent neural network (RNN).

[0031] As an example, the optimization device performs or supports the following processing: applying at least one convolution operation, at least one pooling operation, and at least one fully-connected operation to the test data through the convolutional neural network of the scene overview module, thereby generating multiple test scene features corresponding to the test data.

[0032] As an example, in (b), the optimization device performs or supports the following process: adjusting at least one of the kernel height, kernel width, and number of channels of the superkernel of the large neural network model according to the test architecture information of the test-specific subkernel to optimize the large neural network model, so as to generate the test-specific small neural network model having the test architecture information of the test-specific subkernel.

[0033] According to another aspect of the invention, a learning apparatus for optimizing an on-device neural network model using a sub-kernel search module (SSM) is also disclosed, comprising: at least one memory storing a plurality of instructions; and at least one processor for executing the plurality of instructions, the processor performing or supporting the following processing: (I) training a Big Neural Network Model (B-NN Model) with a maximum capacity capable of executing a target task using the maximum computing power allowed by the edge device on which the neural network model is installed, to generate a first inference result for input data, and upon obtaining learning data, the learning apparatus performing or supporting the following processing: inputting the learning data to the sub-kernel search module, causing the sub-kernel search module (i) to identify at least one constraint corresponding to the learning data, wherein the constraint includes at least one constraint applied to computational complexity. (ii) Constraints of one of load, parameters, latency, accuracy, and floating-point operations; (iii) Generating a state vector corresponding to the learning data, wherein the state vector includes at least information on one of complexity and clarity of the learning data; (iv) Generating architecture information of a specific sub-kernel consisting of the minimum kernel size and minimum number of channels capable of performing the target task on the learning data having the state vector under the constraints, wherein the specific sub-kernel is a subset of the super kernel of the large neural network model corresponding to the maximum capacity, consisting of a kernel size and a number of channels less than or equal to the super kernel; (ii) Optimizing the large neural network model with reference to the architecture information of the specific sub-kernels, generating a specific small neural network model (S-NN Model) with the architecture information of the specific sub-kernels from the large neural network model; (iii) Enabling the specific small neural network model to generate a second inference result for the learning data. (III) Generate at least one first loss with reference to the first inference result and the second inference result, and train the sub-kernel search module using the first loss.

[0034] As an example, in (I), the processor performs or supports performing the following processing: causing the sub-kernel search module (i) to generate multiple scene features corresponding to the learning data through the Scene Overviewing Module (SSM) and to generate the state vector corresponding to the multiple scene features, and (ii) to generate the architecture information of the specific sub-kernel consisting of the minimum kernel size and the minimum number of channels that are capable of performing the target task on the learning data having the state vector under the constraints through the Sub-kernel Network Generator (SNG).

[0035] As an example, the processor performs or supports performing the following processes: (i) training the scene overview module and the sub-kernel network generator to minimize the first loss through back-propagation of the first loss under the given constraints; and (ii) generating at least one second loss with reference to the state vector and the corresponding ground truth, and further training the scene overview module to minimize the second loss.

[0036] As an example, the scene overview module is composed of a convolutional neural network (CNN), and the sub-kernel network generator is composed of a recurrent neural network (RNN). The processor performs or supports the following processing: training the scene overview module and the sub-kernel network generator separately or simultaneously.

[0037] As an example, the processor performs or supports performing the following processing: applying at least one convolution operation, at least one pooling operation, and at least one fully-connected operation to the learning data through the convolutional neural network of the scene overview module, thereby generating multiple scene features corresponding to the learning data.

[0038] As an example, in (II), the processor performs or supports performing the following process: adjusting at least one of the kernel height, kernel width, and number of channels of the superkernel of the large neural network model according to the architecture information of the specific subkernel to optimize the large neural network model, so as to generate the specific small neural network model with the architecture information of the specific subkernel.

[0039] As an example, the processor performs or supports the following processing: referring to the first inference result and the second inference result, when the difference between the first inference result and the second inference result exceeds a preset benchmark value, the learning data and the architecture information of the specific small neural network model are transmitted to the server, so that the server tags the learning data as a hard sample in the specific small neural network model and adds it to the learning database.

[0040] According to another aspect of the present invention, an optimization apparatus for optimizing an on-device neural network model using a sub-kernel search module (SSM) is also disclosed, comprising: at least one memory storing a plurality of instructions; and at least one processor for executing the plurality of instructions, performing the following process: (I) when the learning device (i) has trained a BigNeural Network Model (B-NN Model) with the maximum capacity allowed by the edge device equipped with the neural network model to generate a first inference result for input data, upon obtaining learning data, the learning data is input to the sub-kernel search module, causing the sub-kernel search module (i-1) to identify at least one learning constraint corresponding to the learning data, wherein the learning constraint includes at least one constraint applied to computational complexity. (i-2) Given a constraint of one of load, parameters, latency, accuracy, and floating-point operations, generate a learning state vector corresponding to the learning data, wherein the learning state vector includes at least information regarding the complexity and clarity of the learning data; (i-3) Generate learning architecture information consisting of the minimum kernel size and minimum number of channels required to perform the target task on the learning data with the learning state vector under the learning constraints, wherein the learning-specific sub-kernel is a subset corresponding to the maximum capacity super kernel of the large neural network model, consisting of a kernel size and a number of channels less than or equal to the super kernel; (ii) Optimize the large neural network model with reference to the learning architecture information of the learning-specific sub-kernel, and generate a learning-specific small neural network model with the learning architecture information of the learning-specific sub-kernel from the large neural network model. (ii-2) The process of generating a second inference result for the learning data using a specific small neural network model (NetworkModel, S-NN Model);And (iii) with reference to the first inference result and the second inference result, generating at least one first loss, and using the first loss to train the sub-kernel search module, the processor performs or supports performing the following processing: when obtaining test data, inputting the test data into the sub-kernel search module, so that the sub-kernel search module (i) confirms at least one test constraint corresponding to the test data, wherein the test constraint includes constraints applicable to at least one of computational complexity, parameters, latency, accuracy, and floating-point arithmetic, (ii) generates a test state vector corresponding to the test data, wherein the test state vector includes at least information on the complexity and clarity of the test data, and (iii) generates a system capable of processing the test data under the test constraint. The test architecture information comprises the minimum kernel size and minimum number of channels of the test data executing the target task with the test state vector, wherein the test-specific sub-kernel is a subset of the superkernel corresponding to the maximum capacity of the large neural network model, consisting of a kernel size and a number of channels less than or equal to the superkernel; and (II) the processor performs or supports the following processing: (i) optimizing the large neural network model with reference to the test architecture information of the test-specific sub-kernel to generate a test-specific small neural network model with the test architecture information of the test-specific sub-kernel from the large neural network model, and (ii) causing the test-specific small neural network model to generate a third inference result for the test data.

[0041] As an example, in (I), the processor performs or supports performing the following processes: causing the sub-kernel search module (i) to generate multiple test scene features corresponding to the test data through the Scene Overviewing Module (SSM) and to generate the test state vector corresponding to the multiple test scene features; and (ii) to generate the test architecture information of the test-specific sub-kernel consisting of the minimum kernel size and minimum number of channels capable of performing the target task on the learning data having the test state vector under the test constraints through the Sub-kernel Network Generator (SNG).

[0042] As an example, the scene overview module is composed of a convolutional neural network (CNN), and the sub-kernel network generator is composed of a recurrent neural network (RNN).

[0043] As an example, the processor performs or supports performing the following processing: applying at least one convolution operation, at least one pooling operation, and at least one fully-connected operation to the test data through the convolutional neural network of the scene overview module, thereby generating multiple test scene features corresponding to the test data.

[0044] As an example, in (II), the processor performs or supports performing the following process: adjusting at least one of the kernel height, kernel width, and number of channels of the super kernel of the large neural network model according to the test architecture information of the test-specific sub-kernel to optimize the large neural network model, so as to generate the test-specific small neural network model having the test architecture information of the test-specific sub-kernel.

[0045] In addition, the present invention also provides a computer-readable recording medium for recording a computer program for performing the method of the present invention.

[0046] Beneficial effects

[0047] The advantage of this invention is that it optimizes neural network models installed on edge devices without the need for external servers.

[0048] Furthermore, another advantage of the present invention is that it generates a sub-architecture of the neural network model based on the computing power of the edge device and the surrounding environment to optimize the neural network model installed on the edge device.

[0049] Another effect of the present invention is that it reduces the size of the weights read and the number of computations (mac / flops) during inference by applying computational load optimized according to the difficulty of the input data.

[0050] Furthermore, another effect of the present invention is to improve the performance of each sub-architecture of the neural network model by continuously collecting multiple hard samples based on each computational cost of each sub-architecture. Attached Figure Description

[0051] The following drawings, which are used to describe embodiments of the present invention, are only a part of the embodiments of the present invention, and those skilled in the art to which the present invention pertains (hereinafter referred to as "skilled persons") can obtain other drawings based on these drawings without any creative work.

[0052] Figure 1 and Figure 2 A schematic diagram of an existing neural network optimization system.

[0053] Figure 3 This is a schematic diagram of a learning apparatus for training a sub-kernel search module in a method for optimizing an on-device neural network model using a sub-kernel search module (SSM) according to one aspect of the present invention.

[0054] Figure 4 This diagram illustrates the process of training a subkernel search module in a method for optimizing a neural network model on a device using a subkernel search module, according to an embodiment of the present invention.

[0055] Figure 5 This is a structural diagram of the SceneOverviewing Module (SOM) included in the sub-kernel search module according to an embodiment of the present invention.

[0056] Figure 6 This diagram illustrates a method for generating and training a Big Neural Network Model (B-NN Model) and a Small Neural Network Model (S-NN Model) using the concepts of a super kernel and a sub-kernel, according to an embodiment of the present invention.

[0057] Figure 7 This is a schematic diagram of an optimization apparatus for optimizing a neural network model on a device using a subkernel search module according to an embodiment of the present invention.

[0058] Figure 8 This is a diagram illustrating a method for optimizing a neural network model on a device using a subkernel search module, based on an example of inputs and outputs according to an embodiment of the present invention. Detailed Implementation

[0059] The following detailed description of the invention is illustrated in the accompanying drawings, which show specific embodiments in which the invention can be practiced to illustrate the objectives, technical solutions, and advantages of the invention. These embodiments are described in sufficient detail to enable those skilled in the art to practice the invention. Furthermore, the invention includes all possible combinations of the embodiments shown in this specification. It should be understood that the various embodiments of the invention, although different, are not necessarily mutually exclusive. For example, specific shapes, structures, and characteristics described herein may be implemented in one embodiment by other embodiments without departing from the spirit and scope of the invention. Furthermore, it should be understood that the position or configuration of the components in each disclosed embodiment may be varied without departing from the spirit and scope of the invention. Therefore, the detailed description that follows is not intended to limit the invention; the scope of the invention should be defined by all scopes equivalent to the scope of its claims and the appended claims, provided that appropriate description is possible. Similar reference numerals in the drawings indicate the same or similar functions in several respects.

[0060] The various images involved in this invention can include images related to paved or unpaved roads. In this case, it can be assumed that objects that may appear in a road environment (e.g., cars, people, animals, plants, objects, buildings, aircraft such as airplanes or drones, and other obstacles) can be present, but are not necessarily limited to this. The various images involved in this invention can also be images unrelated to roads (e.g., images related to unpaved roads, alleys, open spaces, seas, lakes, rivers, mountains, forests, deserts, skies, and interiors). In this case, it can be assumed that objects that may appear in unpaved roads, alleys, open spaces, seas, lakes, rivers, mountains, forests, deserts, skies, and interior environments (e.g., cars, people, animals, plants, objects, buildings, aircraft such as airplanes or drones, and other obstacles) can be present, but are not necessarily limited to this. The titles and abstracts of the disclosure provided herein are for convenience only and should not be construed as limiting the scope or meaning of the embodiments.

[0061] For reference, throughout the manual, terms related to the learning process have been replaced with "for learning" or "learning", while terms related to the testing process have been replaced with "for testing" or "testing" to avoid potential confusion.

[0062] Furthermore, in the description and claims of this invention, the term "comprising" and its variations are not intended to exclude other technical features, additions, components, or steps. Other objects, advantages, and features of this invention will be apparent to those skilled in the art, in part from this specification and in part from practice of the invention. The following illustrations and figures are provided as examples and are not intended to limit the scope of the invention.

[0063] The titles and summaries of the disclosures provided herein are for convenience only and should not be construed as limiting the scope or meaning of the embodiments.

[0064] To enable those skilled in the art to readily implement the present invention, preferred embodiments of the invention will now be described in detail with reference to the accompanying drawings.

[0065] first, Figure 3 This is a schematic diagram of a learning apparatus for training a sub-kernel search module in a method for optimizing an on-device neural network model using a sub-kernel search module (SSM) according to one aspect of the present invention.

[0066] See Figure 3 The learning device 1000 may include: a memory 1001 storing a plurality of instructions for training a subkernel search module for optimizing a neural network model on the device; and a processor 1002 that, in response to the plurality of instructions stored in the memory 1001, performs operations to train the subkernel search module.

[0067] Specifically, the learning device 1000 can achieve the required system performance using a combination of computing devices (e.g., computer processors, memory, storage devices, input and output devices, and other components that may include conventional computing devices; electronic communication devices, such as routers, switches, etc.; electronic information storage systems, such as network attached storage (NAS) and storage area networks (SAN)) and computer software (i.e., instructions that enable the computing device to operate in a particular manner).

[0068] Additionally, the processor of a computing device may include hardware configurations such as a microprocessor unit (MPU) or central processing unit (CPU), cache memory, and a data bus. Furthermore, the computing device may include a software configuration such as an operating system and applications for executing specific purposes.

[0069] However, it is not excluded that the computing device may include an integrated processor in the form of an integrated medium, processor, and memory for the purpose of implementing the present invention.

[0070] The following will be referred to Figures 4 to 6 The process of training the subkernel search module 100 using the learning device 100 configured as described above is described.

[0071] First see Figure 4The learning device 1000 can perform or support the following processing: when training a Big Neural Network Model (B-NN Model) 200 with the maximum capacity allowed by the edge device equipped with the neural network model to perform the target task, in a state that generates the first inference result for the input data, when learning data is obtained, the learning data is input to the sub-kernel search module 100, so that the sub-kernel search module (i) confirms the constraints corresponding to the learning data, (ii) generates the state vector corresponding to the learning data, and (iii) generates architecture information of a specific sub-kernel consisting of the minimum kernel size and the minimum number of channels that can perform the target task for the learning data with the state vector under the given constraints.

[0072] The constraints include at least one of the following: computational load, parameters, latency, accuracy, and floating-point operations (flops). These constraints can vary depending on the device state of the edge device and its surrounding environment when the learning data is input. Furthermore, the constraints can be pre-defined by each edge device or modified by additional input. Additionally, the target task described herein can perform data analysis, classification, or detection of information contained in the data. Depending on the purpose, the type of data used can be text, images, or images containing text, but the invention is not limited to these.

[0073] Furthermore, the state vector can be a one-dimensional vector that includes at least information regarding the complexity and clarity of the learning data. Correspondingly, the state vector can be a criterion used to determine the difficulty level of the learning data. Next, a specific sub-kernel is a subset of the super kernel corresponding to the maximum capacity of a large neural network model, and can consist of a kernel size and a number of channels less than or equal to the super kernel.

[0074] On the other hand, the sub-kernel search module 100 may consist of a Scene Overviewing Module (SSM) 110 and a Sub-kernel Network Generator (SNG) 120.

[0075] Specifically, the learning device 1000 can perform or support the following processes: (i) the subkernel search module 100 generates multiple scene features corresponding to the learning data through the scene overview module 110, and generates state vectors corresponding to the multiple scene features; (ii) the subkernel network generator 120 generates architecture information of a specific subkernel consisting of the minimum kernel size and the minimum number of channels capable of performing the target task on the learning data having the state vector under the constraints.

[0076] The computational load, parameters, latency, accuracy, and floating-point operations can be interrelated and influence each other due to given constraints. For example, as the required accuracy increases, the minimum number of parameters required may increase accordingly, and when the computational load decreases, such as by reducing floating-point operations or increasing latency, the allowed number of parameters may decrease. Therefore, the architectural information of a specific sub-kernel can include information about the minimum kernel size and the minimum number of channels, which correspond to the minimum capacity to perform the target task on learning data with a given state vector under given constraints.

[0077] At this time, the scene overview module 110 can be composed of a small convolutional neural network (CNN) that will not significantly affect the overall performance of the device, and the sub-kernel network generator 120 can be composed of a small recurrent neural network (RNN) that will not significantly affect the overall performance.

[0078] See Figure 5 The scene overview module 110, as an example, can consist of convolutional layers, pooling layers, and fully-connected layers. When learning data is input into the scene overview module 110, multiple feature maps generated by the alternating convolutional and pooling layers can be input into the fully-connected layer to generate output predictions. At this point, the output predictions can be a one-dimensional vector of state, i.e., the output of the scene overview module (SOM) 110, or they can be converted into a state vector through a series of processes.

[0079] Furthermore, the data type of the architecture information of a specific sub-kernel output by the sub-kernel network generator 120 can be a dictionary or a string. For example, the architecture information can be output in the form of {{conv1: {k=3, c=26}}, {conv2: {k=7, c=35}}, ..., {ssd_layer: {k=1, c=121}}}. That is, the sub-kernel network generator 120 can generate information about multiple parameters of the architecture of the sub-kernel corresponding to the state vector, wherein the state vector reflects information about multiple scene features generated under given constraints.

[0080] Next, see you again. Figure 4 The learning device 1000 can perform or support the following processes: (i) optimizing a large neural network model 200 with reference to the architecture information of a specific sub-kernel to generate a specific small neural network model (S-NN Model) 300 with the architecture information of a specific sub-kernel from the large neural network model 200; and (ii) enabling the specific small neural network model 300 to generate a second inference result for the learning data.

[0081] The learning device 1000 can perform or supports performing the following process: adjusting at least one of the kernel height, kernel width, and number of channels of the super kernel of the large neural network model 200 according to the architectural information of a specific sub-kernel to optimize the large neural network model 200, so as to generate a specific small neural network model 300 with the architectural information of the specific sub-kernel. That is, the present invention adjusts at least one of the kernel size and number of channels of the super kernel of the large neural network model 200 by adjusting at least one of the kernel height, kernel width, and number of channels, thereby changing the size or volume of the super kernel of the large neural network model 200.

[0082] Specifically, Figure 6This diagram illustrates a method for generating and training a large neural network model (B-NN Model) 200 and a small neural network model (S-NN Model) 300 using the concepts of a super kernel and a sub-kernel, according to an embodiment of the present invention. The large neural network model (B-NN Model) 200 is the neural network model designed by humans, representing the maximum capacity that the computing power of an edge device, such as an autonomous vehicle, autonomous aircraft, or robot, can handle—that is, a neural network model with the maximum computational load and the maximum number of parameters.

[0083] In a large neural network model, the superkernel refers to the kernel within that model, which consists of the maximum superkernel size, the maximum kernel width, and the maximum number of channels, i.e., (max_kernel_w * max_kernel_h * max_ch). In other words, the superkernel can represent a three-dimensional structure, for example, composed of a two-dimensional kernel on the XY plane and one-dimensional channels arranged along the Z-axis perpendicular to the XY plane. Therefore, for ease of description, the following superkernel can be briefly described as having the maximum kernel size and the maximum number of channels.

[0084] In addition, existing one-shot neural architecture search methods can be used as a method for generating and training large neural network models (B-NN Model)200 and small neural network models (S-NN Model)300.

[0085] In particular, when training the superkernel of a large neural network model 200, if the kernel size and number of channels of the superkernel are controlled by training some parameters belonging to a subset of the superkernel at each step, the problem of sudden performance drop can be prevented even if the kernel size and channel size change slightly during the inference of the neural network model, and the performance can be adjusted according to the reduced amount of computation and the number of parameters.

[0086] See Figure 6For example, when training a large neural network model (B-NN Model) 200, the following method can be used to train some parameters belonging to a subset of the super kernel at each step: For the first convolutional layer (conv1) consisting of a kernel size (7*7) and 128 channels corresponding to the kernel size and number of channels of the super kernel, inference and training are performed using the 3*3*64 kernel shape in the middle of the super kernel in the first step and the 5*5*64 kernel shape in the second step.

[0087] On the other hand, for specific methods of training neural network models while adjusting the kernel size and number of channels of the superkernel, refer to papers such as Single-Path NAS: Designing Hardware-Efficient ConvNets in lessthan 4 Hours (Dimitrios Stamoulis, 2019) and Universally Slimmable Networks and Improved Training Techniques (Jiahui Yu, 2019). As an example, by temporarily zeroing out the outer region of the kernel used for inference and learning, performance can be maintained regardless of the kernel size used. Furthermore, if batch statistics are calculated again using a large amount of data after learning, rapid performance degradation due to channel changes can be prevented.

[0088] Therefore, this invention generates and trains a large neural network model (B-NN Model) 200 with a superkernel, and then a small neural network model (S-NN Model) 300 can be generated by optimizing the large neural network model (B-NN Model) 200. At this point, the small neural network model (S-NN Model) 300 can be a neural network model composed of the smallest kernel size and the smallest number of channels, designed to perform a target task on learning data of a given specific difficulty under given constraints, such constraints including computational complexity, parameters, latency, accuracy, and floating-point operations.

[0089] Next, see you again. Figure 4 The learning device 1000 can perform or support the following processing: generating at least one first loss by referring to the first inference result and the second inference result, and training the scene overview module 110 and the sub-kernel network generator 120 by back-propagation of the first loss under given constraints to minimize the first loss.

[0090] Furthermore, the learning device 1000 can perform or support the following processing: generating at least one second loss from a reference state vector and its corresponding ground truth, and further training the scene overview module 110 to minimize the second loss. Thus, the scene overview module 110 can be optimized in the direction of providing accurate information about the scene to the sub-kernel network generator 120. Here, the ground truth corresponding to the state vector can be generated and learned by humans or highly trained neural networks; alternatively, it can also be performed through unsupervised learning methods such as using a Generative Adversarial Network (GAN), but the invention is not limited thereto.

[0091] As described above, the Scene Overview Module (SOM) 110 and the Sub-Kernel Network Generator 120 constituting the Sub-Kernel Search Module 100 can be composed of a Convolutional Neural Network (CNN) and a Recurrent Neural Network (RNN), respectively. Therefore, the Scene Overview Module 110 and the Sub-Kernel Network Generator 120 can be learned separately or together using an ensemble learning method. One ensemble learning method is CNN+RNN combined training, which applies scene understanding methods; see methods such as Show and Tell: A Neural Image Caption Generator (Oriol Vinyals, 2015).

[0092] Furthermore, during the learning process of the scene overview module 110 by the learning device 1000, the learning device 1000 can learn in the following way: by backpropagating the first loss between the large neural network model (B-NN model) 200 and the small neural network model (S-NN Model) 300, the scene overview module 110 minimizes the first loss under at least one given constraint, such as computational cost, parameters, latency, accuracy, and floating-point operations, and provides accurate information to the sub-kernel network generator 120. At this time, if the first loss for the input learning data is greater than a predetermined threshold, it may include information that the capacity of the large neural network model (B-NN Model) 200 for the learning data is sufficient, but the capacity of the small neural network model (S-NN Model) 300 is insufficient.

[0093] As an example, the state vector output from the Scene Overview Module (SOM) 110 can reflect scene information corresponding to scene features. This scene information can reflect the complexity of the foreground and background of the learning data, the clarity of the data, etc., but the invention is not limited to this. The more complex or unclear the learning data, the more difficult it is to identify as a hard sample. In this case, the learning by the Scene Overview Module 110 can be performed in directions requiring more computation and more parameters. Furthermore, the performance of the Scene Overview Module 110 can be adjusted based on the performance of the edge device with the installed neural network model, the size of the original network, and the target performance (e.g., speed, accuracy).

[0094] Furthermore, in the learning process of the sub-kernel network generator 120 by the learning device 1000, the learning device 1000 can learn in the following direction: by backpropagating using the first loss, the sub-kernel network generator 120 minimizes the first loss under at least one given constraint among computational cost, parameters, latency, accuracy, floating-point operations, etc. At this time, for the learning of the sub-kernel network generator 120, methods such as reinforcement learning optimization or gradient descent optimization can be selected based on the differentiability of the constraints.

[0095] On the other hand, when the learning device 1000 trains the sub-kernel search module 100 as described above, if the first loss for the input learning data is greater than a preset threshold, that is, if the difference between the first inference result and the second inference result exceeds a preset benchmark value, it indicates that the learning data is not a hard sample in the large neural network model (B-NN Model) 200, but may be a hard sample in the small neural network model (S-NN Model) 300. Therefore, in this case, the learning device 1000 transmits the learning data and the information of the learning small neural network model (S-NN Model) 300 architecture generated by the sub-kernel search module to the server, so that the server can execute or support the execution of the process of tagging the learning data as a hard sample in the learning small neural network model (S-NN Model) 300 and adding it to the learning database. The preset threshold or preset benchmark value may be the same or different.

[0096] However, unlike this, when a separate computing device is needed to perform hardsample search based on model size, the subkernel search module 100 can be used as a hardsample searcher. Specifically, when the first loss exceeds a preset threshold, it indicates that the computational complexity of the small neural network model (S-NN Model) 300 is insufficient to solve a problem that the computational complexity of the large neural network model (B-NN Model) 200 can handle. Therefore, when the first loss exceeds the preset threshold, the information of the specific small neural network model (S-NN Model) 300 architecture generated by the subkernel search module 100 and the learning data can be transmitted to the server to tag the learning data as a hard sample of the specific small neural network model (S-NN Model) 300 with a specific structure, specific computational complexity, specific number of parameters, etc., and add it to the learning database. Therefore, the hard samples stored in the learning database can be used to learn specific small neural network models (S-NN Models) 300 with similar computational complexity and number of parameters in the future. Since the specific small neural network model (S-NN Model) 300 of the present invention can refer to any structure that can be used as a small neural network model (S-NN Model) 300, the small neural network models (S-NN Model) 300 referred to as specific small neural network models (S-NN Model) 300 can be different from each other.

[0097] On the other hand, see below Figure 7 and Figure 8 The description describes an optimization method using the trained subkernel search module 100 to optimize a neural network model on the device, while the subkernel search module 100 is trained as described above.

[0098] first, Figure 7 This is a schematic diagram of an optimization apparatus 2000 that optimizes a neural network model on a device using a subkernel search module 100 according to an embodiment of the present invention.

[0099] See Figure 7 The optimization apparatus may include: a memory 2001 storing multiple instructions for optimizing a neural network model, namely a large neural network model (B-NN model) 200 on the device, using a subkernel search module 100; and a processor 2002 that, in response to the multiple instructions stored in the memory 2001, performs operations to optimize the large neural network model (B-NN model) 200 using the subkernel search module 100. Figure 7 and Figure 8The large neural network model (B-NN Model) 200 is not described here; it serves as a reference for generating and testing specific small neural network models (S-NN Model) 30. However, the description of the large neural network model (B-NN Model) 200 provided during the training of the subkernel search module 100 still applies. Figure 7 and Figure 8 .

[0100] Specifically, the optimization device 2000 can typically achieve the desired system performance using a combination of computing devices (e.g., computer processors, memory, storage devices, input and output devices, and other components that may include conventional computing devices; electronic communication devices such as routers, switches, etc.; electronic information storage systems such as network attached storage (NAS) and storage area networks (SAN)) and computer software (i.e., instructions that enable the computing device to operate in a particular manner).

[0101] Additionally, the processor of a computing device may include hardware configurations such as a microprocessor unit (MPU) or central processing unit (CPU), cache memory, and a data bus. Furthermore, the computing device may include a software configuration such as an operating system and applications for executing specific purposes.

[0102] However, it is not excluded that the computing device may include an integrated processor in the form of an integrated medium, processor, and memory for the purpose of implementing the present invention.

[0103] On the other hand, the optimization device is used to optimize the large neural network model (B-NN Model) 200 using the sub-kernel search module 100 corresponding to each edge device based on the data input to each edge device, and can be installed on the edge device where the neural network model is applied.

[0104] The following describes an optimization method for a neural network model using the optimization device 2000 configured as described above, which utilizes a sub-kernel search module 100 according to an embodiment of the present invention. (The following will omit information that can be derived from...) Figures 4 to 6 The description is a detailed description of the parts that are easy to understand.

[0105] First, the optimization device 2000 can perform or supports performing the following processes: when test data is obtained, the test data is input to the subkernel search module 100 so that the subkernel search module 100 (i) confirms at least one test constraint corresponding to the test data, wherein the test constraint includes at least one of the constraints applied to computational load, parameters, latency, accuracy, and floating-point operations; (ii) generates a test state vector corresponding to the test data, wherein the test state vector includes at least one of the information regarding the complexity and clarity of the test data; and (iii) generates test architecture information consisting of the minimum kernel size and minimum number of channels of a test-specific subkernel capable of performing a target task on test data having a test state vector under test constraints, wherein the test-specific subkernel is a subset of the test superkernel corresponding to the maximum capacity of a large neural network model, consisting of a kernel size and a number of channels less than or equal to the superkernel.

[0106] Secondly, the optimization device 2000 can perform or support performing the following processes: (i) optimizing the large neural network model 200 with reference to the test architecture information of the test-specific sub-kernel, to generate a test-specific small neural network model 300 with test architecture information of the test-specific sub-kernel from the large neural network model 200; and (ii) enabling the test-specific small neural network model 300 to generate a third inference result for the test data. Since the learning data and test data have different levels of difficulty, the architecture information of the learning-specific sub-kernel obtained during the training of the sub-kernel search module 100 may differ from the architecture information of the test-specific sub-kernel obtained during the testing of the sub-kernel search module 100. As a result, since the capacity of the edge device is fixed, the capacity of the large neural network model 200 may not change during the learning and testing process of the sub-kernel search module 100, while the learning small neural network model 300 and the stored test small neural network model 300 may differ from each other.

[0107] The optimization device 2000 can perform or support the following processes: (i) the subkernel search module 100 generates multiple test scenario features corresponding to the test data through the Scene Overviewing Module (SSM) 110, and generates test state vectors corresponding to the multiple test scenario features; (ii) the subkernel network generator 120 generates test architecture information for a specific subkernel for the test, consisting of the minimum kernel size and the minimum number of channels that can perform the target task on the test data with test state vectors under test constraints.

[0108] Next, the optimization device 2000 may perform or support performing the following process: adjusting at least one of the kernel height, kernel width, and number of channels of the test superkernel of the large neural network model according to the test architecture information of the test specific subkernel to optimize the large neural network model, so as to generate a test specific small neural network model 300 with test architecture information of the test specific subkernel.

[0109] Specifically, Figure 8 This is a method diagram illustrating how a subkernel search module 100 optimizes a neural network model on a device using examples of inputs and outputs according to an embodiment of the present invention.

[0110] See Figure 8 The optimization device 2000 optimizes the large neural network model (B-NN Model) 200 with test architecture information of test-specific sub-kernels to generate test-specific small neural network models (S-NN Models) 300_1 and 300_2 with test architecture information of test-specific sub-kernels. The generated test-specific small neural network models (S-NN Models) 300_1 and 300_2 can perform third inference on the test data.

[0111] At this point, since the test-specific small neural network models (S-NN Models) 300_1 and 300_2 are subsets of the large neural network model (B-NN Model) 200, the performance of the test-specific small neural network models (S-NN Models) 300_1 and 300_2 is always less than or equal to the performance of the large neural network model (B-NN Model) 200. Furthermore, as the difficulty of the test data increases, the computational cost required for the test-specific sub-kernels of the test-specific small neural network models (S-NN Models) 300_1 and 300_2 may increase.

[0112] As an example, when the test data is an easy sample, a test-specific small neural network model (S-NN Model) 300_1 with small weights and low computational cost is generated by optimizing the large neural network model (B-NN Model) 200. This allows the third inference of the test data to be performed with less computation than the large neural network model (B-NN model) 200, for example, 12% of the computation of the large neural network model (B-NN model) 200. On the other hand, when the test data is a hard sample, the large neural network model (B-NN Model) 200 can be optimized to generate a test-specific small neural network model (S-NN Model) 300_2 with larger weights and more operations compared to the test-specific small neural network model (S-NN Model) 300_1, but with smaller weights and fewer operations compared to the large neural network model (B-NN model) 200. This allows the test data to be inferred using more computation than the test-specific small neural network model (S-NN Model) 300_1 but less computation than the large neural network model (B-NN model) 200, for example, 88% of the computation of the large neural network model (B-NN model).

[0113] That is, according to the present invention, the test-specific small neural network models (S-NN Models) 300_1 and 300_2 for optimizing the large neural network model (B-NN Model) have optimized computational load according to the difficulty of the samples, and the optimization of the neural network model can be carried out on edge devices without the help of servers or other means.

[0114] Furthermore, the optimization device 2000 references the third inference result obtained by inferring the test data using the test-specific small neural network models (S-NN models) 300_1 and 300_2 and the fourth inference result obtained by inferring the test data using the large neural network model (B-NN Model) 200. When the difference between the third and fourth inference results exceeds a preset benchmark value, the device can transmit the test data and the architecture information of the test-specific small neural network models (S-NN models) 300_1 and 300_2 to the server. The server then marks the test data as hard samples in the test-specific small neural network models (S-NN models) 300_1 and 300_2 and adds it to the learning database.

[0115] At this point, the learning of the sub-kernel search module 100 can be continuously performed on the edge device using data continuously input to the edge device. When the learning of the sub-kernel search module 100 on the edge device progresses to a certain extent, it can be tested and inferred using the small neural network model (S-NN Model) 300. Depending on the overhead of the sub-kernel search module 100, it can be selected whether to execute the sub-kernel search module 100 every frame to output the architecture information of the new small neural network model (S-NN Model) 300, or to execute the sub-kernel search module 100 every predetermined frame and finally output the architecture information in the corresponding segment.

[0116] The embodiments described above according to the present invention can be implemented and recorded in a computer-readable recording medium in the form of program instructions executable by various computer components. The computer-readable recording medium may include individual or combined program instructions, data files, data structures, etc. The program instructions recorded in the computer-readable recording medium may be specifically designed and configured for the present invention, or may be known and available to those skilled in the art of computer software. Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks, and magnetic tapes; optical recording media such as CD-ROMs and DVDs; magneto-optical media such as floppy disks; and hardware devices specifically configured for storing and executing program instructions, such as ROMs, RAMs, flash memory, etc. Examples of program instructions include not only machine language code generated by a compiler, but also high-level language code that can be executed by a computer using an interpreter, etc. The hardware device may be configured to operate as at least one software module to perform the processing according to the present invention, and vice versa.

[0117] In the foregoing, the invention has been described with reference to specific matters such as particular components, as well as limited embodiments and accompanying drawings. However, this is only to help to understand the invention more fully, and the invention is not limited to the embodiments described above. Various modifications and variations can be designed by those skilled in the art based on these descriptions.

[0118] Therefore, the spirit of the present invention should not be limited to the above embodiments. Except for the appended claims, all modifications that are equivalent or analogous to these claims should be included within the spirit and scope of the present invention.

Claims

1. A method for optimizing a neural network model on a device using a subkernel search module, comprising: (a) In a state where a large neural network model with the maximum capacity allowed by the edge device equipped with the neural network model is trained to generate a first inference result for input data, wherein the target task serves at least one of data analysis, data classification, and information detection, and the data type processed for the target task includes at least one of text, image, and image containing text, when learning data is obtained, the learning device performs or supports performing the following processing: inputting the learning data to a subkernel search module, causing the subkernel search module (i) to identify at least one constraint corresponding to the learning data, wherein the constraint includes at least applicable to computational complexity, parameters (ii) Constraints of one of the following: number, latency, accuracy, and floating-point arithmetic; (iii) Generating a state vector corresponding to the learning data, wherein the state vector includes at least information on one of the complexity and clarity of the learning data; and (iv) Generating architecture information corresponding to the parameters of a specific sub-kernel architecture in the form of a string or dictionary, wherein the parameters of the specific sub-kernel consist of the minimum kernel size and the minimum number of channels required to perform the target task on the learning data having the state vector under the constraints, wherein the specific sub-kernel is a subset of the maximum capacity super-kernel corresponding to the large neural network model, consisting of a kernel size less than or equal to the super-kernel and a number of channels less than or equal to the super-kernel. (b) The learning device performs or supports the following processing: (i) optimizing the large neural network model by adjusting at least one of the kernel height, kernel width, and number of channels of the superkernel of the large neural network model according to the architectural information of the specific subkernel, so as to generate a specific small neural network model from the large neural network model having the architectural information of the specific subkernel; (ii) causing the specific small neural network model to generate a second inference result for the learning data; and (c) The learning device performs or supports the following processing: generating at least one first loss with reference to the first inference result and the second inference result, and training the sub-kernel search module using the first loss; In (a), the learning device performs or supports the following processing: (i) the sub-kernel search module generates multiple scene features corresponding to the learning data through the scene overview module, and generates the state vector corresponding to the multiple scene features; (ii) the sub-kernel network generator generates the architecture information of the specific sub-kernel consisting of the minimum kernel size and the minimum number of channels that are capable of performing the target task on the learning data having the state vector under the constraints. The learning device performs or supports the following processing: (i) Training the scene overview module and the sub-kernel network generator by backpropagation through the first loss under the given constraints to minimize the first loss; (ii) Generating at least one second loss with reference to the state vector and the corresponding real data, and further training the scene overview module to minimize the second loss. The scene overview module is composed of a convolutional neural network, and the sub-kernel network generator is composed of a recurrent neural network. The learning device performs or supports the following processing: training the scene overview module and the sub-kernel network generator separately or simultaneously.

2. The method according to claim 1, characterized in that: The learning device performs or supports the following processing: applying at least one convolution operation, at least one pooling operation, and at least one fully connected operation to the learning data through the convolutional neural network of the scene overview module to generate multiple scene features corresponding to the learning data.

3. The method according to claim 1, characterized in that: The learning device performs or supports the following processing: referring to the first inference result and the second inference result, when the difference between the first inference result and the second inference result exceeds a preset benchmark value, the learning data and the architecture information of the specific small neural network model are transmitted to the server, so that the server marks the learning data as a hard sample in the specific small neural network model and adds it to the learning database.

4. A method for optimizing a neural network model on a device according to claim 1, characterized in that: Further including: when obtaining test data, the optimization device performs or supports performing the following processes: inputting the test data into the sub-kernel search module, so that the sub-kernel search module (i) confirms the test constraints corresponding to the test data, wherein the test constraints include constraints applicable to at least one of computational complexity, parameters, latency, accuracy, and floating-point arithmetic; (ii) generates a test state vector corresponding to the test data, wherein the test state vector includes at least information regarding the complexity and clarity of the test data; (iii) generates test architecture information corresponding to the parameters of the test-specific sub-kernel architecture in the form of a string or dictionary data type, wherein the parameters consist of the minimum kernel size and minimum number of channels for performing the target task under the test constraints on the test data having the test state vector, wherein the test-specific sub-kernel is a subset of the superkernel corresponding to the maximum capacity of the large neural network model, consisting of a kernel size less than or equal to the superkernel and a number of channels less than or equal to the superkernel; and (b) The optimization device performs or supports the following processing: (i) optimizing the large neural network model by adjusting at least one of the kernel height, kernel width, and number of channels of the super kernel of the large neural network model according to the test architecture information of the test-specific sub-kernel, so as to generate a test-specific small neural network model with the test architecture information of the test-specific sub-kernel from the large neural network model; (ii) causing the test-specific small neural network model to generate a third inference result for the test data; In (a): The optimization device performs or supports the following processing: the sub-kernel search module (i) generates multiple test scenario features corresponding to the test data through the scenario overview module, and generates the test state vector corresponding to the multiple test scenario features; (ii) the sub-kernel network generator generates the test architecture information of the test-specific sub-kernel, which consists of the minimum kernel size and the minimum number of channels that can perform the target task on the learning data with the test state vector under the test constraints. The scene overview module is composed of a convolutional neural network, and the sub-kernel network generator is composed of a recurrent neural network.

5. The method according to claim 4, characterized in that: The optimization device performs or supports the following processing: applying at least one convolution operation, at least one pooling operation, and at least one fully connected operation to the test data through the convolutional neural network of the scene overview module to generate multiple test scene features corresponding to the test data.

6. A learning apparatus for optimizing a neural network model on a device using a subkernel search module, comprising: At least one memory that stores multiple instructions; and At least one processor for executing the plurality of said instructions, The processor performs or supports the following processing: (I) training a large neural network model with the maximum capacity to perform the target task using the maximum computing power allowed by the edge device with the neural network model installed, to generate a first inference result for the input data, wherein the target task serves at least one of data analysis, data classification and information detection, and the data type processed for the target task includes at least one of text, image, and image including text. When learning data is obtained, the learning device performs or supports the following processing: inputting the learning data to a subkernel search module, causing the subkernel search module to (i) identify at least one constraint corresponding to the learning data, wherein the constraint includes at least one of the constraints applicable to computational load, parameters, latency, accuracy and floating-point arithmetic, (ii) generate a state vector corresponding to the learning data, wherein the state vector includes at least one of the information regarding the complexity and clarity of the learning data, and (iii) using strings or dictionaries. The architecture information corresponding to the parameters of the architecture of the specific sub-kernel generated according to the type, wherein the parameters of the specific sub-kernel consist of the minimum kernel size and the minimum number of channels that can perform the target task on the learning data with the state vector under the constraints, wherein the specific sub-kernel is a subset of the super kernel with the maximum capacity corresponding to the large neural network model, consisting of a kernel size and a number of channels less than or equal to the super kernel; (II) (i) optimize the large neural network model by adjusting at least one of the kernel height, kernel width and number of channels of the super kernel of the large neural network model according to the architecture information of the specific sub-kernel, so as to generate a specific small neural network model with the architecture information of the specific sub-kernel from the large neural network model; (ii) make the specific small neural network model generate a second inference result for the learning data; and (III) generate at least one first loss with reference to the first inference result and the second inference result, and train the sub-kernel search module using the first loss; In the process described in (I): The processor performs or supports the following processing: the sub-kernel search module (i) generates multiple scene features corresponding to the learning data through the scene overview module, and generates the state vector corresponding to the multiple scene features; (ii) generates the architecture information of the specific sub-kernel consisting of the minimum kernel size and the minimum number of channels that can perform the target task on the learning data with the state vector under the constraints through the sub-kernel network generator. The processor performs or supports performing the following processes: (i) training the scene overview module and the sub-kernel network generator by backpropagation through the first loss under the given constraints to minimize the first loss; (ii) generating at least one second loss with reference to the state vector and the corresponding real data, and further training the scene overview module to minimize the second loss. The scene overview module is composed of a convolutional neural network, and the sub-kernel network generator is composed of a recurrent neural network. The processor performs or supports the following processing: training the scene overview module and the sub-kernel network generator respectively or simultaneously.

7. The learning device according to claim 6, characterized in that: The processor performs or supports the following processing: applying at least one convolution operation, at least one pooling operation, and at least one fully connected operation to the learning data through the convolutional neural network of the scene overview module, thereby generating multiple scene features corresponding to the learning data.

8. The learning device according to claim 6, characterized in that: The processor performs or supports the following processing: referring to the first inference result and the second inference result, when the difference between the first inference result and the second inference result exceeds a preset benchmark value, the learning data and the architecture information of the specific small neural network model are transmitted to the server, so that the server marks the learning data as a hard sample in the specific small neural network model and adds it to the learning database.

9. An optimization apparatus for optimizing a neural network model on a device using a subkernel search module, the subkernel search module being obtained through a process performed by the learning apparatus according to claim 6 and the large neural network model according to claim 6, the optimization apparatus comprising: At least one memory that stores multiple instructions; and At least one processor for executing the plurality of said instructions, The processor is configured to use the subkernel search module to optimize a large neural network model according to instructions in the memory, to perform or support the following processes: when test data is obtained, the test data is input to the subkernel search module, so that the subkernel search module (i) identifies at least one test constraint corresponding to the test data, wherein the test constraint includes constraints applicable to at least one of computational complexity, parameters, latency, accuracy, and floating-point arithmetic; (ii) generates a test state vector corresponding to the test data, wherein the test state vector includes at least information regarding the complexity and clarity of the test data; and (iii) generates test architecture information corresponding to the parameters of the test-specific subkernel architecture in string or dictionary data type, the parameters being determined by the test state vector. The data consists of the minimum kernel size and minimum number of channels for performing the target task under the test constraints, wherein the test-specific sub-kernel is a subset of the superkernel corresponding to the maximum capacity of the large neural network model, consisting of a kernel size and a number of channels less than or equal to the superkernel; and (II) the processor performs or supports the following processing: (i) optimizing the large neural network model by adjusting at least one of the kernel height, kernel width, and number of channels of the superkernel of the large neural network model according to the test architecture information of the test-specific sub-kernel, so as to generate a test-specific small neural network model with the test architecture information of the test-specific sub-kernel from the large neural network model, and (ii) causing the test-specific small neural network model to generate a third inference result for the test data; In the process described in (I): The processor performs or supports the following processing: (i) the sub-kernel search module generates multiple test scenario features corresponding to the test data through the scenario overview module, and generates the test state vector corresponding to the multiple test scenario features; (ii) the sub-kernel network generator generates the test architecture information of the test-specific sub-kernel consisting of the minimum kernel size and minimum number of channels that can perform the target task on the learning data having the test state vector under the test constraints. The scene overview module is composed of a convolutional neural network, and the sub-kernel network generator is composed of a recurrent neural network.

10. The optimization device according to claim 9, characterized in that: The processor performs or supports the following processing: applying at least one convolution operation, at least one pooling operation, and at least one fully connected operation to the test data through the convolutional neural network of the scene overview module, thereby generating multiple test scene features corresponding to the test data.

Images