Method and apparatus for on-device learning of machine learning networks

Abstract

The present invention relates to a method for performing on-device learning on a machine learning network by multi-stage learning using adaptive hyperparameters, comprising: (a) when new learning data meets on-device learning conditions, dividing current learning into 1st stage learning to nth stage learning, generating 1st stage learning data to nth stage learning data, generating 1st hyperparameter set candidates to hyperparameter set candidates based on the default value of each of a plurality of hyperparameters, training the machine learning network, selecting the machine learning network with the highest performance to generate the 1st adaptive hyperparameter set; (b) generating (k_1)th hyperparameter set candidates to (k_h)th hyperparameter set candidates, using the kth stage learning data for training, selecting the machine learning network with the highest performance of the (k-1)th stage learning to generate the kth adaptive hyperparameter set; and (c) generating the nth adaptive hyperparameter set, performing nth stage learning, thereby completing the current learning.

Description

[0001] This application claims priority to and the benefit of U.S. Patent Application No. 63 / 020,101, filed May 5, 2020, and U.S. Patent Application No. 17 / 229,350, filed April 13, 2021, the entireties of which are incorporated herein by reference. Technical Field

[0002] The present invention relates to a method for performing on-device learning on a machine learning network of an autonomous vehicle and an apparatus using the same, and more particularly, to a method for performing on-device learning by using multi-stage learning with an adaptive hyperparameter set on a system having an embedded machine learning network installed in the autonomous vehicle and an apparatus using the same. Background Art

[0003] In order to apply the machine learning network used in autonomous vehicles to new driving environments that have not been learned before, it is necessary to use the information obtained from sensors (such as cameras, lidars, radars, etc.) while the autonomous vehicle is driving to relearn the machine learning network.

[0004] When labeling data obtained from an autonomous vehicle to generate groundtruth for learning, it is difficult to perform this operation on the autonomous vehicle itself. Therefore, a method is proposed for training the machine learning network of an autonomous vehicle by generating a base model of the machine learning network or constructing a learning dataset, wherein the learning dataset is a combination of part of the learning data used for previous learning and data to be used for learning from data obtained from the autonomous vehicle in real time.

[0005] In particular, among existing methods, on-device learning technology is emerging. That is, the data required to train the machine learning network in the autonomous vehicle is transmitted to the cloud using over the air (OTA) technology, and the machine learning network is trained on the server side using the data transmitted from the cloud. Then, only the updated model is transmitted to the autonomous vehicle again through training.

[0006] However, existing on-device learning methods utilizing the over-the-air downloading technology have a slow update cycle and are limited in that they cannot be used when a cloud server and over-the-air downloading are unavailable.

[0007] Therefore, a technology is needed that can train machine learning networks using systems embedded in autonomous vehicles with limited computing power, without the need for connecting to cloud servers and over-the-air downloading. Summary of the Invention

[0008] Technical issues

[0009] The present invention aims to solve all the above problems.

[0010] In addition, another object of the present invention is to improve the performance of machine learning networks by performing on-device multi-stage learning on an embedded system installed in an autonomous vehicle and utilizing the computing power of the autonomous vehicle.

[0011] In addition, another object of the present invention is to improve the performance of a machine learning network through on-device learning by applying an adaptive set of hyperparameters generated through multi-stage learning instead of applying fixed hyperparameters.

[0012] In addition, another object of the present invention is to update a machine learning network with better performance than the current model in a shorter time than the time required to learn the best model of the machine learning network through over-the-air downloading from a cloud server.

[0013] Technical Solution

[0014] In order to achieve the above-mentioned object of the present invention and realize the characteristic effects of the present invention described below, the characteristic structure of the present invention is as follows.

[0015] According to one aspect of the present invention, a method for performing on-device learning on a machine learning network of an autonomous vehicle by multi-stage learning using an adaptive hyperparameter set is characterized in that the method includes: (a) when an on-device learning condition is met in a state where new learning data is acquired during driving of the autonomous vehicle, an on-device learning device (i) divides the current learning into first-stage learning to n-stage learning with reference to a default hyperparameter set, and generates first-stage learning data to n-stage learning data for the first-stage learning to the n-stage learning by using the new learning data and the previous learning data used in the previous learning, where n is 2 or greater, (ii) generating a first hyperparameter set candidate to a (1_h)th hyperparameter set candidate based on each first candidate value within a preset range of a default value combination of each of a plurality of hyperparameters included in the default hyperparameter set, wherein h is an integer greater than 2, (iii) applying the default hyperparameter set and the first hyperparameter set candidate to the (1_h)th hyperparameter set candidate, respectively, and using the first stage learning data to train the machine learning network, respectively, and (iv) evaluating the performance of each of the machine learning networks trained by applying the default hyperparameter set and the first hyperparameter set candidate to the (1_h)th hyperparameter set candidate, respectively. , selecting the machine learning network with the highest performance as the machine learning network learned in the first stage, (v) generating a hyperparameter set used in the training of the machine learning network learned in the first stage as the first adaptive hyperparameter set; (b) the on-device learning device increases k from 2 to (n-1), (i) combining each k-th candidate value within the preset range based on the (k-1)th adaptive value of each of the multiple hyperparameters included in the (k-1)th adaptive hyperparameter set to generate a (k_1)th hyperparameter set candidate to a (k_h)th hyperparameter set candidate, (ii) applying the (k-1)th hyperparameter set and the (k_1)th hyperparameter set candidate respectively to the (k_h)th hyperparameter set candidate, respectively using the k-th stage learning data to train the machine learning network learned in the (k-1)th stage, (iii) evaluating the performance of each machine learning network learned in the (k-1)th stage by respectively applying the (k-1)th adaptive hyperparameter set and the (k_1)th hyperparameter set candidate to the (k_h)th hyperparameter set candidate, selecting the machine learning network learned in the (k-1)th stage with the highest performance as the machine learning network learned in the kth stage, (iv) generating the hyperparameter set used in the training of the machine learning network learned in the kth stage as the k-th adaptive hyperparameter set;and (c) the on-device learning device generates an nth adaptive hyperparameter set using an optimization function constructed with reference to each of the (i-1) first adaptive hyperparameter set to the (n-1)th adaptive hyperparameter set and the performance evaluation results of each of the (i-2) machine learning networks trained in the first stage of learning to the (n-1)th stage of learning, and (ii) applies the nth adaptive hyperparameter set to perform nth stage learning on the machine learning network trained in the (n-1)th stage of learning using the nth stage of learning data, thereby completing the current learning.

[0016] As an example, the method also includes: (d) the on-device learning device performs the following processing: (i) when the performance of the machine learning network does not improve to above a certain threshold after the current learning is completed, the machine learning network is not updated, and the self-driving car is operated using the machine learning network until the next on-device learning condition is met; and (ii) when the performance of the machine learning network improves to above a certain threshold after the current learning is completed, the machine learning network is updated using the currently learned machine learning network, and the self-driving car is operated using the currently learned machine learning network until the next on-device learning condition is met.

[0017] As an example, when the performance of the machine learning network learned in the kth stage is higher than the performance of the machine learning network learned in the (k-1)th stage, the on-device learning device sets the (k+1)th adaptive hyperparameter set to the nth adaptive hyperparameter set to be equal to the kth adaptive hyperparameter set.

[0018] As an example, the on-device learning apparatus maintains the same adaptive value of at least one hyperparameter included in the first adaptive hyperparameter set to the nth adaptive hyperparameter set.

[0019] As an example, when the performance of the machine learning network trained in the k-th stage learning by the on-device learning device is higher than the performance of the machine learning network trained in the (k-1)-th stage learning, the current learning is completed by setting the k-th stage learning as the n-th stage learning.

[0020] As an example, in step (a), when sensor data obtained by a sensor installed on the autonomous vehicle is obtained, in a state where the machine learning network analyzes the sensor data to generate output data for driving information of the autonomous vehicle, the on-device learning device inputs the sensor data and the output data corresponding to each of the sensor data into the data selection network, so that the data selection network selects specific sensor data for training the machine learning network with reference to the output data, and stores the selected specific sensor data as the new learning data.

[0021] As an example, the hyperparameters include at least one of a learning algorithm setting, a mini-batch size, a maximum number of stages, and a maximum epoch per stage.

[0022] As an example, in step (a), the on-device learning device selects a hyperparameter set predetermined as a basic model of the machine learning network as the default hyperparameter set, or selects the best hyperparameter set generated in previous learning as the default hyperparameter set.

[0023] According to one aspect of the present invention, a device for performing on-device learning on a machine learning network of an autonomous vehicle by multi-stage learning using an adaptive hyperparameter set comprises: at least one memory for storing instructions; and at least one processor for executing the instructions, wherein the processor performs the following processing: (I) when a condition for on-device learning is satisfied in a state where new learning data is acquired during driving of the autonomous vehicle, dividing the current learning into first-stage learning to n-stage learning with reference to a default hyperparameter set, generating a plurality of parameters for the first-stage learning to n-stage learning by using the new learning data and the previous learning data used in the previous learning; The first stage learning data to the nth stage learning data, wherein n is an integer greater than 2, based on the default value of each of the multiple hyperparameters included in the default hyperparameter set, each first candidate value within a preset range is combined to generate the first hyperparameter set candidate to the (1_h)th hyperparameter set candidate, wherein h is an integer greater than 2, respectively applying the default hyperparameter set and the first hyperparameter set candidate to the (1_h)th hyperparameter set candidate, respectively using the first stage learning data to train the machine learning network, and evaluating each by respectively applying the default hyperparameter set and the first hyperparameter set candidate to the (1_h)th hyperparameter set candidate. The performance of the machine learning network trained by the set of candidates is selected, the machine learning network with the highest performance is selected as the machine learning network learned in the first stage, and the hyperparameter set applied in the training of the machine learning network learned in the first stage is generated as the first adaptive hyperparameter set; (II) increasing k from 2 to (n-1), combining each k-th candidate value within the preset range based on the (k-1)th adaptive value of each of the multiple hyperparameters included in the (k-1)th adaptive hyperparameter set, to generate the (k_1)th hyperparameter set candidate to the (k_h)th hyperparameter set candidate, and applying the (k-1)th hyperparameter set and the (k_1)th hyperparameter set to the first hyperparameter set respectively. parameter set candidate to the (k_h)th hyperparameter set candidate, respectively using the k-th stage learning data to train the machine learning network learned in the (k-1)th stage, evaluating the performance of each machine learning network learned in the (k-1)th stage by respectively applying the (k-1)th adaptive hyperparameter set and the (k_1)th hyperparameter set candidate to the (k_h)th hyperparameter set candidate, selecting the machine learning network learned in the (k-1)th stage with the highest performance as the machine learning network learned in the kth stage, and generating the hyperparameter set applied in the training of the machine learning network learned in the kth stage as the k-th adaptive hyperparameter set;and (III) generating an nth adaptive hyperparameter set using an optimization function constructed with reference to each of the first to (n-1)th adaptive hyperparameter sets and the respective performance evaluation results of the machine learning networks trained in the first to (n-1)th stages of learning, applying the nth adaptive hyperparameter set and performing nth stage learning on the machine learning network trained in the (n-1)th stage using the nth stage learning data, thereby completing the current learning.

[0024] As an example, (IV) the processor further performs the following processing: when the performance of the machine learning network does not improve to above a certain threshold after the current learning is completed, the machine learning network is not updated, and the autonomous driving vehicle is operated using the machine learning network until the next on-device learning condition is met; and when the performance of the machine learning network improves to above a certain threshold after the current learning is completed, the machine learning network is updated using the currently learned machine learning network, and the autonomous driving vehicle is operated using the currently learned machine learning network until the next on-device learning condition is met.

[0025] As an example, when the performance of the machine learning network learned in the kth stage is higher than the performance of the machine learning network learned in the (k-1)th stage, the processor sets the (k+1)th adaptive hyperparameter set to the nth adaptive hyperparameter set to be equal to the kth adaptive hyperparameter set.

[0026] As an example, in the process (II), the processor keeps the adaptive value of at least one hyperparameter included in the first adaptive hyperparameter set to the nth adaptive hyperparameter set the same.

[0027] As an example, when the performance of the machine learning network trained in the k-th stage learning is higher than the performance of the machine learning network trained in the (k-1)-th stage learning, the processor completes the current learning by setting the k-th stage learning to the n-th stage learning.

[0028] As an example, in the (I) processing, when sensor data obtained by a sensor installed on the autonomous vehicle is obtained, in a state where the machine learning network analyzes the sensor data according to deep learning to generate output data for driving information of the autonomous vehicle, the processor inputs the sensor data and the output data corresponding to each of the sensor data into the data selection network, so that the data selection network selects specific sensor data for network training of the machine learning with reference to the output data, and stores the selected specific sensor data as the new learning data.

[0029] As an example, the hyperparameters include at least one of a learning algorithm setting, a mini-batch size, a maximum stage, and a maximum epoch for each stage.

[0030] As an example, in the step (I), the processor selects a hyperparameter set predetermined as a base model of the machine learning network as the default hyperparameter set, or selects an optimal hyperparameter set generated in previous learning as the default hyperparameter set.

[0031] In addition, the present invention also provides a computer-readable recording medium for recording a computer program for executing the method of the present invention.

[0032] Technical Effects

[0033] Another effect of the present invention is that the performance of the machine learning network can be improved by performing on-device multi-stage learning on the embedded system installed in the autonomous vehicle and utilizing the computing power of the autonomous vehicle.

[0034] In addition, another effect of the present invention is that in order to improve the performance of the machine learning network through on-device learning, learning can be performed by applying an adaptive hyperparameter set generated through multi-stage learning instead of applying fixed hyperparameters.

[0035] In addition, another effect of the present invention is that a machine learning network with better performance than the current model can be updated in a shorter time than the time required to learn the best model of the machine learning network through over-the-air downloading from a cloud server. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] The following drawings used to describe embodiments of the present invention are only part of the embodiments of the present invention, and ordinary technicians in the field to which the present invention belongs (hereinafter referred to as "ordinary technicians") can obtain other drawings based on these drawings without performing any creative work.

[0037] Figure 1 A schematic diagram of an on-device learning apparatus for performing on-device learning of a machine learning network for an autonomous vehicle by multi-stage learning using an adaptive hyperparameter set according to an embodiment of the present invention;

[0038] Figure 2 A schematic diagram of a method for performing on-device learning of a machine learning network for an autonomous vehicle by multi-stage learning using an adaptive hyperparameter set according to an embodiment of the present invention;

[0039] Figure 3 is a structural diagram of a hyperparameter set according to an embodiment of the present invention;

[0040] Figure 4Schematic diagram of a method for generating an adaptive hyperparameter set by learning in each stage and performing multi-stage learning on a machine learning network using the generated adaptive hyperparameter set according to an embodiment of the present invention. DETAILED DESCRIPTION

[0041] The following detailed description of the present invention is made with reference to the accompanying drawings, which illustrate specific embodiments in which the present invention may be implemented to illustrate the purpose, technical solutions and advantages of the present invention. These embodiments are described in sufficient detail to enable those skilled in the art to implement the present invention.

[0042] In addition, in the present disclosure and claims, the term "comprising" and its variations are not intended to exclude other technical features, additions, components, or steps. For those skilled in the art, other objects, advantages, and features of the present invention will be apparent in part from this specification and in part from the practice of the present invention. The following illustrations and figures are provided as examples and are not intended to limit the present invention.

[0043] Furthermore, the present invention includes all possible combinations of the embodiments shown in this specification. It should be understood that the various embodiments of the present invention, although different, are not necessarily mutually exclusive. For example, the specific shapes, structures and characteristics described herein in one embodiment may be implemented by other embodiments without exceeding the spirit and scope of the present invention. Furthermore, it should be understood that the position or configuration of each component in each disclosed embodiment may be changed without exceeding the spirit and scope of the present invention. Therefore, the detailed description to be described later is not intended to limit the present invention, and as long as it can be properly described, the scope of the present invention should be defined according to all scopes equivalent to the scope of its claims and the scope of the appended claims. Similar reference numerals in the drawings indicate the same or similar functions in multiple aspects.

[0044] In order to enable those skilled in the art to easily implement the present invention, preferred embodiments of the present invention will be described in detail below with reference to the accompanying drawings.

[0045] Figure 1 Schematic diagram of an on-device learning apparatus 1000 for performing on-device learning on a machine learning network for an autonomous vehicle by multi-stage learning using an adaptive hyperparameter set according to an embodiment of the present invention.

[0046] See also Figure 1 The on-device learning apparatus 1000 may include: a memory 1001 storing instructions for performing on-device learning on a machine learning network of an autonomous driving vehicle by multi-stage learning using an adaptive hyperparameter set; and a processor 1002 executing, according to the instructions, an operation of performing on-device learning on a machine learning network of an autonomous driving vehicle by multi-stage learning using an adaptive hyperparameter set.

[0047] Specifically, the on-device learning device 1000 can generally use a combination of computing devices (e.g., computer processors, memories, storage devices, input and output devices, and devices that may include other components of conventional computing devices; electronic communication devices, such as routers, switches, etc.; electronic information storage systems, such as network attached storage (NAS) and storage area networks (SAN)) and computer software (i.e., instructions that cause the computing devices to operate in a specific manner) to achieve the desired system performance.

[0048] In addition, the processor of a computing device may include hardware configurations such as a microprocessing unit (MPU) or a central processing unit (CPU), cache memory, and a data bus. In addition, the computing device may further include a software configuration for an operating system and executing application programs for specific purposes.

[0049] However, it is not excluded that the computing device includes an integrated processor in the form of an integrated medium, processor, and memory for implementing the present invention.

[0050] Referring to the on-device learning apparatus 1000 according to an embodiment of the present invention configured as described above, Figures 2 to 4 Describes a method for on-device learning of machine learning networks for self-driving cars via multi-stage learning using adaptive hyperparameter sets.

[0051] See also Figure 2 When the autonomous vehicle is driving, sensor data sensed by various sensors installed on the autonomous vehicle is obtained, and the machine learning network of the autonomous vehicle analyzes the sensor data based on deep learning, obtains output data including information related to the driving of the autonomous vehicle, and enables the autonomous vehicle to drive by referring to the obtained output data.

[0052] In this case, the sensor may include a camera, radar, lidar, etc., and may be a fusion of various sensors. In addition, the sensor data may include environmental information of an area corresponding to a specific viewing angle of the sensor, or environmental information of the entire surrounding area of ​​the sensor. Furthermore, the sensor data may be a video or image captured around the vehicle, or driving information sensed by various sensors of the vehicle, but the present invention is not limited thereto and may include all types of data sensed by the autonomous vehicle while driving.

[0053] In addition, the machine learning network 500 may include a deep learning-based network, etc., and may include an object detector, a classification network, and a segmentation network, etc. that analyze the captured driving images and output driving environment information such as object information and lane information according to the driving environment of the autonomous driving vehicle. In addition, it may include various networks that perform various operations based on driving environment analysis to support the driving of the autonomous driving vehicle, such as behavior prediction and risk detection.

[0054] During the driving process of these autonomous vehicles, the on-device learning device 1000 can refer to the sensor data and the output data of the machine learning network 500 corresponding to each of the sensor data, and select specific sensor data for learning to improve the performance of the machine learning network 500, so that the autonomous vehicle can adapt to repeated driving environments or new driving environments. The selected specific sensor data can be stored in a new learning database 410.

[0055] As an example, when the machine learning network 500 is an object detector based on a convolutional neural network for searching for objects related to the driving environment and classifying the categories of the objects, specific sensor data for training the machine learning network 500 can be selected with reference to detection probability distribution information for each object detected in the sensor data.

[0056] Specifically, when the detection probability of each object detected from the sensor data is lower than a preset probability, the sensor data is classified as a hard sample that is difficult for the machine learning network 500 to classify, and then selected as new learning data and stored in the new learning database 410. However, the method of selecting sensor data for improving the performance of the machine learning network 500 is not limited to this, and various methods of selecting specific sensor data useful for learning can be applied. In this case, the predetermined probability can be a predetermined probability for comparing the classification results of the objects detected by the machine learning network 500.

[0057] On the other hand, in the above, the on-device learning device 1000 selects feature sensor data that is useful for learning the machine learning network 500, but differently from this, the on-device learning device 1000 can input the sensor data and the output data of the machine learning network 500 corresponding to each sensor data into the deep learning-based data selection network 300, so that the data selection network 300 selects specific sensor information that is useful for learning the machine learning network 500 with reference to the output data.

[0058] Next, while new learning data, serving as specific sensory information useful for learning the machine learning network, is collected from sensory data obtained during driving of the autonomous vehicle using the above-described method, and when at least one or more on-device learning conditions are satisfied, such as a preset time condition, a preset amount of learning data condition, and a preset driving state condition of the autonomous vehicle, the on-device learning apparatus 1000 may refer to the default hyperparameter set of the machine learning network 500 and divide the current learning into the first to nth stages of learning. In this case, n is an integer greater than or equal to 2.

[0059] As an example, the on-device learning apparatus 1000 may enable the hyperparameter selection network 200 to refer to the default hyperparameter set of the machine learning network 500, and may divide the current learning into the first stage learning to the nth stage learning.

[0060] On the other hand, see Figure 3 , the hyperparameter set 30 may include one or more hyperparameters. As an example, the hyperparameter set 30 may include parameters (L) of the learning algorithm, the size of the mini-batch (B), the maximum number of stages (D), and the maximum period (E) of each stage, but the present invention is not limited thereto and may include a combination of various hyperparameters for improving learning performance. In addition, the parameters (L) of the learning algorithm may include a category for selecting an optimizer type of a learning algorithm such as SGD, Adam, AdaGrad, RMSProp; and a category for additional parameters such as learning rate, weight decay, momentum, and damping.

[0061] In this case, the on-device learning apparatus 1000 or the hyperparameter selection network 200 of the on-device learning apparatus 1000 can generate a default hyperparameter set as shown in the following equation by using the default values ​​of the hyperparameters predetermined for training the base model of the machine learning network 500. In this case, the base model can be the machine learning network 500 during factory initialization, and the default value is the value of each hyperparameter included in the default hyperparameter set.

[0062] H init ={L pre ,B pre ,D pre ,E pre}

[0063] In addition, the on-device learning apparatus 1000 or the hyperparameter selection network 200 of the on-device learning apparatus 1000 may generate a default hyperparameter set as shown in the following equation by using the highest performance values ​​of the hyperparameters of the machine learning network 500 that have the highest performance in past learning of the machine learning network 500. In this case, the highest performance values ​​are the values ​​of the hyperparameters applied to the machine learning network with the highest performance.

[0064] H init ={L opt ,B opt ,D opt ,E opt}

[0065] At this time, the default values ​​of the additional parameters in L can generally be decimals close to zero, and the optimizer, B, D and E of L can be selected considering the size of the data set used for learning and the computing power of the learning device 1000 on the device.

[0066] See again Figure 2 The on-device learning device 1000 may generate a learning data set for the current learning by sampling the new learning data and the previous learning data stored in the new learning database 410 and the previous learning database 420, and designate the data set as the first-stage learning data to the n-stage learning data, or may generate multiple learning data sets to be used for each stage of learning based on the default hyperparameter set, and designate the data set as the first-stage learning data to the n-stage learning data. Furthermore, the on-device learning device 1000 may cause the network trainer 100 to designate the first-stage learning data as the n-stage learning data in the same manner as described above.

[0067] At this time, the sampling of the learning data set can be performed at a certain time point in the current learning process, or in some cases, multiple samplings can be performed in each stage of learning, but the time point and method of sampling the learning data set in the present invention are not limited to this.

[0068] In addition, as data used in learning or previous learning of the basic model of the machine learning network 500, the previous learning data can be labeled data with real data labels, and the new learning data can be unlabeled data without real data labels. In this case, category labeling is performed by a separately configured automatic labeling network, or the new learning data can be labeled through various methods such as manual labeling. In addition, semi-supervised learning can also be performed by using previously learned labeled data and new learning unlabeled data.

[0069] Next, the on-device learning device 1000 may refer to the default hyperparameter set and generate the first through (1_h)th hyperparameter set candidates based on the default value of each of the multiple hyperparameters included in the default hyperparameter set combined with each first candidate value within a preset range. In this case, h is an integer greater than or equal to 2. Furthermore, the on-device learning device 1000 may cause the hyperparameter selection network 200 to generate the first through (1_h)th hyperparameter set candidates based on the default hyperparameter set.

[0070] As an example, the on-device learning apparatus 1000 or the hyperparameter selection network 200 of the on-device learning apparatus 1000 may generate a default hyperparameter set as shown in the following equation by using the default values ​​of the hyperparameters predetermined by the basic model of the machine learning network 500.

[0071] H init ={L pre ,B pre ,D pre ,E pre}={0.1 pre ,5 pre ,5 pre ,10 pre}

[0072] At this time, the on-device learning device 1000 or the hyperparameter selection network 200 of the on-device learning device 1000 can determine the first candidate value within the preset range based on each default value of each hyperparameter. That is, the first candidate value of each hyperparameter can be configured as L1={0.12, 0.11, 0.09, 0.08}, B1={7, 6, 4, 3}, D1={8, 7, 6, 4, 3, 2} and E1={6, 8, 12, 14}, and different ranges can be preset for each hyperparameter. Moreover, the on-device learning device 1000 or the hyperparameter selection network 200 of the on-device learning device 1000 can generate the hyperparameter set candidates by respectively combining one value of the multiple first candidate values ​​of each hyperparameter. As an example, the first hyperparameter set candidate to the (1_h)th hyperparameter set candidate can be H 1_1can ={0.12,7,8,6} to H 1_hcan ={0.08,3,2,14} combination is generated.

[0073] In contrast, the on-device learning device 1000 or the hyperparameter selection network 200 of the on-device learning device 1000 can generate the 1st hyperparameter set candidate to the (1_h)th hyperparameter set candidate by randomly selecting h from all combinable hyperparameter set candidates using the 1st candidate value.

[0074] Next, the on-device learning apparatus 1000 may train the machine learning network using the first stage learning data according to the default hyperparameter set and each of the first hyperparameter set candidate to the (1_h)th hyperparameter set candidate.

[0075] As an example, the on-device learning apparatus 1000 causes the hyperparameter selection network 200 to transmit the default hyperparameter set and the first to (1_h)th hyperparameter set candidates to the network trainer 100. Furthermore, the on-device learning apparatus 1000 may cause the hyperparameter selection network 200 to train the machine learning network 500 using the first learning data based on the default hyperparameter set and each of the first to (1_h)th hyperparameter set candidates.

[0076] In addition, the on-device learning device 1000 can enable the hyperparameter selection network 200 to transmit each of the default hyperparameter set, the first hyperparameter set candidate to the (1_h)th hyperparameter set candidate in sequence to the network trainer 100, so that the network trainer 100 uses the first learning data to train each machine learning network 500 according to each of the default hyperparameter set, the first hyperparameter set candidate to the (1_h)th hyperparameter set candidate transmitted in sequence.

[0077] That is, the on-device learning apparatus 1000 can cause the hyperparameter selection network 200 to transmit the default hyperparameter set to the network trainer 100, so that the network trainer 100 trains the machine learning network 500 using the first learning data according to the default hyperparameter set. Furthermore, the on-device learning apparatus 1000 can cause the hyperparameter selection network 200 to transmit the first hyperparameter set to the network trainer 100, so that the network trainer 100 trains the machine learning network 500 using the first-stage learning data according to the first hyperparameter set. Then, in the same manner, the on-device learning apparatus 1000 can cause the hyperparameter selection network 200 to sequentially transmit each of the (1_2)th to (1_h)th hyperparameter sets to the network trainer 100, so that the network trainer 100 trains each of the machine learning networks 500 using the first learning data according to each of the sequentially transmitted (1_2)th to (1_h)th hyperparameter sets.

[0078] Next, the on-device learning device 1000 evaluates the performance of each of the machine learning networks 500 trained by respectively applying the default hyperparameter set and the first hyperparameter set candidate to the (1_h)th hyperparameter set candidate, selects the machine learning network 500 with the highest performance as the machine learning network 500 learned in the first stage, and generates the hyperparameter set 30 applied in the training of the machine learning network 500 learned in the first stage as the first adaptive hyperparameter set.

[0079] As an example, the on-device learning apparatus 1000 may cause the network trainer 100 to cross-validate each of the machine learning networks 500 learned by applying the default hyperparameter set and the first hyperparameter set candidate to the (1_h)th hyperparameter set candidate, respectively, using validation data, and transmit the multiple performance evaluation results obtained therefrom to the hyperparameter selection network 200. Furthermore, the on-device learning apparatus 1000 may cause the hyperparameter selection network 200 to select the machine learning network 500 with the highest performance as the machine learning network 500 learned in the first stage, with reference to the multiple cross-validated performance evaluation results, and generate the hyperparameter set 30 used for training the machine learning network 500 learned in the first stage as the first adaptive hyperparameter set.

[0080] Accordingly, the on-device learning apparatus 1000 can generate the first adaptive hyperparameter set by completing the first stage learning, and can perform the next stage learning by using the first adaptive hyperparameter set.

[0081] Next, after completing the first stage of learning, the on-device learning apparatus 1000 may increase k from 2 to (n-1), and generate the (k_1)th to (k_h)th hyperparameter set candidates based on the (k-1)th adaptive value of each of the multiple hyperparameters included in the (k-1)th adaptive hyperparameter set, combining each kth candidate value within the preset range. In this case, the adaptive value is the value of each hyperparameter included in the adaptive hyperparameter set.

[0082] Next, the on-device learning device 1000 can respectively apply the (k-1)th hyperparameter set and the (k_1)th hyperparameter set candidate to the (k_h)th hyperparameter set candidate, and use the kth stage learning data to train the (k-1)th stage learning machine learning network 500.

[0083] Next, the on-device learning device 1000 can evaluate the performance of each of the (k-1)-th stage learned machine learning networks 500 trained by respectively applying the (k-1)-th adaptive hyperparameter set and the (k_1)-th hyperparameter set candidate to the (k_h)-th hyperparameter set candidate, select the (k-1)-th stage learned machine learning network 500 with the highest performance as the k-th stage learned machine learning network 500, and generate the hyperparameter set 30 applied in the training of the k-th stage learned machine learning network 500 as the first adaptive hyperparameter set.

[0084] At this time, the on-device learning device 1000 can repeat the above method through the on-device learning device 1000 or other devices to perform the first stage learning to the (n-1) stage learning in sequence, generating the first adaptive hyperparameter set to the (n-1) adaptive hyperparameter set. The specific execution method is the same as above and will not be repeated here.

[0085] Next, we will refer to Figure 4 Describes the detailed method for performing the n-th stage of learning. Figure 4 , when all other stage learning S41-1, S41-K, S41-(N-1) except the nth stage learning S41-N are completed, the learning device 1000 on the device can use an optimization function constructed with reference to the performance evaluation results of each of the first adaptive hyperparameter set to the (n-1)th adaptive hyperparameter set and each machine learning network 500 learned in the first stage learning S41-1 to the machine learning network 500 learned in the (n-1)th stage learning S41-(N-1) to generate the nth adaptive hyperparameter set.

[0086] As an example, the on-device learning device 1000 enables the hyperparameter selection network 200 to refer to the first stage learning S41-1 to the (n-1)th stage learning S41-(N-1) to generate the nth adaptive hyperparameter set required to predict a performance result that is better than the performance of the machine learning network 500 trained in the previous learning.

[0087] Specifically, the on-device learning device 1000 can enable the hyperparameter selection network 200 to respectively refer to the first adaptive hyperparameter set to the (n-1)th adaptive hyperparameter set and the performance evaluation results of each machine learning network 500 learned using the first adaptive hyperparameter set to the (n-1)th adaptive hyperparameter set to construct an optimization function, and predict the nth adaptive hyperparameter that can improve the performance of the machine learning network 500 trained through the (n-1)th stage learning S41-(N-1).

[0088] At this time, for the above-mentioned optimization function, in order to generate the adaptive hyperparameter set that can produce a result that is superior to the performance of the machine learning network 500 trained from the previous learning or the machine learning network 500 trained from the first stage of learning to the (n-1) stage of learning of the current learning, one or more surrogate models (Surrogate model) and acquisition functions (Acquisition Function) for probability estimation can be applied, and the nth adaptive hyperparameter set can be generated by using the optimal value of each hyperparameter generated by the optimization function operation. In this case, the optimal value is the value of each hyperparameter generated by the operation of the optimization function.

[0089] Next, the on-device learning apparatus 1000 may complete the current learning by applying the nth adaptive hyperparameter set and using the nth stage learning data to perform the nth stage learning on the machine learning network 500 of the (n-1)th stage learning.

[0090] As an example, the on-device learning device 1000 enables the network trainer 100 to train the machine learning network 500 trained through the (n-1)th stage learning using the nth stage learning data according to the nth adaptive hyperparameter set, thereby completing all stages of learning included in the multi-stage learning 40 and completing the current learning.

[0091] Next, if the performance of the machine learning network 500 does not improve above a certain threshold after completing the current learning process, the on-device learning apparatus 1000 may not update the machine learning network 500 and may operate the autonomous vehicle using the machine learning network 500 until the next on-device learning condition is met. In this case, the threshold may be a predetermined value used to compare the performance level of the machine learning network 500.

[0092] As an example, when it is determined that the performance of the machine learning network 500 that has completed the current learning has not improved compared to the performance of the machine learning network 500 that has completed the previous learning, the on-device learning device 1000 causes the network trainer 100 to not update the machine learning network 500 that has completed the current learning to the machine learning network 500 of the autonomous driving vehicle, so that the machine learning network 500 trained in the previous learning can be used to operate the autonomous driving vehicle until the next on-device learning condition is met.

[0093] In addition, when the performance of the machine learning network 500 is improved to above a certain threshold after completing the current learning, the on-device learning device 1000 can update the machine learning network 500 to the currently learned machine learning network 500, and use the currently learned machine learning network 500 to run the autonomous driving vehicle until the next device learning conditions are met.

[0094] For example, when it is determined that the performance of the machine learning network 500 that has completed the current learning process is improved compared to the performance of the machine learning network 500 that has completed the previous learning process, the on-device learning device 1000 causes the network trainer 100 to update the machine learning network 500 that has completed the current learning process as the machine learning network 500 of the autonomous vehicle. This allows the autonomous vehicle to operate using the machine learning network 500 updated through the current learning process until the next on-device learning condition is met. In this case, the method for updating the machine learning network 500 of the autonomous vehicle is as follows, but the present invention is not limited thereto: The machine learning network 500 of the autonomous vehicle is updated by transferring the machine learning network 500 that has completed the current learning process to replace the machine learning network 500 of the autonomous vehicle, or by transferring learning parameters updated from the machine learning network 500 that has completed the current learning process to update the learning parameters of the machine learning network 500 used for the autonomous vehicle, thereby updating the machine learning network 500 of the autonomous vehicle.

[0095] In addition, in the process of generating each of the adaptive hyperparameter sets through learning in each stage, the on-device learning device 1000 or the hyperparameter selection network 200 of the on-device learning device 1000, when the performance of the machine learning network 500 trained in the k-th stage learning is higher than the performance of the machine learning network 500 trained in the (k-1)-th stage learning, the k-th adaptive hyperparameter set can be generated as the (k+1)-th adaptive hyperparameter set to the n-th adaptive hyperparameter set respectively.

[0096] Furthermore, when it is determined that the performance of the machine learning network 500 trained in the k-th stage of learning is higher than the performance of the machine learning network 500 trained in the (k-1)-th stage of learning, the on-device learning apparatus 1000 or the hyperparameter selection network 200 of the on-device learning apparatus 1000 may complete the current learning by setting the k-th stage of learning as the n-th stage of learning S41-N. The on-device learning apparatus 1000 or the hyperparameter selection network 200 of the on-device learning apparatus 1000 may then update the machine learning network 500 to the machine learning network 500 trained in the n-th stage of learning, thereby operating the autonomous vehicle using the machine learning network 500 updated by the current learning until the next device learning condition is met.

[0097] That is, in this way, when a machine learning network 500 with performance better than the existing machine learning network 500 is guaranteed, the current learning can be completed without the need for the remaining stage learning, which not only reduces the additional computing resources consumed by generating the adaptive hyperparameter set through the remaining stage learning, but also allows the machine learning network 500 with improved performance to be quickly updated to the operation of the autonomous driving car.

[0098] In addition, the on-device learning apparatus 1000 may maintain the same adaptive value of at least one hyperparameter included in the first adaptive hyperparameter set to the nth adaptive hyperparameter set. This can be expressed as follows:

[0099] (H st+1 =H st )

[0100] As an example, the on-device learning apparatus 1000 may cause the hyperparameter selection network 200 to maintain some or all of each hyperparameter included in the adaptive hyperparameter set the same, so as to generate each adaptive hyperparameter set through learning at each stage. For example, the on-device learning apparatus 1000 may cause the hyperparameter selection network 200 to select and maintain the adaptive value of the mini-batch size, one of the hyperparameters included in the adaptive hyperparameter set, the same as some or all of the first to nth adaptive hyperparameter sets.

[0101] In addition, the above embodiments according to the present invention can be implemented in the form of program instructions that can be executed by various computer components and recorded in a computer-readable recording medium. The computer-readable recording medium may include program instructions, data files, data structures, etc., either individually or in combination. The program instructions recorded in the computer-readable recording medium may be specially designed and configured for the present invention, or may be known and available to those skilled in the art of computer software. Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks, and magnetic tapes, optical recording media such as CD-ROMs and DVDs, magneto-optical media such as floppy disks, and hardware devices specifically configured to store and execute program instructions, such as ROMs, RAMs, flash memories, etc. Examples of program instructions include not only machine language codes such as those generated by a compiler, but also high-level language codes that can be executed by a computer using an interpreter, etc. The hardware device may be configured to operate as at least one software module to perform processing according to the present invention, or vice versa.

[0102] The present invention has been described above with reference to specific matters such as specific components and limited embodiments and drawings, but this only helps to provide a more comprehensive understanding of the present invention. The present invention is not limited to the above embodiments, and ordinary technicians in the field to which the present invention belongs can design various modifications and changes based on these descriptions.

[0103] Therefore, the spirit of the present invention should not be limited to the above embodiments. In addition to the appended claims, all modifications equivalent or equivalent to these claims should be included in the spirit of the present invention.

Claims

1. A method for on-device learning of a machine learning network for an autonomous vehicle by multi-stage learning using an adaptive set of hyperparameters, characterized in that The method comprises: (a) When a condition for on-device learning is satisfied in a state where new learning data is acquired during driving of an autonomous vehicle, an on-device learning device (a1) divides current learning into 1st stage learning to nth stage learning with reference to a default hyperparameter set, generates 1st stage learning data to nth stage learning data for the 1st stage learning to the nth stage learning by using the new learning data and the previous learning data used in the previous learning, wherein n is an integer greater than 2, and (a2) generates 1st hyperparameter set candidates to (1_h)th hyperparameter set candidates based on each 1st candidate value within a preset range of a default value combination of each of a plurality of hyperparameters included in the default hyperparameter set. parameter set candidates, wherein the h is an integer greater than or equal to 2, (a3) ​​applying the default hyperparameter set and the first hyperparameter set candidate to the (1_h)th hyperparameter set candidate, respectively, and using the first stage learning data to train the machine learning network, (a4) evaluating the performance of each of the machine learning networks trained by respectively applying the default hyperparameter set and the first hyperparameter set candidate to the (1_h)th hyperparameter set candidate, and selecting the machine learning network with the highest performance as the machine learning network learned in the first stage, (a5) generating the hyperparameter set applied in the training of the machine learning network learned in the first stage as the first adaptive hyperparameter set; (b) the on-device learning device increases k from 2 to (n-1), (b1) based on the (k-1)th adaptive value combination of each kth candidate value in the preset range of the plurality of hyperparameters included in the (k-1)th adaptive hyperparameter set, generates the (k_1)th hyperparameter set candidate to the (k_h)th hyperparameter set candidate, (b2) respectively applying the (k-1)th hyperparameter set and the (k_1)th hyperparameter set candidate to the (k_h)th hyperparameter set candidate, respectively using the k-th stage learning data to train the machine learning network of the (k-1) stage learning, (b3) evaluating each by respectively applying the (k-1)th adaptive hyperparameter set and the (k_1)th hyperparameter set candidate to the (k_h)th hyperparameter set candidate. Adapting the performance of the machine learning network learned at the (k-1)th stage by adapting the hyperparameter set and the (k_1)th hyperparameter set candidate to the (k_h)th hyperparameter set candidate, selecting the machine learning network learned at the (k-1)th stage with the highest performance as the machine learning network learned at the kth stage, (b4) generating a hyperparameter set applied in the training of the machine learning network learned at the kth stage as the kth adaptive hyperparameter set, wherein the processes (b1) to (b4) are performed with k initially set to 2, and the processes (b1) to (b4) are repeated after k is increased by 1 until k reaches (n-1); and (c) The on-device learning device (i) generates an nth adaptive hyperparameter set using an optimization function constructed with reference to each of the first to (n-1)th adaptive hyperparameter sets and the respective performance evaluation results of the machine learning networks trained in the first to (n-1)th stage learning, and (ii) applies the nth adaptive hyperparameter set and performs nth stage learning on the machine learning network learned in the (n-1)th stage using the nth stage learning data, thereby completing the current learning.

2. The method according to claim 1, characterized in that Also includes: (d) The on-device learning device performs the following processing: (i) when the performance of the machine learning network does not improve to above a certain threshold after the current learning is completed, the machine learning network is not updated, and the autonomous driving vehicle is operated using the machine learning network until the next on-device learning condition is met; and (ii) when the performance of the machine learning network improves to above a certain threshold after the current learning is completed, the machine learning network is updated using the currently learned machine learning network, and the autonomous driving vehicle is operated using the currently learned machine learning network until the next on-device learning condition is met.

3. The method according to claim 1, wherein: When the performance of the machine learning network learned in the k-th stage is higher than the performance of the machine learning network learned in the (k-1)-th stage, the on-device learning device sets the (k+1)-th adaptive hyperparameter set to the n-th adaptive hyperparameter set to be equal to the k-th adaptive hyperparameter set.

4. The method according to claim 1, wherein: In the step (b), The on-device learning apparatus maintains the same adaptive value of at least one hyperparameter included in the first adaptive hyperparameter set to the nth adaptive hyperparameter set.

5. The method according to claim 1, wherein: When the performance of the machine learning network trained in the k-th stage learning by the on-device learning apparatus is higher than the performance of the machine learning network trained in the (k-1)-th stage learning, the current learning is completed by setting the k-th stage learning as the n-th stage learning.

6. The method according to claim 1, wherein: In the step (a), When sensing data obtained by a sensor mounted on the autonomous vehicle is obtained, the machine learning network analyzes the sensing data to generate output data for driving information of the autonomous vehicle. The on-device learning device inputs the sensor data and the output data corresponding to each of the sensor data into a data selection network, so that the data selection network selects specific sensor data for training the machine learning network with reference to the output data, and stores the selected specific sensor data as the new learning data.

7. The method according to claim 1, wherein: The hyperparameters include at least one of a learning algorithm setting, a mini-batch size, a maximum stage, and a maximum epoch for each stage.

8. The method according to claim 1, wherein: In the step (a), The on-device learning device selects a hyperparameter set predetermined as a basic model of the machine learning network as the default hyperparameter set, or selects an optimal hyperparameter set generated in previous learning as the default hyperparameter set.

9. An apparatus for performing on-device learning of a machine learning network for an autonomous vehicle by multi-stage learning using an adaptive set of hyperparameters, characterized in that include: at least one memory storing instructions; and at least one processor for executing said instructions, The processor performs the following processing: (I) when a learning condition on the device is satisfied in a state where new learning data is acquired during driving of the autonomous vehicle, the current learning is divided into the first stage learning to the nth stage learning with reference to the default hyperparameter set, and the first stage learning data to the nth stage learning data for the first stage learning to the nth stage learning are generated by using the new learning data and the previous learning data used in the previous learning, wherein n is an integer greater than 2, and the first candidate value within a preset range of each of the default values ​​of the multiple hyperparameters included in the default hyperparameter set is combined to generate the first stage learning data to the nth stage learning data. 1 hyperparameter set candidate to the (1_h)th hyperparameter set candidate, where h is an integer greater than 2, respectively applying the default hyperparameter set and the first hyperparameter set candidate to the (1_h)th hyperparameter set candidate, respectively using the first stage learning data to train the machine learning network, evaluating the performance of each of the machine learning networks trained by respectively applying the default hyperparameter set and the first hyperparameter set candidate to the (1_h)th hyperparameter set candidate, selecting the machine learning network with the highest performance as the machine learning network learned in the first stage, and generating a training set of the machine learning network learned in the first stage. as the first adaptive hyperparameter set; (II) increasing k from 2 to (n-1), combining each k-th candidate value within the preset range based on the (k-1)th adaptive value of each of the multiple hyperparameters included in the (k-1)th adaptive hyperparameter set, generating the (k_1)th hyperparameter set candidate to the (k_h)th hyperparameter set candidate, respectively applying the (k-1)th hyperparameter set and the (k_1)th hyperparameter set candidate to the (k_h)th hyperparameter set candidate, respectively using the k-th stage learning data to train the machine learning network learned in the (k-1) stage, and evaluating each of the results obtained by respectively applying The performance of the machine learning network learned in the (k-1)th stage is trained by comparing the (k-1)th adaptive hyperparameter set and the (k_1)th hyperparameter set candidate to the (k_h)th hyperparameter set candidate, selecting the machine learning network learned in the (k-1)th stage with the highest performance as the machine learning network learned in the kth stage, and generating a hyperparameter set used in the training of the machine learning network learned in the kth stage as the kth adaptive hyperparameter set, wherein the process of (II) is performed when k is initially set to 2, and the process of (II) is repeated after k is increased by 1 until k reaches (n-1);and (III) generating an nth adaptive hyperparameter set using an optimization function constructed with reference to each of the first to (n-1)th adaptive hyperparameter sets and the respective performance evaluation results of the machine learning networks trained in the first to (n-1)th stages of learning, applying the nth adaptive hyperparameter set and performing nth stage learning on the machine learning network trained in the (n-1)th stage using the nth stage learning data, thereby completing the current learning. ; 10. The device according to claim 9, characterized in that: (IV) The processor further performs the following processing: when the performance of the machine learning network does not improve to above a certain threshold after the current learning is completed, the machine learning network is not updated, and the autonomous driving vehicle is operated using the machine learning network until the next on-device learning condition is met; and when the performance of the machine learning network improves to above a certain threshold after the current learning is completed, the machine learning network is updated using the currently learned machine learning network, and the autonomous driving vehicle is operated using the currently learned machine learning network until the next on-device learning condition is met.

11. The device according to claim 9, characterized in that: When the performance of the machine learning network learned in the k-th stage is higher than the performance of the machine learning network learned in the (k-1)-th stage, the processor sets the (k+1)-th adaptive hyperparameter set to the n-th adaptive hyperparameter set to be equal to the k-th adaptive hyperparameter set.

12. The device according to claim 9, characterized in that: In the treatment (II), The processor keeps the adaptive value of at least one hyperparameter included in the first adaptive hyperparameter set to the nth adaptive hyperparameter set the same.

13. The device according to claim 9, characterized in that: When the performance of the machine learning network trained in the k-th stage learning is higher than the performance of the machine learning network trained in the (k-1)-th stage learning, the processor completes the current learning by setting the k-th stage learning as the n-th stage learning.

14. The device according to claim 9, characterized in that: In the process (I), When sensing data obtained by a sensor mounted on the autonomous vehicle is obtained, the machine learning network analyzes the sensing data according to deep learning to generate output data for driving information of the autonomous vehicle. The processor inputs the sensor data and the output data corresponding to each of the sensor data into a data selection network, so that the data selection network selects specific sensor data for training the machine learning network with reference to the output data, and stores the selected specific sensor data as the new learning data.

15. The device according to claim 9, characterized in that: The hyperparameters include at least one of a learning algorithm setting, a mini-batch size, a maximum stage, and a maximum epoch for each stage.

16. The device according to claim 9, characterized in that: In the step (I), The processor selects a hyperparameter set predetermined as a base model of the machine learning network as the default hyperparameter set, or selects an optimal hyperparameter set generated in previous learning as the default hyperparameter set.

Images