Hyperparameter transfer method for deep learning framework involving many unspecified users and apparatus therefor

Abstract

A method for transmitting hyperparameters for a deep learning framework in which an unspecified number of users participate in development, and a deep learning training device for the same are disclosed. A method for transmitting hyperparameters for a deep learning framework according to one embodiment includes the steps of: generating a hyperparameter map by converting hyperparameters into a key and a corresponding value; transmitting a hyperparameter map common to each function of the deep learning framework; and searching for a key valid for each function and obtaining a value.

Description

Technology Field

[0001] The present invention relates to computer vision technology, and more specifically, to deep learning technology. Background Technology

[0002] Deep learning is the process of performing machine learning using artificial neural networks with multiple layers, and is also referred to as deep learning. It is a technology in which machines automatically learn important patterns and rules from large-scale data and perform decision-making or predictions based on this.

[0003] A framework is a type of package that bundles various libraries or modules for developing applications into a single unit to enable efficient use. A deep learning framework provides numerous already verified libraries and various pre-trained deep learning algorithms, allowing users (e.g., developers) to use them quickly and easily. This frees users from the wasteful work of implementing redundant functions and helps them focus solely on developing core algorithms for problem solving.

[0004] While deep learning frameworks must provide the specific features desired by a diverse range of users, they must also conversely allow users to directly add the features they want to the framework. To add features desired by this diverse user base, it is necessary to understand the framework's entire system and analyze its input / output structures. However, both understanding the entire system and analyzing the input / output structures incur significant costs for users who simply desire the development of basic functions. The problem to be solved

[0005] According to one embodiment, a hyperparameter transfer method for a deep learning framework in which an unspecified number of users participate in development and a deep learning training device for the same are proposed. means of solving the problem

[0006] A hyperparameter transfer method for a deep learning framework according to one embodiment converts hyperparameters into a key and a corresponding value to form a hyperparameter map<key : value> It includes a step of generating, a step of passing a hyperparameter map common to each function of the deep learning framework, and a step of searching for a valid key for each function and obtaining a value.

[0007] Hyperparameter map<key : value> In the step of generating, if the key attributes of the hyperparameter map are duplicated, a hyperparameter multi-map can be generated by replacing the part corresponding to the value attribute with a map.

[0008] Hyperparameter map<key : value> In the step of generating, if the value attribute of the hyperparameter map is duplicated, a multi-key map can be generated by assigning multiple parts corresponding to the key.

[0009] Hyperparameter map<key : value> In the step of generating, to add a sub-function of a higher-level function based on user operation signals from an unspecified number of users, a key can be added to the existing hyperparameter map without changing the hyperparameter map transmission method.

[0010] Hyperparameter map<key : value> In the step of generating, hyperparameters based on different programming languages ​​can be generated into a unified hyperparameter map.

[0011] The hyperparameter passing method may further include a step of checking only when there is no valid key value in the hyperparameter map and handling this as an exception.

[0012] A deep learning training device according to another embodiment converts hyperparameters into a key and a corresponding value to form a hyperparameter map<key : value> It includes a map generation unit that generates a map, a map transmission unit that transmits a hyperparameter map common to each function of a deep learning framework, a key search unit that searches for a key valid for each function, and a value acquisition unit that obtains a value using the searched key.

[0013] The map generation unit can generate a hyperparameter multi-map that replaces the part corresponding to the value attribute with a map when the key attribute of the hyperparameter map is duplicated.

[0014] The map generation unit can generate a multi-key map that assigns multiple parts corresponding to the key when the value attributes of the hyperparameter map are duplicated.

[0015] The map generation unit can add a key to an existing hyperparameter map without changing the hyperparameter map transmission method in order to add a lower function of a higher function based on user operation signals from an unspecified number of users.

[0016] The map generation unit can generate hyperparameters based on different programming languages ​​into a unified hyperparameter map.

[0017] The deep learning training device may further include an exception handling unit that investigates only when there is no valid key value in the hyperparameter map and handles this as an exception. Effects of the invention

[0018] According to a method for delivering hyperparameters for a deep learning framework in which an unspecified number of users participate in development according to one embodiment, and a deep learning training device for the same, an unspecified number of users can develop a function that uses hyperparameters, and the developed function can be added without changing the structure of the deep learning framework.

[0019] An unspecified number of users can reduce the cost incurred when analyzing the input / output structure of hyperparameters, increase the stability of feature development by aligning the hyperparameter delivery process, and participate in the feature development of the deep learning framework.

[0020] It can solve problems that occur when passing hyperparameters in deep learning frameworks where an unspecified number of users participate in development.

[0021] For example, in a deep learning framework where an unspecified number of users participate in development, the cost of analyzing input / output structures can be reduced by passing a common hyperparameter map.

[0022] As another example, when adding sub-features, hyperparameter maps can be passed without structural changes to the deep learning framework. Development stability can be improved by performing exception handling only for invalid keys.

[0023] As another example, hyperparameters between different languages ​​can also be reliably transmitted. Brief explanation of the drawing

[0024] FIG. 1 is a diagram illustrating a hyperparameter transfer structure for a deep learning framework in which an unspecified number of users participate in development according to an embodiment of the present invention. FIG. 2 is a diagram illustrating a hyperparameter multi-map structure according to an embodiment of the present invention. FIG. 3 is a drawing illustrating a multi-key map structure according to an embodiment of the present invention. FIG. 4 is a diagram illustrating the configuration of a deep learning training device according to an embodiment of the present invention. FIG. 5 is a diagram generalizing a hyperparameter transfer process according to one embodiment of the present invention, FIG. 6 is a diagram illustrating a hyperparameter map generation process according to an embodiment of the present invention. FIG. 7 is a diagram illustrating a key search and value acquisition process according to an embodiment of the present invention. FIG. 8 is a diagram illustrating a hyperparameter addition process according to an embodiment of the present invention. FIG. 9 is a diagram illustrating different programming language-based map hyperparameter transfer processes according to an embodiment of the present invention, FIG. 10 is a diagram illustrating an exception handling process according to an embodiment of the present invention. Specific details for implementing the invention

[0025] The advantages and features of the present invention and the methods for achieving them will become clear by referring to the embodiments described below in detail together with the accompanying drawings. However, the present invention is not limited to the embodiments disclosed below but can be implemented in various different forms. These embodiments are provided merely to ensure that the disclosure of the present invention is complete and to fully inform those skilled in the art of the scope of the invention, and the present invention is defined only by the scope of the claims. Throughout the specification, the same reference numerals refer to the same components.

[0026] In describing the embodiments of the present invention, if it is determined that a detailed description of known functions or configurations may unnecessarily obscure the essence of the invention, such detailed description will be omitted. Furthermore, the terms described below are defined to reflect the functions in the embodiments of the present invention, and these may vary depending on the intentions or conventions of the user or operator. Therefore, their definitions should be based on the content throughout this specification.

[0027] Hereinafter, embodiments of the present invention will be described in detail with reference to the attached drawings. However, the embodiments of the present invention exemplified below may be modified in various different forms, and the scope of the present invention is not limited to the embodiments described below. The embodiments of the present invention are provided to more completely explain the present invention to those skilled in the art to which this invention pertains.

[0028] FIG. 1 is a diagram illustrating a hyperparameter transfer structure for a deep learning framework in which an unspecified number of users participate in development according to an embodiment of the present invention.

[0029] Referring to FIG. 1, a deep learning training device according to one embodiment unifies the process of transmitting hyperparameters to a deep learning framework and transmits common hyperparameters. Accordingly, the cost of analyzing input / output structures is reduced, development stability is increased, and functions can be added by an unspecified number of users (14) without modifying the entire system. Here, hyperparameters include various parameters set for the training of a neural network. For example, the learning rate, the number of epochs (number of training iterations), weight initialization, etc., can be set through hyperparameters.

[0030] The deep learning training device maps to unify the hyperparameter transfer process<key, value> It uses. Here, map is a class included in the C++ Standard Template Library. A Map has key properties, and each key has a value property that corresponds one-to-one.

[0031] The deep learning training device receives user operation signals from an unspecified number of users (14), accesses a common hyperparameter map (100) within the deep learning framework (10) to add key and value attributes, and transmits the hyperparameter map (100) to each function of the deep learning framework (10). The functions may include, for example, function 1 (110), function 2 (120), and function 3 (130), as illustrated in FIG. 1.

[0032] The deep learning training device can reduce the cost of input / output analysis for adding features by using a common hyperparameter map (100).

[0033] The deep learning training device considers only cases where the key (invalid search key) (140) is invalid in each function when handling exceptions. Accordingly, development costs can be minimized and development stability can be increased. Even if a function is added, only the hyperparameter map (100) needs to be passed to the deep learning framework (10), so no structural change occurs in the entire system.

[0034] FIG. 2 is a diagram illustrating a hyperparameter multi-map structure according to an embodiment of the present invention.

[0035] Referring to FIG. 2, the deep learning training device takes a hyperparameter multi-map structure (200) in which, when the key attributes of the hyperparameter map are duplicated, the part corresponding to the value attribute is replaced with a map, and transmits the hyperparameter map to a deep learning framework. For example, in the hyperparameter multi-map structure (200), when the key attributes are duplicated, such as “modellayer”: layermap and “Modelsize”: sizemap in FIG. 2, the part corresponding to the value attribute is replaced with layermap and sizemap, respectively.

[0036] When using this method, duplicate keys stored in the hyperparameter map can be avoided, and an unspecified number of users can develop by dividing attribute levels without encroaching on each other's areas.

[0037] FIG. 3 is a drawing illustrating a multi-key map structure according to one embodiment of the present invention.

[0038] Referring to FIG. 3, the deep learning training device takes a multi-key map structure (300) that assigns multiple parts corresponding to keys when the value attributes of the hyperparameter map are duplicated, and transmits the hyperparameter map to a deep learning framework. For example, as shown in FIG. 3, the multi-key map assigns multiple keys such as "ModelLayer", "LossLayer".

[0039] In a general map structure, key and value correspond one-to-one. Deep learning frameworks may share the same value during the forward and backpropagation processes. For example, the initial hyperparameter generation location is Model, but the same value must be used in the Loss function. Generating different keys and the same value in each function every time is inefficient due to redundant memory resource usage. Furthermore, human error resulting from redundant input cannot be ignored. To avoid redundant input, if the value is used by searching for the key of the hyperparameter map in Model even though it is inside the Loss function, it incurs a cost for an unspecified number of users to understand the system. To prevent these problems, the deep learning training device uses a multi-key map structure (300) that uses the same value but has different key values.

[0040] FIG. 4 is a diagram illustrating the configuration of a deep learning training device according to an embodiment of the present invention.

[0041] Referring to FIG. 4, the deep learning training device (4) includes an input unit (40), a processor (42), an output unit (44), and a storage unit (46).

[0042] The input unit (40) receives a user operation signal and transmits it to the processor (42). For example, it receives a user operation signal for generating and transmitting a hyperparameter map.

[0043] The processor (42) controls the overall configuration of the deep learning training device (4) for transmitting a hyperparameter map. According to one embodiment, the processor (42) includes a map generation unit (421), a map transmission unit (422), a key search unit (423), a value acquisition unit (424), and an exception handling unit (425).

[0044] The map generation unit (421) converts hyperparameters into keys and corresponding values ​​to form a hyperparameter map<key : value> Creates.

[0045] The map generation unit (421) can generate a hyperparameter multi-map structure in which, if the key attribute of the hyperparameter map is duplicated, the part corresponding to the value attribute is replaced with a map.

[0046] The map generation unit (421) can generate a multi-key map structure that assigns multiple parts corresponding to the key when the value attribute of the hyperparameter map is duplicated.

[0047] The map generation unit (421) can add a key to an existing hyperparameter map without changing the hyperparameter map transmission method in order to add a lower function of a higher function based on a user operation signal from an unspecified number of users. An example of this is described later with reference to FIG. 8.

[0048] The map generation unit (421) can generate hyperparameters based on different programming languages ​​into a unified hyperparameter map. An example of this is described later with reference to FIG. 9.

[0049] The map delivery unit (422) delivers a hyperparameter map common to each function of the deep learning framework. Examples of functions include Model, Loss, Optimizer, etc.

[0050] The key search unit (423) searches for a key valid for each function, and the value acquisition unit (424) acquires a value using the key searched through the key search unit (423).

[0051] The exception handling unit (425) investigates only when there is no valid key value in the hyperparameter map and handles it as an exception. An example of exception handling will be described later with reference to FIG. 10.

[0052] The output unit (44) outputs the result of an operation through the processor (42), and the storage unit (46) stores the result of an operation through the processor (42) or provides data necessary for the operation of the processor (42) to the processor (42).

[0053] FIG. 5 is a diagram generalizing a hyperparameter transfer process according to one embodiment of the present invention.

[0054] Referring to FIG. 5, the hyperparameter passing process includes a hyperparameter map generation process (Fig. 6), a key search and value acquisition process (Fig. 7), a hyperparameter addition process (Fig. 8), a map hyperparameter passing process based on different programming languages ​​(Fig. 9), and an exception handling process (Fig. 10). Below, each process will be described in detail with reference to FIGS. 6 to 10.

[0055] FIG. 6 is a diagram illustrating a hyperparameter map generation process according to an embodiment of the present invention.

[0056] Referring to FIGS. 4, 5, and 6, the deep learning training device (4) generates a map by converting hyperparameters into a key and a value corresponding to the key. The Map is<key : value> It has attributes.

[0057] FIG. 7 is a diagram illustrating a key search and value acquisition process according to an embodiment of the present invention.

[0058] Referring to FIGS. 4, 5 and 7, the deep learning training device (1) transmits a hyperparameter map common to each function (Model, Loss, Optimizer, etc.) of the deep learning framework, and searches for a key required in each function (Search key) to obtain a value (Get value).

[0059] FIG. 8 is a diagram illustrating a hyperparameter addition process according to one embodiment of the present invention.

[0060] Referring to FIGS. 4, 5, and 8, the deep learning training device (1) does not change the hyperparameter transmission method even when adding lower functions to upper functions by user operation signals from an unspecified number of users. To add new hyperparameters, the deep learning training device (4) can transmit by adding a key to the existing hyperparameter map.

[0061] FIG. 9 is a diagram illustrating a map hyperparameter transfer process based on different programming languages ​​according to an embodiment of the present invention.

[0062] Referring to FIGS. 4, 5, and 9, hyperparameters between various programming languages ​​(e.g., C, C++, Java, Python, etc.) can be reliably passed to the corresponding functions by constructing a unified map structure and passing it. For example, hyperparameters created in Python can be passed to the corresponding functions of a deep learning framework by constructing a map structure identical to the map structure used to pass hyperparameters created in C++.

[0063] FIG. 10 is a diagram illustrating an exception handling process according to an embodiment of the present invention.

[0064] Referring to FIGS. 4, 5, and 10, the hyperparameter transmission device (1) investigates only when there is no valid key value in the hyperparameter map and handles this as an exception. Accordingly, development costs can be minimized.

[0065] The present invention has been described above with reference to its embodiments. Those skilled in the art will understand that the present invention may be implemented in modified forms without departing from the essential characteristics of the invention. Therefore, the disclosed embodiments should be considered in an illustrative rather than a restrictive sense. The scope of the invention is defined by the claims, not by the foregoing description, and all variations within the scope of the claims should be interpreted as being included in the invention.

Claims

Claim 1 In a method for transferring hyperparameters for a deep learning framework in which an unspecified number of users participate in development, a deep learning training device converts hyperparameters into a key and a corresponding value to form a hyperparameter map<key : value> A method for transferring hyperparameters for a deep learning framework, characterized by comprising: a step of generating; a step of transferring a common hyperparameter map to a plurality of functions operating independently within the deep learning framework; and a step of each function searching for a key corresponding to the function in the hyperparameter map to obtain a value. Claim 2 In claim 1, the hyperparameter map<key : value> A method for passing hyperparameters for a deep learning framework, characterized in that the step of generating a hyperparameter multi-map is generated by replacing the part corresponding to the value attribute with a map when the key attribute of the hyperparameter map is duplicated. Claim 3 In claim 1, the hyperparameter map<key : value> A method for passing hyperparameters for a deep learning framework, characterized in that the step of generating a multi-key map is generated by assigning multiple parts corresponding to the key when the value attribute of the hyperparameter map is duplicated. Claim 4 In claim 1, the hyperparameter map<key : value> A hyperparameter transfer method for a deep learning framework, characterized by adding a key to an existing hyperparameter map without changing the hyperparameter map transfer method in order to add a lower function of a higher function by user operation signals of an unspecified number of users in the step of generating. Claim 5 In claim 1, the hyperparameter map<key : value> A method for passing hyperparameters for a deep learning framework, characterized in that the step of generating [the element] generates hyperparameters based on different programming languages ​​into a unified hyperparameter map. Claim 6 A hyperparameter passing method for a deep learning framework, characterized in that, in claim 1, the hyperparameter passing method further includes the step of investigating only cases where there is no valid key value in the hyperparameter map and handling such cases as exceptions. Claim 7 In a deep learning training device for transmitting hyperparameters for a deep learning framework, the hyperparameters are converted into a key and a corresponding value to form a hyperparameter map<key : value> A deep learning learning device characterized by comprising: a map generation unit that generates a map; a map transmission unit that transmits a common hyperparameter map to a plurality of functions operating independently within a deep learning framework; a key search unit in which each function searches for a key corresponding to the function in the hyperparameter map; and a value acquisition unit that acquires a value using the searched key. Claim 8 A deep learning training device according to claim 7, wherein the map generation unit generates a hyperparameter multi-map by replacing the part corresponding to the value attribute with a map when the key attribute of the hyperparameter map is duplicated. Claim 9 A deep learning training device according to claim 7, wherein the map generation unit generates a multi-key map in which the parts corresponding to the key are assigned multiple times when the value attributes of the hyperparameter map are duplicated. Claim 10 A deep learning training device according to claim 7, wherein the map generation unit adds a key to an existing hyperparameter map without changing the hyperparameter map transmission method in order to add a lower function of a higher function by means of a user operation signal from an unspecified number of users. Claim 11 A deep learning training device according to claim 7, wherein the map generation unit generates hyperparameters based on different programming languages ​​into a unified hyperparameter map. Claim 12 In claim 7, the deep learning training device further comprises an exception handling unit that investigates only cases where there is no valid key value in the hyperparameter map and handles such cases as exceptions.

Images