Machine learning parallelization method using host CPU with multi-socket structure and apparatus therefor

Abstract

A machine learning parallelization method using a host CPU with a multi-socket structure and an apparatus for the same are disclosed. A machine learning parallelization method according to one embodiment of the present invention includes a machine learning parallelization apparatus using a host CPU with a multi-socket structure, comprising: a compilation step of dividing a learning model into pipeline stages at the layer level and assigning them to NUMA (NON-UNIFORM MEMORY ACCESS) nodes for each CPU socket; and a runtime step of initializing parameters required for learning and assigning a plurality of threads created considering a policy for each parallelization algorithm to each core included in the NUMA node for execution.

Description

Technology Field

[0001] The present invention relates to a machine learning parallelization technology using a host CPU with a multi-socket structure, and more specifically, to a technology that enables parallel learning and inference while minimizing performance load when performing large-scale machine learning based on a host CPU on a general multi-socket-based server without a separate computing device such as a GPU. Background Technology

[0002] With the recent explosive proliferation of artificial intelligence and deep learning technologies, the demands for AI accuracy and performance have also risen rapidly. Consequently, the scale of AI models is increasing exponentially, a trend that is outpacing the development of related hardware. Taking GPUs—the most widely used devices for AI training and inference—as an example, the computational and memory requirements of currently popular large-scale models have increased so dramatically that a single GPU can no longer handle them. As a result, to provide training and services utilizing large-scale AI models, it has become inevitable to use methods that distribute models across systems equipped with multiple GPUs for parallel training and inference. In this scenario, when a single model is split and executed across multiple GPUs, the model must be distributed and stored in the memory of multiple non-shared GPUs. This requires collective communication, which incurs a massive communication load, and is one of the primary causes of performance degradation in parallel training.

[0003] While GPUs are the most widely used computing devices for artificial intelligence training and inference, recently released host CPUs are also being equipped with numerous additional features necessary for deep learning acceleration. Consequently, techniques are continuously being introduced that enable machine learning to be performed using only the host CPU, without the need for GPUs or separate computing devices, by accelerating various computations required for deep learning through the utilization of AVX-512, a representative SIMD (Single Instruction Multiple Data) operation of the X86 architecture. It is expected that the utility of such host CPU-based machine learning will increase further in the future.

[0004] The biggest advantage of host CPU-based machine learning is the ability to directly utilize large-scale shared system memory. GPU memory is small and not shared, making it difficult to process large-scale models in a distributed manner. On the other hand, using large system memory allows models to be loaded into memory without partitioning, and since this memory is shared by all CPU cores, inter-core communication is possible through it. Furthermore, the performance of the local bus connecting CPU sockets within a node and the Network-on-Chip (NoC) connecting cores within the socket is excellent, allowing for the minimization of communication load.

[0005] However, most multi-socket servers equipped with multiple CPUs are designed based on the NUMA (Non-Uniform Memory Access) architecture. This structure shares memory connected to each CPU socket, resulting in lower performance when referencing memory in other CPU sockets compared to when referencing memory in one's own socket. When conducting CPU-based distributed learning, it is necessary to recognize this NUMA structure and implement techniques that enable parallel learning optimized for it.

[0006] In other words, in order to divide and execute computations and variables constituting an artificial intelligence model across multiple NUMA nodes, it is necessary to recognize the NUMA structure and perform the division. Furthermore, since most of the various parallelization techniques recently proposed for large-scale machine learning (pipeline parallelization, tensor parallelization, data parallelization, etc.) were proposed assuming a multi-GPU environment, a new method is required to effectively apply them to a host CPU-based deep learning environment. Prior art literature

[0007] U.S. Published Patent US2021 / 0149729, published May 20, 2021 (Title: TASK SCHEDULING FOR MACHINE-LEARNING WORKLOADS) The problem to be solved

[0008] The objective of the present invention is to enable efficient large-scale machine learning while minimizing performance load during host CPU-based parallel machine learning by utilizing the layered physical characteristics of a multi-socket system.

[0009] In addition, the objective of the present invention is to effectively parallelize distributed machine learning models using a host CPU and system memory by utilizing the physical characteristics of a NUMA node system without a separate computing device such as a GPU.

[0010] In addition, the objective of the present invention is to improve performance during parallel training and inference of large-scale models by applying a parallelization technique that minimizes the load level by utilizing the performance gap of multi-layer interconnects constituting the system. means of solving the problem

[0011] A machine learning parallelization method using a host CPU with a multi-socket structure according to the present invention for achieving the above-mentioned purpose comprises: a machine learning parallelization device using a host CPU with a multi-socket structure that divides a learning model into pipeline stages at the layer level and assigns them to NUMA (NON-UNIFORM MEMORY ACCESS) nodes for each CPU socket; and a runtime step that initializes parameters required for learning and assigns a plurality of threads generated considering a policy for each parallelization algorithm to a plurality of cores included in the NUMA nodes for execution.

[0012] At this time, a NUMA node per CPU socket is composed of a CPU including multiple cores and memory, and the multiple cores share the memory through an interconnect between cores, and the NUMA node per CPU socket can share the memory per NUMA node through an interconnect between sockets.

[0013] At this time, the default value of the number of pipeline stages is set to correspond to the number of NUMA nodes, and an equal number of model operations can be distributed to each NUMA node.

[0014] In this case, the parameters may include global parameters for data sharing among the plurality of threads and local parameters used individually by each of the plurality of threads.

[0015] At this time, the local parameters can store the derivative value (GRADIENT) for the error value used in the backpropagation process of the learning model and the optimizer state that determines whether to apply the derivative value.

[0016] At this time, the runtime step may include a step of synchronizing the execution of threads allocated to each NUMA node; and a step of updating parameters to each NUMA node based on the global parameters.

[0017] At this time, the step of updating the parameters can be performed by updating the parameters in either a synchronous manner or an asynchronous manner in which the aforementioned multiple threads update.

[0018] In addition, a machine learning parallelization device using a host CPU with a multi-socket structure according to an embodiment of the present invention includes: a processor that divides a learning model into pipeline stages at the layer level and assigns it to NUMA (NON-UNIFORM MEMORY ACCESS) nodes for each CPU socket, initializes parameters required for learning, and assigns a plurality of threads created by considering a policy for each parallelization algorithm to a plurality of cores included in the NUMA nodes for execution; and a memory that stores the parallelization algorithm.

[0019] At this time, a NUMA node per CPU socket is composed of a CPU including multiple cores and memory, and the multiple cores share the memory through an interconnect between cores, and the NUMA node per CPU socket can share the memory per NUMA node through an interconnect between sockets.

[0020] At this time, the default value of the number of pipeline stages is set to correspond to the number of NUMA nodes, and an equal number of model operations can be distributed to each NUMA node.

[0021] In this case, the parameters may include global parameters for data sharing among the plurality of threads and local parameters used individually by each of the plurality of threads.

[0022] At this time, the local parameters can store the derivative value (GRADIENT) for the error value used in the backpropagation process of the learning model and the optimizer state that determines whether to apply the derivative value.

[0023] At this time, the processor can synchronize the execution of threads allocated to each NUMA node and update parameters to each NUMA node based on the global parameters.

[0024] At this time, the processor can perform parameter updates in either a synchronous update method or an asynchronous update method in which the plurality of threads update. Effects of the invention

[0025] According to the present invention, by utilizing the layered physical characteristics of a multi-socket system, efficient large-scale machine learning can be performed while minimizing the performance load when performing host CPU-based parallel machine learning.

[0026] In addition, the present invention can effectively parallelize distributed machine learning models using a host CPU and system memory by utilizing the physical characteristics of a NUMA node system without a separate computing device such as a GPU.

[0027] In addition, the present invention can improve performance during parallel training and inference of large-scale models by applying a parallelization technique that minimizes the load level by utilizing the performance gap of multi-layer interconnects constituting the system. Brief explanation of the drawing

[0028] FIG. 1 is an operation flowchart illustrating a machine learning parallelization method using a host CPU with a multi-socket structure according to an embodiment of the present invention. FIG. 2 is a diagram showing an example of a host CPU with a multi-socket structure according to the present invention. FIG. 3 is an operation flowchart showing in detail the compilation step of a machine learning parallelization method according to an embodiment of the present invention. FIG. 4 is an operation flowchart showing in detail the runtime step of a machine learning parallelization method according to an embodiment of the present invention. FIGS. 5 to 7 are drawings illustrating an example of multi-thread-based model parallel execution utilizing a host CPU with a multi-socket structure according to the present invention. FIG. 8 is a block diagram showing a machine learning parallelization device using a host CPU with a multi-socket structure according to an embodiment of the present invention. FIG. 9 is a block diagram showing a machine learning parallelization device using a host CPU with a multi-socket structure according to another embodiment of the present invention. FIG. 10 is a block diagram showing the runtime module illustrated in FIG. 9 in detail. FIG. 11 is a drawing showing a computer system according to an embodiment of the present invention. Specific details for implementing the invention

[0029] The present invention will be described in detail below with reference to the accompanying drawings. Hereinafter, repetitive descriptions and detailed descriptions of known functions and configurations that may unnecessarily obscure the essence of the invention are omitted. Embodiments of the present invention are provided to more fully explain the invention to those with average knowledge in the art. Accordingly, the shapes and sizes of elements in the drawings may be exaggerated for clearer explanation.

[0030] In this document, each of the phrases such as "A or B", "at least one of A and B", "at least one of A or B", "A, B or C", "at least one of A, B and C", and "at least one of A, B, or C" may include any one of the items listed together in the corresponding phrase, or all possible combinations thereof.

[0031] Hereinafter, preferred embodiments according to the present invention will be described in detail with reference to the attached drawings.

[0033] FIG. 1 is an operation flowchart illustrating a machine learning parallelization method using a host CPU with a multi-socket structure according to an embodiment of the present invention.

[0034] Referring to FIG. 1, a machine learning parallelization method using a host CPU with a multi-socket structure according to an embodiment of the present invention performs a compilation step in which a machine learning parallelization device using a host CPU with a multi-socket structure divides a learning model into pipeline stages at the layer level and assigns it to NUMA (NON-UNIFORM MEMORY ACCESS) nodes for each CPU socket (S110).

[0035] At this time, a NUMA node per CPU socket is composed of a CPU including multiple cores and memory, and the multiple cores share memory through inter-core interconnects, and a NUMA node per CPU socket can share memory per NUMA node through inter-socket interconnects.

[0036] For example, FIG. 2 shows a host CPU with a multi-socket structure for performing parallel machine learning according to an embodiment of the present invention. According to the example of FIG. 2, the host CPU has four CPU sockets, and a CPU (210, 220, 230, 240) having four cores can be installed in each socket. Each of the four-core CPUs (210, 220, 230, 240) can be connected to memory (211, 221, 231, 241) to form a NUMA node.

[0037] At this time, cores belonging to the same CPU can share the memory of the corresponding NUMA node, and each core can access the memory with the same performance.

[0038] Additionally, the cores within the CPU (210, 220, 230, 240) are interconnected through a core interconnect (201), and each NUMA node can be connected to each other through a socket interconnect (202), but the socket interconnect (202) shows lower performance compared to the core interconnect (201).

[0039] Therefore, at this time, memory access to other NUMA nodes shows lower performance compared to memory access to the same NUMA node.

[0040] At this time, the default value of the number of pipeline stages is set to correspond to the number of NUMA nodes, and an equal number of model operations can be distributed to each NUMA node.

[0041] For example, Figure 3 illustrates in detail the process of performing pipeline parallelization during the compilation stage. First, model operations can be distributed to each stage by considering the number of pipeline stages and the number of operations of the corresponding model, i.e., the number of model layers (S310).

[0042] At this time, since the performance of interconnect between sockets (interconnect between NUMA nodes) is lower than the performance of interconnect between cores within a socket, and since pipeline parallelization is less sensitive to interconnect performance than tensor parallelization or data parallelization, the default value of the number of pipeline stages can be set to the number of NUMA nodes in the system.

[0043] If the number of layers in the model is l and the number of pipeline stages is n, the number of layers ki allocated per pipeline stage can be calculated as [Equation 1].

[0044] [Mathematical Formula 1]

[0045]

[0046] This method corresponds to allocating an equal number of layers per pipeline stage under the assumption that the performance load of each layer constituting the model is at the same level. However, in the case of models with significant performance variance between layers, a method may be applied in which the performance load of each layer is measured through profiling and then allocated based on this assessment.

[0047] In addition, a machine learning parallelization method using a host CPU with a multi-socket structure according to an embodiment of the present invention performs a runtime step in which a machine learning parallelization device using a host CPU with a multi-socket structure initializes parameters required for learning and allocates a plurality of threads created considering a policy for each parallelization algorithm to a plurality of cores included in a NUMA node to execute them (S120).

[0048] In this case, the parameters may include global parameters for data sharing among multiple threads and local parameters used individually by each of the multiple threads.

[0049] For example, once the model layer distribution for each pipeline stage is completed through the compilation phase, global parameters can be created to share output and input values ​​between adjacent pipeline stages. These created or defined global parameters can be used for data sharing between threads responsible for different pipeline stages.

[0050] At this time, the local parameters can store the derivative value (GRADIENT) for the error value used in the backpropagation process of the learning model and the optimizer state that determines whether to apply the derivative value.

[0051] For example, [Table 1] shows an example of code implementing a machine learning model written by a user.

[0052] parameter_1[]...[];parameter_2[]...[];...parameter_16[]...[];model(...) { optimizer_state_1[]...[]; optimizer_state_2[]...[]; ... optimizer_state_16[]...[]; gradient_1[]...[]; gradient_2[]...[]; ... gradient_16[]...[]; operation_1(); operation_2(); ... operation_8(); operation_8_back(); operation_7_back(); ... operation_1_back();}

[0053] For convenience, the example in [Table 1] above will be explained by assuming a machine learning model defined by 8 operations.

[0054] In this case, if the model is used only for inference, only the forward operation needs to be defined, and if the model is trained, the backpropagation operation must also be defined. For example, in [Table 1], the operation defined as 'operation_X' corresponds to the forward operation, and the operation defined as 'operation_X_back' corresponds to the back operation.

[0055] The machine learning model defined in this way can subsequently be executed in parallel in the form of multiple threads.

[0056] In this case, since parameters are shared by multiple threads during parallel execution, they can be defined as global parameters to share data or information between threads. For example, the parameter defined as 'parameter_X[][]' in [Table 1] can correspond to a global parameter.

[0057] In addition, during the backpropagation process performed when training a model, additionally required are the derivative of the loss value (Gradient), an optimizer that determines how to apply the derivative value when updating parameters, and the parameter update operation.

[0058] Therefore, parameters must be declared to store derivative values ​​(gradient_X[]) and optimizer states (optimizer_state_X[]) required for the model training process. Since derivative values ​​and optimizer states are not shared between threads and are used individually, they can be declared as local parameters.

[0059] Recently, many deep learning libraries provide automatic differentiation functions, so by using them, the backpropagation process can be automatically generated and executed simply by defining the model's forward process. When using such automatic differentiation functions, you only need to define and specify the model's forward operations, and the aforementioned derivative values ​​and optimizer states may not need to be defined separately as they are automatically generated.

[0060] At this time, the execution of threads allocated to each NUMA node can be synchronized.

[0061] At this time, parameters can be updated for each NUMA node based on global parameters.

[0062] At this time, multiple threads can perform parameter updates using either a synchronous update method or an asynchronous update method.

[0063] As described above, the present invention is broadly divided into a compilation phase and a runtime phase. In the compilation phase, model partitioning reflecting the characteristics of the target system is performed, and in the runtime phase, the invention can be operated by creating and managing multiple threads based on this. Accordingly, the pipeline parallelization task is handled by the compilation function, and subsequent tasks can be handled by the runtime function.

[0064] For example, through pipeline parallelization, the model is split into pipeline stages at the layer level. Once the compilation stage is complete, the model code written by the user is converted into a multi-threaded form according to the parallelization settings, compiled, and generated as a final executable file. This generated executable file can be run using runtime functions; first, parameter values ​​are initialized, then multiple threads are created and assigned to CPU cores according to the policy for each parallelization algorithm, and the model training can begin by executing the threads assigned to each core.

[0065] At this time, there are three commonly used model parallelization methods: pipeline parallelization, data parallelization, and tensor parallelization. In the compilation step according to the present invention, only pipeline parallelization was processed, for the following reasons.

[0066] The most important task in data parallelization is updating thread parameters through global parameters, and since this is a task that can be processed at the runtime stage of the present invention, it does not require a separate compilation process.

[0067] The purpose of performing tensor parallelization is to partition the interior of a layer when a single layer of a model cannot be loaded into limited GPU memory. However, in the case of host CPU-based deep learning assumed by the present invention, all threads responsible for the same pipeline stage are executed on the same NUMA node and already share the memory of that NUMA node. In this situation, tensor parallelization to partition the interior of a model layer is not meaningful for performance improvement, so it is not considered in the present invention.

[0068] Through a machine learning parallelization method using a host CPU with such a multi-socket structure, it is possible to perform efficient large-scale machine learning while minimizing performance load during host CPU-based parallel machine learning by utilizing the layered physical characteristics of the multi-socket system.

[0069] In addition, distributed machine learning models using the host CPU and system memory can be effectively parallelized by utilizing the physical characteristics of the NUMA node system without a separate computing device such as a GPU.

[0070] In addition, performance can be improved during parallel training and inference of large-scale models by applying parallelization techniques that minimize load levels by utilizing the performance gaps of the multi-layer interconnects constituting the system.

[0072] FIGS. 5 to 7 are drawings illustrating an example of multi-thread-based model parallel execution utilizing a host CPU with a multi-socket structure according to the present invention.

[0073] First, Figure 5 illustrates an example of a multi-threaded model parallel execution utilizing the host CPU in a server equipped with four CPUs each having four cores.

[0074] At this time, assuming that the total number of layers (operations) constituting the model is 8, if pipeline parallelization is performed, a total of 4 pipeline stages can be allocated, one per CPU socket as shown in Fig. 5, and then 2 layers can be allocated to each stage.

[0075] Afterwards, as shown in FIG. 6, parallelization can be performed to process a total of four input data in parallel by applying separate input data (DATA_0 ~ DATA_3) to each thread (610~640) placed in each core within the same CPU socket.

[0076] At this time, the threads at each pipeline stage processing the same input data can perform pipeline parallel processing by passing the current output value as the input value of the corresponding thread in the next socket, as shown in FIG. 7. Accordingly, in a server equipped with four CPUs each having four cores, a total of 16 threads can be created to perform learning, and the synchronization between these threads can be performed during the runtime process described later through FIG. 10.

[0078] FIG. 8 is a block diagram showing a machine learning parallelization device using a host CPU with a multi-socket structure according to an embodiment of the present invention.

[0079] Referring to FIG. 8, a machine learning parallelization device using a host CPU with a multi-socket structure according to an embodiment of the present invention includes a communication unit (810), a processor (820), and a memory (830).

[0080] The communication unit (810) can transmit and receive information for machine learning parallelization through a communication network.

[0081] The processor (820) performs a compilation step that divides the learning model into pipeline stages at the layer level and assigns them to NUMA (NON-UNIFORM MEMORY ACCESS) nodes per CPU socket.

[0082] At this time, a NUMA node per CPU socket is composed of a CPU including multiple cores and memory, and the multiple cores share memory through inter-core interconnects, and a NUMA node per CPU socket can share memory per NUMA node through inter-socket interconnects.

[0083] For example, FIG. 2 shows a host CPU with a multi-socket structure for performing parallel machine learning according to an embodiment of the present invention. According to the example of FIG. 2, the host CPU has four CPU sockets, and a CPU (210, 220, 230, 240) having four cores can be installed in each socket. Each of the four-core CPUs (210, 220, 230, 240) can be connected to memory (211, 221, 231, 241) to form a NUMA node.

[0084] At this time, cores belonging to the same CPU can share the memory of the corresponding NUMA node, and each core can access the memory with the same performance.

[0085] Additionally, the cores within the CPU (210, 220, 230, 240) are interconnected through a core interconnect (201), and each NUMA node can be connected to each other through a socket interconnect (202), but the socket interconnect (202) shows lower performance compared to the core interconnect (201).

[0086] Therefore, at this time, memory access to other NUMA nodes shows lower performance compared to memory access to the same NUMA node.

[0087] At this time, the default value of the number of pipeline stages is set to correspond to the number of NUMA nodes, and an equal number of model operations can be distributed to each NUMA node.

[0088] For example, Figure 3 illustrates in detail the process of performing pipeline parallelization during the compilation stage. First, model operations can be distributed to each stage by considering the number of pipeline stages and the number of operations of the corresponding model, i.e., the number of model layers (S310).

[0089] At this time, since the performance of interconnect between sockets (interconnect between NUMA nodes) is lower than the performance of interconnect between cores within a socket, and since pipeline parallelization is less sensitive to interconnect performance than tensor parallelization or data parallelization, the default value of the number of pipeline stages can be set to the number of NUMA nodes in the system.

[0090] If the number of layers in the model is l and the number of pipeline stages is n, the number of layers ki allocated per pipeline stage can be calculated as [Equation 1].

[0091] This method corresponds to allocating an equal number of layers per pipeline stage under the assumption that the performance load of each layer constituting the model is at the same level. However, in the case of models with significant performance variance between layers, a method may be applied in which the performance load of each layer is measured through profiling and then allocated based on this assessment.

[0092] Additionally, the processor (820) initializes parameters required for learning and performs a runtime step of allocating multiple threads created by considering a policy for each parallelization algorithm to multiple cores included in the NUMA node to execute them.

[0093] In this case, the parameters may include global parameters for data sharing among multiple threads and local parameters used individually by each of the multiple threads.

[0094] For example, once the model layer distribution for each pipeline stage is completed through the compilation phase, global parameters can be created to share output and input values ​​between adjacent pipeline stages. These created or defined global parameters can be used for data sharing between threads responsible for different pipeline stages.

[0095] At this time, the local parameters can store the derivative value (GRADIENT) for the error value used in the backpropagation process of the learning model and the optimizer state that determines whether to apply the derivative value.

[0096] For example, [Table 1] shows an example of code implementing a machine learning model written by a user.

[0097] For convenience, the example in [Table 1] above will be explained by assuming a machine learning model defined by 8 operations.

[0098] In this case, if the model is used only for inference, only the forward operation needs to be defined, and if the model is trained, the backpropagation operation must also be defined. For example, in [Table 1], the operation defined as 'operation_X' corresponds to the forward operation, and the operation defined as 'operation_X_back' corresponds to the back operation.

[0099] The machine learning model defined in this way can subsequently be executed in parallel in the form of multiple threads.

[0100] In this case, since parameters are shared by multiple threads during parallel execution, they can be defined as global parameters to share data or information between threads. For example, the parameter defined as 'parameter_X[][]' in [Table 1] can correspond to a global parameter.

[0101] In addition, during the backpropagation process performed when training a model, additionally required are the derivative of the loss value (Gradient), an optimizer that determines how to apply the derivative value when updating parameters, and the parameter update operation.

[0102] Therefore, parameters must be declared to store derivative values ​​(gradient_X[]) and optimizer states (optimizer_state_X[]) required for the model training process. Since derivative values ​​and optimizer states are not shared between threads and are used individually, they can be declared as local parameters.

[0103] Recently, many deep learning libraries provide automatic differentiation functions, so by using them, the backpropagation process can be automatically generated and executed simply by defining the model's forward process. When using such automatic differentiation functions, you only need to define and specify the model's forward operations, and the aforementioned derivative values ​​and optimizer states may not need to be defined separately as they are automatically generated.

[0104] At this time, the execution of threads allocated to each NUMA node can be synchronized.

[0105] At this time, parameters can be updated for each NUMA node based on global parameters.

[0106] At this time, multiple threads can perform parameter updates using either a synchronous update method or an asynchronous update method.

[0107] Memory (830) stores parallelization algorithms.

[0108] Additionally, the memory (830) stores various information generated in the machine learning parallelization device according to an embodiment of the present invention as described above.

[0109] According to an embodiment, the storage unit (830) may be configured independently of the machine learning parallelization device to support functions for machine learning parallelization. In this case, the storage unit (830) may operate as a separate large-capacity storage and may include control functions for performing operations.

[0110] Through a machine learning parallelization device using a host CPU with such a multi-socket structure, it is possible to perform efficient large-scale machine learning while minimizing performance load during host CPU-based parallel machine learning by utilizing the layered physical characteristics of the multi-socket system.

[0111] In addition, distributed machine learning models using the host CPU and system memory can be effectively parallelized by utilizing the physical characteristics of the NUMA node system without a separate computing device such as a GPU.

[0112] In addition, performance can be improved during parallel training and inference of large-scale models by applying parallelization techniques that minimize load levels by utilizing the performance gap of the multi-layer interconnects constituting the system.

[0114] FIG. 9 is a block diagram showing a machine learning parallelization device using a host CPU with a multi-socket structure according to another embodiment of the present invention.

[0115] Referring to FIG. 9, a machine learning parallelization device using a host CPU with a multi-socket structure according to another embodiment of the present invention may be composed of a compilation module (910) and a runtime module (920).

[0116] At this time, the compilation module (910) can perform model partitioning that reflects the characteristics of the target system, and the runtime module (920) can be operated in a manner that creates and manages multiple threads based on the results performed by the compilation module (910). Accordingly, the pipeline parallelization task is handled by the compilation module (910), and subsequent tasks can be handled by the runtime module (920).

[0118] FIG. 10 is a block diagram showing the runtime module illustrated in FIG. 9 in detail.

[0119] Referring to FIG. 10, a runtime module (920) according to one embodiment of the present invention may be largely composed of a thread initialization and creation / termination management unit (1010), a pipeline parallel execution management unit (1020), and a data parallel execution management unit (1030).

[0120] The thread initialization and creation / termination management unit (1010) can initialize parameters for deep learning execution and manage the creation and termination of threads.

[0121] The pipeline parallel execution management unit (1020) can be responsible for transmitting step-by-step output and input values ​​when pipeline parallelization is applied, and synchronizing thread execution according to the scheduling policy.

[0122] The data parallel execution management unit (1030) can be responsible for thread synchronization tasks related to parameter updates for data parallel execution of the model.

[0123] In this case, parameter updates can be performed through global parameters shared among threads, and a method in which all threads update synchronously or an asynchronous method for performance improvement can be selectively utilized.

[0125] FIG. 11 is a drawing showing a computer system according to an embodiment of the present invention.

[0126] Referring to FIG. 11, an embodiment of the present invention may be implemented in a computer system such as a computer-readable recording medium. As illustrated in FIG. 11, the computer system (1100) may include one or more processors (1110), memory (1130), user input device (1140), user output device (1150), and storage (1160) that communicate with each other via a bus (1120). Additionally, the computer system (1100) may further include a network interface (1170) connected to a network (1180). The processor (1110) may be a central processing unit or a semiconductor device that executes processing instructions stored in memory (1130) or storage (1160). Memory (1130) and storage (1160) may be various forms of volatile or non-volatile storage media. For example, memory may include ROM (1131) or RAM (1132).

[0127] Accordingly, embodiments of the present invention may be implemented as a computer-implemented method or as a non-transient computer-readable medium on which computer-executable instructions are recorded. When computer-readable instructions are executed by a processor, computer-readable instructions may perform a method according to at least one aspect of the present invention.

[0129] As described above, the machine learning parallelization method using a host CPU with a multi-socket structure according to the present invention and the apparatus therefor are not limited to the configurations and methods of the embodiments described above; rather, all or part of each embodiment may be selectively combined to allow for various modifications of the embodiments. Explanation of the symbols

[0130] 201: Inter-core interconnect 202: Inter-socket interconnect 210, 220, 230, 240: CPU 211, 221, 231, 241: Memory 610, 620, 630, 640: Multithreaded 810: Communication unit 820, 1110: Processors 830, 1130: Memory 910: Compile module 920: Runtime module 1010: Thread Initialization and Creation / Termination Management Section 1020: Pipeline Parallel Execution Management Unit 1030: Data Parallel Execution Management Unit 1100: Computer System 1120: Bus 1131: Rome 1132: RAM 1140: User input device 1150: User output device 1160: Storage 1170: Network Interface 1180: Network

Claims

Claim 1 A machine learning parallelization method performed by a machine learning parallelization device using a host CPU with a multi-socket structure, comprising: a compilation step of dividing a learning model into pipeline stages at the layer level and assigning them to NUMA (Non-Uniform Memory Access) nodes for each CPU socket; and a runtime step of initializing parameters required for learning and assigning multiple threads generated considering a policy for each parallelization algorithm to multiple cores included in the NUMA nodes for execution, wherein the NUMA nodes for each CPU socket are composed of a CPU including multiple cores and memory, the multiple cores share the memory through inter-core interconnects, and the NUMA nodes for each CPU socket share memory for each NUMA node through inter-socket interconnects. Claim 2 delete Claim 3 A machine learning parallelization method using a host CPU with a multi-socket structure, characterized in that, in claim 1, the default value of the number of pipeline stages is set to correspond to the number of NUMA nodes, and the same number of model operations are distributed to each NUMA node. Claim 4 A machine learning parallelization method using a host CPU with a multi-socket structure, characterized in that, in claim 1, the parameters include a global parameter for data sharing among the plurality of threads and a local parameter used individually for each of the plurality of threads. Claim 5 A machine learning parallelization method using a host CPU with a multi-socket structure, characterized in that, in claim 4, the local parameters store a derivative value (GRADIENT) for an error value used in the backpropagation process of the learning model and an optimizer state that determines whether to apply the derivative value. Claim 6 A machine learning parallelization method using a host CPU with a multi-socket structure, characterized in that, in claim 4, the runtime step includes the step of synchronizing the execution of threads allocated to each NUMA node; and the step of updating parameters for each NUMA node based on global parameters. Claim 7 A machine learning parallelization method using a host CPU with a multi-socket structure, wherein, in claim 6, the step of updating the parameter is characterized by performing the parameter update in either a synchronous method or an asynchronous method in which the plurality of threads update the parameter. Claim 8 A machine learning parallelization device using a host CPU with a multi-socket structure, comprising: a processor that divides a learning model into pipeline stages at the layer level and assigns it to NUMA (Non-Uniform Memory Access) nodes per CPU socket, initializes parameters required for learning, and assigns multiple threads created considering policies for each parallelization algorithm to multiple cores included in the NUMA nodes for execution; and a memory that stores the parallelization algorithm, wherein the NUMA nodes per CPU socket are composed of a CPU including multiple cores and memory, the multiple cores share the memory through inter-core interconnects, and the NUMA nodes per CPU socket share the memory per NUMA node through inter-socket interconnects. Claim 9 delete Claim 10 A machine learning parallelization device using a host CPU with a multi-socket structure, characterized in that, in claim 8, the default value of the number of pipeline stages is set to correspond to the number of NUMA nodes, and the same number of model operations are distributed to each NUMA node. Claim 11 A machine learning parallelization device using a host CPU with a multi-socket structure, characterized in that, in claim 8, the parameters include a global parameter for data sharing among the plurality of threads and a local parameter used individually for each of the plurality of threads. Claim 12 A machine learning parallelization device using a host CPU with a multi-socket structure, characterized in that, in claim 11, the local parameter stores a derivative value (GRADIENT) for an error value used in the backpropagation process of the learning model and an optimizer state that determines whether to apply the derivative value. Claim 13 A machine learning parallelization device using a host CPU with a multi-socket structure, wherein, in claim 11, the processor synchronizes the execution of threads allocated to each NUMA node and updates parameters to each NUMA node based on global parameters. Claim 14 A machine learning parallelization device using a host CPU with a multi-socket structure, wherein, in claim 13, the processor performs parameter updates in either a synchronous update method or an asynchronous update method in which the plurality of threads update synchronously.

Images