A convolutional neural network layered training method and system based on containerization and virtualization

Abstract

The present invention discloses a containerized and virtualized convolutional neural network layered training method and system, belonging to the field of machine learning technology, including a segmented machine learning framework, an abstract modular template, POD, virtual machine creation scheduling, and a monitoring platform; the segmented machine learning framework is used to provide a machine learning program with communication capabilities and generate complete machine learning model code based on the template; the abstract modular template, based on the layered characteristics of the convolutional neural network structure, provides standard abstract templates for each module of the convolutional neural network, and uses a declarative definition method to define the structure of each layer of the convolutional neural network; POD, virtual machine creation scheduling includes machine learning code generation, POD, virtual machine creation, and POD, virtual machine scheduling. The present invention can fully utilize the performance of computing equipment, reduce the cost of model training for model designers, improve the reuse rate of cloud service providers' equipment, and simplify the design process of convolutional neural network models.

Description

Technical Field

[0001] The present invention relates to the field of machine learning technology, and in particular to a convolutional neural network layered training method and system based on containerization and virtualization. Background Art

[0002] With the advancement of computer hardware performance, more and more excellent convolutional neural network models have been proposed, starting with LeNet and later developing AlexNet, VGG, GoogLeNet, ResNet, and DenseNet. The number of layers in convolutional neural network models has generally increased, and the model structure has become increasingly complex. Generally speaking, with each additional layer in a convolutional neural network, the number of parameters in the entire network increases exponentially.

[0003] Model training relies on a certain amount of computer hardware resources. Whether a model can complete training depends primarily on whether the computer hardware running the model can handle the computing and storage resources required by the model at the time of peak utilization. Throughout the model training process, computer hardware performance is often maximized only when computing and storage resources are most needed; at other times, the computer hardware is idle. While ensuring that the model can train properly, model designers often face a trade-off between computer hardware resources and total training time. Even so, model designers still have to pay for the time during model training when computer hardware resources are not fully utilized, which undoubtedly increases the cost of model training. Summary of the Invention

[0004] The technical task of the present invention is to address the above shortcomings and provide a convolutional neural network hierarchical training method and system based on containerization and virtualization, which can give full play to the performance of computing devices, reduce the cost of model designers to train models, improve the reuse rate of cloud service providers' equipment, and simplify the design process of convolutional neural network models.

[0005] The technical solution adopted by the present invention to solve its technical problem is:

[0006] A containerized and virtualized convolutional neural network layered training method, including a segmented machine learning framework, abstract modular templates, POD and virtual machine creation and scheduling, and a monitoring platform.

[0007] The segmented machine learning framework is used to provide machine learning programs with communication capabilities and generate complete machine learning model code based on templates;

[0008] Abstract modular templates, based on the layered nature of convolutional neural network structures, provide standard templates for abstract convolutional neural network modules. They use a declarative definition approach to define the structure of each layer of the convolutional neural network. Depending on the template type, the declaration needs to define the module type and related configurations, such as resource type (virtual machine or POD), computing resource scale (memory, CPU, GPU, bandwidth), and scheduling order.

[0009] POD and virtual machine creation and scheduling: Utilizing the differences in computing and storage resources required by various convolutional neural network modules, corresponding PODs or virtual machines are created through abstracted template information. This includes machine learning code generation, POD and virtual machine creation, and POD and virtual machine scheduling.

[0010] The monitoring platform includes monitoring of POD and virtual machine performance and monitoring of the model training process. The monitoring data of the monitoring platform for POD and virtual machine comes from the communication modules at each layer, and it aggregates the data into the data dashboard of the monitoring platform.

[0011] Based on the hierarchical relationship between the modules of the convolutional neural network, this method designs a segmented machine learning framework and a set of templates for abstracting the modules of the convolutional neural network. It proposes a method to create PODs or virtual machines with corresponding computing and storage resources based on the computing resources and storage resources required by the abstracted modules, and constructs a platform for creating, scheduling and monitoring PODs and virtual machines.

[0012] Preferably, the segmented machine learning framework encapsulates the codes of each machine learning module in different HTTP services; the segmented machine learning framework includes a code module library and a communication module,

[0013] The code module library mainly defines the code of each machine learning module, including convolutional layer, pooling layer, and fully connected layer modules;

[0014] The communication module mainly provides data transmission capabilities for each module of the model, including transmitting data to the message queue, pulling data from the message queue, and feeding back real-time data to the monitoring platform.

[0015] The segmented machine learning framework encapsulates each machine learning module in a modular form into a module with communication functions. Each module can be flexibly combined into a complete machine learning network and automatically generates a machine learning model suitable for single-machine deployment based on the imported template.

[0016] Encapsulate each machine learning module into a module with communication function, so that each module can run on different machines, achieving the effect of segmented training model.

[0017] Preferably, the abstract modular templates include convolutional layer templates, batch normalization templates, activation templates, pooling templates, discard templates, fully connected templates, data templates and scheduling templates, wherein the scheduling template is automatically generated based on other templates. The main usage of other templates is as follows:

[0018] The convolutional layer template is mainly used to configure the parameters of the convolutional layer of the convolutional neural network. The convolutional layer template mainly contains the following parameters:

[0019] in_channels: the number of channels of network input;

[0020] out_channels: the number of channels output by the network;

[0021] kernel_size: The size of the convolution kernel. If the kernel_size parameter in the convolution layer is an integer n, then the size of the convolution kernel is n*n.

[0022] stride: stride, represents the step length of movement during the convolution process, the default is 1; generally, the convolution kernel moves from left to right and from top to bottom on the input image. If the parameter is an integer, the default is the integer in both horizontal and vertical directions; if the parameter is stride=(2, 1), where 2 represents the height (h), the row stride is 2; 1 represents the width (w) and the stride is 1;

[0023] padding: padding, the default is all 0 padding;

[0024] Dilation: expansion. Generally, the calculation between the convolution kernel and the corresponding position of the input image is the same size. If the convolution kernel size is 3*3, then the area it acts on the input image each time is also 3*3, and dilation=0;

[0025] Groups: Grouping refers to grouping the input channels. If groups=1, the input is a group and the output is also a group. If groups=2, the input is divided into two groups and the corresponding output is also two groups. At the same time, it is necessary to ensure that in_channels and out_channels must be able to divide groups.

[0026] bias: bias parameter, this parameter is of bool type. When bias=True, it means that the parameter b learned in the backward feedback is applied;

[0027] padding_mode: padding mode, padding_mode='zeros' means zero padding;

[0028] The batch normalization template is mainly used to configure the parameters of the batch normalization layer of the convolutional neural network;

[0029] The activation template is mainly used to configure the parameters of the activation layer of the convolutional neural network. The activation layer template mainly contains the following parameters:

[0030] Type: activation function type. Common activation functions include sigmoid, tanh, relu, etc.

[0031] The pooling template is mainly used to configure the parameters of the pooling layer of the convolutional neural network. The pooling layer template mainly contains the following parameters:

[0032] Pooling_kernel_size: the size of the pooling window;

[0033] Pooling_Stride: The step size of the pooling window movement. The default value is the kernel_size of the adjacent convolutional layer.

[0034] Pooling_Padding: The number of layers of 0 added to each edge of the input;

[0035] Pooling_dilation: A parameter that controls the stride of elements in the window;

[0036] return_indices: If True, returns the index of the maximum output value;

[0037] ceil_mode: If True, the calculated output signal will be rounded up if it is too small, instead of the default rounding down operation;

[0038] The discard template is mainly used to configure the parameters of the discard layer of the convolutional neural network. The discard layer template mainly includes the following parameters:

[0039] Rate: the probability of abandonment;

[0040] The fully connected template is mainly used to configure the parameters of the fully connected layer of the convolutional neural network. The fully connected layer template mainly contains the following parameters:

[0041] Inputs: input data;

[0042] Units: the number of neural unit nodes in this layer;

[0043] Activation: activation function;

[0044] use_bias: Boolean type, whether to use bias;

[0045] kernel_initializer: initializer for the convolution kernel;

[0046] bias_initializer: initializer of the bias term, which is initialized to 0 by default;

[0047] kernel_regularizer: regularizer for convolution kernel, optional;

[0048] bias_regularizer: regularization of bias term, optional;

[0049] activity_regularizer: output regularization function;

[0050] trainable: Boolean type, indicating whether the parameters of this layer participate in training;

[0051] reuse: Boolean type, whether to reuse parameters;

[0052] The data template is mainly used to configure data preprocessing information. The data layer template mainly contains the following parameters:

[0053] input_src: data storage path;

[0054] output_src: model configuration save path;

[0055] rate: the ratio of training set to test set;

[0056] count: number of model iterations;

[0057] The parameters common to each template are as follows:

[0058] CPU: used to define the required CPU resources, in Millicore;

[0059] GPU: used to define the required GPU resources;

[0060] Memory: used to define the required memory resources, the unit is Mi;

[0061] name-step: used to define the name of the template and the execution order of the template;

[0062] create_type: used to define the type of POD or virtual machine to be created.

[0063] The number of parameters in a convolutional neural network largely determines the module's required storage resources, such as memory and video memory; the amount of computation required determines the module's required computing resources, such as CPUs and GPUs. Given the distinct hierarchical relationships between convolutional neural network modules and the significant differences in parameter and computational requirements, inconsistent hardware requirements across modules lead to significant waste of storage and computing resources, increasing the cost of model training. Abstracting convolutional neural network modules and forming templates can address the issue of underutilized computer hardware resources during model training. Based on this method of abstracting convolutional neural network modules and creating templates, a set of standard sample templates for abstracting convolutional neural network modules is provided. These templates use a declarative definition approach to configure the parameters of each module.

[0064] Preferably, the POD and virtual machine creation and scheduling,

[0065] Code generation is based on a piecewise machine learning framework, which parses the templates of each module of the model and matches the corresponding deep learning code from the piecewise machine learning framework.

[0066] The segmented machine learning framework automatically generates corresponding machine learning code snippets with communication capabilities by parsing the templates for each model module. The generated machine learning code for each module runs as a program on a pod or virtual machine. To facilitate later model deployment, this method also supports automatically generating machine learning models suitable for single-machine deployment based on imported templates.

[0067] The creation of PODs and virtual machines is based on the templates created in the template definition part and provides automatic expansion and contraction services;

[0068] The scheduling of PODs and virtual machines is implemented based on the scheduling policies and message queues in the scheduling templates.

[0069] Furthermore, the creation of the POD and virtual machine, whether to create a POD or a virtual machine for each layer of the convolutional neural network, can be specified by the model designer through the create_type field in the corresponding template; at the same time, the computing resources and storage resources required by the POD or virtual machine can be configured by setting the values of the CPU, GPU, and memory fields; if the corresponding field values are not configured, based on the computing amount and parameter amount of each layer, according to the pre-set rules, starting from the first layer, a POD or virtual machine with the computing amount and parameter amount that matches the layer is created layer by layer, so that the corresponding computing resources and storage resources can meet the needs of the current module;

[0070] The matching rules work as follows:

[0071] GPU theoretical peak value = number of GPU chips * GPU Boost frequency * number of cores * number of floating-point calculations that can be processed in a single clock cycle;

[0072] The CPU single-cycle double-precision floating-point computing capability = FMA*2*512 / 64, where FMA refers to the floating-point vector multiplication and addition unit, which can complete floating-point vector multiplication and addition operations in one clock cycle. The default value is 2;

[0073] CPU single-cycle single-precision floating-point computing capability = FMA number * 2 * 512 / 32;

[0074] Parameter values are generally of float type, occupying 4 bytes, and storage resources = 4*parameter amount*a, where a is the parameter adjustment coefficient;

[0075] Computing resources = computing amount * b, where b is the parameter adjustment coefficient;

[0076] Estimate the computing and storage resources required for each layer based on the number of parameters and computational complexity of each layer;

[0077] The main function of the created POD or virtual machine is to provide a reliable operating environment for the code programs of each module. If a POD is created, the program runs in the POD; if a virtual machine is created, the scheduling module is created through relevant operation and maintenance tools such as Ansible, SaltStack, etc., to copy the module code to the virtual machine and start the program as a process.

[0078] Furthermore, the scheduling order of PODs and virtual machines is implemented based on the scheduling policy and message queue in the scheduling template. The scheduling template is automatically generated by the creation scheduling module after all PODs and virtual machines required by the model are successfully created. It records the IP address of each POD and virtual machine, as well as their scheduling order and the number of model iterations. After the scheduling template is generated, the creation scheduling module will copy it to the configuration files of all PODs and virtual machines created by the model, so that each POD and virtual machine knows the data transmission flow.

[0079] The specific scheduling steps are as follows:

[0080] 1) According to the scheduling strategy in the scheduling template, first start the data module, load the corresponding data, and pre-process the data;

[0081] 2) After the data processing of this layer is completed, the information related to the scheduling strategy and the processed data are sent to the message queue;

[0082] 3) Other layers periodically poll the messages in the message queue, pull down the messages belonging to this layer, and process the data according to the data processing flow of this layer. After the data processing is completed, step 2) is repeated until the scheduling strategy is executed;

[0083] When model training is completed, the model configuration information is saved in the data template, in the path specified by the output_src field, to provide configuration parameters for subsequent model deployment;

[0084] When the computing resources or storage resources of a certain layer are insufficient or excessive during model training, the pods or virtual machines are automatically expanded or reduced according to the specified rules.

[0085] This method can realize flexible creation and scheduling combinations: POD creation and POD scheduling, virtual machine creation and virtual machine scheduling, and POD-virtual machine mixed creation and mixed scheduling.

[0086] According to the template abstracted from the convolutional neural network model, corresponding PODs or virtual machines are created, and these PODs and virtual machines are scheduled according to the scheduling order defined in the scheduling template. The communication between each POD and virtual machine is implemented based on the message queue.

[0087] Preferably, the monitoring platform's performance monitoring of PODs and virtual machines mainly includes monitoring of CPU, GPU, memory, and traffic;

[0088] Monitoring the model training status mainly refers to monitoring the model training process. Each layer's communication module sends relevant log information to the monitoring platform after processing each data. The monitoring platform will collect the log information of each layer, sort it by time and number of iterations, and dynamically display the model training status information to the model trainer. The log of each transmission of each layer will contain the following information:

[0089] Name: the template name of the current layer;

[0090] data_from: which layer the received data comes from;

[0091] receive_data: received data, optional:

[0092] create_time: the time when the data is received;

[0093] end_time: the time when data is transferred to the next layer;

[0094] to: send data to the next layer;

[0095] send_data: data sent to the next layer, optional;

[0096] message: other information, such as error messages.

[0097] The monitoring platform can monitor the performance of all created PODs and virtual machines and the training status of the model to ensure that the model can be trained according to the scheduling order defined in the scheduling template.

[0098] The present invention also claims a containerized and virtualized convolutional neural network layered training system, including a segmented machine learning framework, an abstract modular template, a POD, a virtual machine creation and scheduling module, and a monitoring platform.

[0099] The system implements the above-mentioned containerization and virtualization-based convolutional neural network layered training method.

[0100] The present invention also claims protection for a containerized and virtualized convolutional neural network layered training device, comprising: at least one memory and at least one processor;

[0101] The at least one memory is configured to store a machine-readable program;

[0102] The at least one processor is used to call the machine-readable program to execute the above-mentioned containerization and virtualization-based convolutional neural network layered training method.

[0103] The present invention also claims protection for a computer-readable medium having computer instructions stored thereon, which, when executed by a processor, causes the processor to perform the above-mentioned containerization and virtualization-based convolutional neural network layer-by-layer training.

[0104] Compared with the prior art, the containerized and virtualized convolutional neural network layered training method and system of the present invention have the following beneficial effects:

[0105] Compared with traditional convolutional neural network model training methods, this method has significant benefits for both model designers and cloud platform service providers:

[0106] For model designers, this method does not require writing relevant model code. Instead, they only need to fill in the corresponding parameters in the declarative template according to the template type, which greatly simplifies the model design and development process. Using this method to train the model can maximize the cost savings of model training.

[0107] For cloud service providers, this method can be used to provide users with a combination service of creating corresponding PODs and virtual machines, which can improve the efficiency of cloud service provider equipment usage. BRIEF DESCRIPTION OF THE DRAWINGS

[0108] Figure 1This is a schematic diagram of the architecture of a convolutional neural network layered training method based on containerization and virtualization provided by an embodiment of the present invention;

[0109] Figure 2 Schematic diagram of the working process of the segmented machine learning framework provided by an embodiment of the present invention;

[0110] Figure 3 It is a schematic diagram of parameter quantities and calculation quantities provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0111] The present invention will be further described below with reference to the accompanying drawings and specific embodiments.

[0112] The embodiment of the present invention provides a convolutional neural network layered training method based on containerization and virtualization, including a segmented machine learning framework, abstract modular templates, POD, virtual machine creation and scheduling, and a monitoring platform;

[0113] The segmented machine learning framework is used to provide machine learning programs with communication capabilities and generate complete machine learning model code based on templates. It encapsulates each machine learning module in a modular form into a module with communication capabilities. Each module can be flexibly combined into a complete machine learning network, and automatically generates a machine learning model suitable for single-machine deployment based on the imported template.

[0114] Abstract modular templates, based on the layered nature of convolutional neural network structures, provide standard templates for abstract convolutional neural network modules. They use a declarative definition approach to define the structure of each layer of the convolutional neural network. Depending on the template type, the declaration needs to define the module type and related configurations, such as resource type (virtual machine or POD), computing resource scale (memory, CPU, GPU, bandwidth), and scheduling order.

[0115] POD and virtual machine creation and scheduling: Utilizing the differences in computing and storage resources required by various convolutional neural network modules, corresponding PODs or virtual machines are created through abstracted template information. This includes machine learning code generation, POD and virtual machine creation, and POD and virtual machine scheduling.

[0116] The monitoring platform includes monitoring of POD and virtual machine performance and monitoring of the model training process. The monitoring data of the monitoring platform for POD and virtual machine comes from the communication modules at each layer, and it aggregates the data into the data dashboard of the monitoring platform.

[0117] Based on the hierarchical relationship between the modules of the convolutional neural network, this method designs a segmented machine learning framework and a set of templates for abstracting the modules of the convolutional neural network. It proposes a method to create PODs or virtual machines with corresponding computing and storage resources based on the computing resources and storage resources required by the abstracted modules, and constructs a platform for creating, scheduling and monitoring PODs and virtual machines.

[0118] The segmented machine learning framework encapsulates each machine learning module into a module with communication capabilities, allowing each module to run on different machines, achieving the effect of segmented training models.

[0119] Because each module requires different computing and storage resources, this method provides a flexible mechanism for creating the optimal pods and virtual machines. Service developers can customize computing and storage resources, and whether to start a pod or virtual machine; or the system can create a pod or virtual machine with a certain amount of computing and storage resources based on the characteristics of the template.

[0120] This method has flexible creation and scheduling combinations: POD creation and POD scheduling; virtual machine creation and virtual machine scheduling; POD and virtual machine mixed creation and mixed scheduling.

[0121] According to the template abstracted from the convolutional neural network model, corresponding PODs or virtual machines are created, and these PODs and virtual machines are scheduled according to the scheduling order defined in the scheduling template. The communication between each POD and virtual machine is implemented based on the message queue.

[0122] The monitoring platform can monitor the performance of all created PODs and virtual machines and the training status of the model to ensure that the model can be trained according to the scheduling order defined in the scheduling template.

[0123] The specific technical solution is as follows:

[0124] 1. Segmented Machine Learning Framework

[0125] The segmented machine learning framework includes a code module library and a communication module. Its main function is to provide a machine learning program with communication capabilities and generate complete machine learning model code based on templates. Its essence is to encapsulate the code of each machine learning module in different HTTP services.

[0126] Among them, the code module library mainly defines the codes of various machine learning modules (such as convolutional layer, pooling layer, fully connected layer, etc.).

[0127] The communication module mainly provides data transmission capabilities for each module of the model, such as transmitting data to the message queue, pulling data from the message queue, and feeding back real-time data to the monitoring platform.

[0128] The working diagram of the segmented machine learning framework is as follows Figure 2 shown.

[0129] 2. Abstract modular template definition

[0130] This method aims to solve the problem that the hierarchical relationship between modules of convolutional neural networks is obvious and the parameters and computational complexity between modules vary greatly. For a comparison of the parameters and computational complexity of each layer of a common convolutional neural network, see Figure 3 As shown in Figure 1, the number of parameters largely determines the storage resources required by the module, such as memory and video memory; the amount of computation determines the computing resources required, such as CPU and GPU. The inconsistent hardware requirements of different convolutional neural network modules result in a large amount of wasted storage and computing resources, increasing the cost of model training. This method, which abstracts the various convolutional neural network modules and forms templates, can address the problem of underutilized computer hardware resources during model training.

[0131] Based on the method of abstracting each module of a convolutional neural network and making it into a template, this method provides a set of standard sample templates for each module of the abstract convolutional neural network. The template uses a declarative definition method to configure the parameters of each module. The template mainly includes:

[0132] Convolutional layer template (Convolutional.yaml), batch normalization template (Bn.yaml), activation template (Activation.yaml), pooling template (Pooling.yaml), dropout template (Dropout.yaml), fully connected template (Fc.yaml), data template (Data.yaml), scheduling template (Scheduling.yaml), etc. The scheduling template is automatically generated based on other templates. The main usage of other templates is as follows:

[0133] The convolutional layer template is mainly used to configure the parameters of the convolutional layer of the convolutional neural network. The convolutional layer template mainly contains the following parameters:

[0134] in_channels: the number of channels of network input;

[0135] out_channels: the number of channels output by the network;

[0136] kernel_size: the size of the convolution kernel. If this parameter is an integer n, the size of the convolution kernel is n*n.

[0137] stride: stride, represents the step length of movement during the convolution process, the default is 1; generally the convolution kernel moves from left to right and from top to bottom on the input image. If the parameter is an integer, the default is the integer in both horizontal and vertical directions. If the parameter is stride=(2, 1), 2 represents the height (h) and the row stride is 2; 1 represents the width (w) and the stride is 1;

[0138] padding: padding, the default is all 0 padding;

[0139] Dilation: expansion. Generally, the convolution kernel and the corresponding position of the input image are calculated to be the same size. If the convolution kernel size is 3*3, then the area it acts on the input image each time is also 3*3, and dilation = 0;

[0140] groups: Grouping refers to grouping the input channels. If groups=1, the input is grouped as one, and the output is also grouped as one. If groups=2, the input is divided into two groups, and the corresponding output is also grouped as two. At the same time, it is necessary to ensure that in_channels and out_channels must be able to divide groups;

[0141] bias: bias parameter, this parameter is of bool type. When bias=True, it means that the parameter b learned in the backward feedback is applied;

[0142] padding_mode: padding mode, padding_mode='zeros' means zero padding.

[0143] The batch normalization template is mainly used to configure the parameters of the batch normalization layer of the convolutional neural network. The batch normalization layer template mainly contains the following parameters:

[0144] name-step: defines the name of the template and the order in which the template executes its steps.

[0145] The activation template is mainly used to configure the parameters of the activation layer of the convolutional neural network. The activation layer template mainly contains the following parameters:

[0146] Type: activation function type. Common activation functions include sigmoid, tanh, relu, etc.

[0147] The pooling template is mainly used to configure the parameters of the pooling layer of the convolutional neural network. The pooling layer template mainly contains the following parameters:

[0148] kernel_size: the size of the pooling window;

[0149] Stride: The step size of the pooling window movement, the default value is kernel_size;

[0150] Padding: The number of layers of 0 added to each edge of the input;

[0151] dilation: a parameter that controls the stride of elements in the window;

[0152] return_indices: If True, returns the index of the maximum output value;

[0153] ceil_mode : If True, the calculated output signal will be rounded up when it is too small, instead of the default rounding down operation.

[0154] The discard template is mainly used to configure the parameters of the discard layer of the convolutional neural network. The discard layer template mainly includes the following parameters:

[0155] Rate: The probability of discarding.

[0156] The fully connected template is mainly used to configure the parameters of the fully connected layer of the convolutional neural network. The fully connected layer template mainly contains the following parameters:

[0157] Inputs: input data;

[0158] Units: the number of neural unit nodes in this layer;

[0159] Activation: activation function;

[0160] use_bias: Boolean type, whether to use bias;

[0161] kernel_initializer: initializer for the convolution kernel;

[0162] bias_initializer: initializer of the bias term, which is initialized to 0 by default;

[0163] kernel_regularizer: regularizer for convolution kernel, optional;

[0164] bias_regularizer: regularization of bias term, optional;

[0165] activity_regularizer: output regularization function;

[0166] trainable: Boolean type, indicating whether the parameters of this layer participate in training;

[0167] reuse: Boolean type, whether to reuse parameters.

[0168] The data template is mainly used to configure data preprocessing information. The data layer template mainly contains the following parameters:

[0169] input_src: data storage path;

[0170] output_src: model configuration save path;

[0171] rate: the ratio of training set to test set;

[0172] count: number of model iterations.

[0173] The common parameters of each template are as follows:

[0174] CPU: used to define the required CPU resources, unit is m;

[0175] GPU: used to define the required GPU resources;

[0176] Memory: used to define the required memory resources, the unit is mi;

[0177] name-step: used to define the name of the template and the execution order of the template;

[0178] create_type: used to define the creation of POD or virtual machine.

[0179] 3. POD, virtual machine creation and scheduling

[0180] It includes code generation, POD, virtual machine creation and POD, virtual machine scheduling. The code generation is based on the segmented machine learning framework. It matches the corresponding deep learning code from the segmented machine learning framework by parsing the templates of each module of the model; the creation of POD and virtual machine is based on the template created in the template definition part and provides automatic scaling services; the scheduling of POD and virtual machine is based on the scheduling strategy and message queue in the scheduling template.

[0181] Code Generation:

[0182] The segmented machine learning framework automatically generates corresponding machine learning code snippets with communication capabilities by parsing the templates for each model module. The generated machine learning code for each module runs as a program on a pod or virtual machine. To facilitate later model deployment, this method also supports automatically generating machine learning models suitable for single-machine deployment based on imported templates.

[0183] POD and virtual machine creation:

[0184] This method provides a flexible mechanism for creating PODs and virtual machines. Whether to create a POD or a virtual machine for each layer of the convolutional neural network can be specified by the model designer through the create_type field in the corresponding template; at the same time, the computing resources and storage resources required by the POD or virtual machine can be configured by setting the values of fields such as CPU, GPU, and memory. If the corresponding field values are not configured, this method will create a POD or virtual machine with the computing and parameter quantities of each layer, starting from the first layer, according to pre-set rules, so that the corresponding computing resources and storage resources can meet the needs of the current module.

[0185] The matching rules work as follows:

[0186] GPU theoretical peak value = number of GPU chips * GPU Boost frequency * number of cores * number of floating-point calculations that can be processed in a single clock cycle;

[0187] CPU single-cycle double-precision floating-point computing capability = FMA*2*512 / 64, where FMA refers to the floating-point vector addition and multiplication fusion unit, and the default value is 2;

[0188] CPU single-cycle double-precision floating-point computing capability = FMA number * 2 * 512 / 32;

[0189] Parameter values are generally of float type, occupying 4 bytes, and storage resources = 4*parameter amount*a, where a is the parameter adjustment coefficient;

[0190] Computing resources = computing amount * b, where b is the parameter adjustment coefficient;

[0191] Estimate the computing and storage resources required for each layer based on the number of parameters and computational complexity of each layer.

[0192] The POD or virtual machine created by this method mainly provides a reliable operating environment for the code programs of each module. If a POD is created, the program runs in the POD; if a virtual machine is created, the scheduling module is created through relevant operation and maintenance tools such as Ansible, SaltStack, etc., the module code is copied to the virtual machine, and the program is started as a process.

[0193] POD, virtual machine scheduling:

[0194] The scheduling order of the Pods and VMs created by this method is based on the scheduling policy and message queue in the scheduling template. The scheduling template is automatically generated by the creation scheduling module after all the Pods and VMs required by the model have been successfully created. It records information such as the IP address accessing each Pod and VM, their scheduling order, and the number of model iterations. Once generated, the scheduling template is copied by the creation scheduling module to the configuration files of all Pods and VMs created by the model, ensuring that each Pod and VM knows the data transmission flow.

[0195] The specific scheduling steps are as follows:

[0196] 1) According to the scheduling strategy in the scheduling template, first start the data module, load the corresponding data, and pre-process the data;

[0197] 2) After the data processing of this layer is completed, the information related to the scheduling strategy and the processed data are sent to the message queue;

[0198] 3) Other layers regularly poll the messages in the message queue, pull down the messages belonging to this layer, and process the data according to the data processing flow of this layer. After the data processing is completed, repeat step 2) until the scheduling strategy is executed.

[0199] When model training is completed, the model configuration information is saved in the data template in the path specified by the output_src field to provide configuration parameters for subsequent model deployment.

[0200] When the computing resources or storage resources of a certain layer are insufficient or excessive during the training process of the model, the present invention will automatically expand or shrink the pod or virtual machine according to certain rules.

[0201] 4. Monitoring Platform

[0202] The monitoring platform includes monitoring of POD and virtual machine performance and monitoring of model training status. The monitoring data of POD and virtual machine on the monitoring platform comes from the communication modules of each layer respectively. The communication modules of each layer aggregate the data into the data dashboard of the monitoring platform.

[0203] Performance monitoring of PODs and virtual machines mainly includes monitoring of CPU, GPU, memory, traffic, etc.

[0204] Monitoring the model training status mainly refers to monitoring the model training process. Each layer's communication module sends relevant log information to the monitoring platform after processing each data. The monitoring platform will collect the log information of each layer, sort it by time and number of iterations, and dynamically display the model training status information to the model trainer. The log of each layer's transmission will contain the following information:

[0205] Name: the template name of the current layer;

[0206] data_from: the layer from which the received data comes;

[0207] receive_data: received data, optional;

[0208] create_time: the time when the data is received;

[0209] end_time: the time when data is transferred to the next layer;

[0210] to: send data to the next layer;

[0211] send_data: data sent to the next layer, optional;

[0212] message: other information, such as error messages.

[0213] The embodiment of the present invention also provides a convolutional neural network layered training system based on containerization and virtualization, including a segmented machine learning framework, an abstract modular template, a POD, a virtual machine creation and scheduling module, and a monitoring platform.

[0214] The system implements the containerization and virtualization-based convolutional neural network layered training method described in the above embodiments of the present invention.

[0215] An embodiment of the present invention further provides a containerized and virtualized convolutional neural network layered training device, comprising: at least one memory and at least one processor;

[0216] The at least one memory is configured to store a machine-readable program;

[0217] The at least one processor is used to call the machine-readable program to execute the containerization and virtualization-based convolutional neural network layered training method described in the above embodiment of the present invention.

[0218] An embodiment of the present invention further provides a computer-readable medium having computer instructions stored thereon. When executed by a processor, the computer instructions cause the processor to execute the method for implementing distributed automatic alarm processing on a cloud computing platform as described in the above-mentioned embodiments of the present invention. Specifically, a system or device equipped with a storage medium can be provided. The storage medium stores software program code that implements the functions of any of the above-mentioned embodiments, and the computer (or CPU or MPU) of the system or device can be configured to read and execute the program code stored in the storage medium.

[0219] In this case, the program code itself read from the storage medium can realize the function of any one of the above-mentioned embodiments, and thus the program code and the storage medium storing the program code constitute part of the present invention.

[0220] Examples of storage media for providing program code include floppy disks, hard disks, magneto-optical disks, optical disks (e.g., CD-ROMs, CD-Rs, CD-RWs, DVD-ROMs, DVD-RAMs, DVD-RWs, and DVD+RWs), magnetic tapes, non-volatile memory cards, and ROMs. Alternatively, the program code may be downloaded from a server computer via a communications network.

[0221] In addition, it should be clear that the functions of any of the above embodiments can be achieved not only by executing the program code read by the computer, but also by enabling the operating system operating on the computer to complete part or all of the actual operations based on the instructions of the program code.

[0222] In addition, it can be understood that the program code read out from the storage medium is written into the memory provided in the expansion board inserted into the computer or into the memory provided in the expansion unit connected to the computer, and then based on the instructions of the program code, the CPU installed on the expansion board or expansion unit is enabled to perform part or all of the actual operations, thereby realizing the functions of any of the above embodiments.

[0223] The present invention has been shown and described in detail above through the accompanying drawings and preferred embodiments. However, the present invention is not limited to these disclosed embodiments. Based on the above multiple embodiments, those skilled in the art can know that the code review methods in the above different embodiments can be combined to obtain more embodiments of the present invention, and these embodiments are also within the scope of protection of the present invention.

Claims

1. A convolutional neural network layer training method based on containerization and virtualization, characterized in that: Includes a segmented machine learning framework, abstract modular templates, POD, virtual machine creation and scheduling, and a monitoring platform; The segmented machine learning framework is used to provide machine learning programs with communication capabilities and generate complete machine learning model code based on templates; Abstract modular templates, based on the layered characteristics of convolutional neural network structures, provide standard templates for abstract convolutional neural network modules. They use a declarative definition approach to define the structure of each layer of the convolutional neural network. Depending on the template type, the module type and related configurations are defined in the declaration, including resource type, computing resource scale, and scheduling order. POD and VM creation and scheduling, including machine learning code generation, POD and VM creation, and POD and VM scheduling; utilizing the differences in computing and storage resources required by convolutional neural network modules, and creating corresponding PODs or VMs through abstracted template information; The monitoring platform includes monitoring of POD and virtual machine performance and monitoring of the model training process. The monitoring data of the monitoring platform for POD and virtual machine comes from the communication modules at each layer, and it aggregates the data into the data dashboard of the monitoring platform.

2. The method for layered training of a convolutional neural network based on containerization and virtualization according to claim 1, characterized in that: The segmented machine learning framework encapsulates the code of each machine learning module in different HTTP services; the segmented machine learning framework includes a code module library and a communication module. The code module library mainly defines the code of each machine learning module, including convolutional layer, pooling layer, and fully connected layer modules; The communication module mainly provides data transmission capabilities for each module of the model, including transmitting data to the message queue, pulling data from the message queue, and feeding back real-time data to the monitoring platform.

3. The method for layered training of a convolutional neural network based on containerization and virtualization according to claim 1, characterized in that: The abstract modular templates include convolutional layer templates, batch normalization templates, activation templates, pooling templates, discard templates, fully connected templates, data templates, and scheduling templates. The scheduling template is automatically generated based on other templates. The main usage of other templates is as follows: The convolutional layer template is mainly used to configure the parameters of the convolutional layer of the convolutional neural network. The convolutional layer template mainly contains the following parameters: in_channels: the number of channels of network input; out_channels: the number of channels output by the network; kernel_size: The size of the convolution kernel. If the kernel_size parameter in the convolution layer is an integer n, then the size of the convolution kernel is n*n. stride: step size, representing the step size of movement during the convolution process, the default is 1; padding: padding, the default is all 0 padding; dilation: Expansion: If the convolution kernel size is 3*3, then the area it acts on the input image each time is also 3*3, and dilation=0. Groups: Grouping refers to grouping the input channels. If groups=1, the input is a group and the output is also a group. If groups=2, the input is divided into two groups and the corresponding output is also two groups. At the same time, it is necessary to ensure that in_channels and out_channels must be able to divide groups. bias: bias parameter, this parameter is of bool type. When bias=True, it means that the parameter b learned in the backward feedback is applied; padding_mode: padding mode, padding_mode='zeros' means zero padding; The batch normalization template is mainly used to configure the parameters of the batch normalization layer of the convolutional neural network. The activation template is mainly used to configure the parameters of the activation layer of the convolutional neural network. The activation layer template mainly contains the following parameters: Type: activation function type. Common activation functions include sigmoid, tanh, and relu. The pooling template is mainly used to configure the parameters of the pooling layer of the convolutional neural network. The pooling layer template mainly contains the following parameters: Pooling_kernel_size: the size of the pooling window; Pooling_Stride: The step size of the pooling window movement. The default value is the kernel_size of the adjacent convolutional layer. Pooling_Padding: The number of layers of 0 added to each edge of the input; Pooling_dilation: A parameter that controls the stride of elements in the window; return_indices: If True, returns the index of the maximum output value; ceil_mode: If True, the calculated output signal will be rounded up if it is too small, instead of the default rounding down operation; The discard template is mainly used to configure the parameters of the discard layer of the convolutional neural network. The discard layer template mainly includes the following parameters: Rate: the probability of abandonment; The fully connected template is mainly used to configure the parameters of the fully connected layer of the convolutional neural network. The fully connected layer template mainly contains the following parameters: Inputs: input data; Units: the number of neural unit nodes in this layer; Activation: activation function; use_bias: Boolean type, whether to use bias; kernel_initializer: initializer for the convolution kernel; bias_initializer: initializer of the bias term, which is initialized to 0 by default; kernel_regularizer: regularizer for convolution kernel, optional; bias_regularizer: regularization of bias term, optional; activity_regularizer: output regularization function; trainable: Boolean type, indicating whether the parameters of this layer participate in training; reuse: Boolean type, whether to reuse parameters; The data template is mainly used to configure data preprocessing information. The data layer template mainly contains the following parameters: input_src: data storage path; output_src: model configuration save path; rate_1: the ratio of training set to test set; count: number of model iterations; The common parameters of each template are as follows: CPU: used to define the required CPU resources, in Millicore; GPU: used to define the required GPU resources; Memory: used to define the required memory resources, the unit is Mi; name-step: used to define the name of the template and the execution order of the template; create_type: used to define the type of POD or virtual machine to be created.

4. A containerized and virtualized convolutional neural network layered training method according to claim 1 or 2, characterized in that: The POD, virtual machine creation and scheduling, Code generation is based on a piecewise machine learning framework, which parses the templates of each module of the model and matches the corresponding deep learning code from the piecewise machine learning framework. The creation of PODs and virtual machines is based on the templates created in the template definition part and provides automatic expansion and contraction services; The scheduling of PODs and virtual machines is implemented based on the scheduling policies and message queues in the scheduling templates.

5. The method for layered training of a convolutional neural network based on containerization and virtualization according to claim 4, characterized in that: The creation of the POD and virtual machine, whether to create a POD or a virtual machine for each layer of the convolutional neural network, can be specified by the model designer through the create_type field in the corresponding template; at the same time, the computing resources and storage resources required by the POD or virtual machine can be configured by setting the values of the CPU, GPU, and memory fields; if the corresponding field values are not configured, based on the computing amount and parameter amount of each layer, according to the pre-set rules, starting from the first layer, a POD or virtual machine with the computing amount and parameter amount that matches the layer is created layer by layer, so that the corresponding computing resources and storage resources can meet the needs of the current module; The matching rules work as follows: GPU theoretical peak value = number of GPU chips * GPU Boost frequency * number of cores * number of floating-point calculations that can be processed in a single clock cycle; The CPU single-cycle double-precision floating-point computing capability = FMA*2*512 / 64, where FMA refers to the floating-point vector multiplication and addition unit, which can complete floating-point vector multiplication and addition operations in one clock cycle. The default value is 2; CPU single-cycle single-precision floating-point computing capability = FMA number * 2 * 512 / 32; Storage resources = 4 * parameter amount * a, where a is the parameter adjustment coefficient; Computing resources = computing amount * b, where b is the parameter adjustment coefficient; Estimate the computing and storage resources required for each layer based on the number of parameters and computational complexity of each layer; If a POD is created, the program runs in the POD; if a virtual machine is created, the creation and scheduling module copies the module code to the virtual machine through relevant operation and maintenance tools, and starts the program as a process.

6. A containerized and virtualized convolutional neural network layered training method according to claim 4, characterized in that The scheduling order of PODs and virtual machines is implemented based on the scheduling policy and message queue in the scheduling template. The scheduling template is automatically generated by the creation and scheduling module after all PODs and virtual machines required by the model are successfully created. It records the IP address of each POD and virtual machine, as well as their scheduling order and model iteration number information. After the scheduling template is generated, the creation and scheduling module will copy it to the configuration files of all PODs and virtual machines created by the model, so that each POD and virtual machine knows the data transmission flow. The specific scheduling steps are as follows: 1) According to the scheduling strategy in the scheduling template, first start the data module, load the corresponding data, and pre-process the data; 2) After the data processing of this layer is completed, the information related to the scheduling strategy and the processed data are sent to the message queue; 3) Other layers periodically poll the messages in the message queue, pull down the messages belonging to this layer, and process the data according to the data processing flow of this layer. After the data processing is completed, step 2) is repeated until the scheduling strategy is executed; When model training is completed, the model configuration information is saved in the data template, in the path specified by the output_src field, to provide configuration parameters for subsequent model deployment; When the computing resources or storage resources of a certain layer are insufficient or excessive during model training, the pods or virtual machines are automatically expanded or reduced according to the specified rules.

7. The method for layered training of a convolutional neural network based on containerization and virtualization according to claim 4, characterized in that: The monitoring platform's performance monitoring of PODs and virtual machines mainly includes monitoring of CPU, GPU, memory, and traffic; Monitoring the model training status mainly refers to monitoring the model training process. Each layer's communication module sends relevant log information to the monitoring platform after processing each data. The monitoring platform will collect the log information of each layer, sort it by time and number of iterations, and dynamically display the model training status information to the model trainer. The log of each transmission of each layer will contain the following information: Name: the template name of the current layer; data_from: which layer the received data comes from; receive_data: received data, optional: create_time: the time when the data is received; end_time: the time when data is transferred to the next layer; to: send data to the next layer; send_data: data sent to the next layer, optional; message: Other information.

8. A convolutional neural network layer training system based on containerization and virtualization, characterized in that: Including segmented machine learning framework, abstract modular template, POD, virtual machine creation and scheduling module, monitoring platform, The system implements the containerization and virtualization-based convolutional neural network layered training method described in any one of claims 1 to 7.

9. A convolutional neural network layer training device based on containerization and virtualization, characterized in that: include: at least one memory and at least one processor; The at least one memory is configured to store a machine-readable program; The at least one processor is used to call the machine-readable program to execute the containerization and virtualization-based convolutional neural network layered training method described in any one of claims 1 to 7.

10. A computer-readable medium, characterized in that The computer-readable medium stores computer instructions, which, when executed by a processor, cause the processor to perform the containerization and virtualization-based convolutional neural network layer-by-layer training described in any one of claims 1 to 7.

Images