Resource configuration method, model training method and device

Abstract

The embodiment of the present application provides a resource configuration method, a model training method and a device. The resource configuration method includes: obtaining model parameters of a neural network model and a target number of graphics processors; obtaining a target graphics processor that matches the model parameters and the target number; determining a parameter update mode between the target graphics processors according to the configuration information corresponding to the target graphics processor; and determining resource configuration information of the neural network model based on the target graphics processor and the parameter update mode. The embodiment of the present application can improve the training and reasoning efficiency of deep learning models.

Description

Technical Field

[0001] The present application relates to the technical field of resource configuration, and in particular to a resource configuration method, a model training method and a device. Background Art

[0002] GPU (Graphics Processing Unit) has powerful matrix calculation and parallel performance, and is widely used in deep learning fields such as computer vision, natural language processing and content recommendation.

[0003] With the rapid development of deep learning technology, the network structure of deep learning models has become more and more complex. With the substantial increase in parameter scale, its training and reasoning need to be carried out using large-scale distributed GPU clusters. In the process of training deep learning models, a large amount of data needs to be transferred between GPUs. However, the existing technical solutions do not fully consider the heterogeneous communication transmission between GPUs, resulting in relatively low efficiency of model training or reasoning. Summary of the invention

[0004] The purpose of the embodiments of the present application is to provide a resource configuration method, a model training method and a device to implement parameter update modes for different deep learning models and communication network configurations, thereby improving the training and reasoning efficiency of deep models. The specific technical solutions are as follows:

[0005] In a first aspect of the implementation of the present application, a resource configuration method is first provided, comprising:

[0006] Obtain model parameters of the neural network model and the target number corresponding to the graphics processor;

[0007] Acquire a target graphics processor that matches the model parameters and the target quantity;

[0008] Determining a parameter update mode between the target graphics processors according to the configuration information corresponding to the target graphics processor;

[0009] Based on the target graphics processor and the parameter update mode, resource configuration information of the neural network model is determined.

[0010] Optionally, acquiring a target graphics processor that matches the model parameters and the target quantity includes:

[0011] Acquire a first number of initial graphics processors matching the model parameters; the first number is greater than or equal to the target number;

[0012] The target number of target graphics processors is selected from the initial graphics processors according to the resource data of the initial graphics processors.

[0013] Optionally, the step of selecting the target number of target graphics processors from the initial graphics processors according to the resource data of the initial graphics processors comprises:

[0014] According to the resource data, selecting a second number of graphics processors whose available resource data is greater than a set data threshold from the initial graphics processors;

[0015] When the second number is greater than the target number, the target number of target graphics processors is screened out from the second number of graphics processors in descending order of the usage frequency.

[0016] Optionally, determining the parameter update mode between the target graphics processors according to the configuration information corresponding to the target graphics processor includes:

[0017] Obtaining the node type of the target graphics processor and the node bandwidth between the target graphics processors;

[0018] A parameter update mode between the target graphics processors is determined according to the node type, the node bandwidth, and the target quantity.

[0019] In a second aspect of the implementation of the present application, a model training method is provided, comprising:

[0020] Obtain resource configuration information of the neural network model to be trained;

[0021] Determining a target graphics processor and a parameter update mode corresponding to the neural network model according to the resource configuration information;

[0022] Obtain training sample data;

[0023] In the process of training the neural network model, transmitting the training sample data based on the target graphics processor;

[0024] After the loss function of the neural network model is calculated, the model parameters of the neural network model are updated according to the loss function and the parameter update mode until the target neural network model is trained.

[0025] Optionally, the resource configuration information is multiple pieces of information.

[0026] After updating the model parameters of the neural network model according to the loss function and the parameter updating mode until the target neural network model is obtained through training, the method further includes:

[0027] Obtaining a score value of the resource configuration information output by the neural network model;

[0028] According to the score value, the resource configuration information with the largest score value is screened out from the resource configuration information, and the resource configuration information with the largest score value is used as the resource configuration information of the target type of network model;

[0029] The target type of network model refers to a model type that is the same as the type of the neural network model.

[0030] In a third aspect of the implementation of the present application, a resource configuration device is provided, including:

[0031] A target quantity acquisition module is used to obtain model parameters of the neural network model and the target quantity corresponding to the graphics processor;

[0032] A target processor acquisition module, used to acquire a target graphics processor that matches the model parameters and the target quantity;

[0033] A parameter update mode determination module, configured to determine a parameter update mode between the target graphics processors according to configuration information corresponding to the target graphics processors;

[0034] A resource configuration information determination module is used to determine the resource configuration information of the neural network model based on the target graphics processor and the parameter update mode.

[0035] Optionally, the target processor acquisition module includes:

[0036] An initial processor acquisition unit, configured to acquire a first number of initial graphics processors matching the model parameters; the first number being greater than or equal to the target number;

[0037] The target processor screening unit is used to screen the target number of target graphics processors from the initial graphics processors according to the resource data of the initial graphics processors.

[0038] Optionally, the target processor screening unit includes:

[0039] A graphics processor screening subunit, configured to screen out a second number of graphics processors having available resource data greater than a set data threshold from the initial graphics processors according to the resource data;

[0040] The target processor screening subunit is used to screen out a target number of target graphics processors from the second number of graphics processors in descending order of usage frequency when the second number is greater than the target number.

[0041] Optionally, the parameter update mode determination module includes:

[0042] A node type acquisition unit, used to acquire the node type of the target graphics processor and the node bandwidth between the target graphics processors;

[0043] The parameter update mode determination unit is used to determine the parameter update mode between the target graphics processors according to the node type, the node bandwidth and the target number.

[0044] In a fourth aspect of the implementation of the present application, a model training device is provided, comprising:

[0045] A resource configuration information acquisition module, used to obtain resource configuration information of a neural network model to be trained;

[0046] A parameter update mode determination module, used to determine a target graphics processor and a parameter update mode corresponding to the neural network model according to the resource configuration information;

[0047] A training sample data acquisition module is used to acquire training sample data;

[0048] A training sample data transmission module, used for transmitting the training sample data based on the target graphics processor during the training of the neural network model;

[0049] The target network model acquisition module is used to update the model parameters of the neural network model according to the loss function and the parameter update mode after calculating the loss function of the neural network model, until the target neural network model is trained.

[0050] Optionally, the resource configuration information is a plurality of pieces of information, and the device further includes:

[0051] A scoring value acquisition module, used to obtain the scoring value of the resource configuration information output by the neural network model;

[0052] A resource configuration information screening module, used to screen out the resource configuration information with the largest score value from the resource configuration information according to the score value, and use the resource configuration information with the largest score value as the resource configuration information of the target type of network model;

[0053] The target type of network model refers to a model type that is the same as the type of the neural network model.

[0054] In another aspect of the implementation of the present application, a computer-readable storage medium is provided, in which instructions are stored. When the computer-readable storage medium is run on a computer, the computer executes any of the resource configuration methods described above, or any of the model training methods described above.

[0055] In another aspect of the implementation of the present application, a computer program product comprising instructions is also provided, which, when executed on a computer, enables the computer to execute any of the resource configuration methods described above, or any of the model training methods described above.

[0056] The resource configuration method, model training method and device provided in the embodiments of the present application obtain the model parameters of the neural network model and the target number corresponding to the graphics processor, obtain the target graphics processor that matches the model parameters and the target number, determine the parameter update mode between the target graphics processors according to the configuration information corresponding to the target graphics processor, and determine the resource configuration information of the neural network model based on the target graphics processor and the parameter update mode. The embodiments of the present application can select the number of target graphics processors for model training in combination with the model parameters, determine the parameter update mode according to the configuration information between the target graphics processors, and determine the resource configuration information of the neural network model according to the number of target graphics processors and the parameter update mode to perform model training or model reasoning, and can perform resource configuration for different deep learning models, thereby improving the training or reasoning efficiency of the deep learning model. BRIEF DESCRIPTION OF THE DRAWINGS

[0057] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art are briefly introduced below.

[0058] Figure 1 A flowchart of a resource configuration method provided in an embodiment of the present application;

[0059] Figure 2 A flowchart of the steps of a model training method provided in an embodiment of the present application;

[0060] Figure 3 A schematic diagram of the structure of a resource configuration device provided in an embodiment of the present application;

[0061] Figure 4 A schematic diagram of the structure of a model training device provided in an embodiment of the present application;

[0062] Figure 5 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0063] The technical solutions in the embodiments of the present application will be described below in conjunction with the drawings in the embodiments of the present application.

[0064] Embodiment 1

[0065] Reference Figure 1 , shows a flow chart of the steps of a resource configuration method provided by an embodiment of the present application, such as Figure 1 As shown, the resource configuration method may specifically include the following steps:

[0066] Step 101: Obtain model parameters of the neural network model and the target number corresponding to the graphics processor.

[0067] The embodiments of the present application can be applied to a scenario in which a graphics processor is selected in combination with model parameters, and a parameter update mode is set according to configuration information between the selected graphics processors to train a model.

[0068] Model parameters refer to parameters such as the network structure of the neural network model to be trained, for example, network model operators and structures, model input features and other parameters.

[0069] The target number refers to the number of graphics processing units (GPUs) required to train the neural network model.

[0070] When a neural network model based on a graphics processor needs to be trained, model parameters of the neural network model and a target number of graphics processors may be obtained.

[0071] In a specific implementation of the present application, the target number of graphics processors can be obtained based on the experience of historical model training or reasoning. For example, the neural network model currently being trained or the neural network model currently performing the reasoning task is a graphic recognition model. When obtaining the number of graphics processors required for training the graphic recognition model, the historical number of graphics processors used when training the graphic recognition model in the past can be obtained, and then the number of graphics processors used in the current training of the graphic recognition model can be determined based on the historical number. For example, the number of graphics processors used in the historical training of the graphic recognition model includes: 5, 8 and 10. At this time, the number of graphics processors with the largest historical number can be selected as the target number of graphics processors for the current training of the graphic recognition model, etc.

[0072] In another specific implementation of the present application, the target number of graphics processors can be determined according to the complexity of the model structure of the neural network model. In actual applications, business personnel can pre-divide the complexity of the model structure into N (N is a positive integer) levels (such as upper, middle and lower levels, etc.), and pre-configure the corresponding number of graphics processors for the N model complexity levels. When training the neural network model or using the neural network model to perform reasoning tasks, the complexity level of the neural network model can be obtained, and then the target number of graphics processors corresponding to the neural network model can be obtained.

[0073] In a specific implementation, the target number of graphics processors can also be pre-input by business personnel, that is, the business personnel determine the target number of graphics processors based on previous historical knowledge or the complexity of the neural network model. When training the neural network model or using the neural network model to perform reasoning tasks, the business personnel can input the required target number of graphics processors.

[0074] It can be understood that the above-mentioned methods of obtaining the target number of graphics processors are only specific implementation methods listed for a better understanding of the technical solution of the embodiment of the present application. In actual applications, the method of obtaining the target number of graphics processors can be selected according to usage requirements. This embodiment does not limit the specific method of obtaining the target number of graphics processors.

[0075] After obtaining the model parameters of the neural network model and the target number of graphics processors, step 102 is executed.

[0076] Step 102: Acquire a target graphics processor that matches the model parameters and the target quantity.

[0077] The target graphics processor (i.e., target GPU) refers to a graphics processor selected based on model parameters and target quantity for model training or model inference tasks.

[0078] After obtaining the model parameters of the training neural network model input by the user and the target number of graphics processors, the target graphics processors for model training can be screened out according to the model parameters and the target number. In a specific implementation, multiple graphics processors that meet the training requirements can be screened out from a graphics processor cluster according to the model parameters, and then a target number of image processors can be screened out from the multiple graphics processors as target graphics processors. This process can be described in detail in conjunction with the following specific implementation method.

[0079] In a specific implementation of the present application, the above step 102 may include:

[0080] Sub-step A1: Acquire a first number of initial graphics processors matching the model parameters; the first number is greater than or equal to the target number.

[0081] The first number refers to the number of graphics processors selected from the graphics processor cluster. The specific value of the first number can be determined according to actual conditions, and this embodiment does not impose any limitation on this.

[0082] The first quantity is greater than or equal to the target quantity.

[0083] The initial graphics processor (ie, initial GPU) refers to a graphics processor selected to match the model parameters.

[0084] After obtaining the model parameters of the neural network model and the target number of graphics processors corresponding to the model parameters, a first number of initial graphics processors matching the model parameters can be screened out from the graphics processor cluster according to the model parameters.

[0085] After obtaining the first number of initial graphics processors matching the model parameters, sub-step A2 is performed.

[0086] Sub-step A2: selecting the target number of target graphics processors from the initial graphics processors according to the resource data of the initial graphics processors.

[0087] After a first number of initial graphics processors matching the model parameters are acquired, a target number of target graphics processors may be screened out from the initial graphics processors according to resource data of the initial graphics processors.

[0088] Specifically, the above process is a screening process performed when the first number is greater than the target number, that is, when the first number is greater than the target number, the resource data and usage rate of the initial graphics processor can be obtained, and then the resource data can be averaged, smoothed, window maximum value, window minimum value and other processing can be performed to screen out the target graphics processor.

[0089] In this embodiment, when the target graphics processor is screened, the screening may also be performed according to the usage frequency of the initial graphics processor. Specifically, it may be described in detail in combination with the following specific implementation manner.

[0090] In another specific implementation of the present application, the above sub-step A2 may include:

[0091] Sub-step B1: According to the resource data, select a second number of graphics processors whose available resource data is greater than a set data threshold from the initial graphics processors.

[0092] The set data threshold refers to a threshold value corresponding to the available resource data of the graphics processor that is preset by the business personnel and is used to filter the graphics processor. The specific value of the set data threshold value can be determined according to business requirements, and this embodiment does not impose any limitation on this.

[0093] First, a second number of graphics processors having available resource data greater than a set data threshold may be selected from the initial graphics processors according to the resource data.

[0094] After selecting a second number of graphics processors whose available resource data is greater than a set data threshold from the initial graphics processors according to the resource data, sub-step B2 is performed.

[0095] Sub-step B2: According to the resource data, select a second number of graphics processors whose available resource data is greater than a set data threshold from the initial graphics processors.

[0096] When the second number is greater than the target number, the second number of graphics processors may be sorted in descending order of usage frequency, and then the target number of target graphics processors may be screened out from the second number of graphics processors according to the sorting result.

[0097] Of course, optimization can also be performed based on rules, such as maximizing the overall bandwidth, minimizing the bandwidth variance, or other rules that match the network parameter update structure.

[0098] When the first number is equal to the target number, all the screened initial graphics processors may be used as target graphics processors. This embodiment will not describe this situation in detail.

[0099] In practical applications, after the neural network model is trained or the neural network model performs the reasoning task, the machine learning model can be trained based on the historical data of the training or reasoning task, and the model can be used to score several configuration options, and the optimal result or the specified result that meets the threshold is given according to the score.

[0100] Of course, in a specific implementation, when screening the target graphics processor, the target graphics processor can also be screened in combination with basic model attributes, such as model type, file, version, storage path, and business attributes. Optimization or recommended deployment requirements can also be given based on historical data of training or reasoning tasks. Specifically, it can be determined according to business needs, and this embodiment does not limit this.

[0101] After the target GPU matching the model parameters and the target quantity is acquired, step 103 is executed.

[0102] Step 103: Determine a parameter update mode between the target graphics processors according to the configuration information corresponding to the target graphics processors.

[0103] Configuration information refers to node configuration information between two nodes in the target GPU. In this example, the configuration information may include information such as the node type, node quantity, and bandwidth between nodes of each GPU node in the GPU cluster formed by the target GPU.

[0104] The parameter update mode refers to a mode for updating the model parameters of the neural network model during the training process of the neural network model or the process of executing an inference task. In this example, the parameter update mode may include an asynchronous update mode, a synchronous update mode, an asynchronous and synchronous combination update mode, and the like.

[0105] After obtaining the target graphics processors that match the model parameters and the target number, the parameter update mode between the target graphics processors can be determined according to the configuration information corresponding to the target graphics processors. Specifically, the parameter update mode between the target graphics processors can be determined according to the node type and node bandwidth between the target graphics processors.

[0106] This process can be described in detail in conjunction with the following specific implementation method.

[0107] In a specific implementation of the present application, the above step 103 may include:

[0108] Sub-step C1: Obtain the node type of the target GPU and the node bandwidth between the target GPUs.

[0109] In this embodiment, the node type refers to the type of each node (i.e., each GPU) in the GPU cluster formed by the target GPU. In this example, the node type may be determined by the model of the GPU, i.e., the node type also indicates the model of the GPU at each node, such as GeForce, Quadro, Tesla, etc.

[0110] Node bandwidth refers to the network bandwidth required for data transmission between two nodes in the GPU cluster formed by the target GPU. In this example, due to the different types of target GPUs in the GPU cluster, the bandwidth required by the GPUs is also different. Therefore, when determining the model update parameters, the node bandwidth required by each target GPU needs to be considered.

[0111] After the target GPU matching the model parameters and the target quantity is obtained, the configuration information of the target GPU, that is, the node type of the target GPU and the node bandwidth between the target GPUs can be obtained.

[0112] After the node type of the target GPU and the node bandwidth between the target GPUs are obtained, sub-step C2 is executed.

[0113] Sub-step C2: Determine a parameter update mode between the target graphics processors according to the node type, the node bandwidth and the target quantity.

[0114] After the node type and node bandwidth of the target graphics processor are obtained, the parameter update mode between the target graphics processors can be determined according to the node type, node bandwidth and target quantity.

[0115] In Example 1, the network bandwidth required by the node types (i.e., the models of the graphics processors) of different graphics processors is different. Generally, the network bandwidth required by the new model of the graphics processor is greater than the network bandwidth required by the old model of the graphics processor. In this example, the node type of each graphics processor in multiple target graphics processors can be obtained, and then the network bandwidth required by each graphics processor can be determined according to the multiple node types. The parameter update mode is determined according to the total bandwidth value of the multiple network bandwidths or the average value of the multiple network bandwidths. For example, the parameter update mode is a synchronous update mode, an asynchronous update mode, and an asynchronous synchronous combination mode. The larger the network bandwidth, the more it tends to the synchronous update mode. On the contrary, the smaller the network bandwidth, the more it tends to the asynchronous update mode. In actual applications, a bandwidth threshold interval can be set. When the total bandwidth value or the average value of the multiple network bandwidths is within the bandwidth threshold interval, the asynchronous synchronous combination mode is selected as the parameter update mode. When the total bandwidth value or the average value of the multiple network bandwidths is greater than the maximum value of the bandwidth threshold interval, the synchronous update mode is selected as the parameter update mode. When the total bandwidth value or the average value of the multiple network bandwidths is less than the minimum value of the bandwidth threshold interval, the asynchronous update mode is selected as the parameter update mode.

[0116] It can be understood that the bandwidth threshold intervals for the total bandwidth value and the average value of multiple network bandwidths are different, and the specific area range can be preset by business personnel.

[0117] In Example 2, when data is transmitted between graphics processors, the network bandwidth required for data transmission between graphics processors of different models (i.e., node types) is also different. Generally, the network bandwidth required for a new model of graphics processor is greater than the network bandwidth required for an old model of graphics processor. When data is transmitted between graphics processors of the same old model, the required network bandwidth is relatively low, while when data is transmitted between new models of graphics processors or when old models of graphics processors transmit data to new models of graphics processors, the required network bandwidth is relatively high. In this example, the parameter update mode can be taken as an example of a synchronous update mode, an asynchronous update mode, and an asynchronous synchronous combination update mode. The larger the network bandwidth, the more it tends to the synchronous update mode. On the contrary, the smaller the network bandwidth, the more it tends to the asynchronous update mode. In actual applications, a bandwidth threshold interval can be set (the bandwidth threshold interval is different from the bandwidth threshold interval in the above example 1). When the total bandwidth value or average value of the network bandwidth between multiple graphics processors is within the bandwidth threshold interval, the asynchronous synchronous combination mode is selected as the parameter update mode. When the total bandwidth value or average value of the network bandwidth between multiple graphics processors is greater than the maximum value of the bandwidth threshold interval, the synchronous update mode is selected as the parameter update mode. When the total bandwidth value or the average value of the network bandwidth between the multiple graphics processors is less than the minimum value of the bandwidth threshold interval, the asynchronous update mode is selected as the parameter update mode.

[0118] In Example 3, the more the target number of target graphics processors, the larger the network bandwidth required. Conversely, the fewer the target number of target graphics processors, the smaller the network bandwidth required. The parameter update mode can be taken as an example of a synchronous update mode, an asynchronous update mode, and an asynchronous synchronous combination update mode. In this example, a quantity threshold interval can be set. When the target number of graphics processors is within the interval, the asynchronous synchronous combination mode is selected as the parameter update mode. When the target image of the graphics processor is greater than the maximum value of the interval, the synchronous update mode is selected as the parameter update mode. When the target number of graphics processors is less than the minimum value of the interval, the asynchronous update mode is selected as the parameter update mode.

[0119] Based on the above three examples, in this embodiment, the parameter update mode may be selected in combination with the node type of the target graphics processor, the network bandwidth between the target graphics processors, and the target number of target graphics processors.

[0120] Of course, in practical applications, the parameter update mode is not limited to the three modes described in the above examples. Other known parameter update modes, such as All-Reduce, etc., may also be used. The specific method of selecting the parameter update mode can refer to the description of the above three examples, and this embodiment will not be repeated here.

[0121] After the parameter update mode between the target graphics processors is determined according to the configuration information corresponding to the target graphics processors, step 104 is executed.

[0122] After the parameter update mode between the target graphics processors is determined according to the configuration information corresponding to the target graphics processors, step 104 is executed.

[0123] Step 104: Determine resource configuration information of the neural network model based on the target graphics processor and the parameter update mode.

[0124] Resource configuration information refers to configuration information for training a neural network model or using a neural network model to perform reasoning tasks. In this example, the resource configuration information may include a target graphics processor for data transmission when training a neural network model or using a neural network model to perform reasoning tasks, and a model parameter update mode of the neural network model.

[0125] After obtaining the target graphics processor and parameter update mode through the above steps, the resource configuration information of the neural network model can be determined according to the target graphics processor and the parameter update mode, and then the subsequent neural network model training or neural network model execution reasoning task can be performed according to the resource configuration information.

[0126] The resource configuration method provided in the embodiment of the present application obtains the model parameters of the neural network model and the target number corresponding to the graphics processor, obtains the target graphics processor that matches the model parameters and the target number, determines the parameter update mode between the target graphics processors according to the configuration information corresponding to the target graphics processor, and determines the resource configuration information of the neural network model based on the target graphics processor and the parameter update mode. The embodiment of the present application can select the number of target graphics processors for model training in combination with the model parameters, determine the parameter update mode according to the configuration information between the target graphics processors, and determine the resource configuration information of the neural network model according to the number of target graphics processors and the parameter update mode to perform model training or model reasoning, and can perform resource configuration for different deep learning models, thereby improving the training or reasoning efficiency of the deep learning model.

[0127] Embodiment 2

[0128] Reference Figure 2 , shows a flow chart of the steps of a model training method provided in an embodiment of the present application, such as Figure 2 As shown, the model training method may include the following steps:

[0129] Step 201: Obtain resource configuration information of the neural network model to be trained.

[0130] In this embodiment, the resource configuration information refers to the configuration information for training the neural network model. In this example, the resource configuration information may include a target graphics processor for data transmission during neural network model training and a model parameter update mode of the neural network model.

[0131] In this example, the resource configuration information of the neural network model to be trained can be obtained by the method of the first embodiment described above.

[0132] After obtaining the resource configuration information of the neural network model to be trained, step 202 is executed.

[0133] Step 202: Determine the target graphics processor and parameter update mode corresponding to the neural network model according to the resource configuration information.

[0134] After obtaining the resource configuration information of the neural network model to be trained, the target graphics processor and parameter update mode corresponding to the neural network model can be determined according to the resource configuration information. Through the target graphics processor and parameter update mode, resource deployment for model training can be performed.

[0135] Step 203: Obtain training sample data.

[0136] Training sample data refers to the sample input used when training the neural network model to be trained. For example, when the neural network model to be trained is an image and text recognition model, the training sample data can be a series of images containing text selected from the image library. When the neural network model to be trained is an order delivery time estimation model, the training sample data can be historical delivery data, etc.

[0137] After obtaining the resource configuration information and training sample data of the neural network model to be trained, step 204 is executed.

[0138] Step 204: During the training of the neural network model, the training sample data is transmitted based on the target graphics processor.

[0139] After obtaining the resource configuration information and training sample data of the neural network model to be trained, the training sample data can be transmitted based on the target graphics processor during the training process of the neural network model to be trained.

[0140] Step 205: After the loss function of the neural network model is calculated, the model parameters of the neural network model are updated according to the loss function and the parameter update mode until the target neural network model is trained.

[0141] In the process of training the neural network model to be trained, the loss function of the neural network model can be calculated based on the training sample data, and then the model parameters of the neural network model can be updated according to the loss function and the parameter update mode until the loss function is within the preset range, and the target neural network can be trained.

[0142] The embodiment of the present application trains the neural network model through pre-configured resource configuration information, and the resource configuration information fully considers the heterogeneous communication transmission between GPUs, thereby improving the efficiency of model training.

[0143] In this embodiment, the resource configuration information may be multiple pieces of information. In the process of training a neural network model based on the resource configuration information, the neural network model may output scoring values ​​of the multiple pieces of resource configuration information, and select resource configurations for subsequent training of the same type of neural network model in combination with the scoring values. Specifically, it may be described in detail in combination with the following specific implementation method.

[0144] In a specific implementation of the present application, the resource configuration information includes multiple pieces of information, and after the above step 205, it may also include:

[0145] Step S1: Obtain the score value of the resource configuration information output by the neural network model.

[0146] In this embodiment, after the neural network model is trained based on the multiple pieces of resource configuration information, the neural network model can score the multiple pieces of resource configuration information according to a set scoring mechanism to obtain scoring values ​​of the multiple pieces of resource configuration information.

[0147] In this example, the scoring mechanism can be a scoring mechanism for training efficiency, that is, in the process of using multiple resource configuration information to train the neural network model, the training efficiency of the neural network model can be obtained by obtaining multiple resource configuration information, and each resource configuration information can be scored according to its training efficiency to obtain a scoring value for the resource configuration information.

[0148] After obtaining the score value of the resource configuration information output by the neural network model, execute step S2.

[0149] Step S2: According to the score value, the resource configuration information with the largest score value is screened out from the resource configuration information, and the resource configuration information with the largest score value is used as the resource configuration information of the target type of network model.

[0150] The target type of network model refers to the same model type as the neural network model. For example, when the neural network model is a picture and text recognition model, the target type is a picture and text recognition model.

[0151] After obtaining the score value of the resource configuration information output by the neural network model, the resource configuration information with the largest score value can be filtered out from the resource configuration information, and the resource configuration information with the largest score value can be used as the resource configuration information of the target type of neural network model. That is, when the target type of neural network model is subsequently trained, the resource configuration information with the largest score can be directly used for model training.

[0152] The embodiment of the present application directly selects the resource configuration information with the largest score value as the resource configuration information for training the same type of neural network model, without the need to perform resource configuration again, thereby speeding up the training of the neural network model.

[0153] The model training method provided in the embodiment of the present application obtains the resource configuration information of the neural network model to be trained, determines the target graphics processor and parameter update mode corresponding to the neural network model according to the resource configuration information, obtains the training sample data, and transmits the training sample data based on the target graphics processor during the training of the neural network model. After the loss function of the neural network model is calculated, the model parameters of the neural network model are updated according to the loss function and the parameter update mode until the target neural network model is trained. The embodiment of the present application trains the neural network model by combining the pre-configured resource configuration information, and the resource configuration information fully considers the heterogeneous communication transmission between GPUs, thereby improving the efficiency of model training.

[0154] Embodiment 3

[0155] Reference Figure 3 , shows a schematic diagram of the structure of a resource configuration device provided in an embodiment of the present application, such as Figure 3 As shown, the resource configuration device 300 may specifically include the following modules:

[0156] A target quantity acquisition module 310 is used to acquire model parameters of a neural network model and a target quantity corresponding to a graphics processor;

[0157] A target processor acquisition module 320 is used to acquire a target graphics processor that matches the model parameters and the target quantity;

[0158] A parameter update mode determination module 330, configured to determine a parameter update mode between the target graphics processors according to configuration information corresponding to the target graphics processors;

[0159] The resource configuration information determination module 340 is used to determine the resource configuration information of the neural network model based on the target graphics processor and the parameter update mode.

[0160] Optionally, the target processor acquisition module 320 includes:

[0161] An initial processor acquisition unit, configured to acquire a first number of initial graphics processors matching the model parameters; the first number being greater than or equal to the target number;

[0162] The target processor screening unit is used to screen the target number of target graphics processors from the initial graphics processors according to the resource data of the initial graphics processors.

[0163] Optionally, the target processor screening unit includes:

[0164] A graphics processor screening subunit, configured to screen out a second number of graphics processors having available resource data greater than a set data threshold from the initial graphics processors according to the resource data;

[0165] The target processor screening subunit is used to screen out a target number of target graphics processors from the second number of graphics processors in descending order of usage frequency when the second number is greater than the target number.

[0166] Optionally, the parameter update mode determination module 330 includes:

[0167] A node type acquisition unit, used to acquire the node type of the target graphics processor and the node bandwidth between the target graphics processors;

[0168] The parameter update mode determination unit is used to determine the parameter update mode between the target graphics processors according to the node type, the node bandwidth and the target number.

[0169] The resource configuration device provided in the embodiment of the present application obtains the resource configuration information of the neural network model to be trained, determines the target graphics processor and parameter update mode corresponding to the neural network model according to the resource configuration information, obtains the training sample data, and transmits the training sample data based on the target graphics processor during the training of the neural network model. After the loss function of the neural network model is calculated, the model parameters of the neural network model are updated according to the loss function and the parameter update mode until the target neural network model is trained. The embodiment of the present application trains the neural network model by combining the pre-configured resource configuration information, and the resource configuration information fully considers the heterogeneous communication transmission between GPUs, thereby improving the efficiency of model training.

[0170] Embodiment 4

[0171] Reference Figure 4 , which shows a schematic diagram of the structure of a model training device provided in an embodiment of the present application, such as Figure 4 As shown, the model training device 400 may specifically include the following modules:

[0172] Resource configuration information acquisition module 410, used to obtain resource configuration information of the neural network model to be trained;

[0173] A parameter update mode determination module 420, configured to determine a target graphics processor and a parameter update mode corresponding to the neural network model according to the resource configuration information;

[0174] A training sample data acquisition module 430 is used to acquire training sample data;

[0175] A training sample data transmission module 440, configured to transmit the training sample data based on the target graphics processor during the training of the neural network model;

[0176] The target network model acquisition module 450 is used to update the model parameters of the neural network model according to the loss function and the parameter update mode after calculating the loss function of the neural network model, until the target neural network model is trained.

[0177] Optionally, the resource configuration information is a plurality of pieces of information, and the device further includes:

[0178] A scoring value acquisition module, used to obtain the scoring value of the resource configuration information output by the neural network model;

[0179] A resource configuration information screening module, used to screen out the resource configuration information with the largest score value from the resource configuration information according to the score value, and use the resource configuration information with the largest score value as the resource configuration information of the target type of network model;

[0180] The target type of network model refers to a model type that is the same as the type of the neural network model.

[0181] The model training device provided in the embodiment of the present application obtains the resource configuration information of the neural network model to be trained, determines the target graphics processor and parameter update mode corresponding to the neural network model according to the resource configuration information, obtains the training sample data, and transmits the training sample data based on the target graphics processor during the training of the neural network model. After the loss function of the neural network model is calculated, the model parameters of the neural network model are updated according to the loss function and the parameter update mode until the target neural network model is trained. The embodiment of the present application trains the neural network model by combining the pre-configured resource configuration information, and the resource configuration information fully considers the heterogeneous communication transmission between GPUs, thereby improving the efficiency of model training.

[0182] The present application also provides an electronic device, such as Figure 5 As shown, it includes a processor 501, a communication interface 502, a memory 503 and a communication bus 504, wherein the processor 501, the communication interface 502, and the memory 503 communicate with each other through the communication bus 504.

[0183] Memory 503, used for storing computer programs;

[0184] The processor 501 is used to execute the program stored in the memory 503, and implements the following steps:

[0185] Obtain model parameters of the neural network model and the target number corresponding to the graphics processor;

[0186] Acquire a target graphics processor that matches the model parameters and the target quantity;

[0187] Determining a parameter update mode between the target graphics processors according to the configuration information corresponding to the target graphics processor;

[0188] Based on the target graphics processor and the parameter update mode, resource configuration information of the neural network model is determined.

[0189] Optionally, acquiring a target graphics processor that matches the model parameters and the target quantity includes:

[0190] Acquire a first number of initial graphics processors matching the model parameters; the first number is greater than or equal to the target number;

[0191] The target number of target graphics processors is selected from the initial graphics processors according to the resource data of the initial graphics processors.

[0192] Optionally, the step of selecting the target number of target graphics processors from the initial graphics processors according to the resource data of the initial graphics processors comprises:

[0193] According to the resource data, selecting a second number of graphics processors whose available resource data is greater than a set data threshold from the initial graphics processors;

[0194] When the second number is greater than the target number, the target number of target graphics processors is screened out from the second number of graphics processors in descending order of the usage frequency.

[0195] Optionally, determining the parameter update mode between the target graphics processors according to the configuration information corresponding to the target graphics processor includes:

[0196] Obtaining the node type of the target graphics processor and the node bandwidth between the target graphics processors;

[0197] A parameter update mode between the target graphics processors is determined according to the node type, the node bandwidth, and the target quantity.

[0198] When the processor 501 is used to execute the program stored in the memory 503, it can also implement the following steps:

[0199] Obtain resource configuration information of the neural network model to be trained;

[0200] Determining a target graphics processor and a parameter update mode corresponding to the neural network model according to the resource configuration information;

[0201] Obtain training sample data;

[0202] In the process of training the neural network model, transmitting the training sample data based on the target graphics processor;

[0203] After the loss function of the neural network model is calculated, the model parameters of the neural network model are updated according to the loss function and the parameter update mode until the target neural network model is trained.

[0204] Optionally, the resource configuration information is multiple pieces of information.

[0205] After updating the model parameters of the neural network model according to the loss function and the parameter updating mode until the target neural network model is obtained through training, the method further includes:

[0206] Obtaining a score value of the resource configuration information output by the neural network model;

[0207] According to the score value, the resource configuration information with the largest score value is screened out from the resource configuration information, and the resource configuration information with the largest score value is used as the resource configuration information of the target type of network model;

[0208] The target type of network model refers to a model type that is the same as the type of the neural network model.

[0209] The communication bus mentioned in the above terminal can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. The communication bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, only one thick line is used in the figure, but it does not mean that there is only one bus or one type of bus.

[0210] The communication interface is used for communication between the above terminal and other devices.

[0211] The memory may include a random access memory (RAM) or a non-volatile memory, such as at least one disk memory. Optionally, the memory may also be at least one storage device located away from the aforementioned processor.

[0212] The above-mentioned processor can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.

[0213] In another embodiment provided in the present application, a computer-readable storage medium is also provided, which stores instructions. When the computer-readable storage medium is run on a computer, the computer executes the resource configuration method described in any one of the above embodiments, or the model training method described in any one of the above embodiments.

[0214] In another embodiment provided in the present application, a computer program product comprising instructions is also provided, which, when executed on a computer, enables the computer to execute the resource configuration method described in any one of the above embodiments, or the model training method described in any one of the above embodiments.

[0215] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium, for example, the computer instructions may be transmitted from a website site, computer, server or data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) mode to another website site, computer, server or data center. The computer-readable storage medium may be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more available media integrated. The available medium may be a magnetic medium, (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state drive Solid State Disk (SSD)), etc.

[0216] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.

[0217] Each embodiment in this specification is described in a related manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the system embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0218] The above description is only a preferred embodiment of the present application and is not intended to limit the protection scope of the present application. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application are included in the protection scope of the present application.

Claims

1. A resource configuration method, characterized in that: include: Obtain model parameters of the neural network model and the target number corresponding to the graphics processor; Acquire a target graphics processor that matches the model parameters and the target quantity; Determining a parameter update mode between the target graphics processors according to the configuration information corresponding to the target graphics processors; Based on the target graphics processor and the parameter update mode, resource configuration information of the neural network model is determined.

2. The method according to claim 1, characterized in that The obtaining of a target graphics processor that matches the model parameters and the target quantity includes: Acquire a first number of initial graphics processors matching the model parameters; the first number is greater than or equal to the target number; The target number of target graphics processors is selected from the initial graphics processors according to the resource data of the initial graphics processors.

3. The method according to claim 2, characterized in that The step of selecting the target number of target graphics processors from the initial graphics processors according to the resource data of the initial graphics processors includes: According to the resource data, selecting a second number of graphics processors having available resource data greater than a set data threshold from the initial graphics processors; When the second number is greater than the target number, the target number of target graphics processors is selected from the second number of graphics processors in descending order of usage frequency.

4. The method according to claim 1, characterized in that The step of determining a parameter update mode between the target graphics processors according to the configuration information corresponding to the target graphics processors includes: Obtaining the node type of the target graphics processor and the node bandwidth between the target graphics processors; A parameter update mode between the target graphics processors is determined according to the node type, the node bandwidth, and the target quantity.

5. A model training method, characterized in that: include: Obtain resource configuration information of the neural network model to be trained; the resource configuration information is generated according to the method according to any one of claims 1 to 4; Determining a target graphics processor and a parameter update mode corresponding to the neural network model according to the resource configuration information; Obtain training sample data; In the process of training the neural network model, transmitting the training sample data based on the target graphics processor; After the loss function of the neural network model is calculated, the model parameters of the neural network model are updated according to the loss function and the parameter update mode until the target neural network model is trained.

6. The method according to claim 5, characterized in that The resource configuration information is multiple pieces of information. After updating the model parameters of the neural network model according to the loss function and the parameter updating mode until the target neural network model is obtained through training, the method further includes: Obtaining a score value of the resource configuration information output by the neural network model; According to the score value, the resource configuration information with the largest score value is screened out from the resource configuration information, and the resource configuration information with the largest score value is used as the resource configuration information of the target type of network model; The target type of network model refers to a model type that is the same as the type of the neural network model.

7. A resource allocation device, characterized in that: include: A target quantity acquisition module is used to obtain model parameters of the neural network model and the target quantity corresponding to the graphics processor; A target processor acquisition module, used to acquire a target graphics processor that matches the model parameters and the target quantity; A parameter update mode determination module, configured to determine a parameter update mode between the target graphics processors according to configuration information corresponding to the target graphics processors; A resource configuration information determination module is used to determine the resource configuration information of the neural network model based on the target graphics processor and the parameter update mode.

8. A model training device, characterized in that: include: A resource configuration information acquisition module, used to acquire resource configuration information of the neural network model to be trained; the resource configuration information is generated according to the device according to claim 7; A parameter update mode determination module, used to determine a target graphics processor and a parameter update mode corresponding to the neural network model according to the resource configuration information; A training sample data acquisition module is used to acquire training sample data; A training sample data transmission module, used for transmitting the training sample data based on the target graphics processor during the training of the neural network model; The target network model acquisition module is used to update the model parameters of the neural network model according to the loss function and the parameter update mode after calculating the loss function of the neural network model, until the target neural network model is trained.

9. An electronic device, characterized in that: It includes a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other through the communication bus; Memory, used to store computer programs; The processor is used to implement the resource configuration method described in any one of claims 1 to 4, or the model training method described in any one of claims 5 to 6 when executing the program stored in the memory.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, it implements the resource configuration method described in any one of claims 1-4, or the model training method described in any one of claims 5-6.

Images