Training method and device of neural network model, and nonvolatile storage medium

Abstract

The application discloses a neural network model training method and device, and a nonvolatile storage medium. The method comprises the following steps: obtaining a sample data set; determining the calculation amount and the memory access amount of a target neural network model running on an image processing unit, and determining the resource configuration parameters corresponding to each historical task in a plurality of historical tasks; determining a multi-element feature function corresponding to the sample data set according to the calculation amount, the memory access amount and the resource configuration parameters; and training a preset decision generation network model according to the sample data set and the multi-element feature function, to obtain a target decision generation network model. The application solves the technical problem of waste of computing resources caused by the failure to reasonably map discrete image processing unit resources to a plurality of tasks to be processed.

Description

Technical Field

[0001] This application relates to the field of information technology, and more specifically, to a method and apparatus for training a neural network model and a non-volatile storage medium. Background Technology

[0002] Graphics Processing Unit (GPU) virtualization is a technology that divides a physical GPU into multiple virtual GPUs and assigns different tasks to different virtual GPUs to achieve efficient utilization of GPU resources. GPU virtualization technology has broad application prospects in fields such as cloud computing, big data analytics, and artificial intelligence; therefore, GPU virtualization resource allocation methods have received widespread attention in these fields.

[0003] The main methods for allocating GPU virtualization resources include static resource allocation and dynamic resource allocation. Static resource allocation divides the physical GPU into multiple virtual GPUs and assigns different tasks to different virtual GPUs, while dynamic resource allocation dynamically allocates GPU resources based on the actual needs of the tasks to achieve more efficient resource utilization.

[0004] In dynamic resource allocation, common algorithms include round-robin scheduling, priority scheduling, and shortest job first (SJF). Round-robin scheduling divides CPU time into time slices, allocating each slice to a task. When a time slice is used up, the task is placed at the end of a queue, awaiting the next time slice. Priority scheduling schedules tasks based on their priority, with higher-priority tasks being executed first. SJF schedules tasks based on their execution time, prioritizing tasks with shorter execution times. Dynamic resource allocation also includes task migration techniques. Task migration allows tasks on a virtual GPU to be moved to other idle virtual GPUs, maximizing GPU resource utilization.

[0005] Related GPU virtualization task resource allocation methods also include GPU resource pooling technology. GPU resource pooling technology aggregates multiple physical GPU resources into a virtualized GPU resource pool for use by virtual machines. GPU resource pooling technology can integrate multiple physical GPU resources, improving GPU resource utilization, and also enables dynamic allocation and adjustment of GPU resources.

[0006] GPU virtualization based on GPU resource pooling technology can achieve resource isolation. By fine-grained partitioning of video memory and computing power resources, memory isolation and computing power isolation can be achieved, thereby enabling secure isolation of business applications. Users allocate resources to the GPU virtualization resource pool, matching different AI applications with appropriate resources. For some large models that occupy a large proportion of GPU resources, the remaining virtualization resources in the GPU resource pool are discretely distributed. How to fully utilize these discrete resources and establish a matching mapping relationship between resources and tasks is a problem that needs to be solved. Summary of the Invention

[0007] This application provides a method and apparatus for training a neural network model, as well as a non-volatile storage medium, to at least solve the technical problem of wasted computing resources caused by the inability to reasonably map discrete image processing unit resources to multiple tasks to be processed.

[0008] According to one aspect of the embodiments of this application, a method for training a neural network model is provided, comprising: acquiring a sample dataset, wherein the sample dataset consists of multiple historical tasks processed by image processing units in an image processing unit resource pool within the same image processing unit cluster; determining the computational cost and memory access cost of a target neural network model running on the image processing unit, and determining resource configuration parameters corresponding to each of the multiple historical tasks, wherein the target neural network model is a neural network model that processes each of the multiple historical tasks; determining a multivariate feature function corresponding to the sample dataset based on the computational cost, memory access cost, and resource configuration parameters; and training a preset decision generation network model based on the sample dataset and the multivariate feature function to obtain a target decision generation network model, wherein the target decision generation network model is used to determine the performance value of virtualized discrete resources of the image processing unit.

[0009] Optionally, determining the computational and memory access requirements of the target neural network model running on the image processing unit includes: determining the computational and memory access requirements of the target neural network model based on the side length of the output feature map of each convolutional kernel, the side length of each convolutional kernel, the number of input channels of each convolutional kernel, and the number of output channels of each convolutional kernel.

[0010] Optionally, the resource configuration parameters corresponding to each historical task among multiple historical tasks are determined, including: determining the resource configuration parameters corresponding to each historical task based on the video memory and computing power of the image processing unit occupied by each historical task.

[0011] Optionally, a multivariate characteristic function is determined based on computational load, memory access volume, and resource configuration parameters, including: determining a first weight corresponding to computational load, a second weight corresponding to memory access volume, and a third weight corresponding to resource configuration parameters; and determining the multivariate characteristic function by weighted summation of computational load, memory access volume, and resource configuration parameters based on the first weight, second weight, and third weight.

[0012] Optionally, a target decision generation network model is obtained by training a preset decision generation network model based on a sample dataset and multivariate feature functions. This includes: determining a first preset number of sample datasets as the training dataset; processing the training dataset and multivariate feature functions through the preset decision generation network model to obtain a target decision tree, wherein the target decision tree is used to output a first type of feature vector and a second type of feature vector, wherein the first type of feature vector is used to characterize the performance of the task as being related to the computational and memory access costs of the target neural network model, and the second type of feature vector is used to characterize the performance of the task as being related to resource allocation parameters; and training the discriminator of the preset decision generation network model based on the actual performance values ​​and the first performance value corresponding to the first type of feature vector and the second performance value corresponding to the second type of feature vector output by the generator of the preset decision generation network model to obtain the target decision generation network model, wherein the target decision generation network model is used to generate the performance value corresponding to the task to be processed.

[0013] According to another aspect of the embodiments of this application, a scheduling method for virtualized discrete resources of an image processing unit is also provided, comprising: acquiring resources to be configured in the resource pool of the image processing unit; acquiring the computational load and memory access load of the task to be processed; inputting the resources to be configured and the computational load and memory access load of the task to be processed into a target decision generation network model obtained by the training method of the above-mentioned neural network model to determine the performance value corresponding to the virtualized discrete resources; and scheduling the virtualized discrete resources according to the performance value.

[0014] Optionally, scheduling virtualized discrete resources based on performance values ​​includes: determining the performance values ​​corresponding to different virtualized discrete resources; and scheduling different virtualized discrete resources in descending order of performance values.

[0015] According to another aspect of the embodiments of this application, a training apparatus for a neural network model is also provided, comprising: an acquisition module, configured to acquire a sample dataset, wherein the sample dataset consists of multiple historical tasks processed by image processing units in the resource pool of image processing units within the same image processing unit cluster; a first determination module, configured to determine the computational load and memory access load of a target neural network model running on the image processing unit, and to determine the resource configuration parameters corresponding to each historical task among the multiple historical tasks, wherein the target neural network model is a neural network model that processes each historical task among the multiple historical tasks; a second determination module, configured to determine a multivariate feature function corresponding to the sample dataset based on the computational load, memory access load, and resource configuration parameters; and a training module, configured to train a preset decision generation network model based on the sample dataset and the multivariate feature function to obtain a target decision generation network model, wherein the target decision generation network model is used to determine the performance value of the virtualized discrete resources of the image processing unit.

[0016] According to another aspect of the embodiments of this application, a non-volatile storage medium is also provided, the storage medium including a stored program, wherein the program, when running, controls the device where the storage medium is located to execute the above-described neural network model training method.

[0017] According to another aspect of the embodiments of this application, an electronic device is also provided, including: a memory and a processor, wherein the processor is configured to run a program stored in the memory, wherein the program executes the training method of the neural network model described above when it runs.

[0018] In this embodiment, a sample dataset is obtained, which consists of multiple historical tasks processed by image processing units within the same image processing unit cluster's resource pool. The computational and memory usage of the target neural network model running on the image processing unit are determined, along with the resource configuration parameters for each historical task. The target neural network model is the neural network model that processes each historical task. Based on the computational, memory usage, and resource configuration parameters, a multivariate feature function corresponding to the sample dataset is determined. A preset decision generation network model is trained based on the sample dataset and the multivariate feature function to obtain a target decision generation network model. This target decision generation network model is used to determine the performance value of the image processing unit's virtualized discrete resources. By training the preset decision generation network model based on the sample dataset and a multivariate feature function containing computational, memory usage, and resource configuration parameters, a target decision generation network model is obtained to determine the performance value of the image processing unit's virtualized discrete resources. This achieves the goal of scheduling the image processing unit's virtualized discrete resources based on performance values, thereby fully utilizing computing resources and solving the technical problem of wasted computing resources caused by the inability to reasonably map discrete image processing unit resources to multiple tasks. Attached Figure Description

[0019] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings:

[0020] Figure 1 This is a flowchart of a method for training a neural network model according to an embodiment of this application;

[0021] Figure 2 This is a flowchart of a method for scheduling virtualized discrete resources of an image processing unit according to an embodiment of this application;

[0022] Figure 3 This is a schematic diagram of a generative adversarial network according to an embodiment of this application;

[0023] Figure 4 This is a schematic diagram of a decision network according to an embodiment of this application;

[0024] Figure 5 This is a structural diagram of a training device for a neural network model according to an embodiment of this application;

[0025] Figure 6 This is a hardware structure block diagram of a computer terminal (or electronic device) for a neural network model training method according to an embodiment of this application. Detailed Implementation

[0026] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort should fall within the scope of protection of the present application.

[0027] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0028] According to an embodiment of this application, a method embodiment for training a neural network model is provided. It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. Furthermore, although a logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in a different order than that shown here.

[0029] Figure 1 This is a flowchart of a neural network model training method according to an embodiment of this application, such as... Figure 1 As shown, the method includes the following steps:

[0030] Step S102: Obtain the sample dataset, wherein the sample dataset consists of multiple historical tasks processed by image processing units in the image processing unit resource pool within the same image processing unit cluster.

[0031] A GPU cluster is a computing cluster composed of multiple GPUs. The GPU cluster utilizes multiple GPUs to perform computing tasks simultaneously, improving computing performance and efficiency through parallel and distributed computing. Each GPU can execute tasks independently, and multiple GPUs can process different data simultaneously, thereby accelerating computation.

[0032] Optionally, completed AI tasks (historical tasks) include tasks that were trained or inferred using GPU virtualization resource configurations. The resource configuration parameters for AI tasks in the sample dataset should, as far as possible, include any proportion of GPU memory and computing power resources. For example, GPU virtualization resource configuration information is as follows:

[0033] "task1":{"gpu":"1","vcuda-core":"30","vcuda-memory":"1024"}

[0034] "task2":{"gpu":"1","vcuda-core":"20","vcuda-memory":"2048"}

[0035] "taskN":{"gpu":"2","vcuda-core":"70","vcuda-memory":"2048"}

[0036] Step S104: Determine the computational and memory access requirements of the target neural network model running on the image processing unit, and determine the resource configuration parameters corresponding to each historical task in the multiple historical tasks, wherein the target neural network model is the neural network model for processing each historical task in the multiple historical tasks.

[0037] Step S106: Determine the multivariate feature function corresponding to the sample dataset based on the computational load, memory access load, and resource configuration parameters.

[0038] Step S108: Based on the sample dataset and multivariate feature functions, train the preset decision generation network model to obtain the target decision generation network model, wherein the target decision generation network model is used to determine the performance value of the image processing unit virtualized discrete resources.

[0039] Based on the above steps, by training the preset decision generation network model with the sample dataset and multivariate feature functions containing computational complexity, memory access complexity, and resource configuration parameters, a target decision generation network model is obtained to determine the performance value of the virtualized discrete resources of the image processing unit. This achieves the purpose of scheduling the virtualized discrete resources of the image processing unit according to the performance value, thereby realizing the technical effect of making full use of computing resources.

[0040] According to some optional embodiments of this application, the computational cost and memory access cost of the target neural network model running on the image processing unit can be determined by the following method: the computational cost and memory access cost of the target neural network model are determined based on the side length of the output feature map of each convolutional kernel, the side length of each convolutional kernel, the number of input channels of each convolutional kernel, and the number of output channels of each convolutional kernel.

[0041] The GPU resource pool within the same cluster contains multiple GPU nodes, each providing different resource configurations for different Artificial Intelligence (AI) tasks, and the AI ​​tasks running on each node are also different. Completed AI tasks, i.e., historical AI tasks, are selected to construct dataset L1.

[0042] Obtain the computational cost A and memory access cost B of the AI ​​task model corresponding to dataset L1. Computational cost A refers to the number of floating-point operations performed by the model during a single forward propagation with a single input sample, i.e., the model's time complexity, measured in FLOPS. Memory access cost B refers to the total amount of memory swapping during a single forward propagation with a single input sample, i.e., the model's space complexity, including the total number of parameters and the output feature maps of each layer. The number of parameters refers to the total weight parameters of all parameterized layers in the model, and the feature map refers to the size of the output feature map calculated by each layer during real-time operation. Memory access cost is measured in bytes.

[0043] Specifically, the computational complexity formula for the target neural network model is as follows:

[0044] M 2 ·K 2 ·C in ·C out

[0045] The formula for calculating the memory access volume of the target neural network model is as follows:

[0046] (K 2 ·C in ·C out +M 2 ·C out )·4

[0047] Where M is the side length of the feature map output by each convolutional kernel, K is the side length of each convolutional kernel, and C is the side length of the feature map output by each convolutional kernel. in The number of input channels for each convolution kernel, C out The number of output channels for each convolution kernel.

[0048] According to some other optional embodiments of this application, the resource configuration parameters corresponding to each historical task in a plurality of historical tasks are determined by the following method: the resource configuration parameters corresponding to each historical task are determined based on the video memory and computing power of the image processing unit occupied by each historical task.

[0049] Optionally, a performance calculation model is used to calculate resource configuration parameters under different memory and computing power configurations, wherein,

[0050] The performance calculation model is derived from the Roofline Model through parameter transformation. The Roofline Model proposes a method for quantitative analysis using computational intensity, which can calculate the theoretical upper limit of computational performance that a computing platform can achieve. The computational intensity includes the platform's computing power parameter and bandwidth parameter. The computing power parameter is the maximum number of floating-point operations that a computing platform can possibly complete per second, measured in FLOPs / s, which is the upper limit of the computing platform's performance. The bandwidth parameter is the maximum amount of memory swapping that a computing platform can possibly complete per second, measured in bytes / s, which is the upper limit of the computing platform's bandwidth.

[0051] Furthermore, the computational intensity cap measures the maximum number of computations performed per unit of memory swap in the aforementioned computing platform, measured in FLOPs per byte. The computational intensity cap can be expressed by the following formula:

[0052]

[0053] Where π is the upper limit of the computing platform's performance, and β is the upper limit of the computing platform's bandwidth.

[0054] It should be noted that the computing platform in this embodiment refers to an AI application using GPU virtualization resources, which calculates performance metrics based on a performance computing model on a certain computing platform. Furthermore, the memory swapping mentioned in this embodiment is memory in a broad sense; for a CPU computing platform, it refers to system memory, and for a GPU computing platform, it refers to video memory.

[0055] In the GPU virtualization scenario mentioned in this embodiment, gm represents video memory and gu represents computing power. The difference between gm and gu has different effects on bandwidth and computing intensity; therefore, a factor is introduced to transform the above formula to obtain the resource configuration parameter C:

[0056]

[0057] In some optional embodiments of this application, determining a multivariate characteristic function based on computational load, memory access load, and resource configuration parameters includes the following steps: determining a first weight corresponding to computational load, a second weight corresponding to memory access load, and a third weight corresponding to resource configuration parameters; and performing a weighted summation of computational load, memory access load, and resource configuration parameters based on the first weight, second weight, and third weight to determine the multivariate characteristic function.

[0058] By using computational complexity A, memory access complexity B, and resource configuration parameter C as features, a multivariate feature function is determined. This involves assigning a weight to each feature and combining each feature with its corresponding weight. The multivariate feature function is used to receive sample data and output a feature vector.

[0059] Specifically, the sample dataset is divided into training and testing sets in a 4:1 ratio. The performance ratio of the AI ​​task depends on the model's computational cost, memory access requirements, and resource configuration parameters. By calculating these three features, the multivariate feature function h can be determined. The multivariate feature function h is expressed by the following formula:

[0060] h = w1*A + w2*B + w3*C

[0061] Among them, w1, w2, and w3 are weight coefficients, namely the first weight, the second weight, and the third weight.

[0062] As some optional embodiments of this application, training a preset decision generation network model based on a sample dataset and multivariate feature functions to obtain a target decision generation network model includes: determining a first preset number of sample datasets as a training dataset; processing the training dataset and multivariate feature functions through the preset decision generation network model to obtain a target decision tree, wherein the target decision tree is used to output a first type of feature vector and a second type of feature vector, wherein the first type of feature vector is used to characterize the performance of the task as being related to the computational and memory access volume of the target neural network model, and the second type of feature vector is used to characterize the performance of the task as being related to resource configuration parameters; training the discriminator of the preset decision generation network model based on the actual performance values ​​and the first performance value corresponding to the first type of feature vector output by the generator of the preset decision generation network model and the second performance value corresponding to the second type of feature vector to obtain the target decision generation network model, wherein the target decision generation network model is used to generate the performance value corresponding to the task to be processed.

[0063] Decision trees are generated by constructing a decision network using training set data and multivariate feature functions. A decision tree is a classification model based on the features of the input data, which can classify the input data into different categories.

[0064] The decision network categorizes input data into two classes, m and c. The generated decision tree will have two leaf nodes, representing classes m and c respectively. Class m tasks indicate that their performance depends on the model's computational and memory usage, while class c tasks indicate that their performance depends on the resource configuration specifications used. For each leaf node, its feature vector is set as the feature represented by that node, and these feature vectors serve as input to the generative network. The generated feature vectors are mapped to the space of the generated data to produce different performance values. During training, the data complexity is increased by adjusting multiple sets of deduplicated resource configuration specifications, thereby generating the target decision generative network model.

[0065] Specifically, the decision network categorizes input data into two classes, m and c. Class m tasks indicate that their performance depends on the model's computational and memory usage, while class c tasks indicate that their performance depends on the resource configuration specifications used. The decision network combines multiple features into a feature vector, calculates the information entropy of each feature vector, and selects the feature with the lowest information entropy as the splitting feature for the current node. For each branch of the current node, the information entropy of the samples on that branch is calculated. If the information entropy is 0, the branch is a leaf node and labeled with that class; otherwise, the splitting steps are recursively executed using the samples on that branch as input, until the information entropy of all leaf nodes is 0 or a predetermined tree depth is reached.

[0066] It should be noted that Generative Adversarial Networks (GANs) are used as the generator network in this embodiment, such as... Figure 3 As shown, a GAN consists of a generator G and a discriminator D. During training, the generator generates samples, and the discriminator judges the generated samples against real samples and provides the accuracy of the judgment. The generator optimizes based on the feedback from the discriminator, making the generated samples increasingly difficult for the discriminator to distinguish; the discriminator, on the other hand, is trained based on the samples generated by the generator and real samples to improve the accuracy of its judgment.

[0067] Decision networks such as Figure 4 As shown, its output feature vector serves as the input to the generator G. The generator G learns the feature distributions of class c and class m tasks respectively, generating false performance values ​​G(c) and G(m). G(c) and G(m) are input to the discriminator D, along with the true performance values ​​of the completed tasks. The discriminator D distinguishes between the true and false performance values, continuously optimizes, improves the discrimination ability, and returns the error to the generator G, thereby improving the authenticity of the false data.

[0068] Figure 2 This is a flowchart of a method for scheduling virtualized discrete resources of an image processing unit according to an embodiment of this application, such as... Figure 2 As shown, the method includes the following steps:

[0069] Step S202: Obtain the resources to be configured in the resource pool of the image processing unit.

[0070] Step S204: Obtain the computational and memory access amounts of the task to be processed.

[0071] Step S206: Input the resources to be configured and the computational and memory access requirements of the tasks to be processed through... Figure 1 The training method of the neural network model shown yields a target decision generation network model, which determines the performance value corresponding to the virtualized discrete resources.

[0072] Step S208: Schedule virtualized discrete resources according to performance values.

[0073] Scheduling virtualized discrete resources based on performance values ​​can be achieved by: determining the performance values ​​corresponding to different virtualized discrete resources; and scheduling the different virtualized discrete resources in descending order of performance values.

[0074] Scheduling virtualized discrete resources also includes:

[0075] 1. Analyze the performance values ​​generated by the tasks to be processed, prioritize scheduling tasks of type c, and schedule tasks of type m with the next highest priority.

[0076] 2. Different AI tasks use different resource configurations. Based on the computational and memory access requirements of the model, as well as the limitations of the resource configuration specifications, the tasks to be processed are divided into two categories. The performance values ​​of category C tasks under the same configuration specification are the same, and the utilization rate of the corresponding configuration specification under this performance value reaches 100%.

[0077] 3. Schedule tasks according to their urgency. Specifically, prioritize tasks based on their urgency, assigning priority codes as follows: the most urgent tasks are assigned priority code 1, the next most urgent tasks are assigned priority code 2, and so on, until the scheduling of tasks of type C is completed.

[0078] 4. Class m tasks are mainly limited by the computational and memory access requirements of the model itself. Under a certain configuration, the higher the prediction performance value of the task, the higher the utilization rate of that configuration. Scheduling is carried out according to the performance value. Class m tasks with the same performance value are randomly scheduled.

[0079] As described above, this application trains a pre-defined decision generation network to predict the performance of multiple AI tasks to be processed and matches them with the remaining resources in the GPU resource pool for scheduling. Feature functions are constructed by calculating the computational and memory access costs of the models corresponding to historical tasks, as well as resource configuration parameters. A decision network is built using the training set data and feature functions to generate a decision tree. The generated decision tree has two leaf nodes, representing classes m and c respectively. The feature vectors of these nodes are set as the features represented by the nodes and used as input to the generation network. The generation network maps the feature vectors to the space of the generated data. During training, the target model is generated by adjusting multiple sets of deduplicated resource configuration specifications. Furthermore, the remaining resource specifications of the current GPU resources and the tasks to be processed are obtained and used as input to the target model to generate performance values ​​for the tasks to be processed. The tasks to be processed are then matched and scheduled with the remaining resources in the GPU resource pool.

[0080] The discrete resource scheduling method provided in this application utilizes a multivariate feature function-based decision generation network that considers not only the impact of a single feature on the generation result but also the interaction between multiple features, significantly reducing the performance loss in value generation. Furthermore, by adding a reference for model performance prediction values, the matching degree between resources and tasks is improved, enabling automatic scheduling between remaining discrete resources and multiple pending tasks, fully utilizing remaining GPU virtualization resources, and further enhancing GPU resource utilization.

[0081] Figure 5 This is a structural diagram of a training device for a neural network model according to an embodiment of this application, such as... Figure 5 As shown, the device includes:

[0082] The acquisition module 50 is used to acquire a sample dataset, wherein the sample dataset consists of multiple historical tasks processed by image processing units in the image processing unit resource pool within the same image processing unit cluster.

[0083] The first determining module 52 is used to determine the computational and memory access volume of the target neural network model running on the image processing unit, and to determine the resource configuration parameters corresponding to each historical task in the multiple historical tasks, wherein the target neural network model is a neural network model for processing each historical task in the multiple historical tasks.

[0084] The second determining module 54 is used to determine the multivariate feature function corresponding to the sample dataset based on the computational load, memory access load and resource configuration parameters;

[0085] Training module 56 is used to train a preset decision generation network model based on a sample dataset and multivariate feature functions to obtain a target decision generation network model, wherein the target decision generation network model is used to determine the performance value of the image processing unit's virtualized discrete resources.

[0086] It should be noted that the above Figure 5 The modules in the above can be program modules (e.g., a set of program instructions that implement a specific function) or hardware modules. For the latter, they can be represented in the following forms, but are not limited to these: each of the above modules is represented by a processor, or the functions of each of the above modules are implemented by a processor.

[0087] It should be noted that, Figure 5 Preferred embodiments of the shown examples can be found in [reference needed]. Figure 1 The relevant descriptions of the embodiments shown will not be repeated here.

[0088] Figure 6 A hardware block diagram of a computer terminal (or mobile device) for implementing a training method for a neural network model is shown. Figure 6As shown, the computer terminal 60 (or mobile device) may include one or more processors 602 (shown as 602a, 602b, ..., 602n in the figure) (processor 602 may include, but is not limited to, a microprocessor MCU or a programmable logic device FPGA, etc.), a memory 604 for storing data, and a transmission module 606 for communication functions. In addition, it may also include: a display, an input / output interface (I / O interface), a universal serial bus (USB) port (which may be included as one of the ports of a BUS bus), a network interface, a power supply, and / or a camera. Those skilled in the art will understand that... Figure 6 The structure shown is for illustrative purposes only and does not limit the structure of the aforementioned electronic device. For example, the computer terminal 60 may also include... Figure 6 The more or fewer components shown, or having the same Figure 6 The different configurations shown.

[0089] It should be noted that the aforementioned one or more processors 602 and / or other data processing circuits are generally referred to herein as "data processing circuits". These data processing circuits may be embodied, in whole or in part, in software, hardware, firmware, or any other combination thereof. Furthermore, the data processing circuits may be a single, independent processing module, or may be integrated, in whole or in part, into any other element within the computer terminal 60 (or mobile device). As involved in the embodiments of this application, the data processing circuits serve as a processor control mechanism (e.g., selection of a variable resistor termination path connected to an interface).

[0090] The memory 604 can be used to store software programs and modules of application software, such as the program instructions / data storage device corresponding to the training method of the neural network model in this embodiment. The processor 602 executes various functional applications and data processing by running the software programs and modules stored in the memory 604, thereby implementing the aforementioned training method of the neural network model. The memory 604 may include high-speed random access memory, and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 604 may further include memory remotely located relative to the processor 602, and these remote memories can be connected to the computer terminal 60 via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0091] The transmission module 606 is used to receive or send data via a network. Specific examples of the network described above may include a wireless network provided by the communication provider of the computer terminal 60. In one example, the transmission module 606 includes a Network Interface Controller (NIC), which can connect to other network devices via a base station to communicate with the Internet. In another example, the transmission module 606 may be a Radio Frequency (RF) module, used for wireless communication with the Internet.

[0092] The display can be, for example, a touchscreen liquid crystal display (LCD) that allows the user to interact with the user interface of the computer terminal 60 (or mobile device).

[0093] It should be noted here that, in some optional embodiments, the above... Figure 6 The computer device (or electronic device) shown may include hardware elements (including circuitry), software elements (including computer code stored on a computer-readable medium), or a combination of both hardware and software elements. It should be noted that... Figure 6 This is only one instance of a particular specific instance, and is intended to illustrate the types of components that may exist in the aforementioned computer equipment (or electronic equipment).

[0094] It should be noted that, Figure 6 The electronic device shown is used to perform Figure 1 The training method of the neural network model shown above, therefore, the relevant explanations in the execution method of the above commands also apply to this electronic device, and will not be repeated here.

[0095] This application also provides a non-volatile storage medium, which includes a stored program, wherein the program, when running, controls the device where the storage medium is located to execute the above-described neural network model training method.

[0096] A non-volatile storage medium performs the following functions: acquiring a sample dataset, wherein the sample dataset consists of multiple historical tasks processed by image processing units within the same image processing unit cluster's resource pool; determining the computational and memory access requirements of the target neural network model running on the image processing unit, and determining the resource configuration parameters corresponding to each of the multiple historical tasks, wherein the target neural network model is the neural network model processing each of the multiple historical tasks; determining the multivariate feature function corresponding to the sample dataset based on the computational, memory access, and resource configuration parameters; and training a pre-defined decision generation network model based on the sample dataset and the multivariate feature function to obtain a target decision generation network model, wherein the target decision generation network model is used to determine the performance values ​​of the image processing unit's virtualized discrete resources.

[0097] This application also provides an electronic device, including: a memory and a processor, wherein the processor is used to run a program stored in the memory, wherein the program executes the training method of the neural network model described above when it runs.

[0098] The processor is used to run a program that performs the following functions: acquiring a sample dataset, wherein the sample dataset consists of multiple historical tasks processed by image processing units in the resource pool of the same image processing unit cluster; determining the computational and memory access requirements of the target neural network model running on the image processing unit, and determining the resource configuration parameters corresponding to each of the multiple historical tasks, wherein the target neural network model is the neural network model that processes each of the multiple historical tasks; determining the multivariate feature function corresponding to the sample dataset based on the computational, memory access, and resource configuration parameters; and training a preset decision generation network model based on the sample dataset and the multivariate feature function to obtain a target decision generation network model, wherein the target decision generation network model is used to determine the performance value of the virtualized discrete resources of the image processing unit.

[0099] The sequence numbers of the embodiments in this application are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0100] In the above embodiments of this application, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0101] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. The device embodiments described above are merely illustrative; for example, the division of units can be a logical functional division, and in actual implementation, there may be other division methods. For instance, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the displayed or discussed mutual coupling, direct coupling, or communication connection may be through some interfaces; the indirect coupling or communication connection between units or modules may be electrical or other forms.

[0102] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0103] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0104] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to related technologies, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, read-only memory (ROM), random access memory (RAM), portable hard drives, magnetic disks, or optical disks.

[0105] The above description is only a preferred embodiment of this application. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of this application, and these improvements and modifications should also be considered within the scope of protection of this application.

Claims

1. A method for training a neural network model, characterized in that, include: Obtain a sample dataset, wherein the sample dataset consists of multiple historical tasks processed by image processing units in the image processing unit resource pool within the same image processing unit cluster; The computational and memory access requirements of the target neural network model running on the image processing unit are determined, and the resource configuration parameters corresponding to each historical task in the plurality of historical tasks are determined, wherein the target neural network model is a neural network model for processing each historical task in the plurality of historical tasks. Based on the computational load, the memory access load, and the resource configuration parameters, determine the multivariate feature function corresponding to the sample dataset; Based on the sample dataset and the multivariate feature function, a preset decision generation network model is trained to obtain a target decision generation network model. The target decision generation network model is used to determine the performance value of the image processing unit's virtualized discrete resources and divides the input data into m-class tasks and c-class tasks. The performance of m-class tasks depends on the computational and memory access volume of the model, and the performance of c-class tasks depends on the resource configuration specifications used. The c-class tasks are prioritized for scheduling, and the m-class tasks are second-priority tasks for scheduling. The c-class tasks are also scheduled according to their urgency.

2. The method according to claim 1, characterized in that, Determine the computational and memory access requirements of the target neural network model running on the image processing unit, including: The computational complexity and memory access complexity of the target neural network model are determined based on the side length of the output feature map of each convolutional kernel, the side length of each convolutional kernel, the number of input channels of each convolutional kernel, and the number of output channels of each convolutional kernel.

3. The method according to claim 1, characterized in that, Determine the resource configuration parameters corresponding to each of the plurality of historical tasks, including: Based on the video memory and computing power of the image processing unit occupied by each historical task, the resource configuration parameters corresponding to each historical task are determined.

4. The method according to claim 1, characterized in that, Based on the computational load, the memory access load, and the resource configuration parameters, a multivariate characteristic function is determined, including: Determine the first weight corresponding to the computational load, the second weight corresponding to the memory access load, and the third weight corresponding to the resource configuration parameters; The computational load, memory access load, and resource configuration parameters are weighted and summed based on the first weight, the second weight, and the third weight to determine the multivariate characteristic function.

5. The method according to claim 1, characterized in that, Based on the sample dataset and the multivariate feature function, a preset decision generation network model is trained to obtain a target decision generation network model, including: The first preset number of sample datasets is determined as the training dataset; The training dataset and the multivariate feature function are processed by the preset decision generation network model to obtain a target decision tree. The target decision tree is used to output a first type of feature vector and a second type of feature vector. The first type of feature vector is used to characterize the performance of the task as being related to the computational cost and memory access cost of the target neural network model. The second type of feature vector is used to characterize the performance of the task as being related to the resource configuration parameters. Based on the actual performance values ​​and the first performance values ​​corresponding to the first type of feature vectors and the second performance values ​​corresponding to the second type of feature vectors output by the generator of the preset decision generation network model, the discriminator of the preset decision generation network model is trained to obtain the target decision generation network model, wherein the target decision generation network model is used to generate the performance values ​​corresponding to the task to be processed.

6. A method for scheduling virtualized discrete resources of an image processing unit, characterized in that, include: Obtain the resources to be configured from the image processing unit resource pool; Obtain the computational and memory access requirements of the task to be processed; The computational load and memory access load of the resource to be configured and the task to be processed are input into the target decision generation network model obtained by the training method of the neural network model according to any one of claims 1 to 5 to determine the performance value corresponding to the virtualized discrete resource. The virtualized discrete resources are scheduled based on the performance values.

7. The method according to claim 6, characterized in that, Scheduling the virtualized discrete resources based on the performance values ​​includes: Determine the performance values ​​corresponding to different virtualized discrete resources; The different virtualized discrete resources are scheduled in descending order of their performance values.

8. A training device for a neural network model, characterized in that, include: The acquisition module is used to acquire a sample dataset, wherein the sample dataset consists of multiple historical tasks processed by image processing units in the image processing unit resource pool within the same image processing unit cluster. The first determining module is used to determine the computational and memory access requirements of the target neural network model running on the image processing unit, and to determine the resource configuration parameters corresponding to each historical task among the plurality of historical tasks, wherein the target neural network model is a neural network model for processing each historical task among the plurality of historical tasks. The second determining module is used to determine the multivariate feature function corresponding to the sample dataset based on the computational load, the memory access load, and the resource configuration parameters. The training module is used to train a preset decision generation network model based on the sample dataset and the multivariate feature function to obtain a target decision generation network model. The target decision generation network model is used to determine the performance value of the image processing unit's virtualized discrete resources and divides the input data into m-class tasks and c-class tasks. The performance of m-class tasks depends on the computational and memory access volume of the model, and the performance of c-class tasks depends on the resource configuration specifications used. The c-class tasks are prioritized for scheduling, and the m-class tasks are second-priority tasks for scheduling. The c-class tasks are also scheduled according to their urgency.

9. A non-volatile storage medium, characterized in that, The non-volatile storage medium includes a stored program, wherein, when the program is executed, the device containing the non-volatile storage medium is controlled to perform the training method of the neural network model according to any one of claims 1 to 5.

10. An electronic device, characterized in that, include: A memory and a processor, the processor being configured to run a program stored in the memory, wherein the program, when running, executes a training method for a neural network model according to any one of claims 1 to 5.

Images