Resource prediction method and device, electronic equipment, storage medium and program product

Abstract

The invention provides a resource prediction method and device, electronic equipment, a storage medium and a program product, and the method comprises the steps: carrying out the resource prediction of a model training task through a resource prediction model of a current round under the condition of receiving the model training task, and obtaining the predicted resource use information corresponding to the model training task; according to the predicted resource use information, allocating corresponding equipment resources to the model training task; in the process of executing the model training task by utilizing the equipment resource, acquiring equipment resource use information corresponding to the model training task; and updating the resource prediction model of the current round by using predicted resource use information and the equipment resource use information to obtain a resource prediction model of the next round, the resource prediction model of the next round being used for performing resource prediction on a model training task received by the next round. According to the embodiment of the invention, the device resources needing to be allocated can be accurately predicted.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to the technical field of computer, and particularly relates to a resource prediction method and device, electronic equipment, computer readable storage medium and computer program product. BACKGROUND

[0002] With the continuous development of computer technology and neural network technology, in various practical application scenarios, a trained neural network model can be used to process specific tasks, thereby meeting actual needs.

[0003] In the related art, for the training operation of a neural network model, device resources allocated by a computing device need to be used for execution. However, in the case of training a neural network model by using device resources, there is a problem of inaccurate resource allocation, which leads to risks in the model training process. Therefore, how to accurately predict the device resources required for the model training operation becomes a technical problem to be solved. SUMMARY

[0004] The present disclosure provides a resource prediction method and device, electronic equipment, computer readable storage medium and computer program product.

[0005] In a first aspect, the present disclosure provides a resource prediction method, which comprises: in the case of receiving a model training task, using a resource prediction model of a current round to perform resource prediction on the model training task, to obtain predicted resource usage information corresponding to the model training task; according to the predicted resource usage information, allocating corresponding device resources for the model training task; in the process of executing the model training task by using the device resources, obtaining device resource usage information corresponding to the model training task; using the predicted resource usage information and the device resource usage information, updating the resource prediction model of the current round to obtain a resource prediction model of a next round, wherein the resource prediction model of the next round is used to perform resource prediction on a model training task received in the next round.

[0006] In a second aspect, the present disclosure provides a resource prediction device, which comprises: a resource prediction module, a resource allocation module, an information acquisition module and a model updating module.

[0007] The resource prediction module is configured to, in the case of receiving a model training task, use a resource prediction model of a current round to perform resource prediction on the model training task, to obtain predicted resource usage information corresponding to the model training task.

[0008] The resource allocation module is configured to, according to the predicted resource usage information, allocate corresponding device resources for the model training task.

[0009] An information obtaining module is configured to obtain device resource usage information corresponding to the model training task in a process of executing the model training task by using the device resource.

[0010] A model updating module is configured to update the resource prediction model of the current round by using the predicted resource usage information and the device resource usage information, to obtain a resource prediction model of a next round, wherein the resource prediction model of the next round is used to perform resource prediction on a model training task received in the next round.

[0011] In a third aspect, the present disclosure provides an electronic device, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores one or more computer programs executable by the at least one processor, and the one or more computer programs are executed by the at least one processor to enable the at least one processor to perform the above-mentioned resource prediction method.

[0012] In a fourth aspect, the present disclosure provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the above-mentioned resource prediction method.

[0013] In a fifth aspect, the present disclosure provides a computer program product comprising computer-readable code, or a non-transitory computer-readable storage medium carrying computer-readable code, when the computer-readable code is run in a processor of an electronic device, the processor in the electronic device performs the above-mentioned resource prediction method.

[0014] The resource prediction method provided by the embodiments of the present disclosure can provide a resource prediction model, which can perform resource prediction on a model training task to obtain predicted resource usage information corresponding to the model training task when the model training task is received, and can obtain device resource usage information corresponding to the model training task in a process of executing the model training task by using the device resource determined by the predicted resource usage information. Then, the resource prediction model of the current round is updated by using the predicted resource usage information and the device resource usage information of the current round, so as to obtain a resource prediction model with higher prediction accuracy, which is used to perform resource prediction on a model training task received in the next round. Through continuous iterative updating of the resource prediction model, the resource prediction model can accurately predict the device resource to be allocated to the model training task.

[0015] It should be understood that the content described in this part is not intended to identify key or important features of the embodiments of the present disclosure, nor is it used to limit the scope of the present disclosure. Other features of the present disclosure will become apparent from the following description. BRIEF DESCRIPTION OF DRAWINGS

[0016] The accompanying drawings are included to provide a further understanding of the present disclosure and are incorporated in and constitute a part of the specification, illustrate embodiments of the present disclosure and are used to explain the principles of the present disclosure, and are not intended to limit the present disclosure. The above and other features and advantages of the present disclosure will become more apparent from the following detailed description when taken in conjunction with the accompanying drawings, in which:

[0017] Figure 1 A flow chart of a resource prediction method provided for an embodiment of the present disclosure.

[0018] Figure 2 A system architecture diagram in a resource prediction method provided for an embodiment of the present disclosure.

[0019] Figure 3 A processing schematic diagram of a resource prediction method provided for an embodiment of the present disclosure.

[0020] Figure 4 A schematic diagram of an error correction feedback mechanism in a resource prediction method provided for an embodiment of the present disclosure.

[0021] Figure 5 A schematic diagram of scheduling suggestion generation and resource state in a resource prediction method provided for an embodiment of the present disclosure.

[0022] Figure 6 A block diagram of a resource prediction apparatus provided for an embodiment of the present disclosure.

[0023] Figure 7 A block diagram of an electronic device provided for an embodiment of the present disclosure. DETAILED DESCRIPTION

[0024] In order for those skilled in the art to better understand the technical solutions of the present disclosure, the following describes exemplary embodiments of the present disclosure in conjunction with the drawings, including various details of the embodiments of the present disclosure to help understanding, which should be considered only as exemplary. Therefore, those skilled in the art should recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of the present disclosure. Also, in order to be clear and concise, the description in the following description omits the description of well-known functions and structures.

[0025] In the case of no conflict, each embodiment of the present disclosure and each feature in the embodiments can be combined with each other.

[0026] As used herein, the term "and / or" includes any and all combinations of one or more of the associated listed items.

[0027] The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the disclosure. As used herein, the singular forms "a", "an" and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will be further understood that the terms "comprises" and / or "comprising," when used in this specification, specify the presence of stated features, integers, steps, operations, elements, and / or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof. "Coupled" or "connected" or similar terms are not restricted to physical or mechanical connections or associations, but can also include electrical connections, whether direct or indirect.

[0028] Unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as commonly understood by one of ordinary skill in the art. It will be further understood that terms, such as those defined in commonly used dictionaries, should be interpreted as having a meaning that is consistent with their meaning in the context of the relevant art and the present disclosure, and will not be interpreted in an overly literal or overly formal sense unless expressly so defined herein.

[0029] The resource prediction method according to the embodiments of the present disclosure can be executed by an electronic device such as a terminal device or a server. The terminal device can be a vehicle-mounted device, a user equipment (UE), a mobile device, a user terminal, a terminal, a cellular phone, a cordless phone, a personal digital assistant (PDA), a handheld device, a computing device, a vehicle-mounted device, a wearable device, etc. The method can be implemented by a processor invoking computer-readable program instructions stored in a memory. Alternatively, the method can be executed by a server.

[0030] The following explains technical terms in the embodiments of the present disclosure.

[0031] Profiling tool: can be called performance analysis tool or profiling tool, is a software tool for detecting, measuring and analyzing program runtime behavior and resource consumption. It can be used to help developers understand what operations the program performs, how much time it takes, and how much resources it uses during execution, so as to find performance bottlenecks and optimize system efficiency. The Profiling tool in the present disclosure can focus on and collect state information and performance indicators of GPU at the kernel level.

[0032] TensorBoard: is a PyTorch Profiling tool for performance analysis and optimization, which can be used for daily training monitoring, experiment management and result visualization.

[0033] PyTorch Profiler: A profiling tool for performance analysis and optimization, which can be used to locate slow operations, analyze CPU and GPU collaboration efficiency, and identify memory bottlenecks.

[0034] NVIDIA Nsight Systems: A profiling tool for performance analysis and optimization, which can be used to analyze detailed performance metrics and troubleshoot problems in detail.

[0035] GPU Tensor: GPU tensor.

[0036] ONNX: A binary file format based on the Open Neural Network Exchange (ONNX) standard, used to store trained deep learning models.

[0037] torch.profiler.profile (also known as torch.profiler context manager): A performance analysis tool provided by PyTorch, which records operation events by inserting hooks during code execution. torch.profiler.profile is used to record and analyze performance metrics (such as CPU / GPU time, memory consumption, and calculation steps) during model training or inference, helping to locate performance bottlenecks and optimize model efficiency.

[0038] GPU Utilization: GPU utilization.

[0039] trace: A performance analysis file that records the execution process of an application, used to locate time-consuming operations.

[0040] timeline: Visualized timeline data generated by performance analysis tools, showing CPU, memory, and other resource usage.

[0041] PyTorch: An open-source deep learning framework for machine learning and deep learning.

[0042] TensorFlow: A machine learning framework for building and deploying deep learning models.

[0043] ResNet-50: A deep convolutional neural network.

[0044] YOLOv5 (You Only Look Once version 5): A deep learning-based efficient object detection model.

[0045] ViT (Vision Transformer): A computer vision model based on the Transformer architecture.

[0046] With the continuous development of computer technology and neural network technology, in various practical application scenarios, a trained neural network model can be used to process specific tasks to meet actual needs. In related technologies, the training operation of the neural network model needs to be executed through the device resources allocated by the computing device; but in the process of training the neural network model by using the device resources, there is a problem of inaccurate resource allocation, which causes risks in the model training process. For example, in the AI model training and deployment process in related technologies, Profiling analysis tools and resource prediction tools are widely used; Profiling tools such as PyTorch Profiler, TensorBoard, NVIDIA Nsight Systems, etc. can provide information such as time consumption, bandwidth, memory, etc. at the operator level in the training process, which is used for offline performance analysis and bottleneck positioning; based on this, a static prediction method predicts resource indicators such as GPU memory demand and running time based on neural network structure, input dimension and other features before task running, which is used for initial scheduling decision of the task. Moreover, in the process of performance analysis, a performance analysis scheme in related technologies can perform performance analysis in combination with the GPU Tensor state and the fine-grained running data. This scheme can collect and analyze the resource occupation of each stage operator during model running, identify the bottlenecks existing in the training process, and make suggestions for scheduling strategy or model structure based on this. This scheme strongly depends on Profiling data, and needs the actual running of the model to obtain performance indicators, and does not have the prediction ability before running. Therefore, although this scheme provides runtime analysis capability, it lacks prediction ability and closed-loop feedback mechanism, and it is difficult to support dynamic adjustment and scheduling optimization.

[0047] That is to say, related technologies have great defects, including: 1. Profiling is accurate, but its full collection has significant performance overhead, which often causes great interference to the model running process, and usually can only be used for offline analysis and cannot provide guidance before task scheduling; 2. The prediction model is limited in inference accuracy by training samples and feature representation ability, lacks dynamic updating mechanism, and the scheduling strategy is not suitable for sudden resource changes in actual running; 3. Related technologies mostly use prediction or Profiling separately, lack a fusion mechanism, and cannot realize the unified closed loop of performance analysis, scheduling optimization and model adaptive adjustment; 4. The GPU resource utilization rate in current AI (artificial intelligence) infrastructure is generally not high, there are a lot of fragmentation, imbalance between hot and cold spots, and other problems, and there is a lack of resource integration and optimization mechanism based on real load perception and prediction feedback.

[0048] Based on this, this disclosure provides a resource prediction method and apparatus, an electronic device, a computer-readable storage medium, and a computer program product. For details, please refer to the specific explanations of the following embodiments.

[0049] Figure 1 A flowchart illustrating a resource prediction method provided in an embodiment of this disclosure. (Refer to...) Figure 1 The method includes steps S11-S14.

[0050] Step S11: Upon receiving a model training task, use the resource prediction model of the current round to predict the resources of the model training task and obtain the predicted resource usage information corresponding to the model training task.

[0051] Here, the model training task can be understood as the task of training the model to be trained; the model to be trained can be understood as the model that needs to be trained, including but not limited to large models, transformer models, convolutional neural network models, etc.

[0052] A resource prediction model can be understood as a model that can predict the equipment resources required for model training tasks. This resource prediction model can be a large model, a deep learning model, an offline trained regression network, or a graph neural network.

[0053] Predicted resource usage information can be understood as the predicted resource usage information. Resource usage information can be understood as the information on the use of device resources during the execution of model training tasks, including but not limited to: CPU utilization, GPU utilization, memory usage, and video memory usage. Device resources can be understood as the hardware resources of the target device, including but not limited to: CPU, GPU, memory, video memory, and bandwidth. The target device can be understood as the computing device used for model training; for example, the target device can be a computer, a server, a node in a distributed cluster, or a training node in a training platform.

[0054] In some embodiments, the step of using the resource prediction model of the current round to predict resources for the model training task and obtain the predicted resource usage information corresponding to the model training task includes: extracting the model parameters corresponding to the model to be trained from the model to be trained corresponding to the model training task; inputting the model parameters into the resource prediction model of the current round to predict resources and obtain the predicted resource usage information corresponding to the model training task.

[0055] The model to be trained can be understood as a model that needs to be trained, including but not limited to large models, transformer models, convolutional neural network models, etc.

[0056] Model parameters can be understood as the parameters of the model to be trained. These parameters can represent the model structure and input data size of the model to be trained.

[0057] Specifically, the resource prediction method provided in this disclosure can be applied to a resource prediction system or a model training platform. When a model training task is received from a target user, the model to be trained corresponding to the model training task will be obtained. The model to be trained can be sent by the target user or selected from the database based on the model information provided by the user (such as model identifier or model type).

[0058] After obtaining the model to be trained, the model parameters corresponding to the model to be trained can be extracted and input into the resource prediction model to make resource predictions, thereby obtaining the predicted resource usage information for performing the model training task.

[0059] Taking the application of the resource prediction method provided in this embodiment of the present disclosure in a training platform as an example, the resource prediction method of this disclosure is explained. The resource prediction method of this disclosure provides a closed-loop performance analysis and scheduling system (hereinafter referred to as the scheduling system) of "prediction + feedback" to improve the resource utilization and execution efficiency of AI model training and inference. That is to say, this disclosure is a hybrid scheme combining system and method. The scheduling system consists of several functional modules, which interact and work together through standard interfaces. The resource prediction method corresponds to a complete execution process, covering the prediction, operation, analysis and feedback of the task. Among them, the scheduling system includes five functional modules: static analysis prediction module, runtime profiling module, fusion correction module, scheduling suggestion engine, and visualization and logging module. Correspondingly, the connection relationship between each module includes: user (i.e. target user) uploads model structure (i.e. model to be trained) — static analysis prediction module — profiling module inserts and runs — fusion correction module collects data for comparison — scheduling suggestion engine outputs suggestions — visualization and logging module displays the entire process changes.

[0060] Specifically, the static analysis and prediction module integrates a lightweight resource estimation model (trained offline via a regression network or graph neural network). After the user initiates a model training task, the module obtains the user-uploaded model structure information (i.e., model parameters) and inputs this information into the current round's resource estimation model (i.e., resource prediction model) to predict resources. It then outputs performance metrics such as the expected peak memory usage, single-step execution time, and I / O (input / output) usage (i.e., predicted resource usage information).

[0061] As can be seen from the above embodiments, the resource prediction method of this disclosure has the ability to predict before execution, breaking through the limitations of post-run analysis in profiling; by introducing static structural analysis and offline trained resource estimation models, this disclosure can provide resource assessments such as video memory and execution time before task execution, improving the foresight and accuracy of resource scheduling decisions, which is not available in traditional profiling tools.

[0062] In some embodiments, extracting model parameters corresponding to the model to be trained from the model to be trained corresponding to the model training task includes: obtaining the model file and model configuration information of the model to be trained corresponding to the model training task; extracting parameters from the model file and the model configuration information to obtain the model parameters of the model to be trained, wherein the model parameters include at least one of: network layer information, input dimension information, and number of hyperparameters.

[0063] Here, the model file can be understood as the file of the model to be trained, such as an ONNX file. The model configuration information can be the configuration information of the model to be trained, such as the model's input configuration information.

[0064] The model parameters include, but are not limited to, network layer information, input dimension information, and the number of hyperparameters; among them, network layer information can be structural information, network layer information, number of network layers, and network layer type; input dimension information can be the feature dimensions in the model to be trained.

[0065] Following the previous example, after receiving the task of training or inference models, the system needs to perform resource prediction. Therefore, it calls the static analysis module to read the structure definition file (such as the ONNX model file) and input configuration (such as batch size=32, input length (sequence length)=512) of the model to be trained (inference model), and inputs this information as source data to the static analysis prediction module. Then, the static analysis module extracts model parameters (such as layer type, dimension, number of parameters, etc.) from the structure definition file and input configuration information; subsequently, it can be fed into the internal regression network (i.e., the resource prediction model), predicting that the GPU memory usage is 24GB and the estimated training time per step is 720ms.

[0066] As can be seen from the above embodiments, the resource prediction method of this disclosure performs static analysis based on the model structure definition file (such as ONNX) and input configuration (such as batch size and input length) before the model training or inference task is executed, to obtain accurate model data, which facilitates the subsequent resource prediction model to accurately predict the consumption of key resources, thereby providing a reliable basis for subsequent scheduling decisions.

[0067] Step S12: Allocate corresponding device resources for the model training task based on the predicted resource usage information.

[0068] Specifically, in this embodiment of the present disclosure, after obtaining the predicted resource usage information, a target device corresponding to the model training task is selected from multiple candidate devices based on the predicted resource usage information, and the idle device resources in the target device are used as the device resources for the model training task. For example, when the training platform receives a model training task for a large model uploaded by a user, it performs resource prediction to obtain the predicted resource usage information corresponding to the model training task; then, based on the predicted resource usage information, it selects a training node from multiple training nodes whose idle device resources match the predicted resource usage information, and uses it as the target training node (i.e., the target device) for executing the model training task; then, the idle device resources in the target training node are used as the device resources for executing the model training task.

[0069] In some embodiments, allocating corresponding device resources to the model training task based on the predicted resource usage information includes: inputting the predicted resource usage information into a scheduling module for processing to obtain a candidate resource allocation strategy corresponding to the model training task, wherein the candidate resource allocation strategy includes: device resource information for the device resources allocated to the model training task; displaying the device resource information to a target user, wherein the target user is the user who triggered the model training task; upon receiving an information confirmation instruction from the target user regarding the device resource information, determining the candidate resource allocation strategy as the target resource allocation strategy corresponding to the model training task; upon receiving an information update instruction from the target user regarding the device resource information, adjusting the candidate resource allocation strategy using the updated device resource information carried by the information update instruction to obtain the target resource allocation strategy corresponding to the model training task; and allocating corresponding device resources to the model training task according to the target resource allocation strategy.

[0070] The resource allocation strategy can be understood as the strategy used to allocate corresponding device resources for the model training task. This strategy includes information such as the target device's load level, device resource information, and the priority of the model training task. Candidate resource allocation strategies can be understood as resource allocation strategies that require user confirmation; the target resource allocation strategy can be understood as the resource allocation strategy after user confirmation.

[0071] In this context, priority can be understood as the priority assigned to the model training task. Different priorities correspond to different target devices and device resources. For example, when the priority is low, the probability of the model training task failing is high. In this case, target devices with high load, low computing power, or poor performance can be assigned to the model training task to avoid wasting device resources. Conversely, when the priority is high, the probability of the model training task succeeding is high. Target devices with low load, high computing power, or better performance can be assigned to the model training task to avoid wasting device resources.

[0072] Device resource information can be understood as the information about the device resources that need to be allocated to the model training task. For example, this device resource information may include the target device's device identifier, device type, number of CPUs, number of GPUs, bandwidth, memory area, and video memory area. Correspondingly, updating device resource information can be understood as the device resource information modified by the user.

[0073] Following the previous example, after using the resource estimation model to predict the peak memory usage, single-step execution time, I / O usage, and other performance metrics corresponding to the model training task, an initial resource allocation strategy (i.e., candidate resource allocation strategy) can be generated based on these performance metrics. This initial resource allocation strategy can be generated using the scheduling engine, or it can be the resource allocation strategy matched by matching the performance metrics with the preset resource allocation strategy.

[0074] After obtaining the initial resource allocation strategy, the device resource information is displayed to the user through the screen or display interface, and the user is notified to confirm whether the device resources allocated for the model training task are appropriate.

[0075] Upon receiving a confirmation instruction from the target user regarding the device resource information, the system confirms the user's agreement to the device resources and thus determines the initial resource allocation strategy as the target resource allocation strategy.

[0076] Upon receiving an information update instruction from the target user regarding the device resource information, it is confirmed that the user does not agree to allocate device resources for the task. Therefore, the updated device resource information provided by the user is used to replace the device resource information in the candidate resource allocation strategy to obtain the target resource allocation strategy corresponding to the model training task.

[0077] Then, using a scheduling system or scheduler, the corresponding device resources are allocated to the model training task according to the target resource allocation strategy.

[0078] In some cases, if the prediction results exceed the platform's acceptable range (such as preset resource thresholds), the scheduling engine can output resource suggestions in advance and display them to the user, such as recommending to reduce the batch size or allocate to large memory nodes.

[0079] As can be seen from the above embodiments, this disclosure introduces a user interaction confirmation and feedback adjustment mechanism based on resource estimation. After automatically generating candidate resource allocation strategies based on key performance indicators such as peak memory usage, single-step time, and I / O usage output by the resource estimation model, device resource information can be displayed to the user through the interface, allowing the user to intuitively understand the system's recommended resource configuration, thus improving the platform's observability and user experience. By receiving the user's "confirmation" or "update" command, the user can review and correct the resource allocation strategy generated by the system, which retains the efficiency of automated recommendation while giving the user control, ensuring that resource allocation conforms to actual business priorities and constraints. Finally, the scheduling system executes resource allocation only after the user confirms or completes the adjustment, effectively avoiding resource mismatch problems caused by prediction bias and improving the success rate and operational stability of model training tasks.

[0080] Step S13: During the process of using the device resources to perform the model training task, obtain the device resource usage information corresponding to the model training task.

[0081] Among them, device resource usage information can be understood as resource usage information collected during the execution of the model training task, including but not limited to GPU utilization, video memory usage, operator execution time, communication latency, etc.

[0082] In some embodiments, obtaining device resource usage information corresponding to the model training task during the execution of the model training task using the device resources includes: receiving device resource usage information sent by an information acquisition module running on the target device when the model training task is executed using device resources in the target device; wherein the information acquisition module is used to collect the resource usage information of the model training task for the device resources during the execution of the model training task using the device resources.

[0083] The information collection module can be understood as a module used to collect resource usage information. This information collection module can be deployed in the target device and can be a profiling tool or a monitoring tool.

[0084] Following the previous example, the training platform of this embodiment can deploy a lightweight ProfilingAgent (i.e., an information collection module) on each training node to collect key performance indicators (i.e., resource usage information) during the execution of the task (i.e., the model training task) in real time, including GPU utilization, memory usage, operator execution time, communication latency, etc.; these key performance indicators will be sent to the resource awareness system according to a preset period. Alternatively, the training platform can deploy monitoring tools such as Prometheus and Grafana on each training node to collect key performance indicators during the execution of the task in real time; and send them to the resource awareness system according to a preset period.

[0085] As can be seen from the above embodiments, this method improves the accuracy of subsequent model updates by collecting accurate resource usage information in real time through an information collection module configured in the target device.

[0086] In some embodiments, the predicted resource usage information includes predicted resource usage information for the target number of iterations in the model training task, and the device resource usage information includes device resource usage information for the target number of iterations in the model training task; and / or the predicted resource usage information includes predicted resource usage information corresponding to the target network layer in the model to be trained, and the device resource usage information includes device resource usage information corresponding to the target network layer in the model to be trained; wherein, the model to be trained is the model corresponding to the model training task, the target number of iterations is a partial number of iterations in multiple model iterations for the model to be trained, and the target network layer is a partial network layer in the model to be trained.

[0087] Using the previous example, regarding the runtime profiling module in the scheduling system, this profiling module is embedded in the execution flow of the AI ​​training framework, that is to say, the profiling tool is deployed in the training node.

[0088] This profiling tool uses a context manager or hook mechanism (such as torch.profiler.profile) to collect only key path nodes such as operator latency, GPU utilization, memory copying, and communication latency. The sampling results are cached locally in the form of traces or timelines. Specifically, when the model to be trained actually starts running, the profiling module instrumentes and samples data at a preset granularity (such as sampling once every 10 steps (i.e., the target number of iterations), or sampling the first N layers of the model to be trained (i.e., the target network layers). The collected data is cached in a local trace file.

[0089] As can be seen from the above embodiments, this disclosure can reduce profiling overhead and improve operating efficiency: compared with the full sampling scheme, this disclosure adopts a lightweight sampling strategy, which only collects key path operators and stages, significantly reducing interference with the operation process. It is a sampling optimization scheme that balances accuracy and efficiency.

[0090] Step S14: Update the resource prediction model for the current round using the predicted resource usage information and the device resource usage information to obtain the resource prediction model for the next round, wherein the resource prediction model for the next round is used to predict resources for the model training task received in the next round.

[0091] In some embodiments, updating the resource prediction model for the current round using the predicted resource usage information and the device resource usage information to obtain the resource prediction model for the next round includes: determining a resource usage error value based on the predicted resource usage information and the device resource usage information; and adjusting the parameters of the resource prediction model for the current round based on the resource usage error value to obtain the resource prediction model for the next round.

[0092] The resource usage error value can be used to represent the error between the predicted resource usage information and the equipment resource usage information. The resource usage error value can be an index deviation value or a loss value for the resource prediction model calculated by a loss function.

[0093] Using the previous example, the fusion correction module in the scheduling system is a module used to implement the fusion correction mechanism. This fusion correction module includes a prediction value and sampled value comparison unit and an error judgment and dynamic update module. Among them, the prediction value and sampled value comparison unit can calculate the key indicator deviation value (i.e., the resource usage error value) based on reading the prediction value and the actual sampled value (i.e., the predicted resource usage information and the equipment resource usage information).

[0094] The error judgment and dynamic update module can reverse-correct the model parameters of the original resource prediction model (i.e., the resource prediction model of the current round) based on runtime data deviations (i.e., deviations in key indicators). In other words, this embodiment can automatically correct the resource prediction model or automatically correct scheduling rules, such as dynamically adjusting estimated bandwidth or memory requirements. Specifically, automatically correcting scheduling rules means using the corrected resource prediction model to perform resource prediction in the next round of resource prediction, thereby correcting subsequent scheduling suggestions (i.e., generating scheduling suggestions for the user), and recording common bottleneck patterns for reuse in the future.

[0095] As can be seen from the above embodiments, this disclosure realizes a closed-loop feedback mechanism for performance analysis and resource prediction; by collecting runtime profiling data to make real-time corrections to the resource prediction model (such as error feedback and dynamic adjustment), the prediction accuracy can be continuously optimized, and a learnable performance model system can be formed to improve the system's intelligence level.

[0096] In some embodiments, after determining the resource usage error value based on the predicted resource usage information and the device resource usage information, the method further includes: using a scheduling module to generate a scheduling suggestion based on the resource usage error value, and displaying the scheduling suggestion to a target user; so that the target user can schedule the device resources allocated to the model training task according to the scheduling suggestion; wherein, the target user is the user who initiated the model training task; the scheduling process includes: reallocating the corresponding device resources for the model training task, reselecting the device to execute the model training task from multiple candidate devices, and adjusting the priority of the model training task.

[0097] Continuing with the previous example, the scheduling engine or scheduling suggestion engine (i.e., the scheduling module) in the scheduling system can generate a new round of scheduling recommendations (i.e., scheduling suggestions) based on the fused metrics (i.e., key metric deviation values) and display these recommendations to the user. Specifically, the scheduling suggestion engine can generate scheduling suggestions such as task scheduling priorities, node suggestions, and migration strategies based on the latest fusion results (i.e., key metric deviation values) and platform resource status, and output these suggestions to an external resource management platform (such as Kubernetes or Slurm) or a user visualization interface, thereby displaying the scheduling suggestions to the user. It should be noted that in the case of concurrent training of multiple models, the model training tasks can also be sorted and a mixed deployment scheme can be proposed.

[0098] As can be seen from the above embodiments, the embodiments of this disclosure can directly participate in scheduling optimization with performance analysis results, thereby improving resource utilization. This allows the analysis results not only to be used for visualization but also as input to generate scheduling suggestions such as task sorting, node allocation, and migration strategies, improving GPU resource utilization and reducing fragmentation and cold start overhead. Furthermore, the embodiments of this disclosure are applicable to multi-model concurrent and mixed-dispatch scheduling scenarios, supporting resource rearrangement and conflict avoidance strategies for parallel tasks of multiple models. This enables dynamic mixed-dispatch and execution rhythm control of heterogeneous tasks, improving overall throughput.

[0099] In some embodiments, adjusting the parameters of the resource prediction model for the current round based on the resource usage error value to obtain the resource prediction model for the next round includes: when the resource usage error value is less than a preset threshold, updating the bias parameters of the resource prediction model for the current round using a preset model update rule to obtain the resource prediction model for the next round; when the resource usage error value is greater than or equal to the preset threshold, generating a model update instruction for the resource prediction model for the current round, and sending the model update instruction to the model update node, wherein the model update instruction is used to instruct the model update node to adjust the parameters of the resource prediction model for the current round to obtain the resource prediction model for the next round.

[0100] The preset threshold can be set according to the actual application scenario. For example, the preset threshold can be 5%, 10%, etc.

[0101] A model update node can be understood as a node used to update the resource prediction model. For example, the model update node can be a training node or a training platform; or the model update node can be a device used by operations and maintenance personnel to update the resource prediction model, such as a console.

[0102] Following the previous example, after the Profiling module collects the actual running value of 22.8GB of video memory and 702ms execution time, the system sends it to the fusion correction module to compare with the predicted value to obtain the deviation value (i.e., the resource usage error value). If the deviation value is found to be less than 5% (i.e., the preset threshold), the system automatically updates the bias parameters of the prediction model according to the preset rules. If the deviation value is equal to or greater than 5%, it indicates that the accuracy of the resource prediction model is low, so it can be fed back to the corresponding console (i.e., the model update node) of the operation and maintenance personnel for manual adjustment.

[0103] As can be seen from the above embodiments, this disclosure compares the actual running values ​​and predicted values ​​of the model training task in the fusion correction module to identify errors, and implements different model update methods according to the degree of deviation. By introducing real running data to analyze the deviation of the prediction results, the system can continuously verify and test the effectiveness of the prediction model after the task is executed, avoiding the gradual failure of the model due to environmental changes or the introduction of new model structures. When the deviation value is less than 5%, the system automatically updates the bias parameters of the prediction model according to preset rules, completing model fine-tuning without manual intervention, improving model maintenance efficiency and reducing operation and maintenance costs. When the deviation is equal to or greater than 5%, the system determines that the prediction result has deviated significantly, triggers an alarm and feeds back to the operation and maintenance console, reminding manual intervention for analysis, effectively preventing the spread of erroneous predictions in subsequent scheduling and ensuring the reliability of system decisions.

[0104] In some embodiments, the method further includes: using an information display interface to display at least one of the predicted resource usage information, the device resource usage information, the resource usage error value, and the task status of the model training task; wherein the resource usage error value is determined based on the predicted resource usage information and the device resource usage information, and the task status is obtained by analyzing the device resource usage information.

[0105] The information display interface can be an interface used to display predicted resource usage information, device resource usage information, resource usage error value, and the task status of the model training task. This information display interface can be a webpage, a user visualization interface, etc.

[0106] Continuing with the previous example, the visualization and logging module of the scheduling system can record performance indicator trends (i.e., device resource usage information), key bottleneck nodes, model running status (i.e., task status), resource usage error values, etc., in the log and present them to the user through a user interface in the form of charts or comparative reports, helping the user understand performance issues and optimization directions. Users can view the prediction and actual comparison curves, resource utilization change graphs, and scheduling suggestion summaries through the visualization module (i.e., the visualization and logging module).

[0107] As can be seen from the above embodiments, the embodiments of this disclosure support performance visualization and interpretable feedback mechanisms. Through the supporting log and graphical feedback modules, it supports the generation of resource assessment reports, scheduling recommendation lists and performance bottleneck location maps, thereby enhancing the interpretability of the system and the diagnostic capabilities of developers.

[0108] It should be noted that the system framework independence and modular deployment capability provided by the embodiments of this disclosure are strong: through context manager instrumentation and external interface sampling design, this disclosure can embed the analysis process of user training code without modifying it, supports mainstream AI frameworks (PyTorch, TensorFlow, etc.) and platforms (bare metal, containers, K8s, etc.), and is flexible in deployment and easy to integrate.

[0109] The resource prediction method provided in this disclosure provides a resource prediction model. Upon receiving a model training task, this model predicts resources for that task to obtain predicted resource usage information. During the execution of the model training task using the device resources determined by this predicted resource usage information, device resource usage information corresponding to the model training task can be obtained. Then, the resource prediction model for the current round is updated using the predicted resource usage information and device resource usage information, resulting in a more accurate resource prediction model for predicting resources for the next round of received model training tasks. Through continuous iterative updates, the resource prediction model can accurately predict the device resources that need to be allocated to the model training task.

[0110] Taking the application of the resource prediction method provided in the embodiments of this disclosure in a training platform as an example, the resource prediction method will be explained and described. Figure 2 This is a system architecture diagram of a resource prediction method provided in an embodiment of this disclosure. Based on... Figure 3 As can be seen, the scheduling system provided by the resource prediction method in this embodiment consists of several functional modules, which interact and work collaboratively through a standard interface. The scheduling system includes five functional modules: a static analysis and prediction module, a runtime profiling module, a fusion correction module, a scheduling suggestion engine, and a visualization and logging module. The connections between the modules include: user-uploaded model structure and configuration; the static analysis and prediction module can perform resource prediction and obtain prediction results (such as predicted resource usage information like video memory / time consumption); during actual task execution, the runtime profiling module can perform sampling and obtain profiling sampling data; the fusion correction module can read the predicted value and the actual sampled value, calculate the key indicator deviation, and automatically correct the prediction model; simultaneously, the key indicator deviation and predicted value can also be provided to the scheduling suggestion engine for processing, generating pre-scheduling decisions, and providing them to the resource management platform and the visualization and logging module respectively; the resource management platform can perform scheduling processing on device resources according to the pre-scheduling decision; the visualization and logging module can display the pre-scheduling decision to the user and record it in the log.

[0111] Figure 3 This is a schematic diagram illustrating the processing of a resource prediction method provided in an embodiment of this disclosure, based on... Figure 3 As can be seen, the process of prediction and profiling is integrated in the embodiments of this disclosure; based on this, the resource prediction method provided in the embodiments of this disclosure specifically includes the following steps S31-S36.

[0112] Step S31: Static structural analysis generates predictions.

[0113] Specifically, the system receives the training or inference model task, calls the static analysis module to read its structural description (such as ONNX file, model class definition, etc.), and generates performance predictions by combining the input parameters.

[0114] Step S32: Determine whether scheduling intervention is needed.

[0115] Specifically, the system determines whether scheduling intervention is needed. If the prediction result exceeds the platform's acceptable range, the scheduling engine can output resource suggestions in advance, such as recommending to reduce the batch size or allocate to large memory nodes; otherwise, step S33 is executed.

[0116] Step S33: Run the task and perform instrumentation sampling.

[0117] Specifically, when the model actually starts running, the Profiling module instrumentes and samples data at a preset granularity (the sampling method can be sampling once every 10 steps or sampling the first N layers), and the sampled data can be cached in a local trace file.

[0118] Step S34: Compare the prediction with the profiling results.

[0119] Specifically, the fusion correction mechanism reads the predicted value and the actual sampled value, calculates the deviation of key indicators, and automatically corrects the prediction model or scheduling rules based on the deviation of key indicators, such as dynamically adjusting the estimated bandwidth or memory requirements.

[0120] Figure 4 This is a schematic diagram of an error correction feedback mechanism in a resource prediction method provided in this embodiment of the disclosure; based on Figure 4 As can be seen, the fusion correction module of this embodiment can obtain the predicted value (static analysis) and the profiling sample value (runtime), and input the predicted value and the sample value into the error calculation module (i.e., the predicted value and sample value comparison unit) for calculation to obtain the error value.

[0121] This embodiment of the disclosure can determine whether the error value exceeds the threshold. If it does, the prediction model parameters are manually updated; if not, the current model is retained and fine-tuned. Finally, in the subsequent resource prediction process, the corrected resource prediction model can be used to output the corrected prediction results.

[0122] Step S35: Feedback to correct the predictor / schedule.

[0123] Specifically, the scheduling engine generates a new round of scheduling recommendations based on the fused metrics (i.e., key metric deviations and sampled data); if it is a multi-model concurrent training method, it can also sort the model training tasks and propose a mixed deployment scheme.

[0124] Figure 5This is a schematic diagram illustrating the generation of scheduling suggestions and resource status in a resource prediction method provided in this disclosure embodiment; based on Figure 5 As can be seen, after obtaining the fusion analysis results (i.e., sampled values ​​and error values), the embodiments of this disclosure can input the fusion analysis results into the scheduling engine, use the scheduling model to generate priority scores, resource node status comparisons, and task migration / reordering suggestions, and generate and output scheduling strategies based on priority scores, resource node status comparisons, and task migration / reordering suggestions; the scheduling strategy can be fed back to the scheduling platform or user interface.

[0125] Step S36: Generate scheduling suggestions and output a report.

[0126] Specifically, users can obtain diagnostic suggestions, bottleneck reports, and historical scheduling results through the log interface or API interface.

[0127] It should be noted that the fusion correction and automatic strategy adjustment mechanism in steps S34-S35 can be adapted to different frameworks (such as PyTorch / TensorFlow) and platforms (single-machine or multi-card environment) and can be directly deployed and implemented.

[0128] Based on the above steps, the resource prediction method in this embodiment provides a method for AI model performance analysis and resource scheduling that integrates profiling and prediction; it involves performance optimization and resource management in the training and inference process of artificial intelligence models, specifically involving a dual-engine architecture that integrates runtime profiling analysis and static resource usage prediction, which is suitable for scenarios such as AI model performance analysis, bottleneck identification and dynamic resource scheduling.

[0129] This disclosure achieves full-cycle perception and optimization of model performance by integrating a dual-engine mechanism of profiling and prediction. Specifically, this disclosure predicts resources such as video memory and execution time through static model structure analysis before task execution, which can guide the initial scheduling strategy. This disclosure introduces a feedback correction mechanism to dynamically update the prediction model in combination with runtime profiling data, thereby improving accuracy and forming a closed loop of performance analysis. This disclosure supports perception and optimization of model resource status at different stages before, during, and after execution, thereby improving the system's intelligent scheduling capabilities and overall performance.

[0130] This disclosure constructs a closed-loop performance awareness system that can provide predictions before resource scheduling, collect necessary data during operation, and provide feedback updates after operation. This enables efficient, intelligent, and interpretable model resource optimization and scheduling strategies without introducing additional operating overhead, thereby improving the overall resource utilization, scheduling efficiency, and training stability of the AI ​​system.

[0131] The following two specific embodiments, combined with method steps and data flow, illustrate the application of the technical solutions of the present disclosure embodiments.

[0132] Example 1: Evaluation and dynamic scheduling of training resources for Transformer models.

[0133] Step 1: The user uploads a BERT-Large model structure definition file (such as the ONNX model) and input configuration (batch size=32, input length 512) through the system interface. This information is used as source data input to the static analysis and prediction module.

[0134] Step 2: The static analysis module extracts model parameters (layer type, dimension, number of parameters, etc.) and feeds them into the internal regression network. The predicted memory usage is 24GB, and the estimated training time per step is 720ms. The prediction results are then sent to the scheduling suggestion engine.

[0135] Step 3: The scheduling engine determines that the current GPU node has 32GB of available video memory, which is sufficient. The task is scheduled for execution, and the runtime profiling module (using the torch.profiler context manager) is automatically wrapped around the training code. Only the time consumption and video memory usage of the key operators in the first 10 steps are collected.

[0136] Step 4: The actual running value collected by the Profiling module is 22.8GB of video memory and 702ms. The system sends it to the fusion correction module to compare with the predicted value for error. If the deviation is within 5%, the system automatically updates the bias parameters of the prediction model. If the deviation is greater than 5%, the system feeds back to the console for manual adjustment.

[0137] Step 5: When similar tasks are submitted again in the future, the output value of the prediction module will refer to the corrected model, and the accuracy will be significantly improved.

[0138] Step 6: Users can view the comparison curves between predictions and actual results, resource utilization change graphs, and scheduling suggestion summaries through the visualization module.

[0139] Example 2: Optimization of scheduling for concurrent inference of multiple models.

[0140] Step 1: Users upload multiple models (ResNet-50, YOLOv5, ViT) and their inference input sizes in batches, and the system sends them to the static analysis and prediction module in batches.

[0141] Step 2: The prediction module outputs the estimated GPU memory, computation time, and I / O latency for each model. For example, ResNet-50 requires 2.3GB of GPU memory, YOLOv5 requires 4.5GB, and ViT requires 3.1GB.

[0142] Step 3: The scheduling engine determines that the total available video memory of a certain GPU node is 12GB. Based on the staggered distribution of the peak values ​​of the three models, it recommends to deploy the three models concurrently on the same node and arrange asynchronous inference to maximize the utilization of video memory and bandwidth.

[0143] Step 4: During the initial inference process, the Profiling module collects model startup latency and device temperature fluctuations, records them in the fusion module, and finds that the YOLOv5 memory allocation fluctuates greatly, suggesting that its batch size be adjusted in the future.

[0144] Step 5: Finally, the scheduling module will feed back the optimization results to the user through a graphical interface, displaying the best mixing order, resource conflict warnings, and adjustable parameter suggestions.

[0145] As can be seen from the above embodiments, the embodiments of this disclosure can realize pre-run resource usage estimation and task priority evaluation, improving the rationality of scheduling strategies; the profiling of the embodiments of this disclosure only collects necessary key indicators, avoiding the performance burden caused by full instrumentation and reducing operational disturbances; the embodiments of this disclosure provide a runtime dynamic feedback mechanism to continuously correct the prediction model and improve estimation accuracy; performance analysis and optimization suggestions can be provided at each stage of model training / inference, covering the entire lifecycle performance management; the embodiments of this disclosure significantly improve the utilization of computing resources such as GPUs, reducing resource fragmentation and idle waste; support multi-model concurrent scheduling optimization and resource rearrangement, enhancing the system's elastic scheduling capability; and provide interpretable scheduling suggestions and bottleneck attribution to help developers quickly locate and optimize task operation.

[0146] In summary, the embodiments disclosed herein provide a dual-engine mechanism that integrates static prediction and runtime profiling: before task execution, the system predicts the memory usage and runtime of the model through a lightweight analysis module, and during execution, it samples key indicators through a profiling module to construct a closed-loop path of prediction-verification-correction, which is the basic capability for achieving high-accuracy resource management and scheduling.

[0147] This disclosure provides a model feedback correction mechanism based on prediction bias: by automatically updating the prediction model or adjusting the scheduling strategy through error judgment and sampling feedback, the performance analysis system has the ability to adapt and evolve.

[0148] This disclosure provides a low-overhead profiling strategy: a lightweight instrumentation mechanism is used to sample only critical path nodes or critical stages, avoiding significant interference with task operation and ensuring system stability.

[0149] This disclosure provides a linkage mechanism between the scheduling engine and the fusion performance analysis module: the analysis results can be directly applied to the scheduling engine to provide priority ranking, resource recommendation and task migration decisions, thereby improving the level of scheduling intelligence.

[0150] This disclosure provides resource reuse and concurrency optimization capabilities for multiple models / tasks: it supports the mixed deployment of multiple tasks, maximizes the utilization of shared computing resources, and improves system throughput and average utilization.

[0151] The embodiments disclosed herein achieve non-intrusive framework and strong platform adaptability: This disclosure does not rely on modifying the user's training code, and can be quickly integrated through context injection, environment variables or scheduling plugins, and is compatible with mainstream frameworks such as PyTorch and TensorFlow.

[0152] It is understood that the various method embodiments mentioned above in this disclosure can be combined with each other to form combined embodiments without violating the principle and logic. Due to space limitations, this disclosure will not elaborate further. Those skilled in the art will understand that in the above methods of specific implementation, the specific execution order of each step should be determined by its function and possible internal logic.

[0153] In addition, this disclosure also provides a resource prediction device, an electronic device, and a computer-readable storage medium, all of which can be used to implement any of the resource prediction methods provided in this disclosure. The corresponding technical solutions and descriptions are described in the corresponding section of the method and will not be repeated here.

[0154] Figure 6 This is a block diagram of a resource prediction device provided in an embodiment of the present disclosure.

[0155] Reference Figure 6 This disclosure provides a resource prediction device, which includes: a resource prediction module 601, a resource allocation module 602, an information acquisition module 603, and a model update module 604.

[0156] The resource prediction module 601 is used to predict the resources of the model training task using the resource prediction model of the current round when a model training task is received, so as to obtain the predicted resource usage information corresponding to the model training task.

[0157] The resource allocation module 602 is used to allocate corresponding device resources for the model training task based on the predicted resource usage information.

[0158] The information acquisition module 603 is used to acquire the device resource usage information corresponding to the model training task during the process of using the device resources to perform the model training task.

[0159] The model update module 604 is used to update the resource prediction model of the current round using the predicted resource usage information and the device resource usage information to obtain the resource prediction model of the next round, wherein the resource prediction model of the next round is used to predict resources for the model training task received in the next round.

[0160] In some embodiments, the model update module 604 is further configured to: determine a resource usage error value based on the predicted resource usage information and the device resource usage information; adjust the parameters of the resource prediction model for the current round based on the resource usage error value to obtain the resource prediction model for the next round.

[0161] In some embodiments, the model update module 604 is further configured to: when the resource usage error value is less than a preset threshold, update the bias parameters of the resource prediction model for the current round using a preset model update rule to obtain the resource prediction model for the next round; when the resource usage error value is greater than or equal to the preset threshold, generate a model update instruction for the resource prediction model for the current round, and send the model update instruction to the model update node, wherein the model update instruction is used to instruct the model update node to adjust the parameters of the resource prediction model for the current round to obtain the resource prediction model for the next round.

[0162] In some embodiments, the information acquisition module 603 is further configured to: receive device resource usage information sent by an information acquisition module running on the target device when performing the model training task using device resources in the target device; wherein the information acquisition module is configured to collect the resource usage information of the model training task for the device resources during the process of performing the model training task using the device resources.

[0163] In some embodiments, the predicted resource usage information includes predicted resource usage information for the target number of iterations in the model training task, and the device resource usage information includes device resource usage information for the target number of iterations in the model training task; and / or the predicted resource usage information includes predicted resource usage information corresponding to the target network layer in the model to be trained, and the device resource usage information includes device resource usage information corresponding to the target network layer in the model to be trained; wherein, the model to be trained is the model corresponding to the model training task, the target number of iterations is a partial number of iterations in multiple model iterations for the model to be trained, and the target network layer is a partial network layer in the model to be trained.

[0164] In some embodiments, the resource prediction module 601 is further configured to: extract model parameters corresponding to the model to be trained from the model to be trained corresponding to the model training task; input the model parameters into the resource prediction model of the current round to perform resource prediction, and obtain the predicted resource usage information corresponding to the model training task.

[0165] In some embodiments, the resource prediction module 601 is further configured to: obtain the model file and model configuration information of the model to be trained corresponding to the model training task; extract parameters from the model file and the model configuration information to obtain the model parameters of the model to be trained, wherein the model parameters include at least one of: network layer information, input dimension information, and number of hyperparameters.

[0166] In some embodiments, the resource allocation module 602 is further configured to: process the predicted resource usage information input scheduling module to obtain a candidate resource allocation strategy corresponding to the model training task, wherein the candidate resource allocation strategy includes: device resource information of the device resources allocated to the model training task; display the device resource information to a target user, wherein the target user is the user who triggered the model training task; upon receiving an information confirmation instruction from the target user regarding the device resource information, determine the candidate resource allocation strategy as the target resource allocation strategy corresponding to the model training task; upon receiving an information update instruction from the target user regarding the device resource information, adjust the candidate resource allocation strategy using the updated device resource information carried by the information update instruction to obtain the target resource allocation strategy corresponding to the model training task; and allocate corresponding device resources to the model training task according to the target resource allocation strategy.

[0167] In some embodiments, the resource prediction device further includes an information display module, which is further configured to: display at least one of the predicted resource usage information, the device resource usage information, the resource usage error value, and the task status of the model training task using an information display interface; wherein the resource usage error value is determined based on the predicted resource usage information and the device resource usage information, and the task status is obtained by analyzing the device resource usage information.

[0168] The resource prediction apparatus provided in this disclosure provides a resource prediction model. Upon receiving a model training task, this model predicts resources for the task and obtains predicted resource usage information corresponding to the task. During the execution of the model training task using the device resources determined by this predicted resource usage information, it acquires device resource usage information corresponding to the task. Then, it updates the resource prediction model using the predicted resource usage information and device resource usage information from the current round, thereby obtaining a more accurate resource prediction model for predicting resources for the next round of received model training tasks. Through continuous iterative updates, the resource prediction model can accurately predict the device resources that need to be allocated to the model training task.

[0169] Figure 7 This is a block diagram of an electronic device provided in an embodiment of the present disclosure.

[0170] Reference Figure 7 This disclosure provides an electronic device, which includes: at least one processor 701; at least one memory 702; and one or more I / O interfaces 703 connected between the processor 701 and the memory 702; wherein the memory 702 stores one or more computer programs that can be executed by the at least one processor 701, and the one or more computer programs are executed by the at least one processor 701 to enable the at least one processor 701 to perform the resource prediction method described above.

[0171] This disclosure also provides a computer-readable storage medium storing a computer program thereon, wherein the computer program, when executed by a processor, implements the resource prediction method described above. The computer-readable storage medium may be volatile or non-volatile.

[0172] This disclosure also provides a computer program product, including computer-readable code, or a non-volatile computer-readable storage medium carrying computer-readable code, wherein when the computer-readable code is run in a processor of an electronic device, the processor in the electronic device executes the resource prediction method described above.

[0173] Those skilled in the art will understand that all or some of the steps, systems, and apparatuses disclosed above, and their functional modules / units, can be implemented as software, firmware, hardware, or suitable combinations thereof. In hardware implementations, the division between functional modules / units mentioned above does not necessarily correspond to the division of physical components; for example, a physical component may have multiple functions, or a function or step may be performed collaboratively by several physical components. Some or all physical components may be implemented as software executed by a processor, such as a central processing unit, digital signal processor, or microprocessor, or as hardware, or as an integrated circuit, such as an application-specific integrated circuit (ASIC). Such software can be distributed on a computer-readable storage medium, which may include computer storage media (or non-transitory media) and communication media (or transient media).

[0174] As is known to those skilled in the art, the term computer storage medium includes volatile and non-volatile, removable and non-removable media implemented in any method or technology for storing information, such as computer-readable program instructions, data structures, program modules, or other data. Computer storage media includes, but is not limited to, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM), static random access memory (SRAM), flash memory or other memory technologies, portable compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical disc storage, magnetic cartridges, magnetic tape, disk storage or other magnetic storage devices, or any other medium that can be used to store desired information and is accessible to a computer. Furthermore, it is known to those skilled in the art that communication media typically contain computer-readable program instructions, data structures, program modules, or other data in modulated data signals such as carrier waves or other transmission mechanisms, and may include any information delivery medium.

[0175] The computer-readable program instructions described herein can be downloaded from computer-readable storage media to various computing / processing devices, or downloaded via a network, such as the Internet, local area network, wide area network, and / or wireless network, to an external computer or external storage device. The network may include copper transmission cables, fiber optic transmission, wireless transmission, routers, firewalls, switches, gateway computers, and / or edge servers. A network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards them to the computer-readable storage media in the respective computing / processing device.

[0176] Computer program instructions used to perform the operations of this disclosure may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, status setting data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Smalltalk, C++, etc., and conventional procedural programming languages ​​such as the "C" language or similar programming languages. The computer-readable program instructions may execute entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or may be connected to an external computer (e.g., via the Internet using an Internet service provider). In some embodiments, electronic circuitry, such as programmable logic circuitry, field-programmable gate arrays (FPGAs), or programmable logic arrays (PLAs), is personalized by utilizing the status information of the computer-readable program instructions to implement various aspects of this disclosure.

[0177] The computer program product described herein can be implemented specifically through hardware, software, or a combination thereof. In one alternative embodiment, the computer program product is specifically embodied in a computer storage medium; in another alternative embodiment, the computer program product is specifically embodied in a software product, such as a software development kit (SDK), etc.

[0178] Various aspects of this disclosure are described herein with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this disclosure. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer-readable program instructions.

[0179] These computer-readable program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus to produce a machine such that, when executed by the processor of the computer or other programmable data processing apparatus, they create means for implementing the functions / actions specified in one or more blocks of the flowchart and / or block diagram. These computer-readable program instructions can also be stored in a computer-readable storage medium that causes a computer, programmable data processing apparatus, and / or other device to operate in a particular manner; thus, the computer-readable medium storing the instructions comprises an article of manufacture that includes instructions for implementing aspects of the functions / actions specified in one or more blocks of the flowchart and / or block diagram.

[0180] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions executed on the computer, other programmable data processing apparatus, or other device to perform the functions / actions specified in one or more boxes of a flowchart and / or block diagram.

[0181] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of an instruction containing one or more executable instructions for implementing a specified logical function. In some alternative implementations, the functions marked in the blocks may occur in a different order than those shown in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, may be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions.

[0182] Example embodiments have been disclosed herein, and while specific terminology has been used, it is for illustrative purposes only and should be construed as such, and is not intended to be limiting. In some instances, it will be apparent to those skilled in the art that features, characteristics, and / or elements described in connection with particular embodiments may be used alone, or in combination with features, characteristics, and / or elements described in connection with other embodiments, unless otherwise expressly indicated. Therefore, those skilled in the art will understand that various changes in form and detail may be made without departing from the scope of this disclosure as set forth by the appended claims.

Claims

1. A resource forecasting method, characterized in that, include: Upon receiving a model training task, the resource prediction model of the current round is used to predict the resources of the model training task to obtain the predicted resource usage information corresponding to the model training task. Based on the predicted resource usage information, allocate corresponding device resources for the model training task; During the process of using the device resources to perform the model training task, the device resource usage information corresponding to the model training task is obtained; Using the predicted resource usage information and the device resource usage information, the resource prediction model for the current round is updated to obtain the resource prediction model for the next round, wherein the resource prediction model for the next round is used to predict resources for the model training task received in the next round.

2. The method according to claim 1, characterized in that, The step of updating the resource prediction model for the current round using the predicted resource usage information and the equipment resource usage information to obtain the resource prediction model for the next round includes: Based on the predicted resource usage information and the equipment resource usage information, a resource usage error value is determined; The parameters of the resource prediction model for the current round are adjusted based on the resource usage error value to obtain the resource prediction model for the next round.

3. The method according to claim 2, characterized in that, The step of adjusting the parameters of the resource prediction model for the current round based on the resource usage error value to obtain the resource prediction model for the next round includes: If the resource usage error value is less than a preset threshold, the bias parameters of the resource prediction model for the current round are updated using a preset model update rule to obtain the resource prediction model for the next round. If the resource usage error value is greater than or equal to a preset threshold, a model update instruction for the resource prediction model in the current round is generated and sent to the model update node. The model update instruction is used to instruct the model update node to adjust the parameters of the resource prediction model in the current round to obtain the resource prediction model for the next round.

4. The method according to any one of claims 1 to 3, characterized in that, The step of obtaining device resource usage information corresponding to the model training task during the process of using the device resources to perform the model training task includes: When performing the model training task using the device resources in the target device, the device resource usage information sent by the information acquisition module running in the target device is received; The information acquisition module is used to collect resource usage information of the model training task for the device resources during the execution of the model training task using the device resources.

5. The method according to claim 4, characterized in that, The predicted resource usage information includes the predicted resource usage information for the target number of iterations in the model training task, and the device resource usage information includes the device resource usage information for the target number of iterations in the model training task. and / or The predicted resource usage information includes the predicted resource usage information corresponding to the target network layer in the model to be trained, and the device resource usage information includes the device resource usage information corresponding to the target network layer in the model to be trained. Wherein, the model to be trained is the model corresponding to the model training task, the target iteration number is a partial iteration number in multiple model iterations for the model to be trained, and the target network layer is a partial network layer in the model to be trained.

6. The method according to claim 1, characterized in that, The step of using the resource prediction model of the current round to predict resources for the model training task, and obtaining the predicted resource usage information corresponding to the model training task, includes: Extract the model parameters corresponding to the model to be trained from the model to be trained corresponding to the model training task; The model parameters are input into the resource prediction model of the current round to perform resource prediction, thereby obtaining the predicted resource usage information corresponding to the model training task.

7. The method according to claim 6, characterized in that, The step of extracting the model parameters corresponding to the model to be trained from the model to be trained for the model training task includes: Obtain the model file and model configuration information of the model to be trained in the model training task; The model file and the model configuration information are used to extract parameters to obtain the model parameters of the model to be trained. The model parameters include at least one of the following: network layer information, input dimension information, and number of hyperparameters.

8. The method according to claim 1, characterized in that, The step of allocating corresponding device resources for the model training task based on the predicted resource usage information includes: The predicted resources are processed by the information input scheduling module to obtain the candidate resource allocation strategy corresponding to the model training task. The candidate resource allocation strategy includes: device resource information of the device resources allocated to the model training task. The device resource information is displayed to the target user, who is the user who triggered the model training task; Upon receiving a confirmation instruction from the target user regarding the device resource information, the candidate resource allocation strategy is determined as the target resource allocation strategy corresponding to the model training task. Upon receiving an information update instruction from the target user regarding the device resource information, the candidate resource allocation strategy is adjusted using the updated device resource information carried by the information update instruction to obtain the target resource allocation strategy corresponding to the model training task. According to the target resource allocation strategy, allocate corresponding device resources for the model training task.

9. The method according to claim 1, characterized in that, The method further includes: The information display interface displays at least one of the following: the predicted resource usage information, the device resource usage information, the resource usage error value, and the task status of the model training task. The resource usage error value is determined based on the predicted resource usage information and the equipment resource usage information, and the task status is obtained by analyzing the equipment resource usage information.

10. A resource prediction device, characterized in that, include: The resource prediction module is used to predict the resources of the model training task using the resource prediction model of the current round when a model training task is received, and to obtain the predicted resource usage information corresponding to the model training task. The resource allocation module is used to allocate corresponding device resources to the model training task based on the predicted resource usage information. The information acquisition module is used to acquire the device resource usage information corresponding to the model training task during the process of using the device resources to perform the model training task. The model update module is used to update the resource prediction model of the current round using the predicted resource usage information and the device resource usage information to obtain the resource prediction model of the next round, wherein the resource prediction model of the next round is used to predict resources for the model training task received in the next round.

11. An electronic device, characterized in that, include: At least one processor; as well as A memory communicatively connected to the at least one processor; wherein, The memory stores one or more computer programs that can be executed by the at least one processor, the one or more computer programs being executed by the at least one processor to enable the at least one processor to perform the method as described in any one of claims 1-9.

12. A computer-readable storage medium having a computer program stored thereon, characterized in that, The computer program, when executed by a processor, implements the method as described in any one of claims 1-9.

13. A computer program product, characterized in that, Includes computer-readable code, or a non-volatile computer-readable storage medium carrying computer-readable code, wherein when the computer-readable code is run in a processor of an electronic device, the processor in the electronic device performs the method as described in any one of claims 1-9.

Images