Training resource configuration method, device and equipment

Abstract

The invention discloses a training resource configuration method, device and equipment, and the method comprises the steps: building a duration model based on the training parameter information of a large model training task and a duration impact factor extracted from the historical training record of an intelligent calculation center; on the basis of the unit time resource use cost of a single GPU card and the relationship between the training duration evaluated by the duration model and the number of the GPU cards, constructing a total resource cost function taking the number of the GPU cards as a variable, and determining the corresponding number of the target GPU cards when the minimum function value of the total resource cost is taken as a target condition; and based on the number of the target GPU cards, configuring training resources of the large model training task.

Description

A method, apparatus, and device for configuring training resources Technical Field

[0001] This application belongs to the field of artificial intelligence technology, specifically relating to a data processing method, apparatus, and device. Background Technology

[0002] With the rapid development of artificial intelligence technology, especially large-scale pre-trained models, intelligent computing centers have become an indispensable core infrastructure supporting model training. Intelligent computing centers are typically configured with large-scale, high-performance graphics processing unit (GPU) server clusters to meet the massive computing power demands of large model training. However, the construction, operation, and maintenance costs of intelligent computing centers are high, and their core training resource—GPU cards—is a scarce, high-value, and energy-intensive resource.

[0003] Therefore, how to efficiently and economically utilize the GPU resources of intelligent computing centers to reduce the total cost of training large models while ensuring the successful completion of training tasks has become a key issue that the industry urgently needs to address. Summary of the Invention

[0004] This application provides a training resource configuration method, system, and apparatus to efficiently and economically utilize the GPU resources of a computing center while ensuring the successful completion of training tasks, thereby reducing the total cost of training large models.

[0005] To address the aforementioned technical problems, this application provides the following: Firstly, embodiments of this application offer a training resource configuration method, comprising: constructing a duration model based on training parameter information of a large model training task and duration influencing factors extracted from historical training records of an intelligent computing center; constructing a total resource cost function with the number of GPU cards as a variable based on the unit time resource usage cost of a single GPU card, the relationship between the training duration evaluated by the duration model and the number of GPU cards, and determining the target number of GPU cards corresponding to minimizing the total resource cost function value; and configuring training resources for the large model training task based on the target number of GPU cards.

[0006] Secondly, embodiments of this application also provide a training resource configuration device, comprising: a model building module, used to construct a duration model based on training parameter information of a large model training task and duration influencing factors extracted from historical training records of an intelligent computing center; a numerical solution module, used to construct a total resource cost function with the number of GPU cards as a variable based on the unit time resource usage cost of a single GPU card, the relationship between the training duration evaluated by the duration model and the number of GPU cards, and to determine the target number of GPU cards corresponding to the minimum value of the total resource cost function as the objective condition; and a resource configuration module, used to configure the training resources of the large model training task based on the target number of GPU cards.

[0007] Thirdly, embodiments of this application provide an electronic device including a processor and a memory, wherein the memory stores a program or instructions executable on the processor, and the program or instructions, when executed by the processor, implement the steps of the training resource configuration method as described in the first aspect.

[0008] Fourthly, embodiments of this application provide a readable storage medium on which a program or instructions are stored, which, when executed by a processor, implement the steps of the training resource configuration method as described in the first aspect.

[0009] Fifthly, embodiments of this application provide a chip, the chip including a processor and a communication interface, the communication interface being coupled to the processor, the processor being used to run programs or instructions to implement the training resource configuration method as described in the first aspect.

[0010] In a sixth aspect, embodiments of this application provide a computer program product stored in a storage medium, which is executed by at least one processor to implement the training resource configuration method as described in the first aspect.

[0011] The training resource allocation method provided in this application constructs a duration model to evaluate the relationship between the training duration of a large model and the number of GPUs used in training the large model. Based on this duration model, the relationship between training duration and the number of GPUs is evaluated, thereby constructing a total resource cost function with the number of GPUs as a variable. The duration model is solved with the goal of minimizing the training resource cost to obtain the target number of GPUs. Training resources for the large model training task are then allocated based on this number of GPUs, ensuring that the minimum amount of GPU resources is allocated while meeting the training task requirements. This provides an objective and calculable scientific basis for resource allocation decisions. Attached Figure Description

[0012] To more clearly illustrate the technical solutions of the embodiments of this application, the drawings used in the description of the embodiments of this application will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0013] Figure 1 is a flowchart illustrating the application of the training resource configuration method provided in this application to a real-world scenario; Figure 2 is a flowchart illustrating the implementation of a training resource configuration method provided in this application; Figure 3 is a schematic diagram illustrating the relationship between training duration and GPU card size in the training resource configuration method provided in this application; Figure 4 is a schematic diagram illustrating the training of the duration influencing factor in the training resource configuration method provided in this application; Figure 5 is a schematic diagram illustrating the apparatus for implementing the training resource configuration method provided in this application; Figure 6 is a schematic diagram illustrating the structure of a training resource configuration apparatus provided in this application; Figure 7 is a schematic diagram illustrating the structure of an electronic device provided in this application. Detailed Implementation

[0014] The technical solutions of the embodiments of this application will be clearly described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this application. All other embodiments obtained by those skilled in the art based on the embodiments of this application are within the scope of protection of this application.

[0015] The terms "first," "second," etc., used in the specification and claims of this application are used to distinguish similar objects and not to describe a specific order or sequence. It should be understood that such use of data can be interchanged where appropriate so that embodiments of this application can be implemented in orders other than those illustrated or described herein, and the objects distinguished by "first," "second," etc., are generally of the same class and the number of objects is not limited; for example, a first object can be one or more. Furthermore, in the specification and claims, "and / or" indicates at least one of the connected objects, and the character " / " generally indicates that the preceding and following objects are in an "or" relationship.

[0016] The step counting method provided in this application will be described in detail below with reference to the accompanying drawings, through specific embodiments and application scenarios.

[0017] As described in the background section, regarding the problems with existing technologies, the industry has not yet formed a scientific and unified quantitative decision-making method for the number of GPU cards to be used for large-scale model training tasks in intelligent computing centers. In practice, the following empirical or reactive configuration methods are mainly relied upon: 1. Setting at fixed time intervals: Based on the estimated duration of the training task (e.g., short, medium, long term), a fixed number of GPU cards are roughly allocated. Although this method is simple and easy to implement, it heavily relies on the personal experience of engineers and fails to fully consider key variables such as the number of model parameters, the scale of training data, and the computational efficiency of a single GPU. This can easily lead to a mismatch between resource allocation and actual needs, resulting in idle and wasted resources or training bottlenecks.

[0018] 2. Dynamic Adjustment Based on Training Progress: By monitoring changes in the model training epochs, iterations, or key performance indicators (such as the decreasing trend of the loss function), the number of GPUs deployed can be adjusted manually or semi-automatically. While this method offers some flexibility, its adjustment decisions still lack a systematic consideration of the underlying hardware resource costs and failure risks. For example, when training needs to be interrupted and rolled back to a checkpoint due to hardware failure in the later stages, this method struggles to proactively incorporate the additional time and resource costs caused by such interruptions into the adjustment logic.

[0019] 3. Event-triggered configuration: Resource reallocation is triggered by monitoring cluster hardware status (such as GPU failure, network congestion) or training anomalies. This method places extremely high demands on the real-time performance and accuracy of the monitoring system and is essentially a "post-event remediation" strategy. Its adjustment occurs after resource waste or training interruption, making it impossible to perform optimal resource allocation planning before the training task starts, thus hindering proactive cost control.

[0020] In addition, academia and industry have proposed some related technical solutions. For example, some related technologies focus on recommending optimal distributed parallel strategies (such as data parallelism and model parallelism) and their hyperparameter configurations for training tasks to improve the computational efficiency of a single training session. Other solutions focus on improving the overall resource utilization of the cluster by optimizing the resource topology or scheduling strategy when dynamically allocating GPU resources among multiple tasks. However, none of these solutions have taken an economic perspective of "total cost of resource usage" to build a decision model that can quantitatively evaluate and minimize the resource consumption of a single large model training task. They have failed to uniformly model and jointly optimize factors such as "GPU unit time cost," "training duration changes due to changes in the number of GPUs," and "additional costs caused by the risk of training interruption."

[0021] In summary, current GPU resource allocation methods rely heavily on empirical judgment or local efficiency optimization, lacking a precise and intelligent decision-making model and method that aims to minimize total training resource costs and comprehensively considers multiple factors such as training data volume, model size, hardware computing power, parallel efficiency, and failure / interruption. This results in the inability to achieve optimal return on investment for valuable GPU resources in intelligent computing centers, leading to significant economic costs and resource efficiency losses.

[0022] To address this, this application provides a training resource configuration method, apparatus, and device that can accurately assess and recommend the optimal number of GPUs for large-scale model training tasks from the perspective of resource cost optimization, thereby achieving efficient and economical utilization of training resources in intelligent computing centers. Figure 1 illustrates the flowchart of the training resource configuration method provided in this application in a real-world scenario, including: Step 1, constructing a duration model based on the amount of training data, model parameters, number of GPUs, effective computing power per GPU, number of tokens processed per unit of parameters using computing power, number of bytes occupied by tokens, and duration influencing factors; Step 2, constructing a total resource cost function with the number of GPUs as the variable, based on the unit time resource usage cost of a single GPU, the relationship between training duration evaluated by the duration model, and the number of GPUs; Step 3, establishing an "optimal resource cost GPU number model" by differentiating the formula for the resource usage cost of large-scale model training in intelligent computing centers; and finally, obtaining the target number of GPUs with the optimal resource usage cost for training. The duration impact factor is derived and updated based on historical statistical data such as the effective computing power of a single GPU card in the previous intelligent computing center, the number of GPU cards configured in the training cluster, the actual training time, the number of tokens processed by computing power per unit parameter, the number of model parameters, the number of bytes occupied by tokens, and the amount of training data for a single training task. The model training method is forward computation > loss function calculation > back propagation.

[0023] The following section describes the implementation process of the training resource configuration method in detail, using the schematic diagram of the specific implementation process shown in Figure 2. This includes: S110, constructing a duration model based on the training parameter information of the large model training task and the duration influencing factors extracted from the historical training records of the intelligent computing center.

[0024] In some exemplary embodiments, the training parameter information includes at least: the number of model parameters, the amount of training data, the effective computing power of a single GPU card, and a token processing metric used to measure the computational efficiency per unit of parameters.

[0025] Theoretically, the training time of large models in intelligent computing centers is usually directly proportional to the amount of training data and the number of model parameters, and inversely proportional to the number of GPUs, the effective computing power of a single GPU, and the token processing metrics used to measure the computational efficiency per unit of parameters (including the number of tokens processed per unit of parameters and the number of bytes occupied by each token). Therefore, the theoretical training time of large models is... The formula can be expressed as: .in, The effective computing power of a single GPU (floating-point operations per second, FLOPS), measured in TFLOPS; GPU refers to the number of GPUs configured in this training cluster. This refers to the duration of theoretical training, expressed in hours. P represents the number of tokens processed using computing power per unit of parameters, expressed as tokens per (flops per parameter); P represents the number of model parameters, expressed as Billions. ); The number of bytes occupied by the token, in bytes. The amount of training data for a single training task, in bytes.

[0026] Figure 3 is a schematic diagram illustrating the relationship between training duration and GPU card size in the training resource configuration method provided in this application embodiment. In Figure 3, the horizontal axis represents the number of GPU cards used for training (cluster size), and the vertical axis represents the training duration (unit: hours). It can be seen that the training duration decreases sharply and gradually stabilizes as the number of GPU cards increases. This curve reveals that there exists a configuration range for the optimal number of GPU cards in terms of resource cost.

[0027] However, in reality, training large models involves many factors beyond theory. For example, firstly, large model training involves parallel computing, and the training data is divided into multiple batches. These training batches are typically allocated to different GPUs or nodes. Each GPU independently computes the forward and backward propagation of the model, calculates the gradients, and then synchronizes the gradients across all GPUs and updates the model parameters through some method (such as All-Reduce). During this process, on the one hand, some GPU computing power is used for batch splitting and synchronizing model parameters; on the other hand, GPU computing power also has idle periods (this is usually due to different parallel strategies resulting in different idle percentages).

[0028] Secondly, since hardware failures at the intelligent computing center are common occurrences, training interruptions require reverting to the most recent Checkpoint for resumed training (the revert process includes: saving Checkpoint data → hardware failure → troubleshooting at the intelligent computing center → training revert → training task restarted), which also increases training time.

[0029] Based on factors other than those mentioned above, the embodiments of this application extract duration-influencing factors from historical training records of the intelligent computing center as one of the bases for constructing the duration model.

[0030] In some exemplary embodiments, the duration influencing factor is learned through a neural network algorithm based on the training duration of historical training tasks in historical training records and the number of GPU cards.

[0031] For example, the duration impact factor can be dynamically learned from historical training records through machine learning methods. This duration impact factor includes at least a parallel factor for correcting theoretical computational efficiency and a recovery factor for reflecting the impact of fault interruptions.

[0032] Among them, the machine learning method is a feedforward neural network, which can be trained using the actual training time of historical tasks, the number of GPU cards, and training parameter information, and dynamically updated the time influence factor through backpropagation.

[0033] For example, Figure 4 is a training schematic diagram of the duration influence factor in the training resource configuration method provided in this application embodiment. In Figure 4, the input data are x1, x2, x3, ..., x from historical training records. n Specifically, this includes the actual training time of historical training tasks, the number of GPUs, and training parameter information. These are the input features of the neural network, used to learn patterns related to training time. The neural network used employs a feedforward structure; the input data is processed through hidden layers and finally reaches the output layer. The parameter determination stage of the output layer outputs the target that this training aims to solve: the optimal values ​​of the time-influence factors α, β, and χ.

[0034] Each time, based on historical training records and preset duration influence factors, the data is input into the duration model. After obtaining the predicted training duration, it can be compared with the actual training duration in the historical training records for each input. Specifically, a loss function can be used to quantify the prediction error by comparing the predicted training duration with the actual training duration in the historical training records for each input. Then, based on the gradient calculated by the loss function, the error is propagated layer by layer from the output layer to the input layer through the backpropagation algorithm to update the values ​​of the duration influence factors α, β, and χ. This ensures that the error between the predicted training duration under the updated values ​​of duration influence factors α, β, and χ and the actual training duration in the historical training records for each input gradually decreases. Finally, through iterative training on a large amount of historical data, the α, β, and χ learned by the network can accurately reflect the actual operational characteristics of the intelligent computing center in terms of parallel efficiency and fault interruption.

[0035] In some exemplary embodiments, the duration model is constructed based on the product of training duration, training data volume, and model parameter volume, the number of GPUs deployed, a comprehensive computing performance factor, and a duration-influencing factor; wherein, the comprehensive computing performance factor is jointly determined by the effective computing power of a single GPU, the number of tokens processed per unit of parameter volume, and the number of bytes occupied by each token. This comprehensive computing performance factor corresponds to the formula below. .

[0036] For example, given a fixed model, training data, and GPU server, the actual training time of a large model is usually inversely proportional to the number of GPUs used. Considering the overhead of parallel computing and the impact of breakpoint training, the training time model can be constructed as follows: the actual training time is directly proportional to the product of the amount of training data and the number of model parameters, and inversely proportional to the number of GPUs used and the overall computing performance of a single GPU. Furthermore, a time-related factor χ is added to this model.

[0037] Specifically, the formula for this duration model can be expressed as: .

[0038] in, , P , Given a fixed model, training data, and GPU server, these are all constants. α, β, and χ are model adjustment coefficients for the actual training time, i.e., time-influencing factors, used to adjust the theoretical training time of the aforementioned large model. Make corrections.

[0039] S120 constructs a total resource cost function with the number of GPUs as the variable, based on the unit time resource usage cost of a single GPU card, the relationship between the training duration evaluated by the duration model and the number of GPUs, and determines the target number of GPUs when the total resource cost function value is minimized.

[0040] Optionally, the unit time resource usage cost of a single GPU card includes the construction investment amortization cost and the operation and maintenance amortization cost of the GPU card; wherein, the operation and maintenance amortization cost includes at least equipment energy consumption, cooling and rack rental fees.

[0041] In some exemplary embodiments, the total training resource cost is constructed by multiplying the unit time resource usage cost of a single GPU card, the number of GPU cards deployed, and the training duration evaluated by the duration model.

[0042] For example, the cost of a single GPU card per unit time consists of investment cost allocation and operating cost allocation. Investment cost allocation is determined based on the overall cost per card and the investment payback period, while operating cost allocation includes at least rack rental, equipment energy consumption, cooling, and maintenance personnel costs. After the intelligent computing center is operational, the unit time cost of a single GPU card can be considered a fixed value. Therefore, the total resource cost for training large models mainly depends on the number of GPU cards used and the duration of resource usage. Assume the unit time resource usage cost of the m-th type of GPU card in the intelligent computing center is... (This value is a constant and can be calculated from construction costs, operating costs, and return on investment.) The resource usage time for training a large model is T, and the resource usage cost for training this large model is... Then there is the resource cost of training this large model. .

[0043] For example, the total resource cost function can be constructed as the product of the resource usage cost per unit time of a single GPU card, the number of GPU cards deployed, and the training duration evaluated by the duration model.

[0044] In some exemplary embodiments, determining the target number of GPUs when the total resource cost function value is minimized includes: taking the derivative of the total resource cost function with respect to the number of GPUs; and finding the number of GPUs that makes the derivative of the total resource cost function with respect to the number of GPUs zero, which is taken as the target number of GPUs.

[0045] In summary, the training cost of large models in an intelligent computing center is mainly determined by resource usage (the number of GPUs of a certain model deployed) and resource usage duration. Since resource usage duration is related to the number of GPUs deployed, it is possible to determine the appropriate number of GPUs (as long as other factors remain constant) to minimize the resource usage cost during large model training. Specifically, the derivative of the total resource cost function with respect to the number of GPUs can be calculated as follows:

[0046]

[0047] Right now

[0048] Therefore, the optimal resource cost GPU number model for deployment is:

[0049] When using a smart computing center for large model training, the effective computing power of a single GPU card for this training session is input. The number of tokens processed using computing power per unit parameter Model parameter count P, Token byte count The amount of training data for a single training task With the latest duration-influencing factors α, β, and χ determined using a feedforward neural network algorithm, the optimal resource-cost GPU card count can be calculated using the optimal resource-cost GPU card count model. That is, the target number of GPU cards when the total resource cost function value is minimized.

[0050] S130 configures training resources for large model training tasks based on the number of target GPU cards.

[0051] Figure 5 is a schematic diagram of the apparatus for implementing the training resource configuration method provided in the embodiments of this application. In Figure 5, the apparatus may include an input module for inputting information such as the effective computing power of a single GPU card, the number of tokens processed per unit of parameter quantity, the number of model parameters, the number of bytes occupied by tokens, and the amount of training data for a single training task; an intelligent parameter training module for propagating the input data to the output layer and intelligently updating the values ​​of duration influence factors α, β, and χ in real time through backpropagation; an intelligent computing center large model actual training time evaluation module for evaluating the total training time of intelligent computing center resources used for training a large model; a large model training optimal resource cost GPU card number evaluation module for evaluating the optimal number of GPU cards used when intelligent computing center resources are used for training a large model; and an output module for outputting the optimal number of GPU cards used, i.e., the target number of GPU cards.

[0052] The training resource allocation method provided in this application constructs a duration model to evaluate the relationship between the training duration of a large model and the number of GPUs used in training the large model. Based on this duration model, the relationship between training duration and the number of GPUs is evaluated, thereby constructing a total resource cost function with the number of GPUs as the variable. The duration model is solved with the goal of minimizing the training resource cost to obtain the target number of GPUs. Training resources for the large model training task are then allocated based on this number of GPUs, ensuring that the minimum amount of GPU resources is allocated while meeting the training task requirements. This provides an objective and calculable scientific basis for resource allocation decisions.

[0053] Figure 6 is a schematic diagram of a training resource configuration device 600 provided in an exemplary embodiment of this application. As shown in Figure 6, the device 600 includes: a model building module 610, a numerical solution module 620, and a resource configuration module 630, wherein: the model building module 610 is used to build a duration model based on the training parameter information of the large model training task and the duration influence factor extracted from the historical training records of the intelligent computing center; the numerical solution module 620 is used to build a total resource cost function with the number of GPU cards as the variable based on the unit time resource usage cost of a single GPU card, the relationship between the training duration evaluated by the duration model and the number of GPU cards, and determine the target number of GPU cards corresponding to the minimum value of the total resource cost function as the objective condition; the resource configuration module 630 is used to configure the training resources of the large model training task based on the target number of GPU cards.

[0054] The training resource allocation device provided in this application constructs a duration model to evaluate the relationship between the training duration of a large model and the number of GPUs used in training the large model. Based on this duration model, the relationship between training duration and the number of GPUs is evaluated, thereby constructing a total resource cost function with the number of GPUs as a variable. The duration model is solved with the goal of minimizing the training resource cost to obtain the target number of GPUs. Training resources for the large model training task are then allocated based on this number of GPUs, ensuring that the minimum amount of GPU resources is used while meeting the training task requirements. This provides an objective and calculable scientific basis for resource allocation decisions.

[0055] Optionally, the training parameter information includes at least: the number of model parameters, the amount of training data, the effective computing power of a single GPU card, and a token processing metric used to measure the computational efficiency per unit of parameters.

[0056] Optionally, the duration influencing factor is learned through a neural network algorithm based on the training duration of historical training tasks in the historical training records and the number of GPU cards.

[0057] Optionally, the duration model is constructed based on the product of training duration and training data volume and model parameter volume, the number of GPUs deployed, the comprehensive computing performance factor, and the duration influence factor; wherein, the comprehensive computing performance factor is jointly determined by the effective computing power of a single GPU, the number of Tokens processed per unit of parameter volume using computing power, and the number of bytes occupied by the Token.

[0058] Optionally, the total training resource cost is constructed by multiplying the unit time resource usage cost of a single GPU card, the number of GPU cards invested, and the training duration evaluated by the duration model.

[0059] Optionally, when the numerical solution module 620 determines the target number of GPU cards corresponding to the objective condition of minimizing the total resource cost function value, it is specifically used to: calculate the derivative of the total resource cost function with respect to the number of GPU cards; and find the number of GPU cards that makes the derivative of the total resource cost function with respect to the number of GPU cards zero, which is taken as the target number of GPU cards.

[0060] The training resource configuration device 600 can implement the methods of the embodiments shown in Figures 1 to 5. For details, please refer to the training resource configuration methods of the embodiments shown in Figures 1 to 5, which will not be described again.

[0061] Figure 7 is a schematic diagram of the structure of an electronic device provided by an exemplary embodiment of this application. As shown in Figure 7, the device includes a memory 71 and a processor 72.

[0062] Memory 71 is used to store computer programs and can be configured to store various other data to support operation on the computing device. Examples of this data include instructions for any application or method used to operate on the computing device, contact data, phone book data, messages, images, videos, etc.

[0063] The processor 72, coupled to the memory 71, is used to execute the computer program in the memory 71 for: constructing a duration model based on training parameter information of a large model training task and duration influencing factors extracted from historical training records of the intelligent computing center; constructing a total resource cost function with the number of GPU cards as a variable based on the unit time resource usage cost of a single GPU card, the relationship between the training duration evaluated by the duration model and the number of GPU cards, and determining the target number of GPU cards corresponding to the minimum value of the total resource cost function as the objective condition; and configuring training resources for the large model training task based on the target number of GPU cards.

[0064] The electronic device provided in this application can construct a duration model to evaluate the relationship between the training duration of a large model and the number of GPUs used in training the large model. Based on this duration model, the relationship between training duration and the number of GPUs is evaluated, thereby constructing a total resource cost function with the number of GPUs as a variable. The duration model is solved with the goal of minimizing training resource cost to obtain the target number of GPUs. Training resources for large model training tasks are then configured based on this number of GPUs, ensuring that the minimum amount of GPU resources is allocated while meeting the training task requirements. This provides an objective and calculable scientific basis for resource allocation decisions.

[0065] Furthermore, as shown in Figure 7, the electronic device also includes other components such as a communication component 73, a display 74, a power supply component 75, and an audio component 76. Figure 7 only schematically shows some components and does not imply that the electronic device only includes the components shown in Figure 7. Additionally, depending on the implementation of the traffic playback device, the components within the dashed boxes in Figure 7 are optional, not mandatory. For example, when the electronic device is implemented as a terminal device such as a smartphone, tablet, or desktop computer, it may include the components within the dashed boxes in Figure 7; when the electronic device is implemented as a server-side device such as a conventional server, cloud server, data center, or server array, it may not include the components within the dashed boxes in Figure 7.

[0066] Accordingly, embodiments of this application also provide a computer-readable storage medium storing a computer program, which, when executed by a processor, enables the processor to implement the steps in the above-described wire harness detection method embodiments.

[0067] The communication component in Figure 7 above is configured to facilitate wired or wireless communication between the device containing the communication component and other devices. The device containing the communication component can access wireless networks based on communication standards, such as WiFi, 2G, or 3G, or combinations thereof. In one exemplary embodiment, the communication component receives broadcast signals or broadcast-related information from an external broadcast management system via a broadcast channel. In one exemplary embodiment, the communication component may further include a Near Field Communication (NFC) module, Radio Frequency Identification (RFID) technology, Infrared Data Association (IrDA) technology, Ultra Wideband (UWB) technology, Bluetooth (BT) technology, etc.

[0068] The memory in Figure 7 above can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk or optical disk.

[0069] The display in Figure 7 above includes a screen, which may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen can be implemented as a touchscreen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touches, swipes, and gestures on the touch panel. The touch sensors can sense not only the boundaries of the touch or swipe action, but also the duration and pressure associated with the touch or swipe operation.

[0070] The power supply component in Figure 7 above provides power to various components of the device in which it resides. The power supply component may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power to the device in which it resides.

[0071] The audio component in Figure 7 above can be configured to output and / or input audio signals. For example, the audio component includes a microphone (MIC) configured to receive external audio signals when the device containing the audio component is in an operating mode, such as call mode, recording mode, and voice recognition mode. The received audio signals can be further stored in memory or transmitted via a communication component. In some embodiments, the audio component also includes a speaker for outputting audio signals.

[0072] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0073] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in one or more blocks of the flowchart illustrations and / or one or more blocks of the block diagrams.

[0074] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means that implement the functions specified in one or more flowcharts and / or one or more block diagrams.

[0075] These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process, such that the instructions, which execute on the computer or other programmable apparatus, provide steps for implementing the functions specified in one or more flowcharts and / or one or more block diagrams.

[0076] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0077] Memory may include non-persistent storage in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.

[0078] Computer-readable media includes both permanent and non-permanent, removable and non-removable media that can store information using any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other classes of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic magnetic disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.

[0079] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0080] The above description is merely an embodiment of this application and is not intended to limit this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principle of this application should be included within the scope of the claims of this application.

Claims

1. A method for allocating training resources, characterized in that, include: A duration model is constructed based on the training parameter information of the large model training task and the duration influencing factors extracted from the historical training records of the intelligent computing center. Based on the unit time resource usage cost of a single GPU card, the relationship between the training duration evaluated by the duration model and the number of GPU cards, a total resource cost function with the number of GPU cards as the variable is constructed, and the target number of GPU cards corresponding to the minimum value of the total resource cost function is determined; based on the target number of GPU cards, the training resources for the large model training task are configured.

2. The method according to claim 1, characterized in that, The training parameter information includes at least: the number of model parameters, the amount of training data, the effective computing power of a single GPU card, and the token processing metric used to measure the computational efficiency per unit of parameters.

3. The method according to claim 1, characterized in that, The duration influencing factor is learned through a neural network algorithm based on the training duration of historical training tasks in the historical training records and the number of GPU cards.

4. The method according to any one of claims 1 to 3, characterized in that, The duration model is constructed based on the product of training duration, training data volume, and model parameter volume, the number of GPUs deployed, the comprehensive computing performance factor, and the duration impact factor; wherein, the comprehensive computing performance factor is jointly determined by the effective computing power of a single GPU, the number of tokens processed per unit of parameter volume using computing power, and the number of bytes occupied by the token.

5. The training resource allocation method according to claim 1, characterized in that, The total training resource cost is calculated by multiplying the unit time resource usage cost of a single GPU card, the number of GPU cards deployed, and the training duration evaluated by the duration model.

6. The method according to claim 1 or 5, characterized in that, The step of determining the target number of GPUs when the total resource cost function value is minimized includes: taking the derivative of the total resource cost function with respect to the number of GPUs; and finding the number of GPUs that makes the derivative of the total resource cost function with respect to the number of GPUs zero, which is taken as the target number of GPUs.

7. A training resource allocation device, characterized in that, include: The model building module is used to build a duration model based on the training parameter information of the large model training task and the duration influencing factors extracted from the historical training records of the intelligent computing center; the numerical solution module is used to build a total resource cost function with the number of GPUs as the variable based on the unit time resource usage cost of a single GPU card, the relationship between the training duration evaluated by the duration model and the number of GPU cards, and to determine the target number of GPU cards corresponding to the minimum value of the total resource cost function; the resource configuration module is used to configure the training resources of the large model training task based on the target number of GPU cards.

8. An electronic device, characterized in that, The electronic device includes a processor and a memory, the memory storing programs or instructions that can run on the processor, the programs or instructions being executed by the processor to implement the steps of the method as described in any one of claims 1 to 6.

9. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it causes the processor to perform the steps in the method as described in any one of claims 1 to 6.

10. A computer program product, characterized in that, The computer program product stores instructions that, when executed by a computer, cause the computer to perform the steps of the method according to any one of claims 1 to 6.

Images