Distributed training method and device of text language model, computer device and medium

Abstract

The application relates to a distributed training method and device of a text language model, computer equipment and a medium. The method comprises the following steps: respectively deploying a distributed scheduling system and an inference engine on distributed hardware resources; for a training process of a text language model to be trained, the training process is configured as multiple training tasks and corresponding data streams, and multiple services capable of respectively executing different training tasks are configured; wherein the multiple training tasks comprise an inference task, and a training inference service corresponding to the inference task is configured based on the inference engine; based on the multiple training tasks and corresponding data streams, the distributed scheduling system and idle hardware resources, one or more services executing different training tasks are deployed, and the text language model to be trained is trained to generate a target text language model. The method can improve the resource utilization rate in the training process of the text language model.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence technology, and in particular to a distributed training method, apparatus, computer device, and medium for a text language model. Background Technology

[0002] In traditional technologies, language models use different scheduling frameworks for reinforcement learning training and inference services, utilizing isolated hardware resources. However, traditional technologies suffer from low resource utilization, low training efficiency, and high costs. Furthermore, traditional distributed system development often faces challenges such as complex resource scheduling, difficulty in cross-scenario adaptation, and high deployment and maintenance costs. For example, scenarios like reinforcement learning training of text language models, online text dialogue, and online text inference can require hundreds of thousands of nodes and millions of cores of computing resources in a single application, which traditional microservice architectures simply cannot support. Therefore, how to seamlessly extend single-machine programming logic to a distributed environment, allowing model developers to quickly build large-scale applications without needing in-depth knowledge of complex distributed theories, is a significant technical challenge for those skilled in the art. Summary of the Invention

[0003] Therefore, it is necessary to provide a distributed training method, apparatus, computer equipment, and medium for text language models that can combine the training process of text language models with the online inference process to improve the utilization of distributed resources in the training process of text language models, thereby addressing the aforementioned technical problems.

[0004] Firstly, this application provides a distributed training method for a text language model, including:

[0005] Deploy a distributed scheduling system and an inference engine on distributed hardware resources respectively;

[0006] For the training process of the text language model to be trained, the training process is configured into multiple training tasks and corresponding data streams, and multiple services that can execute different training tasks are configured; wherein, the multiple training tasks include inference tasks, and training inference services corresponding to the inference tasks are configured based on the inference engine.

[0007] Based on multiple training tasks and corresponding data streams, a distributed scheduling system, and idle hardware resources, one or more services are deployed to perform different training tasks, train the text language model to be trained, and generate the target text language model.

[0008] In one embodiment, the training process for the text language model to be trained is configured as multiple training tasks and corresponding data streams, and multiple services capable of executing different training tasks are configured, including:

[0009] Reference information for the training process of multiple different text language models is provided, and multiple parameter configuration templates are preset, including multiple training tasks and corresponding data streams, as well as services that execute the multiple training tasks respectively.

[0010] Based on the reference information of the training process of the text language model to be trained, candidate parameter configuration templates that are suitable for the text language model to be trained are determined from the various parameter configuration templates. Based on any operation triggered by the user in the modification operation, configuration operation, or determination operation of the candidate parameter configuration template, the training process is configured into multiple training tasks and corresponding data streams, and multiple services that can execute different training tasks are configured.

[0011] In one embodiment, the inference engine is further configured with an online inference service for online inference requests, and the method further includes:

[0012] A first resource pool is configured for the training inference service and the online inference service, and a second resource pool is configured for services that perform other training tasks besides the training inference service; or,

[0013] A third resource pool is configured for the services performing different training tasks and the online inference service.

[0014] In one embodiment, with the first resource pool and the second resource pool configured, the step of deploying one or more services to perform different training tasks based on multiple training tasks and corresponding data streams, a distributed scheduling system, and idle hardware resources to train the text language model to be trained and generate a target text language model includes:

[0015] The priorities of the inference tasks and online inference requests are determined, and resources corresponding to the inference tasks are allocated from the first resource pool according to the priorities. One or more training inference services that execute the inference tasks are deployed to train the text language model to be trained and generate the target text language model.

[0016] In one embodiment, with the third resource pool configured, the step of deploying one or more services to perform different training tasks based on multiple training tasks and corresponding data streams, a distributed scheduling system, and idle hardware resources to train the text language model to be trained and generate a target text language model includes:

[0017] Idle resources are allocated from the third resource pool for each service performing different training tasks. The idle resources are used to asynchronously and in parallel process the multiple training tasks to train the text language model to be trained and generate the target text language model.

[0018] In one embodiment, before utilizing the idle resources to perform asynchronous parallel processing on the plurality of training tasks, the method further includes any one of the following:

[0019] Determine the processing speed of each service performing different training tasks, and based on the processing speed of each service, the resource requirements of each service, and the amount of idle resources, determine the number of different services to be deployed.

[0020] The system monitors the number of pending requests in the task queue corresponding to each training task in real time, and dynamically adjusts the number of services executing different training tasks based on the number of pending requests in each task queue.

[0021] In one embodiment, the method further includes:

[0022] Real-time monitoring of transmission pressure and hardware resource utilization in the data link;

[0023] The current model training batch size is dynamically set based on the transmission pressure, the utilization rate of hardware resources, the complexity of the training task, and the current training stage.

[0024] Secondly, this application also provides a distributed training device for a text language model, comprising:

[0025] The deployment module is used to deploy the distributed scheduling system and inference engine on distributed hardware resources respectively;

[0026] The first configuration module is used to configure the training process of the text language model to be trained into multiple training tasks and corresponding data streams, and to configure multiple services that can execute different training tasks respectively; wherein, the multiple training tasks include inference tasks, and the training inference service corresponding to the inference task is configured based on the inference engine.

[0027] The training module is used to deploy one or more services that perform different training tasks based on multiple training tasks and corresponding data streams, a distributed scheduling system, and idle hardware resources, to train the text language model to be trained and generate the target text language model.

[0028] Thirdly, this application also provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to perform the following steps:

[0029] Deploy a distributed scheduling system and an inference engine on distributed hardware resources respectively;

[0030] For the training process of the text language model to be trained, the training process is configured into multiple training tasks and corresponding data streams, and multiple services that can execute different training tasks are configured; wherein, the multiple training tasks include inference tasks, and training inference services corresponding to the inference tasks are configured based on the inference engine.

[0031] Based on multiple training tasks and corresponding data streams, a distributed scheduling system, and idle hardware resources, one or more services are deployed to perform different training tasks, train the text language model to be trained, and generate the target text language model.

[0032] Fourthly, this application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, performs the following steps:

[0033] Deploy a distributed scheduling system and an inference engine on distributed hardware resources respectively;

[0034] For the training process of the text language model to be trained, the training process is configured into multiple training tasks and corresponding data streams, and multiple services that can execute different training tasks are configured; wherein, the multiple training tasks include inference tasks, and training inference services corresponding to the inference tasks are configured based on the inference engine.

[0035] Based on multiple training tasks and corresponding data streams, a distributed scheduling system, and idle hardware resources, one or more services are deployed to perform different training tasks, train the text language model to be trained, and generate the target text language model.

[0036] Fifthly, this application also provides a computer program product, including a computer program that, when executed by a processor, performs the following steps:

[0037] Deploy a distributed scheduling system and an inference engine on distributed hardware resources respectively;

[0038] For the training process of the text language model to be trained, the training process is configured into multiple training tasks and corresponding data streams, and multiple services that can execute different training tasks are configured; wherein, the multiple training tasks include inference tasks, and training inference services corresponding to the inference tasks are configured based on the inference engine.

[0039] Based on multiple training tasks and corresponding data streams, a distributed scheduling system, and idle hardware resources, one or more services are deployed to perform different training tasks, train the text language model to be trained, and generate the target text language model.

[0040] The aforementioned distributed training method, apparatus, computer equipment, and medium for text language models, by deploying a distributed scheduling system and an inference engine on distributed hardware resources respectively, can configure the training process of the text language model to be trained into multiple training tasks and corresponding data streams based on the deployed distributed scheduling system. This allows for the deployment of one or more services running on distributed hardware resources, each with independent functions, to execute different training tasks, based on the deployed inference engine, multiple training tasks and corresponding data streams, the distributed scheduling system, and idle hardware resources. Among these training tasks are inference tasks. By using multiple services capable of executing different training tasks independently, it ensures that services for different training tasks can be executed uninterruptedly during the training of the text language model. Simultaneously, the efficient inference capabilities of the inference engine accelerate the training process, optimize the use of distributed hardware resources, improve the utilization rate of hardware resources during the training process of the text language model, and enhance model training efficiency. Attached Figure Description

[0041] To more clearly illustrate the technical solutions in the embodiments of this application or related technologies, the drawings used in the description of the embodiments of this application or related technologies will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0042] Figure 1 This is a flowchart illustrating a distributed training method for a text language model in one embodiment.

[0043] Figure 2 This is a flowchart illustrating a distributed training method for a text language model in another embodiment;

[0044] Figure 3 This is a schematic diagram of resource allocation in one embodiment;

[0045] Figure 4 This is a flowchart illustrating a distributed training method for a text language model in another embodiment;

[0046] Figure 5 This is a block diagram of the structure of a distributed training device for a text language model in one embodiment;

[0047] Figure 6 This is an internal structural diagram of a computer device in one embodiment. Detailed Implementation

[0048] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0049] It should be noted that the terms "comprising" and "having," and any variations thereof, as used in this application, are intended to cover non-exclusive inclusion. The term "multiple" as used in this application refers to two or more. The term "and / or" as used in this application refers to one of the solutions, or any combination of multiple solutions.

[0050] In traditional technologies, text language models utilize different hardware resources for reinforcement learning training and inference services, resulting in low resource utilization. Furthermore, separate optimizations for resource scheduling are required for model training and inference, leading to high system complexity, high development costs, and low training efficiency. The distributed training method for text language models provided in this application combines layered abstraction of full-stack development components, distributed management of a distributed scheduling system, and efficient inference of a high-performance inference engine to construct an efficient, clear, and scalable reinforcement learning training system for text language models. Full-stack development components can be, for example, the Forge framework; the distributed scheduling system can be, for example, Ray; and the model inference engine can be, for example, vLLM.

[0051] Specifically, full-stack development components are used to abstract the complex text language model reinforcement learning training process (including data generation, reward scoring, policy updates, etc.) into different tasks. The distributed scheduling system, as a unified resource management layer, manages services such as the inference service working group, reward scoring service working group, and policy update service working group in the form of distributed executors, providing a resource pool for all services. For example, when generation is not required, the graphics processing unit (GPU) occupied by the inference engine's inference service can be dynamically released to the reward scoring service for model training, and the data flow and dependencies between them can be handled efficiently, improving the overall cluster resource utilization and achieving higher cost-effectiveness. The efficient inference capability of the inference engine is used to specifically execute the training inference tasks in the generation phase of text language model training. By combining the above three elements, and defining task dependencies through the full-stack development component layer, the mid-level scheduler of the full-stack development component treats "generating a batch of answers" as an atomic task. This task is then scheduled to a set of training inference services dedicated to training generation through a distributed scheduling system. These services internally call the inference engine to execute the atomic tasks, thereby decoupling the complex model training process. This reduces the need for separate optimization of the most time-consuming inference generation stage during model training, lowers development costs, and improves the training performance of the text language model.

[0052] By leveraging the inference engine's specialized optimizations for inference generation capabilities, resource utilization is maximized, accelerating the training cycle. The distributed scheduling system's object store serves as a hub for data sharing between tasks. For example, text generated by the inference engine is passed as an object to the reward scoring service (Actor) for scoring, and then to the policy update service (Actor) to update the policy, achieving collaborative optimization. Furthermore, visualization tools can be integrated to visualize the entire data flow during text language model training, facilitating the configuration of training tasks and data streams, the configuration of corresponding services, hyperparameter tuning, and performance profiling. For example, visualization tools such as FlowInsight can be used.

[0053] Through extensive experimental research, the embodiments of this application utilize layered abstraction of full-stack development components to decouple the end-to-end workflow and component set of reinforcement learning tasks in text language model reinforcement learning training. A distributed scheduling system is used for distributed scheduling of data acquisition and model learning / training, and an inference engine is used for high-speed inference sampling. This enables the construction of a high-throughput, low-latency, automatic scaling, and cost-optimized text language model reinforcement learning system, thereby improving training speed, significantly reducing engineering complexity, and increasing resource utilization.

[0054] In one embodiment, such as Figure 1 As shown, a distributed training method for a text language model is provided. This embodiment illustrates the method by applying it to a computer device. It is understood that this method can also be applied to a server, or to a system including both a computer device and a server, and is implemented through the interaction between the computer device and the server. In this embodiment, the method includes the following steps:

[0055] S201 deploys a distributed scheduling system and an inference engine on distributed hardware resources respectively.

[0056] In this context, distributed hardware resources refer to a unified computing cluster formed by connecting multiple independent physical / virtual computing devices, i.e., computing nodes, through a network. These computing nodes include computer devices that execute the distributed training method for the text language model according to embodiments of this application. For example, the distributed scheduling system deployed in this embodiment could be Ray, and the inference engine could be vLLM. The distributed scheduling system is a distributed computing framework that integrates distributed hardware resources into a virtual supercomputer, automatically scheduling tasks during the text language model training process to idle computing nodes. The inference engine is a high-performance inference / service framework designed for text language models, capable of achieving ultra-high throughput and ultra-low latency inference for large language models.

[0057] As an optional implementation, in this embodiment, a unified isolated running environment can be created for all nodes using containerized images (such as Docker images) on distributed hardware resources. Then, a distributed scheduling system (such as Ray) and an inference engine (such as vLLM) can be installed in the containers of each node using package management tools (such as Python package management tools, Pip Installs Packages, pip).

[0058] Furthermore, for large-scale distributed training, cluster management of distributed hardware resources can be achieved by writing cluster startup scripts for a distributed scheduling system. When using an open-source distributed scheduling system, such as Ray, the officially provided cluster startup scripts can be used to manage distributed hardware resources.

[0059] S202, for the training process of the text language model to be trained, the training process is configured into multiple training tasks and corresponding data streams, and multiple services that can execute different training tasks are configured respectively; among them, the multiple training tasks include inference tasks, and training inference services corresponding to the inference tasks are configured based on the inference engine.

[0060] For example, taking the text language model to be trained as a text generation language model, its training process may include acquiring training data, performing inference sampling on each piece of training data, determining feedback values, and updating model parameters.

[0061] As an optional implementation, in this embodiment, the distributed scheduling system such as Ray can be used to break down the complete training process of the text language model to be trained into multiple asynchronous training tasks, executed in a pipeline, and the data source, data format, and data flow path used by each training task can be configured as the corresponding data flow for each training task. Furthermore, the multiple services configured in this embodiment that can execute different training tasks can be understood as independent threads or dedicated services configured for each training task, with services for different training tasks not interfering with each other.

[0062] It is understood that the training process of the text language model to be trained includes an inference process. Therefore, the multiple training tasks configured may include inference tasks. Based on this, as an optional implementation, the inference engine deployed above can also be used to configure training inference services corresponding to the inference tasks to execute the inference tasks in the training process of the text language model to be trained.

[0063] Taking the reinforcement learning training of a text language model to be trained as an example, the multiple training tasks configured in the training process may include: inference task, reward evaluation task, parameter update task, etc. For example, the data flow corresponding to multiple training tasks can be: obtaining a batch of 1000 training data from a specified data source to the inference task, the result of the inference task to the reward evaluation task, the result of the reward evaluation task to the parameter update task, the result of the parameter update to the inference task, and updating the model parameters for executing the inference task.

[0064] Furthermore, the hardware resources and business process parameters (inference rules) configured for services performing different training tasks can vary. For example, each training inference service requires hardware resources of 1 unit of Central Processing Unit (CPU), 8 units of GPU, and 200GB of memory, as well as how to perform inference for each piece of data, such as the batch size for parallel inference, output length, and number of inferences. The reward evaluation service performing the reward evaluation task requires hardware resources of 1 unit of CPU, 0.5 units of GPU, and 8GB of memory, as well as specific reward evaluation methods, such as rule-based reward evaluation for training tasks with correct answers in mathematics, science, and logic, or reward evaluation based on reward model evaluation scores for training tasks without correct answers in literature and art. Different reward evaluation services can be configured for different reward evaluation methods. The parameter update service requires hardware resources of 1 unit of CPU, 8 units of GPU, and 100GB of memory, as well as specific configurations for gradient calculation, gradient pruning, gradient propagation paths, etc.

[0065] S203, based on multiple training tasks and corresponding data streams, a distributed scheduling system and idle hardware resources, deploys one or more services to perform different training tasks, trains the text language model to be trained, and generates the target text language model.

[0066] For example, in this embodiment, a distributed scheduling system can be used to match multiple training tasks and their corresponding data streams with idle resources, deploy one or more services to perform different training tasks, and ensure that multiple training tasks work together on distributed hardware resources to train the text language model to be trained and generate the target text language model.

[0067] As an example, a distributed scheduling system can be used to match multiple training tasks and their corresponding data streams with available resources based on the amount of data in the available resources. When available resources are less than a preset threshold (i.e., when resources are scarce), the distributed scheduling system can deploy services one by one according to the data streams of multiple training tasks, executing the corresponding training tasks serially with each deployed service. When available resources are not less than the preset threshold (i.e., when resources are plentiful), the distributed scheduling system can follow the principle of "one service exclusively owns a set of resources, without sharing or preemption," allocating corresponding resources to each training task separately, deploying services to execute each training task, and using the deployed service pipeline to execute the corresponding tasks asynchronously and in parallel.

[0068] Additionally, it should be noted that configuring multiple services capable of performing different training tasks refers to defining the attributes and rules of services for different training tasks. The objects of operation are configuration files / parameter tables. Deploying one or more services that perform different training tasks refers to applying the configured services to hardware resources and running the configured services. The objects of operation are specific entities such as hardware resources / containers / processes. Configuring services is the prerequisite and basis for deploying services.

[0069] In the aforementioned distributed training method for text language models, by deploying a distributed scheduling system and an inference engine on distributed hardware resources, the training process of the text language model to be trained can be configured into multiple training tasks and corresponding data streams based on the deployed distributed scheduling system. This allows for the deployment of one or more services running on distributed hardware resources, each with independent functions, to execute different training tasks, based on the deployed inference engine, multiple training tasks and corresponding data streams, the distributed scheduling system, and idle hardware resources. Among these training tasks are inference tasks. By using multiple services capable of executing different training tasks independently, it ensures that different training tasks can be executed uninterruptedly during the training of the text language model. Simultaneously, the efficient inference capabilities of the inference engine accelerate the training process, optimize the use of distributed hardware resources, improve the utilization rate of hardware resources during the training process of the text language model, and enhance model training efficiency.

[0070] In some scenarios, the training process of a text language model can be configured into multiple training tasks and corresponding data streams based on preset parameter configuration templates, and multiple services capable of executing different training tasks can be configured. In an exemplary embodiment, such as... Figure 2 As shown, the above S202 includes:

[0071] S301 provides reference information for the training process of multiple different text language models, and pre-sets various parameter configuration templates including multiple training tasks and corresponding data streams, as well as services that execute multiple training tasks respectively.

[0072] For example, the reference information for the training process of a text language model may include modal information, structural information, training stage information, and training task type information of the text language model. The reference information for the training process of different text language models may be different. As an example, different modalities include text models, speech models, music models, multimodal models, video models, etc.; different structures include 30 layers with 200B parameters or 9 layers with 30B parameters; different training stages include pre-training stage, supervised fine-tuning stage, and reinforcement learning training stage. More specifically, for the reinforcement learning training stage, it can be the initial stage of reinforcement learning training from step 1 to 5000, the middle stage of reinforcement learning training from step 5001 to 10000, and the later stage of reinforcement learning training after step 10001, etc.; different training task types include mathematical tasks, scientific tasks, logical reasoning tasks, literary reasoning tasks, etc. It can be understood that different task types will correspond to different training tasks, and correspondingly, different services.

[0073] In this embodiment, multiple parameter configuration templates can be preset for reference information in the training processes of various text language models, establishing a mapping relationship between different reference information and corresponding parameter configuration templates. Furthermore, each parameter configuration template can include multiple training tasks and corresponding data streams, as well as multiple services that execute different training tasks. The information included in the parameter configuration templates has a certain degree of universality. In practical applications, the information included in the parameter configuration templates can be adjusted to match the actual reference information in the training process of the text language models. For example, the batch size of the inference task can be appropriately reduced from 1000 training data points per batch to 800 training data points per batch; or the threshold for the reward evaluation task can be appropriately changed, increasing the reward from scores exceeding 80 to scores exceeding 90.

[0074] S302, based on the reference information of the training process of the text language model to be trained, determine the candidate parameter configuration template that is suitable for the text language model to be trained from a variety of parameter configuration templates, and based on any operation triggered by the user in the modification operation, configuration operation, and determination operation of the candidate parameter configuration template, configure the training process into multiple training tasks and corresponding data streams, and configure multiple services that can execute different training tasks respectively.

[0075] In this embodiment, a candidate parameter configuration template suitable for the text language model to be trained can be determined from a variety of preset parameter configuration templates based on reference information from the training process of the text language model to be trained. Furthermore, the determined candidate parameter configuration templates can be visualized. Based on any of the user's modification, configuration, or confirmation operations triggered on the candidate parameter configuration templates, the information included in the candidate parameter configuration templates can be modified and / or confirmed. For example, at least one piece of information can be modified regarding the multiple training tasks and corresponding data streams included in the candidate parameter configuration templates, as well as the services that execute the multiple training tasks respectively. For instance, the batch size of the inference task can be appropriately reduced, from 1000 training data points per batch to 800 training data points per batch; or the threshold for the reward evaluation task can be appropriately changed, from rewarding scores above 80 to rewarding scores above 90, etc.

[0076] Then, based on the modified parameter configuration template, the training process of the text language model to be trained can be configured into multiple training tasks and corresponding data streams, and services capable of executing each training task can be configured for each task. The modified parameter configuration template can also be saved as a preset parameter configuration template for future selection, thereby saving configuration time for training tasks, data streams, and corresponding services, and improving training efficiency.

[0077] In this embodiment, for reference information regarding the training processes of multiple different text language models, several parameter configuration templates are preset, each including multiple training tasks and corresponding data streams, as well as services that execute these training tasks. Based on the reference information of the training process of the text language model to be trained, a candidate parameter configuration template suitable for the model can be quickly determined from these templates. Then, based on any of the modification, configuration, or confirmation operations triggered by the user on the candidate parameter configuration templates, the training process of the text language model to be trained can be configured as multiple training tasks and corresponding data streams, while simultaneously configuring multiple services capable of executing different training tasks. This improves the efficiency of configuring the training process of the text language model to be trained as multiple training tasks, corresponding data streams, and corresponding services, thereby increasing training efficiency. Furthermore, by using any of the modification, configuration, or confirmation operations triggered on the candidate parameter configuration templates, the accuracy of the configuration is ensured.

[0078] This embodiment explains the resource configuration during the training process of the text language model to be trained. In an exemplary embodiment, based on the above-mentioned inference engine, an online inference service is also configured for online inference requests. The method further includes: configuring a first resource pool for the training inference service and the online inference service, and configuring a second resource pool for services that perform other training tasks besides the training inference service; or, configuring a third resource pool for services that perform different training tasks and the online inference service.

[0079] For example, in this embodiment, an online inference request refers to a real-time inference call initiated by an external or internal module to another text language model that has already been trained and deployed, while the text language model to be trained is still undergoing continuous training (weights are not fixed, training is not yet complete). The online inference request and the training process of the text language model are carried out in parallel. As an optional implementation, in this scenario, an online inference service can also be configured for the online inference request based on the above-mentioned inference engine, so as to process the online inference request through the online inference service.

[0080] As an optional implementation, one resource allocation method in this embodiment can be: such as Figure 3 As shown, a first resource pool is configured for the training inference service and the online inference service, while a second resource pool is configured for services performing other training tasks besides the training inference service. Different hardware resources are allocated to each resource pool, and the resource pools are dynamically expanded or reduced as needed. Both the first and second resource pools can include hardware resources such as CPU, GPU, and memory. In this scenario, as an example, the first resource pool can be configured with multiple inference workers, and the second resource pool can be configured with multiple training workers. Each worker has a fixed resource configuration corresponding to its task. For example, an inference worker, based on the number of model parameters of the online inference model to be deployed or the training inference model, can be configured with 1 unit of CPU, 8 units of GPU, and 200GB of memory. A training worker, based on the number of model parameters of the text language model to be trained, can be configured with 2 units of CPU, 4 units of GPU, and 100GB of memory. This ensures that the inference service and the training service do not interfere with each other and operate efficiently.

[0081] As an alternative implementation, another resource allocation method in this embodiment can be: configuring a third resource pool for services performing different training tasks and online inference services. That is, in this scenario, training tasks and online inference services share a single resource pool. In other words, training and inference are treated as different workgroups, using the same hardware resources in the same resource pool for time-sharing reuse. This reduces the barriers caused by resource grouping and resource idleness due to untimely dynamic scaling, thereby improving resource utilization.

[0082] The following will explain and illustrate the application examples after configuring the resource pool:

[0083] Example 1: Regardless of the resource allocation method described above, when performing one or more training tasks in the text language model training process, the training tasks can be executed serially or asynchronously in parallel.

[0084] Example 2: With the aforementioned first and second resource pools configured, as an optional implementation, during the training of the text language model to be trained, if the inference engine receives a training inference task but the first resource pool has no idle resources, then on the one hand, the number of resources in the first resource pool can be dynamically expanded; on the other hand, the priorities of the training inference tasks and online inference requests during the text language model training process can be determined. Based on the priority ranking of the training inference tasks and online inference requests, corresponding resources can be allocated from the first resource pool for the training inference tasks, and one or more training inference services can be deployed to execute the training inference tasks, thereby training the text language model to be trained and generating the target text language model. Additionally, based on the priority ranking of the training inference tasks and online inference requests, corresponding resources can also be allocated from the first resource pool for the online inference requests to process them.

[0085] As an optional implementation, services executing low-priority online inference requests can be evicted to release resources, and training inference services corresponding to higher-priority training inference tasks can be redeployed. For example, when a high-priority online inference request is received, the inference worker can be configured as an inference model parameter to prioritize processing high-priority online inference requests. Simultaneously, to improve resource utilization, currently received low-priority online inference requests can be inferred together with high-priority online inference request data. For instance, if an inference batch size is 3 and there are 2 high-priority online inference requests, to maximize resource utilization, one low-priority online inference request and two high-priority online inference requests can be included in the same inference batch for inference output. If a high-priority inference request is received again during the inference process, the inference for that high-priority request continues until inference is complete and no more high-priority inference requests are received. At this point, computing resources can be released into the first resource pool, and idle resources in the first resource pool can be used to load training model parameters and execute the training inference task. It can be seen that the above scheme involves executing multiple online inference requests for the inference model before executing one training inference task. This configuration aligns with the speed matching relationship between different stages and links in model training; that is, the model inference speed is much faster than the transmission speed of training data in the data stream and the speed of executing other training tasks. Therefore, configuring the training inference service intermittently to execute model training inference tasks allows sufficient time to transmit training data and train model parameters, avoiding long waiting times for computing resources to acquire training data, thus improving resource utilization. Furthermore, when there are multiple inference workers, some inference workers can be configured to process online inference request data for inference model parameters, while others are configured to execute training inference tasks for training model parameters, thereby ensuring the effectiveness of online inference request processing.

[0086] Furthermore, in this embodiment, after obtaining the training inference results, the inference results can be fed back to the training workgroup. The resources occupied by each inference worker can be released back to the resource pool. In this way, feeding back the inference results to the training workgroup consumes network bandwidth, not computing power. At this time, the computing power resources in the resource pool are called in parallel to configure the inference model parameters, perform batch inference on a batch of online inference request data that has been acquired, and return the online inference results. This transforms the existing technology of occupying fixed computing power resources to infer a small number of online inference requests, resulting in low resource utilization during inference, into time-sharing multiplexing of limited computing power resources and increasing the amount of data in the inference batch, further improving the utilization of inference resources. This is because when the amount of data in each batch is small, the model's inference speed is very fast, and a lot of time is spent acquiring data, resulting in low resource utilization. For example, if the inference worker executing the online inference service is set to infer once per second, and the network bandwidth acquires about 100 online inference request data per second, it takes about 0.1 seconds for the inference worker to infer 100 data. In this case, about 90% of the computing resources are waiting for inference data, and the resource utilization is only 10%. If configuring both training and inference model parameters takes 0.5 seconds using the above method, and a training batch includes 1000 training data points, the inference worker executing the training inference task takes 8 seconds to process 1000 training inference tasks. Adding the 1 second spent configuring different model parameters twice, the online inference request queue receives 9 seconds of online inference request data, totaling approximately 900 requests. The inference worker executing the online inference service takes approximately 0.9 seconds to execute these 900 online inference requests. In this case, the utilization rate of training resources is approximately (8+0.9) / (8+0.9+0.5+0.5)≈90%. Therefore, by time-sharing the computing resources between the training inference service executing training inference tasks and the online inference service executing online inference requests, the utilization rate of computing resources can be significantly improved.

[0087] Furthermore, during the configuration of inference model parameters and the inference process of online inference requests, the training inference results of the current training batch can be transmitted synchronously, and the next batch of training data can be received. The training worker can also calculate the feedback value synchronously, update the training model parameters based on the calculated feedback value, and transmit the updated training model parameters to the inference worker performing the training inference task. This allows the inference worker performing the training inference task to perform training inference on the acquired training data based on the latest training model parameters and obtain the training inference results when training the next batch of models. This setup further improves data parallelism and comprehensively enhances resource utilization.

[0088] Example 3: With the above-mentioned third resource pool configured, as an optional implementation, idle resources can be allocated from the third resource pool for each service performing different training tasks. The configured idle resources are used to asynchronously and in parallel process multiple training tasks to train the text language model to be trained and generate the target text language model.

[0089] Understandably, by configuring multiple services that perform different training tasks using idle resources, pipelined asynchronous parallel processing of different training tasks is achieved, which can significantly shorten the model training time and improve training efficiency.

[0090] In this embodiment, before utilizing idle resources to asynchronously and parallelly process multiple training tasks, the deployment quantity of different services can be determined through any of the following embodiments:

[0091] 1) Determine the processing speed of each service performing different training tasks, and based on the processing speed of each service, the resource requirements of each service, and the amount of idle resources, determine the number of different services to be deployed.

[0092] In this embodiment, the processing speeds of the model inference service, reward evaluation service, and parameter update service for training the text language model to be trained, as well as the resource requirements of each service, can be determined first. Then, based on the amount of idle resources, combined with the processing speed and resource requirements of each service, the number of instances (deployment quantity) of different services can be determined. For example, the number of each service can be used as a variable to form an equation by matching resource requirements with idle resources, and an equation can be formed by matching the processing speed of each service. This forms a set of equations, and the deployment quantity of different services can be determined by solving the set of equations.

[0093] 2) Monitor the number of pending requests in the task queue corresponding to each training task in real time, and dynamically adjust the number of services executing different training tasks based on the number of pending requests in each task queue.

[0094] In this embodiment, for example, during the initial training phase where multiple training tasks are asynchronously and parallelly processed using configured idle resources, the inference generation task queue contains a large amount of pending data, while the subsequent reward evaluation task queue and parameter update task queue have few or no pending requests. In this case, the resources originally allocated for reward evaluation and parameter update services can be dynamically allocated to the training inference service, thereby dynamically increasing the number of a particular service and improving parallel processing capabilities. Furthermore, the automatic scaling function of the distributed scheduling system can be utilized to dynamically adjust the number of online inference workers and training inference workers in the inference engine based on the task queue depth. During peak generation periods, the number of online inference workers in the inference engine can be increased, and during peak training periods, the number of training inference workers can be increased. This adaptive load change optimization can improve the resource cost of text language model training while maintaining speed.

[0095] In this embodiment, for online inference requests received during the training process of the text language model to be trained, an online inference service can be configured based on the inference engine for the online inference request. On this basis, a first resource pool can be configured for the training inference service and the online inference service, and a second resource pool can be configured for services that perform other training tasks besides the training inference service. Alternatively, a third resource pool can be configured for services that perform different training tasks and the online inference service. Through any of the above resource configuration methods, it is ensured that one or more services that perform different training tasks can make full use of distributed hardware resources as much as possible, thereby improving the utilization rate of distributed hardware resources by the text language model during the training process.

[0096] During the training of the text language model, the batch size can be dynamically set. In an exemplary embodiment, such as... Figure 4 As shown, the above method also includes:

[0097] S401 monitors the transmission pressure and hardware resource utilization in the data link in real time.

[0098] S402 dynamically sets the current model training batch size based on transmission pressure, hardware resource utilization, training task complexity, and the current training stage.

[0099] Understandably, the training process of a text language model can include an early exploration phase, a mid-term stable learning phase, and a late-term fine-tuning and convergence phase. The early exploration phase primarily involves extensively exploring the policy space to avoid premature convergence, resulting in poor initial policies, low data quality, and high noise. In the mid-term stable learning phase, the policy has found a relatively optimal region, requiring efficient and stable learning. In the late-term fine-tuning and convergence phase, the policy is close to its optimal state, requiring fine-tuning and stable convergence to avoid oscillations. Smaller batch sizes during text language model training, such as 100 data points per batch, result in a larger model exploration range, more unstable training, more frequent model parameter updates, larger data transfer volumes, and lower GPU utilization (a significant amount of time is spent acquiring data, reducing the proportion of computation time). Conversely, larger batch sizes have the opposite effect.

[0100] Furthermore, regarding transmission pressure in the data link, for example, if the current bandwidth has low queries per second (QPS) and bits per second (BPS), it indicates a low volume of transmission requests and a small amount of data transmitted per request. In this case, the batch size can be reduced to make fuller use of bandwidth resources to transmit more data and parameters, thereby enhancing the model's exploration capabilities. Regarding hardware resource utilization in the data link, for example, if current hardware resources are limited, the batch size can be appropriately increased to make better use of every unit of computing power and improve computing resource utilization; the more complex the training task, the larger the batch size can be to ensure stable training.

[0101] Therefore, for different stages of the training process of a text language model, the transmission pressure in the data link and the utilization rate of hardware resources can be monitored in real time. Based on the transmission pressure in the data link, the utilization rate of hardware resources, the complexity of the training task in the current training stage, and the current training stage, different training batch sizes can be used to optimize and adjust the training batch size for different training stages of the text language model.

[0102] In this embodiment, by monitoring the transmission pressure and hardware resource utilization in the data link in real time, the current model training batch size can be dynamically set according to the transmission pressure in the data link, hardware resource utilization, the complexity of the training task in the current training stage of the text language model, and the current training stage. Dynamically setting the model training batch size maximizes the use of hardware resources and improves the resource utilization rate in the text language model training process.

[0103] It should be understood that although the steps in the flowcharts of the above embodiments are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the above embodiments may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages in other steps. It is understood that the steps in different embodiments can be freely combined as needed, and all non-contradictory solutions formed by such combinations are within the scope of protection of this application.

[0104] Based on the same inventive concept, this application also provides a distributed training apparatus for a text language model to implement the distributed training method for the text language model described above. The solution provided by this apparatus is similar to the implementation scheme described in the above method. Therefore, the specific limitations of one or more embodiments of the distributed training apparatus for a text language model provided below can be found in the limitations of the distributed training method for the text language model described above, and will not be repeated here.

[0105] In one exemplary embodiment, such as Figure 5 As shown, a distributed training device for a text language model is provided, comprising: a deployment module 10, a first configuration module 11, and a training module 12, wherein:

[0106] Deployment module 10 is used to deploy a distributed scheduling system and an inference engine on distributed hardware resources respectively.

[0107] The first configuration module 11 is used to configure the training process of the text language model to be trained into multiple training tasks and corresponding data streams, and to configure multiple services that can execute different training tasks respectively; among the multiple training tasks, there are inference tasks, and the training inference service corresponding to the inference task is configured based on the inference engine.

[0108] Training module 12 is used to deploy one or more services that perform different training tasks based on multiple training tasks and corresponding data streams, distributed scheduling systems and idle hardware resources, to train the text language model to be trained and generate the target text language model.

[0109] The distributed training device for the text language model provided in this embodiment can execute the above method embodiment, and its implementation principle and technical effect are similar, so it will not be described again here.

[0110] Based on the above embodiments, optionally, the first configuration module 11 includes: a processing unit and a configuration unit, wherein:

[0111] The processing unit is used to provide reference information for the training process of multiple different text language models, and to pre-set various parameter configuration templates that include multiple training tasks and corresponding data streams, as well as services that execute multiple training tasks respectively.

[0112] The configuration unit is used to determine candidate parameter configuration templates that are suitable for the text language model to be trained from a variety of parameter configuration templates based on the reference information of the training process of the text language model to be trained. Based on any operation triggered by the user in the modification operation, configuration operation, or determination operation of the candidate parameter configuration template, the training process is configured into multiple training tasks and corresponding data streams, and multiple services that can execute different training tasks are configured.

[0113] The distributed training device for the text language model provided in this embodiment can execute the above method embodiment, and its implementation principle and technical effect are similar, so it will not be described again here.

[0114] Based on the above embodiments, optionally, the above-mentioned inference engine is further configured with an online inference service for online inference requests, and the above-mentioned apparatus further includes: a second configuration module, or a third configuration module, wherein:

[0115] The second configuration module is used to configure the first resource pool for the training inference service and the online inference service, and to configure the second resource pool for services that perform other training tasks besides the training inference service.

[0116] The third configuration module is used to configure a third resource pool for services that perform different training tasks and online inference services.

[0117] The distributed training device for the text language model provided in this embodiment can execute the above method embodiment, and its implementation principle and technical effect are similar, so it will not be described again here.

[0118] Based on the above embodiments, optionally, when a first resource pool and a second resource pool are configured, the training module 12 includes: a first training unit, wherein:

[0119] The first training unit is used to determine the priority of inference tasks and online inference requests, sort them according to priority, allocate corresponding resources for inference tasks from the first resource pool, deploy one or more training inference services to perform inference tasks, train the text language model to be trained, and generate the target text language model.

[0120] The distributed training device for the text language model provided in this embodiment can execute the above method embodiment, and its implementation principle and technical effect are similar, so it will not be described again here.

[0121] Based on the above embodiments, optionally, when a third resource pool is configured, the training module 12 includes: a second training unit, wherein:

[0122] The second training unit is used to allocate idle resources from the third resource pool for each service performing different training tasks, and to use the idle resources to perform asynchronous parallel processing on multiple training tasks, train the text language model to be trained, and generate the target text language model.

[0123] The distributed training device for the text language model provided in this embodiment can execute the above method embodiment, and its implementation principle and technical effect are similar, so it will not be described again here.

[0124] Based on the above embodiments, optionally, the second training unit is further configured to perform any one of the following:

[0125] Determine the processing speed of each service performing different training tasks, and based on the processing speed of each service, the resource requirements of each service, and the amount of idle resources, determine the number of different services to deploy.

[0126] The system monitors the number of pending requests in the task queue corresponding to each training task in real time, and dynamically adjusts the number of services executing different training tasks based on the number of pending requests in each task queue.

[0127] The distributed training device for the text language model provided in this embodiment can execute the above method embodiment, and its implementation principle and technical effect are similar, so it will not be described again here.

[0128] Based on the above embodiments, optionally, the device further includes: a monitoring module and a setting module, wherein:

[0129] The monitoring module is used to monitor the transmission pressure and hardware resource utilization in the data link in real time.

[0130] The settings module is used to dynamically set the current model training batch size based on transmission pressure, hardware resource utilization, training task complexity, and the current training stage.

[0131] The distributed training device for the text language model provided in this embodiment can execute the above method embodiment, and its implementation principle and technical effect are similar, so it will not be described again here.

[0132] The modules in the distributed training device for the aforementioned text language model can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device, or stored in the memory of a computer device in software form, so that the processor can call and execute the operations corresponding to each module.

[0133] In one exemplary embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be as follows: Figure 6 As shown, this computer device includes a processor, memory, input / output (I / O) interfaces, and a communication interface. The processor, memory, and I / O interfaces are connected via a system bus, and the communication interface is also connected to the system bus via the I / O interfaces. The processor provides computational and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and databases. The internal memory provides the environment for the operating system and computer programs stored in the non-volatile storage media. The I / O interfaces are used for exchanging information between the processor and external devices. The communication interface is used for communicating with external terminals via a network connection. When the computer program is executed by the processor, it implements a distributed training method for a text language model.

[0134] Those skilled in the art will understand that Figure 6 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0135] In one exemplary embodiment, a computer device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to perform the following steps:

[0136] Deploy a distributed scheduling system and an inference engine on distributed hardware resources respectively;

[0137] For the training process of the text language model to be trained, the training process is configured into multiple training tasks and corresponding data streams, and multiple services that can execute different training tasks are configured. Among them, the multiple training tasks include inference tasks, and training inference services corresponding to the inference tasks are configured based on the inference engine.

[0138] Based on multiple training tasks and corresponding data streams, a distributed scheduling system, and idle hardware resources, one or more services are deployed to perform different training tasks, train the text language model to be trained, and generate the target text language model.

[0139] In one embodiment, the processor, when executing a computer program, also performs the following steps:

[0140] Reference information for the training process of multiple different text language models is provided, and various parameter configuration templates are preset, including multiple training tasks and corresponding data streams, as well as services that execute multiple training tasks respectively.

[0141] Based on the reference information of the training process of the text language model to be trained, candidate parameter configuration templates suitable for the text language model to be trained are determined from a variety of parameter configuration templates. Based on any operation triggered by the user in the modification operation, configuration operation, and determination operation of the candidate parameter configuration template, the training process is configured into multiple training tasks and corresponding data streams, and multiple services that can execute different training tasks are configured.

[0142] In one embodiment, an online inference service for online inference requests is also configured based on the inference engine, and the processor further performs the following steps when executing the computer program:

[0143] Configure a first resource pool for the training inference service and the online inference service, and configure a second resource pool for services that perform other training tasks besides the training inference service; or,

[0144] Configure a third resource pool for services that perform different training tasks and online inference services.

[0145] In one embodiment, when a first resource pool and a second resource pool are configured, the processor, when executing a computer program, further performs the following steps:

[0146] Determine the priority of inference tasks and online inference requests, sort them according to priority, allocate corresponding resources for inference tasks from the first resource pool, deploy one or more training inference services to execute inference tasks, train the text language model to be trained, and generate the target text language model.

[0147] In one embodiment, when a third resource pool is configured, the processor also performs the following steps when executing a computer program:

[0148] Idle resources are allocated from the third resource pool to each service performing different training tasks. The idle resources are used to asynchronously and in parallel process multiple training tasks to train the text language model to be trained and generate the target text language model.

[0149] In one embodiment, the processor, when executing a computer program, also performs the following steps:

[0150] Determine the processing speed of each service performing different training tasks, and based on the processing speed of each service, the resource requirements of each service, and the amount of idle resources, determine the number of different services to deploy.

[0151] or,

[0152] The system monitors the number of pending requests in the task queue corresponding to each training task in real time, and dynamically adjusts the number of services executing different training tasks based on the number of pending requests in each task queue.

[0153] In one embodiment, the processor, when executing a computer program, also performs the following steps:

[0154] Real-time monitoring of transmission pressure and hardware resource utilization in the data link;

[0155] The current model training batch size is dynamically set based on transmission pressure, hardware resource utilization, training task complexity, and the current training stage.

[0156] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon, the computer program performing the following steps when executed by a processor:

[0157] Deploy a distributed scheduling system and an inference engine on distributed hardware resources respectively;

[0158] For the training process of the text language model to be trained, the training process is configured into multiple training tasks and corresponding data streams, and multiple services that can execute different training tasks are configured. Among them, the multiple training tasks include inference tasks, and training inference services corresponding to the inference tasks are configured based on the inference engine.

[0159] Based on multiple training tasks and corresponding data streams, a distributed scheduling system, and idle hardware resources, one or more services are deployed to perform different training tasks, train the text language model to be trained, and generate the target text language model.

[0160] In one embodiment, when the computer program is executed by a processor, it also performs the following steps:

[0161] Reference information for the training process of multiple different text language models is provided, and various parameter configuration templates are preset, including multiple training tasks and corresponding data streams, as well as services that execute multiple training tasks respectively.

[0162] Based on the reference information of the training process of the text language model to be trained, candidate parameter configuration templates suitable for the text language model to be trained are determined from a variety of parameter configuration templates. Based on any operation triggered by the user in the modification operation, configuration operation, and determination operation of the candidate parameter configuration template, the training process is configured into multiple training tasks and corresponding data streams, and multiple services that can execute different training tasks are configured.

[0163] In one embodiment, an online inference service for online inference requests is also configured based on the inference engine, and the computer program, when executed by the processor, further implements the following steps:

[0164] Configure a first resource pool for the training inference service and the online inference service, and configure a second resource pool for services that perform other training tasks besides the training inference service; or,

[0165] Configure a third resource pool for services that perform different training tasks and online inference services.

[0166] In one embodiment, when a first resource pool and a second resource pool are configured, the computer program, when executed by the processor, further performs the following steps:

[0167] Determine the priority of inference tasks and online inference requests, sort them according to priority, allocate corresponding resources for inference tasks from the first resource pool, deploy one or more training inference services to execute inference tasks, train the text language model to be trained, and generate the target text language model.

[0168] In one embodiment, when a third resource pool is configured, the computer program, when executed by the processor, also performs the following steps:

[0169] Idle resources are allocated from the third resource pool to each service performing different training tasks. The idle resources are used to asynchronously and in parallel process multiple training tasks to train the text language model to be trained and generate the target text language model.

[0170] In one embodiment, when the computer program is executed by a processor, it also performs the following steps:

[0171] Determine the processing speed of each service performing different training tasks, and based on the processing speed of each service, the resource requirements of each service, and the amount of idle resources, determine the number of different services to deploy.

[0172] or,

[0173] The system monitors the number of pending requests in the task queue corresponding to each training task in real time, and dynamically adjusts the number of services executing different training tasks based on the number of pending requests in each task queue.

[0174] In one embodiment, when the computer program is executed by a processor, it also performs the following steps:

[0175] Real-time monitoring of transmission pressure and hardware resource utilization in the data link;

[0176] The current model training batch size is dynamically set based on transmission pressure, hardware resource utilization, training task complexity, and the current training stage.

[0177] In one embodiment, a computer program product is provided, including a computer program that, when executed by a processor, performs the following steps:

[0178] Deploy a distributed scheduling system and an inference engine on distributed hardware resources respectively;

[0179] For the training process of the text language model to be trained, the training process is configured into multiple training tasks and corresponding data streams, and multiple services that can execute different training tasks are configured. Among them, the multiple training tasks include inference tasks, and training inference services corresponding to the inference tasks are configured based on the inference engine.

[0180] Based on multiple training tasks and corresponding data streams, a distributed scheduling system, and idle hardware resources, one or more services are deployed to perform different training tasks, train the text language model to be trained, and generate the target text language model.

[0181] In one embodiment, when the computer program is executed by a processor, it also performs the following steps:

[0182] Reference information for the training process of multiple different text language models is provided, and various parameter configuration templates are preset, including multiple training tasks and corresponding data streams, as well as services that execute multiple training tasks respectively.

[0183] Based on the reference information of the training process of the text language model to be trained, candidate parameter configuration templates suitable for the text language model to be trained are determined from a variety of parameter configuration templates. Based on any operation triggered by the user in the modification operation, configuration operation, and determination operation of the candidate parameter configuration template, the training process is configured into multiple training tasks and corresponding data streams, and multiple services that can execute different training tasks are configured.

[0184] In one embodiment, an online inference service for online inference requests is also configured based on the inference engine, and the computer program, when executed by the processor, further implements the following steps:

[0185] Configure a first resource pool for the training inference service and the online inference service, and configure a second resource pool for services that perform other training tasks besides the training inference service; or,

[0186] Configure a third resource pool for services that perform different training tasks and online inference services.

[0187] In one embodiment, when a first resource pool and a second resource pool are configured, the computer program, when executed by the processor, further performs the following steps:

[0188] Determine the priority of inference tasks and online inference requests, sort them according to priority, allocate corresponding resources for inference tasks from the first resource pool, deploy one or more training inference services to execute inference tasks, train the text language model to be trained, and generate the target text language model.

[0189] In one embodiment, when a third resource pool is configured, the computer program, when executed by the processor, also performs the following steps:

[0190] Idle resources are allocated from the third resource pool to each service performing different training tasks. The idle resources are used to asynchronously and in parallel process multiple training tasks to train the text language model to be trained and generate the target text language model.

[0191] In one embodiment, when the computer program is executed by a processor, it also performs the following steps:

[0192] Determine the processing speed of each service performing different training tasks, and based on the processing speed of each service, the resource requirements of each service, and the amount of idle resources, determine the number of different services to deploy.

[0193] or,

[0194] The system monitors the number of pending requests in the task queue corresponding to each training task in real time, and dynamically adjusts the number of services executing different training tasks based on the number of pending requests in each task queue.

[0195] In one embodiment, when the computer program is executed by a processor, it also performs the following steps:

[0196] Real-time monitoring of transmission pressure and hardware resource utilization in the data link;

[0197] The current model training batch size is dynamically set based on transmission pressure, hardware resource utilization, training task complexity, and the current training stage.

[0198] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, artificial intelligence (AI) processors, etc., and are not limited to these.

[0199] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.

[0200] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A distributed training method for a text language model, characterized in that, The method includes: Deploy a distributed scheduling system and an inference engine on distributed hardware resources respectively; For the training process of the text language model to be trained, the training process is configured into multiple training tasks and corresponding data streams, and multiple services that can execute different training tasks are configured; wherein, the multiple training tasks include inference tasks, and training inference services corresponding to the inference tasks are configured based on the inference engine. Based on multiple training tasks and corresponding data streams, a distributed scheduling system, and idle hardware resources, one or more services are deployed to perform different training tasks, train the text language model to be trained, and generate the target text language model.

2. The method according to claim 1, characterized in that, The training process for the text language model to be trained is configured as multiple training tasks and corresponding data streams, and multiple services capable of executing different training tasks are configured, including: Reference information for the training process of multiple different text language models is provided, and multiple parameter configuration templates are preset, including multiple training tasks and corresponding data streams, as well as services that execute the multiple training tasks respectively. Based on the reference information of the training process of the text language model to be trained, candidate parameter configuration templates that are suitable for the text language model to be trained are determined from the various parameter configuration templates. Based on any operation triggered by the user in the modification operation, configuration operation, or determination operation of the candidate parameter configuration template, the training process is configured into multiple training tasks and corresponding data streams, and multiple services that can execute different training tasks are configured.

3. The method according to claim 1, characterized in that, The method further includes: an online inference service configured for online inference requests based on the inference engine; A first resource pool is configured for the training inference service and the online inference service, and a second resource pool is configured for services that perform other training tasks besides the training inference service; or, A third resource pool is configured for the services performing different training tasks and the online inference service.

4. The method according to claim 3, characterized in that, With the first resource pool and the second resource pool configured, the step of deploying one or more services to execute different training tasks based on multiple training tasks and corresponding data streams, a distributed scheduling system, and idle hardware resources to train the text language model to be trained and generate the target text language model includes: The priorities of the inference tasks and online inference requests are determined, and resources corresponding to the inference tasks are allocated from the first resource pool according to the priorities. One or more training inference services that execute the inference tasks are deployed to train the text language model to be trained and generate the target text language model.

5. The method according to claim 3, characterized in that, With the third resource pool configured, the process of deploying one or more services to execute different training tasks based on multiple training tasks and corresponding data streams, a distributed scheduling system, and idle hardware resources to train the text language model to be trained and generate the target text language model includes: Idle resources are allocated from the third resource pool for each service performing different training tasks. The idle resources are used to asynchronously and in parallel process the multiple training tasks to train the text language model to be trained and generate the target text language model.

6. The method according to claim 5, characterized in that, Before utilizing the idle resources to perform asynchronous parallel processing on the multiple training tasks, the method further includes any one of the following: Determine the processing speed of each service performing different training tasks, and based on the processing speed of each service, the resource requirements of each service, and the amount of idle resources, determine the number of different services to be deployed. The system monitors the number of pending requests in the task queue corresponding to each training task in real time, and dynamically adjusts the number of services executing different training tasks based on the number of pending requests in each task queue.

7. The method according to claim 1, characterized in that, The method further includes: Real-time monitoring of transmission pressure and hardware resource utilization in the data link; The current model training batch size is dynamically set based on the transmission pressure, the utilization rate of hardware resources, the complexity of the training task, and the current training stage.

8. A distributed training device for a text language model, characterized in that, The device includes: The deployment module is used to deploy the distributed scheduling system and inference engine on distributed hardware resources respectively; The first configuration module is used to configure the training process of the text language model to be trained into multiple training tasks and corresponding data streams, and to configure multiple services that can execute different training tasks respectively; wherein, the multiple training tasks include inference tasks, and the training inference service corresponding to the inference task is configured based on the inference engine. The training module is used to deploy one or more services that perform different training tasks based on multiple training tasks and corresponding data streams, a distributed scheduling system, and idle hardware resources, to train the text language model to be trained and generate the target text language model.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 7.

Images