A task parallel processing intelligent scheduling method and system for large model services

Abstract

The present invention proposes a method and system for intelligent scheduling of parallel processing of tasks for large model services, which belongs to the field of service computing technology. The method comprises the following steps: using the LaBSE model to extract semantic features from the instructions and text of parallel subtasks and generate embedded vectors, reducing the vector dimension of the embedded vectors through a compression module, and inputting the compressed embedded vectors into a regressor to predict the generated length; constructing a segmentation algorithm based on semantic integrity constraints to segment the text into a list of text fragments; using a multi-constraint task batching algorithm to divide the segmented text fragments in the text fragment list into different batches of tasks according to constraints of similar lengths and identical instructions, thereby obtaining a batch task set; and using a task allocation algorithm for heterogeneous resources to optimize the batch task set and determine the optimized subtask scheduling scheme. The present invention realizes efficient parallel processing of large model services, significantly improving the inference speed, resource utilization, and the intelligent level of task scheduling.

Description

Technical Field

[0001] The present invention belongs to the field of service computing technology, and in particular relates to a method and system for intelligent scheduling of task parallel processing for large model services. Background Art

[0002] The statements in this section merely provide background information related to the present invention and do not necessarily constitute prior art.

[0003] Large language model services are intelligent services based on deep learning and natural language processing technologies. They leverage pre-trained large models to provide users with a variety of services, including accurate text generation, conversational question-and-answering, language translation, and code assistance. However, as the scale of models and the number of users continue to grow, the computing resources and storage requirements required to deploy and execute these models also increase dramatically, placing a significant cost and resource burden on service providers. To address these challenges, service providers deploy large language models in the cloud and provide inference services to users through application programming interfaces (APIs). This service model, known as Model as a Service (MaaS), not only reduces providers' upfront investment and operational costs, but also promotes efficient model utilization and scalable application. Furthermore, this model supports on-demand resource utilization, elastic scaling, and continuous model updates, ensuring users always have access to the optimal model version without the need for manual maintenance and updates.

[0004] The Model-as-a-Service model optimizes the cost of large language model services, but there are certain problems and challenges in practical applications. First, the slow service response speed is a prominent issue, especially in scenarios with high concurrency or high real-time requirements. The computational complexity and network latency of the model may lead to a decline in user experience. Second, due to the scale and complexity of large models, their behavior and output are difficult to predict, making it difficult for existing service system optimization algorithms to achieve efficient optimization of large model services in a short period of time, increasing the risk of uncertainty. Finally, the resource allocation of the cloud platform may not accurately match the task requirements, resulting in irrational resource allocation, which in turn leads to resource waste or insufficient performance.

[0005] Existing technologies primarily encompass methods at the data, model, and system levels. Data-level optimization primarily reduces computational burden through input compression and increases inference parallelism through output organization. While this effectively reduces computational overhead, its applicability to different tasks is limited. Model-level optimization primarily involves methods such as model quantization, pruning, structural optimization, and dynamic inference. For example, quantization reduces storage and computational costs through low bit precision, but this can lead to reduced accuracy and requires additional compensation strategies. Pruning can reduce redundant parameters and improve inference speed, but high pruning rates can impact model performance. System-level optimization improves hardware utilization by optimizing the inference engine and inference system. For example, methods based on speculative decoding and FlashAttention reduce computational overhead, but are limited by specific architectures and are not universally applicable. Distributed inference and batch optimization can improve throughput, but their scheduling mechanisms and resource management are complex. Overall, existing technologies for large language model services suffer from slow inference speed, high computational and storage overhead, low throughput, and high inference latency. Summary of the Invention

[0006] To overcome the shortcomings of the aforementioned prior art, this paper proposes an intelligent scheduling method and system for parallel processing of tasks for large model services. This method comprehensively optimizes the performance of large language model services, improves inference speed and system throughput, and fully leverages user data features and system resources to achieve dual-level optimization and acceleration at both the data and system levels. First, to address the difficulty in predicting the generation length of requests, this paper proposes a LaBSE-based generation length prediction model for large language model tasks. This model utilizes the LaBSE model to extract semantic features and generate embedding vectors. This model reduces the vector dimension through a compression module and inputs the low-dimensional vector into a LightGBM regressor for generation length prediction, thereby improving the accuracy and efficiency of generation length prediction. Second, based on the generation length prediction results, a text segmentation algorithm based on semantic integrity constraints is proposed. Under semantic integrity constraints, this algorithm segments long text into multiple segments based on parallel requirements for parallel processing. Furthermore, a multi-constrained task batching algorithm is developed to assign text segments of similar length to the same batch, reducing the waiting time for tasks in the same batch. Finally, to improve system throughput, a batch task allocation algorithm for heterogeneous cloud resources is proposed. This algorithm utilizes an improved discrete binary particle swarm optimization algorithm, combined with an adaptive strategy and local search optimization, to rationally distribute batch tasks across a heterogeneous cluster of virtual machines, minimizing batch task completion time within memory constraints. This approach enables efficient parallel processing of large model services, significantly improving inference speed, resource utilization, and intelligent task scheduling, while reducing computational costs and latency.

[0007] To achieve the above objectives, one or more embodiments of the present invention provide the following technical solutions:

[0008] In a first aspect, a method for intelligent scheduling of task parallel processing for large model services is disclosed, comprising:

[0009] Receive the user's task instruction and text, and determine whether the text can be divided into subtasks to be executed in parallel according to the instruction, thereby obtaining parallel subtasks;

[0010] The LaBSE model is used to extract semantic features from the instructions and text of the parallel subtasks and generate an embedding vector. The vector dimension of the embedding vector is reduced by a compression module to obtain a compressed embedding vector. The compressed embedding vector is input into the LightGBM regressor to predict the generated length.

[0011] Based on the generated length, constructing a segmentation algorithm based on semantic integrity constraints to segment the text into a list of text segments and tags;

[0012] A multi-constrained task batching algorithm is used to divide the segmented text segments in the text segment list into different batches of tasks according to similar length constraints and the same instruction constraints, thereby obtaining a batch task set;

[0013] Optimizing the batch task set by using a heterogeneous resource-oriented task allocation algorithm and determining an optimized subtask scheduling scheme;

[0014] The subtask scheduling schemes of the same task are spliced in sequence according to the labels of the parallel subtasks to obtain a task parallel processing scheduling scheme.

[0015] Secondly, a task parallel processing intelligent scheduling system for large model services is disclosed, including:

[0016] The task receiving module is configured to: receive the instruction and text of the user task, determine whether the text can be divided into subtasks to be executed in parallel according to the instruction, and obtain parallel subtasks;

[0017] A generation length prediction module is configured to: extract semantic features from the instructions and text of the parallel subtasks using the LaBSE model and generate an embedding vector, reduce the vector dimension of the embedding vector using a compression module to obtain a compressed embedding vector, and input the compressed embedding vector into the LightGBM regressor to predict the generation length;

[0018] A text segmentation module is configured to: construct a segmentation algorithm based on semantic integrity constraints based on the generated length, and segment the text into a list of text segments and tags;

[0019] The task batching module is configured to: adopt a multi-constraint task batching algorithm to divide the segmented text segments in the text segment list into different batches of tasks according to similar length constraints and same instruction constraints, thereby obtaining a batch task set;

[0020] A scheduling optimization module is configured to: optimize the batch task set using a task allocation algorithm oriented to heterogeneous resources and determine an optimized subtask scheduling solution;

[0021] The scheduling generation module is configured to: sequentially splice the subtask scheduling schemes of the same task according to the labels of the parallel subtasks to obtain a task parallel processing scheduling scheme.

[0022] In a third aspect, an electronic device is disclosed, including a memory and a processor, and computer instructions stored in the memory and running on the processor. When the computer instructions are executed by the processor, the steps of the above-mentioned task parallel processing intelligent scheduling method for large model services are completed.

[0023] In a fourth aspect, a computer-readable storage medium is disclosed for storing computer instructions. When the computer instructions are executed by a processor, the steps of the above-mentioned method for intelligent scheduling of task parallel processing for large model services are completed.

[0024] Compared with the prior art, the present invention has the following beneficial effects:

[0025] 1. LaBSE-based large language model task generation length prediction model: The LaBSE model extracts semantic features and combines it with the LightGBM regressor to accurately predict the text generation length, significantly improving the accuracy and efficiency of prediction. This provides a reliable basis for subsequent text segmentation and batch processing, reducing the waste of computing resources.

[0026] 2. Text segmentation algorithm based on semantic integrity constraints: Based on the predicted generation length, a text segmentation algorithm is used to cut long text into multiple semantically complete segments, ensuring that the segmented text segments are suitable for parallel processing, effectively reducing computational complexity and improving inference efficiency.

[0027] 3. Multi-constrained task batching algorithm: This algorithm allocates text fragments of similar length and the same instruction to the same batch, reducing the use of padding tags and the waiting time of tasks in the same batch, optimizing service quality, and reducing memory overhead and computational costs.

[0028] 4. Text segmentation algorithm based on semantic integrity constraints: This algorithm uses an improved discrete binary particle swarm optimization (PSO) algorithm for task scheduling. Combined with adaptive strategies and local search optimization, it rationally allocates batch tasks to virtual machine resources, minimizing the maximum task completion time and significantly improving the intelligence level of task scheduling and resource utilization.

[0029] 5. By optimizing the task processing flow and resource scheduling strategy, the present invention significantly improves the inference efficiency of large language model services, reduces computing costs and delays, and ensures the integrity and accuracy of the output results, providing users with an efficient, stable, and low-cost large language model service experience.

[0030] Advantages of additional aspects of the present invention will be given in part in the following description and in part will be obvious from the following description, or will be learned through practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] The accompanying drawings, which constitute a part of the present invention, are used to provide a further understanding of the present invention. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute improper limitations on the present invention.

[0032] Figure 1 This is a schematic diagram of a specific application of the intelligent scheduling method for parallel processing of tasks for large model services described in the first embodiment of the present invention.

[0033] Figure 2 This is an overall flow chart of the intelligent scheduling method for parallel processing of tasks for large model services described in Example 1 of the present invention.

[0034] Figure 3 This is a flow chart of generating a length prediction model for the LaBSE-based large language model task described in Example 1 of the present invention.

[0035] Figure 4 This is a flow chart of the text segmentation algorithm based on semantic integrity constraints described in Example 1 of the present invention.

[0036] Figure 5 This is a flow chart of the multi-constrained task batching algorithm described in Example 1 of the present invention.

[0037] Figure 6 This is a flow chart of the heterogeneous resource-oriented task allocation algorithm described in the first embodiment of the present invention. DETAILED DESCRIPTION

[0038] It should be noted that the following detailed descriptions are exemplary and intended to provide further explanation of the present invention. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by those skilled in the art to which the present invention belongs.

[0039] It should be noted that the terms used herein are for describing particular embodiments only and are not intended to limit the exemplary embodiments according to the present invention.

[0040] In the absence of conflict, the embodiments of the present invention and the features thereof may be combined with each other.

[0041] Example 1

[0042] In one or more embodiments, a method for intelligent scheduling of parallel processing of tasks for large model services is disclosed, which aims to improve the reasoning efficiency and system throughput of large language models in cloud environments. First, the task type judgment module is used to distinguish between indivisible tasks and divisible tasks to avoid invalid processing; secondly, the generation length prediction module is used to accurately predict the generation length of the task text to provide a basis for subsequent processing; then, the text segmentation module is used to segment the long text into segments of similar length to facilitate parallel processing; finally, the text batching module and the intelligent scheduling module are combined to reasonably allocate tasks to heterogeneous virtual machine resources to minimize task completion time. The present invention significantly improves reasoning efficiency by optimizing task processing procedures and resource scheduling strategies, while ensuring the integrity and accuracy of the output results.

[0043] It should be noted that the embodiments described herein utilize a cloud server cluster as the platform for deployment, supporting the deployment of a variety of large models. This cloud platform boasts high-performance computing resources and automated management capabilities, enabling dynamic resource adjustments to meet the needs of diverse tasks. Furthermore, it supports on-demand elastic scaling, allocating and releasing resources in real time based on task load, ensuring both computational efficiency and cost reduction.

[0044] like Figure 1 Figure 2 shows a schematic diagram of a specific application scenario of the present invention. After a user submits a task to the cloud platform, the cloud scheduling system first parallelizes the task, splitting it into multiple subtasks and processing them in batches based on task type and resource requirements. The system then dynamically allocates these batches to cloud resources deployed on large models (such as GPU / TPU clusters) for execution, fully leveraging the high-performance computing capabilities of the cloud environment to ensure efficient task completion. Through parallel processing and intelligent scheduling, the present invention significantly improves task processing efficiency while reducing computing costs.

[0045] The specific implementation process of the task parallel processing intelligent scheduling method for large model services proposed in this invention is as follows: Figure 2 As shown, the following steps are included:

[0046] Step S1: Receive the user task instruction and text, and determine whether the text can be divided into subtasks for parallel execution based on the instruction using a large model to obtain parallel subtasks.

[0047] In this embodiment, the user task refers to a large language model inference task, which is a service request. The service request can be a request sent by the user's large language model service, and can include natural language processing tasks such as text translation, entity recognition, language polishing, text continuation, summary generation, and sentiment analysis.

[0048] Specifically, tasks with weak context dependencies, such as text translation, entity recognition, and language polishing, are parallelizable, while tasks with strong context dependencies, such as text continuation, summary generation, and sentiment analysis, are not. If the judgment result is negative, scheduling is performed directly. If the judgment result is positive, parallel subtasks are generated and subsequent parallel optimization operations are performed. This type of judgment can be used to screen large model tasks that are suitable for parallel optimization.

[0049] Step S2: Figure 3 As shown in the figure, the LaBSE model is used to extract semantic features from the instructions and text of the parallel subtasks and generate an embedding vector, and the vector dimension of the embedding vector is reduced through the compression module. The compressed embedding vector is input into the LightGBM regressor to predict the generated length.

[0050] Step S2-1: Based on the instructions and text of the parallel subtasks obtained in step S1, the LaBSE model is used to capture the semantic information of the text and generate an embedding vector;

[0051] Specifically, in this embodiment, the LaBSE model is used to extract semantic features and generate a 768-dimensional embedding vector, which is expressed as follows:

[0052] (1)

[0053] (2)

[0054] in, is the text embedding vector, For user text, is the instruction embedding vector, For instructions, , is the vector dimension, and = 768. The high-dimensional embedding vector generated by the LaBSE model can provide a basis for subsequent compression and prediction.

[0055] Step S2-2: Based on the embedding vector obtained in step S2-1, the embedding vector is compressed by a compression module to form a compressed embedding vector.

[0056] The embedded vector is compressed by the compression module to form a compressed embedded vector:

[0057] The vector Average share Group, vector Average share groups, the number of elements in each group is 、 . Sum each group and normalize to get the compressed embedding vector and , the calculation formula for the elements in the group is:

[0058] (3)

[0059] (4)

[0060] in, is the compressed text embedding vector, is the compressed instruction embedding vector, is the element in the i-th group, The number of text vector groups, optional =16, The number of instruction vector groups, optional =4 (optimal result after multiple trials), and , , The index of the element within the group.

[0061] The compression module reduces the dimension of the embedding vector, reduces the computational complexity, and avoids the overfitting problem.

[0062] Step S2-3: Compress the embedded vector and The length of the text entered by the user Concatenate into the first input vector , then the first input vector is fed into the trained LightGBM regressor to predict the generated length. The regressor is used to predict the generated length of the text, ensuring that the prediction results are both accurate and efficient.

[0063] Step S3: Based on the generated length, construct a segmentation algorithm based on semantic integrity constraints to segment the text into a list of text segments and tags, such as Figure 4 As shown, this facilitates subsequent parallel processing and improves reasoning efficiency.

[0064] Step S3-1: Receive the text within a unit time and its corresponding predicted length to form an input text list ( For the Input text, is the total number of texts, ) and its corresponding predicted generation length list ( for The service provider sets the parallel threshold based on resource allocation. ,control the granularity of text segmentation to ensure that the segmented segments are suitable for parallel processing;

[0065] Step S3-2: Initialize the text segment list ,provides a container for storing segmented text fragments;

[0066] Step S3-3, traverse the input text list , for text , initialize the text sublist , calculate the number of text segmentation:

[0067] (5)

[0068] in, Generates the predicted length for the text, Is the preset generation length threshold, text exceeding this length will be cut to ensure that the lengths of the segments after segmentation are similar, so as to facilitate subsequent parallel acceleration processing. The expected size of each text segment after being cut is:

[0069] (6)

[0070] in, For each text fragment size expected value, For text The actual length;

[0071] Step S3-4: Cut the first sub-segment from the input text (Here The starting index of Start by calculating the end position of the first sub-segment :

[0072] (7)

[0073] in, is the end position of the sub-segment, The starting index of the input text.

[0074] To ensure semantic integrity, if Characters other than periods (i.e. ), then search backward for the nearest period and update , ensuring that the text fragment is not corrupted.

[0075] Then, extract the fragment For cut text (Here ), stored in the text sublist Update The starting index of , increasing Continue to split subsequent segments until the current text is processed. Set to 1 to add text sublist Add to text snippet list . Increasing , process the next text until all text processing is completed.

[0076] Step S3-5: Return to the text fragment list , text sublist ( For text No. text snippets, where For the The number of text segments to be cut, the index of the list is recorded as a mark), providing input for subsequent batch processing.

[0077] The above algorithm can be expressed as: a text segmentation algorithm based on semantic integrity constraints, as shown in Table 1.

[0078]

[0079] Step S4: Figure 5 As shown, a multi-constraint task batching algorithm is adopted to divide the segmented text segments in the text segment list into different batch tasks according to the constraints of similar lengths and the same instruction constraints. This can reduce the use of padding marks and the differences in batch tasks, so as to ensure that tasks of the same time and subtasks of the same task are completed at a similar time, thereby shortening the task execution time.

[0080] Step S4-1: List of text segments segmented based on step S3 , set the relative error threshold according to resource allocation , batch capacity , controls the accuracy and size of the batch. This setting allows for continuous optimization of parameters based on the returned results.

[0081] Step S4-2: Initialize the batch queue and create an empty batch , provides a container for storing batched text fragments.

[0082] Step S4-3: Traverse the text fragment list Each sublist of , at the beginning of each cycle, the relative error The initial value is set to ,in is the relative error threshold. For each sublist , calculate the first batch The relative error in the length of the text fragment in this sublist is:

[0083] (8)

[0084] in, is the batch length, i.e. the length of the longest text in the batch. The predicted lengths of all segments after the same text is cut are regarded as .

[0085] Step S4-4: Traverse the batch queue Each batch in Then, determine whether there is a and batch capacity If the result is yes, the sublist The text fragments in the generated sublist are inserted into the batch with the closest length. If the judgment result is negative, a new batch is created and the text fragments in the sublist are inserted. This conditional judgment allows for batched insertion of text fragments, ensuring the constraints of relative error thresholds and batch capacity while dynamically expanding the batch queue. This mechanism balances accuracy and flexibility during batching, thereby improving the overall efficiency and adaptability of the algorithm.

[0086] The above algorithm can be expressed as: multi-constrained task batching algorithm, as shown in Table 2.

[0087]

[0088] Step S5: Figure 6 As shown, a task allocation algorithm oriented to heterogeneous resources is used to optimize the batch task set and determine an optimized subtask scheduling scheme;

[0089] Specifically, a task allocation algorithm for heterogeneous resources is used to optimize the allocation of batch tasks with the goal of minimizing task completion time. An improved binary PSO algorithm is used to reasonably allocate batch tasks to heterogeneous virtual machine resources, thereby improving overall reasoning efficiency.

[0090] Step S5-1: Randomly generate an initial scheduling scheme based on the batch task set and the generated length. The initial scheduling scheme includes a batch task memory requirement vector and a virtual machine memory capacity vector. ,common A batch task, a collection of virtual machines in the cloud service ,common virtual machines. Estimated batch task execution time matrix , where the matrix Chinese elements Indicates the The task in Estimated running time on virtual machines, representing batch tasks In the virtual machine The execution time of the task with the longest execution time in the batch is . The memory requirement vector of the batch task obtained based on the predicted text generation length is expressed as ,in is the memory requirement vector The middle element represents a batch task Memory required for execution, is the element index of the memory requirement vector, . Virtual machine memory capacity vector ,in is the memory capacity vector The middle element represents the virtual machine The remaining memory capacity after loading the large model weights, is the memory capacity vector element index, Because virtual machines in a cloud environment are heterogeneous, the memory capacity of all virtual machines is inconsistent.

[0091] Step S5-2: Initialize the particle swarm, corresponding to the initial scheduling scheme in the task generated in step S5-1, and set the number of particles and the maximum number of iterations , set the learning factor , (in is the individual learning factor, is the global learning factor). Each particle Represent a task allocation scheme using a binary matrix If It means the task Assigned to a virtual machine Execute on; if This indicates a batch task Not assigned to a virtual machine Initialize the velocity matrix is a zero matrix. The optimization objective (fitness function) is:

[0092] (9)

[0093] in, It is a virtual machine The total task completion time, is the fitness value;

[0094] (10)

[0095] in, For the virtual machine batch task queue, For the The task in Each virtual machine VM maintains a FIFO batch task queue, and tasks enter the queue in the order of arrival.

[0096] The total memory requirement of all batch tasks assigned to a virtual machine cannot exceed the remaining memory capacity of the virtual machine. Therefore, the constraint is:

[0097] (11)

[0098] in, For the allocation plan, When Assign the task to virtual machines; When The task is not assigned to virtual machines; ;

[0099] (12)

[0100] in, ;

[0101] Initialize the individual optimal solution through the above optimization objectives and constraints and the global optimal solution ; Calculate the initial fitness value .

[0102] Step S5-3: Loop until the maximum number of iterations is reached : Calculate the inertia weight of this round;

[0103] (13)

[0104] in, is the inertia weight of this round, is the maximum weight set, is the minimum weight set, For the current round.

[0105] Update particle speed in this round:

[0106] (14)

[0107] in, is the particle speed of this round, is the particle velocity in the previous round, 、 is a random number uniformly distributed on [0,1]. is the individual optimal solution of the particle.

[0108] Then update the particle position:

[0109] = ,when < hour;

[0110] = ,when hour;

[0111] in, For the updated allocation plan, is the Sigmoid activation function, and ; Set the speed Mapped to the interval [0,1], is a random number uniformly distributed on [0,1]. is the particle speed. Check the task allocation constraints, otherwise redistribute. Calculate the fitness value of this round Update the individual optimal solution of each particle , update the global optimal solution of the current round ;

[0112] Step S5-4: Perform local search optimization every five rounds. If the new solution is better than the global optimal solution, , then the global optimal solution Update to the new solution, otherwise The probability of accepting the new solution is is the fitness difference between the new solution and the current solution, is the temperature parameter of the current iteration;

[0113] Step S5-5: Based on the return task scheduling scheme ,The system allocates batch tasks to the most appropriate resources for execution according to the task scheduling plan;

[0114] Step S6: Return the inference result and continue to optimize: splice the subtask scheduling plans of the same task in order according to the labels of the parallel subtasks, obtain the task parallel processing scheduling plan, and return the result to the user.

[0115] In addition, the system continuously optimizes the parameters of the above-mentioned step models and algorithms based on the task completion status to achieve optimal reasoning efficiency.

[0116] Example 2

[0117] In one or more embodiments, a task parallel processing intelligent scheduling system for large model services is disclosed, specifically including:

[0118] The task receiving module is configured to: receive the instruction and text of the user task, determine whether the text can be divided into subtasks to be executed in parallel according to the instruction, and obtain parallel subtasks;

[0119] A generation length prediction module is configured to: extract semantic features from the instructions and text of the parallel subtasks using the LaBSE model and generate an embedding vector, reduce the vector dimension of the embedding vector using a compression module to obtain a compressed embedding vector, and input the compressed embedding vector into the LightGBM regressor to predict the generation length;

[0120] A text segmentation module is configured to: construct a segmentation algorithm based on semantic integrity constraints based on the generated length, and segment the text into a list of text segments and tags;

[0121] The task batching module is configured to: adopt a multi-constraint task batching algorithm to divide the segmented text segments in the text segment list into different batches of tasks according to similar length constraints and same instruction constraints, thereby obtaining a batch task set;

[0122] A scheduling optimization module is configured to: optimize the batch task set using a task allocation algorithm oriented to heterogeneous resources and determine an optimized subtask scheduling solution;

[0123] The scheduling generation module is configured to: sequentially splice the subtask scheduling schemes of the same task according to the labels of the parallel subtasks to obtain a task parallel processing scheduling scheme.

[0124] Example 3

[0125] This embodiment provides an electronic device, including a memory and a processor, and computer instructions stored in the memory and running on the processor. When the computer instructions are run by the processor, the steps of the above-mentioned task parallel processing intelligent scheduling method for large model services are completed.

[0126] Example 4

[0127] This embodiment provides a computer-readable storage medium for storing computer instructions. When the computer instructions are executed by a processor, the steps of the above-mentioned task parallel processing intelligent scheduling method for large model services are completed.

[0128] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowcharts and / or block diagrams, as well as combinations of processes and / or blocks in the flowcharts and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowcharts and / or block diagrams. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0129] These computer program instructions may also be stored in a computer readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0130] These computer program instructions can also be loaded onto a computer or other programmable data processing device, and a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, so that the instructions executed on the computer or other programmable device provide the functions for implementing the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0131] The descriptions of the various embodiments in the above embodiments have different focuses. For parts not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0132] The foregoing description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Those skilled in the art will readily appreciate that various modifications and variations of the present invention are possible. Any modifications, equivalent substitutions, or improvements made within the spirit and principles of the present invention are intended to be within the scope of protection of the present invention.

Claims

1. A method for intelligent scheduling of task parallel processing for large model services, characterized in that: include: Receive the user's task instruction and text, and determine whether the text can be divided into subtasks to be executed in parallel according to the instruction, thereby obtaining parallel subtasks; The LaBSE model is used to extract semantic features from the instructions and text of the parallel subtasks and generate an embedding vector. The vector dimension of the embedding vector is reduced by a compression module to obtain a compressed embedding vector. The compressed embedding vector is input into the LightGBM regressor to predict the generated length. Based on the generated length, constructing a segmentation algorithm based on semantic integrity constraints to segment the text into a list of text segments and tags; A multi-constrained task batching algorithm is used to divide the segmented text segments in the text segment list into different batches of tasks according to similar length constraints and the same instruction constraints, thereby obtaining a batch task set; Optimizing the batch task set by using a heterogeneous resource-oriented task allocation algorithm and determining an optimized subtask scheduling scheme; The subtask scheduling schemes of the same task are spliced in sequence according to the labels of the parallel subtasks to obtain a task parallel processing scheduling scheme.

2. The method for intelligent scheduling of task parallel processing for large model services according to claim 1, characterized in that: The method of reducing the vector dimension of the embedded vector by the compression module is specifically as follows: Embed text into vectors Average share group, embed instructions into vectors Average share groups, the number of elements in each group is 、 , sum each group and normalize to get the compressed embedding vector, and the calculation formula of the elements in the group is: in, is the compressed text embedding vector, is the compressed instruction embedding vector, is the element in the i-th group, is the vector dimension, is the number of text vector groups, is the number of instruction vector groups, The index of the element within the group.

3. The method for intelligent scheduling of task parallel processing for large model services according to claim 1, characterized in that: Based on the generated length, a segmentation algorithm based on semantic integrity constraints is constructed to segment the text into a list of text segments and tags, specifically: According to the generated lengths, forming an input text list and a generated length list; Initialize the text fragment list; Traverse the input text list and its corresponding generated length list, and calculate the number of text segments and the corresponding generated length of each text under the parallel threshold constraint to ensure that the generated lengths of text segments after segmentation of the same text are similar; Cut each text in the text list according to the calculated number of cutting segments and store them in the corresponding text sublist; Integrate the segmented sublists to get a text fragment list.

4. The method for intelligent scheduling of task parallel processing for large model services according to claim 1, characterized in that: The text segments in the text segment list are divided into different batches of tasks according to the similar length constraint and the same instruction constraint, to obtain a batch task set, including: Receive the segmented text segments in the text segment list; Initialize the batch queue and create a batch; Access each sublist in the text fragment list in turn. For each sublist, calculate the relative error between it and the longest generated length of each batch task in the batch queue. By comparing the error values, locate the batch with the most matching generated length. Determine whether there is a batch in the batch queue whose relative error meets the error threshold and the sublist generation length meets the batch capacity constraint; if the judgment result is yes, insert the text in the sublist into the batch with the closest generation length; if the judgment result is no, create a new batch and insert the text in the sublist to obtain a batch task set.

5. The method for intelligent scheduling of task parallel processing for large model services according to claim 4, characterized in that: The relative error value of the generated length is: in, is the relative error in length, is the batch length, is the predicted generation length, n is the A text fragment.

6. The method for intelligent scheduling of task parallel processing for large model services according to claim 1, characterized in that: The step of optimizing the batch task set by using a heterogeneous resource-oriented task allocation algorithm and determining an optimized subtask scheduling scheme includes: Randomly forming an initial scheduling scheme based on the batch task set and the generation length; Based on the initial iteration scheme, initialize the particle swarm, set the number of particles, maximum number of iterations, learning factor, calculate the initial local and global optimal solutions, and set the optimization goal; Calculate the weights in a loop and update the speed and position of each particle to calculate the optimal solution; Based on the number of cycles, local search optimization is performed every five rounds to determine the optimized subtask scheduling scheme.

7. The method for intelligent scheduling of task parallel processing for large model services according to claim 6, characterized in that: The optimization goal is: in, It is a virtual machine The total task completion time, in, For the A virtual machine batch task queue, For the The task in Estimated running time on 1 virtual machine; The constraints are: in, are the elements in the memory requirement vector, ; in, For the allocation plan, When Assign the task to virtual machines; When The task is not assigned to virtual machines, ; The loop calculates weights and updates the velocity and position of each particle to calculate the optimal solution, including: Calculate the inertia weight of this round; in, is the inertia weight of this round, is the maximum weight set, is the minimum weight set, For the current round; Update particle velocity: in, is the particle speed of this round, is the particle velocity in the previous round, , is a random number uniformly distributed on [0,1]. is the individual optimal solution of the particle, is the global optimal solution, For the allocation plan, is the individual learning factor, is the global learning factor; Then update the particle position: = ,when < hour; = ,when hour; in, The updated allocation plan, is the Sigmoid activation function, is a random number uniformly distributed on [0,1]. is the particle speed.

8. A task parallel processing intelligent scheduling system for large model services, characterized by: include: The task receiving module is configured to: receive the instruction and text of the user task, determine whether the text can be divided into subtasks to be executed in parallel according to the instruction, and obtain parallel subtasks; A generation length prediction module is configured to: extract semantic features from the instructions and text of the parallel subtasks using the LaBSE model and generate an embedding vector, reduce the vector dimension of the embedding vector using a compression module to obtain a compressed embedding vector, and input the compressed embedding vector into the LightGBM regressor to predict the generation length; A text segmentation module is configured to: construct a segmentation algorithm based on semantic integrity constraints based on the generated length, and segment the text into a list of text segments and tags; The task batching module is configured to: adopt a multi-constraint task batching algorithm to divide the segmented text segments in the text segment list into different batches of tasks according to similar length constraints and same instruction constraints, thereby obtaining a batch task set; A scheduling optimization module is configured to: optimize the batch task set using a task allocation algorithm oriented to heterogeneous resources, and determine an optimized subtask scheduling solution; The scheduling generation module is configured to: splice the subtask scheduling plans of the same task in sequence according to the labels of the parallel subtasks to obtain a task parallel processing scheduling plan.

9. An electronic device, characterized in that: The method comprises a memory and a processor, and computer instructions stored in the memory and executed on the processor. When the computer instructions are executed by the processor, the method completes the intelligent scheduling method for parallel processing of tasks for large model services as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that Used to store computer instructions, which, when executed by a processor, complete the task parallel processing intelligent scheduling method for large model services as described in any one of claims 1 to 7.

Images