Task allocation method and apparatus, and device, storage medium and program product

Abstract

Disclosed in the present application are a task allocation method and apparatus, and a device, a storage medium and a program product in the technical field of computer applications. The method comprises: acquiring state information of a processor in a server cluster, and writing into a target record idle-processor identifiers determined by using the state information; sequentially reading the idle-processor identifiers from the target record, and sequentially reading dataset paths from an input file; determining one read idle-processor identifier and one read dataset path as an allocation information pair, on the basis of each allocation information pair, performing splicing to obtain a task command and writing same into an output file, and deleting from the input file dataset paths that have been read and used; and on the basis of the output file, allocating an inference task to an idle processor in the server cluster. The technical effects are as follows: the present application can automatically allocate an inference task of a model to a server cluster in a distributed manner, thereby reducing the learning cost for developers on a distributed framework and the time cost of task allocation and deployment.

Description

Task allocation method, device, equipment, storage medium and program product

[0001] Cross-reference of related applications

[0002] The present application claims priority to the Chinese patent application No. 202410781699.3, filed on June 18, 2024, and entitled "Task allocation method, device, equipment, storage medium and program product", the content of which is incorporated herein by reference in its entirety. TECHNICAL FIELD

[0003] The present application relates to the technical field of computer application, in particular to a task allocation method, device, equipment, storage medium and program product. BACKGROUND

[0004] When a large model performs a reasoning task (i.e., executes a reasoning task), in the case of a large amount of data, distributed reasoning is often adopted, and the reasoning operation of the deep learning model is performed on multiple computing nodes or devices at the same time to speed up the reasoning process and process large-scale data.

[0005] For some popular open source models, their communities provide relatively complete distributed reasoning solutions, and after deployment, large-scale reasoning tasks can be performed immediately. For example, the CodeGeeX (multi-language code generation model) model can use Megatron (a deep learning library for training extremely large models) for distributed reasoning, and the code repository of CodeGeeX can be downloaded from github (a hosting platform for open source and private software projects). Based on the code repository, the computing resources of the distributed system can be utilized to process large-scale data sets.

[0006] However, for the latest large models or some large models mainly used for research and learning, the support for distributed reasoning is not enough, and the community does not provide a distributed reasoning solution. At this time, developers need to learn some distributed reasoning frameworks and adapt these models. The learning cost and time cost are relatively high, and for some tasks with time requirements, these costs may not be acceptable.

[0007] In summary, how to effectively solve the problem of distributed allocation of reasoning tasks of models is a technical problem that needs to be solved by technical personnel in the field at present. SUMMARY

[0008] The purpose of the present application is to provide a task allocation method, device, equipment, storage medium and program product, which can automatically implement distributed reasoning task allocation of models.

[0009] To solve the above technical problems, the present application provides the following technical solutions:

[0010] A task allocation method comprises:

[0011] Obtaining state information of processors in a server cluster, and writing idle processor identifiers determined by using the state information into a target record;

[0012] In a case where the target record is not empty, sequentially reading idle processor identifiers from the target record and sequentially reading dataset paths from an input file until paired idle processor identifiers and dataset paths cannot be read; wherein one dataset path corresponds to one inference task in use of a model;

[0013] Determining one idle processor identifier and one dataset path read to be one pair of allocation information, and respectively splicing one task command according to each pair of allocation information;

[0014] Writing the plurality of task commands spliced to an output file, and deleting the dataset paths used and read in the input file;

[0015] Based on the output file, allocating inference tasks to idle processors in the server cluster.

[0016] In some embodiments of the present application, obtaining state information of processors in a server cluster, and writing idle processor identifiers determined by using the state information into a target record comprises:

[0017] Obtaining state information of processors in a server cluster, and writing idle processor identifiers determined by using the state information into a first record;

[0018] After a specified time interval, obtaining state information of processors in the server cluster, and writing idle processor identifiers determined by using the state information into a second record;

[0019] Determining an intersection of the first record and the second record as the target record.

[0020] In some embodiments of the present application, the state information comprises processor usage, obtaining state information of processors in a server cluster, and writing idle processor identifiers determined by using the state information into a target record comprises:

[0021] Using a parallel distributed operation and maintenance tool, calling a processor state checking command line tool to obtain the processor usage;

[0022] Determining a processor with a processor usage lower than a preset threshold as an idle processor;

[0023] Determining an IP address and a number of the idle processor as information in the idle processor identifier;

[0024] write the idle processor identifier into the target record.

[0025] In some embodiments of the present application, writing the idle processor identifier into the target record comprises:

[0026] writing the IP address and the number in the idle processor identifier into the target record in the form of a key-value pair.

[0027] In some embodiments of the present application, after calling the processor state checking command line tool by using the parallel distributed operation and maintenance tool, further comprising:

[0028] if the processor usage rate is not obtained, outputting prompt information of capturing an exception.

[0029] In some embodiments of the present application, writing the IP address and the number in the idle processor identifier into the target record in the form of a key-value pair comprises:

[0030] taking the IP address and the number of the idle processor as the key, and taking the number of the idle processor as the value.

[0031] writing the key and the value into the target record according to a preset key-value pair format; wherein the IP address corresponds to a node to which the idle processor belongs.

[0032] In some embodiments of the present application, in the case that the target record is not empty, sequentially reading the idle processor identifiers from the target record and sequentially reading the dataset paths from the input file until paired idle processor identifiers and dataset paths cannot be read, comprising:

[0033] judging whether the target record is empty;

[0034] if the target record is empty, outputting prompt information of failure of reasoning task allocation;

[0035] if the target record is not empty, reading a dataset path from the input file each time an idle processor identifier is read from the target record until paired idle processor identifiers and dataset paths cannot be read.

[0036] In some embodiments of the present application, obtaining the state information of the processors in the server cluster comprises:

[0037] periodically obtaining the state information of the processors in the server cluster;

[0038] or, obtaining the state information of the processors in the server cluster in the case that the input file is obtained;

[0039] or, obtaining the state information of the processors in the server cluster in the case that the input file is not empty.

[0040] In some embodiments of the present application, the spliced task commands are written into an output file, and the dataset paths that have been read and used are deleted from the input file, including:

[0041] In the output file, the task commands are written in sequence;

[0042] The dataset paths that have been read and used for task command splicing are deleted from the input file.

[0043] In some embodiments of the present application, based on the output file, the inference tasks are allocated to idle processors in the server cluster, including:

[0044] The inference tasks are allocated to idle processors in the server cluster by executing the task commands in the output file.

[0045] In some embodiments of the present application, further comprising:

[0046] Using the idle processors allocated to the tasks, the datasets corresponding to the dataset paths are read, and the read datasets are processed and inferred to obtain results and output;

[0047] After outputting the results, the processors that have completed the tasks are cleaned up.

[0048] In some embodiments of the present application, one idle processor identifier and one dataset path read are determined as a pair of allocation information, and one task command is spliced for each pair of allocation information;

[0049] According to the reading order, one idle processor identifier and one dataset path read at the same time are determined as a pair of allocation information;

[0050] According to the standard format of the preset command line, each pair of allocation information is spliced to obtain a plurality of task commands.

[0051] In some embodiments of the present application, the state information of the processors in the server cluster is obtained, including:

[0052] A preset command for obtaining processor state information is executed to obtain the state information of each processor in the server cluster.

[0053] In some embodiments of the present application, the state information of the processors in the server cluster is obtained, including:

[0054] A preset tool for obtaining processor state information is run to obtain the state information of each processor in the server cluster.

[0055] In some embodiments of the present application, the state information of the processors in the server cluster is obtained, including:

[0056] receiving the state information of the processors periodically reported by the processors in the server cluster.

[0057] In some embodiments of the present application, the method for determining the idle processor is:

[0058] When the processor usage of the target processor is lower than the preset usage threshold, the target processor is determined as an idle processor.

[0059] A task allocation apparatus comprises:

[0060] A state acquisition module is configured to acquire the state information of the processors in the server cluster, and write the idle processor identified by using the state information into a target record;

[0061] An information reading module is configured to, when the target record is not empty, read the idle processor identifiers from the target record and the dataset paths from the input file in sequence until a pair of idle processor identifier and dataset path cannot be read; wherein one dataset path corresponds to one inference task in the model usage.

[0062] A task command determination module is configured to determine one idle processor identifier and one dataset path as one pair of allocation information, and splice one task command according to each pair of allocation information.

[0063] An output file determination module is configured to write the spliced task commands into an output file, and delete the dataset paths in the input file that have been read and used.

[0064] A task allocation module is configured to allocate the inference tasks to the idle processors in the server cluster based on the output file.

[0065] An electronic device comprises:

[0066] A memory is configured to store a computer program.

[0067] A processor is configured to implement the steps of the above task allocation method when executing the computer program.

[0068] A non-volatile readable storage medium has a computer program stored thereon, and the computer program is executed by a processor to implement the steps of the above task allocation method.

[0069] A computer program product comprises a computer program / instruction, and the computer program / instruction is executed by a processor to implement the steps of the above task allocation method.

[0070] The method provided in the embodiment of the application is used to obtain the state information of the processors in the server cluster, and write the idle processor identifiers determined by using the state information into a target record; in the case that the target record is not empty, the idle processor identifiers are read from the target record one by one, and the dataset paths are read from the input file one by one until the paired idle processor identifiers and dataset paths cannot be read; wherein one dataset path corresponds to one inference task in the model use; one idle processor identifier and one dataset path read are determined as one pair of allocation information, and one task command is spliced according to each pair of allocation information; the plurality of task commands obtained by splicing are written into an output file, and the dataset paths used in the input file are deleted; and based on the output file, the inference tasks are allocated to the idle processors in the server cluster.

[0071] In the application, the state information of the processors in the server cluster capable of implementing the distributed execution of a certain model can be obtained first, and then the idle processors can be determined based on the state information, and the idle processors can be written into a target record. Then, in the case that the target record is not empty, the idle processor identifiers are read from the target record, and at the same time, the dataset paths are read from the input file one by one until the paired idle processor identifiers and dataset paths cannot be read. That is, in the application, the dataset paths and the idle processor identifiers read are read in pairs, and one dataset path corresponds to one inference task in the model test. In order to effectively complete the inference task allocation, one idle processor identifier and one dataset path are determined as one pair of allocation information, and each allocation information is spliced to obtain a plurality of task commands. That is, which idle processor processes which inference task corresponding to which dataset path corresponds to one task command. The plurality of task commands are written into an output file, and the dataset paths used in the output file are deleted. In this way, based on the output file, the inference tasks can be allocated to the idle processors in the server cluster.

[0072] Technical effects: The application relates to distributed task distribution, and automatically distributes inference tasks to the server cluster for the latest or community-supported incomplete model, reduces the learning cost of the developer for the distributed framework and the time cost of task allocation and deployment.

[0073] Correspondingly, the embodiment of the application also provides a task allocation device, equipment, nonvolatile readable storage medium and computer program product corresponding to the above-mentioned task allocation method, which has the above-mentioned technical effects, and details are not repeated here. BRIEF DESCRIPTION OF DRAWINGS

[0074] In order to make the technical scheme of the embodiments of the present application or the related art clearer, the accompanying drawings needed in the embodiments or the related art description will be briefly introduced. Obviously, the accompanying drawings in the following description only represent some embodiments of the present application, and all other embodiments obtained by those of ordinary skill in the art without creative effort based on the accompanying drawings are within the protection scope of the present application.

[0075] FIG. 1 is an implementation flowchart of a task allocation method in an embodiment of the present application;

[0076] FIG. 2 is a flowchart of an output file determination process in an embodiment of the present application;

[0077] FIG. 3 is a specific implementation flowchart of a task allocation method in an embodiment of the present application;

[0078] FIG. 4 is a structural schematic diagram of a task allocation device in an embodiment of the present application;

[0079] FIG. 5 is a structural schematic diagram of an electronic device in an embodiment of the present application;

[0080] FIG. 6 is a specific structural schematic diagram of an electronic device in an embodiment of the present application. DETAILED DESCRIPTION

[0081] In order to make the technical scheme of the embodiments of the present application or the related art clearer, the accompanying drawings needed in the embodiments or the related art description will be briefly introduced. Obviously, the accompanying drawings in the following description only represent some embodiments of the present application, and all other embodiments obtained by those of ordinary skill in the art without creative effort based on the accompanying drawings are within the protection scope of the present application.

[0082] Please refer to FIG. 1, which is a flowchart of a task allocation method in an embodiment of the present application. The method includes the following steps:

[0083] S101, obtaining state information of processors in a server cluster, and writing an idle processor identifier determined by using the state information into a target record.

[0084] The server cluster can include multiple nodes, and each node can include several processors, which can be GPUs, CPUs, or other processors.

[0085] In the present application, the preset task allocation model can execute a preset command for obtaining processor state information to obtain the state information of each processor in the server cluster, or can run a preset tool for obtaining processor state information to obtain the state information of each processor in the server cluster, or can be set to periodically report the state information of the processors in each server cluster by the processors themselves.

[0086] The state information can be information indicating whether it has an assigned task, or information indicating whether it is executing a related task, or a related performance indicator of the processor, such as memory usage, etc. That is, as long as the state information can clearly indicate which are idle processors. It should be noted that in the embodiments of the present application, the idle processor refers to a processor to which a reasoning task can be assigned at this time.

[0087] In an embodiment of the present application, the state information of the processors in the server cluster is obtained, comprising:

[0088] Periodically obtaining the state information of the processors in the server cluster;

[0089] Or, in the case of obtaining an input file, obtaining the state information of the processors in the server cluster;

[0090] Or, in the case of the input file being not empty, obtaining the state information of the processors in the server cluster.

[0091] That is, in this embodiment, the timing of obtaining the state information of the processors in the server cluster can be through periodic acquisition, or through triggering acquisition in the case of obtaining an input file, or triggering acquisition of the state of the processors in the server cluster in the case of the input file being not empty. The three situations can be used in combination in actual applications. For example, after the state information of the processors in the server cluster is obtained periodically, it is first determined whether there is an input file at present, and the input file is not empty, if so, the state information is obtained, otherwise the current round of acquisition is not performed.

[0092] In an embodiment of the present application, the state information includes processor usage, the state information of the processors in the server cluster is obtained, and the idle processor identifier determined by the state information is written into the target record, comprising:

[0093] Using a parallel distributed operation and maintenance tool to call a processor state checking command line tool to obtain the processor usage;

[0094] Determining the processor whose processor usage is lower than a preset threshold as an idle processor;

[0095] Determining the IP address and number of the idle processor as information in the idle processor identifier;

[0096] Writing the idle processor identifier into the target record.

[0097] For ease of description, the above steps will be described in combination.

[0098] In the embodiment, the state information can specifically include processor usage rates of the processors in the server cluster. Specifically, a preset usage rate threshold can be set in advance, and when the processor usage rate is lower than the preset usage rate threshold, the corresponding processor can be determined as an idle processor. For example, the threshold can be set as 0, and of course, other values lower than the value can also be set according to actual needs, which can indicate that the value belongs to an idle state and can be allocated a reasoning task.

[0099] After the idle processor is determined, the IP address and number of the idle processor can be determined as information in the idle processor identifier, and then the idle processor identifier is written into the target record. In this way, by reading the target record, it can be determined which idle processors are currently available.

[0100] In an embodiment of the present application, the processing of the state information includes:

[0101] The state information is obtained, and an output result is obtained; the output result includes the IP address and number of the idle processor;

[0102] The output result is assigned to a temporary variable;

[0103] The output result is processed based on the temporary variable to obtain a list-form output result.

[0104] Specifically, a command to be executed can be defined in advance, the command uses pdsh to execute the nvidia-smi command on the specified host, queries the index and used memory of the GPU, and filters out the idle GPU. The pre-defined command is executed, and the output result is assigned to a temporary variable. The output result is processed, and its format is modified to a list form, so as to be further processed based on the list form subsequently.

[0105] In an embodiment of the present application, writing the idle processor identifier into the target record includes:

[0106] The IP address and number in the idle processor identifier are written into the target record in the form of a key-value pair.

[0107] The idle processor identifier is recorded in the form of a key-value pair, and the key of the dictionary can be used as the name of the subsequent log file, such as 127.0.0.1_0.log, so that the execution of each GPU on each node on the task can be well distinguished.

[0108] In an embodiment of the present application, the IP address and number in the idle processor identifier are written into the target record in the form of a key-value pair, including:

[0109] The IP address and number of the idle processor are used as the key, and the number of the idle processor is used as the value.

[0110] According to the preset key-value pair format, the key and value are written into the target record; wherein the IP address corresponds to the node to which the idle processor belongs.

[0111] That is, in the present embodiment, the processor is a GPU, and the processing of the state information includes:

[0112] Traverse the output list corresponding to the idle processor to extract the IP address and number of each GPU;

[0113] The extracted IP address and number of each GPU are added to the dictionary as a key-value pair; wherein the key-value pair format of the dictionary is {'IP address and number of GPU': 'number of GPU'}.

[0114] Specifically, the output result of obtaining the state information can be processed to modify its format to a list form. By traversing the output list, the IP address and number of each GPU are extracted and added to the dictionary as a key-value pair, and the key-value pair format of the dictionary is {'IP address and number of GPU': 'number of GPU'}, such as {'127.0.0.1_0': '0', '127.0.0.1_1': '1', '127.0.0.1_2': '2'} indicates that the IP of the server is 127.0.0.1, and the idle GPUs are the 0th card, 1st card and 2nd card of the current server.

[0115] In this way, the key of the dictionary can be used as the naming of the subsequent log file, such as 127.0.0.1_0.log, which can well distinguish the execution of each GPU of each node on the task; the number of the GPU is to facilitate the splicing of the command of the inference task. In addition, the key-value pair of the dictionary is designed in this way to avoid the complexity of subsequent processing of the idle processor identifier.

[0116] Since the naming of the key of the dictionary cannot be repeated, and each node corresponds to multiple GPUs; based on the above two points, if the IP is simply used as the key, then the format of the dictionary will be {'127.0.0.1': '[0,1,2]'}, that is, the 0th, 1st and 2nd cards of 127.0.0.1 are idle.

[0117] In order to ensure that the inference task allocation does not conflict, it is more simple to compare and analyze the idle state processor identifier. Of course, the list can also be directly selected as the returned data structure here, but the list is not conducive to the data expansion in the later period, and the dictionary can add other information in the value in the later period. In summary, it is recommended that the data structure selected here is the dictionary, and the format of the dictionary is {'IP address and number of GPU': 'number of GPU'}.

[0118] In a specific embodiment of the present application, after the parallel distributed operation and maintenance tool is used to call the processor state check command line tool, if the processor usage rate is not obtained, prompt information of capturing an exception is output. That is, in order to the robustness of the program, the exception capturing is added, if an error occurs in the execution command, the exception is captured, and the error information is printed, so as to facilitate subsequent problem positioning.

[0119] S102, in the case that the target record is not empty, read the idle processor identifiers from the target record and the dataset paths from the input file in sequence until the paired idle processor identifiers and dataset paths cannot be read.

[0120] Wherein, one dataset path corresponds to one inference task in model use.

[0121] Specifically, one dataset corresponds to one inference task, and one dataset path corresponds to one dataset, so one dataset path also corresponds to one inference task.

[0122] In the case that the target record is not empty, the idle processor identifiers can be read from the target record, and the dataset paths can be read from the input file in sequence. Since one inference task needs to be assigned to one processor for processing, and one processor can only process one inference task in the same allocation period, that is, the inference task needs to correspond to one processor.

[0123] In order to realize one-to-one correspondence between the inference task and the processor, first, the target record and the input file are read only in the case that the target record is not empty; second, the reading can be ended when the paired idle processor identifiers and dataset paths cannot be read.

[0124] That is, the idle processor identifiers obtained by reading always appear in pairs with the dataset paths. That is, one idle processor identifier is read, and one dataset path is read.

[0125] For example, in the case that there are 2 idle processor identifiers in the target record, if there are 4 dataset paths in the input file, after reading 2 idle processor identifiers from the target record and 2 dataset paths from the input file, the reading is ended;

[0126] In the case that there are 3 idle processor identifiers in the target record, if there is 1 dataset path in the input file, after reading 1 idle processor identifier from the target record and 1 dataset path from the input file, the reading is ended;

[0127] If there are three idle processor identifiers in the target record, and there are three data set paths in the input file, then after reading the three idle processor identifiers from the target record and the three data set paths from the input file, the reading is ended;

[0128] If there are three idle processor identifiers in the target record, and there are no data set paths in the input file, then no reading is performed.

[0129] In actual implementation, the number A of idle processor identifiers in the target record and the number B of data set paths in the input file can be determined first, and if A and B are both not 0, the minimum value is determined from A and B, and the minimum value is read from the target record and the input file in turn, so that the number of idle processor identifiers and data set paths of the minimum value is obtained. Of course, in the reading file, one reading can be defined, that is, one data set path is read from the input file, and one idle processor identifier is read from the target record. If one reading cannot be met, the reading is stopped. If one reading can be met, the reading is continuously cycled until the condition of one reading is not met.

[0130] In the input file, one line of information can correspond to one data set path, so when reading the input file, one line of information or several lines of information can be read each time.

[0131] In an embodiment of the present application, if the target record is not empty, idle processor identifiers are read from the target record in turn, and data set paths are read from the input file in turn, until paired idle processor identifiers and data set paths cannot be read, including:

[0132] It is judged whether the target record is empty;

[0133] If the target record is empty, a reasoning task allocation failure prompt information is output;

[0134] If the target record is not empty, each time an idle processor identifier is read from the target record, a data set path is read from the input file, until paired idle processor identifiers and data set paths cannot be read.

[0135] Specifically, if the target record is empty, a reasoning task allocation failure prompt information is directly output; if the target record is not empty, each time an idle processor identifier is read from the target record, a data set path is read from the input file, until paired idle processor identifiers and data set paths cannot be read. At the same time, traversal reading can be implemented to iterate according to the shortest input sequence, that is, the input file line number and the target record length are iteratively read according to the shortest one.

[0136] S103, determine the read idle processor identifier and a data set path as a pair of allocation information, and splice a task command according to each pair of allocation information respectively.

[0137] Since one inference task needs to be one-to-one corresponding to one idle processor, in the embodiment, the read idle processor identifier and a data set path are determined as a pair of allocation information, and a task command is spliced according to each pair of allocation information respectively.

[0138] That is, a task command is spliced based on a pair of allocation information, and the task command can specify that an inference task is to be allocated to an idle processor.

[0139] In an embodiment of the present application, the read idle processor identifier and a data set path are determined as a pair of allocation information, and a task command is spliced according to each pair of allocation information respectively;

[0140] According to the reading order, an idle processor identifier and a data set path read at the same time are determined as a pair of allocation information;

[0141] According to the standard format of the preset command line, each pair of allocation information is spliced to obtain a plurality of task commands.

[0142] For ease of description, the above steps will be described together.

[0143] In the embodiment, an idle processor identifier and a data set path read at each time can be determined as a pair of allocation information according to the reading order. Then, according to the standard format of the preset command line, each pair of allocation information is spliced to obtain a plurality of task commands. That is, there are as many pairs of allocation information as there are task commands.

[0144] The task command realizes the allocation of the inference task corresponding to the data set path in the allocation information to the idle processor corresponding to the idle processor identifier in the allocation information. In actual application, the preset command line identifier format can be specifically capable of allocating the inference task based on the data set path to the idle processor.

[0145] In an embodiment of the present application, the processor is a GPU, and the idle processor identifier includes the IP address and number of the GPU. According to the standard format of the preset command line, each pair of allocation information is spliced to obtain a plurality of task commands, including:

[0146] The absolute address in each pair of allocation information and the IP and number of the idle GPU are written into the corresponding position of the standard format of the preset command line to obtain a plurality of task commands;

[0147] wherein, the standard format of the preset command line is: pdsh-w TARGET_IP–R ssh–f PORT"CUDA_VISIBLE_DEVICES=gpu_num python infer.py--input_file inputfile.txt--output_file outputfile.txt 2>&1|tee. / log / TARGET_IP.log"&;

[0148] Pdsh, representing a parallel distributed operation tool;

[0149] -w TARGET_IP, representing a target server IP;

[0150] -R ssh, representing specifying to use a secure shell protocol;

[0151] -f, representing executing a remote command of the secure shell protocol in the background;

[0152] PORT, representing a port number accessed by the secure shell protocol;

[0153] the command inside the quotation marks is executed by the parallel distributed operation tool on the target server:

[0154] CUDA_VISIBLE_DEVICES=gpu_num, representing specifying a GPU number of the current server;

[0155] python infer.py, representing a model inference script executed;

[0156] --input_file, representing an input file;

[0157] --output_file, representing an output file;

[0158] inputfile.txt, representing an output file list;

[0159] 2>&1|tee. / log / log.log, representing printing a log on a screen and writing the log into a. / log / TARGET_IP.log file;

[0160] &, representing putting a current command into the background for execution.

[0161] For example, if the processor identifier corresponding to the allocation information is {'127.0.0.1_0':'0'} and the dataset path is / home / 1.txt, the task command is obtained by splicing based on the standard format of the preset command line as follows:

[0162] pdsh -w 127.0.0.1 -R ssh -f 500"CUDA_VISIBLE_DEVICES=0python infer.py--input_file / home / 1.txt--output_file / home / 1.txt_res 2>&1|tee. / log / 127_0_0_1_0.log"&.

[0163] Correspondingly, if there are multiple pairs of allocation information, there are multiple corresponding task commands.

[0164] S104, write the spliced several task commands to the output file, and delete the dataset path read and used in the input file.

[0165] After completing the splicing of the task command, the task command can be written to the input file. In order to avoid the repeated allocation of inference tasks, the dataset path read and used in the task command splicing needs to be deleted from the input file.

[0166] In an embodiment of the present application, writing the spliced several task commands to the output file and deleting the dataset path read and used in the input file includes:

[0167] sequentially writing several task commands in the output file;

[0168] delete the dataset path read and used in the task command splicing from the input file.

[0169] That is, the task command generated by splicing can be written to the output file one by one. Then, for the dataset path read and used in the task command splicing in the input file, it can be deleted.

[0170] S105, based on the output file, allocate inference tasks to idle processors in the server cluster.

[0171] The output file has task commands. Through the output file, inference tasks can be allocated to idle processors in the server cluster. Thus, the distributed inference application of the model is realized.

[0172] In an embodiment of the present application, based on the output file, allocating inference tasks to idle processors in the server cluster includes:

[0173] By executing the task command in the output file, inference tasks are allocated to idle processors in the server cluster. That is, by executing the task command, the distributed deployment of the inference task is realized, that is, the idle processor is caused to execute the task inference processing of the corresponding dataset path corresponding to the dataset.

[0174] In an embodiment of the present application, further comprising:

[0175] Using the idle processor allocated to the task, reading the data set corresponding to the data set path, and performing data processing and inference on the read data set to obtain the result summary and output;

[0176] After outputting the result, the processor completing the task is cleaned up of resources.

[0177] That is, the idle processor allocated to the task can read the data set corresponding to the data set path, and perform data processing and inference on the data set to obtain the summary combination and output.

[0178] After outputting the result, the processor completing the task can be cleaned up of resources so that it can be allocated a new inference task in subsequent task allocation.

[0179] It should be noted that after each task ends, the GPU resources can be released for subsequent GPU state detection, so the model can be reloaded after each task allocation; the model loading time is about 1 minute. Among them, the resource cleaning records the output file, such as recording to the record.txt file, and then deletes the output file; clears the model information loaded on the GPU, etc.

[0180] By applying the method provided in the present application, the state information of the processors in the server cluster is obtained, and the idle processor identifiers determined using the state information are written into the target record; in the case that the target record is not empty, the idle processor identifiers are read from the target record one by one and the data set paths are read from the input file one by one until paired idle processor identifiers and data set paths cannot be read; wherein one data set path corresponds to one inference task in use of the model; one idle processor identifier and one data set path read are determined as one pair of allocation information, and one task command is spliced according to each pair of allocation information; the obtained several task commands are written into the output file, and the data set paths read and used in the input file are deleted; based on the output file, inference tasks are allocated to the idle processors in the server cluster.

[0181] In the present application, the state information of the processors in the server cluster capable of implementing distributed execution of a certain model can be acquired first, and then based on the state information, the idle processors can be determined and written into the target record. Then, in the case that the target record is not empty, the idle processor identifiers are read from the target record, and at the same time, the dataset paths are sequentially read from the input file until the paired idle processor identifiers and data and paths cannot be read. That is, in the present application, the dataset paths and idle processor identifiers read are read in pairs, and one dataset path corresponds to one inference task in the model test. In order to effectively complete the inference task allocation, one idle processor identifier and one data path are determined as the allocation information, and then each allocation information is spliced respectively to obtain a plurality of task commands. That is, which idle processor processes which inference task corresponding to which data set path corresponds to one task command. The plurality of task commands are written into the output file, and the dataset paths read and used in the output file are deleted. In this way, based on the output file, the inference tasks can be allocated to the idle processors in the server cluster.

[0182] Technical effect: The present application relates to distributed task distribution, and realizes automatic distribution of inference tasks to the server cluster for the latest or community-supported incomplete model, reduces the learning cost of developers on the distributed framework and the time cost of task allocation and deployment

[0183] It should be noted that based on the above embodiments, the present application embodiments also provide corresponding improvement schemes. In some embodiments, the steps involved in the above embodiments or the corresponding steps can be mutually referenced, and the corresponding beneficial effects can also be mutually referenced. In some embodiments herein, they will not be described one by one.

[0184] In an embodiment of the present application, the state information of the processors in the server cluster is acquired, and the idle processor identifiers determined by using the state information are written into the target record, comprising:

[0185] The state information of the processors in the server cluster is acquired, and the idle processor identifiers determined by using the state information are written into the first record.

[0186] After the interval specified duration, the state information of the processors in the server cluster is acquired, and the idle processor identifiers determined by using the state information are written into the second record.

[0187] The intersection of the first record and the second record is determined as the target record.

[0188] For ease of description, the above steps will be described in combination.

[0189] Wherein, the first and second in the first record, the second record is only used to show the record obtained at different periods.

[0190] For convenience of description, the first record is represented by record 1, the second record is represented by record 2, and the target record is represented by record 3.

[0191] If the GPU is assigned a task, when its state information is obtained, the GPU is found to be loading a model, at this time the state information may be wrong, resulting in its being wrongly identified as belonging to an idle GPU.

[0192] To avoid such cases, in the embodiment, two state information acquisitions are performed, and records 1 and 2 are generated, and then the intersection of the two, that is, record 3, is taken as the basis for subsequent task allocation.

[0193] According to the speed of the GPU loading the model, if there is a GPU loading the model, 5 minutes is enough to make the state of the GPU no longer show idle. That is, the specified time length can be set to 5 minutes, and of course the specified time length can also be set to other time length values in actual application.

[0194] Specifically, in actual application, idle GPU information can be obtained, and the obtained return result is saved in a dictionary, which is named record 1. If no idle GPU is found (i.e., record 1 is empty), it indicates that all GPUs currently have tasks to execute and cannot execute new tasks, and then the program is exited.

[0195] After waiting for 5 minutes (300 seconds), idle GPU information is obtained again, and the obtained return result is saved in a dictionary, which is named record 2. If no idle GPU is found (i.e., record 2 is empty), the program is exited.

[0196] An empty dictionary is created, which is named record 3. Each key-value pair (i.e., {'IP address and number of GPU': 'number of GPU'}) in record 2 is traversed. If the 'IP address and number of GPU' also exists in record 1, it indicates that the GPU of the node is in an idle state obtained by continuous two queries, then the key-value pair is added to record 3. The purpose of continuously calling the first module twice and comparing the return results of the two times is that a certain GPU card has just finished executing the previous task and is executing the initialization of the next task, at this time the GPU may still be in an idle state, but in fact the task has been assigned to the GPU, avoiding task execution failure due to resource competition.

[0197] In an embodiment of the present application, the above task allocation method is implemented, comprising:

[0198] Periodically execute the auto_write_bash_script.py script; the auto_write_bash_script.py script is a program for monitoring idle GPUs and generating a corresponding command script;

[0199] Determine whether new new_bash.sh has been generated; new new_bash.sh is an output file;

[0200] If yes, execute new new_bash.sh, record new new_bash.sh in record.txt, and delete new new_bash.sh after completing the recording.

[0201] If no, exit the program.

[0202] By combining the pdsh and nvidia-smi commands, the idle GPUs can be queried, and then an execution script can be generated, the script content is the command of the distributed inference task that needs to be executed, and finally a shell script is used to periodically execute the above task allocation method process to complete the distribution of the distributed task execution.

[0203] The specific implementation process includes:

[0204] 1. Write a program for monitoring idle GPUs and generating a corresponding command script, named auto_write_bash_script.py (see the following Figure 2 for the flowchart), the script has only two modules, one module is to monitor the idle GPU state, and the other module is to generate a command script file to utilize the idle GPU for task computation.

[0205] Function description of the first module (named: monitor_idle_gpu):

[0206] ① Define the command to be executed, which uses pdsh to execute the nvidia-smi command on the specified host to query the index and used memory of the GPU, and filters out the idle GPU.

[0207] ② Execute the command defined in step ①, and assign the output result to a temporary variable.

[0208] ③ Process the output result and modify its format to list form. The purpose of modifying the format is to facilitate the processing of the second module.

[0209] IV. Traverse the output list, extract the IP address and number of each GPU, and add it as a key-value pair to the dictionary, the key-value pair format of the dictionary is {'IP address and number of GPU': 'number of GPU'}, such as {'127.0.0.1_0': '0', '127.0.0.1_1': '1', '127.0.0.1_2': '2'} indicates that the IP of the server is 127.0.0.1, and the idle GPUs are the 0th, 1st, and 2nd cards of the current server;

[0210] In this way, the key of the dictionary can be used as the name of the subsequent log file, such as 127.0.0.1_0.log, which can well distinguish the execution of each GPU of each node on the task; the number of GPUs is to facilitate the splicing of the command of the inference task. In addition, the purpose of designing the key-value pair of the dictionary in this way is to avoid the complexity of the processing of the second module. Since the naming of the key of the dictionary cannot be repeated, and each node corresponds to multiple GPUs. The data structure selected here is a dictionary, and the format of the dictionary is {'IP address and number of GPU': 'number of GPU'};

[0211] V. For the robustness of the program, add exception capture, if the execution command has an error, capture the exception and print the error information, which is convenient for subsequent problem positioning.

[0212] VI. Return the recorded idle GPU information, that is, return a dictionary, such as {'127.0.0.1_0': '0', '127.0.0.1_1': '1', '127.0.0.2_3': '3', '127.0.0.5_7': '7'} This result shows that the 0th and 1st cards of the 127.0.0.1 server are in idle state, the 3rd card of the 127.0.2 is in idle state, and the 7th card of the 127.0.5 is in idle state.

[0213] Function description of the second module (named: get_real_idle_gpu_info):

[0214] I. Call the first module to get the idle GPU information, and save the returned result in a dictionary named record 1. If no idle GPU is found (i.e. record 1 is empty), it means that all GPUs are currently executing tasks and cannot execute new tasks, so the program exits.

[0215] II. The program waits for 5 minutes (300 seconds); 5 minutes is an empirical time, according to the speed of loading the model on the GPU, if there is a GPU loading the model, 5 minutes is enough to make the state of the GPU no longer show idle.

[0216] ③, Call the first module again to get the idle GPU information, and save the returned result in a dictionary named record 2. If no idle GPU is found (i.e. record 2 is empty), exit the program.

[0217] ④, Create an empty dictionary named record 3. Traverse each key-value pair in record 2 (i.e. {'IP address and number of GPU': 'number of GPU'}), if 'IP address and number of GPU' also exists in record 1, it means that the GPU of this node is idle state after two consecutive queries, then add the key-value pair to record 3. The purpose of calling the first module twice and comparing the results returned twice is: a certain GPU card has just finished the last task and is performing the initialization of the next task, at this time the GPU may still be idle, but in fact the task has been assigned to the GPU, avoiding task execution failure due to resource competition;

[0218] ⑤, Open the input file and the output file; the input file is a txt file, each line can be the absolute path of a data set. The output file is named: new_bash.sh.

[0219] ⑥, Traverse each line of the input file and record 3 at the same time, and splice the command of the inference task according to the data set path of the input file and the node IP and GPU information in record 3. The purpose of simultaneous traversal is to iterate according to the shortest input sequence, that is, according to the shortest one of the input file line number and record 3 length. After traversal, if there is an output file generated, the used information will be deleted from the input file. Among them, suppose infer.py is the script that really executes the inference task.

[0220] Example 1:

[0221] The content of the input file is as follows: / home / 1.txt; / home / 2.txt; / home / 3.txt;

[0222] The data content of record 3 is as follows: {'127.0.0.1_0':'0'};

[0223] Process the key value of record 3, then splice the input file information, and the output file content after splicing is as follows:

[0224] pdsh -w 127.0.0.1 -R ssh -f 500"CUDA_VISIBLE_DEVICES=0 python infer.py --input_file / home / 1.txt --output_file / home / 1.txt_res 2>&1 | tee. / log / 127_0_0_1_0.log" &

[0225] After the loop ends, the input file content is: / home / 2.txt; / home / 3.txt;

[0226] Example 2:

[0227] The input file content is as follows: / home / 1.txt; / home / 2.txt; / home / 3.txt;

[0228] The data content of record 3 is as follows: {'127.0.0.1_0': '0', '127.0.0.1_1': '1', '127.0.0.1_2': '2', '127.0.0.1_3': '3'};

[0229] The key value of record 3 is processed, and then the input file information is spliced, and the output file content after splicing is as follows:

[0230] pdsh -w 127.0.0.1 -R ssh -f 500"CUDA_VISIBLE_DEVICES=0 python infer.py --input_file / home / 1.txt --output_file / home / 1.txt_res 2>&1 | tee. / log / 127_0_0_1_0.log" &

[0231] pdsh -w 127.0.0.1 -R ssh -f 500"CUDA_VISIBLE_DEVICES=1 python infer.py --input_file / home / 2.txt --output_file / home / 2.txt_res 2>&1 | tee. / log / 127_0_0_1_1.log" &

[0232] pdsh -w 127.0.0.1 -R ssh -f 500"CUDA_VISIBLE_DEVICES=2 python infer.py --input_file / home / 3.txt --output_file / home / 3.txt_res 2>&1 | tee. / log / 127_0_0_1_2.log" &

[0233] After the loop, the input file content is empty, and the 3rd GPU card of 127.0.0.1 is still in idle state.

[0234] Example 3:

[0235] The input file content is as follows: / home / 1.txt; / home / 2.txt; / home / 3.txt;

[0236] The data content of record 3 is as follows: {":"};

[0237] There is no output file at this time, i.e. no new_bash.sh;

[0238] After the loop, the input file content is: / home / 1.txt; / home / 2.txt; / home / 3.txt;

[0239] 2. Write a shell script named auto_submit.sh (see flowchart in Figure 3). The script is a timing execution script, which is executed every minute. The execution content is:

[0240] ①. First execute auto_write_bash_script.py;

[0241] ②. Check if the new_bash.sh file is generated. If the file is generated, execute the new_bash.sh script directly, and record the file content to a record file, such as recored.txt, for later problem positioning. After recording, delete the new_bash.sh file, and then the auto_submit.sh script exits, waiting for the next execution;

[0242] ③. If no new_bash.sh file is generated, the auto_submit.sh script exits, waiting for the next execution;

[0243] For example, Qwen1.5 is a test version of the model, and its peripheral support is basically 0, which cannot perform multi-node inference tasks, and the model does not provide the use method of the megatron distributed framework. Under the task requirements of evaluating the model, due to the shortage of GPU resources, the window period is short, and the inference task needs to be completed in a specific short time. Using the task allocation method provided in the application, without any adaptation of the model and without learning any distributed framework, the model can be directly loaded into data for distributed inference tasks. The learning cost and related time cost of the developer on the distributed framework can be effectively reduced.

[0244] Corresponding to the above method embodiments, the embodiments of the application also provide a task allocation device. The task allocation device described below can be mutually corresponding and referred to with the task allocation method described above.

[0245] Referring to FIG. 4, the device includes the following modules:

[0246] The state acquisition module 101 is configured to acquire state information of processors in a server cluster, and write an idle processor identifier determined by using the state information into a target record;

[0247] The information reading module 102 is configured to, in a case where the target record is not empty, read idle processor identifiers and dataset paths from the input file in sequence from the target record until a pair of idle processor identifiers and dataset paths cannot be read; wherein one dataset path corresponds to one inference task in use of the model;

[0248] The task command determination module 103 is configured to determine one idle processor identifier and one dataset path read to be a pair of allocation information, and splice a task command according to each pair of allocation information;

[0249] The output file determination module 104 is configured to write the spliced task commands into an output file, and delete the dataset paths read and used in the input file;

[0250] The task allocation module 105 is configured to allocate inference tasks to idle processors in the server cluster based on the output file.

[0251] The device provided in the embodiment of the application is used for obtaining the state information of the processors in the server cluster, and writing the idle processor identifiers determined by using the state information into the target record; in the case that the target record is not empty, the idle processor identifiers are read from the target record in sequence and the dataset paths are read from the input file in sequence until a pair of idle processor identifier and dataset path cannot be read; wherein one dataset path corresponds to one inference task in the model use; one idle processor identifier and one dataset path read are determined as one pair of allocation information, and one task command is spliced according to each pair of allocation information; the plurality of task commands obtained by splicing are written into the output file, and the dataset paths used by reading are deleted in the input file; and the inference tasks are allocated to the idle processors in the server cluster based on the output file.

[0252] In the application, the state information of the processors in the server cluster capable of implementing the distributed execution of a certain model can be obtained first, and then the idle processors can be determined based on the state information and written into the target record. Then, in the case that the target record is not empty, the idle processor identifiers are read from the target record, and at the same time, the dataset paths are read from the input file in sequence until a pair of idle processor identifier and dataset path cannot be read. That is, in the application, the dataset paths and the idle processor identifiers are read in pairs, and one dataset path corresponds to one inference task in the model test. In order to effectively complete the inference task allocation, one idle processor identifier and one dataset path are determined as one pair of allocation information, and each allocation information is spliced to obtain a plurality of task commands. That is, which idle processor processes which inference task corresponding to which dataset path is corresponding to one task command. The plurality of task commands can be written into the output file, and the dataset paths used by reading in the output file are deleted. In this way, the inference tasks can be allocated to the idle processors in the server cluster based on the output file.

[0253] Technical effects: The application relates to distributed task distribution, and automatically distributes inference tasks to the server cluster for the latest or community-supported incomplete model, reduces the learning cost of the developers for the distributed framework and the time cost of task allocation and deployment.

[0254] In one specific embodiment of the application, the state acquisition module is specifically configured to obtain the state information of the processors in the server cluster, and write the idle processor identifiers determined by using the state information into the first record;

[0255] After the interval specified duration, the state information of the processors in the server cluster is obtained, and the idle processor identifiers determined by using the state information are written into the second record;

[0256] An intersection of the first record and the second record is determined as a target record.

[0257] In an embodiment of the present application, the state information includes a processor usage rate, and the state acquisition module is specifically configured to obtain the processor usage rate by invoking a processor state checking command line tool using a parallel distributed operation and maintenance tool.

[0258] A processor with a processor usage rate lower than a preset threshold is determined as an idle processor.

[0259] An IP address and a number of the idle processor are determined as information in an idle processor identifier.

[0260] The idle processor identifier is written into the target record.

[0261] In an embodiment of the present application, the state acquisition module is specifically configured to write the IP address and the number in the idle processor identifier into the target record in the form of a key-value pair.

[0262] In an embodiment of the present application, the state acquisition module is further configured to output prompt information of a capture exception if the processor usage rate is not obtained after the processor state checking command line tool is invoked using the parallel distributed operation and maintenance tool.

[0263] In an embodiment of the present application, the state acquisition module is specifically configured to take the IP address and the number of the idle processor as a key and take the number of the idle processor as a value.

[0264] The key and the value are written into the target record in a preset key-value pair format; and the IP address corresponds to a node to which the idle processor belongs.

[0265] In an embodiment of the present application, the information reading module is specifically configured to determine whether the target record is empty.

[0266] If yes, a reasoning task allocation failure prompt information is output.

[0267] If no, a data set path is read from the input file each time an idle processor identifier is read from the target record, until a pair of idle processor identifier and data set path cannot be read.

[0268] In an embodiment of the present application, the state acquisition module is specifically configured to periodically acquire state information of processors in a server cluster.

[0269] Or, in a case where an input file is obtained, the state information of the processors in the server cluster is acquired.

[0270] Or, in the case of an empty input file, obtain the state information of the processors in the server cluster.

[0271] In an embodiment of the present application, the output file determination module is specifically configured to sequentially write a plurality of task commands in the output file.

[0272] The dataset path that has been read and used to splice the task command is deleted from the input file.

[0273] In an embodiment of the present application, the task allocation module is specifically configured to allocate an inference task to an idle processor in the server cluster by executing the task command in the output file.

[0274] In an embodiment of the present application, the method further comprises:

[0275] The resource cleaning module is configured to read a dataset corresponding to the dataset path by using the idle processor allocated to the task, and perform data processing and inference on the read dataset to obtain a result summary and output.

[0276] After the output result, the processor that completes the task is subjected to resource cleaning.

[0277] In an embodiment of the present application, the task command determination module is specifically configured to determine, in the order of reading, one idle processor identifier read at the same time and one dataset path read at the same time as one pair of allocation information.

[0278] According to a standard format of a preset command line, each pair of allocation information is spliced to obtain a plurality of task commands.

[0279] In an embodiment of the present application, the processor is a GPU, and the idle processor identifier includes an IP address and a number of the GPU; the task command determination module is specifically configured to write an absolute address in each pair of allocation information and an IP and a number of the idle GPU into corresponding positions of the standard format of the preset command line to obtain a plurality of task commands.

[0280] The standard format of the preset command line is: pdsh -w TARGET_IP –R ssh –f PORT"CUDA_VISIBLE_DEVICES=gpu_num python infer.py--input_file inputfile.txt--output_file outputfile.txt 2>&1|tee. / log / TARGET_IP.log"&.

[0281] Pdsh represents a parallel distributed operation and maintenance tool.

[0282] -w TARGET_IP, indicating the target server IP;

[0283] -R ssh, indicating that the secure shell protocol is specified to be used;

[0284] -f, indicating that the remote command of the secure shell protocol is executed in the background;

[0285] PORT, indicating the port number accessed by the secure shell protocol;

[0286] The command inside the quotation marks is the command executed by the parallel distributed operation and maintenance tool on the target server:

[0287] CUDA_VISIBLE_DEVICES=gpu_num, indicating the GPU number of the current server;

[0288] python infer.py, indicating the model inference script executed;

[0289] --input_file, indicating the input file;

[0290] --output_file, indicating the output file;

[0291] inputfile.txt, indicating the output file list;

[0292] 2>&1|tee. / log / log.log, indicating that the log is printed on the screen and written into the. / log / TARGET_IP.log file;

[0293] &, indicating that the current command is put into the background for execution.

[0294] Corresponding to the above method embodiment, the embodiments of the present application also provide an electronic device. The electronic device described below can be mutually corresponding with reference to the task allocation method described above.

[0295] Referring to FIG. 5, the electronic device includes:

[0296] a memory 332 for storing a computer program;

[0297] a processor 322 for executing the computer program to implement the steps of the task allocation method of the above method embodiment.

[0298] Specifically, please refer to FIG. 6, which is a schematic diagram of a specific structure of an electronic device provided in the embodiment. The electronic device can be quite different in configuration or performance, and can include one or more processors (central processing units, CPUs) (for example, one or more processors) and a memory 332 storing one or more computer programs 342 or data 344. The memory 332 can be temporary storage or persistent storage. The programs stored in the memory 332 can include one or more modules (not shown in the figure), each of which can include a series of instruction operations in the data processing device. Further, the processor 322 can be configured to communicate with the memory 332 and execute the series of instruction operations in the memory 332 on the electronic device 301.

[0299] The electronic device 301 can also include one or more power supplies 326, one or more wired or wireless network interfaces 350, one or more input / output interfaces 358, and / or one or more operating systems 341.

[0300] The steps in the task allocation method described above can be implemented by the structure of the electronic device.

[0301] Corresponding to the above method embodiment, the embodiment of the application also provides a non-volatile readable storage medium. The non-volatile readable storage medium described below can be mutually corresponding with reference to the task allocation method described above.

[0302] A non-volatile readable storage medium, the non-volatile readable storage medium storing a computer program, the computer program being executed by a processor to implement the steps of the task allocation method of the above method embodiment.

[0303] The non-volatile readable storage medium can be a U disk, a mobile hard disk, a read-only memory (Read-Only Memory, ROM), a random access memory (Random Access Memory, RAM), a magnetic disk or an optical disk, and various non-volatile readable storage media that can store program codes.

[0304] Corresponding to the above method embodiment, the embodiment of the application also provides a computer program product. The computer program product described below can be mutually corresponding with reference to the task allocation method described above.

[0305] A computer program product, including computer programs / instructions, when executed by a processor, implements the steps of the above task allocation method.

[0306] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the apparatus disclosed in the embodiments, since it corresponds to the method disclosed in the embodiments, the description is relatively simple; relevant parts can be referred to in the method section.

[0307] Those skilled in the art will further recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0308] The steps of the methods or algorithms described in conjunction with the embodiments disclosed herein can be implemented directly by hardware, a software module executed by a processor, or a combination of both. The software module can be located in random access memory (RAM), main memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, removable disk, CD-ROM, or any other form of storage medium known in the art.

[0309] Finally, it should be noted that in this document, relationships such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "include," "contain," or any other variations are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus.

[0310] This document uses specific examples to illustrate the principles and implementation methods of this application. The descriptions of the above embodiments are only for the purpose of helping to understand the methods and core ideas of this application. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of this application. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. A method of task allocation, characterized by, The method comprises the following steps: acquiring state information of processors in a server cluster, and writing idle processor identifiers determined by using the state information into a target record; in the case that the target record is not empty, reading idle processor identifiers from the target record and dataset paths from an input file in sequence until paired idle processor identifiers and dataset paths cannot be read; wherein one of the dataset paths corresponds to one inference task in model use; determining one idle processor identifier and one dataset path read as one pair of allocation information, and splicing one task command according to each pair of the allocation information; writing a plurality of the task commands spliced into an output file, and deleting the dataset paths used in the input file; based on the output file, allocating the inference task to idle processors in the server cluster.

2. The method of claim 1, wherein, The method comprises the following steps: acquiring state information of processors in a server cluster, and writing idle processor identifiers determined by using the state information into a target record; acquiring state information of processors in the server cluster, and writing idle processor identifiers determined by using the state information into a first record; after a specified time interval, acquiring state information of processors in the server cluster, and writing idle processor identifiers determined by using the state information into a second record; 3. The method of claim 1, wherein, determining the intersection of the first record and the second record as the target record. The method comprises the following steps: using a parallel distributed operation and maintenance tool, calling a processor state checking command line tool to obtain processor usage; determining processors with processor usage lower than a preset threshold as idle processors; determining IP addresses and numbers of the idle processors as information in idle processor identifiers; 4. The method of claim 3, wherein, writing the idle processor identifiers into the target record. The method comprises the following steps:

5. The method of claim 4, wherein, writing IP addresses and numbers in the idle processor identifiers into the target record in the form of key-value pairs. After using a parallel distributed operation and maintenance tool to call a processor state checking command line tool, the method further comprises the following steps:

6. The method of claim 4, wherein, in the case that the processor usage is not obtained, outputting prompt information for capturing an exception. The method comprises the following steps: taking the IP addresses and numbers of the idle processors as keys, and taking the numbers of the idle processors as values; 7. The method of claim 1, wherein, writing the keys and the values into the target record in a preset key-value pair format; wherein the IP addresses correspond to nodes to which the idle processors belong. The method comprises the following steps: determining whether the target record is empty; If the target record is empty, output a reasoning task allocation failure prompt information; If the target record is not empty, read a data set path from the input file for each idle processor identifier read from the target record until a pair of idle processor identifier and data set path cannot be read.

8. The method of claim 1, wherein, Obtaining state information of processors in a server cluster, comprising: Periodically obtaining state information of processors in the server cluster; Or, in the case of obtaining the input file, obtaining state information of processors in the server cluster; Or, in the case of the input file being not empty, obtaining state information of processors in the server cluster.

9. The method of claim 1, wherein, Writing the spliced task commands into an output file, and deleting the data set paths read and used in the input file, comprising: Writing the task commands into the output file in sequence; Deleting the data set paths read and used in the task command splicing from the input file.

10. The method of claim 1, wherein, Based on the output file, allocating the reasoning task to the idle processors in the server cluster, comprising: Allocating the reasoning task to the idle processors in the server cluster by executing the task commands in the output file.

11. The method of claim 10, wherein, Further comprising: Using the idle processors allocated to the task to read the data set corresponding to the data set path, and performing data processing and reasoning on the read data set to obtain the result summary and output; After outputting the result, cleaning up the resources of the processors completing the task.

12. The method according to any one of claims 1 to 11, characterized in that, Determining one idle processor identifier and one data set path as a pair of allocation information, and splicing a task command according to each pair of the allocation information; According to the reading order, determining one idle processor identifier and one data set path read at the same time as a pair of the allocation information; According to the standard format of the preset command line, splicing each pair of the allocation information to obtain a plurality of task commands.

13. The method of claim 1, wherein, The obtaining state information of processors in a server cluster, comprising: Executing a preset command for obtaining processor state information to obtain state information of each processor in the server cluster.

14. The method of claim 1, wherein, The obtaining state information of processors in a server cluster, comprising: Running a preset tool for obtaining processor state information to obtain state information of each processor in the server cluster.

15. The method of claim 1, wherein, The obtaining state information of processors in a server cluster, comprising: Receiving the state information of the processors periodically reported by the processors in the server cluster.

16. The method of claim 1, wherein, The determination method of the idle processor is: When the processor utilization of the target processor is lower than the preset utilization threshold, the target processor is determined as an idle processor.

17. A task allocation apparatus characterized by comprising: Comprising: A state acquisition module for obtaining state information of processors in a server cluster, and writing idle processor identifiers determined using the state information into a target record; The information reading module is configured to, when the target record is not empty, read idle processor identifiers from the target record and dataset paths from the input file in sequence until a pair of idle processor identifier and dataset path cannot be read; wherein one of the dataset paths corresponds to one inference task in model use; The task command determining module is configured to determine one idle processor identifier and one dataset path as one pair of allocation information, and splice one task command according to each pair of the allocation information; The output file determining module is configured to write the spliced task commands to an output file, and delete the dataset paths that have been read and used in the input file; The task allocation module is configured to allocate the inference tasks to idle processors in the server cluster based on the output file.

18. An electronic device, comprising: The computer program is stored in the memory and executed by the processor to implement the steps of the task allocation method according to any one of claims 1 to 16. The computer program is stored in the non-volatile readable storage medium and executed by the processor to implement the steps of the task allocation method according to any one of claims 1 to 16. The computer program / instruction is executed by the processor to implement the steps of the task allocation method according to any one of claims 1 to 16.

19. A non-volatile readable storage medium, characterized by ​ 20. A computer program product comprising computer programs / instructions, characterized in that, ​

Images