A software distributed task allocation method, device, equipment and storage medium

Abstract

The present invention discloses a software distributed task allocation method, device, equipment and storage medium, which are applied to the field of electronic technology. The method comprises: inputting the characteristic information of the distributed task into the generated machine learning model to obtain the running time of the distributed task; determining according to the preset rules that the long-time distributed tasks not greater than the number of computing units are stored in the computing unit for calculation; when the computing unit is idle, storing the remaining distributed tasks equal to the number of idle computing units in the computing unit for calculation. The method can give priority to calculating the long-time distributed tasks to reduce the tailing effect, thereby shortening the overall running time and improving the global operation efficiency.

Description

Technical Field

[0001] The present invention relates to the field of electronic technology, and in particular to a software distributed task allocation method, device, equipment and computer-readable storage medium. Background Art

[0002] At present, the allocation principle of distributed tasks in software is often relatively simple, and the calculation completion time of different distributed tasks will vary greatly. In some cases, long-term distributed tasks will be calculated last, which will make their running time much longer than other tasks, causing a tailing effect, thereby increasing the overall running time and reducing the global operating efficiency.

[0003] Therefore, how to reasonably allocate distributed tasks to eliminate the tailing effect, thereby shortening the overall running time and improving the global operating efficiency is a technical problem that technical personnel in this field need to solve. Summary of the invention

[0004] The purpose of the present invention is to provide a software distributed task allocation method, which can solve the tailing effect generated in the process of distributed task processing, shorten the overall running time, and improve the global running efficiency. The present invention also provides a software distributed task allocation device, equipment and computer-readable storage medium.

[0005] In order to solve the above technical problems, the present invention provides a software distributed task allocation method, comprising:

[0006] Input the feature information of the distributed task into the generated machine learning model to obtain the running time of the distributed task;

[0007] According to the preset rules, long-term distributed tasks that are not larger than the number of computing units are determined and stored in the computing units for calculation;

[0008] When there are idle computing units, the remaining distributed tasks equal to the number of idle computing units are stored in the computing units for calculation.

[0009] Optionally, determining according to a preset rule that the long-duration distributed tasks not greater than the number of computing units are stored in the computing units for computing includes:

[0010] Distributed tasks whose running time is greater than or equal to a first threshold are stored in the computing unit for computing.

[0011] Optionally, before storing the remaining distributed tasks equal to the number of idle computing units into the computing units for computing, the process further includes:

[0012] The distributed tasks whose running time is greater than or equal to the second threshold among the remaining distributed tasks are stored in a slow buffer, and the distributed tasks whose running time is less than the second threshold are stored in a fast buffer.

[0013] Optionally, storing the distributed tasks whose running time is greater than or equal to the second threshold among the remaining distributed tasks into the slow buffer includes:

[0014] The distributed tasks whose running time is greater than or equal to the second threshold and less than the third threshold among the remaining distributed tasks are stored in the slow buffer, and the distributed tasks whose running time is greater than or equal to the third threshold are stored in the non-buffer.

[0015] Optionally, when the computing unit is idle, storing the remaining distributed tasks equal to the number of idle computing units into the computing unit for computing, including:

[0016] When there are idle computing units, the distributed tasks in the non-buffer equal to the number of idle computing units are stored in the computing units; when the distributed tasks in the non-buffer are calculated, the distributed tasks in the slow buffer equal to the number of idle computing units are stored in the computing units; when the distributed tasks in the slow buffer are calculated, the distributed tasks in the fast buffer equal to the number of idle computing units are stored in the computing units.

[0017] Optionally, the feature information of the distributed task is input into the generated machine learning model to obtain the running time of the distributed task, including:

[0018] The feature information of the distributed task is input into a machine learning model generated corresponding to the feature information to obtain the running time of the distributed task, wherein the machine learning model includes a linear function model and a two-layer neural network model.

[0019] Optionally, inputting the feature information of the distributed task into a machine learning model generated corresponding to the feature information includes:

[0020] The feature information of the distributed tasks of the chip manufacturing software is input into a machine learning model generated corresponding to the feature information, wherein the feature information includes one or more of the number of polygons, the total number of vertices, and the overall coverage area and coverage rate.

[0021] The present invention also provides a software distributed task allocation device, comprising:

[0022] A running time calculation module is used to input the characteristic information of the distributed task into the generated machine learning model to obtain the running time of the distributed task;

[0023] The allocation calculation module is used to determine according to preset rules that long-term distributed tasks that are no greater than the number of computing units are stored in the computing units for calculation; when the computing units are idle, the remaining distributed tasks equal to the number of idle computing units are stored in the computing units for calculation.

[0024] The present invention also provides a software distributed task allocation device, comprising:

[0025] Memory for storing computer programs;

[0026] A processor is used to implement the steps of a software distributed task allocation method as described in any one of the above when executing the computer program.

[0027] The present invention also provides a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the steps of the method for distributed task allocation of software as described in any one of the above items are implemented.

[0028] The present invention provides a distributed task allocation method for software, which inputs characteristic information of distributed tasks into a generated machine learning model to obtain the running time of the distributed tasks; according to preset rules, long-time distributed tasks that are no greater than the number of computing units are stored in the computing units for calculation; when the computing units are idle, the remaining distributed tasks equal to the number of idle computing units are stored in the computing units for calculation.

[0029] The beneficial effect of the present invention is that the characteristic information of the distributed task is input into the generated machine learning model, the reasoning time of the machine learning model is very short relative to the processing time of the distributed task, the reasoning process is short and very efficient, that is, the method selects the machine learning model to calculate the running time, can quickly and efficiently obtain the running time of the distributed task, and according to the preset rules, the long-term distributed tasks that are not greater than the number of computing units are determined to be stored in the computing unit for calculation; when the computing unit is idle, the remaining distributed tasks equal to the number of idle computing units are stored in the computing unit for calculation, such an allocation method can realize the calculation of the long-term distributed tasks first, and in the process, other short-term tasks are calculated in parallel, so that the time of the last calculated distributed task will be relatively short, thereby reducing the tailing effect, shortening the overall running time, and improving the global operation efficiency. In addition, the present invention also provides a software distributed task allocation device, equipment and computer-readable storage medium, which also have the above-mentioned beneficial effects. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying creative work.

[0031] Figure 1 A flowchart of a software distributed task allocation method provided by an embodiment of the present invention;

[0032] Figure 2 A flowchart of another software distributed task allocation method provided by an embodiment of the present invention;

[0033] Figure 3 A schematic diagram of a two-layer neural network provided by an embodiment of the present invention;

[0034] Figure 4 An example flow chart of distributed task allocation of a chip manufacturing software provided by an embodiment of the present invention;

[0035] Figure 5 A schematic diagram of the structure of a software distributed task allocation device provided by an embodiment of the present invention;

[0036] Figure 6 A schematic diagram of the structure of a software distributed task allocation device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0037] The core of the present invention is to provide a distributed task allocation method for software, which can calculate distributed tasks according to preset rules, give priority to calculating long-time distributed tasks, and finally calculate the distributed tasks in a relatively short time, so as to reduce or even eliminate the tailing effect, thereby shortening the overall running time and improving the global running efficiency; the present invention also provides a distributed task allocation device, equipment and computer-readable storage medium for software.

[0038] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0039] At present, various large-scale tasks (such as large chip manufacturing EDA (electronic design automation) software running design rule detection, optical proximity effect correction and other functional tasks) often use distributed parallel computing to reduce the overall running time and improve the overall operating efficiency. However, in the distribution of distributed tasks, the distribution principle is often relatively simple. For example, the design layout is divided into equal parts according to coordinates, and batches are distributed to distributed computing units for calculation in a fixed order. Due to the different complexity of the task slice layouts assigned to each computing unit in the chip design layout, the calculation completion time of different distributed tasks will vary greatly. In some cases, the last batch of tasks distributed contains tasks with longer running times, making their running time much longer than other tasks, resulting in a tailing effect that increases the overall running time and reduces the global operating efficiency. To solve the above technical problems, please refer to Figure 1 , Figure 1 A flowchart of distributed distribution of software is provided for an embodiment of the present invention. The method may include:

[0040] Step S100: Input the characteristic information of the distributed task into the generated machine learning model to obtain the running time of the distributed task.

[0041] In software such as data processing, image recognition, and chip manufacturing, distributed task allocation methods are often used to process tasks to improve efficiency. Distributed tasks refer to the process in which tasks are decomposed and assigned to multiple physically or logically separated computing nodes in a distributed system. This task allocation method can improve the scalability, availability, and fault tolerance of the system. When the software runs tasks, distributed parallel computing is often used to reduce the overall running time and improve computing efficiency. For example, large-scale data processing software requires distributed task scheduling to process large amounts of data in parallel to improve efficiency; in image recognition software, distributed task allocation is required to process images to improve the efficiency of image recognition; chip manufacturing software will also apply distributed tasks to distribute slice layout information to computing units for parallel processing. This embodiment does not limit the type of software distributed tasks. It can be a distributed task of data processing software or a distributed task of chip manufacturing software, as long as the application of the present invention can be implemented to distribute distributed tasks.

[0042] It is understandable that this embodiment does not limit the feature information of distributed tasks, and different types of distributed tasks correspond to different feature information. For example, the feature information of distributed tasks of image recognition software includes area, edge features, resolution, etc.; the feature information of chip manufacturing software includes one or more of the number of polygons, the total number of vertices, and the overall coverage area and coverage rate.

[0043] Further, in order to improve the reasoning efficiency of the running time of the distributed tasks, before the distributed tasks are assigned, the machine learning model is trained to process the input feature information of the distributed tasks to obtain the running time of the corresponding distributed tasks. To generate the machine learning model, the machine learning model should be fitted according to the feature information and running time of the distributed tasks actually collected. Since the present embodiment does not limit the type of distributed tasks, the type and training process of the machine learning model are not limited. Different distributed tasks can correspond to different machine learning models. In the training process, there is no restriction on the number of input feature information, which can be one or more of the feature information, as long as the feature information of the generated machine learning model and the feature information of the input distributed task running time correspond to each other. For example, for low-complexity computing tasks such as design rule detection in chip manufacturing software, a linear function model can be selected for fitting, and the actually collected feature information such as the number of vertices can be input into the selected linear function model for training to obtain a linear function model corresponding to the number of vertices of this distributed task; for highly complex tasks such as optical proximity effect correction, a decision tree model can be selected, and the actually collected feature information such as the number of polygons can be input into the selected decision tree model for training to obtain a decision tree model corresponding to the number of polygons of this task. That is, the model can be selected for training according to the complexity of the distribution task.

[0044] Furthermore, this embodiment does not limit the number of models included in the machine learning model, and it can be selected according to the characteristics of the distributed task. The machine learning model may only include a learning model, such as a neural network model or a decision tree model; of course, it can also be a combination of multiple models such as a learning model and a linear function model to shorten the inference time of the running time of the distributed task as much as possible. For example, a linear model can be selected for the task of design rule detection, and a decision tree model can be selected for the task of optical proximity effect correction. Since the complexity of the neural network model inference model is simpler and more efficient than other machine learning models, and the neural network model can also be directly applied to linear fitting, the neural network model can also be applied to these two tasks. This embodiment does not limit the number of layers of the neural network model. Preferably, in order to comprehensively consider the reliability and computational efficiency of the running time of the distributed task, a two-layer neural network model can be selected. The schematic diagram of the two-layer neural network model can be referenced. Figure 3 , where x1, x2, x3 are inputs and y is the output.

[0045] Compared with distributed tasks that take tens of seconds to several minutes to run in the computing unit, the inference time of the above models is in milliseconds. For most distributed tasks, the running time of the distributed tasks is much longer than the time required for inference running time. The addition of the running time calculation of the distributed tasks has basically no negative impact on the time consumption of the distributed process itself. Therefore, using the machine learning model to calculate the running time of the distributed tasks before assigning them will not cause unnecessary time consumption, and the running time of the corresponding feature information can be inferred to facilitate the following operations. That is, data fitting is performed based on the characteristic information of the distributed tasks, so as to use the machine learning model to predict their running time, and then the long-term distributed tasks are entered into the computing unit first, and the short-term distributed tasks are calculated later, thereby reducing or eliminating the tailing effect.

[0046] Step S101: Determine, according to a preset rule, that the number of long-term distributed tasks not greater than the number of computing units is stored in the computing units for computing.

[0047] It is understandable that in this embodiment, long-time tasks can be stored in the calculation unit for calculation first, so as to prevent the long-time tasks from being calculated at the end and causing a tailing effect. This embodiment does not limit the preset rules, and can be set according to actual conditions. For example, a threshold can be set, and distributed tasks that are not greater than the number of calculation units are selected from the distributed tasks greater than the threshold and stored in the calculation unit for processing; or the running time of the distributed tasks can be sorted in order from long to short, and the long-time distributed tasks that are not greater than the number of calculation units and ranked at the top are taken out and stored in the calculation unit for calculation. As long as the preset rules set can select the long-time distributed tasks from all tasks and store them in the calculation unit for calculation first. Furthermore, there is no limitation on the specific time of the long-time distributed tasks. After the running time of all distributed tasks is calculated, the running time is greater than the running time of other distributed tasks, or if the distributed task is calculated at the end and may produce a tailing effect, such tasks can be called long-time tasks.

[0048] Furthermore, this embodiment does not limit the number of distributed tasks stored in the computing unit. Among them, the long-term distributed tasks that are determined to be no greater than the number of computing units can be determined based on preset rules and running time to determine the specific number of computing units to be stored. For example, if there are 10 computing units, if a time threshold is set and it is found that there are 15 distributed tasks greater than the threshold, then 10 long-term distributed tasks can be stored in the computing unit for calculation, and the remaining tasks can be stored when the computing unit is idle; if there are 8 long-term distributed tasks, then these 8 long-term tasks can be stored in the computing unit for calculation, and 2 distributed tasks can be randomly selected from the remaining tasks and stored in the idle computing unit for calculation. If the running time of the distributed tasks is sorted from long to short, it can also be calculated according to the above method. If there are 10 computing units, 10 distributed tasks can be taken from long to short in time, or the top 8 distributed tasks can be taken out and stored in the computing unit for calculation.

[0049] Step S102: when there are idle computing units, the remaining distributed tasks equal to the number of idle computing units are stored in the computing units for calculation.

[0050] When there are distributed tasks that have been completed in the computing unit, the distributed tasks that have been completed are collected. If there are idle computing units at this time, the tasks to be processed equal to the number of idle computing units are stored in the computing unit for calculation.

[0051] It is understandable that after the computing unit stores the distributed tasks, the distributed tasks are processed in parallel. When there are idle computing units, the remaining distributed tasks will be stored in the idle computing units for calculation to improve processing efficiency. This embodiment does not limit the order in which the remaining distributed tasks are stored in the computing units, and the user can choose according to the actual situation. For example, the distributed tasks corresponding to the number of idle computing units can be randomly selected from the remaining distributed tasks and stored in the computing unit for calculation; a buffer can also be set, and the buffer and the remaining distributed tasks can be classified according to the time threshold, and the remaining distributed tasks can be stored in the buffer. It is also possible to select distributed tasks with a longer time and store them in the computing unit for calculation according to different time thresholds, and then select distributed tasks with a shorter time and store them in the computing unit for calculation. The number of selected distributed tasks must be equal to the number of idle computing units.

[0052] It should be noted that this embodiment does not limit the number of buffers to be set, nor does it limit the time threshold for storing the running time of distributed tasks in each buffer. For example, when there are two buffers, the distributed tasks whose running time is greater than or equal to the second threshold among the remaining distributed tasks can be stored in the slow buffer, and the distributed tasks whose running time is less than the second threshold can be stored in the fast buffer.

[0053] Furthermore, the present embodiment may also set up a non-buffer for storing distributed tasks whose running time is greater than the buffer, so as to achieve the advance calculation of distributed tasks with longer running time as much as possible, thereby avoiding the appearance of tasks with too long running time in the last batch. For example, the distributed tasks whose running time is greater than or equal to the second threshold among the remaining distributed tasks are stored in the slow buffer, and the distributed tasks whose running time is less than the second threshold are stored in the fast buffer. The distributed tasks whose running time is greater than or equal to the second threshold and less than the third threshold among the remaining distributed tasks are stored in the slow buffer, and the distributed tasks whose running time is greater than or equal to the third threshold are stored in the non-buffer. Correspondingly, when there are idle computing units, the distributed tasks in the non-buffer equal to the number of idle computing units are stored in the computing units; when the distributed tasks in the non-buffer are calculated, the distributed tasks in the slow buffer equal to the number of idle computing units are stored in the computing units; when the distributed tasks in the slow buffer are calculated, the distributed tasks in the fast buffer equal to the number of idle computing units are stored in the computing units.

[0054] This embodiment uses a machine learning model to infer the running time of distributed tasks. The inference time of the machine learning model is very short and will not affect the processing flow of distributed tasks. Therefore, the use of a machine learning model to infer the running time can improve the efficiency of subsequent distributed task allocation; long-term tasks are stored in the computing unit for priority calculation, and the remaining tasks with relatively short time are stored in the computing unit for calculation when there are idle computing units. Long-term distributed tasks can be calculated first, and the distributed tasks calculated at the end have shorter time, which can eliminate the tailing effect, reduce the overall running time, and improve the global operation efficiency.

[0055] Please refer to Figure 2 , Figure 2 A flowchart of another software distributed task allocation method provided by an embodiment of the present invention. The method may include:

[0056] Step S200: Input the characteristic information of the distributed task into the generated machine learning model to obtain the running time of the distributed task.

[0057] It can be understood that step S200 is the same as the corresponding step S100 in the above embodiment, and will not be described in detail here.

[0058] Furthermore, in order to more quickly classify and store the remaining distributed tasks into computing units, and reduce or even eliminate the distributed tasks that produce the tailing effect in the last few batches of calculations, when there are idle computing units, before storing the remaining distributed tasks equal to the number of idle computing units into the computing units for calculation, the following steps may be included:

[0059] Step S201: storing the distributed tasks whose running time is greater than or equal to a first threshold into the computing unit for computing.

[0060] Step S202: among the remaining distributed tasks, the distributed tasks whose running time is greater than or equal to the second threshold are stored in a slow buffer, and the distributed tasks whose running time is less than the second threshold are stored in a fast buffer.

[0061] Among them, this embodiment does not limit the specific values ​​of the first threshold and the second threshold. The first threshold and the second threshold can be set according to actual conditions, and the first threshold is greater than the second threshold. Furthermore, distributed tasks greater than or equal to the second threshold can be called slow tasks, and distributed tasks with a running time less than the second threshold can be called fast tasks. Slow tasks are stored in a slow buffer, and fast tasks are stored in a fast buffer. It should be noted that this embodiment does not limit the number of buffers, which can also be understood as the number of types of distributed tasks. The number of types of distributed tasks can be determined based on the usage scenario. For example, distributed tasks can be divided into fast tasks, medium-speed tasks, and slow tasks, and the processing order of distributed tasks can be adjusted accordingly.

[0062] Step S203: storing the distributed tasks whose running time is greater than or equal to the second threshold and less than the third threshold in the slow buffer, and storing the distributed tasks whose running time is greater than or equal to the third threshold in the non-buffer.

[0063] Among them, this embodiment does not limit the specific value of the third threshold value, and the third threshold value can be set according to actual conditions, and the third threshold value is greater than the second threshold value. When the running time of the remaining distributed tasks is greater than or equal to the second threshold value, the tasks are stored in the slow buffer, and the slow buffer tasks are maintained in real time. The maintenance logic is to save the tasks with shorter duration in the slow buffer as much as possible, and set the third threshold value, and store the distributed tasks with a running time greater than the third threshold value in the non-buffer. Storing tasks with longer duration in the non-buffer and storing them in the computing unit first after the computing unit completes the calculation can more efficiently process long-term distributed tasks and improve operating efficiency.

[0064] Step S204: When there are idle computing units, the distributed tasks in the non-buffer equal to the number of idle computing units are stored in the computing units; when the distributed tasks in the non-buffer are calculated, the distributed tasks in the slow buffer equal to the number of idle computing units are stored in the computing units; when the distributed tasks in the slow buffer are calculated, the distributed tasks in the fast buffer equal to the number of idle computing units are stored in the computing units.

[0065] This embodiment uses a machine learning model to infer the running time of distributed tasks. The inference time of the machine learning model is very short and will not affect the processing flow of distributed tasks. Therefore, using the machine learning model to infer the running time can improve the efficiency of subsequent distributed task allocation; long-time tasks are stored in the computing unit for priority calculation. When there are idle computing units, they are calculated in the order of non-buffer, slow buffer, and fast buffer. Tasks are divided according to time thresholds and stored in non-buffer, slow buffer, and fast buffer. This can achieve more efficient storage of longer-time tasks in computing units and give priority to calculating long-time distributed tasks, thereby reducing or even eliminating slow tasks that produce tailing effects in the final calculation, shortening the overall running time and improving global operating efficiency.

[0066] In order to make the embodiment of the present invention easier to understand, the chip manufacturing distributed task is taken as an example for explanation. Figure 4 , Figure 4 This is an example flow chart of distributed task allocation of a chip manufacturing software provided by an embodiment of the present invention. The above process is described in detail:

[0067] 1. Select the full size or partial layout of chip design of common nodes as the data collection sample of distributed tasks;

[0068] 2. Based on the distributed tasks of chip manufacturing software such as design rule detection and optical proximity effect correction as benchmarks, extract the feature information of each distributed task, such as the number of polygons, the total number of vertices, the overall coverage area and coverage rate, and other corresponding distributed task feature information (simple features of subtasks);

[0069] 3. Using the actual running time of distributed tasks and the characteristic information of distributed tasks collected, a linear function and / or a machine learning nonlinear model based on the characteristic information is fitted;

[0070] It can be understood that steps 1-3 implement the training process of the machine learning model. For low-complexity computing tasks such as design rule detection, linear fitting can be used to reduce the real-time reasoning running time and obtain better reasoning results. For high-complexity computing tasks such as simulation, such as optical proximity effect correction, relatively simple nonlinear fitting models (mainly used to improve computing efficiency) such as decision trees or two-layer neural networks can be used. Taking a two-layer neural network as an example, its reasoning model complexity is simpler and more efficient than other machine learning models. At the same time, since the two-layer neural network can also be directly applied to linear fitting, it is suitable for most task running time fitting scenarios and can provide high-accuracy reasoning results.

[0071] 4. Before constructing distributed tasks and assigning them, extract the feature information of each distributed task to perform runtime inference to obtain the runtime of each distributed task. Based on the inferred runtime and with a certain runtime threshold, divide the tasks into fast tasks and slow tasks;

[0072] 5. In common distributed application scenarios, the preparation phase often prepares distributed tasks twice the number of computing units in advance so that the remaining tasks can be immediately assigned to the computing units when the computing units are idle. Long-lasting distributed tasks are stored in computing units. During this process, some of the remaining distributed tasks that have not yet been distributed are stored in the fast task buffer and the slow task buffer, and are maintained in real time. The maintenance logic is to save relatively short-duration slow tasks in the slow task buffer as much as possible and store tasks longer than the slow task buffer in the non-buffer. This will adjust the slow tasks in the buffer so that relatively long slow tasks are processed first to avoid tasks that run too long in the final batch;

[0073] 6. After all distributed tasks in the non-buffered areas are distributed, buffered areas are distributed, with slow buffered areas being the first to be distributed, followed by fast buffered areas. After completing slow tasks, the computing unit will compute fast tasks with shorter running times, thereby reducing or even eliminating slow tasks that produce tailing effects in the last few batches of computation.

[0074] It can be seen that by extracting the number of polygons, the total number of vertices and the overall coverage area of ​​the sliced ​​layout in the distributed task, and extracting the features of the distributed task, the machine learning model is used for linear and / or nonlinear reasoning; its nonlinear reasoning is fitted using a two-layer neural network. After completing the reasoning of the running time, the fast and slow task buffers are maintained during the task distribution process. After the distributed tasks stored in the non-buffer are completed, the distributed tasks in the slow buffer are distributed first, and finally the distributed tasks in the fast buffer are distributed to eliminate the tailing effect.

[0075] A software distributed task allocation device provided by an embodiment of the present invention is introduced below. The software distributed task allocation device described below and the software distributed task allocation method described above can be referenced to each other.

[0076] Please refer to Figure 5 , Figure 5 A schematic diagram of the structure of a software distributed task allocation device provided in an embodiment of the present invention, the device may include:

[0077] A running time calculation module 100 is used to input the characteristic information of the distributed task into the generated machine learning model to obtain the running time of the distributed task;

[0078] The allocation calculation module 200 is used to determine the long-term distributed tasks that are no greater than the number of computing units according to preset rules and store them into the computing units for calculation; when the computing units are idle, the remaining distributed tasks equal to the number of idle computing units are stored in the computing units for calculation.

[0079] Based on any of the above embodiments, the allocation calculation module 200 may include:

[0080] A first allocation module, configured to store distributed tasks whose running time is greater than the first threshold into a computing unit according to a first threshold;

[0081] The second allocation module is used to store the distributed tasks whose running time is less than the second threshold in the fast buffer according to the second threshold and the third threshold; store the distributed tasks whose running time is greater than or equal to the second threshold and less than the third threshold in the slow buffer; and store the distributed tasks whose running time is greater than the third threshold in the non-buffer.

[0082] Correspondingly, the runtime calculation module 100 specifically calculates the distributed tasks in the computing units. When there are idle computing units, the distributed tasks in the non-buffer equal to the number of idle computing units are stored in the computing units; when the distributed tasks in the non-buffer are calculated, the distributed tasks in the slow buffer equal to the number of idle computing units are stored in the computing units; when the distributed tasks in the slow buffer are calculated, the distributed tasks in the fast buffer equal to the number of idle computing units are stored in the computing units.

[0083] Based on the above technical solution, an embodiment of the present invention provides a software distributed task allocation device, which can reduce or even eliminate the tailing effect in the calculation of the last few batches, thereby shortening the overall running time and improving the global operating efficiency.

[0084] A distributed task allocation device provided by an embodiment of the present invention is introduced below. The distributed task allocation device described below and the distributed task allocation method described above can refer to each other.

[0085] Please refer to Figure 6 , Figure 6 A structural diagram of a distributed task allocation device provided by an embodiment of the present invention may include:

[0086] A memory 400, used for storing computer programs;

[0087] The processor 500 is used to execute a computer program to implement the above-mentioned software distributed task allocation method.

[0088] In the embodiment of the present invention, the memory 400 is used to store one or more programs, and the program may include program code, and the program code includes computer operation instructions. In the embodiment of the present invention, the memory 400 may store a computer program 41 for implementing the following functions:

[0089] Input the feature information of the distributed task into the generated machine learning model to obtain the running time of the distributed task;

[0090] According to the preset rules, long-term distributed tasks that are not larger than the number of computing units are determined and stored in the computing units for calculation;

[0091] When there are idle computing units, the remaining distributed tasks equal to the number of idle computing units are stored in the computing units for calculation.

[0092] In one possible implementation, the memory 400 may include a program storage area and a data storage area, wherein the program storage area may store an operating system and an application required for at least one function, etc.; the resources stored in the memory 400 may also include an operating system 42 and data 43, etc., and the data storage area may store data created during use.

[0093] In addition, the memory 400 may include a read-only memory and a random access memory, and provide instructions and data to the processor. A portion of the memory may also include an NVRAM. The memory stores an operating system and operating instructions, executable modules or data structures, or a subset thereof, or an extended set thereof, wherein the operating instructions may include various operating instructions for implementing various operations. The operating system may include various system programs for implementing various basic tasks and processing hardware-based tasks.

[0094] The processor 500 may be a central processing unit (CPU), a specific application integrated circuit, a digital signal processor, a field programmable gate array or other programmable logic device, and the processor 500 may be a microprocessor or any conventional processor. The processor 500 may call the program stored in the memory 400. In some embodiments, the distributed task allocation device may further include a display screen 402, an input and output interface 403, a communication interface 404, a power supply 405 and a communication bus 406.

[0095] Those skilled in the art will understand that Figure 6 The structure shown in the figure does not constitute a limitation on the distributed task allocation device, and may include more or fewer components than those shown in the figure.

[0096] The following is an introduction to a readable storage medium provided in an embodiment of the present invention. The readable storage medium described below and the distributed task allocation method of a software described above can be referred to in correspondence with each other.

[0097] The present invention also provides a readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the above-mentioned software distributed task allocation method are implemented.

[0098] The computer-readable storage medium may include: a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, and other media that can store program codes.

[0099] In this specification, each embodiment is described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the embodiments can be referred to each other. For the device disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant parts can be referred to the method part.

[0100] Professionals may further appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in the above description according to function. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians may use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.

[0101] Finally, it should be noted that, in this article, relationships such as first and second, etc. are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprises" or any other variations are intended to cover non-exclusive inclusion, so that a process, method, article or device that includes a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device.

[0102] The above is a detailed introduction to the distributed task allocation method, device, equipment and computer-readable storage medium of a software provided by the present invention. Specific examples are used in this article to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core idea; at the same time, for a general technician in this field, according to the idea of ​​the present invention, there will be changes in the specific implementation method and application scope. In summary, the content of this specification should not be understood as a limitation on the present invention.

Claims

1. A software distributed task allocation method, characterized in that: include: Input the feature information of the distributed task into the generated machine learning model to obtain the running time of the distributed task; According to the preset rules, long-term distributed tasks that are not larger than the number of computing units are determined and stored in the computing units for calculation; When the computing unit is idle, the remaining distributed tasks equal to the number of idle computing units are stored in the computing unit for calculation; The step of inputting the feature information of the distributed task into the machine learning model generated corresponding to the feature information includes: Inputting feature information of the distributed tasks of the chip manufacturing software into a machine learning model generated corresponding to the feature information, wherein the feature information includes one or more of the number of polygons, the total number of vertices, and the overall coverage area and coverage rate; Among them, for the calculation task of design rule detection in chip manufacturing software, a linear function model is selected for fitting, and the number of vertices actually collected is input into the selected linear function model for training to obtain a linear function model corresponding to the number of vertices of this distributed task; For the task of correcting the optical proximity effect, a decision tree model is selected, and the number of polygons actually collected is input into the selected decision tree model for training to obtain a decision tree model corresponding to the number of polygons for this task; Wherein, determining according to preset rules that the long-term distributed tasks not greater than the number of computing units are stored in the computing units for computing includes: storing the distributed tasks whose running time is greater than or equal to a first threshold into the computing unit for computing; Before storing the remaining distributed tasks equal to the number of idle computing units into the computing units for computing, the following steps are also included: The distributed tasks whose running time is greater than or equal to the second threshold value among the remaining distributed tasks are stored in the slow buffer, and the distributed tasks whose running time is less than the second threshold value are stored in the fast buffer; the slow buffer tasks are maintained in real time, and the maintenance logic is to store the tasks with shorter running time in the slow buffer; The distributed tasks whose running time is greater than or equal to the second threshold among the remaining distributed tasks are stored in the slow buffer, including: The distributed tasks whose running time is greater than or equal to the second threshold and less than the third threshold among the remaining distributed tasks are stored in the slow buffer, and the distributed tasks whose running time is greater than or equal to the third threshold are stored in the non-buffer; When the computing unit is idle, the remaining distributed tasks equal to the number of idle computing units are stored in the computing unit for computing, including: When there are idle computing units, the distributed tasks in the non-buffer equal to the number of idle computing units are stored in the computing units; when the distributed tasks in the non-buffer are calculated, the distributed tasks in the slow buffer equal to the number of idle computing units are stored in the computing units; when the distributed tasks in the slow buffer are calculated, the distributed tasks in the fast buffer equal to the number of idle computing units are stored in the computing units; Input the feature information of the distributed task into the generated machine learning model to obtain the running time of the distributed task, including: The feature information of the distributed task is input into a machine learning model generated corresponding to the feature information to obtain the running time of the distributed task, wherein the machine learning model includes a linear function model, a selection decision tree model and a two-layer neural network model.

2. A software distributed task allocation device, characterized in that: include: A running time calculation module is used to input the characteristic information of the distributed task into the generated machine learning model to obtain the running time of the distributed task; An allocation computing module is used to determine, according to a preset rule, that the long-term distributed tasks not exceeding the number of computing units are stored in the computing units for computing; When the computing unit is idle, the remaining distributed tasks equal to the number of idle computing units are stored in the computing unit for calculation; The runtime calculation module is specifically used to input feature information of the distributed tasks of the chip manufacturing software into a machine learning model generated corresponding to the feature information, wherein the feature information includes one or more of the number of polygons, the total number of vertices, and the overall coverage area and coverage rate; Among them, for the calculation task of design rule detection in chip manufacturing software, a linear function model is selected for fitting, and the number of vertices actually collected is input into the selected linear function model for training to obtain a linear function model corresponding to the number of vertices of this distributed task; For the task of correcting the optical proximity effect, a decision tree model is selected, and the number of polygons actually collected is input into the selected decision tree model for training to obtain a decision tree model corresponding to the number of polygons for this task; Wherein, determining according to preset rules that the long-term distributed tasks not greater than the number of computing units are stored in the computing units for computing includes: storing the distributed tasks whose running time is greater than or equal to a first threshold into the computing unit for computing; Before storing the remaining distributed tasks equal to the number of idle computing units into the computing units for computing, the following steps are also included: The distributed tasks whose running time is greater than or equal to the second threshold value among the remaining distributed tasks are stored in the slow buffer, and the distributed tasks whose running time is less than the second threshold value are stored in the fast buffer; the slow buffer tasks are maintained in real time, and the maintenance logic is to store the tasks with shorter running time in the slow buffer; The distributed tasks whose running time is greater than or equal to the second threshold among the remaining distributed tasks are stored in the slow buffer, including: The distributed tasks whose running time is greater than or equal to the second threshold and less than the third threshold among the remaining distributed tasks are stored in the slow buffer, and the distributed tasks whose running time is greater than or equal to the third threshold are stored in the non-buffer; When the computing unit is idle, the remaining distributed tasks equal to the number of idle computing units are stored in the computing unit for computing, including: When there are idle computing units, the distributed tasks in the non-buffer equal to the number of idle computing units are stored in the computing units; when the distributed tasks in the non-buffer are calculated, the distributed tasks in the slow buffer equal to the number of idle computing units are stored in the computing units; when the distributed tasks in the slow buffer are calculated, the distributed tasks in the fast buffer equal to the number of idle computing units are stored in the computing units; Input the feature information of the distributed task into the generated machine learning model to obtain the running time of the distributed task, including: The feature information of the distributed task is input into a machine learning model generated corresponding to the feature information to obtain the running time of the distributed task, wherein the machine learning model includes a linear function model, a selection decision tree model and a two-layer neural network model.

3. A software distributed task allocation device, characterized in that: include: Memory for storing computer programs; A processor is used to implement the steps of a software distributed task allocation method as claimed in claim 1 when executing the computer program.

4. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the software distributed task allocation method according to claim 1 are implemented.

Images