Algorithm scheduling method and device, equipment, storage medium and product

Abstract

The invention discloses an algorithm scheduling method and device, equipment, a storage medium and a product. The method comprises the following steps: determining an algorithm type of an algorithm required for auditing to-be-audited content; according to the algorithm type, a first server set conforming to the algorithm is determined in a machine resource cluster, the first server set comprises at least one first server, and the algorithm is arranged in each first server; designing an algorithm scheduling model according to the to-be-audited content and the first server set; calculating the algorithm scheduling model to obtain strategy scheduling parameters of the algorithm scheduling model; and performing algorithm scheduling on the to-be-audited content by adopting the strategy scheduling parameters. According to the embodiment of the invention, the efficiency of the to-be-audited content can be improved.

Description

Technical Field

[0001] The present application belongs to the technical field of algorithm scheduling, and in particular, relates to an algorithm scheduling method, device, equipment, storage medium and product. Background Art

[0002] In the content security audit system, various algorithms are usually used to audit a multimedia content (images, audio, video, etc.) to detect whether the rich media violates the rules in some aspects. After receiving the algorithm requirements from the user, the content security audit system will dispatch different algorithms required by the user, use multiple algorithms to process and detect the content to be audited in parallel, and notify the user of the results after all algorithms return.

[0003] The above algorithm services are usually deployed in GPU servers. Considering factors such as cost and resource utilization, multiple algorithm services are usually deployed in one GPU service, thereby improving GPU resource utilization and reducing costs.

[0004] The purpose of algorithm scheduling is to execute a specific algorithm on a specific GPU server, that is, to send the data content to be reviewed to a specific algorithm on a specific GPU server. In the existing solution, a polling + FIFO scheduling strategy and method are adopted. That is, when the user needs to use multiple algorithms to review and detect rich media content, the system adopts a polling strategy to send the content to be reviewed to different algorithm servers in sequence. In a certain server, the algorithm service adopts a FIFO strategy to detect the content to be reviewed in order. The FIFO strategy uses the order in which the content to be tested enters the queue as the scheduling basis, and allocates GPU resources according to the order in which the content to be tested is submitted or the process becomes ready. The GPU is released only when the task is completed or blocked. However, the polling strategy is used to send the content to be reviewed to the GPU, and the order in which the content to be reviewed enters the queue is used as the scheduling basis. The characteristics of different algorithms in the GPU and the machine resources of the GPU server are not considered, resulting in low review efficiency. Summary of the invention

[0005] The embodiments of the present application provide an algorithm scheduling method, apparatus, device and computer storage medium to solve the problem of low efficiency in reviewing content to be reviewed.

[0006] In a first aspect, an embodiment of the present application provides an algorithm scheduling method, the method comprising:

[0007] Determine the type of algorithm required to review the content to be reviewed;

[0008] According to the algorithm type, determining a first server set that complies with the algorithm in a machine resource cluster, wherein the first server set includes at least one first server, and the algorithm is deployed in each first server;

[0009] Designing an algorithm scheduling model according to the content to be reviewed and the first server set;

[0010] Calculating the algorithm scheduling model to obtain the policy scheduling parameters of the algorithm scheduling model;

[0011] The policy scheduling parameters are used to perform algorithmic scheduling on the content to be reviewed.

[0012] In a second aspect, an embodiment of the present application provides an algorithm scheduling device, the device comprising:

[0013] A first determination module, used to determine the algorithm type of the algorithm required to review the content to be reviewed;

[0014] A second determination module is used to determine, according to the algorithm type, a first server set that conforms to the algorithm in the machine resource cluster, wherein the first server set includes at least one first server, and the algorithm is deployed in each first server;

[0015] A design module, used to design an algorithm scheduling model according to the content to be reviewed and the first server set;

[0016] A calculation module, used to calculate the algorithm scheduling model to obtain the policy scheduling parameters of the algorithm scheduling model;

[0017] The scheduling module is used to perform algorithmic scheduling on the content to be reviewed by using the policy scheduling parameters.

[0018] In a third aspect, an embodiment of the present application provides a terminal device, the device comprising: a processor and a memory storing computer program instructions;

[0019] When the processor executes the computer program instructions, the algorithm scheduling method of the first aspect is implemented.

[0020] In a fourth aspect, an embodiment of the present application provides a computer storage medium, on which computer program instructions are stored. When the computer program instructions are executed by a processor, the algorithm scheduling method of the first aspect is implemented.

[0021] In a fifth aspect, an embodiment of the present application provides a computer program product. When instructions in the computer program product are executed by a processor of an electronic device, the electronic device executes the algorithm scheduling method of the first aspect.

[0022] The algorithm scheduling method provided in the embodiment of the present application determines the algorithm type of the algorithm required to audit the content to be audited; according to the algorithm type, determines a first server set that conforms to the algorithm in the machine resource cluster, wherein the first server set includes at least one first server, and each first server is equipped with the algorithm; designs an algorithm scheduling model according to the content to be audited and the first server set; calculates the algorithm scheduling model to obtain the policy scheduling parameters of the algorithm scheduling model; and uses the policy scheduling parameters to perform algorithm scheduling on the content to be audited. Compared with the existing polling strategy, this model fully considers the characteristics of different algorithms, classifies the algorithms, selects the optimal machine resources to execute the corresponding algorithms, processes the data content to be audited, reduces the algorithm execution time, increases the algorithm execution speed, and thus improves the audit efficiency of the content to be audited. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] In order to more clearly illustrate the technical solution of the embodiments of the present application, the following is a brief introduction to the drawings required for use in the embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0024] Figure 1 It is a flowchart of the algorithm scheduling method provided in the embodiment of the present application;

[0025] Figure 2 It is a schematic diagram of the allocation relationship between the algorithm and the server provided in the embodiment of the present application;

[0026] Figure 3 It is a schematic diagram of the overall process of the algorithm scheduling method provided in the embodiment of the present application;

[0027] Figure 4 It is an overall architecture diagram of a scheduling system applied to an algorithm scheduling method provided in an embodiment of the present application;

[0028] Figure 5 It is a structural diagram of an algorithm scheduling device provided in an embodiment of the present application;

[0029] Figure 6 It is a schematic diagram of the structure of the terminal device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0030] The features and exemplary embodiments of various aspects of the present application will be described in detail below. In order to make the purpose, technical solutions and advantages of the present application clearer, the present application will be further described in detail below in conjunction with the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are only intended to explain the present application, rather than to limit the present application. For those skilled in the art, the present application can be implemented without the need for some of these specific details. The following description of the embodiments is only to provide a better understanding of the present application by illustrating the examples of the present application.

[0031] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the statement "include..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.

[0032] In order to solve the problems of the prior art, the embodiments of the present application provide an algorithm scheduling method, device, equipment, storage medium and product.

[0033] The following first introduces the algorithm scheduling method provided in the embodiment of the present application.

[0034] Figure 1 FIG. 1 is a schematic diagram showing an interactive process of an algorithm scheduling method provided by an embodiment of the present application. Figure 1 As shown, the method includes:

[0035] Step 101, determining the algorithm type of the algorithm required to audit the content to be audited;

[0036] In this embodiment, when auditing the content to be audited, various algorithms are usually required to perform content audit on the content to be audited (pictures, audio, video, etc.) to detect whether the content to be audited violates the rules in some aspect. The above-mentioned algorithm types can be specifically classified by the type of resource demand of the algorithm, so as to obtain the algorithm type to which the algorithm belongs. It can include core (GPU) busy algorithms, interface (I / O) busy algorithms, and memory busy algorithms. It should be noted that the algorithms required for auditing the content to be audited can be one or more, and the algorithm type can also include other algorithm types, such as network busy algorithms, which are not limited in this embodiment.

[0037] Specifically, in one embodiment, the algorithm type of the algorithm required for reviewing the content to be reviewed includes:

[0038] Determining a plurality of first resources required for reviewing the content to be reviewed;

[0039] Determine a third resource corresponding to each second resource in the plurality of first resources, the third resource being a resource provided by the machine resource cluster;

[0040] Calculating a ratio between the second resource and the third resource to obtain a plurality of ratios corresponding one-to-one to the plurality of first resources;

[0041] Among the multiple ratios, the algorithm type corresponding to the ratio with the largest value is determined to be the algorithm type of the required algorithm.

[0042] In this embodiment, the first resources required for the to-be-audited content of different sizes are different, and the first resources include the number of GPU cores G, the memory size R, and the I / O usage I.

[0043] A machine resource cluster refers to a cluster of multiple servers, which may be located in different physical locations and have different computing capabilities and storage resources. The third resource refers to the resources that the machine resource cluster can provide, including the number of CPU cores and G s 、Memory size R s And I / O usage I s Among them, G s is the third resource corresponding to G, R s is the third resource corresponding to R, I s Is the third resource corresponding to I.

[0044] The ratios between the second resource and the third resource are Among these ratios, the ratio with the largest value is determined by the formula:

[0045]

[0046] Among them, if The maximum, then the required algorithm type is a CPU busy algorithm; if Maximum, then the required algorithm type is a memory busy algorithm; if Maximum, the required algorithm type is an I / O busy algorithm.

[0047] In this embodiment, the resource requirements of the content to be reviewed and the availability of resources in the machine resource cluster are comprehensively considered in a quantitative manner, so that the most appropriate algorithm type can be accurately selected.

[0048] Step 102: determining, according to the algorithm type, a first server set that conforms to the algorithm in a machine resource cluster, wherein the first server set includes at least one first server, and the algorithm is deployed in each first server;

[0049] In this embodiment, once the algorithm type is determined, the next step is to find servers that have deployed these algorithms in the machine resource cluster. These servers may be located in different physical locations and have different computing capabilities and storage resources. According to the requirements of the algorithm type, servers that meet the requirements will be screened out to form one or more first server sets.

[0050] Specifically, in one embodiment, determining, in a machine resource cluster, a first set of servers that conform to the algorithm according to the algorithm type includes:

[0051] Step 1021: for any first algorithm type among the at least one algorithm type, using the first algorithm type, perform a status evaluation on a second server in the machine resource cluster to obtain a status evaluation value of the second server, wherein the status evaluation value indicates a status of the second server when executing the algorithm corresponding to the first algorithm type;

[0052] In this embodiment, the algorithm type of the algorithm for auditing the content to be audited includes at least one algorithm type. When determining the server corresponding to any first algorithm type in the at least one algorithm type, it is necessary to perform a status evaluation on the second server in the machine resource cluster before determining.

[0053] Specifically, in one embodiment, using the first algorithm type to perform a status evaluation on a second server in the machine resource cluster to obtain a status evaluation value of the second server includes:

[0054] Obtaining a percentage of available cores, a percentage of available memory, and a percentage of available interfaces of the second server;

[0055] According to the first algorithm type, using a first formula to perform a status evaluation on the second server to obtain a status evaluation value of the second server;

[0056] The first formula is:

[0057] S i =a*G i +b*R i +c*I i ;

[0058] Among them, S i is the state evaluation value, a is the weight value corresponding to the core busy algorithm, G iis the percentage of available cores, b is the weight value corresponding to the memory busy algorithm, R i is the percentage of available memory, c is the weight value corresponding to the interface busy algorithm, I i The ratio is the available interface ratio.

[0059] In this embodiment, the first algorithm type can be any one of a core busy algorithm, an interface busy algorithm, and a memory busy algorithm, and a, b, and c are weight values ​​of the corresponding algorithm, and the weight value is related to the first algorithm type, and the corresponding algorithm type has a higher weight value. For example, for a memory busy algorithm, the corresponding ratio of a, b, and c can be 8:1:1, and this ratio can be adjusted dynamically.

[0060] Step 1022, selecting a preset number of second servers from each of the second servers in the machine resource cluster according to the order of the state evaluation values ​​from large to small, to obtain a second server set;

[0061] In this embodiment, after calculating the status evaluation values ​​of all second servers, a preset number of second servers are selected by size sorting to form a second server set. gx Indicates that, and M gx ={M1,M2,M3,...M N}, where M N Represents the Nth second server. The set represents N second servers that meet the requirements of the first algorithm type, where N can be a preset number, which refers to the total number of first algorithm types. For example, if there are 3 first algorithm types, then N is 3.

[0062] By using the above method, the second server set of each specific type of algorithm is screened out. Since there are three types of first algorithms in this embodiment, three second server sets are obtained, and each second server set corresponds to one algorithm type.

[0063] Step 1023: perform intersection processing on multiple second server sets to obtain a first server set that complies with the algorithm.

[0064] In this embodiment, all second server sets are subjected to intersection processing to obtain a machine resource cluster set M that conforms to all first algorithm types = {M gx ∩M gy ∩M gz ∩...M gn}, finally, the set of first servers that meets all algorithm requirements is M, and the number of first servers in the set is m.

[0065] In this embodiment, the above method ensures that the final selected set can meet the requirements of multiple algorithm types at the same time, thereby improving the flexibility and efficiency of the audit task. At the same time, by introducing the state evaluation value, it is also ensured that the selected server has a high performance when executing the algorithm.

[0066] Step 103, designing an algorithm scheduling model according to the content to be reviewed and the first server set;

[0067] In this embodiment, M is the m identical parallel physical servers after screening, A is the n selected algorithms, and T is the required resource type, including the number of GPU cores, memory, I / O volume, etc.

[0068] M={M1,M2,M3...M I}, I = 1, 2, 3...m

[0069] A={A1,A2,A3...A J}, J = 1, 2, 3...n

[0070] T={T1,T2,T3...T K}, K = 1, 2, 3...k

[0071] According to the needs and characteristics of algorithm scheduling, the algorithm scheduling model is designed with the algorithm scheduling of "multi-algorithms-multi-resources-multi-constraints" as the optimization problem. The design process of the algorithm scheduling model is divided into model design, objective function design and constraint function design.

[0072] Specifically, when designing the model, first determine the distribution relationship between the algorithm and the server of the content to be reviewed, such as Figure 2 As shown, Figure 2 This is a diagram of the distribution relationship between algorithms and servers. All algorithms in the figure need to be distributed to the braking server to run. Among them, all algorithms are deployed on each server.

[0073] Secondly, the objective function is designed, considering aspects such as algorithm queue length, resource utilization, and algorithm priority. The objective function of the model is defined as:

[0074] minf1=ρ i∈I ρ j∈J (t ij × ijk ) (1)

[0075]

[0076] minf3=ρ i∈I ρ j∈J (a ij / C) (3)

[0077] Among them, x ijk Indicates that the content to be reviewed is sent to the third server i in the first server set where algorithm j is located, and the supply of resource k by the third server i;

[0078] d jk represents the demand of algorithm j for resource k;

[0079] t ij Indicates the time when the content to be reviewed is sent to the third server i and processed by algorithm j;

[0080] a ij It represents the length of the queue of pending tasks of the process where algorithm j is located after the pending tasks are assigned to algorithm j in the third server i.

[0081] It should be noted that This time consists of two parts. is the waiting time, which is determined by the queue length L of algorithm j in the third server i. is the actual processing time, which is related to the amount of allocated resources x. j It represents the priority coefficient of algorithm j demand, which can be determined by the user. C is the maximum value of the queue that does not affect the system runtime in each algorithm process under normal circumstances.

[0082] The objective function f1 in formula (1) indicates that in each third server i, the total processing time turnover of algorithm j is minimized, that is, the sum of the products of the server allocation of resources and the total algorithm processing time is minimized. The objective function f2 in formula (2) indicates that the degree to which the resources of each third server do not meet the resource requirements of the algorithm is minimized, taking into account the priority coefficient of the algorithm that may be set by the user, reflecting the fairness of the algorithm scheduling. The objective function f3 in formula (3) represents the load rate of all algorithm processing queues. The three-objective scheduling optimization model is defined by formulas (1) to (3), which represent the efficiency target, fairness target and load target in algorithm scheduling respectively.

[0083] Finally, the model constraint design is performed. In order to represent the allocation relationship between GPU servers and algorithms, the allocation matrix S is introduced to represent the mapping relationship between the selected algorithm set A and the server resource set M:

[0084]

[0085] The allocation matrix can represent all possible mappings, and each mapping is uniquely represented. Since each algorithm can only be processed by one server, the model constraints include (4)(5):

[0086]

[0087] At the same time, the model should meet the resource constraints. Formula (6) indicates that the allocation of resource k by the third server i is less than or equal to its demand. Formula (7) indicates that the total demand of algorithm j for resource k is less than the storage capacity of resource k of the third server i.

[0088]

[0089]

[0090] p ik Represents the storage capacity of resource k in the third server i.

[0091] In this embodiment, the algorithm scheduling model comprehensively considers the total algorithm execution time, execution efficiency, algorithm process load, overall resource consumption, algorithm priority, etc., and the scheduling is more reasonable.

[0092] Step 104, calculating the algorithm scheduling model to obtain the policy scheduling parameters of the algorithm scheduling model;

[0093] Step 105: Use the policy scheduling parameters to perform algorithm scheduling on the content to be reviewed.

[0094] In this embodiment, after the algorithm scheduling model is designed, the particle swarm algorithm is used to solve the optimal scheduling strategy, and the scheduling strategy parameters that meet the optimization goals (efficiency goal, fairness goal, load goal) under the above constraints are calculated. The strategy parameters are: on the third server i, the j algorithm is scheduled to process the content to be reviewed, and the allocation amount of the scheduled resource type k is x.

[0095] By calculating these parameters, the algorithm is dynamically scheduled for each content to be reviewed.

[0096] The algorithm scheduling method provided in the embodiment of the present application determines the algorithm type of the algorithm required to audit the content to be audited; according to the algorithm type, determines a first server set that conforms to the algorithm in the machine resource cluster, wherein the first server set includes at least one first server, and each first server is equipped with the algorithm; designs an algorithm scheduling model according to the content to be audited and the first server set; calculates the algorithm scheduling model to obtain the policy scheduling parameters of the algorithm scheduling model; and uses the policy scheduling parameters to perform algorithm scheduling on the content to be audited. Compared with the existing polling strategy, this model fully considers the characteristics of different algorithms, classifies the algorithms, selects the optimal machine resources to execute the corresponding algorithms, processes the data content to be audited, reduces the algorithm execution time, increases the algorithm execution speed, and thus improves the audit efficiency of the content to be audited.

[0097] In one embodiment of the present application, Figure 3 The overall flow diagram of the algorithm scheduling method provided by one embodiment of the present application is shown;

[0098] like Figure 3 As shown, including:

[0099] S110, classify the required algorithms according to the content to be processed and the algorithm attributes, and determine the algorithm type: (that is, determine the algorithm type of the algorithm required to review the content to be reviewed)

[0100] S120, obtaining the status of each machine in the current cluster, evaluating and screening the machine resources that meet the requirements; (that is, according to the algorithm type, determining a first server set that meets the algorithm in the machine resource cluster, the first server set includes at least one first server, and each first server is deployed with the algorithm;)

[0101] S130, designing a multi-objective algorithm scheduling optimization model and constraint conditions; (i.e., designing an algorithm scheduling model according to the content to be reviewed and the first server set;)

[0102] S140, based on the particle swarm optimization algorithm, calculate the optimal scheduling method and strategy parameters, and then perform algorithm scheduling; (i.e., calculate the algorithm scheduling model to obtain the strategy scheduling parameters of the algorithm scheduling model; and use the strategy scheduling parameters to perform algorithm scheduling on the content to be reviewed.)

[0103] Correspondingly, in one embodiment of the present application, Figure 4 The overall architecture diagram of the scheduling system provided by one embodiment of the present application is shown;

[0104] The scheduling system includes: task analysis and algorithm classification module, state acquisition and evaluation module, model building and strategy solving module, task distribution and algorithm scheduling module.

[0105] The main functions of each module of the system are as follows.

[0106] Task analysis and algorithm classification module: Analyze the size of the data to be processed and obtain the type of algorithm required by the user. According to the type of resource requirements of the algorithm, classify the algorithm and obtain the type of algorithm.

[0107] Status acquisition and resource evaluation module: obtains the status of GPU computing servers in real time, including resource status and operation status, and combines the different resources required by different types of algorithms to screen out all GPU servers that meet the algorithm conditions.

[0108] Model building and strategy solving module: Aiming at goals such as data processing time, total resource consumption, algorithm priority, etc., taking into account factors such as the total number of algorithms to be classified, the number of GPU cores in the GPU cluster, the total amount of computing resources, the allocation amount, and the demand amount, a scheduling model is established to calculate the scheduling strategy parameters.

[0109] Task dispatching and algorithm scheduling module: After calculating the scheduling strategy and scheduling parameters, the data to be processed is sent to the specific algorithm on the designated server. Once the algorithm processing is completed, the algorithm scheduling process ends.

[0110] Based on the same inventive concept, the embodiment of the present application also provides an algorithm scheduling device. Figure 5 The algorithm scheduling device provided in the embodiment of the present application is described in detail.

[0111] Figure 5 It is a structural diagram of an algorithm scheduling device provided in an embodiment of the present application.

[0112] like Figure 5 As shown, the algorithm scheduling device 500 may include:

[0113] A first determination module 501 is used to determine the algorithm type of the algorithm required to review the content to be reviewed;

[0114] A second determination module 502 is used to determine, according to the algorithm type, a first server set in the machine resource cluster that conforms to the algorithm, wherein the first server set includes at least one first server, and the algorithm is deployed in each first server;

[0115] A design module 503, configured to design an algorithm scheduling model according to the content to be reviewed and the first server set;

[0116] A calculation module 504 is used to calculate the algorithm scheduling model to obtain the policy scheduling parameters of the algorithm scheduling model;

[0117] The scheduling module 505 is used to perform algorithmic scheduling on the content to be reviewed by using the policy scheduling parameters.

[0118] Optionally, the first determining module 501 includes:

[0119] A first determination submodule, used to determine a plurality of first resources required for reviewing the content to be reviewed;

[0120] A second determining submodule, configured to determine a third resource corresponding to each second resource in the plurality of first resources, wherein the third resource is a resource provided by the machine resource cluster;

[0121] a calculation submodule, configured to calculate a ratio between the second resource and the third resource, and obtain a plurality of ratios corresponding one-to-one to the plurality of first resources;

[0122] The third determination submodule is used to determine, among the multiple ratios, that the algorithm type corresponding to the ratio with the largest value is the algorithm type of the required algorithm.

[0123] Optionally, the second determining module 502 includes:

[0124] an evaluation submodule, configured to, for any first algorithm type among the at least one algorithm type, use the first algorithm type to perform a status evaluation on a second server in the machine resource cluster to obtain a status evaluation value of the second server, wherein the status evaluation value represents a status of the second server when executing the algorithm corresponding to the first algorithm type;

[0125] A selection submodule, configured to select a preset number of second servers from each of the second servers in the machine resource cluster according to the order of the state evaluation values ​​from large to small, to obtain a second server set;

[0126] The processing submodule is used to perform intersection processing on multiple second server sets to obtain a first server set that conforms to the algorithm.

[0127] Optionally, evaluate submodules. Includes:

[0128] An acquisition unit, used to acquire a percentage of available cores, a percentage of available memory, and a percentage of available interfaces of the second server;

[0129] an evaluation unit, configured to perform a status evaluation on the second server using a first formula according to the first algorithm type to obtain a status evaluation value of the second server;

[0130] The first formula is:

[0131] S i =a*G i +b*R i +c*I i ;

[0132] Among them, S i is the state evaluation value, a is the weight value corresponding to the core busy algorithm, G i is the percentage of available cores, b is the weight value corresponding to the memory busy algorithm, R i is the percentage of available memory, c is the weight value corresponding to the interface busy algorithm, I i The ratio is the available interface ratio.

[0133] Optionally, the design module 503 is specifically used to:

[0134] Determining the objective function according to the content to be reviewed and the first server set;

[0135] Among them, the objective function includes:

[0136]

[0137] Among them, x ijk Indicates that the content to be reviewed is sent to the third server i in the first server set where algorithm j is located, and the supply of resource k by the third server i;

[0138] d jk represents the demand of algorithm j for resource k;

[0139] t ij Indicates the time when the content to be reviewed is sent to the third server i and processed by algorithm j;

[0140] a ij It represents the length of the queue of pending tasks of the process where algorithm j is located after the pending tasks are assigned to algorithm j in the third server i.

[0141] Optionally, the design module 503 is specifically used to:

[0142] Determining the constraint function according to the content to be reviewed and the first server set;

[0143] The constraint functions include:

[0144]

[0145] Among them, S ij represents the allocation matrix between the first server set and the algorithm, x ijk Indicates that the content to be reviewed is sent to the third server i in the first server set where algorithm j is located, and the supply of resource k by the third server i; d jk represents the demand of algorithm j for resource k; p ik Represents the storage capacity of resource k in the third server i.

[0146] It should be noted that the algorithm scheduling device 500 is a device corresponding to the above-mentioned algorithm scheduling method. All implementation methods in the above-mentioned method embodiments are applicable to the embodiments of the device and can achieve the same technical effects.

[0147] Figure 5 A schematic diagram of the hardware structure of a terminal device provided in an embodiment of the present application is shown.

[0148] The terminal device may include a processor 501 and a memory 502 storing computer program instructions.

[0149] Specifically, the processor 501 may include a central processing unit (CPU), or an application specific integrated circuit (ASIC), or may be configured to implement one or more integrated circuits of the embodiments of the present application.

[0150] The memory 502 may include a large capacity memory for data or instructions. By way of example and not limitation, the memory 502 may include a hard disk drive (HDD), a floppy disk drive, a flash memory, an optical disk, a magneto-optical disk, a magnetic tape, or a universal serial bus (USB) drive or a combination of two or more of these. In appropriate cases, the memory 502 may include a removable or non-removable (or fixed) medium. In appropriate cases, the memory 502 may be inside or outside the integrated gateway disaster recovery device. In a specific embodiment, the memory 502 is a non-volatile solid-state memory.

[0151] In certain embodiments, the memory 502 may include a read-only memory (ROM), a random access memory (RAM), a magnetic disk storage medium device, an optical storage medium device, a flash memory device, an electrical, optical or other physical / tangible memory storage device. Thus, in general, the memory includes one or more tangible (non-transitory) computer-readable storage media (e.g., a memory device) encoded with software including computer-executable instructions, and when the software is executed (e.g., by one or more processors), it is operable to perform the operations described with reference to the method according to an aspect of the present disclosure.

[0152] The processor 501 implements any one of the algorithm scheduling methods in the above embodiments by reading and executing computer program instructions stored in the memory 502 .

[0153] In one example, the terminal device may further include a communication interface 503 and a bus 510. Figure 5 As shown, the processor 501, the memory 502, and the communication interface 503 are connected via a bus 510 and communicate with each other.

[0154] The communication interface 503 is mainly used to implement communication between various modules, devices, units and / or equipment in the embodiments of the present application.

[0155] Bus 510 includes hardware, software or both, and the parts of online data flow billing equipment are coupled to each other. For example, but not limitation, bus may include accelerated graphics port (AGP) or other graphics bus, enhanced industrial standard architecture (EISA) bus, front-end bus (FSB), hypertransport (HT) interconnection, industrial standard architecture (ISA) bus, infinite bandwidth interconnection, low pin count (LPC) bus, memory bus, micro channel architecture (MCA) bus, peripheral component interconnection (PCI) bus, PCI-Express (PCI-X) bus, serial advanced technology attachment (SATA) bus, video electronics standard association local (VLB) bus or other suitable bus or two or more of these combinations. In appropriate cases, bus 510 may include one or more buses. Although the present application embodiment describes and shows a specific bus, the present application considers any suitable bus or interconnection.

[0156] In addition, in combination with the algorithm scheduling method in the above embodiment, the embodiment of the present application can provide a computer storage medium for implementation. The computer storage medium stores computer program instructions; when the computer program instructions are executed by a processor, any one of the algorithm scheduling methods in the above embodiment is implemented.

[0157] It should be clear that the present application is not limited to the specific configuration and processing described above and shown in the figures. For the sake of simplicity, a detailed description of the known method is omitted here. In the above embodiments, several specific steps are described and shown as examples. However, the method process of the present application is not limited to the specific steps described and shown, and those skilled in the art can make various changes, modifications and additions, or change the order between the steps after understanding the spirit of the present application.

[0158] The functional blocks shown in the above block diagram can be implemented as hardware, software, firmware or a combination thereof. When implemented in hardware, it can be, for example, an electronic circuit, an application specific integrated circuit (ASIC), appropriate firmware, a plug-in, a function card, etc. When implemented in software, the elements of the present application are programs or code segments that are used to perform the required tasks. The program or code segment can be stored in a machine-readable medium, or transmitted on a transmission medium or a communication link by a data signal carried in a carrier wave. "Machine-readable medium" can include any medium capable of storing or transmitting information. Examples of machine-readable media include electronic circuits, semiconductor memory devices, ROM, flash memory, erasable ROM (EROM), floppy disks, CD-ROMs, optical disks, hard disks, optical fiber media, radio frequency (RF) links, etc. The code segment can be downloaded via a computer network such as the Internet, an intranet, etc.

[0159] It should also be noted that the exemplary embodiments mentioned in this application describe some methods or systems based on a series of steps or devices. However, this application is not limited to the order of the above steps, that is, the steps can be performed in the order mentioned in the embodiment, or in a different order from the embodiment, or several steps can be performed simultaneously.

[0160] Aspects of the present disclosure are described above with reference to the flowchart and / or block diagram of the method, device (system) and computer program product according to the embodiment of the present disclosure. It should be understood that each box in the flowchart and / or block diagram and the combination of each box in the flowchart and / or block diagram can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable algorithm scheduling device to produce a machine so that these instructions executed by the processor of the computer or other programmable algorithm scheduling device enable the implementation of the function / action specified in one or more boxes of the flowchart and / or block diagram. Such a processor can be, but is not limited to, a general-purpose processor, a special-purpose processor, a special application processor, or a field programmable logic circuit. It can also be understood that each box in the block diagram and / or flowchart and the combination of boxes in the block diagram and / or flowchart can also be implemented by dedicated hardware that performs a specified function or action, or can be implemented by a combination of dedicated hardware and computer instructions.

[0161] The above are only specific implementation methods of the present application. Those skilled in the art can clearly understand that for the convenience and simplicity of description, the specific working processes of the systems, modules and units described above can refer to the corresponding processes in the aforementioned method embodiments, and will not be repeated here. It should be understood that the protection scope of the present application is not limited to this. Any technician familiar with the technical field can easily think of various equivalent modifications or replacements within the technical scope disclosed in this application, and these modifications or replacements should be included in the protection scope of this application.

Claims

1. An algorithm scheduling scheme, characterized in that: The method comprises: Determine the type of algorithm required to review the content to be reviewed; According to the algorithm type, determining a first server set that complies with the algorithm in a machine resource cluster, wherein the first server set includes at least one first server, and the algorithm is deployed in each first server; Designing an algorithm scheduling model according to the content to be reviewed and the first server set; Calculating the algorithm scheduling model to obtain the policy scheduling parameters of the algorithm scheduling model; The policy scheduling parameters are used to perform algorithmic scheduling on the content to be reviewed.

2. The algorithm scheduling method according to claim 1, characterized in that: The algorithm type for determining the algorithm required to review the content to be reviewed includes: Determining a plurality of first resources required for reviewing the content to be reviewed; Determine a third resource corresponding to each second resource in the plurality of first resources, the third resource being a resource provided by the machine resource cluster; Calculating a ratio between the second resource and the third resource to obtain a plurality of ratios corresponding one-to-one to the plurality of first resources; Among the multiple ratios, the algorithm type corresponding to the ratio with the largest value is determined to be the algorithm type of the required algorithm.

3. The algorithm scheduling method according to claim 1, characterized in that: The algorithm type includes at least one algorithm type; Determining, in a machine resource cluster, a first server set that complies with the algorithm according to the algorithm type includes: For any first algorithm type among the at least one algorithm type, using the first algorithm type, performing a state evaluation on a second server in the machine resource cluster to obtain a state evaluation value of the second server, wherein the state evaluation value represents a state of the second server when executing an algorithm corresponding to the first algorithm type; In each of the second servers in the machine resource cluster, a preset number of second servers are selected according to the order of the state evaluation values ​​from large to small to obtain a second server set; An intersection process is performed on a plurality of the second server sets to obtain a first server set that complies with the algorithm.

4. The algorithm scheduling method according to claim 3, characterized in that: The first algorithm type includes a core busy algorithm, an interface busy algorithm or a memory busy algorithm; The step of using the first algorithm type to perform a status evaluation on a second server in the machine resource cluster to obtain a status evaluation value of the second server includes: Obtaining a percentage of available cores, a percentage of available memory, and a percentage of available interfaces of the second server; According to the first algorithm type, using a first formula to perform a status evaluation on the second server to obtain a status evaluation value of the second server; The first formula is: S i =a*G i +b*R i +c*I i ; Among them, S i is the state evaluation value, a is the weight value corresponding to the core busy algorithm, G i is the percentage of available cores, b is the weight value corresponding to the memory busy algorithm, R i is the percentage of available memory, c is the weight value corresponding to the interface busy algorithm, I i The percentage of available interfaces.

5. The algorithm scheduling method according to claim 1, characterized in that: The algorithm scheduling model includes an objective function; The step of designing an algorithm scheduling model according to the content to be reviewed and the first server set includes: Determining the objective function according to the content to be reviewed and the first server set; Among them, the objective function includes: Among them, x ijk Indicates that the content to be reviewed is sent to the third server i in the first server set where algorithm j is located, and the supply of resource k by the third server i; d jk represents the demand of algorithm j for resource k; t ij Indicates the time when the content to be reviewed is sent to the third server i and processed by algorithm j; a ij It represents the length of the queue of pending tasks of the process where algorithm j is located after the pending tasks are assigned to algorithm j in the third server i.

6. The algorithm scheduling method according to claim 5, characterized in that: The algorithm scheduling model includes a constraint function; The step of designing an algorithm scheduling model according to the content to be reviewed and the first server set includes: Determining the constraint function according to the content to be reviewed and the first server set; The constraint functions include: Among them, S ij represents the allocation matrix between the first server set and the algorithm, x ijk Indicates that the content to be reviewed is sent to the third server i in the first server set where algorithm j is located, and the supply of resource k by the third server i; d jk represents the demand of algorithm j for resource k; p ik Represents the storage capacity of resource k in the third server i.

7. The algorithm scheduling method according to claim 6, characterized in that: The scheduling parameters include the third server i, the algorithm j, the resource k and the allocation amount of the resource k.

8. An algorithm scheduling device, characterized in that: The device comprises: A first determination module, used to determine the algorithm type of the algorithm required to review the content to be reviewed; A second determination module is used to determine, according to the algorithm type, a first server set that conforms to the algorithm in the machine resource cluster, wherein the first server set includes at least one first server, and the algorithm is deployed in each first server; A design module, used to design an algorithm scheduling model according to the content to be reviewed and the first server set; A calculation module, used to calculate the algorithm scheduling model to obtain the policy scheduling parameters of the algorithm scheduling model; The scheduling module is used to perform algorithmic scheduling on the content to be reviewed by using the policy scheduling parameters.

9. A terminal device, characterized in that: The device comprises: a processor and a memory storing computer program instructions; When the processor executes the computer program instructions, the algorithm scheduling method as described in any one of claims 1-7 is implemented.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer program instructions, and when the computer program instructions are executed by a processor, the algorithm scheduling method according to any one of claims 1 to 7 is implemented.

11. A computer program product, characterized in that The computer product comprises a computer program, and when the computer program is executed by a processor, the algorithm scheduling method according to any one of claims 1 to 7 is implemented.

Images