Cache task scheduling method and device, equipment, storage medium and program product

Abstract

The application relates to a cache task scheduling method and device, equipment, a storage medium and a program product. The method comprises the following steps: firstly, in the case of meeting a preset condition, obtaining the task priorities of cache tasks in a task pool comprising a plurality of cache tasks to be processed on a platform side; then, obtaining the network states of a plurality of cluster sides for processing the cache tasks on the platform side; then, determining a target cache task according to the network states of the cluster sides and the task priorities; and finally, performing task publishing processing on the target cache task, so that the target cache task is acquired and processed by the cluster sides. According to the method, the target cache task to be processed preferentially can be determined according to the task priorities of the cache tasks and the network states of the clusters, the cache task with high priority and good cluster network state can be processed more timely, and the scheduling of the cache task is more flexible.

Description

Technical Field

[0001] This application relates to the field of computer task processing technology, and in particular to a cached task scheduling method, apparatus, device, storage medium, and program product. Background Technology

[0002] With the explosive growth of data volume and the sharp increase in computing demands, traditional single-server architectures are no longer sufficient to meet the needs of efficient and stable data processing. Therefore, distributed computing systems have emerged. These systems significantly improve computing efficiency and reliability by breaking down tasks into multiple subtasks and distributing them to multiple clusters for parallel processing.

[0003] In distributed computing systems, caching data from a central repository on the cluster side can improve data access speed and reduce latency. Currently, in large-scale cluster environments, cluster caching task scheduling schemes typically use a first-come, first-served (FIFO) approach, which is inflexible. Summary of the Invention

[0004] Therefore, it is necessary to provide a more flexible cache task scheduling method, apparatus, device, storage medium, and program product to address the aforementioned technical problems.

[0005] Firstly, this application provides a method for scheduling cache tasks, including:

[0006] Under the condition that the preset conditions are met, the task priority of each cached task in the task pool is obtained. The task pool includes multiple cached tasks to be processed on the platform side.

[0007] Obtain the network status of multiple cluster endpoints, which are used to process caching tasks on the platform.

[0008] The target caching task is determined based on the network status of each cluster and the priority of each task.

[0009] The target cache task is published so that each cluster can obtain and process the target cache task.

[0010] In the above embodiments, firstly, under preset conditions, the task priorities of each cache task in a task pool containing multiple cache tasks to be processed on the platform are obtained. Then, the network status of multiple clusters used to process the cache tasks on the platform is obtained. Next, based on the network status of each cluster and the priority of each task, the target cache task is determined. Finally, the target cache task is published for each cluster to acquire and process. In this way, the target cache task can be prioritized based on the task priority of each cache task and the network status of each cluster, ensuring that high-priority cache tasks with good cluster network status are processed more promptly, and making cache task scheduling more flexible.

[0011] In one embodiment, obtaining the task priority of each cached task in the task pool includes:

[0012] Obtain data volume information for each cache task;

[0013] Get the task wait time information for each cached task;

[0014] Based on the data volume information and the waiting time information of each task, the task priority of each cache task is determined.

[0015] In the above embodiments, the task priority of each cache task is determined by the data volume information and task waiting time information of each cache task, so that each cache task can be scheduled and processed according to the task priority, making task scheduling and processing more flexible.

[0016] In one embodiment, the task priority of the caching task is negatively correlated with the amount of data indicated by the data volume information;

[0017] The task priority of a cached task is positively correlated with the task wait time indicated by the task wait time information.

[0018] In the above embodiments, it can be ensured that both smaller caching tasks and caching tasks with excessively long waiting times can obtain higher scheduling priorities and be scheduled to each cluster to complete caching jobs more quickly. This will result in shorter waiting times for users to use the data packet on that cluster, thereby improving the user experience.

[0019] In one embodiment, each caching task includes a cluster identifier, and the target caching task is determined based on the network status of each cluster and the priority of each task, including:

[0020] The target cluster is determined based on the network status of each cluster terminal.

[0021] Based on the target cluster identifier of the target cluster, select cache tasks in the task pool that have the same cluster identifier as the target cluster identifier as the first cache task;

[0022] The cache tasks with a priority higher than a preset priority threshold in each of the first cache tasks are identified as target cache tasks.

[0023] In the above embodiments, the target cluster terminal for sending cached tasks is determined based on the network status of each cluster terminal, thereby implementing task rate limiting for each cluster terminal. This allows for more efficient use of the cluster's network resources and enables the entire distributed system to operate more rationally and efficiently.

[0024] In one embodiment, the method further includes:

[0025] Obtain task completion status information from each cluster endpoint;

[0026] The target cluster is determined based on the network status of each cluster, including:

[0027] The target cluster is determined based on the task completion status information and network status of each cluster.

[0028] In the above embodiments, by obtaining the task completion status information of each cluster, the completion status of each cache task can be determined, which facilitates the platform to schedule and process the cache tasks according to the completion status of each cache task.

[0029] In one embodiment, the target cluster is determined based on the task completion status information and network status of each cluster, including:

[0030] Clusters whose task completion status information is determined to be more than a preset threshold are identified as candidate clusters.

[0031] The target cluster is determined based on the network status of each candidate cluster.

[0032] In the above embodiments, the target cluster is determined based on the task completion status information and network status of each cluster, thereby realizing real-time dynamic task rate limiting and improving the stability of the distributed system.

[0033] Secondly, this application also provides a cache task scheduling apparatus, comprising:

[0034] The first acquisition module is used to acquire the task priority of each cached task in the task pool when the preset conditions are met. The task pool includes multiple cached tasks to be processed on the platform side.

[0035] The second acquisition module is used to acquire the network status of multiple cluster terminals, which are used to process the caching tasks on the platform.

[0036] The determination module is used to determine the target cache task based on the network status and task priority of each cluster end;

[0037] The publishing module is used to publish target cache tasks so that each cluster end can obtain and process the target cache tasks.

[0038] Thirdly, this application also provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the cached task scheduling method described in any of the first aspects above.

[0039] Fourthly, this application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the cache task scheduling method described in any of the first aspects above.

[0040] Fifthly, this application also provides a computer program product, including a computer program that, when executed by a processor, implements the steps of the cached task scheduling method described in any of the first aspects above.

[0041] The aforementioned cache task scheduling method, apparatus, device, storage medium, and program product first, under preset conditions, obtains the task priorities of each cache task in a task pool containing multiple cache tasks to be processed on the platform side. Then, it obtains the network status of multiple clusters used to process the cache tasks on the platform side. Next, based on the network status of each cluster and the priority of each task, it determines the target cache task. Finally, it publishes the target cache task for each cluster to obtain and process. In this way, the target cache task can be prioritized based on the task priority of each cache task and the network status of each cluster, enabling higher-priority cache tasks with good cluster network status to be processed more promptly, and making cache task scheduling more flexible. Attached Figure Description

[0042] To more clearly illustrate the technical solutions in the embodiments of this application or related technologies, the drawings used in the description of the embodiments of this application or related technologies will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0043] Figure 1 This is an application environment diagram of a cache task scheduling method in one embodiment;

[0044] Figure 2 This is a flowchart illustrating a cache task scheduling method in one embodiment;

[0045] Figure 3This is a flowchart illustrating the task priority calculation steps for each cached task in one embodiment;

[0046] Figure 4 This is a flowchart illustrating the target cache task determination steps in one embodiment;

[0047] Figure 5 This is a flowchart illustrating the steps for determining the target cluster in one embodiment;

[0048] Figure 6 This is a flowchart illustrating the cache task scheduling method in another embodiment;

[0049] Figure 7 This is a schematic diagram of the system structure of the cache task scheduling method in one embodiment;

[0050] Figure 8 This is a structural block diagram of a cache task scheduling device in one embodiment;

[0051] Figure 9 This is an internal structural diagram of a computer device in one embodiment. Detailed Implementation

[0052] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0053] In distributed computing systems, caching data from a central repository on the cluster side can improve data access speed and reduce latency. Currently, in large-scale cluster environments, cluster caching task scheduling schemes typically use a first-come, first-served (FIFO) approach. In this case, smaller data packets may be forced to wait for a long time because larger packets are already being cached. Furthermore, under network congestion, continuously adding new caching tasks can lead to performance degradation or even crashes of the entire distributed system. Therefore, this caching task scheduling method is not flexible.

[0054] In view of this, this application provides a cache task scheduling method. Under preset conditions, it obtains the task priorities of each cache task in a task pool containing multiple cache tasks to be processed on the platform side. Then, it obtains the network status of multiple clusters used to process the cache tasks on the platform side. Next, based on the network status of each cluster and the priority of each task, it determines the target cache task. Finally, it publishes the target cache task for each cluster to obtain and process. This method can determine the target cache task to be processed first based on the task priority of each cache task and the network status of each cluster, enabling higher-priority cache tasks with good cluster network status to be processed more promptly, and making cache task scheduling more flexible.

[0055] The cache task scheduling method provided in this application embodiment can be applied to, for example, Figure 1 In the application environment shown, platform 102 communicates with each cluster terminal 106 via message middleware platform 104. Each cluster terminal 106 can cache data packets from platform 102. Platform 102 can be an independent physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing cloud computing services. Message middleware platform 104 can be a communication middleware capable of supporting communication between different systems in a distributed system. Each cluster terminal 106 can be an independent physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing cloud computing services.

[0056] In one exemplary embodiment, such as Figure 2 As shown, a cache task scheduling method is provided, which can be applied to... Figure 1 Taking the platform side as an example, the explanation includes the following steps 201 to 204. Wherein:

[0057] Step 201: Under the condition that the preset conditions are met, obtain the task priority of each cached task in the task pool.

[0058] The task pool includes multiple cached tasks awaiting processing on the platform. A preset condition could be timed intervals, at which point the task priority of each cached task in the pool is calculated. Task priority characterizes the order in which cached tasks are processed; higher-priority tasks are processed more promptly. Optionally, task priority can be calculated based on the size of the cached task's data and its waiting time.

[0059] Optionally, after receiving a cache task, the platform assigns a task identifier to the cache task, calculates its initial priority, and stores it in the task pool. Then, the platform can periodically recalculate the task priority of each cache task in the task pool according to a preset time.

[0060] Step 202: Obtain the network status of multiple cluster terminals.

[0061] Multiple clusters are used to process caching tasks on the platform. Network status can include information such as the cluster identifier, total bandwidth, bandwidth used, and bandwidth utilization of the current cluster. To improve data access efficiency and reduce latency on each cluster, each cluster can cache data from the platform. However, if a cluster is currently executing too many tasks and experiencing high load, continuing to receive cached tasks may affect its normal operation. Therefore, to achieve dynamic task rate limiting on the clusters, the network status of multiple clusters is obtained. Based on the network status of each cluster, it is determined whether to send cached tasks to each cluster, thereby achieving task rate limiting for each cluster. For example, if the network status of a cluster is good, such as low bandwidth utilization, cached tasks can be sent to that cluster; if the network status of a cluster is poor, such as high bandwidth utilization, cached tasks will not be sent to that cluster.

[0062] Step 203: Determine the target cache task based on the network status of each cluster and the priority of each task.

[0063] Optionally, the target cache task can be a cache task that has been filtered and prioritized. The cache tasks are grouped according to each cluster. The target cache task can be determined based on the network status of each cluster and the task priority of the cache tasks in their respective groups. For example, if the network status of cluster A is good, the group of cache tasks corresponding to cluster A is determined. Then, the target cache task is determined based on the task priority of each cache task in the group corresponding to cluster A.

[0064] Step 204: The target cache task is published so that each cluster can obtain and process the target cache task.

[0065] Optionally, the platform can publish the target caching tasks. For example, the platform can publish the target caching tasks through a message middleware. The message middleware enables communication between the platform and the cluster, such as using Kafka. Kafka uses message topics for message publishing and subscription. The platform publishes the caching tasks to the corresponding topics in Kafka, and each cluster retrieves the caching tasks from the corresponding topics in Kafka through subscription. It is understood that the message middleware of this application can also be implemented using other communication middleware, and this embodiment does not limit this.

[0066] Optionally, obtaining the network status of multiple clusters can include each cluster publishing its network status to a corresponding Kafka topic at regular intervals, such as every 30 seconds, and the platform obtaining the network status of each cluster from the corresponding Kafka topic by subscribing.

[0067] In the above embodiments, firstly, under preset conditions, the task priorities of each cache task in a task pool containing multiple cache tasks to be processed on the platform are obtained. Then, the network status of multiple clusters used to process the cache tasks on the platform is obtained. Next, based on the network status of each cluster and the priority of each task, the target cache task is determined. Finally, the target cache task is published for each cluster to acquire and process. In this way, the target cache task can be prioritized based on the task priority of each cache task and the network status of each cluster, ensuring that high-priority cache tasks with good cluster network status are processed more promptly, and making cache task scheduling more flexible.

[0068] In one embodiment, the steps for obtaining the task priority of each cached task in the task pool are as follows: Figure 3 As shown, it includes:

[0069] Step 301: Obtain the data volume information for each cache task.

[0070] The data volume information for the caching task can be the data packet size of the caching task, and the unit of data packet size can be bytes. Optionally, in order to allow smaller data packets to be processed first, smaller data packets are given higher priority. Optionally, the data packet size Ws can be calculated according to the following formula.

[0071]

[0072] Where C is a constant value that can be set as needed, and S is the data packet size.

[0073] Step 302: Obtain the task wait time information for each cached task.

[0074] The task waiting time is the time a cached task has been waiting to be sent in the task pool, and the unit of the task waiting time can be seconds. Optionally, to avoid excessively long task waiting times, cached tasks with longer waiting times have higher priority. Optionally, the task waiting time Wt can be calculated according to the following formula.

[0075]

[0076] Where T is the task waiting time.

[0077] Step 303: Determine the task priority of each cached task based on the data volume information and the waiting time information of each task.

[0078] Optionally, the task priority of a cache task is negatively correlated with the amount of data indicated by the data volume information; the task priority of a cache task is positively correlated with the task waiting time indicated by the task waiting time information. Therefore, the task priority P can be calculated according to the following formula.

[0079]

[0080] Here, α and β are the importance coefficients for the size of each cached data packet and the task waiting time, respectively, and α + β = 1. α and β can be set according to the actual situation.

[0081] Optionally, the cached tasks in the task pool can be sorted according to their cluster groups based on the calculated task priority P of each cached task.

[0082] In the above embodiments, the task priority of each cache task is determined by the data volume information and task waiting time information of each cache task, so that each cache task can be scheduled and processed according to the task priority, making task scheduling and processing more flexible.

[0083] Furthermore, it ensures that both smaller caching tasks and caching tasks with excessively long waiting times can receive higher scheduling priorities and be scheduled to various clusters to complete caching jobs more quickly. This results in shorter waiting times for users to access the data packets on those clusters, thereby improving the user experience.

[0084] In the embodiments of this application, each caching task includes a cluster identifier. The steps for determining the target caching task based on the network status of each cluster and the priority of each task are as follows: Figure 4 As shown, it includes:

[0085] Step 401: Determine the target cluster based on the network status of each cluster terminal.

[0086] Optionally, after obtaining the network status of each cluster, it is possible to determine parameters such as the total bandwidth, the bandwidth used, and the bandwidth utilization rate of each cluster. In order to prevent the cluster from being overloaded due to too many cluster tasks being executed, it is preferable to determine the cluster with a large total bandwidth, low bandwidth used, and low bandwidth utilization rate as the target cluster. That is, the target cluster is the cluster that is currently relatively idle and can receive and process cached tasks.

[0087] Step 402: Based on the target cluster identifier of the target cluster, select cache tasks in the task pool that have the same cluster identifier as the target cluster identifier as the first cache task.

[0088] Optionally, if the cluster identifier of the cache task is the same as the target cluster identifier, that is, the cache task is a cache task on the target cluster side, then the first cache task is also the cache task corresponding to the target cluster side. If the cache tasks in the task pool are grouped according to each cluster side, then the first cache task is the cache task in the group corresponding to the target cluster side.

[0089] Step 403: Select the cache tasks with a priority higher than a preset priority threshold from the first cache tasks as target cache tasks.

[0090] Optionally, in order to prioritize the processing of tasks with high priority, cache tasks in the first cache task whose priority is higher than a preset priority threshold are selected as target cache tasks and published with priority.

[0091] In the above embodiments, the target cluster terminal for sending cached tasks is determined based on the network status of each cluster terminal, thereby implementing task rate limiting for each cluster terminal. This allows for more efficient use of the cluster's network resources and enables the entire distributed system to operate more rationally and efficiently.

[0092] Optionally, the method further includes: obtaining task completion status information from each cluster endpoint. That is, step 401 above may also include: determining the target cluster endpoint based on the task completion status information and network status of each cluster endpoint.

[0093] Optionally, task completion status information from multiple clusters can be obtained through a message broker. This information may include task identifier, cluster identifier, task priority, and completion status. Each cluster publishes its task completion status information to a corresponding Kafka topic. The platform subscribes to this topic to retrieve the task completion status information from each cluster. The platform can then record and display the completion status of each cached task based on this information for future reference.

[0094] In the above embodiments, by obtaining the task completion status information of each cluster, the completion status of each cache task can be determined, which facilitates the platform to schedule and process the cache tasks according to the completion status of each cache task.

[0095] Furthermore, the target cluster can be determined based on the task completion status information and network status of each cluster terminal, such as... Figure 5 As shown, it includes:

[0096] Step 501: Select clusters whose task completion status information success rate is greater than a preset threshold as candidate clusters.

[0097] Optionally, based on the task completion status information of each cluster, the completion status of the cache tasks on each cluster can be statistically analyzed to obtain the success rate of cache task execution on each cluster. It is understandable that if the success rate of cache task execution on a cluster is low, it can be determined that there is a fault on that cluster causing cache task execution to fail. Therefore, sending cache tasks to that cluster can be temporarily suspended. Thus, clusters with a success rate greater than a preset threshold are identified as candidate clusters that can send cache tasks.

[0098] Step 502: Determine the target cluster based on the network status of each candidate cluster.

[0099] Then, based on the network status of each candidate cluster, the candidate cluster with good network status is selected as the target cluster.

[0100] In the above embodiments, the target cluster is determined based on the task completion status information and network status of each cluster, thereby realizing real-time dynamic task rate limiting and improving the stability of the distributed system.

[0101] In embodiments of this application, a cache task scheduling method is provided, such as... Figure 6 As shown, it includes:

[0102] Step 601: Under the condition that the preset conditions are met, obtain the data volume information of each cache task.

[0103] Step 602: Obtain the task wait time information for each cached task.

[0104] Step 603: Determine the task priority of each cached task based on the data volume information and the waiting time information of each task.

[0105] Step 604: Obtain the network status of multiple cluster terminals.

[0106] Step 605: Determine the target cluster based on the network status of each cluster terminal.

[0107] Step 606: Based on the target cluster identifier of the target cluster, select cache tasks in the task pool that have the same cluster identifier as the target cluster identifier as the first cache task.

[0108] Step 607: Select the cache tasks with a priority higher than a preset priority threshold from the first cache tasks as target cache tasks.

[0109] Step 608: The target cache task is published so that each cluster end can obtain and process the target cache task.

[0110] In the examples of this application, Figure 7The overall system structure diagram of the cache task scheduling and processing method provided in the embodiments of this application is shown.

[0111] The platform can include a task scheduling service, a cluster task rate limiting service, a task sending service, a priority calculation job, and a status feedback service. Upon receiving a cached task, the platform assigns a task identifier, calculates its initial priority, and stores it in the task pool. The priority calculation job is a background service used to periodically recalculate the priorities of waiting cached tasks in the task pool. The task scheduling service determines whether to send cached tasks to each cluster based on the network status of each cluster obtained from the status feedback service, thus achieving dynamic task rate limiting. For target clusters where cached tasks need to be sent, it retrieves and publishes higher-priority target cached tasks based on cluster grouping. The task sending service publishes the target cached tasks to be executed to Kafka's task topics (tasks). The status feedback service obtains the network status of each cluster by subscribing to Kafka's cluster network status topic (net-status) and the task completion status information of each cluster by subscribing to the task completion status topic (tasks-status). Optionally, the data format of the task topic can be as shown in Table 1 below.

[0112] Table 1 Data Format of Task Topics

[0113]

[0114] The data format for the network status topic of each cluster can be shown in Table 2 below.

[0115] Table 2 Data Format for Network Status Topic

[0116]

[0117] The data format for the task completion status topic on each cluster can be shown in Table 3 below.

[0118] Table 3 Data Format for Task Completion Status Topic

[0119]

[0120] Each cluster endpoint includes a task receiving service, a network monitoring service, and multiple task execution threads. The task receiving service retrieves cached tasks from the Kafka `tasks` topic, first determining if the task belongs to this cluster. If so, it submits the task to a thread pool for execution by the task execution threads. The task execution threads publish the execution results (failure or completion) of the cached tasks to the `tasks-status` topic. The network monitoring task periodically (e.g., every 30 seconds) analyzes the network status of the current cluster and publishes the results to the `net-status` topic.

[0121] It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.

[0122] Based on the same inventive concept, this application also provides a cache task scheduling apparatus for implementing the cache task scheduling method described above. The solution provided by this apparatus is similar to the implementation described in the above method; therefore, the specific limitations in one or more cache task scheduling apparatus embodiments provided below can be found in the limitations of the cache task scheduling method described above, and will not be repeated here.

[0123] In one exemplary embodiment, such as Figure 8 As shown, a cache task scheduling device 800 is provided, including: a first acquisition module 801, a second acquisition module 802, a determination module 803, and a publishing module 804, wherein:

[0124] The first acquisition module 801 is used to acquire the task priority of each cached task in the task pool when the preset conditions are met. The task pool includes multiple cached tasks to be processed on the platform side.

[0125] The second acquisition module 802 is used to acquire the network status of multiple cluster terminals, which are used to process the caching tasks on the platform.

[0126] The determination module 803 is used to determine the target cache task based on the network status and task priority of each cluster end;

[0127] The publishing module 804 is used to publish the target cache task so that each cluster end can obtain and process the target cache task.

[0128] In one embodiment, the first acquisition module 801 is specifically used to acquire data volume information of each cache task; acquire task waiting time information of each cache task; and determine the task priority of each cache task based on the data volume information and the task waiting time information.

[0129] In one embodiment, the task priority of the caching task is negatively correlated with the amount of data indicated by the data volume information; the task priority of the caching task is positively correlated with the task waiting time indicated by the task waiting time information.

[0130] In one embodiment, each cache task includes a cluster identifier. The determining module 803 is specifically used to determine the target cluster based on the network status of each cluster; based on the target cluster identifier of the target cluster, filter cache tasks in the task pool that have the same cluster identifier as the target cluster identifier as first cache tasks; and determine the cache tasks in each first cache task whose task priority is higher than a preset priority threshold as target cache tasks.

[0131] In one embodiment, the device further includes a third acquisition module for acquiring task completion status information of each cluster terminal; the determination module 803 is also used to determine the target cluster terminal based on the task completion status information of each cluster terminal and the network status of each cluster terminal.

[0132] In one embodiment, the determining module 803 is further configured to determine the cluster terminals whose success rate in determining the task completion status information of each cluster terminal is greater than a preset threshold as candidate cluster terminals; and determine the target cluster terminal based on the network status of each candidate cluster terminal.

[0133] Each module in the aforementioned cached task scheduling device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device, or stored in the memory of a computer device as software, so that the processor can invoke and execute the operations corresponding to each module.

[0134] In one exemplary embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be as follows: Figure 9As shown, this computer device includes a processor, memory, input / output interfaces (I / O), and a communication interface. The processor, memory, and I / O interfaces are connected via a system bus, and the communication interface is also connected to the system bus via the I / O interfaces. The processor provides computational and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and a database. The internal memory provides the environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The database stores cached task data. The I / O interfaces are used for exchanging information between the processor and external devices. The communication interface is used for communicating with external terminals via a network connection. When the computer program is executed by the processor, it implements a cached task scheduling method.

[0135] Those skilled in the art will understand that Figure 9 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0136] In one exemplary embodiment, a computer device is provided, including a memory and a processor. The memory stores a computer program, and the processor executes the computer program to perform the following steps: under the condition of satisfying preset conditions, obtaining the task priority of each cached task in a task pool, the task pool including multiple cached tasks to be processed on the platform side; obtaining the network status of multiple cluster ends, the multiple cluster ends being used to process the cached tasks on the platform side; determining a target cached task based on the network status of each cluster end and the priority of each task; and publishing the target cached task for each cluster end to obtain and process the target cached task.

[0137] In one embodiment, when the processor executes the computer program, it further performs the following steps: obtaining data volume information for each cache task; obtaining task wait time information for each cache task; and determining the task priority of each cache task based on the data volume information and the task wait time information.

[0138] In one embodiment, the task priority of a caching task is negatively correlated with the amount of data indicated by the data volume information; the task priority of a caching task is positively correlated with the task waiting time indicated by the task waiting time information.

[0139] In one embodiment, each cache task includes a cluster identifier, and when the processor executes the computer program, it further implements the following steps: determining the target cluster based on the network status of each cluster; selecting cache tasks with the same cluster identifier as the target cluster identifier from the task pool as first cache tasks based on the target cluster identifier of the target cluster; and determining the cache tasks with a priority higher than a preset priority threshold among the first cache tasks as target cache tasks.

[0140] In one embodiment, when the processor executes the computer program, it further performs the following steps: obtaining task completion status information of each cluster terminal; and determining the target cluster terminal based on the task completion status information of each cluster terminal and the network status of each cluster terminal.

[0141] In one embodiment, when the processor executes the computer program, it further performs the following steps: determining the cluster terminals whose success rate in determining the task completion status information of each cluster terminal is greater than a preset threshold as candidate cluster terminals; and determining the target cluster terminal based on the network status of each candidate cluster terminal.

[0142] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, it performs the following steps: under the condition of satisfying preset conditions, obtaining the task priority of each cached task in a task pool, the task pool including multiple cached tasks to be processed on the platform side; obtaining the network status of multiple cluster ends, the multiple cluster ends being used to process the cached tasks on the platform side; determining the target cached task based on the network status of each cluster end and the priority of each task; and publishing the target cached task for each cluster end to obtain and process the target cached task.

[0143] In one embodiment, when the computer program is executed by the processor, it further performs the following steps: obtaining data volume information for each cache task; obtaining task wait time information for each cache task; and determining the task priority of each cache task based on the data volume information and the task wait time information.

[0144] In one embodiment, the task priority of a caching task is negatively correlated with the amount of data indicated by the data volume information; the task priority of a caching task is positively correlated with the task waiting time indicated by the task waiting time information.

[0145] In one embodiment, each cache task includes a cluster identifier, and when the computer program is executed by the processor, it further implements the following steps: determining the target cluster based on the network status of each cluster; selecting cache tasks with the same cluster identifier as the target cluster identifier from the task pool as first cache tasks based on the target cluster identifier of the target cluster; and determining the cache tasks with a priority higher than a preset priority threshold among the first cache tasks as target cache tasks.

[0146] In one embodiment, when the computer program is executed by the processor, it further performs the following steps: obtaining task completion status information of each cluster terminal; and determining the target cluster terminal based on the task completion status information of each cluster terminal and the network status of each cluster terminal.

[0147] In one embodiment, when the computer program is executed by the processor, it further performs the following steps: identifying clusters with a success rate of task completion status information determination greater than a preset threshold as candidate clusters; and determining the target cluster based on the network status of each candidate cluster.

[0148] In one embodiment, a computer program product is provided, including a computer program that, when executed by a processor, performs the following steps: under the condition of satisfying preset conditions, obtaining the task priority of each cached task in a task pool, the task pool including multiple cached tasks to be processed on the platform side; obtaining the network status of multiple cluster ends, the multiple cluster ends being used to process the cached tasks on the platform side; determining a target cached task based on the network status of each cluster end and the priority of each task; and publishing the target cached task for each cluster end to obtain and process the target cached task.

[0149] In one embodiment, when the computer program is executed by the processor, it further performs the following steps: obtaining data volume information for each cache task; obtaining task wait time information for each cache task; and determining the task priority of each cache task based on the data volume information and the task wait time information.

[0150] In one embodiment, the task priority of a caching task is negatively correlated with the amount of data indicated by the data volume information; the task priority of a caching task is positively correlated with the task waiting time indicated by the task waiting time information.

[0151] In one embodiment, each cache task includes a cluster identifier, and when the computer program is executed by the processor, it further implements the following steps: determining the target cluster based on the network status of each cluster; selecting cache tasks with the same cluster identifier as the target cluster identifier from the task pool as first cache tasks based on the target cluster identifier of the target cluster; and determining the cache tasks with a priority higher than a preset priority threshold among the first cache tasks as target cache tasks.

[0152] In one embodiment, when the computer program is executed by the processor, it further performs the following steps: obtaining task completion status information of each cluster terminal; and determining the target cluster terminal based on the task completion status information of each cluster terminal and the network status of each cluster terminal.

[0153] In one embodiment, when the computer program is executed by the processor, it further performs the following steps: identifying clusters with a success rate of task completion status information determination greater than a preset threshold as candidate clusters; and determining the target cluster based on the network status of each candidate cluster.

[0154] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of the relevant data must comply with relevant regulations.

[0155] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, artificial intelligence (AI) processors, etc., and are not limited to these.

[0156] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.

[0157] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A method for scheduling cached tasks, characterized in that, The method includes: Under the condition that the preset conditions are met, the task priority of each cached task in the task pool is obtained, and the task pool includes multiple cached tasks to be processed on the platform side. Obtain the network status of multiple cluster terminals, which are used to process the caching tasks of the platform terminal; The target caching task is determined based on the network status of each cluster terminal and the priority of each task. The target cache task is published so that each of the cluster terminals can obtain and process the target cache task.

2. The method according to claim 1, characterized in that, The process of obtaining the task priority of each cached task in the task pool includes: Obtain the data volume information for each of the aforementioned cache tasks; Obtain the task wait time information for each of the aforementioned cached tasks; The task priority of each cache task is determined based on the data volume information and the task waiting time information.

3. The method according to claim 2, characterized in that, The task priority of the cache task is negatively correlated with the amount of data indicated by the data volume information. The task priority of the cached task is positively correlated with the task waiting time indicated by the task waiting time information.

4. The method according to claim 1, characterized in that, Each of the aforementioned caching tasks includes a cluster identifier. The step of determining the target caching task based on the network status of each cluster and the priority of each task includes: The target cluster terminal is determined based on the network status of each cluster terminal. Based on the target cluster identifier of the target cluster, cache tasks with the same cluster identifier as the target cluster identifier are selected from the task pool as the first cache task; The cache task with a priority higher than a preset priority threshold in each of the first cache tasks is identified as the target cache task.

5. The method according to claim 4, characterized in that, The method further includes: Obtain the task completion status information of each of the aforementioned cluster terminals; The step of determining the target cluster terminal based on the network status of each cluster terminal includes: The target cluster is determined based on the task completion status information and network status of each cluster terminal.

6. The method according to claim 5, characterized in that, The step of determining the target cluster based on the task completion status information and network status of each cluster terminal includes: Clusters whose success rate in determining the task completion status information of each cluster terminal is greater than a preset threshold are identified as candidate cluster terminals. The target cluster is determined based on the network status of each candidate cluster.

7. A cached task scheduling device, characterized in that, The device includes: The first acquisition module is used to acquire the task priority of each cached task in the task pool when a preset condition is met. The task pool includes multiple cached tasks to be processed on the platform side. The second acquisition module is used to acquire the network status of multiple cluster terminals, which are used to process the caching tasks of the platform terminal; The determination module is used to determine the target cache task based on the network status of each cluster terminal and the task priority; The publishing module is used to publish the target cache task so that each of the cluster terminals can obtain and process the target cache task.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 6.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 6.

Images