Offline queue resource optimization method and device, electronic equipment and storage medium

By adjusting the execution time of offline tasks when detecting the peak value of computing resources, peak staggering and peak cutting are achieved, the problem of insufficient computing resources is solved, resource utilization and task stability are improved, and cost is reduced.

CN119065824APending Publication Date: 2024-12-03SF TECH CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202310653330.X
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2023-06-02
Publication Date
2024-12-03

AI Technical Summary

Technical Problem

As the amount of business data increases and there is insufficient computing resources, the existing technology expands computing resources is the fastest solution, but it is not the most fundamental solution. How to reduce the additional costs brought by low-value tasks and duplicate construction and reduce the consumption of computing resources. , maintain the health of data assets.

Method used

By detecting that when the target offline queue resource usage reaches its peak, the offline task is used as a pending task to obtain its execution time information. If the preset conditions are met, the task execution time will be adjusted to avoid multiple tasks being executed simultaneously, and the peaking and cutting will be achieved. peak, reduce resource utilization.

Benefits of technology

It effectively increases the available amount of computing resources, avoids resource scarcity, ensures the effectiveness and stability of task execution, and reduces server procurement costs.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119065824A_ABST
    Figure CN119065824A_ABST
Patent Text Reader

Abstract

The embodiment of the invention discloses an off-line queue resource optimization method and device, electronic equipment and a storage medium, and the method comprises the steps: taking an off-line task in a target off-line queue at a target peak value as a to-be-processed off-line task if it is detected that the resource use of the target off-line queue reaches the target peak value; obtaining execution time efficiency information of the to-be-processed offline task, wherein the execution time efficiency information describes a time efficiency requirement which needs to be met by execution of the to-be-processed offline task; if the execution time efficiency information meets a first preset condition, the original execution time of the to-be-processed offline task is adjusted to be target execution time, and the target execution time is different from the original execution time; and when the target execution time is reached, executing the to-be-processed offline task. According to the method, the execution time of each task is adjusted through the execution timeliness of the tasks, the task execution effectiveness and the relevance of upstream and downstream execution of the tasks can be guaranteed while peak shifting and peak clipping execution of each task is achieved, and the task execution stability is guaranteed.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] Embodiments of the present application relate to the field of computer technology, and specifically to an offline queue resource optimization method, device, electronic device, and storage medium, wherein the storage medium includes a computer-readable storage medium. Background Art

[0002] As the amount of business data increases, the computing resources and costs of the cluster continue to increase, and data computing consumes a lot of resources. Therefore, it is urgent to reduce the extra costs caused by low-value tasks and duplicate construction, maintain the high health of data assets, reduce the purchase of servers by reducing costs and increasing efficiency, and effectively reduce the consumption of computing resources.

[0003] The amount of business data continues to increase, computing tasks gradually increase, and computing resources are increasingly insufficient. Expanding computing resources is the fastest solution, but it is not the most fundamental solution. Summary of the invention

[0004] The embodiments of the present application provide an offline queue resource optimization method, device, electronic device and storage medium, which can optimize offline queue resources and improve the available amount of computing resources.

[0005] The present application embodiment provides an offline queue resource optimization method, including:

[0006] If it is detected that the resource usage of the target offline queue reaches the target peak value, the offline tasks in the target offline queue at the target peak value are taken as the offline tasks to be processed;

[0007] Acquire execution timeliness information of the offline task to be processed, where the execution timeliness information describes the timeliness that needs to be met when executing the offline task to be processed;

[0008] If the execution timeliness information satisfies a first preset condition, the original execution time of the to-be-processed offline task is adjusted to a target execution time, where the target execution time is different from the original execution time;

[0009] When the target execution time is reached, the to-be-processed offline task is executed.

[0010] Accordingly, the embodiment of the present application further provides an offline queue resource optimization device, including:

[0011] A determination module, configured to, if it is detected that the resource usage of the target offline queue reaches a target peak value, use the offline tasks in the target offline queue at the target peak value as offline tasks to be processed;

[0012] An acquisition module, used to acquire execution timeliness information of the offline task to be processed, wherein the execution timeliness information describes the timeliness conditions that need to be met when executing the offline task to be processed;

[0013] An adjustment module, configured to adjust the original execution time of the to-be-processed offline task to a target execution time if the execution timeliness information satisfies a first preset condition, wherein the target execution time is different from the original execution time;

[0014] The execution module is used to execute the pending offline task when the target execution time is reached.

[0015] Optionally, in some embodiments of the present application, the acquisition module includes:

[0016] A first acquisition unit is used to acquire the output time of the execution result generated by the to-be-processed offline task after execution;

[0017] A second acquisition unit, used to acquire a usage time of the execution result used by a downstream task of the to-be-processed offline task;

[0018] The first determining unit is configured to determine the execution timeliness information of the to-be-processed offline task according to the production time and the use time.

[0019] Optionally, in some embodiments of the present application, the acquisition module includes:

[0020] The second determining unit is configured to determine the execution timeliness information of the offline task to be processed according to a preset execution timeliness tag of the offline task to be processed.

[0021] In some embodiments of the present application, the offline task to be processed needs to be executed continuously for at least two days, and the acquisition module includes:

[0022] A subtask determining unit, used for taking the tasks of the to-be-processed offline task on each day as to-be-processed offline subtasks;

[0023] A subtask timeliness acquisition unit, used to acquire execution timeliness information of each of the to-be-processed offline subtasks;

[0024] Adjustment modules, including:

[0025] The adjustment unit is configured to adjust, for each offline subtask to be processed, an original execution time of the offline subtask to be processed to a target execution time if the execution timeliness information of the offline subtask to be processed meets a first preset condition.

[0026] In some embodiments of the present application, the device further includes a offline module, and the offline module includes:

[0027] An execution status acquisition unit, used to acquire execution status information of the offline task to be optimized in the target offline queue, wherein the execution status information describes the necessity of executing the offline task to be optimized;

[0028] An offline task determination unit, configured to use the offline task to be optimized that is unnecessary to be executed as described in the execution status information as the offline task to be offline;

[0029] The offline processing unit is used to perform offline processing on the offline tasks to be offline in the target offline queue.

[0030] In some embodiments of the present application, the execution status acquisition unit includes:

[0031] A lineage relationship acquisition subunit is used to acquire the lineage relationship of the offline task to be optimized, wherein the lineage relationship describes the upstream and downstream relationship between tasks, and describes the belonging relationship between tasks and business systems;

[0032] An existence status determination subunit, used to determine the task existence status of the downstream task corresponding to the offline task to be optimized according to the upstream and downstream relationship;

[0033] A business system determination subunit, used to determine the target business system to which the offline task to be optimized belongs according to the belonging relationship;

[0034] An online status determination subunit, used to obtain the system online status of the target business system;

[0035] The first execution status acquisition subunit is used to take execution unnecessary as the execution status information of the offline task to be optimized if the task existence status describes that the downstream task does not exist or the system online status describes that the target business system is offline.

[0036] In some embodiments of the present application, the execution status acquisition unit includes:

[0037] A storage status acquisition subunit, used to acquire the table storage status of the actual processing table corresponding to the offline task to be optimized;

[0038] The second execution status acquisition subunit is used to, if the table storage status describes that the actual processing table is empty or has been empty for a period of time, determine that execution is unnecessary as the execution status information of the offline task to be optimized.

[0039] In some embodiments of the present application, the execution status acquisition unit includes:

[0040] A table determination subunit, used to determine an actual processing table corresponding to the offline task to be optimized, and to determine an actual upstream table corresponding to the actual processing table;

[0041] The third execution status acquisition subunit is used to execute unnecessary as the execution status information of the offline task to be optimized if there is a reference processing table with a table structure consistent with the actual processing table, and the reference upstream table corresponding to the reference processing table is consistent with the actual upstream table.

[0042] In a third aspect, an embodiment of the present application further provides an electronic device, which includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program implements the steps in the above-mentioned offline queue resource optimization method when executed by the processor.

[0043] In a fourth aspect, an embodiment of the present application further provides a storage medium, the storage medium comprising a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the steps in the above-mentioned offline queue resource optimization method are implemented.

[0044] In a fifth aspect, the embodiments of the present application further provide a computer program product or a computer program, which includes a computer instruction stored in a computer-readable storage medium. The processor of the computer device reads the computer instruction from the computer-readable storage medium, and the processor executes the computer instruction, so that the computer device executes the method provided in various optional implementations described in the embodiments of the present application.

[0045] In the embodiment of the present application, if it is detected that the resource usage of the target offline queue reaches the target peak value, the offline tasks in the target offline queue at the target peak value are taken as the offline tasks to be processed, and the execution timeliness information of the offline tasks to be processed is obtained. The execution timeliness information describes the timeliness that needs to be met for the execution of the offline tasks to be processed. If the execution timeliness information satisfies the first preset condition, the original execution time of the offline tasks to be processed is adjusted to the target execution time, which is different from the original execution time, and when the target execution time is reached, the offline tasks to be processed are executed. Among them, by staggering and shaving the peak value of offline tasks under peak conditions, the situation of peak value occurring again when multiple offline tasks are executed simultaneously is avoided, and the problem of shortage of available resources is avoided. Among them, the execution time of each task is adjusted by the execution timeliness of the task. While achieving staggered and shaving the peak value of each task, the effectiveness of task execution and the correlation between upstream and downstream execution of tasks can be guaranteed, thereby ensuring the stability of task execution. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] In order to more clearly illustrate the technical solutions in the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative work.

[0047] Figure 1 Schematic diagram of a scenario of an offline queue resource optimization method provided in an embodiment of the present application;

[0048] Figure 2 It is a flowchart of the offline queue resource optimization method provided in an embodiment of the present application;

[0049] Figure 3 is another flow chart of the offline queue resource optimization method provided in an embodiment of the present application;

[0050] Figure 4 This is a schematic diagram of CPU resource usage of an offline queue provided in an embodiment of the present application;

[0051] Figure 5 This is a schematic diagram of the usage of content resources by an offline queue provided in an embodiment of the present application;

[0052] Figure 6 It is a structural diagram of an offline queue resource optimization device provided in an embodiment of the present application;

[0053] Figure 7 It is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0054] The following will be combined with the drawings in this application to clearly and completely describe the technical solutions in this application. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of them. Based on the embodiments in the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present invention.

[0055] The embodiments of the present application provide an offline queue resource optimization method, device, electronic device and storage medium. Specifically, the embodiments of the present application provide an offline queue resource optimization device suitable for electronic devices, wherein the electronic device includes a server, which can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content distribution networks (CDNs), and big data and artificial intelligence platforms, and the server can be directly or indirectly connected via wired or wireless communications.

[0056] See also Figure 1 , Figure 1 : is a schematic diagram of a scenario in which a server according to an embodiment of the present application executes the offline queue resource optimization method, wherein the specific execution process of the server executing the offline queue resource optimization method is as follows:

[0057] The server 10 starts the offline queue to execute the offline task, and completes the acquisition and update of the corresponding data in an offline manner. If it is detected that the resource usage of the offline queue reaches a peak, the offline task in the offline queue at the peak is used as the offline task to be processed, and the execution timeliness information of the offline task to be processed is obtained. If the execution timeliness information meets the first preset condition, the original execution time of the offline task to be processed is adjusted to a target execution time different from the original execution time, and the offline task to be processed is executed when the target execution time is reached when the offline task to be processed is executed next time.

[0058] In the embodiment of the present application, the execution timeliness information describes the timeliness that needs to be met by the execution of the offline task to be processed, that is, the timeliness requirement that needs to be met by the execution of the offline task to be processed. The timeliness requirement is usually affected by the execution time of the downstream task of the offline task to be processed. For example, if the start execution time of the downstream task is far away from the end execution time of the offline task to be processed, the execution timeliness requirement for the offline task to be processed is low; if the start execution time of the downstream task is close to the end execution time of the offline task to be processed, the execution timeliness requirement for the offline task to be processed is high.

[0059] It can be understood that a task corresponds to an operation that needs to be performed, which corresponds to a piece of code in the program, such as querying corresponding data, etc. Offline tasks refer to tasks that are performed offline, such as tasks that are configured to be automatically performed at a certain time period at night, such as regular transaction data updates, regular waybill data updates, etc.

[0060] It is understandable that by staggering and shaving the peak of offline tasks under peak conditions, we can avoid the peak situation when multiple offline tasks are executed simultaneously, and avoid the problem of lack of available resources. Among them, by adjusting the execution time of each task through the execution time of the task, while achieving staggered and shaving the peak of each task, we can also ensure the effectiveness of task execution and the correlation between upstream and downstream execution of tasks, and ensure the stability of task execution.

[0061] It should be noted that the order of description of the following embodiments is not intended to limit the priority order of the embodiments.

[0062] See also Figure 2 , Figure 2 A schematic diagram of the process of the offline queue resource optimization method provided in the embodiment of the present application. The specific process of the offline queue resource optimization method is as follows:

[0063] 101. If it is detected that the resource usage of a target offline queue reaches a target peak value, the offline tasks in the target offline queue at the target peak value are taken as offline tasks to be processed.

[0064] It can be understood that a queue is a first-in-first-out data structure. In the embodiment of the present application, a queue refers to a task queue, which is a container for storing tasks and controlling the execution of tasks. Correspondingly, a task queue is used to execute a time-consuming task, and an offline queue refers to a task queue for executing offline tasks.

[0065] A task corresponds to an operation that needs to be performed, and corresponds to a piece of code in the program, such as querying corresponding data. Offline tasks refer to tasks that are performed offline, such as tasks that are configured to be automatically performed during a certain period of time at night, such as regular transaction data updates, regular waybill data updates, etc.

[0066] Resources refer to system resources, such as memory and CPU, and resource usage peak refers to the peak usage of memory and CPU. Correspondingly, target peak refers to reaching the set peak, such as peak value, which means that the available resources of the system or queue are insufficient.

[0067] It is understandable that by using the offline tasks in the offline queue when the target peak value is reached as the offline tasks to be processed, the purpose is to reduce the usage of offline queue resources and avoid the peak value from occurring again by processing the offline tasks to be processed at the peak value.

[0068] 102. Obtain execution timeliness information of the offline task to be processed, where the execution timeliness information describes timeliness requirements that need to be met for execution of the offline task to be processed.

[0069] It can be understood that, by acquiring the execution timeliness information of the offline task to be processed, it is convenient to subsequently adjust the execution time of the offline task to be processed according to the execution timeliness information, thereby realizing the staggered execution of each task.

[0070] 103. If the execution timeliness information satisfies a first preset condition, the original execution time of the to-be-processed offline task is adjusted to a target execution time, where the target execution time is different from the original execution time.

[0071] It can be understood that in the embodiment of the present application, the execution time limit information includes whether there is a requirement for the execution time limit or whether there is no requirement for the execution time limit. Accordingly, the first preset condition refers to that there is no requirement for the execution time limit.

[0072] Among them, by adjusting the execution time of the offline tasks to be processed, the execution of offline tasks will not occur at the time corresponding to the target peak, reducing the tasks executed simultaneously at that time, reducing the utilization of resources at the same time, and avoiding the occurrence of peaks (high peaks).

[0073] Accordingly, the target execution time may be before or after the original execution time. For example, if the original execution time is at midnight every day, the target execution time may be at 3:00 or 22:00 every day. In the embodiment of the present application, it is emphasized that the target execution time is different from the original execution time, and the specific value or specific time of the target execution time is not limited. Those skilled in the art may advance or adjust the execution time of the corresponding offline task to be processed to a later time according to the requirements of the execution timeliness information.

[0074] It can be understood that by adjusting the execution time of the offline task to be processed based on the execution time information, while avoiding peak conditions, it can also ensure the effectiveness of the execution of the offline task to be processed and ensure the correlation between the execution of tasks. For example, the adjustment of the execution time of the offline task to be processed does not affect the acquisition or use of the execution results of the offline task by the downstream task.

[0075] 104. When the target execution time is reached, execute the pending offline task.

[0076] It can be understood that the target execution time is the next execution time of the pending offline task. For example, each offline task is configured with a fixed execution time. For example, the data of the previous day will be updated at midnight every day. Accordingly, the original execution time of the offline task for data update is at midnight every day. When the offline task is a pending offline task, it indicates that the task appears at the time corresponding to the peak value, that is, the task is being executed or is about to be completed in the offline queue. Accordingly, in the embodiment of the present application, the next execution time will be determined according to its execution timeliness information, for example, the offline task for data update will be executed at three o'clock tomorrow. Accordingly, when it reaches three o'clock tomorrow, the offline task will be executed.

[0077] It is understandable that by staggering and shaving the peak of offline tasks under peak conditions, we can avoid the peak situation when multiple offline tasks are executed simultaneously, and avoid the problem of lack of available resources. Among them, by adjusting the execution time of each task through the execution time of the task, while achieving staggered and shaving the peak of each task, we can also ensure the effectiveness of task execution and the correlation between upstream and downstream execution of tasks, and ensure the stability of task execution.

[0078] It should be noted that the execution timeliness information describes the timeliness requirements that need to be met for the execution of the offline task to be processed. In the embodiment of the present application, the timeliness requirements may include timeliness requirements or no timeliness requirements. Among them, the timeliness requirement means that the offline task needs to be executed in a timely manner, so it is not convenient to adjust the execution time of the offline task. On the contrary, no timeliness requirement means that the offline task does not need to be executed in a timely manner, and there is time space that can be adjusted. Therefore, the execution time of this type of offline task can be adjusted to stagger the number of tasks executed at the same time, avoid concentrated utilization of resources, and avoid the occurrence of resource utilization peaks.

[0079] Optionally, tasks usually have an upstream and downstream relationship, that is, the execution of downstream tasks depends on the execution of upstream tasks. For example, the data generated by the execution of upstream tasks is required to execute downstream tasks. Therefore, for this type of task, its execution often has some timeliness requirements. However, since it is an offline task, the execution of downstream tasks has a corresponding start time. Therefore, in an embodiment of the present application, it is also possible to judge whether the execution of the upstream task meets the corresponding timeliness requirements based on the interval between the start execution time of the downstream task and the end execution time of the upstream task.

[0080] Therefore, the timeliness requirement of the execution of the offline task to be processed, i.e., the execution timeliness information, can be determined according to the interval between the start execution time of the downstream task and the end execution time of the offline task to be processed. Wherein, the offline task to be processed is the upstream task of the downstream task, i.e., optionally, in some embodiments of the present application, the step of "obtaining the execution timeliness information of the offline task to be processed" includes:

[0081] Obtaining the output time of the execution result of the offline task to be processed after execution;

[0082] Obtaining the usage time of the execution result used by the downstream task of the to-be-processed offline task;

[0083] Determine execution timeliness information of the to-be-processed offline task according to the production time and the use time.

[0084] It should be noted that in order to balance the scheduling of resources for each task, each task will be scheduled with a corresponding start execution time and end execution time. The start execution time and end execution time of each task can be obtained from the task information of the queue, by capturing relevant code or calling relevant node functions. There is no limitation on the method of obtaining the execution time.

[0085] Among them, the end execution time is also the output time of the execution result. Therefore, the end execution time of the offline task to be processed can be directly used as the output time of the execution result, and the interval between the output time and the use time (start execution time) of the downstream task can be used to determine the execution timeliness information of the offline task to be processed.

[0086] For example, if the usage time of the downstream task is far from the output time, it indicates that the pending offline task does not have a strong timeliness requirement. On the contrary, if the usage time of the downstream task is close to the output time, it indicates that the pending offline task has a strong timeliness requirement. For example, if the output time is 2 a.m. and the usage time is 3 a.m., it indicates that the execution of the pending offline task has a strong timeliness requirement and needs to be completed in time. Correspondingly, if the output time is 2 a.m. and the usage time is 8 a.m., it indicates that the execution of the pending offline task does not have a strong timeliness requirement. Therefore, it is not necessary to complete the execution of the pending offline task in time, that is, the execution of the pending offline task can be postponed for a period of time.

[0087] Optionally, for each offline task, the timeliness of the task may be marked in advance to obtain an execution timeliness label. When the execution time of the offline task needs to be adjusted, the execution time of the task may be adjusted according to the execution timeliness label. That is, optionally, in some embodiments of the present application, the step of "obtaining the execution timeliness information of the offline task to be processed" includes:

[0088] The execution timeliness information of the offline task to be processed is determined according to the preset execution timeliness tag of the offline task to be processed.

[0089] It is understandable that the execution timeliness label can be pre-configured based on experience parameters, or upstream and downstream relationships of tasks, etc.

[0090] Optionally, some offline tasks need to be continued for multiple days, but the timeliness requirements of each day are not the same. For example, for logistics update data, the duration of the data update task may be as long as half a month, and in half a month, the user may only care about the logistics information of some days. Therefore, the entire logistics data update task can be split, and the logistics data of each day is updated based on the corresponding timeliness requirements to minimize the number of tasks executed at a fixed time on the same day. That is, optionally, in some embodiments of the present application, the offline task to be processed needs to be executed continuously for at least two days, and the step of "obtaining the execution timeliness information of the offline task to be processed" includes:

[0091] The tasks of the pending offline tasks on each day are taken as pending offline subtasks;

[0092] Obtaining execution time information of each of the offline subtasks to be processed;

[0093] If the execution timeliness information satisfies a first preset condition, adjusting the original execution time of the to-be-processed offline task to the target execution time includes:

[0094] For each offline subtask to be processed, if the execution timeliness information of the offline subtask to be processed meets the first preset condition, the original execution time of the offline subtask to be processed is adjusted to the target execution time.

[0095] By dividing the task into subtasks on a daily basis, the execution time efficiency information of each subtask can be obtained, and then the execution time of each subtask on a daily basis can be adjusted according to its corresponding execution time efficiency information.

[0096] It can be understood that by splitting the tasks, the precision and accuracy of obtaining task execution time efficiency information is improved, and peak-shifting or peak-shaving control can be performed for the execution of the same task at multiple times.

[0097] Among them, for some offline tasks, if the execution of the offline task will not produce corresponding results, or there is no downstream task waiting for the execution result of the upstream offline task, or the offline task has repeated execution, you can consider taking some offline tasks offline to reduce the execution of "unnecessary" impacts and reduce the occupancy of system resources.

[0098] Accordingly, the offline task offline processing can occur during the normal execution of the offline queue, or during the period when the offline queue encounters a resource utilization peak. For example, during the normal execution of the offline queue (a period without a peak), it can be detected whether there are any unnecessary offline tasks in the offline queue. If so, the offline task can be offline processed; or, when the execution of the offline queue reaches a peak in queue resource utilization, the necessity of executing the pending offline tasks in the queue at that moment can be detected. If there are any unnecessary offline tasks, the pending offline tasks can also be offline processed.

[0099] It is understandable that offline processing can occur in the processing process before the peak period, or in the processing process during the peak period. In the embodiment of the present application, the necessity detection of offline task execution and offline processing can be selected to be processed in the processing process of one of the time periods.

[0100] Among them, in the embodiment of the present application, taking offline processing of offline tasks in other time periods other than peak time periods as an example, specifically, during the execution of the offline queue, it is detected whether there are unnecessary offline tasks to be optimized in the offline queue, and if so, the offline tasks to be optimized are offline processed, that is, optionally, in some embodiments of the present application, before the step of "if it is detected that the resource usage of the target offline queue reaches the target peak value, the offline tasks in the target offline queue at the target peak value are used as offline tasks to be processed", the method also includes:

[0101] Acquiring execution status information of the offline task to be optimized in the target offline queue, wherein the execution status information describes the necessity of executing the offline task to be optimized;

[0102] The execution status information describes that unnecessary offline tasks to be optimized are executed as offline tasks to be taken offline;

[0103] The offline tasks to be taken offline in the target offline queue are processed offline.

[0104] Among them, in the embodiment of the present application, the execution status information describes the necessity of executing the offline task to be optimized. By using the execution status information description in the offline queue to perform unnecessary offline processing of the offline task to be optimized, the occupancy of the queue resources can be reduced, and the peak utilization of the queue resources can be effectively avoided.

[0105] Optionally, whether the execution of the offline task to be optimized in the offline queue is necessary can be analyzed and determined based on the lineage relationship of the offline task to be optimized, that is, the execution status information of the offline task to be optimized is determined based on the lineage relationship. For example, it is determined based on the lineage relationship whether there is a downstream task for the offline task to be optimized, and whether the business system corresponding to the offline task to be optimized has been offline, etc., and then whether the execution of the offline task to be optimized is necessary is determined based on whether there is a downstream task and whether the associated business system is offline. That is, optionally, in some embodiments of the present application, the step of "obtaining the execution status information of the offline task to be optimized in the target offline queue" includes:

[0106] Acquire the blood relationship of the offline task to be optimized, where the blood relationship describes the upstream and downstream relationship between tasks, and describes the belonging relationship between tasks and business systems;

[0107] Determine the task existence status of the downstream task corresponding to the offline task to be optimized according to the upstream and downstream relationship;

[0108] Determining the target business system to which the offline task to be optimized belongs according to the belonging relationship;

[0109] Obtaining the system online status of the target business system;

[0110] If the task existence status describes that the downstream task does not exist or the system online status describes that the target business system is offline, execution unnecessary is used as the execution status information of the offline task to be optimized.

[0111] Among them, data lineage is an important component of data assets, which is used to analyze the lineage path of tables and fields from the data source to the current table, and whether the relationship between lineage fields is satisfied, and pays attention to data consistency and the rationality of table design. In the embodiment of the present application, the lineage relationship describes the association relationship between tasks, such as upstream and downstream relationships, and the lineage relationship also describes the relationship between tasks and business systems, that is, describes the business system to which the offline task belongs.

[0112] It can be understood that the task existence status describes the status of whether the task exists, for example, whether the task exists or not. In an embodiment of the present application, the task existence status is described by describing the existence status of the downstream task corresponding to the offline task to be optimized, and describing the status information of whether the downstream task exists.

[0113] Among them, the business system is the system to which the task belongs. The business system generates corresponding tasks during operation. For example, for the logistics order tracking business system, when the system is running, tasks such as obtaining and displaying logistics order data are triggered. Accordingly, when the system is configured to run at a scheduled time, the corresponding task is configured as an offline task.

[0114] It is understandable that when there is no downstream task for the offline task to be optimized, it means that no other task needs to rely on the execution result of the offline task to be optimized, and the execution necessity of the offline task to be optimized is insufficient; when the business system corresponding to the offline task to be optimized is offline, it means that the data relied on by the offline task to be optimized during execution can no longer guarantee timeliness and accuracy, so the execution necessity of the offline task to be optimized is also insufficient. Therefore, offline tasks to be optimized that have no downstream tasks or whose corresponding business systems have been offline can be processed offline, and after these tasks are offline, the impact on subsequent tasks is small, which can ensure the stability of queue task execution.

[0115] Optionally, in an embodiment of the present application, the necessity of executing the offline task to be optimized may also be determined by analyzing the storage status of the processing table corresponding to the offline task to be optimized, that is, determining the execution status information of the offline task to be optimized. That is, optionally, in some embodiments of the present application, the step of "obtaining the execution status information of the offline task to be optimized in the target offline queue" includes:

[0116] Obtaining a table storage status of an actual processing table corresponding to the offline task to be optimized;

[0117] If the table storage status describes that the actual processing table is empty or is empty for a period of time, unnecessary execution is used as the execution status information of the offline task to be optimized.

[0118] The table storage status describes the status of data stored in the table, including stored, not stored, not stored for a period of time, or not updated for a period of time.

[0119] Among them, if the processing table corresponding to the offline task to be optimized is empty or has not been empty for a period of time, it means that the execution of the offline task to be optimized will not produce any actual effect, that is, it means that the task belongs to the empty run type task. Therefore, for this part of the offline tasks to be optimized, offline processing can also be done, and this type of task will not affect the execution of other tasks in the queue after offline processing, ensuring the stability of the execution of other tasks. Therefore, this type of offline task to be optimized can be marked as an unnecessary task, that is, the execution status information of the offline task to be optimized is determined according to the table storage status of the actual processing table corresponding to the offline task to be optimized.

[0120] Optionally, in an embodiment of the present application, the execution status information of the offline task to be optimized may be determined according to whether the offline task to be optimized is repeatedly constructed, wherein whether the offline task to be optimized is repeatedly constructed may be determined according to whether there is a processing table with the same table structure as the offline task to be optimized, and whether there is the same upstream table. That is, optionally, in some embodiments of the present application, the step of "obtaining the execution status information of the offline task to be optimized in the target offline queue" includes:

[0121] Determine an actual processing table corresponding to the offline task to be optimized, and determine an actual upstream table corresponding to the actual processing table;

[0122] If there is a reference processing table having a table structure consistent with the actual processing table, and the reference upstream table corresponding to the reference processing table is consistent with the actual upstream table, execution unnecessary is used as the execution status information of the offline task to be optimized.

[0123] Among them, the processing tables corresponding to the tasks are the same, and the data of the upstream tables used are also the same, which fully indicates that the two tasks are the same and belong to duplicate construction. Therefore, one of the tasks can be processed offline. Therefore, for the offline task to be optimized, execution unnecessary can be used as its corresponding execution status information.

[0124] It can be understood that the embodiment of the present application adjusts the execution time of offline tasks in the offline queue during peak moments to achieve the control of staggered and peak-shaving execution of offline tasks, avoid the re-appearance of peak values ​​in the execution of offline tasks in the queue, avoid insufficient queue resources, and ensure the stability of system execution.

[0125] Among them, by taking unnecessary offline tasks in the queue offline for processing, the resource occupation is further reduced to ensure the execution of necessary tasks. In addition, the offline processing of some offline tasks not only optimizes queue resources, but also avoids peak resource utilization.

[0126] It is understandable that offline processing of offline tasks can also be applied to situations where offline queue resource usage reaches a peak value. For example, when the resources of the offline queue reach a peak value, the execution status information of the offline tasks in the offline queue at that moment is collected to perform offline processing on the corresponding offline tasks that can be offline.

[0127] See also Figure 3 , Figure 3 is another flow chart of the offline queue resource optimization method provided in an embodiment of the present application, wherein the specific process of the offline queue resource optimization includes:

[0128] 201. In response to a business execution requirement, generate an offline queue, where the offline queue includes a plurality of offline tasks to be optimized;

[0129] 202. Obtain the blood relationship of the offline tasks to be optimized in the offline queue, and obtain the actual processing table corresponding to the offline tasks to be optimized;

[0130] 203. Determine, based on the blood relationship, the task existence status of the downstream task of the offline task to be optimized, the target business system to which the offline task to be optimized belongs, and the actual upstream table of the actual processing table corresponding to the offline task to be optimized;

[0131] In this embodiment of the present application, whether the task is used can be determined by whether there is a downstream task for the offline task to be optimized.

[0132] 204. Obtain the system online status of the target business system;

[0133] 205. Obtaining the table storage status of the actual processing table;

[0134] 206. Determine the execution status information of the task to be optimized according to the task existence status, system online status, table storage status or table structure information;

[0135] If the task existence status describes that the downstream task does not exist, or the system online status describes that the target business system is offline, or the table storage status describes that the actual processing table is empty or has been empty for a period of time, or there is a reference processing table with the same table structure as the actual processing table and the reference upstream table corresponding to the reference processing table is consistent with the actual upstream table, then the execution is unnecessary as the execution status information of the offline task to be optimized;

[0136] 207. If the execution status information of the offline task to be optimized indicates that execution is unnecessary, execute step 208;

[0137] 208. Offline processing of the offline task to be optimized;

[0138] 209. If it is detected that the resource usage of the offline queue reaches a peak value, execute step 210;

[0139] Among them, in the embodiment of the present application, resources mainly refer to memory and CPU, and the peak value refers to the peak usage of memory and CPU. Among them, in the embodiment of the present application, the offline task is a T-1 day mode, and yesterday's data is produced today, so the computing peak is mainly in the early morning.

[0140] like Figure 4 As shown, Figure 4 This is a schematic diagram of CPU resource usage of an offline queue provided in an embodiment of the present application, wherein CPU resource usage in the early morning period is shown, with the horizontal axis representing the time trend and the vertical axis representing CPU usage.

[0141] like Figure 5 As shown, Figure 5 This is a schematic diagram of the usage of content resources by an offline queue provided in an embodiment of the present application, wherein the usage of memory resources in the early morning period is shown, the horizontal axis is the time trend, and the vertical axis is the CPU usage.

[0142] Among them, Figure 4 and Figure 5 As shown, since most offline tasks are scheduled to be executed in the early morning, the peak period of queue resource usage mainly occurs in the early morning.

[0143] 210. The offline tasks in the offline queue at the peak moment are taken as offline tasks to be processed;

[0144] 211. Obtain execution time information of the pending offline task;

[0145] In an embodiment of the present application, the execution timeliness information of the offline task to be processed can be determined according to the end execution time of the offline task to be processed and the start execution time of the corresponding downstream task; or, the execution timeliness information of the offline task to be processed can be determined according to a preset execution timeliness tag of the offline task to be processed.

[0146] 212. If the execution time limit information describes that the offline task to be processed has no time limit requirement, execute step 213;

[0147] 213; adjusting the to-be-processed offline task from the original execution time to the target execution time;

[0148] 214. When the target execution time is reached, the pending offline task is executed.

[0149] Among them, in the embodiment of the present application, in case that the execution of an offline task needs to last for multiple days, the offline task can be divided into multiple sub-offline tasks on a daily basis, and the corresponding execution time limit information is determined for each sub-offline task, and the execution time of each sub-offline task is adjusted according to the execution time limit information of each sub-offline task.

[0150] For example, a very small number of abnormal waybills may take one month from shipment to receipt, so 30 days of data need to be updated when updating the receipt status of the waybill. However, in business scenarios with time requirements, only 7 days of data are generally required, so the other 23 days can be updated during the day to reduce the pressure of the early morning peak.

[0151] It can be understood that the embodiment of the present application adjusts the execution time of offline tasks in the offline queue during peak moments to achieve the control of staggered and peak-shaving execution of offline tasks, avoid the re-appearance of peak values ​​in the execution of offline tasks in the queue, avoid insufficient queue resources, and ensure the stability of system execution.

[0152] Among them, by taking unnecessary offline tasks in the queue offline for processing, the resource occupation is further reduced to ensure the execution of necessary tasks. In addition, the offline processing of some offline tasks not only optimizes queue resources, but also avoids peak resource utilization.

[0153] In order to better implement the offline queue resource optimization method of the present application, the present application also provides an offline queue resource optimization device based on the offline queue resource optimization method. The meanings of the terms are the same as those in the offline queue resource optimization method, and the specific implementation details can refer to the description in the method embodiment.

[0154] See also Figure 6 , Figure 6 : is a schematic diagram of the structure of an offline queue resource optimization device provided in an embodiment of the present application, wherein the offline queue resource optimization device may be specifically as follows:

[0155] The determination module 301 is used to, if it is detected that the resource usage of the target offline queue reaches a target peak value, take the offline tasks in the target offline queue at the target peak value as the offline tasks to be processed;

[0156] The acquisition module 302 is used to acquire the execution timeliness information of the offline task to be processed, where the execution timeliness information describes the timeliness that needs to be met when executing the offline task to be processed;

[0157] An adjustment module 303, configured to adjust the original execution time of the offline task to be processed to a target execution time if the execution timeliness information satisfies a first preset condition, wherein the target execution time is different from the original execution time;

[0158] The execution module 304 is used to execute the to-be-processed offline task when the target execution time is reached.

[0159] Optionally, in some embodiments of the present application, the acquisition module 302 includes:

[0160] A first acquisition unit is used to acquire the output time of the execution result generated by the to-be-processed offline task after execution;

[0161] A second acquisition unit, used to acquire a usage time of the execution result used by a downstream task of the to-be-processed offline task;

[0162] The first determining unit is configured to determine the execution timeliness information of the to-be-processed offline task according to the production time and the use time.

[0163] Optionally, in some embodiments of the present application, the acquisition module 302 includes:

[0164] The second determining unit is configured to determine the execution timeliness information of the offline task to be processed according to a preset execution timeliness tag of the offline task to be processed.

[0165] In some embodiments of the present application, the offline task to be processed needs to be executed continuously for at least two days, and the acquisition module includes:

[0166] A subtask determining unit, used for taking the tasks of the to-be-processed offline task on each day as to-be-processed offline subtasks;

[0167] A subtask timeliness acquisition unit, used to acquire execution timeliness information of each of the to-be-processed offline subtasks;

[0168] The adjustment module 303 includes:

[0169] The adjustment unit is configured to adjust, for each offline subtask to be processed, an original execution time of the offline subtask to be processed to a target execution time if the execution timeliness information of the offline subtask to be processed meets a first preset condition.

[0170] In some embodiments of the present application, the device further includes a offline module, and the offline module includes:

[0171] An execution status acquisition unit, used to acquire execution status information of the offline task to be optimized in the target offline queue, wherein the execution status information describes the necessity of executing the offline task to be optimized;

[0172] An offline task determination unit, configured to use the offline task to be optimized that is unnecessary to be executed as described in the execution status information as the offline task to be offline;

[0173] The offline processing unit is used to perform offline processing on the offline tasks to be offline in the target offline queue.

[0174] In some embodiments of the present application, the execution status acquisition unit includes:

[0175] A lineage relationship acquisition subunit is used to acquire the lineage relationship of the offline task to be optimized, wherein the lineage relationship describes the upstream and downstream relationship between tasks, and describes the belonging relationship between tasks and business systems;

[0176] An existence status determination subunit, used to determine the task existence status of the downstream task corresponding to the offline task to be optimized according to the upstream and downstream relationship;

[0177] A business system determination subunit, used to determine the target business system to which the offline task to be optimized belongs according to the belonging relationship;

[0178] An online status determination subunit, used to obtain the system online status of the target business system;

[0179] The first execution status acquisition subunit is used to take execution unnecessary as the execution status information of the offline task to be optimized if the task existence status describes that the downstream task does not exist or the system online status describes that the target business system is offline.

[0180] In some embodiments of the present application, the execution status acquisition unit includes:

[0181] A storage status acquisition subunit, used to acquire the table storage status of the actual processing table corresponding to the offline task to be optimized;

[0182] The second execution status acquisition subunit is used to, if the table storage status describes that the actual processing table is empty or has been empty for a period of time, determine that execution is unnecessary as the execution status information of the offline task to be optimized.

[0183] In some embodiments of the present application, the execution status acquisition unit includes:

[0184] A table determination subunit, used to determine an actual processing table corresponding to the offline task to be optimized, and to determine an actual upstream table corresponding to the actual processing table;

[0185] The third execution status acquisition subunit is used to execute unnecessary as the execution status information of the offline task to be optimized if there is a reference processing table with a table structure consistent with the actual processing table, and the reference upstream table corresponding to the reference processing table is consistent with the actual upstream table.

[0186] In the embodiment of the present application, first, if the determination module 301 detects that the resource usage of the target offline queue has reached the target peak value, the offline task in the target offline queue at the target peak value is used as the offline task to be processed. Then, the acquisition module 302 acquires the execution timeliness information of the offline task to be processed, and the execution timeliness information describes the timeliness that needs to be met for the execution of the offline task to be processed. Subsequently, if the execution timeliness information satisfies the first preset condition, the adjustment module 303 adjusts the original execution time of the offline task to be processed to the target execution time, and the target execution time is different from the original execution time. Then, when the target execution time is reached, the execution module 304 executes the offline task to be processed.

[0187] It is understandable that by staggering and shaving the peak of offline tasks under peak conditions, we can avoid the peak situation when multiple offline tasks are executed simultaneously, and avoid the problem of lack of available resources. Among them, by adjusting the execution time of each task through the execution time of the task, while achieving staggered and shaving the peak of each task, we can also ensure the effectiveness of task execution and the correlation between upstream and downstream execution of tasks, and ensure the stability of task execution.

[0188] In addition, the present application also provides an electronic device, such as Figure 7 As shown, it shows a schematic diagram of the structure of the electronic device involved in this application, specifically:

[0189] The electronic device may include components such as a processor 401 with one or more processing cores, a memory 402 with one or more computer-readable storage media, a power supply 403, and an input unit 404. Those skilled in the art will appreciate that Figure 7 The electronic device structure shown in the figure does not constitute a limitation on the electronic device, and may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently.

[0190] The processor 401 is the control center of the electronic device, and uses various interfaces and lines to connect various parts of the entire electronic device. By running or executing software programs and / or modules stored in the memory 402, and calling data stored in the memory 402, the processor 401 performs various functions of the electronic device and processes data, thereby monitoring the electronic device as a whole. Optionally, the processor 401 may include one or more processing cores; preferably, the processor 401 may integrate an application processor and a modem processor, wherein the application processor mainly processes the operating system, user interface, and application programs, and the modem processor mainly processes wireless communications. It is understandable that the above-mentioned modem processor may not be integrated into the processor 401.

[0191] The memory 402 can be used to store software programs and modules. The processor 401 executes various functional applications and data processing by running the software programs and modules stored in the memory 402. The memory 402 may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc.; the data storage area may store data created according to the use of the electronic device, etc. In addition, the memory 402 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other volatile solid-state storage devices. Accordingly, the memory 402 may also include a memory controller to provide the processor 401 with access to the memory 402.

[0192] The electronic device also includes a power supply 403 for supplying power to each component. Preferably, the power supply 403 can be logically connected to the processor 401 through a power management system, so as to manage charging, discharging, power consumption and other functions through the power management system. The power supply 403 can also include one or more DC or AC power supplies, recharging systems, power failure detection circuits, power converters or inverters, power status indicators and other arbitrary components.

[0193] The electronic device may further include an input unit 404, which may be used to receive input digital or character information and generate keyboard, mouse, joystick, optical or trackball signal input related to user settings and function control.

[0194] Although not shown, the electronic device may further include a display unit, etc., which will not be described in detail herein. Specifically in this embodiment, the processor 401 in the electronic device will load the executable files corresponding to the processes of one or more application programs into the memory 402 according to the following instructions, and the processor 401 will run the application programs stored in the memory 402, thereby implementing the steps in any of the offline queue resource optimization methods provided in the embodiments of the present application.

[0195] In an embodiment of the present application, if it is detected that the resource usage of the target offline queue reaches a target peak value, the offline tasks in the target offline queue at the target peak value are used as offline tasks to be processed, and execution timeliness information of the offline tasks to be processed is obtained, where the execution timeliness information describes the timeliness that needs to be met for the execution of the offline tasks to be processed. If the execution timeliness information satisfies a first preset condition, the original execution time of the offline tasks to be processed is adjusted to a target execution time, where the target execution time is different from the original execution time, and when the target execution time is reached, the offline tasks to be processed are executed.

[0196] By staggering and shaving the peak load of offline tasks during peak loads, we can avoid peak loads when multiple offline tasks are executed simultaneously, and avoid the problem of insufficient available resources. By adjusting the execution time of each task according to the execution time of the task, we can ensure the effectiveness of task execution, the correlation between upstream and downstream execution of tasks, and the stability of task execution.

[0197] The specific implementation of the above operations can be found in the previous embodiments, which will not be described in detail here.

[0198] A person of ordinary skill in the art will appreciate that all or part of the steps in the various methods of the above embodiments may be accomplished by instructions, or by controlling related hardware through instructions. The instructions may be stored in a computer storage medium and loaded and executed by a processor.

[0199] To this end, the present application provides a storage medium, which includes a computer-readable storage medium, on which a computer program is stored. The computer program can be loaded by a processor to execute the steps in any offline queue resource optimization method provided in the present application.

[0200] The specific implementation of the above operations can be found in the previous embodiments, which will not be described in detail here.

[0201] The computer-readable storage medium may include: a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, etc.

[0202] Since the instructions stored in the computer-readable storage medium can execute the steps in any offline queue resource optimization method provided in the present application, the beneficial effects that can be achieved by any offline queue resource optimization method provided in the present application can be achieved. Please refer to the previous embodiments for details, which will not be repeated here.

[0203] The above is a detailed introduction to an offline queue resource optimization method, device, electronic device and computer-readable storage medium provided by the present application. Specific examples are used in this article to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core idea; at the same time, for those skilled in the art, according to the idea of ​​the present invention, there will be changes in the specific implementation method and application scope. In summary, the content of this specification should not be understood as a limitation on the present invention.

[0204] It should be noted that in the specific implementation of the present application, related data such as the usage of terminal resources (including memory, CPU), execution time information of offline tasks, processing data table information, table structure information, table storage information, blood relationship information, etc., when the above embodiments of the present application are applied to specific products or technologies, user permission or consent is required, and the collection, use and processing of relevant data need to comply with relevant laws, regulations and standards of relevant countries and regions.

Claims

1. A method for optimizing offline queue resources, characterized in that: include: If it is detected that the resource usage of the target offline queue reaches the target peak value, the offline tasks in the target offline queue at the target peak value are taken as the offline tasks to be processed; Acquire execution timeliness information of the offline task to be processed, where the execution timeliness information describes the timeliness requirement that needs to be met for the execution of the offline task to be processed; If the execution timeliness information satisfies a first preset condition, the original execution time of the to-be-processed offline task is adjusted to a target execution time, where the target execution time is different from the original execution time; When the target execution time is reached, the to-be-processed offline task is executed.

2. The method according to claim 1, characterized in that: The obtaining of the execution timeliness information of the offline task to be processed includes: Obtaining the output time of the execution result of the offline task to be processed after execution; Obtaining the usage time of the execution result used by the downstream task of the to-be-processed offline task; Determine execution timeliness information of the to-be-processed offline task according to the production time and the use time.

3. The method according to claim 1, characterized in that The obtaining of the execution timeliness information of the offline task to be processed includes: The execution timeliness information of the offline task to be processed is determined according to the preset execution timeliness tag of the offline task to be processed.

4. The method according to claim 1, characterized in that: The offline task to be processed needs to be executed continuously for at least two days, and the obtaining of the execution timeliness information of the offline task to be processed includes: The tasks of the pending offline tasks on each day are taken as pending offline subtasks; Obtaining execution time information of each of the offline subtasks to be processed; If the execution timeliness information satisfies a first preset condition, adjusting the original execution time of the to-be-processed offline task to the target execution time includes: For each offline subtask to be processed, if the execution timeliness information of the offline subtask to be processed meets the first preset condition, the original execution time of the offline subtask to be processed is adjusted to the target execution time.

5. The method according to claim 1, characterized in that If it is detected that the resource usage of the target offline queue reaches the target peak value, before taking the offline tasks in the target offline queue at the target peak value as the offline tasks to be processed, the method further includes: Acquiring execution status information of the offline task to be optimized in the target offline queue, wherein the execution status information describes the necessity of executing the offline task to be optimized; The execution status information describes that unnecessary offline tasks to be optimized are executed as offline tasks to be taken offline; The offline tasks to be taken offline in the target offline queue are processed offline.

6. The method according to claim 5, characterized in that The obtaining the execution status information of the offline task to be optimized in the target offline queue includes: Acquire the blood relationship of the offline task to be optimized, where the blood relationship describes the upstream and downstream relationship between tasks, and describes the belonging relationship between tasks and business systems; Determine the task existence status of the downstream task corresponding to the offline task to be optimized according to the upstream and downstream relationship; Determining the target business system to which the offline task to be optimized belongs according to the belonging relationship; Obtaining the system online status of the target business system; If the task existence status describes that the downstream task does not exist or the system online status describes that the target business system is offline, execution unnecessary is used as the execution status information of the offline task to be optimized.

7. The method according to claim 5, characterized in that The obtaining the execution status information of the offline task to be optimized in the target offline queue includes: Obtaining a table storage status of an actual processing table corresponding to the offline task to be optimized; If the table storage status describes that the actual processing table is empty or is empty for a period of time, unnecessary execution is used as the execution status information of the offline task to be optimized.

8. The method according to claim 5, characterized in that The obtaining the execution status information of the offline task to be optimized in the target offline queue includes: Determine an actual processing table corresponding to the offline task to be optimized, and determine an actual upstream table corresponding to the actual processing table; If there is a reference processing table having a table structure consistent with the actual processing table, and the reference upstream table corresponding to the reference processing table is consistent with the actual upstream table, execution unnecessary is used as the execution status information of the offline task to be optimized.

9. An offline queue resource optimization device, characterized in that: include: A determination module, configured to, if it is detected that the resource usage of the target offline queue reaches a target peak value, use the offline tasks in the target offline queue at the target peak value as offline tasks to be processed; An acquisition module, used to acquire execution timeliness information of the offline task to be processed, wherein the execution timeliness information describes the timeliness conditions that need to be met when executing the offline task to be processed; An adjustment module, configured to adjust the original execution time of the to-be-processed offline task to a target execution time if the execution timeliness information satisfies a first preset condition, wherein the target execution time is different from the original execution time; The execution module is used to execute the pending offline task when the target execution time is reached.

10. An electronic device, characterized in that: The method comprises a memory, a processor and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps in the offline queue resource optimization method according to any one of claims 1 to 8 when executing the computer program.

11. A storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps in the offline queue resource optimization method according to any one of claims 1 to 8 are implemented.