Data downloading method and device and storage medium

Abstract

According to the data downloading method and device and the storage medium, the preset data volume threshold value is configured for the first time period, so that only asynchronous downloading tasks with small data volumes can be completed, priority processing of the asynchronous downloading tasks with the small data volumes is achieved, and delayed queuing is reduced; and the asynchronous downloading task with the large data volume can be paused and delayed to be processed, and the data acquired by the asynchronous downloading task with the large data volume in the first time period is temporarily stored, so that the time consumption and the performance pressure caused by repeated acquisition are reduced, the downloading efficiency is improved, resources are fully utilized in different time periods, and the downloading efficiency is improved. The memory overflow caused by overlarge concurrent processing data volume is avoided, and the system operation risk is reduced.

Description

Technical Field

[0001] This application relates to the field of communications, and in particular, to a data download method, device, and storage medium. Background Art

[0002] In some business scenarios, customers usually have a need to download data. For example, in e-banking services, customers may need to download transaction details or other business files.

[0003] Before submitting a download request, customers cannot accurately evaluate the amount of data to be downloaded. The system generally provides an asynchronous download function, that is, first register the customer's needs, and then batch process the registered asynchronous download tasks uniformly.

[0004] However, when batch processing asynchronous download tasks uniformly, due to the inability to estimate the amount of data to be downloaded, multiple large files may be generated by parallel processing of multiple asynchronous download tasks, resulting in overly large generated files and bringing operational risks to the system. Summary of the Invention

[0005] Embodiments of this application provide a data download method, device, and storage medium to reduce the operational risk of the system during the process of processing asynchronous download tasks.

[0006] In a first aspect, embodiments of this application provide a data download method, including:

[0007] When the current time is within a first time period, obtain any unprocessed asynchronous download task from the asynchronous download task list; wherein the asynchronous download task is a download task for data within a required time range corresponding to a data request policy, and the first time period is a time period configured with a preset data volume threshold;

[0008] According to the asynchronous download task, sequentially obtain target data within the required time range from the database, and accumulate the data volume of the obtained target data;

[0009] If the obtaining operation is not completed and the data volume reaches the preset data volume threshold, stop continuing to obtain target data, generate a temporary file for the obtained target data, cache it in an additional storage device, and mark the status of the asynchronous download task as a paused state in the asynchronous download task list to indicate deferred processing;

[0010] If the obtaining operation is completed and the data volume does not reach the preset data volume threshold, generate a target file according to the obtained target data and return it to the data requester.

[0011] In a second aspect, embodiments of this application provide a data download device, including:

[0012] A task acquisition module, configured to acquire any unprocessed asynchronous download task from an asynchronous download task list when the current time is within a first time period; wherein the asynchronous download task is a download task for data within a required time range according to a data request policy, and the first time period is a time period configured with a preset data volume threshold.

[0013] A task execution module, configured to sequentially acquire target data within the required time range from a database according to the asynchronous download task, and accumulate the data volume of the acquired target data; if the acquisition operation is not completed and the data volume reaches the preset data volume threshold, then stop continuing to acquire target data, generate a temporary file for the acquired target data, cache it in an additional storage device, and mark the status of the asynchronous download task as a paused state in the asynchronous download task list to indicate deferred processing; if the acquisition operation is completed and the data volume does not reach the preset data volume threshold, then generate a target file according to the acquired target data and return it to the data requester.

[0014] In a third aspect, an embodiment of the present application provides an electronic device, including: a memory, a processor;

[0015] The memory stores computer execution instructions;

[0016] The processor executes the computer execution instructions stored in the memory, so that the processor executes the above first aspect and / or various possible implementation manners of the first aspect.

[0017] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, in which computer execution instructions are stored, and when the computer execution instructions are executed by a processor, they are used to implement the above first aspect and / or various possible implementation manners of the first aspect.

[0018] In a fifth aspect, an embodiment of the present application provides a computer program product, including a computer program, and when the computer program is executed by a processor, it implements the above first aspect and / or various possible implementation manners of the first aspect.

[0019] The data download method, device, and storage medium provided by the embodiments of the present application obtain any unprocessed asynchronous download task from the asynchronous download task list when the current time is within the first time period; where the asynchronous download task is a download task for data within the required time range according to the data request policy, and the first time period is a time period configured with a preset data volume threshold; sequentially obtain the target data within the required time range from the database according to the asynchronous download task, and accumulate the data volume of the obtained target data; if the acquisition operation is not completed and the data volume reaches the preset data volume threshold, stop continuing to obtain the target data, generate a temporary file for the obtained target data, cache it in the additional storage device, and mark the status of the asynchronous download task as paused in the asynchronous download task list to indicate deferred processing; if the acquisition operation is completed and the data volume does not reach the preset data volume threshold, generate a target file according to the obtained target data and return it to the data requester. The embodiments of the present application configure different time periods and configure a preset data volume threshold for the first time period, so that only asynchronous download tasks with a small data volume can be completed, realizing the priority processing of asynchronous download tasks with a small data volume, reducing delay queuing; while asynchronous download tasks with a large data volume can be paused for processing, and the data already obtained by the asynchronous download tasks with a large data volume within the first time period is temporarily stored, without causing resource waste, reducing the time consumption and performance pressure caused by repeated acquisition, and improving the download efficiency. By making full use of resources in different time periods, it is avoided that the data volume of concurrent processing is too large to cause memory overflow, and the system operation risk is reduced. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] The drawings herein are incorporated into the specification and form a part of the specification, showing embodiments consistent with the present application, and are used together with the specification to explain the principles of the present application.

[0021] Figure 1 It is a schematic diagram of the scenario of the data download method provided by an embodiment of the present application;

[0022] Figure 2 It is a flowchart of the data download method provided by an embodiment of the present application;

[0023] Figure 3 It is a flowchart of the data download method provided by an embodiment of the present application;

[0024] Figure 4 It is a flowchart of the data download method provided by an embodiment of the present application;

[0025] Figure 5 It is a schematic diagram of the structure of the data download device provided by an embodiment of the present application;

[0026] Figure 6 It is a schematic diagram of the structure of the electronic device provided by an embodiment of the present application.

[0027] Through the above-mentioned drawings, specific embodiments of the present application have been shown, and will be described in more detail hereinafter. These drawings and the written description are not intended to limit the scope of the concept of the present application in any way, but to illustrate the concept of the present application to those skilled in the art by reference to specific embodiments. Detailed Description of the Embodiments

[0028] Here, the exemplary embodiments will be described in detail, and the examples are shown in the drawings. When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present application. On the contrary, they are merely examples of devices and methods consistent with some aspects of the present application as detailed in the appended claims.

[0029] Figure 1 The following is a schematic diagram of the scenario of the data download method provided for the present application. As Figure 1 shown, the specific application of the present application is in a business scenario where the customer has a need to download data. For example, in e-banking, the customer may have a need to download transaction information or other business documents. The customer, that is, the data requester, can send a download request to the server. After the server obtains the data from the database, it generates a target file and sends it to the data requester.

[0030] Combined with the above scenario, in the prior art, before submitting a download request, the customer cannot accurately evaluate the amount of data to be downloaded. For example, in e-banking, the customer needs to download transaction information for a specific period, but the customer cannot evaluate the amount of data of the transaction information for a specific period. If the time span is relatively large or the transactions are frequent, the amount of data to be downloaded is huge. Generally, the system will provide an asynchronous download function, that is, first register the customer's needs, and then batch process the registered asynchronous download tasks uniformly.

[0031] However, when batch processing asynchronous download tasks uniformly, since the amount of data to be downloaded cannot be estimated, multiple large files may be generated due to parallel processing of multiple asynchronous download tasks, resulting in the generated files being too large and bringing operation risks to the system, such as memory overflow, resulting in download task failures, etc.

[0032] To solve the above technical problems, the data download method provided by the embodiments of the present application obtains any unprocessed asynchronous download task from the asynchronous download task list when the current time is in the first time period; wherein the asynchronous download task is a download task for the data within the required time range by the data request policy, and the first time period is a time period configured with a preset data volume threshold. According to the asynchronous download task, the target data within the required time range is sequentially obtained from the database, and the data volume of the obtained target data is accumulated. If the acquisition operation is not completed and the data volume reaches the preset data volume threshold, the acquisition of the target data is stopped, a temporary file is generated for the obtained target data, cached in the additional storage device, and the status of the asynchronous download task in the asynchronous download task list is marked as the paused state to indicate deferred processing. If the acquisition operation is completed and the data volume does not reach the preset data volume threshold, a target file is generated according to the obtained target data and returned to the data request party. By configuring different time periods and configuring a preset data volume threshold for the first time period, the embodiments of the present application enable only the asynchronous download tasks with a small data volume to be completed, realizing the priority processing of the asynchronous download tasks with a small data volume, reducing the delay queue; while the asynchronous download tasks with a large data volume can be paused for processing, and the data obtained by the asynchronous download tasks with a large data volume within the first time period is temporarily stored, and there is no need to repeatedly obtain it in the second time period, which will not cause resource waste, reducing the time consumption and performance pressure caused by repeated acquisition, and improving the download efficiency. By making full use of resources in different time periods, it is avoided that the data volume of concurrent processing is too large to cause memory overflow, reducing the system operation risk.

[0033] The following uses specific embodiments to elaborate in detail on the technical solutions of the present application and how the technical solutions of the present application solve the above technical problems. These specific embodiments below can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments. The embodiments of the present application will be described below with reference to the accompanying drawings.

[0034] Figure 2 It is a schematic flowchart of the data download method provided by an embodiment of the present application, as Figure 2 shown, the method includes:

[0035] S201. When the current time is in the first time period, obtain any unprocessed asynchronous download task from the asynchronous download task list; wherein the asynchronous download task is a download task for the data within the required time range by the data request policy, and the first time period is a time period configured with a preset data volume threshold.

[0036] In this embodiment, the time can be divided into a first time period and a second time period. A preset data volume threshold is configured for the first time period. Based on the preset data volume threshold, an asynchronous download task with a small data volume (not exceeding the preset data volume threshold) can be completed. For an asynchronous download task with a large data volume, it is paused and postponed when the acquired data reaches the preset data volume threshold, that is, it is executed within the second time period. And no preset data volume threshold is configured for the second time period, and it can be unrestricted by the preset data volume threshold. The preset data volume threshold can be the number of data items or the size of the data.

[0037] Optionally, considering that a large number of active users submit asynchronous download tasks during the daytime time period, while relatively few asynchronous download tasks are submitted at night. Therefore, the first time period can be the daytime time period, such as 8:00 - 18:00, and the second time period can be the nighttime time period, such as 18:00 - 8:00. Thus, small-data-volume asynchronous download tasks can be preferentially processed during the daytime time period to improve the download efficiency and reduce delayed queuing, while large-data-volume asynchronous download tasks can be processed during the nighttime time period to make full use of resources. Or, the first time period can also be the time period when the number of unprocessed asynchronous download tasks in the asynchronous download task list exceeds a preset number, and the second time period is the time period when the number of unprocessed asynchronous download tasks in the asynchronous download task list does not exceed the preset number. That is, it is not limited to the daytime and nighttime time periods, and the first and second time periods can be divided with a finer granularity.

[0038] In specific implementation, it can be determined which time period the current time is in. If the current time reaches the first time period, the configuration of the preset data volume threshold is enabled. If the current time reaches the second time period, the configuration of the preset data volume threshold is disabled. Through the above process, the automatic opening and closing of the preset data volume threshold are realized, so as to control asynchronous download tasks with different data volumes.

[0039] It should be noted that there can be more than one first time period and second time period in this embodiment. There can be multiple first time periods, and there can also be multiple second time periods. In addition, the preset data volume thresholds configured for multiple first time periods can be the same or different, and the corresponding preset data volume thresholds for different first time periods can be configured according to the actual situation.

[0040] When the data requester needs to download data within the required time range, such as the transaction details of the data requester in banking business, including but not limited to customer transfer records such as payroll, collection, and transfer, an asynchronous download task can be submitted first. The asynchronous download task can be added to the asynchronous download task list for queuing first, and then the asynchronous download task is taken from the asynchronous download task list for execution. The asynchronous download task is a download task that does not process the download request in real time but registers it for deferred processing first.

[0041] When the current time is within the first time period, obtain any unprocessed asynchronous download task from the asynchronous download task list and execute it.

[0042] S202. Obtain target data within the required time range from the database in sequence according to the asynchronous download task, and accumulate the data volume of the obtained target data.

[0043] In this embodiment, when the current time is within the first time period, any unprocessed asynchronous download task can be directly executed. According to the asynchronous download task, obtain target data within the required time range from the database in sequence. Since a preset data volume threshold is configured for the first time period, it is necessary to accumulate the data volume of the obtained target data during the process of obtaining target data within the required time range.

[0044] Further, S203 or S204 can be executed according to whether the data volume reaches the preset data volume threshold.

[0045] S203. If the acquisition operation is not completed and the data volume reaches the preset data volume threshold, stop continuously obtaining target data, generate a temporary file for the obtained target data, cache it to an additional storage device, and mark the status of the asynchronous download task as paused in the asynchronous download task list to indicate deferred processing.

[0046] In this embodiment, during the process of an asynchronous download task obtaining target data, if the acquisition operation is not completed, that is, not all target data within the required time range has been obtained, and the data volume reaches the preset data volume threshold, it indicates that this asynchronous download task is an asynchronous download task with a large data volume. It can be paused first and then deferred for processing, that is, it can be continued to be executed in a non-first time period (such as the second time period).

[0047] To avoid resource waste and reduce the time consumption and performance pressure caused by repeated acquisition, the obtained target data can be generated into a temporary file and cached to an additional storage device. In the second time period, the acquisition of target data can continue without having to repeatedly obtain the obtained target data. Optionally, the additional storage device can be a NAS (Network Attached Storage) device. A NAS device is a dedicated data storage server that completely separates the storage device from the server and centrally manages data, with the characteristics of high performance and low cost. Of course, it can also be any other feasible additional storage device.

[0048] In addition, to distinguish these paused asynchronous download tasks from unprocessed asynchronous download tasks, the status of these paused asynchronous download tasks can be marked as the paused status in the asynchronous download task list to indicate that these paused asynchronous download tasks can be processed later, that is, to indicate that these paused asynchronous download tasks can continue to be executed within the second time period. Then, after reaching the second time period, these paused asynchronous download tasks can be retrieved from the asynchronous download task list according to the paused status and continue to be executed.

[0049] S204. If the acquisition operation is completed and the data volume does not reach the preset data volume threshold, generate a target file based on the acquired target data and return it to the data requester.

[0050] In this embodiment, during the process of an asynchronous download task acquiring target data, if the acquisition operation is completed, that is, all the target data within the required time range is acquired, and the data volume has not reached the preset data volume threshold, it indicates that this asynchronous download task is an asynchronous download task with a small data volume. The acquired target data can be directly used to generate a target file and returned to the data requester, that is, this asynchronous download task is completed. The target file is a file in a format such as a table or PDF generated based on the target data.

[0051] The data download method provided in this embodiment includes: when the current time is within the first time period, obtaining any unprocessed asynchronous download task from the asynchronous download task list, where the asynchronous download task is a download task of the data requester for the data within the required time range, and the first time period is a time period configured with a preset data volume threshold; sequentially obtaining the target data within the required time range from the database according to the asynchronous download task and accumulating the data volume of the acquired target data; if the acquisition operation is not completed and the data volume reaches the preset data volume threshold, stop continuing to obtain the target data, generate a temporary file for the acquired target data, cache it in the additional storage device, and mark the status of the asynchronous download task as the paused status in the asynchronous download task list to indicate delayed processing; if the acquisition operation is completed and the data volume does not reach the preset data volume threshold, generate a target file based on the acquired target data and return it to the data requester. By configuring different time periods and configuring a preset data volume threshold for the first time period in this application embodiment, only asynchronous download tasks with a small data volume can be completed, realizing the priority processing of asynchronous download tasks with a small data volume and reducing delay queuing. For asynchronous download tasks with a large data volume, they can be paused for processing, and the data already acquired by the asynchronous download tasks with a large data volume within the first time period is temporarily stored, so there is no need to repeatedly acquire it within the second time period, which will not cause resource waste, reduce the time consumption and performance pressure caused by repeated acquisition, and improve the download efficiency. By making full use of resources in different time periods, it is avoided that the data volume of concurrent processing is too large to cause memory overflow and reduce the system operation risk.

[0052] Based on the above embodiments, when the current time is in the second time period, as Figure 3 shown, the following process can be specifically executed:

[0053] S301. When the current time is in the second time period, obtain any asynchronous download task in a paused state from the asynchronous download task list; wherein the second time period is a time period without a preset data volume threshold configured;

[0054] S302. Obtain the temporary file from the additional storage device, and continue to sequentially obtain target data within the required time range from the database according to the asynchronous download task;

[0055] S303. After the obtaining operation is completed, generate a target file according to the temporary file and the obtained target data, and return it to the data requester.

[0056] In this embodiment, when the current time is in the second time period, any asynchronous download task in a paused state can be first obtained from the asynchronous download task list, and the temporary file can be obtained from the additional storage device. On the basis of the temporary file, the target data within the required time range is sequentially obtained from the database according to the asynchronous download task, and the target data included in the temporary file does not need to be obtained repeatedly. Since the second time period is a time period without a preset data volume threshold configured, all the target data within the required time range can be directly obtained. Then, after the obtaining operation is completed, a target file is generated according to the temporary file and the obtained target data, and returned to the data requester, so as to ensure that the asynchronous download task paused in the first time period can be completed, and there is no need to repeatedly obtain the target data already obtained in the first time period throughout the second time period, which will not cause resource waste, reduce the time consumption and performance pressure caused by repeated obtaining, improve the download efficiency, and since there are relatively few asynchronous download tasks in the second time period, the resources can be fully utilized, avoiding the idleness of resources in the second time period, and also avoiding the memory overflow caused by the excessive data volume to be processed concurrently when the asynchronous download tasks are concentrated in the first time period.

[0057] Optionally, after stopping the continuous acquisition of target data in the first time period, the download progress information can be determined, a download progress file can be generated according to the download progress information, the download progress file can be stored in the attached storage device, and associated with the temporary file; furthermore, in the second time period, the temporary file and the download progress file can be obtained from the attached storage device, and according to the download progress information in the download progress file, continue to obtain the target data within the required time range from the database in sequence. By recording the download progress information, it is possible to continue to obtain the target data within the required time range based on the download progress information in the second time period, ensuring that the target data included in the temporary file does not need to be obtained repeatedly.

[0058] Based on any of the above embodiments, as Figure 4 shown, the method may further include:

[0059] S401. When the current time is in the second time period, if there is no asynchronous download task in the paused state in the asynchronous download task list, or all asynchronous download tasks in the paused state have been completed, obtain any unprocessed asynchronous download task from the asynchronous download task list;

[0060] S402. Obtain the target data within the required time range from the database in sequence according to the asynchronous download task;

[0061] S403. After the acquisition operation is completed, generate a target file according to the obtained target data and return it to the data requester.

[0062] In this embodiment, in the second time period, since the asynchronous download tasks in the paused state in the asynchronous download task list are the asynchronous download tasks that have been started and processed in the first time period, and the unprocessed asynchronous download tasks are the asynchronous download tasks that have not been started and processed in the first time period, or the asynchronous download tasks submitted in the second time period, the priority of the asynchronous download tasks in the paused state is relatively high, and the asynchronous download tasks in the paused state can be processed first. When there is no asynchronous download task in the paused state in the asynchronous download task list, or all asynchronous download tasks in the paused state have been completed, then execute the unprocessed asynchronous download tasks in the asynchronous download task list.

[0063] When processing the unprocessed asynchronous download tasks in the second time period, since there is no preset data volume threshold configured in the second time period, all the target data within the required time range can be directly obtained. After the acquisition operation is completed, a target file is generated according to the obtained target data and returned to the data requester, ensuring the successful execution of the unprocessed asynchronous download tasks. Of course, the unprocessed asynchronous download tasks in the second time period can include asynchronous download tasks with a small data volume or asynchronous download tasks with a large data volume, and are not restricted by the preset data volume threshold.

[0064] Based on any of the above embodiments, considering factors such as limited resources such as transmission bandwidth and memory, the amount of asynchronous download task data processed within the second time period is not unlimited, so as not to affect the execution of other tasks. Therefore, an upper limit on the downloaded data volume can also be configured. When the current time is within the second time period and any asynchronous download task is executed, the amount of the target data obtained cumulatively can also be determined. If the data volume exceeds the upper limit of the downloaded data volume, an error message is sent to the data requester to prompt the data requester to shorten the required time range of the asynchronous download task, thereby reducing the data volume of the asynchronous download task.

[0065] Optionally, for an asynchronous download task with an error, the obtained target data can also be generated into a temporary file and cached in an additional storage device. After receiving an instruction from the data requester to shorten the required time range of the asynchronous download task, if there is an intersection between the adjusted time range and the original time range, the target data within the adjusted required time range can be retrieved from the temporary file. For the target data that does not exist in the temporary file, it is continuously retrieved from the database, avoiding resource waste and reducing the time consumption and performance pressure caused by repeated retrieval.

[0066] Figure 5 The following is a schematic structural diagram of the data download device provided by the present application, as Figure 5 shown, the data download device 500 provided in this embodiment includes: a task acquisition module 501 and a task execution module 502.

[0067] The task acquisition module 501 is configured to, when the current time is within the first time period, acquire any unprocessed asynchronous download task from the asynchronous download task list; wherein the asynchronous download task is a download task of the data requester for the data within the required time range, and the first time period is a time period configured with a preset data volume threshold;

[0068] The task execution module 502 is configured to sequentially acquire the target data within the required time range from the database according to the asynchronous download task, and accumulate the amount of the target data obtained; if the acquisition operation is not completed and the data volume reaches the preset data volume threshold, stop continuously acquiring the target data, generate a temporary file for the obtained target data, cache it in an additional storage device, and mark the status of the asynchronous download task as a paused state in the asynchronous download task list to indicate deferred processing; if the acquisition operation is completed and the data volume does not reach the preset data volume threshold, generate a target file according to the obtained target data and return it to the data requester.

[0069] In one or more embodiments of the present application, the task acquisition module 501 is further configured to, when the current time is within the second time period, obtain any asynchronous download task in a paused state from the asynchronous download task list; wherein the second time period is a time period without a preset data volume threshold configured.

[0070] The task execution module 502 is further configured to obtain the temporary file from the additional storage device, and continue to sequentially obtain target data within the required time range from the database according to the asynchronous download task; after the acquisition operation is completed, generate a target file according to the temporary file and the obtained target data, and return it to the data requester.

[0071] In one or more embodiments of the present application, if the acquisition operation of the task execution module 502 is not completed and the data volume reaches the preset data volume threshold, after stopping the continuous acquisition of target data, it is further configured to:

[0072] Determine the download progress information, generate a download progress file according to the download progress information, store the download progress file in the additional storage device, and associate it with the temporary file.

[0073] Correspondingly, when the task execution module 502 obtains the temporary file from the additional storage device and continues to sequentially obtain target data within the required time range from the database according to the asynchronous download task, it is configured to:

[0074] Obtain the temporary file and the download progress file from the additional storage device, and continue to sequentially obtain target data within the required time range from the database according to the download progress information in the download progress file.

[0075] In one or more embodiments of the present application, the task acquisition module 501 is further configured to, when the current time is within the second time period, if there is no asynchronous download task in a paused state in the asynchronous download task list, or all asynchronous download tasks in a paused state have been completed, obtain any unprocessed asynchronous download task from the asynchronous download task list.

[0076] The task execution module 502 is further configured to sequentially obtain target data within the required time range from the database according to the asynchronous download task; after the acquisition operation is completed, generate a target file according to the obtained target data, and return it to the data requester.

[0077] In one or more embodiments of the present application, the task execution module 502 is further configured to:

[0078] When the current time is within the second time period, when executing any asynchronous download task, determine the data volume of the target data obtained cumulatively.

[0079] If the amount of data exceeds the upper limit of the downloaded data amount, an error message is sent to the data requester to prompt the data requester to shorten the required time range of the asynchronous download task.

[0080] In one or more embodiments of the present application, the task execution module 502 is further configured to:

[0081] Judge the time period in which the current time is located;

[0082] If the current time reaches the first time period, the configuration of the preset data volume threshold is enabled;

[0083] If the current time reaches the second time period, the configuration of the preset data volume threshold is disabled.

[0084] In one or more embodiments of the present application, the first time period is a daytime time period, and the second time period is a nighttime time period; or

[0085] The first time period is a time period in which the number of unprocessed asynchronous download tasks in the asynchronous download task list exceeds a preset number, and the second time period is a time period in which the number of unprocessed asynchronous download tasks in the asynchronous download task list does not exceed the preset number.

[0086] The data download device provided in this embodiment can execute the method provided in the above method embodiment, and its implementation principle and technical effect are similar, and will not be elaborated here in this embodiment.

[0087] Figure 6 It is a schematic structural diagram of an electronic device provided in the present application. As Figure 6 shown, the electronic device 600 provided in this embodiment includes: at least one processor 601 and a memory 602. Optionally, the electronic device 600 further includes a communication component 603. Among them, the processor 601, the memory 602, and the communication component 603 are connected through a bus 604.

[0088] In a specific implementation process, at least one processor 601 executes computer execution instructions stored in the memory 602, so that at least one processor 601 executes the above method.

[0089] The specific implementation process of the processor 601 can refer to the above method embodiment, and its implementation principle and technical effect are similar, and will not be elaborated here in this embodiment.

[0090] In the above embodiments, it should be understood that the processor may be a central processing unit (CPU for short), or other general-purpose processors, digital signal processors (DSP for short), application specific integrated circuits (ASIC for short), etc. The general-purpose processor may be a microprocessor or any conventional processor, etc. The steps of the method disclosed in combination with the invention can be directly implemented by the execution of the hardware processor, or implemented by the combination of the hardware and software modules in the processor.

[0091] The memory may include a high-speed memory (Random Access Memory, RAM), and may also include non-volatile memory (Non-volatile Memory, NVM), such as at least one disk memory.

[0092] The bus may be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, an Extended Industry Standard Architecture (EISA) bus, etc. The bus can be divided into an address bus, a data bus, a control bus, etc. For the convenience of representation, the buses in the drawings of this application are not limited to only one bus or one type of bus.

[0093] This application also provides a computer program product, including a computer program, which implements the above method when executed by a processor.

[0094] This application also provides a computer-readable storage medium, in which computer-executable instructions are stored, and when the processor executes the computer-executable instructions, the above method is implemented.

[0095] The above-readable storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, a magnetic disk or an optical disk. The readable storage medium can be any available medium accessible by a general-purpose or special-purpose computer.

[0096] An exemplary readable storage medium is coupled to a processor, enabling the processor to read information from and write information to the readable storage medium. Of course, the readable storage medium can also be a component of the processor. The processor and the readable storage medium can be located in an Application Specific Integrated Circuit (ASIC). Of course, the processor and the readable storage medium can also exist as discrete components in a device.

[0097] The division of units is only a logical function division. In actual implementation, there may be other division methods. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the couplings or direct couplings or communication connections shown or discussed among each other can be indirect couplings or communication connections through some interfaces, devices or units, and can be in electrical, mechanical or other forms.

[0098] The units described as separate components may or may not be physically separated. The components shown as units may or may not be physical units, that is, they can be located in one place or distributed to multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0099] In addition, in each embodiment of the present invention, the functional units can be integrated in a processing unit, or each unit can exist physically alone, or two or more units can be integrated in one unit.

[0100] If the function is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art or part of this technical solution can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods in each embodiment of the present invention. The aforementioned storage medium includes: various media such as USB flash drives, mobile hard disks, read-only memories (ROMs), random access memories (RAMs), magnetic disks or optical discs that can store program codes.

[0101] Those of ordinary skill in the art can understand that all or part of the steps to implement the above method embodiments can be completed by hardware related to program instructions. The foregoing program can be stored in a computer-readable storage medium. When the program is executed, it performs the steps including those of the above method embodiments; and the foregoing storage medium includes: various media such as ROM, RAM, magnetic disk, or optical disc that can store program code.

[0102] Finally, it should be noted that: After considering the specification and practicing the invention disclosed herein, those skilled in the art will readily think of other implementation schemes of the present invention. The present invention is intended to cover any variations, uses, or adaptations of the present invention, which follow the general principles of the present invention and include the common general knowledge or conventional technical means in the technical field not disclosed by the present invention. It is not limited to the exact structure described above and shown in the drawings, and various modifications and changes can be made without departing from its scope. The scope of the present invention is only limited by the appended claims.

Claims

1. A data download method, characterized in that, Including: When the current time is in the first time period, obtain any unprocessed asynchronous download task from the asynchronous download task list; Wherein the asynchronous download task is a download task for data within the required time range according to the data request policy, and the first time period is a time period configured with a preset data volume threshold; Obtain the target data within the required time range from the database in sequence according to the asynchronous download task, and accumulate the data volume of the obtained target data; If the acquisition operation is not completed and the data volume reaches the preset data volume threshold, stop continuing to obtain the target data, generate a temporary file for the obtained target data, cache it in the additional storage device, and mark the status of the asynchronous download task as paused in the asynchronous download task list for indicating deferred processing; If the acquisition operation is completed and the data volume does not reach the preset data volume threshold, generate a target file according to the obtained target data and return it to the data requester.

2. The method according to claim 1, characterized in that, The method further includes: When the current time is in the second time period, obtain any asynchronous download task with a paused status from the asynchronous download task list; wherein the second time period is a time period not configured with a preset data volume threshold; Obtain the temporary file from the additional storage device, and continue to obtain the target data within the required time range from the database in sequence according to the asynchronous download task; After the acquisition operation is completed, generate a target file according to the temporary file and the obtained target data and return it to the data requester.

3. The method according to claim 2, wherein After stopping continuing to obtain the target data when the acquisition operation is not completed and the data volume reaches the preset data volume threshold, it further includes: Determine the download progress information, generate a download progress file according to the download progress information, store the download progress file in the additional storage device, and associate it with the temporary file; Correspondingly, obtaining the temporary file from the additional storage device and continuing to obtain the target data within the required time range from the database in sequence according to the asynchronous download task includes: Obtain the temporary file and the download progress file from the additional storage device, and continue to obtain the target data within the required time range from the database in sequence according to the download progress information in the download progress file.

4. The method according to claim 2, wherein The method further includes: When the current time is in the second time period, if there is no asynchronous download task with a paused status in the asynchronous download task list, or all asynchronous download tasks with a paused status have been completed, obtain any unprocessed asynchronous download task from the asynchronous download task list; Obtain the target data within the required time range from the database in sequence according to the asynchronous download task; After the acquisition operation is completed, generate a target file according to the obtained target data and return it to the data requester.

5. The method according to any one of claims 2-4, characterized in that, The method further includes: When the current time is in the second time period, when executing any asynchronous download task, determine the data volume of the accumulated obtained target data; If the data volume exceeds the download data volume upper limit, send an error message to the data requester to prompt the data requester to shorten the required time range of the asynchronous download task.

6. The method according to any one of claims 1-4, characterized in that, The method further includes: Determine the time period in which the current time is located; If the current time reaches the first time period, configure the preset data volume threshold; If the current time reaches the second time period, close the configuration of the preset data volume threshold.

7. The method according to any one of claims 1-4, characterized in that The first time period is the daytime time period, and the second time period is the nighttime time period; or The first time period is the time period when the number of unprocessed asynchronous download tasks in the asynchronous download task list exceeds a preset number, and the second time period is the time period when the number of unprocessed asynchronous download tasks in the asynchronous download task list does not exceed the preset number.

8. A data download device, characterized in that, It includes: A task acquisition module, which is used to acquire any unprocessed asynchronous download task from the asynchronous download task list when the current time is in the first time period; Wherein the asynchronous download task is a download task for data within the required time range by the data request policy, and the first time period is the time period configured with a preset data volume threshold; A task execution module, which is used to sequentially acquire target data within the required time range from the database according to the asynchronous download task, and accumulate the data volume of the acquired target data; if the acquisition operation is not completed and the data volume reaches the preset data volume threshold, stop continuing to acquire target data, generate a temporary file for the acquired target data, cache it in an additional storage device, and mark the status of the asynchronous download task as paused in the asynchronous download task list to indicate deferred processing; If the acquisition operation is completed and the data volume does not reach the preset data volume threshold, generate a target file according to the acquired target data and return it to the data requester.

9. An electronic device, characterized in that, It includes: A memory, a processor; The memory stores computer execution instructions; The processor executes the computer execution instructions stored in the memory, so that the processor executes the method according to any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, Computer execution instructions are stored in the computer-readable storage medium, and when the computer execution instructions are executed by the processor, they are used to implement the method according to any one of claims 1-7.

Images