A file downloading method, device and computer equipment

By dividing large files into multiple slices and reasonably allocating download tasks according to the IO load occupancy of storage space, the problems of extended download time and reduced device reliability are solved, and faster and reliable file transfer is achieved.

CN112199052BActive Publication Date: 2024-11-01TSD ELECTRONICS TECH
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202011218108.X
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2020-11-04
Publication Date
2024-11-01
Estimated Expiration
2040-11-04

AI Technical Summary

Technical Problem

The prior art can easily extend the download time due to network fluctuations or other users' bandwidth occupation during the downloading of large files, resulting in a longer download time, resulting in a decrease in device reliability.

Method used

By obtaining the size of the target file, the number of cache storage space is determined, and the file is divided into multiple slice files according to the IO load occupancy of each cache storage space, the download task is reasonably allocated, and the number of slice files is determined based on the IO load occupancy and the number of slice files. The download volume of each storage space can realize parallel download.

Benefits of technology

It improves the transmission speed and reliability of large file data downloads, ensuring the stable operation of the equipment during file transfer and update.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN112199052B_ABST
    Figure CN112199052B_ABST
Patent Text Reader

Abstract

The present invention discloses a file downloading method, device and computer equipment, which include: obtaining a target file, determining the number of cache storage spaces according to the size of the target file; obtaining the IO load occupation of multiple cache storage spaces respectively, and determining the target cache storage space that meets the number accordingly; copying and storing the target files in the target cache storage space respectively; segmenting the target file to generate multiple target slice files; then determining the target download amount of each target cache storage space; and downloading the corresponding target slice file in each target cache storage space according to the target download amount. Storing the target file in multiple target storage spaces solves the problem that the download time of large files is prolonged and the reliability of the device is reduced. According to the resource occupation of the current target cache storage space, the file download task can be reasonably allocated to improve the reliability of the device operation.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the technical field of data transmission processing, and in particular to a file downloading method, device and computer equipment. Background Art

[0002] With the development of the Internet, the program files used on various Internet devices are becoming larger and larger. When downloading large files, the existing download method is continuous downloading. When encountering network fluctuations (for example, network disconnection) or other users suddenly occupying the network bandwidth, the network speed of the device will drop sharply, which can easily lead to the failure of the large file download task, thereby extending the overall download time of the file, making the device's response time longer during file transmission and update, etc., which may reduce the reliability of the device operation. Summary of the Invention

[0003] In view of this, embodiments of the present invention provide a file downloading method, apparatus, and computer device to solve the technical problems in the prior art of prolonged response time and reduced reliability of device operation during file transmission and updating.

[0004] According to a first aspect, an embodiment of the present invention provides a file downloading method, comprising: obtaining a target file, and determining the number of cache storage spaces based on the size of the target file; respectively obtaining the IO load occupancy of multiple cache storage spaces, and determining target cache storage spaces that meet the said number based on the IO load occupancy of each cache storage space; respectively copying and storing the target files in the target cache storage spaces; splitting the target file to generate multiple target slice files; determining the target download volume of each target cache storage space based on the IO load occupancy of the target cache storage space, the said number, and the number of the target slice files; and downloading the corresponding target slice files in each target cache storage space based on the target download volume.

[0005] In combination with the first aspect, in the first embodiment of the first aspect, the step of obtaining the target file includes: when the number of download requests corresponding to the download file meets the target condition, determining that the download file is the target file; the target condition is that the ratio of the number of download requests in the current preset period to the number of download requests in the previous preset period adjacent to the current preset period is greater than or equal to a preset threshold.

[0006] In combination with the first aspect, in a second implementation of the first aspect, the method further includes: determining a target download amount for each target cache storage space by the following formula:

[0007]

[0008]

[0009] Among them, Pi represents the IO load occupancy of the i-th target cache storage space, C represents the number of target slice files, n represents the preset storage quantity of the cache storage space, Ti represents the target download volume of the i-th target cache storage space, i is a positive integer, and n is a positive integer.

[0010] In combination with the first aspect, in the third embodiment of the first aspect, the method also includes: when the downloading of the corresponding target slice file in each target cache storage space is paused, storing the current download progress and download link; when the corresponding target slice file is re-downloaded in each target cache storage space, determining the current download position according to the current download progress and download link; and re-downloading the corresponding target slice file according to the current download position.

[0011] In combination with the first aspect, in a fourth implementation of the first aspect, the method further includes: obtaining a download address of each target cache storage space, and updating a preset address database list according to the download address of each target cache storage space.

[0012] In combination with the first aspect, in the fifth implementation of the first aspect, the method also includes: copying and storing the target file in a main storage space, and when the target cache storage space fails, downloading the corresponding target slice file in the main storage space according to the target download amount corresponding to the failed target cache storage space.

[0013] In combination with the first aspect, in the sixth embodiment of the first aspect, the method also includes: obtaining the file name of the target file; obtaining the splitting timestamp for splitting the target file according to preset conditions; and generating a target slice folder name based on the file name, the splitting timestamp and a random number.

[0014] According to the second aspect, an embodiment of the present invention provides a file downloading device, including: a quantity determination module, used to obtain a target file, and determine the quantity of cache storage space according to the size of the target file; a target cache storage space determination module, used to respectively obtain the IO load occupancy of multiple cache storage spaces, and determine the target cache storage space that meets the quantity according to the IO load occupancy of each cache storage space; a copy module, used to copy and store the target files in the target cache storage space respectively; a splitting module, used to split the target file to generate multiple target slice files; a target download quantity determination module, used to determine the target download quantity of each target cache storage space according to the IO load occupancy of the target cache storage space, the quantity and the number of target slice files; a download module, used to download the corresponding target slice file in each target cache storage space according to the target download quantity.

[0015] According to a third aspect, an embodiment of the present invention provides a computer device comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the one processor, and the instructions are executed by the at least one processor to enable the at least one processor to execute the method for downloading a file described in the first aspect or any one embodiment of the first aspect.

[0016] According to a fourth aspect, an embodiment of the present invention provides a computer-readable storage medium, which stores computer instructions, and the computer instructions are used to enable the computer to execute the method for downloading a file described in the second aspect or any one of the embodiments of the second aspect.

[0017] The technical solution of the present invention has the following advantages:

[0018] An embodiment of the present invention provides a file downloading method, apparatus, and computer device, wherein the file downloading method includes: obtaining a target file, determining the number of cache storage spaces based on the size of the target file; obtaining the IO load occupancy of multiple cache storage spaces respectively, and determining a target cache storage space that meets the number based on the IO load occupancy of each cache storage space; copying and storing the target file in the target cache storage space respectively; segmenting the target file to generate multiple target slice files; determining the target download volume of each target cache storage space based on the IO load occupancy and number of the target cache storage space and the number of target slice files; and downloading the corresponding target slice file in each target cache storage space based on the target download volume. Storing the target file in multiple target storage spaces solves the problem of extended large file download time and reduced device reliability in the related art. File download tasks can be reasonably allocated based on the resource occupancy of the current target cache storage space, thereby improving the reliability of device operation and improving the transmission speed and reliability of large file data downloads. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] In order to more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the specific embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0020] Figure 1 This is a flowchart of a specific example of a file downloading method according to an embodiment of the present invention;

[0021] Figure 2is a flowchart of another specific example of the file downloading method according to an embodiment of the present invention;

[0022] Figure 3 is a principle block diagram of a specific example of a device for downloading files in an embodiment of the present invention;

[0023] Figure 4 FIG. 4 is a diagram showing a specific example of a computer device in an embodiment of the present invention. DETAILED DESCRIPTION

[0024] The technical solution of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the embodiments described are only some embodiments of the present invention, not all embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments of the present invention without making any creative efforts shall fall within the scope of protection of the present invention.

[0025] In the description of the present invention, it should be noted that the terms "center," "upper," "lower," "left," "right," "vertical," "horizontal," "inner," and "outer," etc., indicating orientations or positional relationships, are based on the orientations or positional relationships shown in the accompanying drawings and are intended solely to facilitate and simplify the description of the present invention. They are not intended to indicate or imply that the devices or components referred to must have, be constructed, or operate in a specific orientation, and therefore should not be construed as limitations on the present invention. Furthermore, the terms "first," "second," and "third" are used for descriptive purposes only and should not be construed as indicating or implying relative importance.

[0026] In the description of the present invention, it should be noted that, unless otherwise expressly specified or limited, the terms "installed," "connected," and "connected" should be understood in a broad sense. For example, they can refer to fixed connections, detachable connections, or integral connections; mechanical connections or electrical connections; direct connections or indirect connections through an intermediate medium; internal connections between two components; wireless connections or wired connections. Those skilled in the art will understand the specific meanings of the above terms in the present invention based on specific circumstances.

[0027] In addition, the technical features involved in the different embodiments of the present invention described below can be combined with each other as long as they do not conflict with each other.

[0028] When other network users suddenly occupy the network bandwidth, the transmission and download processes of large files will inevitably be affected, resulting in prolonged file download time. To solve the above problem, the embodiments of the present invention provide a file downloading method, apparatus and computer device.

[0029] The embodiment of the present invention provides a method for downloading files. Figure 1 As shown, the file download method includes:

[0030] Step S11: Obtain the target file and determine the amount of cache storage space based on the size of the target file. In this embodiment, the target file can be any file required by the smart device during operation, such as the installation file of the new version of the smart device system and the installation file corresponding to the new version of the software. The cache storage space can be a storage medium for storing various files, such as a cache solid-state drive. Specifically, obtain the target file that is currently at the peak of downloading, and determine the amount of cache storage space based on the size of the target file itself. That is, determine the number of cache solid-state drives based on the size of the target file, that is, determine how many cache solid-state drives the target file needs to be copied to. For example, if the current target file size is 500MB, the target file can be copied to four cache solid-state drives.

[0031] Step S12: Obtain the IO load occupancy of each of the multiple cache storage spaces. Based on the IO load occupancy of each cache storage space, determine a target cache storage space that meets the required quantity. In this embodiment, the storage occupancy refers to the currently occupied cache storage space. The IO load occupancy refers to the current processing speed of the cache storage space, or it may be the read / write utilization rate of the cache solid-state drive. A hard drive's "low IO load occupancy" generally indicates a low speed at which the hard drive currently processes tasks, which can be understood as indicating that the cache storage space is currently relatively idle and can be assigned more tasks. Specifically, obtain the IO load occupancy of all cache solid-state drives currently in the system. Based on the number of cache solid-state drives determined in the above step, select the cache solid-state drive with the lowest IO load occupancy. For example, if the number is 4, select the four cache solid-state drives with the lowest current IO load occupancy. Specifically, the current IO read / write occupancy of the cache solid-state drives can be obtained using the IO load detector (disk_io_counters) of the monitoring module (psutil).

[0032] For example, the storage occupancy is the current remaining capacity of the cache storage space. When tasks are assigned to the hard disk with the lowest storage occupancy, overload may occur. That is, although the cache storage space occupancy is low, there are still many tasks to be processed. Therefore, multiple cache storage spaces with the lowest IO load occupancy need to be selected.

[0033] Specifically, when the IO load occupancy is relatively low, the cache storage space may be overloaded due to the lack of judgment on the remaining space of the cache storage space, even though it is currently relatively idle. To prevent this problem from occurring, on the one hand, the capacity of the cache space can be increased so that the capacity of the cache storage space is much larger than the capacity of each file, which will reduce the probability of this problem occurring. On the other hand, an expiration cleanup time can be set for each file (for example, 5 days, the user can also determine the expiration cleanup time based on actual conditions). When the expiration cleanup time is reached, the download request volume of the corresponding file is judged. When the download request volume of this file returns to zero, this file is deleted, which can also ensure that the cache storage space is sufficient. In addition, the IO load occupancy of the cache storage space is regularly analyzed. When it exceeds a preset threshold (for example, 80%), the file closest to the expiration time and with a current download request volume of 0 will be forcibly deleted.

[0034] Exemplarily, when a file in the cache storage space is deleted, the file download address will be updated to the location of the source file in the main storage space.

[0035] Step S13: Slice the target file to generate multiple target slice files. In this embodiment, the target file is sliced ​​according to preset conditions to generate multiple target slice files. The generated multiple target slice files are numbered according to the slice order to obtain the target slice file order. The preset conditions may be that when the target file is less than 100MB, the slice size is 4MB, and the target file is sliced ​​according to the 4MB slice size; when the target file is between 100-500MB, the slice size is 10MB; when the target file is larger than 500MB, the slice size is 20MB.

[0036] Step S14: Copy and store the target slice file in multiple target cache storage spaces. In this embodiment, the target cache storage space is the cache storage space with the lowest current IO load determined based on the actual hardware configuration, that is, the cache solid-state drive with the lowest current IO load. Specifically, the IO load occupancy can be determined based on the detection results of the counter (disk_io_counters) of the monitoring module (psutil).

[0037] Step S15: Determine the target download amount of each target cache storage space based on the IO load occupancy and number of the target slice files of the target cache storage space; in this embodiment, the number of target slice files that need to be downloaded in each target cache storage space, that is, the target download amount, is proportionally allocated based on the IO load occupancy of the target cache storage space and the number of target slice files; for example, when the number of target cache storage spaces determined according to the size of the target file is 4, and the IO load occupancy is 20%, 30%, 50% and 10% respectively, and the number of target slice files is 20, the allocation ratio can be determined to be 8:7:5:9, that is, in the first target cache storage space, 5 target slice files need to be downloaded; in the second target cache storage space, 4 target slice files need to be downloaded; in the third target cache storage space, 3 target slice files need to be downloaded; in the fourth target cache storage space, 8 target slice files need to be downloaded.

[0038] Step S16: Download the corresponding target slice files in each target cache storage space according to the target download amount. In this embodiment, according to the order of the target slice files, the target slice files with corresponding sequence numbers and quantities are downloaded in each cache solid state drive.

[0039] A file download method provided by an embodiment of the present invention includes: obtaining a target file, determining the number of cache storage spaces based on the size of the target file; obtaining the IO load occupancy of multiple cache storage spaces respectively, and determining a target cache storage space that meets the number based on the IO load occupancy of each cache storage space; copying and storing the target file in the target cache storage space respectively; segmenting the target file to generate multiple target slice files; determining the target download volume of each target cache storage space based on the IO load occupancy and number of the target cache storage space and the number of target slice files; and downloading the corresponding target slice file in each target cache storage space based on the target download volume. Storing the target file in multiple target storage spaces solves the problem of extended large file download time and reduced device reliability in the related art. File download tasks can be reasonably allocated based on the resource occupancy of the current target cache storage space, thereby improving the transmission speed and reliability of large file data downloads, thereby ensuring stable operation of the device.

[0040] As an optional embodiment of the present invention, step S11 of obtaining the target file specifically includes determining that the download file is the target file when the number of download requests corresponding to the download file meets a target condition; the target condition is that the ratio of the number of download requests in the current preset period to the number of download requests in the previous preset period adjacent to the current preset period is greater than or equal to a preset threshold. In this embodiment, the system contains multiple download files available for users to download, each of which has a corresponding download link. Users can use the download link to issue a download request and subsequently download the download file in the system.

[0041] Specifically, when the number of download requests determined according to the download link meets the target condition, it can be determined that the currently downloaded file is the target file; the preset period can be ten minutes, the current preset period can be within the current ten minutes, and the previous preset period adjacent to the current preset period can be the previous ten minutes; the target condition can be that the number of download requests within the current ten minutes is a target multiple of the number of download requests within the previous ten minutes, for example, five times, etc.; that is, the target condition is that the number of download requests for the same download link reaches 500% of the number of download requests in the previous ten minutes within 10 minutes.

[0042] Specifically, when the number of download requests corresponding to the downloaded file meets the target condition, that is, when the currently downloaded file is determined to be the target file, it can be considered that the target file has reached the download peak at this time. The actual application scenario can be within 24 hours after the system pushes the new version update file.

[0043] Based on the above process, an embodiment of the present invention provides a file downloading method that can periodically analyze download requests and timely adjust the strategy for providing each target slice file according to the increase in the number of download requests.

[0044] Specifically, the target download volume of each target cache storage space is determined by the following formula:

[0045]

[0046]

[0047] Where Pi represents the IO load occupied by the i-th target cache storage space, C represents the number of target slice files, n represents the preset storage quantity of the cache storage space, Ti represents the target download quantity of the i-th target cache storage space, i is a positive integer, n is a positive integer, and the number of slice files required to be provided by the first n-1 hard disks are T1…T n-1 , A represents the number of slice files to be provided by the nth hard disk (i.e., the hard disk with the highest IO load).

[0048] A file downloading method provided by an embodiment of the present invention determines the target download amount in combination with the current IO occupancy ratio of each target cache solid-state hard disk, that is, according to the resource occupancy of the current cache solid-state hard disk, reasonably allocates tasks, reduces the possibility of system failure caused by excessive load of a certain hard disk, improves the stability of the system, and when providing a target slice file, can be downloaded from multiple cache solid-state hard disks at the same time, thereby improving the speed of downloading the target file of the device.

[0049] As an optional embodiment of the present invention, the file downloading method further includes:

[0050] First, when the download of the corresponding target slice file in each target cache storage space is paused, the current download progress and download link are stored; in this embodiment, in the scenario of downloading the target slice file in a target cache storage space, if the current network condition is abnormal and the download action is paused, the download link and download progress corresponding to the current download target file can be automatically stored, and the download progress can be fifty percent, etc.

[0051] Secondly, when re-downloading the corresponding target slice file in each target cache storage space, the current download position is determined based on the current download progress and download link; based on the current download position, the corresponding target slice file is re-downloaded. In this embodiment, when the network condition recovers and the target slice file can be downloaded again through the download link, the pause position can be determined based on the download link and download progress stored in the above steps; the pause position is the download position where the network condition becomes abnormal. When re-downloading, the current download position can be determined based on the preset copy amount. That is, when re-downloading, the download will be restarted 100 bytes before the pause position.

[0052] Specifically, after the target slice file is re-downloaded, the hash value of the target slice file can be calculated and compared with the hash value of the previously downloaded target slice file; when they do not match, the pause position and the current download position can be re-determined, and then the target slice file can be downloaded.

[0053] As an optional embodiment of the present invention, the file download method further includes: obtaining the download address of each target cache storage space, and updating the preset address database list according to the download address of each target cache storage space. In this embodiment, when multiple target slice files are generated by slicing in the first cache solid-state drive and the generated multiple target slice files are copied to multiple other cache solid-state drives, the location of the target slice files in each target cache solid-state drive is obtained, and the storage location of the target slice file is mapped to a download link through a preset NGINX tool, a download address is generated according to the mapped download location, and the current download address of the file is updated in the preset address database list.

[0054] As an optional embodiment of the present invention, the file downloading method also includes: copying and storing the target file in a main storage space, and when the target cache storage space fails, downloading the corresponding target slice file in the main storage space according to the target download amount corresponding to the failed target cache storage space. In this embodiment, the main storage space can be a main storage hard disk; the failure can be that the target slice file cannot be downloaded in the cache solid-state hard disk, that is, the cache solid-state hard disk cannot provide the corresponding target slice file, and the generated multiple target slice files are copied to the main storage hard disk. Specifically, in the specific application scenario of downloading the target slice file, if a target cache solid-state hard disk is unresponsive and cannot provide the corresponding target slice file, the corresponding target slice file can be downloaded from another target cache solid-state hard disk adjacent to the current target cache solid-state hard disk; if all target cache solid-state hard disks cannot provide the target slice file, the corresponding target slice file can be downloaded from the main storage hard disk.

[0055] Specifically, for example, the target file is divided into 20 target slice files. In the first target cache solid-state drive, 5 target slice files are downloaded; in the second target cache solid-state drive, 4 target slice files are downloaded; in the third target cache solid-state drive, 3 target slice files are downloaded; in the fourth target cache solid-state drive, 8 target slice files are downloaded; when the third target cache solid-state drive fails, the corresponding tenth to twelfth target slice files can be downloaded from the second target cache solid-state drive or the fourth target cache solid-state drive. When the first target cache solid-state drive, the second target cache solid-state drive, and the fourth target cache solid-state drive all fail, all 20 target slice files can be downloaded from the primary storage hard drive.

[0056] Specifically, the target file is generally a large file such as a system upgrade package file. Since the download address corresponding to the system upgrade package generally has periodic access peaks, that is, within one week of the system upgrade package being pushed, it can be considered as the download peak period of the above-mentioned system upgrade package. When the administrator is managing his own equipment in a specific application scenario, when the upgrade rate of the current device determined according to the device list is high (for example, more than 90%), the system can set the download address corresponding to the download link of the system upgrade package to the download address in the main storage hard disk, and delete multiple target slice files in each target cache solid-state hard disk, that is, when the device downloads the system upgrade package again, it will be downloaded from the main storage hard disk, and will not be downloaded from other cache solid-state hard disks. The system administrator can flexibly arrange the cycle of cleaning the slice file data in the cache solid-state hard disk according to the current upgrade status of each device to ensure the stability of the system operation.

[0057] A file download method provided by an embodiment of the present invention combines the copying of multiple generated target slice files to a primary storage hard disk and multiple target cache solid-state drives, and a switching mechanism that can obtain corresponding target slice files from adjacent cache solid-state drives. This can ensure the reliability of users downloading target slice files, thereby greatly improving the stability of system operation.

[0058] As an optional embodiment of the present invention, the file download method further includes: obtaining the file name of the target file; obtaining a splitting timestamp for splitting the target file according to preset conditions; and generating a target slice folder name based on the file name, the splitting timestamp, and the random number. In this embodiment, the file name of the target file is first obtained, that is, the file name of the complete target file pre-stored in the system; then the time node of splitting the target file to obtain multiple target slice files is obtained; the name of the target slice folder is generated based on the file name of the target file, the time node of generating multiple target slice files, and randomly generated numbers or letters. The generated multiple target slice files can be stored in the target slice folder. The randomly generated numbers or letters can be three, four, or any other number of digits determined according to the actual application scenario. The name of the target slice folder needs to remove the decimal point to prevent errors when identifying the file type.

[0059] A file downloading method provided by an embodiment of the present invention combines the file name of the target file with the timestamp of the current segmentation time (for example, the accuracy can be accurate to the microsecond level), and creates a folder for storing the target slice file by splicing a name composed of a combination of three random letters or numbers. The target slice file is saved in the folder, which can ensure that even if multiple target files are segmented in a short period of time, the slice file storage location of the same target file can be unique and not confused.

[0060] The embodiment of the present invention also provides a device for downloading files, such as Figure 3 As shown, the device includes:

[0061] The quantity determination module 21 is used to obtain the target file and determine the quantity of cache storage space according to the size of the target file; the detailed implementation content can be found in the relevant description of step S11 in the above method embodiment.

[0062] The target cache storage space determination module 22 is used to obtain the IO load occupancy of multiple cache storage spaces respectively, and determine the target cache storage space that meets the quantity according to the IO load occupancy of each cache storage space; the detailed implementation content can be found in the relevant description of step S12 in the above method embodiment.

[0063] The copy module 23 is used to copy the target files and store them in the target cache storage space respectively; the detailed implementation content can refer to the relevant description of step S13 in the above method embodiment.

[0064] The segmentation module 24 is used to segment the target file to generate multiple target slice files; the detailed implementation content can be found in the relevant description of step S14 in the above method embodiment.

[0065] The target download amount determination module 25 is used to determine the target download amount of each target cache storage space according to the IO load occupancy and quantity of the target cache storage space and the number of target slice files; for detailed implementation content, please refer to the relevant description of step S15 in the above method embodiment.

[0066] The download module 26 is used to download the corresponding target slice file in each target cache storage space according to the target download amount. Detailed implementation content can be found in the description of step S16 in the above method embodiment.

[0067] An embodiment of the present invention provides a file downloading device, comprising: a quantity determination module 21 for obtaining a target file and determining the number of cache storage spaces based on the size of the target file; a target cache storage space determination module 22 for respectively obtaining the IO load occupancy of multiple cache storage spaces and determining a target cache storage space that meets the quantity based on the IO load occupancy of each cache storage space; a copy module 23 for respectively copying the target file and storing it in the target cache storage space; a segmentation module 24 for segmenting the target file to generate multiple target slice files; a target download quantity determination module 25 for determining the target download quantity of each target cache storage space based on the IO load occupancy and quantity of the target cache storage space and the number of target slice files; and a download module 26 for downloading the corresponding target slice file from each target cache storage space based on the target download quantity. Storing the target file in multiple target storage spaces solves the problem of prolonged large file download time and reduced device reliability in the related art. File download tasks can be reasonably allocated based on the resource occupancy of the current target cache storage space, thereby improving the transmission speed and reliability of large file data downloads and ensuring stable operation of the device.

[0068] The embodiment of the present invention also provides a computer device, such as Figure 4 As shown, the computer device may include a processor 31 and a memory 32, wherein the processor 31 and the memory 32 may be connected via a bus 30 or other means. Figure 4 The connection via bus 30 is taken as an example.

[0069] The processor 31 may be a central processing unit (CPU). The processor 31 may also be other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, or a combination of the above chips.

[0070] Memory 32, as a non-transitory computer-readable storage medium, can be used to store non-transitory software programs, non-transitory computer executable programs, and modules, such as the program instructions / modules corresponding to the file download method in the embodiments of the present invention. Processor 31 executes the non-transitory software programs, instructions, and modules stored in memory 32 to perform various processor functions and data processing, thereby implementing the file download method in the above-mentioned method embodiments.

[0071] The memory 32 may include a program storage area and a data storage area, wherein the program storage area may store an operating system and application programs required for at least one function; the data storage area may store data created by the processor 31, etc. In addition, the memory 32 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other non-volatile solid-state storage device. In some embodiments, the memory 32 may optionally include a memory remotely located relative to the processor 31, and these remote memories may be connected to the processor 31 via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0072] The one or more modules are stored in the memory 32 and when executed by the processor 31, perform the following steps: Figure 1 The file downloading method in the illustrated embodiment.

[0073] For details of the above computer equipment, please refer to Figure 1 The corresponding descriptions and effects in the embodiments shown can be understood and will not be repeated here.

[0074] An embodiment of the present invention further provides a non-transitory computer-readable medium, which stores computer instructions. The computer instructions are used to enable a computer to execute the file download method described in any one of the above embodiments, wherein the storage medium may be a magnetic disk, an optical disk, a read-only memory (ROM), a random access memory (RAM), a flash memory (Flash Memory), a storage space (Hard Disk Drive, abbreviated as: HDD) or a storage space (Solid-State Drive, SSD), etc.; the storage medium may also include a combination of the above types of memory.

[0075] Obviously, the above embodiments are merely examples for clarity of explanation and are not intended to limit the implementation methods. Those skilled in the art will readily appreciate that other variations or modifications based on the above descriptions are possible. It is not necessary and impossible to enumerate all implementation methods here. Obvious variations or modifications arising therefrom remain within the scope of protection of the present invention.

Claims

1. A file downloading method, characterized in that: include: Acquire a target file that is currently in a download peak period, and determine the amount of cache storage space according to the size of the target file; Respectively obtain the IO load occupancy of multiple cache storage spaces, and determine the target cache storage space that meets the quantity according to the IO load occupancy of each cache storage space, wherein the IO load occupancy indicates the current processing task speed of the cache storage space; Slicing the target file to generate multiple target slice files; Copying and storing the target slice file in multiple target cache storage spaces; Determine the target download amount of each target cache storage space according to the IO load occupancy of the target cache storage space, the number and the number of target slice files, wherein the target download amount represents the number of target slice files that need to be downloaded by each target cache storage space; According to the target download amount, the corresponding target slice file is downloaded in each target cache storage space.

2. The method according to claim 1, characterized in that The step of obtaining the target file that is currently in a download peak period includes: when the number of download requests corresponding to the download file meets the target condition, determining that the download file is the target file; the target condition is that the ratio of the number of download requests in the current preset period to the number of download requests in the previous preset period adjacent to the current preset period is greater than or equal to a preset threshold.

3. The method according to claim 1, characterized in that: The target download volume of each target cache storage space is determined by the following formula: Among them, P i represents the IO load usage of the i-th target cache storage space, C represents the number of target slice files, n represents the preset storage quantity of the cache storage space, T i represents the target download amount of the i-th target cache storage space, i is a positive integer, and n is a positive integer; A represents the number of slice files provided by the n-th target cache storage space.

4. The method according to claim 1, characterized in that: Also includes: When downloading of the corresponding target slice file in each target cache storage space is paused, the current download progress and download link are stored; When re-downloading the corresponding target slice file in each target cache storage space, determining the current download position according to the current download progress and the download link; According to the current download location, re-download the corresponding target slice file.

5. The method according to claim 1, characterized in that Also includes: The download address of each target cache storage space is obtained, and the preset address database list is updated according to the download address of each target cache storage space.

6. The method according to claim 1, characterized in that Also includes: The target file is copied and stored in a main storage space. When the target cache storage space fails, the corresponding target slice file is downloaded from the main storage space according to the target download amount corresponding to the failed target cache storage space.

7. The method according to claim 1, characterized in that Also includes: Get the file name of the target file; Obtaining a split timestamp for splitting the target file according to a preset condition; Generate a target slice folder name based on the file name, the segmentation timestamp and the random number.

8. A file downloading device, characterized in that: include: A quantity determination module is used to obtain a target file that is currently in a download peak period and determine the amount of cache storage space according to the size of the target file; a target cache storage space determination module, used to respectively obtain the IO load occupancy of a plurality of cache storage spaces, and determine the target cache storage space that meets the quantity according to the IO load occupancy of each cache storage space, wherein the IO load occupancy indicates the current processing task speed of the cache storage space; A copy module, used for copying the target files respectively and storing them in the target cache storage space; A segmentation module, used for segmenting the target file to generate multiple target slice files; A target download amount determination module is used to determine the target download amount of each target cache storage space according to the IO load occupation of the target cache storage space, the number and the number of target slice files, wherein the target download amount represents the number of target slice files that need to be downloaded by each target cache storage space; The download module is used to download the corresponding target slice file in each target cache storage space according to the target download amount.

9. A computer device, characterized in that: include: at least one processor; And a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the one processor, and the instructions are executed by the at least one processor so that the at least one processor executes the steps of the file downloading method described in any one of claims 1-7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the file downloading method according to any one of claims 1 to 7 are implemented.