File downloading method and device, equipment, storage medium and program product

Abstract

The invention discloses a file downloading method and device, equipment, a storage medium and a program product, and relates to the technical field of data processing. The file downloading method comprises the following steps: receiving a file auditing request, wherein the request comprises file types of a plurality of files and downloading links of the files; storing the file type of each file and the downloading link of each file in a first downloading queue; according to the file type, sequentially moving the downloading link of each file from the first downloading queue to a second downloading queue corresponding to the file type; setting a corresponding sliding window length for a second downloading queue corresponding to each file type according to a reference downloading thread count of the auditing platform and an auditing amount proportion of each file type in a preset time period; according to the lengths of the sliding windows, downloading links are sequentially obtained from the corresponding second downloading queues for file downloading. According to the embodiment, the file auditing accuracy and the file auditing efficiency can be improved.

Description

Technical Field

[0001] The present application belongs to the field of data processing technology, and in particular, relates to a file downloading method, device, equipment, storage medium and program product. Background Art

[0002] With the continuous development of cloud disk business, the storage content in cloud disk is increasing. The management and control of storage content is a problem that needs to be solved at present. It can prevent criminals from using cloud disk to spread illegal content and have a negative impact on society.

[0003] In the existing method, the review platform receives the review request from the business, obtains the corresponding file download address, and downloads and reviews the review file by requesting the download.

[0004] However, due to the diversity of document types and the imbalance of document numbers, the accuracy of document review is poor and the efficiency of document review is low. Summary of the invention

[0005] The embodiments of the present application provide a file downloading method, apparatus, device, storage medium and program product, which can improve the accuracy and efficiency of file review.

[0006] In one aspect of an embodiment of the present application, a method for downloading a file is provided, the method comprising:

[0007] Receive a file review request, the request including file types of multiple files and download links of each file;

[0008] storing the file type of each file in the request and the download link of each file in the first download queue;

[0009] According to the file type, the download link of each file is moved from the first download queue to the second download queue corresponding to the file type in sequence;

[0010] According to the reference download thread number of the audit platform and the audit volume ratio of each file type in a preset time period, a corresponding sliding window length is set for the second download queue corresponding to each file type;

[0011] According to the length of each sliding window, download links are sequentially obtained from the corresponding second download queue to download the file.

[0012] In one aspect of an embodiment of the present application, a file downloading device is provided, the device comprising:

[0013] A request receiving module, used to receive a file review request, the request including the file types of multiple files and the download links of each file;

[0014] A file storage module, used for storing the file type of each file in the request and the download link of each file in the first download queue;

[0015] A link migration module, used to move the download link of each file from the first download queue to the second download queue corresponding to the file type in sequence according to the file type;

[0016] A length setting module, used to set a corresponding sliding window length for the second download queue corresponding to each file type according to the number of reference download threads of the review platform and the review volume ratio of each file type within a preset time period;

[0017] The link acquisition module is used to sequentially acquire download links from the corresponding second download queue according to the length of each sliding window to download the file.

[0018] In one aspect of an embodiment of the present application, an electronic device is provided, which includes: a memory and a program or instruction stored in the memory and executable on a processor, wherein when the program or instruction is executed by the processor, a file download method as provided in any aspect of the above-mentioned embodiment of the present application is implemented.

[0019] In one aspect of an embodiment of the present application, a readable storage medium is provided, on which a program or instruction is stored. When the program or instruction is executed by a processor, a file download method provided in any aspect of the above-mentioned embodiment of the present application is implemented.

[0020] According to one aspect of an embodiment of the present application, a computer program product is provided. When instructions in the computer program product are executed by a processor of an electronic device, the electronic device executes a file downloading method provided by any aspect of the embodiment of the present application described above.

[0021] In the processing method of the audit file provided in the embodiment of the present application, multiple second download queues are created according to the file type, and the download links of the files of the corresponding file type are stored in each second download queue. According to the file type corresponding to each second download queue, the corresponding file audit standard can be set for each second download queue, so as to improve the accuracy of the file audit. At the same time, according to the reference number of download threads of the audit platform and the audit volume ratio of each file type in the preset time period, the corresponding sliding window length is set for the second download queue corresponding to each file type. According to the audit volume of each file type, the corresponding number of download threads is allocated, which can ensure balanced processing between the file types and avoid the low efficiency of the overall file audit caused by the excessive audit volume of a certain file type. In this way, the embodiment of the present application can improve the accuracy of file audit and the efficiency of file audit by creating multiple second download queues and configuring the corresponding number of download threads for each second download queue. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] In order to more clearly illustrate the technical solution of the embodiments of the present application, the following is a brief introduction to the drawings required for use in the embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0023] Figure 1 It is a flowchart of a file downloading method provided by an embodiment of the present application;

[0024] Figure 2 This is a schematic diagram of a process of downloading files by a second download queue provided in an embodiment of the present application;

[0025] Figure 3 It is a schematic diagram of a process of adjusting the length of a sliding window provided by an embodiment of the present application;

[0026] Figure 4 It is a structural diagram of a file downloading device provided by an embodiment of the present application;

[0027] Figure 5 It is a structural diagram of a file downloading device provided by an embodiment of the present application. DETAILED DESCRIPTION

[0028] The features and exemplary embodiments of various aspects of the present application will be described in detail below. In order to make the purpose, technical solutions and advantages of the present application clearer, the present application will be further described in detail below in conjunction with the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are only intended to explain the present application, rather than to limit the present application. For those skilled in the art, the present application can be implemented without the need for some of these specific details. The following description of the embodiments is only to provide a better understanding of the present application by illustrating the examples of the present application.

[0029] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the statement "include..." do not exclude the presence of other identical elements in the process, method, article or device including the elements.

[0030] It should be noted that the acquisition, storage, use, and processing of data in the technical solution of this application are in compliance with the relevant provisions of national laws and regulations.

[0031] It should be noted that in the embodiments of the present application, certain software, components, models and other existing solutions in the industry may be mentioned, and they should be regarded as exemplary. Their purpose is only to illustrate the feasibility of implementing the technical solution of the present application, but it does not mean that the applicant has or will necessarily use the solution.

[0032] In the related art, the review platform receives the review request from the business, obtains the corresponding file download address, and downloads and reviews the review file by requesting download. However, due to the diversity of file types and the imbalance of file numbers, the accuracy of file review is poor and the efficiency of file review is low.

[0033] The purpose of the present application is to provide a file download method, device, equipment, storage medium and program product. In the file download method provided in the embodiment of the present application, multiple second download queues are created according to the file type, and the download links of the files of the corresponding file type are stored in each second download queue. According to the file type corresponding to each second download queue, the corresponding file review standard can be set for each second download queue, so as to improve the accuracy of the file review. At the same time, according to the reference download thread number of the review platform and the review volume ratio of each file type in the preset time period, the corresponding sliding window length is set for the second download queue corresponding to each file type. According to the review volume of each file type, the corresponding number of download threads is allocated, which can ensure the balanced processing between the file types and avoid the low efficiency of the overall file review caused by the excessive review volume of a certain file type. In this way, the embodiment of the present application can improve the accuracy of file review and the efficiency of file review by creating multiple second download queues and configuring the corresponding number of download threads for each second download queue.

[0034] The following describes specific embodiments of the file downloading method, device, equipment, storage medium and program product provided by the embodiments of the present application. First, the case handling method is introduced.

[0035] Figure 1 A flowchart of a file downloading method is provided. The file downloading method can be applied to a server. The file downloading method can include the following S101 to S105.

[0036] S101, receiving a file review request, the request including file types of multiple files and download links of each file.

[0037] In this embodiment, the file review request includes the file types of multiple files to be reviewed and the download links corresponding to the files. For example, the file types may include picture types, video types, audio types, and document types.

[0038] As an example, the server responds to the file review request initiated by the upstream service end and receives the file type of each file to be reviewed and the download link corresponding to the file included in the file review request. Furthermore, the file review request may also include basic information such as file size and file suffix.

[0039] S102: Store the file type of each file in the request and the download link of each file in a first download queue.

[0040] In this embodiment, the first download queue is a main queue responsible for maintaining the order and status of the files to be reviewed. It is only responsible for retrieving and storing the file type and download link of the files to be reviewed, and does not perform actual download tasks.

[0041] As an example, after receiving the file review request, the server stores the file type and download link of each to-be-reviewed file included in the file review request in the first download queue.

[0042] S103: According to the file type, the download link of each file is moved from the first download queue to the second download queue corresponding to the file type in sequence.

[0043] In this embodiment, the second download queue is a subqueue responsible for executing the actual download task, and each file type corresponds to a second download queue. For example, the picture type may correspond to the second download queue A, the video type may correspond to the second download queue B, the audio type may correspond to the second download queue C, and the document type may correspond to the second download queue D.

[0044] As an example, after the server stores the file type and download link of each file to be reviewed in the first download queue, it then transfers the corresponding download links from the first download queue to the second download queue matching the file type according to the file type of each file to be reviewed in the first download queue.

[0045] S104, according to the number of reference download threads of the review platform and the review volume ratio of each file type within a preset time period, a corresponding sliding window length is set for the second download queue corresponding to each file type.

[0046] In this embodiment, the reference number of download threads is the maximum number of download threads that the audit platform can provide for downloading files.

[0047] The review volume ratio is the proportion of the review volume of the corresponding file type to the review volume of all file types, and the sliding window length is used to represent the number of files that can be downloaded at one time.

[0048] As an example, when the number of reference download threads of the review platform is 10, the review ratio of picture type within the preset time period is 20%, the review ratio of video type within the preset time period is 30%, the review ratio of audio type within the preset time period is 40%, and the review ratio of document type within the preset time period is 10%.

[0049] The corresponding sliding window length can be allocated according to the relationship between the review amount ratios corresponding to each file type, that is, the sliding window length of the picture type is 2, the sliding window length of the video type is 3, the sliding window length of the audio type is 4, and the sliding window length of the document type is 1.

[0050] S105: According to the length of each sliding window, download links are sequentially obtained from the corresponding second download queue to download the file.

[0051] In this embodiment, after determining the sliding window length corresponding to the second download queue of each file type, the server acquires the download link according to the sliding window length to download the file to be reviewed.

[0052] As an example, Figure 2 As shown, a schematic diagram of a process of downloading files using a second download queue is provided.

[0053] Specifically, when the sliding window length corresponding to the sliding window is 2, the server first uses the sliding window to obtain the download links of file 1 and file 2 from the second download queue to download the files.

[0054] Then, after completing the download of file 1 and file 2, the sliding window is slid in the preset sliding direction to continue to obtain the download links of file 3 and file 4 to download the files, and so on, until the download links of all files in the second download queue are obtained, then the acquisition of download links is stopped.

[0055] Through this embodiment, multiple second download queues are created according to the file type, and each second download queue stores the download link of the file of the corresponding file type. According to the file type corresponding to each second download queue, the corresponding file review standard can be set for each second download queue, so as to improve the accuracy of the file review. At the same time, according to the reference number of download threads of the review platform and the review volume ratio of each file type in the preset time period, the corresponding sliding window length is set for the second download queue corresponding to each file type. According to the corresponding number of download threads allocated according to the review volume of each file type, balanced processing between the file types can be guaranteed, and the low efficiency of the overall file review caused by the excessive review volume of a certain file type can be avoided. In this way, the embodiment of the present application can improve the accuracy and efficiency of file review by creating multiple second download queues and configuring the corresponding number of download threads for each second download queue.

[0056] As an optional embodiment, S104 may specifically include:

[0057] Multiply the number of reference download threads of the audit platform by the preset length coefficient to obtain the total length of the window;

[0058] Determine the window split coefficient for each file type based on the proportion of each audit volume and the proportion threshold;

[0059] The total window length is multiplied by the window splitting coefficient of each file type to obtain the sliding window length of the second download queue corresponding to each file type.

[0060] In this embodiment, the total window length is the accumulated value of the sliding window lengths corresponding to the second download queues of each file type.

[0061] The window splitting coefficient is the ratio of the total window length to the sliding window length corresponding to the second download queue.

[0062] As an example, the preset length coefficient is 0.6, and the number of reference download threads of the audit platform is 10, then the total window length is 10*0.6=6.

[0063] Then, the review quantity ratio corresponding to each file type is compared with the preset ratio threshold. If it is greater than or equal to the preset ratio threshold, the window splitting coefficient of the corresponding file type can be set to 0.3; if it is less than the preset ratio threshold, the window splitting coefficient of the corresponding file type can be set to 0.1, thereby obtaining the window splitting coefficient of each file type.

[0064] Finally, the total window length is multiplied by the window splitting coefficient of each file type, so as to obtain the sliding window length of the second download queue corresponding to each file type.

[0065] As another example, when the calculated sliding window length is a non-integer, the calculated sliding window length may be rounded up or down. At the same time, the sliding window length of the corresponding type may be rounded up or down according to the audit volume ratio of each file type.

[0066] For example, assuming that the result of sorting the review quantity ratio of each file type is: picture review quantity ratio>video review quantity ratio>audio review quantity ratio>document review quantity ratio. Then, if the calculated sliding window length is a non-integer, the sliding window length of the picture type and the sliding window length of the video type are rounded up, and the sliding window length of the audio type and the sliding window length of the document type are rounded down.

[0067] Through this embodiment, the total window length is determined according to the reference download thread number of the audit platform and the preset length coefficient. Then, according to the audit volume ratio and the ratio threshold, the window splitting coefficient of each file type is obtained. Finally, according to the total window length and the window splitting coefficient, the sliding window length of the second download queue corresponding to each file type can be obtained. In this way, this embodiment can accurately obtain the sliding window length of the second download queue corresponding to each file type by comprehensively considering the reference download thread number of the audit platform and the audit volume ratio of each file type, thereby improving the overall efficiency of file audit.

[0068] As an optional embodiment, the ratio threshold includes a first threshold and a second threshold, and the second threshold is greater than the first threshold;

[0069] According to the proportion of each audit volume and the proportion threshold, the window split coefficient of each file type is determined, which may include:

[0070] Compare the maximum ratio among the audit amount ratios with the ratio threshold to obtain a first comparison result;

[0071] When the first comparison result indicates that the maximum ratio is greater than the second threshold, setting the window splitting coefficient of the file type with the maximum ratio to the second threshold;

[0072] The remaining split coefficients are allocated according to the relationship between the audit volume ratios of each first file type to obtain the window split coefficients of each first file type, where the first file type is each file type except the file type with the largest ratio.

[0073] In this embodiment, after obtaining the review ratio corresponding to each file type, the server first sorts them according to the proportion. Assume that the sorting result is: picture review ratio > video review ratio > audio review ratio > document review ratio.

[0074] Then, the image review amount ratio is compared with the ratio threshold. When the image review amount ratio is greater than the second threshold, the window splitting coefficient of the image type is set to the second threshold.

[0075] For the window splitting coefficients of other file types, the remaining splitting coefficients are allocated according to the relationship between the audit quantity ratios. For example, the window splitting coefficient of the video type = (1-second threshold)*video audit quantity ratio / (video audit quantity ratio+audio audit quantity ratio+document audit quantity ratio).

[0076] Through this embodiment, the window splitting coefficient of each file type is determined according to each audit volume ratio and ratio threshold. By comprehensively considering the audit volume ratio of each file type, the window splitting coefficient corresponding to each file type can be accurately obtained, which helps to reasonably allocate the sliding window length, thereby improving the overall efficiency of file auditing.

[0077] As an optional embodiment, after comparing the maximum audit amount ratio among the audit amount ratios with the ratio threshold and obtaining the first comparison result, the file download method specifically further includes:

[0078] When the first comparison result indicates that the maximum ratio is between the first threshold and the second threshold, the sum of the maximum ratio and the second largest ratio among the audit amount ratios is compared with the second threshold to obtain a second comparison result;

[0079] When the second comparison result indicates that the sum value is not less than the second threshold, the second threshold is allocated according to the relationship between the maximum proportion and the second largest proportion to obtain the window splitting coefficient of the file type with the largest proportion and the window splitting coefficient of the file type with the second largest proportion;

[0080] The remaining split coefficients are allocated according to the relationship between the audit volume ratios of each second file type to obtain the window split coefficients of each second file type, where the second file types are all file types except the file type with the largest ratio and the file type with the second largest ratio.

[0081] In this embodiment, it is assumed that the result obtained by sorting the review quantity ratio of each file type is: picture review quantity ratio > video review quantity ratio > audio review quantity ratio > document review quantity ratio.

[0082] When the image review amount ratio is greater than or equal to the first threshold and less than or equal to the second threshold, the sum of the image review amount ratio and the video review amount ratio is further compared with the second threshold.

[0083] When the sum of the image review ratio and the video review ratio is greater than or equal to the second threshold, the second threshold is allocated according to the relationship between the maximum ratio and the second largest ratio. Specifically, the window splitting coefficient of the image type = the second threshold * image review ratio / (image review ratio + video review ratio), and the window splitting coefficient of the video type = the second threshold * video review ratio / (image review ratio + video review ratio).

[0084] Furthermore, the window splitting coefficients of other file types are allocated according to the relationship between the audit quantity ratios. Specifically, the window splitting coefficient of the audio type = (1-second threshold) * audio audit quantity ratio / (audio audit quantity ratio + document audit quantity ratio), and the window splitting coefficient of the document type = (1-second threshold) * document audit quantity ratio / (audio audit quantity ratio + document audit quantity ratio).

[0085] Through this embodiment, the window splitting coefficient of each file type is determined according to each audit volume ratio and ratio threshold. By comprehensively considering the audit volume ratio of each file type, the window splitting coefficient corresponding to each file type can be accurately obtained, which helps to reasonably allocate the sliding window length, thereby improving the overall efficiency of file auditing.

[0086] As an optional embodiment, when the first comparison result indicates that the maximum ratio is between the first threshold and the second threshold, the sum of the maximum ratio and the second largest ratio among the audit amount ratios is compared with the second threshold, and after obtaining the second comparison result, the file download method specifically further includes:

[0087] If the second comparison result indicates that the sum value is less than the second threshold, setting the window splitting coefficient of the file type with the largest proportion to the first threshold, and setting the window splitting coefficient of the file type with the second largest proportion to the difference between the second threshold and the first threshold;

[0088] The remaining splitting coefficients are distributed according to the relationship between the audit quantity ratios of each second file type to obtain the window splitting coefficients of each second file type.

[0089] In this embodiment, it is assumed that the result obtained by sorting the review quantity ratio of each file type is: picture review quantity ratio > video review quantity ratio > audio review quantity ratio > document review quantity ratio.

[0090] When the picture review amount ratio is greater than or equal to the first threshold and less than or equal to the second threshold, the sum of the picture review amount ratio and the video review amount ratio is further compared with the second threshold.

[0091] When the sum of the image review ratio and the video review ratio is also less than the second threshold, the window splitting coefficient of the file type is set to the first threshold, and the window splitting coefficient of the video type is set to the difference between the second threshold and the first threshold.

[0092] Furthermore, the window splitting coefficients of other file types are allocated according to the relationship between the audit quantity ratios. Specifically, the window splitting coefficient of the audio type = (1-second threshold) * audio audit quantity ratio / (audio audit quantity ratio + document audit quantity ratio), and the window splitting coefficient of the document type = (1-second threshold) * document audit quantity ratio / (audio audit quantity ratio + document audit quantity ratio).

[0093] Through this embodiment, the window splitting coefficient of each file type is determined according to each audit volume ratio and ratio threshold. By comprehensively considering the audit volume ratio of each file type, the window splitting coefficient corresponding to each file type can be accurately obtained, which helps to reasonably allocate the sliding window length, thereby improving the overall efficiency of file auditing.

[0094] As an optional embodiment, after comparing the maximum ratio of each audit amount ratio with the ratio threshold and obtaining the first comparison result, the file download method specifically further includes:

[0095] When the first comparison result indicates that the maximum ratio is less than the first threshold, the window splitting coefficient of each file type is set to the audit amount ratio corresponding to the file type.

[0096] In this embodiment, it is assumed that the result obtained by sorting the review quantity ratio of each file type is: picture review quantity ratio > video review quantity ratio > audio review quantity ratio > document review quantity ratio.

[0097] When the image review amount ratio is less than the first threshold, the window splitting coefficient of each file type is directly set to the review amount ratio corresponding to the file type. Specifically, the window splitting coefficient of the image type = the image review amount ratio, the window splitting coefficient of the video type = the video review amount ratio, the window splitting coefficient of the audio type = the audio review amount ratio, and the window splitting coefficient of the document type = the document review amount ratio.

[0098] Through this embodiment, the window splitting coefficient of each file type is determined according to each audit volume ratio and ratio threshold. By comprehensively considering the audit volume ratio of each file type, the window splitting coefficient corresponding to each file type can be accurately obtained, which helps to reasonably allocate the sliding window length, thereby improving the overall efficiency of file auditing.

[0099] As an optional embodiment, S105 may specifically include:

[0100] The following steps are executed in a loop until all unprocessed download links in the second download queue are traversed:

[0101] According to the order relationship between the download links in the second download queue, starting from the first unprocessed download link, obtain the download links corresponding to the length of the sliding window to download the file;

[0102] When a new download link is added to the second download queue, the second download queue is updated.

[0103] In this embodiment, it is assumed that the sliding window length corresponding to the sliding window is 2. Then, according to the order relationship between the download links in the second download queue, the server first obtains the first unprocessed download link and the second unprocessed download link to download the file; then, when the first and second download links are completed, the sliding window slides according to the preset sliding direction, and continues to obtain the third unprocessed download link and the fourth unprocessed download link to download the file, and so on, until all the unprocessed download links in the second download queue are obtained, then the download link acquisition is stopped.

[0104] Meanwhile, when there is a new download link to be added in the second download queue, the second download queue is updated in a timely manner.

[0105] Through this embodiment, according to the sliding window length, the download links are obtained from the second download queue in sequence to download files in a loop, thereby ensuring that all files to be reviewed in the second download queue can be downloaded smoothly, avoiding the phenomenon of missing files to be reviewed, and improving the reliability of file review.

[0106] As an optional embodiment, after S105, the file downloading method further includes:

[0107] Monitor each load type of the audit platform and obtain the business load score of the audit platform;

[0108] Determine the target adjustment strategy of the audit platform based on the relationship between the business load score and the score threshold;

[0109] According to the target adjustment strategy, the sliding window length of each second download queue is adjusted.

[0110] In this embodiment, the load type is a factor that describes the load situation during the file download process. For example, the load type may include a download thread number load type and an algorithm load type.

[0111] The target adjustment strategy is a strategy for adjusting the sliding window length of each second download queue.

[0112] In this embodiment, the server dynamically monitors the file review process, grasps the operation status of each dimension in the file review process, and thus obtains the business load score of the review platform.

[0113] Then the business load score is compared with the score threshold, and the corresponding target adjustment strategy of the audit platform is determined according to the size relationship between the business load score and the score threshold. Finally, the target adjustment strategy is used to dynamically adjust the sliding window length of each second download queue.

[0114] Through this embodiment, each load type of the audit platform is monitored to obtain the business load score corresponding to the audit platform. Then, according to the business load score corresponding to the audit platform, the current load of the audit platform is determined, so as to select the corresponding target adjustment strategy to timely adjust the sliding window length of each second download queue. In this way, the embodiment of the present application can further improve the efficiency of file auditing and ensure the smooth operation of the audit business by dynamically adjusting the sliding window length.

[0115] As an optional embodiment, each load type of the audit platform is monitored to obtain a business load score of the audit platform, including:

[0116] Monitor each load type of the audit platform and obtain the load ratio corresponding to each load type;

[0117] Multiply each load ratio by the corresponding load factor to obtain the load sub-score corresponding to each load type;

[0118] The load sub-scores corresponding to each load type are accumulated to obtain the business load score of the audit platform.

[0119] In this embodiment, the load ratio is information used to describe the actual load situation of the load type.

[0120] As an example, the server first presets a load factor for each load type according to the importance of each load type to the file review, and the sum of the load factors of each load type is 1.

[0121] Then, by monitoring each load type of the audit platform, the load ratio corresponding to each load type is obtained. Then, the load ratio corresponding to the load type is multiplied by the load factor corresponding to the preset load type to obtain the load sub-score corresponding to each load type. Finally, the load sub-scores corresponding to each load type are accumulated to obtain the business load score of the audit platform.

[0122] Through this embodiment, each load type of the audit platform is monitored, and the load sub-score corresponding to each load type is obtained. Finally, each load sub-score is accumulated to obtain the business load score of the audit platform. In this way, the embodiment of the present application quantifies the load conditions of each load type, thereby realizing the quantification of the business load conditions of the audit platform. According to the business load score, the target adjustment strategy suitable for the audit platform can be accurately obtained, and the accuracy of the dynamic adjustment of the sliding window length can be improved.

[0123] As an optional embodiment, the load type includes at least one of a download thread number load type, an algorithm load type, a file download speed load type, a bandwidth load type, and a cache load type;

[0124] The load ratio includes at least one of a download thread number load ratio, an algorithm load ratio, a file download speed load ratio, a bandwidth load ratio, and a cache load ratio;

[0125] The load sub-score includes at least one of a download thread number load score, an algorithm load ratio score, a file download speed load score, a bandwidth load score, and a cache load score.

[0126] In this embodiment, if Figure 3 As shown, a flow chart of adjusting the length of a sliding window is provided. When the server adjusts the length of the sliding window, the server monitors the five load types of the audit platform, namely, the number of download threads, the algorithm, the file download speed, the bandwidth, and the cache, and obtains the corresponding number of download threads, the algorithm, the file download speed, the bandwidth, and the cache load ratio, thereby further obtaining the corresponding number of download threads, the algorithm, the file download speed, the bandwidth, and the cache load scores.

[0127] Then, the download thread number load score, algorithm load ratio score, file download speed load score, bandwidth load score and cache load score are accumulated to obtain the business load score of the audit platform, and the corresponding target adjustment strategy is determined according to the business load score of the audit platform. Among them, the target adjustment strategy includes the basic adjustment method when the business load score is at the first level, the high load adjustment strategy when the business load score is at the second level, and the heavy load adjustment strategy when the business load score is at the third level.

[0128] For example, the load ratio of the number of download threads = the currently occupied number of download threads / the total number of available download threads on the platform, the load ratio of the file download speed = the current average file download speed / the upper limit of the file download speed, the bandwidth load ratio = the used bandwidth of the current download link / the total available bandwidth of the download link, and the cache load ratio = the current cache consumption / the total cache amount.

[0129] Through this embodiment, a set of audit business load scoring system is constructed based on the real-time monitoring of the audit platform on the load ratio of the number of download threads, the algorithm load ratio, the file download speed load ratio, the bandwidth load ratio and the cache load ratio, thereby realizing the quantification of the business load of the audit platform. According to the business load score, the target adjustment strategy suitable for the audit platform can be accurately obtained, which can improve the accuracy of the dynamic adjustment of the sliding window length.

[0130] As an optional embodiment, the scoring threshold includes a first scoring threshold and a second scoring threshold, and the second scoring threshold is greater than the first scoring threshold;

[0131] In the case where the service load score is less than the first score threshold, the target adjustment strategy is used as the basic adjustment method;

[0132] According to the target adjustment strategy, the sliding window length of each second download queue is adjusted, which may specifically include:

[0133] Obtaining an algorithm load factor corresponding to the algorithm load type, a first bandwidth load factor corresponding to the bandwidth load type, and a first cache load factor corresponding to the cache load type;

[0134] Accumulating the product of the algorithm load factor and the algorithm load ratio, the product of the first bandwidth load factor and the bandwidth load ratio, and the product of the first cache load factor and the cache load ratio to obtain a first adjustment score;

[0135] The difference between the first adjustment score and the preset score total value is multiplied by the number of reference download threads of the audit platform to obtain the first adjustment window length;

[0136] The sliding window length of each second download queue is adjusted using the maximum value of the first adjustment window length and the total window length.

[0137] In this embodiment, as an example, when the business load score is less than the first load score threshold, it indicates that the business load of the audit platform is in a healthy state. At this time, the basic adjustment method of the business load score at the first level can be selected to adjust the sliding window length of each second download queue.

[0138] Specifically, the server selects the algorithm load type, bandwidth load type, and cache load type that have a greater impact on the business load score as analysis targets. At the same time, three adjustment factors are pre-set, namely, the algorithm load factor corresponding to the algorithm load type, the first bandwidth load factor corresponding to the bandwidth load type, and the first cache load factor corresponding to the cache load type, wherein the sum of the algorithm load factor, the first bandwidth load factor, and the first cache load factor is 1.

[0139] Then, the server adds up the product of the algorithm load factor and the algorithm load ratio, the product of the first bandwidth load factor and the bandwidth load ratio, and the product of the first cache load factor and the cache load ratio to obtain a first adjustment score corresponding to the basic adjustment method.

[0140] Then, the server side subtracts the difference between the first adjustment score and the total value of the preset score and multiplies it by the number of reference download threads of the audit platform to obtain the first adjustment window length.

[0141] Finally, the maximum value is selected from the first adjustment window length and the total window length, and the sliding window length of each second download queue is adjusted.

[0142] Through this embodiment, the business load score is used to quantify the business load of the audit platform, and a suitable target adjustment strategy is selected according to the business load of the audit platform. The adjusted window split coefficient corresponding to each file type can be accurately obtained, which helps to reasonably adjust the sliding window length, thereby improving the overall efficiency of file auditing.

[0143] As an optional embodiment, the scoring threshold includes a first scoring threshold and a second scoring threshold, and the second scoring threshold is greater than the first scoring threshold;

[0144] When the service load score is not less than the first score threshold and not greater than the second score threshold, the target adjustment strategy is a high load adjustment method;

[0145] According to the target adjustment strategy, the sliding window length of each second download queue is adjusted, which may specifically include:

[0146] Multiplying the total window length by a first preset reduction factor to obtain a high-load window length;

[0147] Acquire a second bandwidth load factor corresponding to the bandwidth load type and a second cache load factor corresponding to the cache load type;

[0148] determining a second adjustment score according to the second bandwidth load factor, the second cache load factor, the bandwidth load ratio, and the cache load ratio;

[0149] Determine the second adjustment window length according to the second adjustment score and the number of reference download threads of the audit platform;

[0150] The sliding window length of each file download queue is adjusted using the minimum value between the second adjustment window length and the high-load window length.

[0151] In this embodiment, as an example, when the business load score is not less than the first score threshold and not greater than the second score threshold, it indicates that the business load of the audit platform fluctuates greatly in a short period of time and is already in a high load state. Therefore, the high load adjustment method with the business load score at the second level can be selected to adjust the sliding window length of each second download queue.

[0152] Specifically, the server first reduces the total window length to half of the original length to obtain the high-load window length. Then, the bandwidth load type and the cache load type are selected as the analysis targets. At the same time, two adjustment factors are set, namely, the second bandwidth load factor corresponding to the bandwidth load type and the second cache load factor corresponding to the cache load type. The sum of the second bandwidth load factor and the second cache load factor is 0.5.

[0153] Then, the server adds the product of the second bandwidth load factor and the bandwidth load ratio to the product of the second cache load factor and the cache load ratio to obtain a first calculated value, and subtracts the first calculated value from the sum of the second bandwidth load factor and the second cache load factor to obtain a second adjustment score.

[0154] Then, the server multiplies the number of available download threads by the second adjustment score and then by 2 to obtain the second adjustment window length. Finally, the minimum value is selected from the second adjustment window length and the high-load window length to adjust the sliding window length of each second download queue.

[0155] Furthermore, the server can also set multiple monitoring time periods. After the bandwidth load ratio and cache load ratio monitored by the platform meet the target conditions at the same time within the target number of consecutive monitoring time periods, the current high load adjustment ends, and the subsequent sliding window adjustment re-adjusts the window size through the basic adjustment method. Among them, the target condition can be that the bandwidth load ratio is less than or equal to the first threshold, and the cache load ratio is less than or equal to the second threshold.

[0156] Through this embodiment, the business load score is used to quantify the business load of the audit platform, and a suitable target adjustment strategy is selected according to the business load of the audit platform. The adjusted window split coefficient corresponding to each file type can be accurately obtained, which helps to reasonably adjust the sliding window length, thereby improving the overall efficiency of file auditing.

[0157] As an optional embodiment, the scoring threshold includes a first scoring threshold and a second scoring threshold, and the second scoring threshold is greater than the first scoring threshold;

[0158] When the service load score is greater than the second score threshold, the target adjustment strategy is a heavy load adjustment method;

[0159] According to the target adjustment strategy, the sliding window length of each second download queue is adjusted, which may specifically include:

[0160] Multiplying the total length of the initialization window by a second preset reduction factor to obtain a heavy load window length;

[0161] Obtain the maximum file type corresponding to the maximum ratio and the minimum file type corresponding to the minimum ratio among the audit volume ratios;

[0162] Determine, according to the heavy load window length, a first heavy load length of the largest file type, a second heavy load length of the smallest file type, and a third heavy load length of the intermediate file types excluding the largest file type and the smallest file type;

[0163] Acquire a third bandwidth load factor corresponding to the bandwidth load type and a third cache load factor corresponding to the cache load type;

[0164] determining a third adjustment score according to the third bandwidth load factor, the third cache load factor, the bandwidth load ratio, and the cache load ratio;

[0165] The first heavy load length, the second heavy load length, and the third heavy load length are adjusted according to the length recovery level corresponding to the third adjustment score.

[0166] In this embodiment, as an example, when the business load score is greater than the second score threshold, it indicates that the business load of the audit platform is in a heavy load state and the sliding window length needs to be strictly controlled. At this time, the heavy load adjustment method with the business load score at the third level can be selected to adjust the sliding window length of each second download queue.

[0167] Specifically, the server first reduces the total window length to one eighth of the original length to obtain the heavy load window length. Then, the bandwidth load type and the cache load type are selected as the analysis targets. At the same time, two adjustment factors are set, namely, the third bandwidth load factor corresponding to the bandwidth load type and the third cache load factor corresponding to the cache load type. The sum of the third bandwidth load factor and the third cache load factor is 1.

[0168] Then obtain the maximum file type corresponding to the maximum ratio and the minimum file type corresponding to the minimum ratio in each audit ratio, and only split the heavy load window length into the first heavy load length corresponding to the maximum file type and the second heavy load length corresponding to the minimum file type, and the third heavy load lengths corresponding to the remaining intermediate file types are set to 0.

[0169] Then, the server adds the product of the third bandwidth load factor and the bandwidth load ratio and the product of the third cache load factor and the cache load ratio to obtain a second calculated value, and subtracts the second calculated value from the sum of the third bandwidth load factor and the third cache load factor to obtain a third adjustment score.

[0170] Finally, the server sets three corresponding length recovery levels for the third adjustment score, and gradually adjusts the sliding window length according to the threshold reached by the third adjustment score, and finally restores it to the initialization length.

[0171] Specifically, assuming that the score thresholds are set to 0.2, 0.35, and 0.5, when the third adjustment score reaches 0.2, the sliding window length is adjusted to 2 times the current length. When the third adjustment score reaches 0.35, the sliding window length is adjusted to 2 times the current length (i.e., 4 times the heavy load window length), and when the third adjustment score reaches 0.5, the sliding window length is adjusted to 2 times the current length (i.e., 8 times the heavy load window length), and the window is restored to the initialization length.

[0172] Through this embodiment, the business load score is used to quantify the business load of the audit platform, and a suitable target adjustment strategy is selected according to the business load of the audit platform. The adjusted window split coefficient corresponding to each file type can be accurately obtained, which helps to reasonably adjust the sliding window length, thereby improving the overall efficiency of file auditing.

[0173] Based on the file downloading method, accordingly, the present application also provides a specific embodiment of the file downloading device.

[0174] like Figure 4 As shown, the file downloading device provided in the embodiment of the present application includes a request receiving module 410, a file storage module 420, a link migration module 430, a length setting module 440 and a link acquisition module 450.

[0175] The request receiving module 410 is used to receive a file review request, where the request includes the file types of multiple files and the download links of each file.

[0176] The file storage module 420 is used to store the file type of each file in the request and the download link of each file in the first download queue.

[0177] The link migration module 430 is used to move the download link of each file from the first download queue to the second download queue corresponding to the file type in sequence according to the file type.

[0178] The length setting module 440 is used to set the corresponding sliding window length for the second download queue corresponding to each file type according to the number of reference download threads of the review platform and the review volume ratio of each file type within a preset time period.

[0179] The link acquisition module 450 is used to sequentially acquire download links from the corresponding second download queue according to the length of each sliding window to download the file.

[0180] As an optional embodiment, the length setting module 440 may specifically include the following units:

[0181] A total length determination unit, used to multiply the number of reference download threads of the audit platform by a preset length coefficient to obtain the total length of the window;

[0182] A coefficient determination unit, used to determine the window splitting coefficient of each file type according to each audit amount ratio and ratio threshold;

[0183] The length setting unit is used to multiply the total window length by the window splitting coefficient of each file type to obtain the sliding window length of the second download queue corresponding to each file type.

[0184] As an optional embodiment, the ratio threshold includes a first threshold and a second threshold, and the second threshold is greater than the first threshold;

[0185] The coefficient determination unit specifically includes the following subunits:

[0186] A ratio comparison subunit, used for comparing the maximum ratio among the audit amount ratios with the ratio threshold value to obtain a first comparison result;

[0187] a coefficient setting subunit, configured to set the window splitting coefficient of the file type with the largest proportion to the second threshold value if the first comparison result indicates that the largest proportion is greater than the second threshold value;

[0188] The coefficient allocation subunit is used to allocate the remaining splitting coefficients according to the relationship between the audit quantity proportions of each first file type to obtain the window splitting coefficients of each first file type. The first file type is each file type except the file type with the largest proportion.

[0189] As an optional embodiment, after comparing the maximum audit quantity ratio among the audit quantity ratios with the ratio threshold and obtaining the first comparison result, the coefficient determination unit further includes the following subunits:

[0190] The ratio comparison subunit is further configured to compare the sum of the maximum ratio and the second largest ratio among the audit quantity ratios with the second threshold value to obtain a second comparison result when the first comparison result indicates that the maximum ratio is between the first threshold value and the second threshold value;

[0191] The coefficient allocation subunit is further configured to allocate the second threshold according to the relationship between the maximum ratio and the second largest ratio to obtain the window splitting coefficient of the file type with the largest ratio and the window splitting coefficient of the file type with the second largest ratio when the second comparison result indicates that the sum value is not less than the second threshold;

[0192] The coefficient allocation subunit is also used to allocate the remaining splitting coefficients according to the relationship between the review amount ratios of each second file type to obtain the window splitting coefficients of each second file type. The second file types are all file types except the file type with the largest ratio and the file type with the second largest ratio.

[0193] As an optional embodiment, when the first comparison result indicates that the maximum ratio is between the first threshold value and the second threshold value, the sum of the maximum ratio and the second largest ratio among the audit amount ratios is compared with the second threshold value, and after obtaining the second comparison result, the coefficient determination unit further includes the following subunits:

[0194] The coefficient setting subunit is further configured to set the window splitting coefficient of the file type with the largest proportion to the first threshold value, and to set the window splitting coefficient of the file type with the second largest proportion to the difference between the second threshold value and the first threshold value, if the second comparison result indicates that the sum value is less than the second threshold value;

[0195] The coefficient allocation subunit is further used to allocate the remaining splitting coefficients according to the relationship between the audit quantity ratios of each second file type, so as to obtain the window splitting coefficients of each second file type.

[0196] As an optional embodiment, after comparing the maximum ratio among the audit amount ratios with the ratio threshold and obtaining the first comparison result, the coefficient determination unit further includes the following subunits:

[0197] The coefficient setting subunit is also used to set the window splitting coefficient of each file type to the audit quantity ratio corresponding to the file type when the first comparison result indicates that the maximum ratio is less than the first threshold.

[0198] As an optional embodiment, the link acquisition module 450 specifically includes the following units:

[0199] The loop execution unit is used to loop the following steps until all unprocessed download links in the second download queue are traversed:

[0200] According to the order relationship between the download links in the second download queue, starting from the first unprocessed download link, obtain the download links corresponding to the length of the sliding window to download the file;

[0201] When a new download link is added to the second download queue, the second download queue is updated.

[0202] As an optional embodiment, after downloading the file by sequentially acquiring the download links from the corresponding second download queue according to the length of each sliding window, the file downloading device further includes the following modules:

[0203] A score determination module is used to monitor various load types of the audit platform and obtain a business load score of the audit platform;

[0204] A strategy determination module, used to determine the target adjustment strategy of the audit platform according to the relationship between the business load score and the score threshold;

[0205] The length adjustment module is used to adjust the sliding window length of each second download queue according to the target adjustment strategy.

[0206] As an optional embodiment, the score determination module includes the following units:

[0207] A ratio determination unit, used to monitor each load type of the audit platform and obtain the load ratio corresponding to each load type;

[0208] A sub-score determination unit, used to multiply each load ratio by the corresponding load coefficient to obtain a load sub-score corresponding to each load type;

[0209] The sub-score accumulation unit is used to accumulate the load sub-scores corresponding to each load type to obtain the business load score of the audit platform.

[0210] Based on the file download method, accordingly, the present application also provides a specific embodiment of the file download device.

[0211] Figure 5 A schematic diagram of the hardware structure of a file downloading device provided in an embodiment of the present application is shown.

[0212] The file downloading device may include a processor 501 and a memory 502 storing computer program instructions.

[0213] Specifically, the processor 501 may include a central processing unit (CPU), or an application specific integrated circuit (ASIC), or may be configured to implement one or more integrated circuits of the embodiments of the present application.

[0214] The memory 502 may include a large capacity memory for data or instructions. By way of example and not limitation, the memory 502 may include a hard disk drive (HDD), a floppy disk drive, a flash memory, an optical disk, a magneto-optical disk, a magnetic tape, or a universal serial bus (USB) drive or a combination of two or more of these. In appropriate cases, the memory 502 may include a removable or non-removable (or fixed) medium. In appropriate cases, the memory 502 may be inside or outside the integrated gateway disaster recovery device. In a specific embodiment, the memory 502 is a non-volatile solid-state memory.

[0215] The processor 501 implements any one of the file downloading methods in the above embodiments by reading and executing the computer program instructions stored in the memory 502 .

[0216] In one example, the file download device may further include a communication interface 503 and a bus 510. Figure 5 As shown, the processor 501, the memory 502, and the communication interface 503 are connected via a bus 510 and communicate with each other.

[0217] The communication interface 503 is mainly used to implement communication between various modules, devices, units and / or equipment in the embodiments of the present application.

[0218] Bus 510 includes hardware, software or both, and the parts of file download device are coupled to each other. For example, but not limitation, bus may include accelerated graphics port (AGP) or other graphics bus, enhanced industrial standard architecture (EISA) bus, front side bus (FSB), hypertransport (HT) interconnection, industrial standard architecture (ISA) bus, infinite bandwidth interconnection, low pin count (LPC) bus, memory bus, micro channel architecture (MCA) bus, peripheral component interconnection (PCI) bus, PCI-Express (PCI-X) bus, serial advanced technology attachment (SATA) bus, video electronics standard association local (VLB) bus or other suitable bus or two or more of these combinations. In appropriate cases, bus 510 may include one or more buses. Although the present application embodiment describes and shows a specific bus, the present application considers any suitable bus or interconnection.

[0219] In addition, in combination with the file downloading method in the above embodiment, the embodiment of the present application can provide a computer storage medium for implementation. The computer storage medium stores computer program instructions; when the computer program instructions are executed by a processor, any one of the file downloading methods in the above embodiment is implemented.

[0220] In addition, in combination with the file downloading method in the above embodiment, the present application embodiment can provide a computer program product to implement. When the instructions in the computer program product are executed by the processor of the electronic device, the electronic device executes any one of the file downloading methods in the above embodiment.

[0221] It should be clear that the present application is not limited to the specific configuration and processing described above and shown in the figures. For the sake of simplicity, a detailed description of the known method is omitted here. In the above embodiments, several specific steps are described and shown as examples. However, the method process of the present application is not limited to the specific steps described and shown, and those skilled in the art can make various changes, modifications and additions, or change the order between the steps after understanding the spirit of the present application.

[0222] The functional blocks shown in the above-described block diagram can be implemented as hardware, software, firmware or a combination thereof. When implemented in hardware, it can be, for example, an electronic circuit, an application specific integrated circuit (ASIC), appropriate firmware, a plug-in, a function card, etc. When implemented in software, the elements of the present application are programs or code segments that are used to perform the required tasks. The program or code segment can be stored in a machine-readable medium, or transmitted on a transmission medium or a communication link by a data signal carried in a carrier wave. "Machine-readable medium" can include any medium capable of storing or transmitting information. Examples of machine-readable media include electronic circuits, semiconductor memory devices, ROM, flash memory, erasable ROM (EROM), floppy disks, CD-ROMs, optical disks, hard disks, optical fiber media, radio frequency (RF) links, etc. The code segment can be downloaded via a computer network such as the Internet, an intranet, etc.

[0223] It should also be noted that the exemplary embodiments mentioned in this application describe some methods or systems based on a series of steps or devices. However, this application is not limited to the order of the above steps, that is, the steps can be performed in the order mentioned in the embodiment, or in a different order from the embodiment, or several steps can be performed simultaneously.

[0224] Aspects of the present disclosure are described above with reference to the flowchart and / or block diagram of the method, device (system) and computer program product according to the embodiment of the present disclosure. It should be understood that each box in the flowchart and / or block diagram and the combination of each box in the flowchart and / or block diagram can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device to produce a machine so that these instructions executed by the processor of the computer or other programmable data processing device enable the implementation of the function / action specified in one or more boxes of the flowchart and / or block diagram. Such a processor can be, but is not limited to, a general-purpose processor, a special-purpose processor, a special application processor, or a field programmable logic circuit. It can also be understood that each box in the block diagram and / or flowchart and the combination of boxes in the block diagram and / or flowchart can also be implemented by dedicated hardware that performs a specified function or action, or can be implemented by a combination of dedicated hardware and computer instructions.

[0225] The above is only a specific implementation of the present application. Those skilled in the art can clearly understand that for the convenience and simplicity of description, the specific working processes of the systems, modules and units described above can refer to the corresponding processes in the aforementioned method embodiments, and will not be repeated here. It should be understood that the protection scope of the present application is not limited to this. Any technician familiar with the technical field can easily think of various equivalent modifications or replacements within the technical scope disclosed in this application, and these modifications or replacements should be included in the protection scope of this application.

Claims

1. A file downloading method, characterized in that: The method comprises: receiving a file review request, the request including file types of multiple files and download links of each of the files; storing the file type of each file in the request and the download link of each file in the first download queue; According to the file type, sequentially move the download link of each file from the first download queue to a second download queue corresponding to the file type; According to the number of reference download threads of the audit platform and the audit volume ratio of each file type within a preset time period, a corresponding sliding window length is set for the second download queue corresponding to each file type; According to the length of each sliding window, the download links are sequentially obtained from the corresponding second download queue to download the file.

2. The method according to claim 1, characterized in that The step of setting a corresponding sliding window length for the second download queue corresponding to each file type according to the reference download thread number of the audit platform and the audit volume ratio of each file type within a preset time period includes: Multiply the number of reference download threads of the audit platform by a preset length coefficient to obtain the total window length; Determine the window splitting coefficient of each file type according to each audit amount ratio and ratio threshold; The total window length is multiplied by the window splitting coefficient of each file type to obtain the sliding window length of the second download queue corresponding to each file type.

3. The method according to claim 2, characterized in that The ratio threshold comprises a first threshold and a second threshold, wherein the second threshold is greater than the first threshold; Determining the window splitting coefficient of each file type according to each audit amount ratio and ratio threshold includes: Compare the maximum ratio among the audit amount ratios with the ratio threshold to obtain a first comparison result; When the first comparison result indicates that the maximum ratio is greater than the second threshold, setting the window splitting coefficient of the file type of the maximum ratio to the second threshold; The remaining splitting coefficients are allocated according to the relationship between the audit quantity ratios of each first file type to obtain the window splitting coefficients of each first file type, where the first file type is each file type except the file type with the largest ratio.

4. The method according to claim 3, characterized in that After comparing the maximum audit quantity ratio among the audit quantity ratios with the ratio threshold to obtain a first comparison result, the method further includes: When the first comparison result indicates that the maximum ratio is between the first threshold and the second threshold, the sum of the maximum ratio and the second largest ratio among the audit amount ratios is compared with the second threshold to obtain a second comparison result; When the second comparison result indicates that the sum is not less than the second threshold, allocating the second threshold according to the relationship between the maximum proportion and the second largest proportion to obtain a window splitting coefficient for the file type with the maximum proportion and a window splitting coefficient for the file type with the second largest proportion; The remaining splitting coefficients are allocated according to the relationship between the audit quantity proportions of each second file type to obtain the window splitting coefficients of each second file type, where the second file type is each file type except the file type with the largest proportion and the file type with the second largest proportion.

5. The method according to claim 4, characterized in that In the case where the first comparison result indicates that the maximum ratio is between the first threshold and the second threshold, comparing the sum of the maximum ratio and the second largest ratio among the audit amount ratios with the second threshold, and obtaining a second comparison result, the method further includes: If the second comparison result indicates that the sum is less than the second threshold, setting the window splitting coefficient of the file type with the largest proportion to the first threshold, and setting the window splitting coefficient of the file type with the second largest proportion to the difference between the second threshold and the first threshold; The remaining splitting coefficients are distributed according to the relationship between the audit quantity ratios of each second file type to obtain the window splitting coefficients of each second file type.

6. The method according to claim 3, characterized in that After comparing the maximum ratio of each of the audit amount ratios with the ratio threshold to obtain a first comparison result, the method further includes: When the first comparison result indicates that the maximum ratio is less than the first threshold value, the window splitting coefficient of each file type is set to the audit amount ratio corresponding to the file type.

7. The method according to claim 1, characterized in that The step of sequentially acquiring the download links from the corresponding second download queues according to the lengths of the sliding windows to download the files includes: The following steps are executed in a loop until all the unprocessed download links in the second download queue are traversed: According to the order relationship between the download links in the second download queue, starting from the first unprocessed download link, obtaining the download links corresponding to the length of the sliding window to download the file; When a new download link is added to the second download queue, the second download queue is updated.

8. The method according to any one of claims 1 to 7, characterized in that: After sequentially acquiring the download links from the corresponding second download queues according to the lengths of the sliding windows to download the files, the method further includes: Monitor each load type of the audit platform to obtain a business load score of the audit platform; Determining a target adjustment strategy for the audit platform according to a magnitude relationship between the business load score and the score threshold; According to the target adjustment strategy, the sliding window length of each of the second download queues is adjusted.

9. The method according to claim 8, characterized in that The monitoring of each load type of the audit platform to obtain the business load score of the audit platform includes: Monitor each load type of the audit platform to obtain the load ratio corresponding to each load type; Multiply each of the load proportions by the corresponding load coefficient to obtain a load sub-score corresponding to each of the load types; The load sub-scores corresponding to each load type are accumulated to obtain the business load score of the audit platform.

10. The method according to claim 9, characterized in that The load type includes at least one of a download thread number load type, an algorithm load type, a file download speed load type, a bandwidth load type, and a cache load type; The load ratio includes at least one of a download thread number load ratio, an algorithm load ratio, a file download speed load ratio, a bandwidth load ratio, and a cache load ratio; The load sub-score includes at least one of a download thread number load score, an algorithm load ratio score, a file download speed load score, a bandwidth load score, and a cache load score.

Images