File transmission method and device, computer equipment, storage medium and program product

Abstract

The invention relates to a file transmission method and device, computer equipment, a storage medium and a program product. The method comprises the steps of obtaining a first number of tasks in a task queue and the size of a first file corresponding to the tasks in response to a file transmission request, and obtaining a file to be uploaded according to the first number, the size of the first file, the size of a file to be uploaded corresponding to the file transmission request, a preset maximum concurrent file number and a maximum concurrent file size, and the file corresponding to the task in the task queue is transmitted, so that the number and the size of the large files uploaded at the same time can be limited through the maximum concurrent file number and the maximum concurrent file size, the occurrence probability of the UI blocking problem is reduced, and the efficiency and the reliability of large file transmission are improved.

Description

Technical Field

[0001] The present application relates to the technical field of large file transmission, and in particular to a file transmission method, device, computer equipment, storage medium and program product. Background Art

[0002] With the rapid development of the Internet and the sharp increase in data volume, large file transfer has become an indispensable function in many online services and applications. However, when facing huge files, the traditional single-threaded transmission method often has problems such as long transmission time and low efficiency, and cannot meet users' demand for high-speed transmission. Therefore, large file parallel transmission technology came into being.

[0003] At present, during the large file transmission process, it is usually necessary to verify the integrity of the large file. However, the integrity verification of large files usually takes a lot of time. When multiple large files are transmitted in parallel, it is easy to cause the user interface (UI) to be blocked, which in turn reduces the transmission efficiency of large files. Summary of the invention

[0004] Based on this, it is necessary to provide a file transmission method, device, computer equipment, storage medium and program product that can improve the transmission efficiency of large files in response to the above technical problems.

[0005] In a first aspect, the present application provides a file transmission method. The method comprises:

[0006] In response to a file transfer request, obtaining a first number of tasks in the task queue and a size of a first file corresponding to the task, wherein the file transfer request includes a size of a file to be uploaded;

[0007] The files corresponding to the tasks in the task queue are transmitted according to the first number, the size of the first file, the size of the file to be uploaded, the preset maximum number of concurrent files, and the maximum concurrent file size.

[0008] In one embodiment, transmitting the file corresponding to the task in the task queue according to the first number, the size of the first file, the size of the file to be uploaded, the preset maximum number of concurrent files, and the maximum concurrent file size includes:

[0009] Determining whether the file to be uploaded meets the transmission condition according to the first number, the size of the first file, the size of the file to be uploaded, the preset maximum number of concurrent files, and the maximum concurrent file size;

[0010] If the file to be uploaded meets the transmission condition, the task corresponding to the file transmission request is added to the end of the task queue, and the files corresponding to the tasks in the task queue are transmitted in sequence.

[0011] In one embodiment, judging whether the file to be uploaded meets the transmission condition according to the first number, the size of the first file, the size of the file to be uploaded, the preset maximum number of concurrent files, and the maximum concurrent file size includes:

[0012] Determine the sum of the size of the first file and the size of the file to be uploaded;

[0013] If the sum is not greater than the maximum concurrent file size, and the first number is less than the maximum concurrent file number, it is determined that the file to be uploaded meets the transmission condition;

[0014] If the sum result is greater than the maximum concurrent file size, and / or the first number is not less than the maximum concurrent file number, it is determined that the file to be uploaded does not meet the transmission condition.

[0015] In one embodiment, the method further comprises:

[0016] If the file to be uploaded does not meet the transmission condition, the task corresponding to the file transfer request is added to the waiting queue until the file to be uploaded meets the transmission condition, then the task corresponding to the file to be uploaded in the waiting queue is added to the task queue to transfer the file to be uploaded.

[0017] In one embodiment, the transmitting the file corresponding to the task in the task queue includes:

[0018] Get the task to be transferred from the task queue;

[0019] Splitting the file corresponding to the task to be transferred into multiple file segments;

[0020] When the number of shard transfer tasks in the shard request pool is less than a preset threshold, the shard transfer task corresponding to the file shard is added to the shard request pool, and the file shard corresponding to the shard transfer task in the shard request pool is transmitted; the preset threshold is determined according to the maximum concurrent number of the shard request pool.

[0021] In one embodiment, the method further comprises:

[0022] When the number of shard transfer tasks in the shard request pool is not less than the preset threshold, the file shards corresponding to the shard transfer tasks in the shard request pool are transmitted, until the number of shard transfer tasks in the shard request pool is less than the preset threshold, the shard transfer tasks corresponding to the file shards under the task to be transferred are added to the shard request pool.

[0023] In a second aspect, the present application also provides a file transmission device. The device includes:

[0024] an acquisition module, configured to acquire, in response to a file transfer request, a first number of tasks in the task queue and a size of a first file corresponding to the task, wherein the file transfer request includes a size of a file to be uploaded;

[0025] The transmission module is used to transmit the file corresponding to the task in the task queue according to the first number, the size of the first file, the size of the file to be uploaded, the preset maximum number of concurrent files and the maximum concurrent file size.

[0026] In a third aspect, the present application further provides a computer device, comprising a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of any of the above methods when executing the computer program.

[0027] In a fourth aspect, the present application further provides a computer-readable storage medium having a computer program stored thereon, which implements the steps of any of the above methods when executed by a processor.

[0028] In a fifth aspect, the present application also provides a computer program product, including a computer program, which implements the steps of any of the above methods when executed by a processor.

[0029] The above-mentioned file transfer method, apparatus, computer equipment, storage medium and program product obtain the first number of tasks in the task queue and the size of the first file corresponding to the task in response to a file transfer request, and transfer the file corresponding to the task in the task queue according to the first number, the size of the first file, the size of the file to be uploaded corresponding to the file transfer request, the preset maximum number of concurrent files and the maximum concurrent file size, so that the number and size of large files uploaded simultaneously can be limited by the maximum number of concurrent files and the maximum concurrent file size, thereby reducing the probability of UI blocking problems and improving the efficiency and reliability of large file transmission. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] FIG1 is an internal structure diagram of a computer device provided in an embodiment of the present application;

[0031] FIG2 is a schematic diagram of a flow chart of a file transmission method provided in an embodiment of the present application;

[0032] FIG3 is a flow chart of another file transmission method provided in an embodiment of the present application;

[0033] FIG4 is a flow chart of a transmission condition determination method provided in an embodiment of the present application;

[0034] FIG5 is a flow chart of a method for dynamically controlling concurrent file uploads provided in an embodiment of the present application;

[0035] FIG6 is a schematic diagram of a flow chart of a file segment transmission method provided in an embodiment of the present application;

[0036] 7 is a schematic diagram of a flow chart of a method for dynamic control of segmented upload provided in an embodiment of the present application;

[0037] FIG8 is a structural block diagram of a file transmission device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0038] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0039] With the rapid development of the Internet and the sharp increase in data volume, large file transfer has become an indispensable function in many online services and applications. However, when facing huge files, the traditional single-threaded transmission method often has problems such as long transmission time and low efficiency, and cannot meet users' demand for high-speed transmission. Therefore, large file parallel transmission technology came into being.

[0040] At present, during the large file transmission process, it is usually necessary to verify the integrity of the large file. However, the integrity verification of large files usually takes a lot of time. When multiple large files are transmitted in parallel, it is easy to cause the user interface (UI) to be blocked, which in turn reduces the transmission efficiency of large files.

[0041] The file transfer method provided in the embodiment of the present application can be applied in the application environment as shown in Figure 1. Figure 1 is an internal structure diagram of a computer device provided in the embodiment of the present application. The computer device can be a server, and its internal structure diagram can be as shown in Figure 1. The computer device includes a processor, a memory and a network interface connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The network interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a file transfer method is implemented.

[0042] Those skilled in the art will understand that the structure shown in FIG. 1 is merely a block diagram of a partial structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than shown in the figure, or combine certain components, or have a different arrangement of components.

[0043] In one embodiment, as shown in FIG. 2 , FIG. 2 is a flow chart of a file transmission method provided in an embodiment of the present application, and the method can be applied to the computer device in FIG. 1 , and the method includes the following steps:

[0044] S201, in response to a file transfer request, obtaining a first number of tasks in a task queue and a size of a first file corresponding to the task.

[0045] In one embodiment, the file to be uploaded selected by the user can be obtained through the browser. After detecting that the user has selected the file to be uploaded, a file transfer request can be generated according to the file to be uploaded selected by the user. In response to the file transfer request, the total number of all tasks in the current task queue (i.e., the first number) and the total size of the first files corresponding to all tasks can be obtained. The file transfer request includes the size of the file to be uploaded, and the task queue includes the task of the file being uploaded, which is the file transfer task.

[0046] S202: Transmit files corresponding to tasks in the task queue according to the first number, the size of the first file, the size of the file to be uploaded, the preset maximum number of concurrent files, and the maximum concurrent file size.

[0047] In one embodiment, the sum of the size of the first file and the size of the file to be uploaded can be determined, and then the sum is compared with the maximum concurrent file size, the first number and the maximum concurrent file number. If the sum is not greater than the maximum concurrent file size, and the first number is less than the maximum concurrent file number, the task corresponding to the file transfer request is added to the end of the task queue, and the files corresponding to the tasks in the task queue are transferred in sequence.

[0048] Alternatively, if the sum is greater than the maximum concurrent file size, the task corresponding to the file transfer request is added to the waiting queue until the file to be uploaded meets the transmission conditions, then the task corresponding to the file to be uploaded in the waiting queue is added to the task queue to transfer the file to be uploaded.

[0049] Alternatively, if the first number is not less than the maximum concurrent file number, the task corresponding to the file transfer request is added to the waiting queue until the file to be uploaded meets the transmission conditions, then the task corresponding to the file to be uploaded in the waiting queue is added to the task queue to transfer the file to be uploaded.

[0050] Alternatively, if the sum result is greater than the maximum concurrent file size and the first number is not less than the maximum concurrent file number, the task corresponding to the file transfer request is added to the waiting queue until the file to be uploaded meets the transmission conditions, then the task corresponding to the file to be uploaded in the waiting queue is added to the task queue to transfer the file to be uploaded.

[0051] Optionally, when transferring files corresponding to tasks in the task queue, the tasks to be transferred can be obtained from the task queue, and the files corresponding to the tasks to be transferred can be divided into multiple file fragments. When the number of fragment transfer tasks in the fragment request pool is less than a preset threshold, the fragment transfer tasks corresponding to the file fragments are added to the fragment request pool, and the file fragments corresponding to the fragment transfer tasks in the fragment request pool are transferred. When the number of fragment transfer tasks in the fragment request pool is not less than a preset threshold, the file fragments corresponding to the fragment transfer tasks in the fragment request pool are transferred, until the number of fragment transfer tasks in the fragment request pool is less than the preset threshold, the fragment transfer tasks corresponding to the file fragments under the tasks to be transferred are added to the fragment request pool.

[0052] Exemplarily, refer to Figure 5, which is a flow chart of a method for dynamic control of concurrent file uploads provided in an embodiment of the present application. File transfer can be performed by the method shown in Figure 5: define a function createMultiUploadFn for transferring files, which returns an asynchronous task corresponding to the upload task, and set the variable maxUploadNum representing the maximum number of concurrent files and the variable maxUploadSize representing the maximum concurrent file size in the function createMultiUploadFn. Then define the array variable unUploadPool of the waiting queue in the function createMultiUploadFn. The array variable unUploadPool is used to store the list of file tasks to be uploaded. Each file task is executed by the runFn function. The runFn function contains file information corresponding to each file task, such as file size, file content, etc. The corresponding file can be uploaded according to the file information corresponding to each file task. The initial value of the array variable unUploadPool is empty. Secondly, the array variable uploadingPool of the task queue can be defined in the function createMultiUploadFn. The array variable uploadingPool is used to store the file task information being uploaded. The initial value of the array variable uploadingPool is empty, and the variable uploadingFileSize representing the size of the first file is defined, and the initial value is 0. The function createMultiUploadFn returns the asynchronous upload method asyncMultiUpload. Any task that needs to perform file upload can be started by calling asyncMultiUpload.

[0053] Optionally, in the method asyncMultiUpload, you can first determine whether the first number (uploadingPool.length) of the first files being uploaded in the task queue is less than the maximum number of concurrent files (maxUploadNum), and whether the sum of the size of the first file (uploadingFileSize) and the size of the file to be uploaded is less than or equal to the maximum concurrent file size (maxUploadSize). If both of the above conditions are met, the task corresponding to the file transfer request is added to the task queue, the first number and the size of the first file (uploadingFileSize) are updated, and the file corresponding to the task in the task queue is transferred. If at least one of the above conditions is not met, the task corresponding to the file transfer request and the size of the file to be uploaded are stored in the waiting queue unUploadPool and returned directly.

[0054] In one embodiment, if the tasks in the task queue are completed, the task can be removed from the task queue uploadingPool, and the first number and the size of the first file (uploadingFileSize) are updated. Then a task is obtained from the waiting queue unUploadPool, added to the task queue uploadingPool, and the above file number and size determination steps are repeated until all tasks are completed.

[0055] In particular, when the sum of the size of the first file (uploadingFileSize) and the size of the file to be uploaded is greater than the maximum concurrent file size (maxUploadSize), if the first number of first files (uploadingPool.length) is 0, it means that there is no file being transferred, and the task corresponding to the file transfer request can be added to the task queue, the first number and the size of the first file (uploadingFileSize) are updated, and the file corresponding to the task in the task queue is transferred.

[0056] In an embodiment of the present application, by responding to a file transfer request, the first number of tasks in the task queue and the size of the first file corresponding to the task are obtained, and the file corresponding to the task in the task queue is transmitted according to the first number, the size of the first file, the size of the file to be uploaded corresponding to the file transfer request, the preset maximum number of concurrent files and the maximum concurrent file size. This allows the number and size of large files uploaded simultaneously to be limited by the maximum number of concurrent files and the maximum concurrent file size, thereby reducing the probability of UI blocking problems and improving the efficiency and reliability of large file transmission.

[0057] Referring to Figure 3, Figure 3 is a flow chart of another file transfer method provided by an embodiment of the present application. This implementation involves a possible implementation method of how to transfer files corresponding to tasks in a task queue according to a first quantity, a size of a first file, a size of a file to be uploaded, a preset maximum number of concurrent files, and a maximum concurrent file size. Based on the above embodiment, the above S202 includes the following steps:

[0058] S301, judging whether the file to be uploaded meets the transmission condition according to the first number, the size of the first file, the size of the file to be uploaded, the preset maximum number of concurrent files and the maximum concurrent file size.

[0059] In one embodiment, the sum of the size of the first file and the size of the file to be uploaded can be determined, and then the sum is compared with the maximum concurrent file size, and the first number is compared with the maximum concurrent file number. If the sum is not greater than the maximum concurrent file size, and the first number is less than the maximum concurrent file number, it is determined that the file to be uploaded meets the transmission condition.

[0060] Alternatively, if the sum result is greater than the maximum concurrent file size, it is determined that the file to be uploaded does not meet the transmission condition.

[0061] Alternatively, if the first number is not less than the maximum concurrent file number, it is determined that the file to be uploaded does not meet the transmission condition.

[0062] Alternatively, if the sum is greater than the maximum concurrent file size and the first number is not less than the maximum concurrent file number, it is determined that the file to be uploaded does not meet the transmission condition.

[0063] S302: If the file to be uploaded meets the transmission condition, the task corresponding to the file transmission request is added to the end of the task queue, and the files corresponding to the tasks in the task queue are transmitted in sequence.

[0064] Optionally, if the file to be uploaded meets the transmission conditions, the task corresponding to the file transfer request is added to the end of the task queue, and then the task to be transferred is obtained from the task queue, and the file corresponding to the task to be transferred is divided into multiple file fragments. When the number of fragment transfer tasks in the fragment request pool is less than a preset threshold, the fragment transfer task corresponding to the file fragment is added to the fragment request pool, and the file fragment corresponding to the fragment transfer task in the fragment request pool is transferred. When the number of fragment transfer tasks in the fragment request pool is not less than the preset threshold, the file fragment corresponding to the fragment transfer task in the fragment request pool is transferred, until the number of fragment transfer tasks in the fragment request pool is less than the preset threshold, the fragment transfer task corresponding to the file fragment under the task to be transferred is added to the fragment request pool.

[0065] In an embodiment of the present application, whether the file to be uploaded meets the transmission conditions is determined based on the first quantity, the size of the first file, the size of the file to be uploaded, the preset maximum number of concurrent files and the maximum concurrent file size. If the file to be uploaded meets the transmission conditions, the task corresponding to the file transfer request is added to the end of the task queue, and the files corresponding to the tasks in the task queue are transmitted in sequence. In this way, the number and size of large files uploaded at the same time can be limited by the maximum number of concurrent files and the maximum concurrent file size, thereby reducing the probability of UI blocking problems and improving the efficiency and reliability of large file transmission.

[0066] Referring to Figure 4, Figure 4 is a flow chart of a transmission condition determination method provided by an embodiment of the present application. This embodiment involves a possible implementation method of how to determine whether the file to be uploaded meets the transmission condition based on the first number, the size of the first file, the size of the file to be uploaded, the preset maximum number of concurrent files, and the maximum concurrent file size. Based on the above embodiment, the above S301 includes the following steps:

[0067] S401, determining a sum of the size of a first file and the size of a file to be uploaded.

[0068] Exemplarily, the total size of the first file being uploaded in the task queue may be obtained, and the size of the file to be uploaded may be obtained from the file transfer request, and then the total size of the first file and the size of the file to be uploaded may be added to obtain a sum result.

[0069] S402: If the sum is not greater than the maximum concurrent file size and the first number is less than the maximum concurrent file number, it is determined that the file to be uploaded meets the transmission condition.

[0070] Exemplarily, the sum result is compared with the maximum concurrent file size, and the first number is compared with the maximum concurrent file number. If the sum result is not greater than the maximum concurrent file size and the first number is less than the maximum concurrent file number, it is determined that the file to be uploaded meets the transmission conditions.

[0071] That is, the file to be uploaded meets the transmission condition only when the file to be uploaded meets the following two conditions at the same time: 1. The sum result is not greater than the maximum concurrent file size. 2. The first number is less than the maximum concurrent file number.

[0072] S403: If the sum is greater than the maximum concurrent file size and / or the first number is not less than the maximum concurrent file number, it is determined that the file to be uploaded does not meet the transmission condition.

[0073] In one embodiment, if the sum is greater than the maximum concurrent file size, it is determined that the file to be uploaded does not meet the transmission condition.

[0074] Alternatively, if the first number is not less than the maximum concurrent file number, it is determined that the file to be uploaded does not meet the transmission condition.

[0075] Alternatively, if the sum is greater than the maximum concurrent file size and the first number is not less than the maximum concurrent file number, it is determined that the file to be uploaded does not meet the transmission condition.

[0076] That is, as long as the file to be uploaded does not meet any of the following two conditions, it is determined that the file to be uploaded does not meet the transmission condition: 1. The sum result is not greater than the maximum concurrent file size. 2. The first number is less than the maximum concurrent file number.

[0077] In an embodiment of the present application, the sum of the size of the first file and the size of the file to be uploaded is determined. If the sum is not greater than the maximum concurrent file size and the first number is less than the maximum concurrent file number, it is determined that the file to be uploaded meets the transmission condition; if the sum is greater than the maximum concurrent file size, and / or the first number is not less than the maximum concurrent file number, it is determined that the file to be uploaded does not meet the transmission condition, thereby enabling the number and size of large files to be uploaded simultaneously to be limited by the maximum concurrent file number and the maximum concurrent file size. Only when the file to be uploaded meets the restrictions of the maximum concurrent file number and the maximum concurrent file size, the file to be uploaded is allowed to be transmitted, thereby reducing the probability of UI blocking problems and improving the efficiency and reliability of large file transmission.

[0078] Based on the above embodiment, the method further includes the following steps:

[0079] If the file to be uploaded does not meet the transmission conditions, the task corresponding to the file transfer request is added to the waiting queue until the file to be uploaded meets the transmission conditions, then the task corresponding to the file to be uploaded in the waiting queue is added to the task queue to transfer the file to be uploaded.

[0080] Exemplarily, if the file to be uploaded does not meet the transmission condition, the task corresponding to the file transfer request and the size of the file to be uploaded can be stored in the unUploadPool array. If the task in the task queue is completed, the task can be removed from the task queue uploadingPool, and the first number and the size of the first file (uploadingFileSize) are updated. Then a task is obtained from the waiting queue unUploadPool, and the above steps of determining the number and size of files are repeated until all tasks are completed.

[0081] In an embodiment of the present application, if the file to be uploaded does not meet the transmission conditions, the task corresponding to the file transfer request is added to the waiting queue until the file to be uploaded meets the transmission conditions, then the task corresponding to the file to be uploaded in the waiting queue is added to the task queue to transmit the file to be uploaded, thereby limiting the number and size of large files uploaded simultaneously by the maximum number of concurrent files and the maximum concurrent file size, and dynamically controlling file upload by removing and adding tasks in the task queue, thereby reducing the probability of UI blocking problems and improving the efficiency and reliability of large file transmission.

[0082] Referring to Figure 6, Figure 6 is a flow chart of a file segment transmission method provided by an embodiment of the present application. This implementation relates to a possible implementation of how to transmit files corresponding to tasks in a task queue. Based on the above embodiment, the files corresponding to tasks in the above transmission task queue can be implemented in the following manner:

[0083] S601, obtaining a task to be transmitted from a task queue.

[0084] Exemplarily, a fileInfo object of the task to be transferred may be created to store relevant information during the execution of the task to be transferred. A totalChunks array is defined in the fileInfo object of the task to be transferred, and totalChunks is initialized to empty to store subsequent fragmentation results.

[0085] In one embodiment, the hash value of the file corresponding to the task to be transferred can be calculated based on the Message Digest Algorithm 5 (MD5), and the hash value can be used as the integrity identifier of the file. Then, the hash value of the file corresponding to the task to be transferred can be added to the fileInfo object as the file-md5 attribute.

[0086] S602: Split the file corresponding to the task to be transmitted into multiple file segments.

[0087] Optionally, the file corresponding to the task to be transferred may be segmented according to a preset size to obtain a plurality of file segments of preset sizes.

[0088] In one embodiment, each file segment can be numbered, and the hash value of each file segment can be calculated based on MD5, which is used as the integrity identifier of the file segment. The hash value and number uploadId of each file segment are added to the totalChunks array. Create an object uploadCenter of type Map to store uploadId and file information. Among them, the file information includes the task file to be transferred, the list of file segment information totalChunks, the list of uploaded file segment information uploadedChunks, and the file upload progress percent.

[0089] Exemplarily, after the transmission of the file segments corresponding to the task to be transmitted is completed, the integrity identifiers of each file segment and the integrity identifier of the file corresponding to the task to be transmitted can be uploaded, and then integrity verification and file merging can be performed based on the integrity identifiers of each file segment and the integrity identifier of the file corresponding to the task to be transmitted.

[0090] S603, when the number of shard transmission tasks in the shard request pool is less than a preset threshold, add the shard transmission tasks corresponding to the file shards to the shard request pool, and transmit the file shards corresponding to the shard transmission tasks in the shard request pool.

[0091] The preset threshold is determined according to the maximum concurrent number of the shard request pool.

[0092] Optionally, the maximum concurrent number of the shard request pool can be obtained, and then a preset threshold is determined based on the maximum concurrent number of the shard request pool, so that the preset threshold is smaller than the maximum concurrent number of the shard request pool.

[0093] Exemplarily, assuming that the maximum concurrent number of the shard request pool is 6, the preset threshold can be set to 4, thereby limiting the number of file shards uploaded simultaneously, and setting redundancy to improve the reliability of file shard upload.

[0094] Exemplarily, refer to FIG. 7, which is a flow chart of a method for dynamic control of segmented upload provided by an embodiment of the present application. File segments can be transferred by the method shown in FIG. 7: set the function createRequestPoolFn for uploading file segments, define the array requestPool representing the segment request pool, and use it to store the segment transfer tasks being uploaded. And set three parameters: limit, paramsItems, and promiseFn. Among them, limit is a preset threshold, the paramsItems array is a segment information queue for storing file segment information, and promiseFn is a function that sends a segment upload request and returns a promise type.

[0095] In one embodiment, the file segments corresponding to the tasks to be transferred can be stored in the segment information queue paramsItems. When uploading file segments, the segment request pool requestPool can read an upload request for a file segment from the segment information queue paramsItems, and generate a segment transfer task corresponding to the file segment in response to the upload request for the file segment. Then, it can be determined whether the number of segment transfer tasks in the current segment request pool requestPool is less than the preset threshold limit. If the number of segment transfer tasks in the current segment request pool requestPool is less than the preset threshold limit, the segment transfer task is added to the segment request pool requestPool, and then the file segment corresponding to the segment transfer task in the segment request pool is transferred.

[0096] Optionally, a clean function may be defined. After a shard transmission task in the transmission shard request pool is completed, the clean function may be used to remove the completed shard transmission task from the transmission shard request pool.

[0097] In an embodiment of the present application, a task to be transmitted is obtained from a task queue, and the file corresponding to the task to be transmitted is divided into multiple file fragments. When the number of fragment transmission tasks in a fragment request pool is less than a preset threshold, the fragment transmission task corresponding to the file fragment is added to the fragment request pool, and the file fragment corresponding to the fragment transmission task in the fragment request pool is transmitted, thereby limiting the number of file fragments uploaded simultaneously by a preset threshold, and removing and adding fragment transmission tasks in the fragment request pool to achieve dynamic regulation of file fragment upload, further reducing the probability of UI blocking problems, and improving the efficiency and reliability of large file transmission.

[0098] Based on the above embodiment, the method further includes the following steps:

[0099] When the number of shard transfer tasks in the shard request pool is not less than the preset threshold, the file shards corresponding to the shard transfer tasks in the shard request pool are transmitted. When the number of shard transfer tasks in the shard request pool is less than the preset threshold, the shard transfer tasks corresponding to the file shards under the tasks to be transferred are added to the shard request pool.

[0100] Exemplarily, as shown in FIG7 , it is possible to determine whether the number of shard transfer tasks in the current shard request pool requestPool is less than a preset threshold value limit. If the number of shard transfer tasks in the current shard request pool requestPool is not less than the preset threshold value limit, then await and Promise.race can be used to wait for the file shard transfer corresponding to the shard transfer task in the shard request pool until any shard transfer task in the shard request pool is completed, and the shard transfer task corresponding to the file shard under the task to be transferred is added to the shard request pool, and the above steps are repeated until all file shards in the shard information queue paramsItems and the shard request pool are transferred.

[0101] In an embodiment of the present application, when the number of shard transmission tasks in the shard request pool is not less than a preset threshold, the file shards corresponding to the shard transmission tasks in the shard request pool are transmitted, until the number of shard transmission tasks in the shard request pool is less than the preset threshold, the shard transmission tasks corresponding to the file shards under the tasks to be transmitted are added to the shard request pool, thereby being able to limit the number of file shards uploaded simultaneously by the preset threshold, and by removing and adding shard transmission tasks in the shard request pool, dynamic regulation of file shard upload is achieved, further reducing the probability of UI blocking problems and improving the efficiency and reliability of large file transmission.

[0102] It should be understood that, although the various steps in the flowcharts involved in the above-mentioned embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps does not have a strict order restriction, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above-mentioned embodiments can include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.

[0103] Based on the same inventive concept, the embodiment of the present application also provides a file transmission device for implementing the file transmission method involved above. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme recorded in the above method, so the specific limitations in the one or more file transmission device embodiments provided below can refer to the limitations on the file transmission method above, and will not be repeated here.

[0104] In one embodiment, as shown in FIG8 , FIG8 is a structural block diagram of a file transmission device provided in an embodiment of the present application, and the device 800 includes:

[0105] The acquisition module 801 is used to acquire a first number of tasks in the task queue and a size of a first file corresponding to the task in response to a file transfer request, wherein the file transfer request includes a size of a file to be uploaded.

[0106] The transmission module 802 is used to transmit the file corresponding to the task in the task queue according to the first number, the size of the first file, the size of the file to be uploaded, the preset maximum number of concurrent files and the maximum concurrent file size.

[0107] In one embodiment, the transmission module 802 includes:

[0108] The judging unit is used to judge whether the file to be uploaded meets the transmission condition according to the first number, the size of the first file, the size of the file to be uploaded, the preset maximum number of concurrent files and the maximum concurrent file size.

[0109] The file transmission unit is used to add the task corresponding to the file transmission request to the end of the task queue if the file to be uploaded meets the transmission condition, and transmit the files corresponding to the tasks in the task queue in sequence.

[0110] In one embodiment, the determining unit includes:

[0111] The first determining subunit is used to determine the sum of the size of the first file and the size of the file to be uploaded.

[0112] The second determining subunit is used to determine that the file to be uploaded meets the transmission condition if the sum result is not greater than the maximum concurrent file size and the first number is less than the maximum concurrent file number.

[0113] The third determining subunit is used to determine that the file to be uploaded does not meet the transmission condition if the sum result is greater than the maximum concurrent file size and / or the first number is not less than the maximum concurrent file number.

[0114] In one embodiment, the apparatus 800 further includes:

[0115] The first waiting module is used to add the task corresponding to the file transfer request to the waiting queue if the file to be uploaded does not meet the transmission conditions, until the file to be uploaded meets the transmission conditions, then add the task corresponding to the file to be uploaded in the waiting queue to the task queue to transmit the file to be uploaded.

[0116] In one embodiment, the transmission module 802 includes:

[0117] The acquisition unit is used to acquire the tasks to be transmitted from the task queue.

[0118] The segmentation unit is used to segment the file corresponding to the task to be transmitted into multiple file segments.

[0119] The shard transfer unit is used to add the shard transfer tasks corresponding to the file shards to the shard request pool and transfer the file shards corresponding to the shard transfer tasks in the shard request pool when the number of shard transfer tasks in the shard request pool is less than a preset threshold; the preset threshold is determined according to the maximum concurrent number of the shard request pool.

[0120] In one embodiment, the apparatus 800 further includes:

[0121] The second waiting module is used to transmit the file slices corresponding to the slice transmission tasks in the slice request pool when the number of slice transmission tasks in the slice request pool is not less than the preset threshold, until the number of slice transmission tasks in the slice request pool is less than the preset threshold, and then add the slice transmission tasks corresponding to the file slices under the tasks to be transmitted to the slice request pool.

[0122] Each module in the above-mentioned file transmission device can be implemented in whole or in part by software, hardware or a combination thereof. Each module can be embedded in or independent of a processor in a computer device in the form of hardware, or can be stored in a memory in a computer device in the form of software, so that the processor can call and execute operations corresponding to each module above.

[0123] In one embodiment, a computer device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and when the processor executes the computer program, the following steps are implemented:

[0124] In response to a file transfer request, obtaining a first number of tasks in the task queue and a size of a first file corresponding to the task, the file transfer request including a size of a file to be uploaded;

[0125] The files corresponding to the tasks in the task queue are transmitted according to the first quantity, the size of the first file, the size of the file to be uploaded, the preset maximum number of concurrent files, and the maximum concurrent file size.

[0126] In one embodiment, when the processor executes the computer program, the processor further implements the following steps:

[0127] Determining whether the file to be uploaded meets the transmission condition according to the first number, the size of the first file, the size of the file to be uploaded, the preset maximum number of concurrent files, and the maximum concurrent file size;

[0128] If the file to be uploaded meets the transmission conditions, the task corresponding to the file transfer request is added to the end of the task queue, and the files corresponding to the tasks in the task queue are transmitted in sequence.

[0129] In one embodiment, when the processor executes the computer program, the processor further implements the following steps:

[0130] Determine the sum of the size of the first file and the size of the file to be uploaded;

[0131] If the sum is not greater than the maximum concurrent file size, and the first number is less than the maximum concurrent file number, it is determined that the file to be uploaded meets the transmission condition;

[0132] If the sum result is greater than the maximum concurrent file size, and / or the first number is not less than the maximum concurrent file number, it is determined that the file to be uploaded does not meet the transmission condition.

[0133] In one embodiment, when the processor executes the computer program, the processor further implements the following steps:

[0134] If the file to be uploaded does not meet the transmission conditions, the task corresponding to the file transfer request is added to the waiting queue until the file to be uploaded meets the transmission conditions, then the task corresponding to the file to be uploaded in the waiting queue is added to the task queue to transfer the file to be uploaded.

[0135] In one embodiment, when the processor executes the computer program, the processor further implements the following steps:

[0136] Get the tasks to be transferred from the task queue;

[0137] Split the file corresponding to the task to be transferred into multiple file segments;

[0138] When the number of shard transfer tasks in the shard request pool is less than a preset threshold, the shard transfer tasks corresponding to the file shards are added to the shard request pool, and the file shards corresponding to the shard transfer tasks in the shard request pool are transmitted; the preset threshold is determined according to the maximum concurrent number of the shard request pool.

[0139] In one embodiment, when the processor executes the computer program, the processor further implements the following steps:

[0140] When the number of shard transfer tasks in the shard request pool is not less than the preset threshold, the file shards corresponding to the shard transfer tasks in the shard request pool are transmitted. When the number of shard transfer tasks in the shard request pool is less than the preset threshold, the shard transfer tasks corresponding to the file shards under the tasks to be transferred are added to the shard request pool.

[0141] In one embodiment, a computer readable storage medium is provided, on which a computer program is stored, and when the computer program is executed by a processor, the following steps are implemented:

[0142] In response to a file transfer request, obtaining a first number of tasks in the task queue and a size of a first file corresponding to the task, the file transfer request including a size of a file to be uploaded;

[0143] The files corresponding to the tasks in the task queue are transmitted according to the first quantity, the size of the first file, the size of the file to be uploaded, the preset maximum number of concurrent files, and the maximum concurrent file size.

[0144] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0145] Determining whether the file to be uploaded meets the transmission condition according to the first number, the size of the first file, the size of the file to be uploaded, the preset maximum number of concurrent files, and the maximum concurrent file size;

[0146] If the file to be uploaded meets the transmission conditions, the task corresponding to the file transfer request is added to the end of the task queue, and the files corresponding to the tasks in the task queue are transmitted in sequence.

[0147] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0148] Determine the sum of the size of the first file and the size of the file to be uploaded;

[0149] If the sum is not greater than the maximum concurrent file size, and the first number is less than the maximum concurrent file number, it is determined that the file to be uploaded meets the transmission condition;

[0150] If the sum result is greater than the maximum concurrent file size, and / or the first number is not less than the maximum concurrent file number, it is determined that the file to be uploaded does not meet the transmission condition.

[0151] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0152] If the file to be uploaded does not meet the transmission conditions, the task corresponding to the file transfer request is added to the waiting queue until the file to be uploaded meets the transmission conditions, then the task corresponding to the file to be uploaded in the waiting queue is added to the task queue to transfer the file to be uploaded.

[0153] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0154] Get the tasks to be transferred from the task queue;

[0155] Split the file corresponding to the task to be transferred into multiple file segments;

[0156] When the number of shard transfer tasks in the shard request pool is less than a preset threshold, the shard transfer tasks corresponding to the file shards are added to the shard request pool, and the file shards corresponding to the shard transfer tasks in the shard request pool are transmitted; the preset threshold is determined according to the maximum concurrent number of the shard request pool.

[0157] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0158] When the number of shard transfer tasks in the shard request pool is not less than the preset threshold, the file shards corresponding to the shard transfer tasks in the shard request pool are transmitted. When the number of shard transfer tasks in the shard request pool is less than the preset threshold, the shard transfer tasks corresponding to the file shards under the tasks to be transferred are added to the shard request pool.

[0159] In one embodiment, a computer program product is provided, comprising a computer program, which, when executed by a processor, implements the following steps:

[0160] In response to a file transfer request, obtaining a first number of tasks in the task queue and a size of a first file corresponding to the task, the file transfer request including a size of a file to be uploaded;

[0161] The files corresponding to the tasks in the task queue are transmitted according to the first quantity, the size of the first file, the size of the file to be uploaded, the preset maximum number of concurrent files, and the maximum concurrent file size.

[0162] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0163] Determining whether the file to be uploaded meets the transmission condition according to the first number, the size of the first file, the size of the file to be uploaded, the preset maximum number of concurrent files, and the maximum concurrent file size;

[0164] If the file to be uploaded meets the transmission conditions, the task corresponding to the file transfer request is added to the end of the task queue, and the files corresponding to the tasks in the task queue are transmitted in sequence.

[0165] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0166] Determine the sum of the size of the first file and the size of the file to be uploaded;

[0167] If the sum is not greater than the maximum concurrent file size, and the first number is less than the maximum concurrent file number, it is determined that the file to be uploaded meets the transmission condition;

[0168] If the sum result is greater than the maximum concurrent file size, and / or the first number is not less than the maximum concurrent file number, it is determined that the file to be uploaded does not meet the transmission condition.

[0169] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0170] If the file to be uploaded does not meet the transmission conditions, the task corresponding to the file transfer request is added to the waiting queue until the file to be uploaded meets the transmission conditions, then the task corresponding to the file to be uploaded in the waiting queue is added to the task queue to transfer the file to be uploaded.

[0171] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0172] Get the tasks to be transferred from the task queue;

[0173] Split the file corresponding to the task to be transferred into multiple file segments;

[0174] When the number of shard transfer tasks in the shard request pool is less than a preset threshold, the shard transfer tasks corresponding to the file shards are added to the shard request pool, and the file shards corresponding to the shard transfer tasks in the shard request pool are transmitted; the preset threshold is determined according to the maximum concurrent number of the shard request pool.

[0175] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0176] When the number of shard transfer tasks in the shard request pool is not less than the preset threshold, the file shards corresponding to the shard transfer tasks in the shard request pool are transmitted. When the number of shard transfer tasks in the shard request pool is less than the preset threshold, the shard transfer tasks corresponding to the file shards under the tasks to be transferred are added to the shard request pool.

[0177] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., but are not limited to this.

[0178] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0179] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. A file transmission method, characterized in that: The method comprises: In response to a file transfer request, obtaining a first number of tasks in the task queue and a size of a first file corresponding to the task, wherein the file transfer request includes a size of a file to be uploaded; The files corresponding to the tasks in the task queue are transmitted according to the first number, the size of the first file, the size of the file to be uploaded, the preset maximum number of concurrent files, and the maximum concurrent file size.

2. The method according to claim 1, characterized in that The transmitting the file corresponding to the task in the task queue according to the first number, the size of the first file, the size of the file to be uploaded, the preset maximum number of concurrent files, and the maximum concurrent file size includes: Determining whether the file to be uploaded meets the transmission condition according to the first number, the size of the first file, the size of the file to be uploaded, the preset maximum number of concurrent files, and the maximum concurrent file size; If the file to be uploaded meets the transmission condition, the task corresponding to the file transmission request is added to the end of the task queue, and the files corresponding to the tasks in the task queue are transmitted in sequence.

3. The method according to claim 2, characterized in that The determining whether the file to be uploaded meets the transmission condition according to the first number, the size of the first file, the size of the file to be uploaded, a preset maximum number of concurrent files, and a maximum concurrent file size includes: Determine the sum of the size of the first file and the size of the file to be uploaded; If the sum is not greater than the maximum concurrent file size, and the first number is less than the maximum concurrent file number, it is determined that the file to be uploaded meets the transmission condition; If the sum result is greater than the maximum concurrent file size, and / or the first number is not less than the maximum concurrent file number, it is determined that the file to be uploaded does not meet the transmission condition.

4. The method according to claim 2 or 3, characterized in that: The method further comprises: If the file to be uploaded does not meet the transmission condition, the task corresponding to the file transfer request is added to the waiting queue, until the file to be uploaded meets the transmission condition, the task corresponding to the file to be uploaded in the waiting queue is added to the task queue to transmit the file to be uploaded.

5. The method according to any one of claims 1 to 3, characterized in that: The transmitting the file corresponding to the task in the task queue includes: Obtaining the task to be transmitted from the task queue; Splitting the file corresponding to the task to be transferred into multiple file segments; When the number of shard transfer tasks in the shard request pool is less than a preset threshold, the shard transfer task corresponding to the file shard is added to the shard request pool, and the file shard corresponding to the shard transfer task in the shard request pool is transmitted; the preset threshold is determined according to the maximum concurrent number of the shard request pool.

6. The method according to claim 5, characterized in that The method further comprises: When the number of shard transfer tasks in the shard request pool is not less than the preset threshold, the file shards corresponding to the shard transfer tasks in the shard request pool are transmitted, until the number of shard transfer tasks in the shard request pool is less than the preset threshold, and then the shard transfer tasks corresponding to the file shards under the tasks to be transferred are added to the shard request pool.

7. A file transmission device, characterized in that: The device comprises: an acquisition module, configured to acquire, in response to a file transfer request, a first number of tasks in the task queue and a size of a first file corresponding to the task, wherein the file transfer request includes a size of a file to be uploaded; A transmission module is used to transmit the files corresponding to the tasks in the task queue according to the first number, the size of the first file, the size of the file to be uploaded, the preset maximum number of concurrent files and the maximum concurrent file size.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 6 are implemented.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

Images