File transmission method, device, electronic device and storage medium

Abstract

The present application provides a file transmission method, device, electronic device and storage medium, the method comprising: when the file to be transmitted is a first file that needs to be uploaded in segments, the first file is sequentially segmented based on a coroutine method, and N file objects corresponding to the N file segments are obtained, the file objects include first identification information and segment objects, N is an integer greater than or equal to 2; according to the first identification information corresponding to the N file objects, respectively, M segment objects to be uploaded are determined in the N segment objects, M is greater than or equal to 1 and less than or equal to N; for the M segment objects, upload tasks are sequentially registered in a coroutine manner, and the M segment objects are uploaded to the receiving device in a parallel upload manner. The present application can reduce the pressure on the front end based on the coroutine segment processing, ensure segment retransmission or continuation based on the identification information, optimize the transmission method, and improve the transmission efficiency.

Description

Technical Field

[0001] The present application relates to the field of communication technology, and in particular to a file transmission method, device, electronic device and storage medium. Background Art

[0002] In the JavaScript scenario, when uploading files to the server, it is usually necessary to encode and verify the entire file. For small files, the encoding speed is acceptable, but for large files, the encoding speed is significantly slower, affecting the transmission efficiency.

[0003] Currently, a few JavaScript solutions use js's Blob and FormData objects to implement the multi-part upload function. However, when using js's Blob and FormData objects for transmission, if the transmission is interrupted during the transmission process, it needs to be re-uploaded, and the upload speed is very slow.

[0004] It can be seen that in the prior art, when uploading files in a JavaScript scenario, if the file is uploaded as a whole, the transmission efficiency will be affected due to the limited encoding speed; if the file is uploaded in pieces, the entire file needs to be re-uploaded when the transmission is interrupted and the transmission speed is limited, which also affects the transmission efficiency. Summary of the invention

[0005] The embodiments of the present application provide a file transmission method, device, electronic device and storage medium to solve the problem of low transmission efficiency in file transmission in the prior art.

[0006] In a first aspect, an embodiment of the present application provides a file transmission method, which is applied to a client device, including:

[0007] In the case where the file to be transmitted is a first file that needs to be uploaded in segments, the first file is sequentially segmented based on the coroutine method to obtain N file objects corresponding to the N file segments, where the file objects include first identification information and segment objects, and N is an integer greater than or equal to 2;

[0008] According to the first identification information respectively corresponding to the N file objects, determine M fragment objects to be uploaded from the N fragment objects, where M is an integer greater than or equal to 1 and less than or equal to N;

[0009] For the M shard objects, upload tasks are registered in sequence in a coroutine manner, and the M shard objects are uploaded to the receiving end device in a parallel upload manner.

[0010] In a second aspect, an embodiment of the present application provides a file transmission device, which is applied to a client device, including:

[0011] A fragment acquisition module is used to, when the file to be transmitted is a first file that needs to be uploaded in fragments, sequentially fragment the first file based on a coroutine method, and obtain N file objects corresponding to the N file fragments, wherein the file objects include first identification information and fragment objects, and N is an integer greater than or equal to 2;

[0012] A determination module, configured to determine M fragment objects to be uploaded from the N fragment objects according to the first identification information corresponding to the N file objects, where M is an integer greater than or equal to 1 and less than or equal to N;

[0013] The processing module is used to register upload tasks in sequence in a coroutine manner for the M shard objects, and upload the M shard objects to the receiving end device in a parallel upload manner.

[0014] In a third aspect, an embodiment of the present application provides an electronic device, comprising a processor, a memory, and a computer program stored in the memory and executable on the processor, wherein the computer program, when executed by the processor, implements the steps of the file transfer method described in the first aspect above.

[0015] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the file transfer method described in the first aspect above are implemented.

[0016] The technical solution of the embodiment of the present application, by sequentially segmenting the first file in a coroutine manner, obtains N file objects corresponding to the N file segments, including first identification information and segment objects, which can reduce the pressure on the front end based on the coroutine segmentation processing, and can ensure segment retransmission or continuation based on the first identification information, thereby optimizing the transmission method; after obtaining the first identification information corresponding to the N file objects respectively, determine the M segment objects to be uploaded in the N segment objects according to the N first identification information, register the upload tasks for the M segment objects in sequence in a coroutine manner, and upload the M segment objects to the receiving device in a parallel upload manner, thereby improving the transmission efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 A schematic diagram showing a file transmission method provided in an embodiment of the present application;

[0018] Figure 2 A schematic diagram showing the interaction between a client device and a receiving device provided in an embodiment of the present application;

[0019] Figure 3 A flowchart showing the implementation of performing file pre-checking and executing corresponding operations according to the pre-checking results provided by an embodiment of the present application;

[0020] Figure 4 A schematic diagram showing a file transmission device provided in an embodiment of the present application;

[0021] Figure 5 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application is shown. DETAILED DESCRIPTION

[0022] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0023] It should be understood that the references to "one embodiment" or "an embodiment" throughout the specification mean that the specific features, structures, or characteristics associated with the embodiment are included in at least one embodiment of the present application. Therefore, the references to "in one embodiment" or "in an embodiment" appearing throughout the specification do not necessarily refer to the same embodiment. In addition, these specific features, structures, or characteristics may be combined in any suitable manner in one or more embodiments.

[0024] In the various embodiments of the present application, it should be understood that the size of the serial numbers of the following processes does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0025] In view of the problems in the prior art of uploading files in JavaScript scenarios, such as the overall upload affecting the transmission efficiency due to the limited encoding speed, and the problem that the overall file needs to be re-uploaded in case of transmission interruption when uploading fragments, which affects the transmission efficiency, the present application provides a file transmission method, which implements file coroutine fragment processing and fragment parallel uploading through a JavaScript Generator mechanism, thereby improving the transmission speed, and by setting a fragment identifier, it is possible to ensure fragment retransmission or continuation based on the fragment identifier, thereby avoiding re-uploading of the entire file, thereby optimizing the transmission method and improving the transmission efficiency.

[0026] The following is an introduction to the file transmission method provided in the embodiment of the present application. Figure 1 As shown, the method includes:

[0027] Step 101: When the file to be transmitted is a first file that needs to be uploaded in segments, segment the first file in sequence based on a coroutine method to obtain N file objects corresponding to the N file segments, where the file objects include first identification information and segment objects, and N is an integer greater than or equal to 2.

[0028] The file transfer method provided in the embodiment of the present application is an application scenario in which the browser or application of the client device uploads files to the receiving device under the JavaScript scenario. JavaScript, as a directly interpreted scripting language, needs to be run in a browser or application environment, and the client device is a device installed with a browser or application. The client device first determines whether the file to be transferred is the first file that needs to be uploaded in pieces or the second file that does not need to be uploaded in pieces, which can be determined specifically according to the file size. In the case where it is determined that the file to be transferred is the first file that needs to be uploaded in pieces, the first file is processed in sequence based on the coroutine method. When the fragmentation processing is successful, at least two file objects corresponding to the first file are obtained. Among them, the coroutine can be considered as a user space thread, and the operating system knows nothing about it. The developer needs to design the scheduling in the thread to perform collaborative multitasking. js does not have direct multi-threading capabilities in the browser, and uses coroutine scheduling for thread reuse, which indirectly has multi-threading capabilities.

[0029] When the first file is processed in sequence by fragmentation based on the coroutine method, the first file is processed in sequence by fragmentation according to the first preset order, where the first preset order is the arrangement order (cutting order) of the N file fragments corresponding to the first file. By sequentially fragmenting the first file, a file object including the first identification information and the fragment object corresponding to each file fragment can be obtained. The present application is based on sequential processing in a coroutine manner, which can reduce the pressure on the front end compared to the method of processing the entire file. Among them, the first identification information is used to distinguish different file fragments, each file fragment corresponds to a file object, and the fragment object included in the file object is the content generated based on the file fragment for transmission to the receiving device. By setting the corresponding first identification information for the file fragment, the retransmission or continuation of the fragment object can be guaranteed, avoiding re-uploading of the entire file, and optimizing the transmission method.

[0030] Step 102: Determine M fragment objects to be uploaded from the N fragment objects according to the first identification information respectively corresponding to the N file objects, where M is an integer greater than or equal to 1 and less than or equal to N.

[0031] After obtaining the first identification information corresponding to the N file objects, respectively, the M fragment objects to be uploaded can be determined from the N fragment objects corresponding to the N file objects according to the N first identification information. The value of M is greater than or equal to 1 and less than or equal to N. When the value of M is less than N, it is determined that some of the N fragment objects corresponding to the N file fragments need to be uploaded. When the value of M is equal to N, it is determined that all of the N fragment objects corresponding to the N file fragments need to be uploaded.

[0032] Step 103: For the M slice objects, register upload tasks in sequence in a coroutine manner, and upload the M slice objects to the receiving device in a parallel upload manner.

[0033] After determining the M shard objects to be uploaded from the N shard objects, for the M shard objects, the upload tasks are registered on the system in sequence in a coroutine manner according to a second preset order, and after completing the registration of the upload task, the M shard objects are uploaded to the receiving device in a parallel upload manner. The transmission efficiency can be improved by registering the upload tasks and transmitting in parallel through the coroutine.

[0034] In the above implementation process of the present application, the first file is processed in sequence in a coroutine manner to obtain N file objects corresponding to the N file slices, including the first identification information and the slice objects. The pressure on the front end can be reduced based on the coroutine slice processing, and the retransmission or continuation of the slices can be guaranteed based on the first identification information, thereby optimizing the transmission method. After obtaining the first identification information corresponding to the N file objects, the M slice objects to be uploaded are determined in the N slice objects according to the N first identification information. For the M slice objects, the upload tasks are registered in sequence in a coroutine manner, and the M slice objects are uploaded to the receiving device in a parallel upload manner, thereby improving the transmission efficiency.

[0035] The following is an introduction to the process of obtaining N file objects corresponding to the first file. In the case where the file to be transmitted is the first file that needs to be uploaded in segments, the first file is segmented in sequence based on the coroutine method to obtain N file objects corresponding to the N file segments, including: when the number of segments corresponding to the file to be transmitted is greater than a preset threshold, determining that the file to be transmitted is the first file; constructing a segment queue for the first file based on the coroutine method, and segmenting the first file in sequence according to the segment queue; when the segment processing is successful, obtaining N file objects corresponding to the first file, and the N file objects are stored in the segment queue.

[0036] For the file to be transmitted, the number of shards is determined according to the file size of the file to be transmitted, specifically, the number of shards is determined based on the ratio of the file size of the file to be transmitted and the preset file shard size (fixed value). After determining the number of shards, the number of shards is compared with a preset threshold value, where the preset threshold value can be the single maximum transmission amount of the shard object. If the number of shards is greater than the preset threshold value, that is, the transmission of the shard objects corresponding to all file shards cannot be completed at a single time, then the file to be transmitted is determined to be the first file, otherwise, the file to be transmitted is determined to be the second file. After determining that the file to be transmitted is the first file, for the first file, a shard queue corresponding to the first file is constructed based on the coroutine method. At this time, the queue is an empty queue. Based on the constructed shard queue, the first file is sequentially sharded, and when the sharding processing is successful, the N file objects corresponding to the first file are obtained, and the N file objects are stored in the shard queue.

[0037] The above implementation process of the present application can determine whether the file to be transmitted is the first file or the second file based on the relationship between the number of fragments corresponding to the file to be transmitted and the preset threshold, so as to realize the simple and fast determination of the file category according to the number of fragments; when it is determined that the file to be transmitted is the first file, the first file is sequentially fragmented using a coroutine method, so that the file can be uploaded in the form of fragments.

[0038] The following is an introduction to the process of sequentially slicing the first file according to the slicing queue. When sequentially slicing the first file according to the slicing queue, it includes: sequentially cutting the first file according to the slicing queue; obtaining a file slicing for each cutting, determining the first identification information corresponding to the file slicing according to the file slicing, and generating the slicing object corresponding to the file slicing according to the file slicing and the associated information of the file slicing.

[0039] After constructing a shard queue for the first file based on the coroutine method, the first file is cut in sequence according to the shard queue. For each cut, a file shard can be obtained. Then, for the file shard, the first identification information corresponding to the file shard is determined according to the shard content corresponding to the file shard. A shard object corresponding to the file shard is generated according to the file shard and the associated information of the file shard. Then, a file object including the first identification information and the shard object is generated. After obtaining the file object, the file object is stored in the shard queue.

[0040] This application realizes the switching of code execution control rights through the characteristic of the generator automatically maintaining its own state. The code execution control right here can be understood as the control right of generating file objects. In the initial state, the generator function does not execute any code, and an iterable object is calculated based on the large file (the first file). The iterable object here is the file fragment. It is necessary to set the iterable object to be iterated only once, so as to realize the sequential calculation of the file objects of the fragmented files.

[0041] The yield keyword of the generator function can hand over the execution right of the function, and then the js engine will execute the statements following this function. Using the yield and next() methods, you can continuously hand over and restore the execution right of the function, thereby realizing a non-blocking calculation file object.

[0042] It should be noted that the acquisition order corresponding to the N file objects obtained through the sharding process may be the same as or different from the arrangement order corresponding to the N file shards, that is, the acquisition order of the file objects may be different from the cutting order of the file shards corresponding to the file objects. Among them, the first identification information is the SHA value obtained by calculating the file shard using the secure hash algorithm SHA; the shard object is generated based on the file shard, the offset of the file shard, the shard start position of the file shard and the second identification information, the second identification information is determined based on the first identification information and the transmission identification, and the transmission identification is used for upload authentication.

[0043] When determining the first identification information, this embodiment can use a secure hash algorithm (SHA) to calculate the slice content of the file slice, obtain the corresponding SHA value, and determine the corresponding SHA value as the first identification information. Since the slice contents of different file slices are different, the uniqueness of the first identification information corresponding to the file slice can be guaranteed by calculating the first identification information based on the slice content.

[0044] The SHA algorithm in this embodiment may be a SHA1 algorithm or other algorithms in the SHA series. For example, if the SHA algorithm is a SHA1 algorithm, the SHA1 algorithm may be used to calculate the fragment content of the file fragment to obtain a SHA1 value, and the obtained SHA1 value may be determined as the first identification information of the file fragment.

[0045] When generating a shard object, it is generated based on the file shard and the associated information of the file shard, and the associated information of the file shard includes at least the offset of the file shard, the shard start position of the file shard, and the second identification information, wherein the second identification information is determined based on the first identification information and the transmission identification, and the transmission identification is an identification used for upload authentication, which is obtained when the user logs in to the client. By obtaining the transmission identification, the permission authentication for file upload can be performed to ensure that the file can be uploaded normally. By determining the second identification information based on the first identification information and the transmission identification, the second identification information can be integrated with the upload authentication function and the shard object identification function.

[0046] In the above implementation process of the present application, by sequentially cutting the first file based on the shard queue, sharding can be performed through the coroutine processing method, and N file objects including the first identification information and the shard objects are obtained in sequence; by determining the first identification information based on the shard content, the uniqueness of the first identification information can be guaranteed.

[0047] In one embodiment of the present application, after the shard object is uploaded to the receiving device and parsed by the receiving device, the receiving device obtains the file shard, the offset of the file shard, the shard start position of the file shard, and the second identification information;

[0048] The second identification information is used for upload authentication and identification of the shard object, and the receiving end device performs shard splicing according to the file shards corresponding to the M shard objects, the offsets of the file shards, and the shard starting positions of the file shards.

[0049] After the client device uploads the M slice objects to the receiving device, the receiving device parses the M slice objects, and obtains the file slice, the offset of the file slice, the slice start position of the file slice, and the second identification information for each slice object. The second identification information integrates the upload authentication function and the slice object identification function. The integrated upload authentication function can ensure that the slice object is uploaded to the receiving device, and the integrated slice object identification function can distinguish different slice objects.

[0050] After obtaining the file fragments, the offsets of the file fragments, and the fragment start positions of the file fragments corresponding to the M fragment objects, the receiving end device can perform fragment splicing. When performing fragment splicing, the M file fragments are spliced ​​based on the offsets of the file fragments and the fragment start positions, or the M file fragments are spliced ​​with other file fragments that belong to the same file as the M file fragments and are stored in the receiving end device. By performing fragment splicing by the receiving end device, it is possible to obtain a complete file on the receiving end device side.

[0051] In an embodiment of the present application, after uploading the M slice objects to the receiving end device in a parallel uploading manner, the method further includes:

[0052] When it is determined according to the second identification information corresponding to the shard object that at least one target shard object has failed to be uploaded, the target shard object is retransmitted or the target shard object is retransmitted at a breakpoint.

[0053] Since each shard object corresponds to a second identification information, when uploading M shard objects to the receiving device in parallel, when at least one target shard object fails to be uploaded, at least one target shard object can be determined according to the second identification information. At this time, it is necessary to determine the transmission status of at least one target shard object. If at least one target shard object has only uploaded a part of the content due to network disconnection, the breakpoint can be resumed; if at least one target shard object needs to be retransmitted (such as the need for retransmission due to the failure of uploading when the client device is disconnected from the network, or the need for retransmission due to the failure of receiving when the receiving device is abnormal), then re-upload at least one target shard object. And in the case of multiple target shard objects, the transmission can be performed according to the order of failure when retransmitting or resuming the transmission, that is, the transmission can be performed based on the generated retransmission task queue.

[0054] By setting a second identification information for each fragment object, when a transmission failure occurs, the retransmission efficiency can be improved by resuming the transmission from the breakpoint or retransmitting the erroneous fragment object based on the second identification information.

[0055] The following is an introduction to the process of determining M shard objects to be uploaded from N shard objects. When determining the M shard objects to be uploaded from N shard objects based on the first identification information corresponding to the N file objects, it includes: uploading the N first identification information corresponding to the N file objects to the receiving end device; determining the M shard objects to be uploaded from the N shard objects based on the first comparison result fed back by the receiving end device that does not include the file shards corresponding to the M first identification information; wherein the first comparison result is determined by the receiving end device comparing the N first identification information with the shard identification information corresponding to the file shards stored in the receiving end device.

[0056] After obtaining the first identification information corresponding to the N file objects respectively, the N first identification information can be uploaded to the receiving device, and the receiving device compares the N first identification information with the fragment identification information corresponding to the file fragment stored in the receiving device. When the N first identification information all match the fragment identification information corresponding to the file fragment stored in the receiving device, it is determined that the receiving device has stored N file fragments corresponding to the N first identification information, and the receiving device can feedback a prompt that the file has been uploaded to the client device and feedback the file address. When the N first identification information and the fragment identification information corresponding to the file fragment stored in the receiving device do not match, the client device receives the prompt feedback from the receiving device and uploads the fragment object corresponding to the file fragment to the receiving device. When part of the N first identification information matches the fragment identification information corresponding to the file fragment stored in the receiving device, the client device receives the prompt feedback from the receiving device and uploads the fragment object corresponding to the corresponding file fragment to the receiving device.

[0057] Wherein, when the N first identification information do not match the fragment identification information corresponding to the file fragment stored in the receiving device, or when part of the N first identification information matches the fragment identification information corresponding to the file fragment stored in the receiving device, the client device receives the first comparison result fed back by the receiving device that does not include the file fragment corresponding to the M first identification information, and the value of M is less than or equal to N. When the N first identification information do not match the fragment identification information corresponding to the file fragment stored in the receiving device, the value of M is equal to N; when part of the N first identification information matches the fragment identification information corresponding to the file fragment stored in the receiving device, the value of M is less than N. After obtaining the first comparison result, the client device determines the M fragment objects to be uploaded from the N fragment objects based on the first comparison result. For the receiving device, it can store the file fragment, the first identification information of the file fragment, the offset of the file fragment and the starting position of the fragment.

[0058] The following is an introduction to the above process through a specific implementation process. Figure 2As shown, when the client device determines that the file to be transmitted is the first file based on the number of fragments corresponding to the file to be transmitted, the N first identification information of the N file fragments corresponding to the first file are uploaded to the receiving device. The receiving device detects whether the file fragments corresponding to the N first identification information have been stored based on the N first identification information. If the file fragments corresponding to the N first identification information are not stored, the first prompt information is fed back, and the client device uploads the fragment objects corresponding to the N file fragments based on the first prompt information. If the file fragments corresponding to some of the first identification information in the N first identification information are stored, the second prompt information is fed back, and the client device determines the file fragments not stored in the receiving device based on the second prompt information, and uploads the fragment objects corresponding to the corresponding file fragments to the receiving device. If the file fragments corresponding to the N first identification information are stored, the third prompt information and the file address are fed back, and the client device receives the third prompt information and the file address. The file address can be carried in the third prompt information or sent independently.

[0059] In the above implementation process of the present application, by uploading N first identification information to the receiving device, the receiving device detects whether the corresponding file segment is stored based on the first identification information, and thus duplicate uploading can be avoided through pre-check, thereby saving uploading time and storage space; by sending N first identification information at one time, the number of requests sent can be reduced.

[0060] The following is an introduction to the process of uploading M shard objects to a receiving device in a parallel upload manner. When uploading the M shard objects to the receiving device in a parallel upload manner, the process includes: in a coroutine manner, according to the arrangement order corresponding to the M shard objects, sequentially registering upload tasks for the M shard objects on the system; and uploading the M shard objects to the receiving device in a parallel manner.

[0061] When the client device uploads M slice objects to the receiving device, the upload task is registered on the system in sequence in a coroutine manner according to the arrangement order corresponding to the M slice objects, wherein the arrangement order corresponding to the M slice objects can be the acquisition order of the M slice objects. By registering the upload tasks in sequence in a coroutine manner, it is possible to establish an upload task on the system based on the coroutine processing method. After completing the registration of the upload task, the M slice objects are uploaded to the receiving device in a parallel manner, and the transmission efficiency can be improved by parallel uploading. When uploading M slice objects in a parallel manner, it is necessary to consider the transmission capacity and the number of slice objects. If the value of M is 100, and the number of slice objects transmitted by the browser at a time is at most 10, then the upload of the M slice objects is completed through 10 parallel uploads, and each time the upload is performed, 10 slice objects are uploaded simultaneously, so as to upload the slice objects in a parallel upload manner; if the value of M is less than 10, the upload of the slice objects is completed through one parallel upload.

[0062] In an embodiment of the present application, when the file to be transmitted is a second file that does not need to be uploaded in segments, the method further includes:

[0063] Obtaining second file identification information corresponding to the second file and uploading it to the receiving end device;

[0064] Upon receiving a second comparison result fed back by the receiving end device that does not include the second file corresponding to the second file identification information, uploading the second file to the receiving end device;

[0065] Upon receiving the third comparison result of the second file corresponding to the second file identification information fed back by the receiving end device, acquiring the file address of the second file;

[0066] Among them, when the number of fragments corresponding to the file to be transmitted is less than or equal to a preset threshold, the file to be transmitted is determined to be the second file; the second comparison result or the third comparison result is determined by the receiving end device by comparing the second file identification information corresponding to the second file with the file identification information corresponding to the file stored in the receiving end device.

[0067] When the number of fragments corresponding to the file to be transmitted is less than or equal to the preset threshold, it is determined that the file to be transmitted is a second file that does not need to be uploaded in fragments. At this time, the second file identification information corresponding to the second file is obtained and uploaded to the receiving end device, and the receiving end device detects whether the receiving end device stores the second file based on the second file identification information corresponding to the second file. Wherein, when the receiving end device determines that the second file identification information corresponding to the second file matches the file identification information corresponding to a file stored in the receiving end device, it indicates that the receiving end device stores the second file, and the receiving end device feeds back the third comparison result, and the receiving end device can feed back the file address of the second file while feeding back the third comparison result, and the client device receives the third comparison result and the file address of the second file. When the receiving end device determines that the second file identification information corresponding to the second file does not match the file identification information corresponding to the file stored in the receiving end device, the receiving end device feeds back the second comparison result, the client device receives the second comparison result, and uploads the second file based on the second comparison result.

[0068] Among them, the second file identification information corresponding to the second file is obtained by calculating the file content using the SHA algorithm. When uploading the second file, there is no need to segment the file, and the entire file can be uploaded directly. When uploading the second file, an upload object corresponding to the second file can be generated. The upload object can be generated based on the second file, the second file identification information corresponding to the second file, and the transmission identifier. Specifically, the target file identification information can be generated according to the second file identification information corresponding to the second file and the transmission identifier, and the upload object can be generated based on the target file identification information and the second file.

[0069] In the above implementation process of the present application, when it is determined that the file to be transmitted is the second file based on the number of fragments, the second file identification information corresponding to the second file is determined based on the file content of the second file and uploaded to the receiving device. The receiving device detects whether the second file is stored based on the second file identification information corresponding to the second file, and feeds back a comparison result based on the detection result. The client device uploads the file or obtains the file address based on the comparison result. The second file can be uploaded in a normal upload manner, or the uploading of the second file can be stopped based on the file address to avoid repeated uploading through pre-check.

[0070] The following is an introduction to the implementation process of distinguishing file types, pre-checking files, and performing corresponding operations based on the pre-check results. Figure 3 As shown, the following steps are included:

[0071] Step 301, the client device obtains the number of fragments corresponding to the file to be transmitted. When the number of fragments is greater than a preset threshold, it is determined that the file to be transmitted is a first file, and steps 302 to 305 are executed; when the number of fragments is less than or equal to the preset threshold, it is determined that the file to be transmitted is a second file, and steps 306 to 309 are executed.

[0072] Step 302: The client device obtains N first identification information corresponding to N file segments of the first file.

[0073] Step 303: The client device uploads N first identification information to the receiving device.

[0074] Step 304: The receiving end device determines, based on the N first identification information, a first comparison result that does not include file segments corresponding to the M first identification information, or the receiving end device determines, based on the N first identification information, a fourth comparison result that includes file segments corresponding to the N first identification information.

[0075] Step 305: The client device receives the first comparison result or the fourth comparison result fed back by the receiving device, determines the segment objects corresponding to the M file segments to be uploaded according to the first comparison result, and uploads the M segment objects in parallel, or determines that the first file has been uploaded according to the fourth comparison result, and obtains the file address of the first file.

[0076] Step 306: The client device obtains second file identification information of the second file.

[0077] Step 307: The client device uploads the second file identification information of the second file to the receiving device.

[0078] Step 308: The receiving end device determines that the receiving end device does not include the second comparison result of the second file according to the second file identification information of the second file, or determines that the receiving end device includes the third comparison result of the second file according to the second file identification information of the second file.

[0079] Step 309: The client device receives the second comparison result or the third comparison result fed back by the receiving device, determines that the second file needs to be uploaded according to the second comparison result, and uploads the second file in a normal manner, or determines that the second file has been uploaded according to the third comparison result, and obtains the file address of the second file.

[0080] In the above implementation process, a pre-check is required before the file is uploaded. If the pre-check result is that the file has been uploaded, the client device obtains a notification that the file has been uploaded and can obtain the file address. If the pre-check result is that the second file has not been uploaded, the normal upload process is followed. If the pre-check result is that at least part of the first file has not been uploaded, the segmented upload process is followed. By performing a pre-check, repeated uploading can be avoided, thereby saving upload time and storage space.

[0081] The above is the overall implementation process of the file transfer method provided in the embodiment of the present application. By sequentially segmenting the first file in a coroutine manner, N file objects including first identification information and segment objects corresponding to the N file segments are obtained. The pressure on the front end can be reduced based on the coroutine segment processing, and segment retransmission or continuation can be guaranteed based on the first identification information, thereby optimizing the transmission method. After obtaining the first identification information corresponding to the N file objects, the M segment objects to be uploaded are determined in the N segment objects according to the N first identification information. For the M segment objects, the upload tasks are registered in sequence in a coroutine manner, and the M segment objects are uploaded to the receiving device in a parallel upload manner, thereby improving the transmission efficiency.

[0082] Furthermore, the file category can be determined simply and quickly based on the number of fragments. When it is determined that the file to be transmitted is the first file, the first file is fragmented, so that the file can be uploaded in the form of fragments. By determining the first identification information based on the fragment content, the uniqueness of the first identification information can be guaranteed. By determining the file identification information based on the file content, the uniqueness of the file identification information can be guaranteed.

[0083] By performing a pre-check, repeated uploading can be avoided, thereby saving uploading time and storage space; by sending N first identification information at one time, the number of request sending can be reduced.

[0084] The present application also provides a file transmission device, which is applied to a client device, such as Figure 4 As shown, the device comprises:

[0085] The fragment acquisition module 401 is used to, when the file to be transmitted is a first file that needs to be uploaded in fragments, sequentially fragment the first file based on the coroutine method, and obtain N file objects corresponding to the N file fragments, wherein the file objects include first identification information and fragment objects, and N is an integer greater than or equal to 2;

[0086] A determination module 402 is used to determine M fragment objects to be uploaded from the N fragment objects according to the first identification information respectively corresponding to the N file objects, where M is an integer greater than or equal to 1 and less than or equal to N;

[0087] The processing module 403 is used to register upload tasks in sequence in a coroutine manner for the M slice objects, and upload the M slice objects to the receiving end device in a parallel upload manner.

[0088] Optionally, the fragment acquisition module includes:

[0089] A first determination submodule, configured to determine that the file to be transmitted is the first file when the number of fragments corresponding to the file to be transmitted is greater than a preset threshold;

[0090] A construction submodule, used to construct a shard queue for the first file based on a coroutine method;

[0091] A processing submodule, configured to sequentially perform fragment processing on the first file according to the fragment queue;

[0092] The acquisition submodule is used to acquire N file objects corresponding to the first file when the sharding process is successful, and the N file objects are stored in the sharding queue.

[0093] Optionally, the processing submodule is further used for:

[0094] According to the fragmentation queue, the first file is sequentially cut;

[0095] A file segment is obtained for each cutting, first identification information corresponding to the file segment is determined according to the file segment, and a segment object corresponding to the file segment is generated according to the file segment and association information of the file segment.

[0096] Optionally, the first identification information is a SHA value obtained by calculating the file segment using a secure hash algorithm SHA;

[0097] The shard object is generated based on the file shard, the offset of the file shard, the shard start position of the file shard and second identification information, the second identification information is determined based on the first identification information and a transmission identifier, and the transmission identifier is used for upload authentication.

[0098] Optionally, after the shard object is uploaded to the receiving end device and parsed by the receiving end device, the receiving end device obtains the file shard, the offset of the file shard, the shard start position of the file shard, and the second identification information;

[0099] The second identification information is used for upload authentication and identification of the shard object, and the receiving end device performs shard splicing according to the file shards corresponding to the M shard objects, the offsets of the file shards, and the shard starting positions of the file shards.

[0100] Optionally, the device further comprises:

[0101] A transmission module is used to retransmit the target shard object or resume the target shard object after the processing module uploads the M shard objects to the receiving end device in a parallel upload manner, if it is determined according to the second identification information corresponding to the shard object that at least one target shard object has failed to be uploaded.

[0102] Optionally, the determining module includes:

[0103] A first uploading submodule, used to upload N first identification information corresponding to the N file objects to the receiving end device;

[0104] A second determination submodule, configured to determine the M fragment objects to be uploaded from the N fragment objects according to the first comparison result fed back by the receiving end device that does not include the file fragments corresponding to the M first identification information;

[0105] The first comparison result is determined by the receiving end device comparing the N first identification information with the fragment identification information corresponding to the file fragments stored in the receiving end device.

[0106] Optionally, the processing module includes:

[0107] A registration submodule, used to register upload tasks for the M shard objects in sequence on the system in a coroutine manner according to the arrangement order corresponding to the M shard objects;

[0108] The second uploading submodule is used to upload the M slice objects to the receiving end device in parallel.

[0109] Optionally, when the file to be transmitted is a second file that does not need to be uploaded in pieces, the device further includes:

[0110] An acquisition upload module, used to acquire second file identification information corresponding to the second file and upload the second file identification information to the receiving end device;

[0111] an uploading module, configured to upload the second file to the receiving end device upon receiving a second comparison result fed back by the receiving end device that does not include the second file corresponding to the second file identification information;

[0112] an acquisition module, configured to acquire a file address of the second file upon receiving a third comparison result of the second file corresponding to the second file identification information fed back by the receiving end device;

[0113] Among them, when the number of fragments corresponding to the file to be transmitted is less than or equal to a preset threshold, the file to be transmitted is determined to be the second file; the second comparison result or the third comparison result is determined by the receiving end device by comparing the second file identification information corresponding to the second file with the file identification information corresponding to the file stored in the receiving end device.

[0114] As for the device embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0115] An embodiment of the present application also provides an electronic device, including: a processor, a memory, and a computer program stored in the memory and executable on the processor. When the computer program is executed by the processor, the various processes of the above-mentioned file transfer method embodiment are implemented, and the same technical effect can be achieved. To avoid repetition, it will not be described here.

[0116] For example, Figure 5 FIG. 1 shows a schematic diagram of the physical structure of an electronic device. Figure 5 As shown, the electronic device may include: a processor 510, a communication interface 520, a memory 530 and a communication bus 540, wherein the processor 510, the communication interface 520 and the memory 530 communicate with each other through the communication bus 540. The processor 510 may call the logic instructions in the memory 530, and the processor 510 is used to perform the following steps: in the case where the file to be transmitted is a first file to be uploaded in pieces, the first file is sequentially processed in a coroutine manner, and N file objects corresponding to N file pieces are obtained, the file object includes first identification information and a piece object, N is an integer greater than or equal to 2; according to the first identification information corresponding to the N file objects, respectively, M piece objects to be uploaded are determined in the N piece objects, M is an integer greater than or equal to 1 and less than or equal to N; for the M piece objects, the upload tasks are sequentially registered in a coroutine manner, and the M piece objects are uploaded to the receiving end device in a parallel upload manner. The processor 510 may also execute other solutions in the embodiments of the present application, which will not be further elaborated here.

[0117] In addition, the logic instructions in the above-mentioned memory 530 can be implemented in the form of software functional units and can be stored in a computer-readable storage medium when sold or used as an independent product. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product, which is stored in a storage medium and includes a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present application.

[0118] The embodiment of the present application also provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, each process of the above-mentioned file transfer method embodiment is implemented, and the same technical effect can be achieved. To avoid repetition, it is not repeated here. The computer-readable storage medium is, for example, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0119] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the existence of other identical elements in the process, method, article or device including the element.

[0120] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, a magnetic disk, or an optical disk), and includes a number of instructions for a terminal (which can be a mobile phone, a computer, a server, an air conditioner, or a network device, etc.) to execute the methods described in each embodiment of the present application.

[0121] The embodiments of the present application are described above in conjunction with the accompanying drawings, but the present application is not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are merely illustrative and not restrictive. Under the guidance of the present application, ordinary technicians in this field can also make many forms without departing from the purpose of the present application and the scope of protection of the claims, all of which are within the protection of the present application.

[0122] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed in the present application can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0123] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0124] In the embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0125] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0126] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0127] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, server, or network device, etc.) to perform all or part of the steps of the methods described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as USB flash drives, mobile hard drives, ROM, RAM, magnetic disks, or optical disks.

[0128] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A file transmission method, applied to a client device, characterized in that: include: In the case where the file to be transmitted is a first file that needs to be uploaded in segments, the first file is sequentially segmented based on the coroutine method to obtain N file objects corresponding to the N file segments, where the file objects include first identification information and segment objects, and N is an integer greater than or equal to 2; According to the first identification information respectively corresponding to the N file objects, determine M fragment objects to be uploaded from the N fragment objects, where M is an integer greater than or equal to 1 and less than or equal to N; For the M shard objects, register upload tasks in sequence in a coroutine manner, and upload the M shard objects to the receiving end device in a parallel upload manner; The first file is a file to be transmitted whose number of fragments is greater than a preset threshold, the number of fragments is determined based on the ratio of the file size of the file to be transmitted to a preset file fragment size, the preset threshold is the single maximum transmission amount of the fragment object, and the first identification information is a SHA value obtained by calculating the file fragment using a secure hash algorithm SHA; The shard object is generated based on the file shard, the offset of the file shard, the shard starting position of the file shard and the second identification information, the second identification information is determined based on the first identification information and the transmission identification, the transmission identification is used for upload authentication, and the second identification information is used for upload authentication and identification of the shard object.

2. The method according to claim 1, characterized in that In the case where the file to be transmitted is a first file that needs to be uploaded in segments, sequentially segmenting the first file based on a coroutine method to obtain N file objects corresponding to the N file segments includes: When the number of fragments corresponding to the file to be transmitted is greater than a preset threshold, determining that the file to be transmitted is the first file; Building a shard queue for the first file based on a coroutine method; According to the fragmentation queue, the first file is fragmented in sequence; When the sharding process is successful, N file objects corresponding to the first file are obtained, and the N file objects are stored in the sharding queue.

3. The method according to claim 2, characterized in that The step of sequentially processing the first file into slices according to the slice queue includes: According to the fragmentation queue, the first file is sequentially cut; A file segment is obtained for each cutting, first identification information corresponding to the file segment is determined according to the file segment, and a segment object corresponding to the file segment is generated according to the file segment and association information of the file segment.

4. The method according to claim 1, characterized in that After the slice object is uploaded to the receiving end device and parsed by the receiving end device, the receiving end device obtains the file slice, the offset of the file slice, the slice start position of the file slice, and the second identification information; The receiving end device performs fragment splicing according to the file fragments corresponding to the M fragment objects, the offsets of the file fragments, and the fragment starting positions of the file fragments.

5. The method according to claim 1, characterized in that After uploading the M slice objects to the receiving end device in a parallel uploading manner, the method further includes: When it is determined according to the second identification information corresponding to the shard object that at least one target shard object has failed to be uploaded, the target shard object is retransmitted or the target shard object is retransmitted at a breakpoint.

6. The method according to claim 1, characterized in that The determining, according to the first identification information respectively corresponding to the N file objects, M fragment objects to be uploaded from the N fragment objects includes: Uploading N first identification information corresponding to the N file objects to the receiving end device; Determining the M fragment objects to be uploaded from the N fragment objects according to the first comparison result fed back by the receiving end device that does not include the file fragments corresponding to the M first identification information; The first comparison result is determined by the receiving end device comparing the N first identification information with the fragment identification information corresponding to the file fragments stored in the receiving end device.

7. The method according to claim 1, characterized in that The method of registering upload tasks in sequence in a coroutine manner for the M slice objects and uploading the M slice objects to a receiving end device in a parallel upload manner includes: In a coroutine manner, according to the arrangement order corresponding to the M shard objects, the upload tasks are sequentially registered on the system for the M shard objects; The M slice objects are uploaded to the receiving device in parallel.

8. The method according to claim 1, characterized in that In the case where the file to be transmitted is a second file that does not need to be uploaded in segments, the method further includes: Obtaining second file identification information corresponding to the second file and uploading it to the receiving end device; Upon receiving a second comparison result fed back by the receiving end device that does not include the second file corresponding to the second file identification information, uploading the second file to the receiving end device; Upon receiving the third comparison result of the second file corresponding to the second file identification information fed back by the receiving end device, acquiring the file address of the second file; Among them, when the number of fragments corresponding to the file to be transmitted is less than or equal to a preset threshold, the file to be transmitted is determined to be the second file; the second comparison result or the third comparison result is determined by the receiving end device by comparing the second file identification information corresponding to the second file with the file identification information corresponding to the file stored in the receiving end device.

9. A file transmission device, applied to a client device, characterized in that: include: A fragment acquisition module is used to, when the file to be transmitted is a first file that needs to be uploaded in fragments, sequentially fragment the first file based on a coroutine method, and obtain N file objects corresponding to the N file fragments, wherein the file objects include first identification information and fragment objects, and N is an integer greater than or equal to 2; A determination module, configured to determine M fragment objects to be uploaded from the N fragment objects according to the first identification information respectively corresponding to the N file objects, where M is an integer greater than or equal to 1 and less than or equal to N; A processing module, configured to sequentially register upload tasks in a coroutine manner for the M shard objects, and upload the M shard objects to a receiving end device in a parallel upload manner; The first file is a file to be transmitted whose number of fragments is greater than a preset threshold, the number of fragments is determined based on the ratio of the file size of the file to be transmitted to a preset file fragment size, the preset threshold is the single maximum transmission amount of the fragment object, and the first identification information is a SHA value obtained by calculating the file fragment using a secure hash algorithm SHA; The shard object is generated based on the file shard, the offset of the file shard, the shard starting position of the file shard and the second identification information, the second identification information is determined based on the first identification information and the transmission identification, the transmission identification is used for upload authentication, and the second identification information is used for upload authentication and identification of the shard object.

10. An electronic device, characterized in that: The method comprises a processor, a memory and a computer program stored in the memory and executable on the processor, wherein the computer program implements the steps in the file transmission method according to any one of claims 1 to 8 when executed by the processor.

11. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps in the file transmission method according to any one of claims 1 to 8 are implemented.

Images