Front-end large file uploading method

Abstract

The invention relates to a front-end large file uploading method, and belongs to the technical field of digital information transmission. According to the method, a global uploading component is realized on the basis of a vue-simple-loader plug-in and a spark-md5. Js library, a state of calculating MD5 is added, the md5 value is calculated in a fragmentation manner, and an uploading page is optimized. According to the global uploading assembly, the user experience is optimized, the user can monitor the uploading progress in real time, the uploading progress comprises the file uploading state and the completion percentage, and the control of the user on the uploading process is enhanced; the method is easy to integrate with an existing Vue project, is convenient to develop and maintain, and has good universality and compatibility.

Description

Technical Field

[0001] The invention belongs to the technical field of digital information transmission, and in particular relates to a front-end large file uploading method. Background Art

[0002] With the rapid development of Internet technology, users' demand for file upload is increasing, especially in scenarios involving large amounts of data or large files, such as high-definition videos, large documents, image datasets, etc. However, traditional front-end file upload methods face many challenges when processing large files.

[0003] In early network applications, files were uploaded based on the simple HTTP protocol. However, due to the limitations of the HTTP protocol itself, such as the request body size limit, unstable network connections, and browser timeouts caused by long uploads, large file uploads often fail or are extremely inefficient.

[0004] The traditional upload method usually transfers the entire file as a whole. For large files, this means that the entire file needs to be loaded into the memory at once, which not only places high demands on the client's hardware resources (such as memory, CPU, etc.), but also easily causes page freezes or even crashes. Moreover, once a network interruption or other abnormal situation occurs during the upload process, the entire upload process needs to start from the beginning, which greatly wastes the user's time and energy.

[0005] In addition, since it takes a long time to upload large files, users often cannot understand the progress and status of the upload in real time, which also brings a bad experience to users.

[0006] In order to overcome these problems, the industry has conducted a series of explorations and studies. At present, some common solutions include using Flash plug-ins or Java Applets to upload large files, but these technologies have compatibility issues and are gradually being eliminated by mainstream browsers.

[0007] In recent years, with the promotion of HTML5 standards and the continuous advancement of Web technology, some new front-end large file upload technologies have emerged, such as block upload, breakpoint resume, etc. However, these technologies still have some shortcomings in practical applications, such as the time-consuming calculation of the MD5 value of large files, which affects subsequent uploads or causes upload failures, and the accurate calculation of upload progress and real-time feedback still need to be further improved and perfected.

[0008] It takes a long time to upload large files. You may encounter the following problems during the upload process:

[0009] (1) Close the current page or jump to another page during the upload of large files.

[0010] (2) Memory overflow during large file upload.

[0011] (3) The progress of uploading large files is uncontrollable.

[0012] (4) It takes a long time to calculate the MD5 value of a large file, and the user is not given feedback on the current upload status. Summary of the invention

[0013] 1. Technical issues to be resolved

[0014] The technical problem to be solved by the present invention is how to provide a front-end large file uploading method to solve the above problems occurring in large file uploading.

[0015] (II) Technical solution

[0016] In order to solve the above technical problems, the present invention proposes a front-end large file uploading method, which comprises the following steps:

[0017] Step S1, install and reference the vue-simple-loader plug-in and spark-md5.js library;

[0018] Step S2, cancel the default automatic upload of the vue-simple-loader plug-in, and set the plug-in attribute auto-start to "false";

[0019] Step S3, split the file into pieces and calculate the md5 value; at this time, the front-end upload status is displayed as "calculating md5...";

[0020] Step S4, start uploading in pieces. The front end uses the resume method provided by the vue-simple-loader plug-in to start uploading. When the resume method is called, the vue-simple-loader plug-in will start uploading in pieces and send an upload request to the back end. Each upload request sends the piece file data, and also sends the piece number, total number of pieces, and file md5 value parameters;

[0021] Step S5, the back end receives the fragment file and fragment number, total number of fragments, file md5 value and other information in the front end request, calculates the md5 value of the temporary file after all the received fragments are merged, and compares the calculated md5 value with the md5 value sent by the front end; reads the stored fragment files in the order of fragment numbers, creates a new file for storing the merged complete file, and writes the data of each fragment into the new file.

[0022] (III) Beneficial effects

[0023] The present invention proposes a front-end large file upload method, and implements a global upload component based on the vue-simple-uploader plug-in. The fragmented upload divides the large file into smaller fragments for parallel upload, which significantly reduces the time of a single upload request, greatly improves the overall upload speed, effectively reduces the repetitive work and time waste caused by upload failure, and improves the success rate of file upload; it can also make full use of network bandwidth to avoid network congestion and transmission delays caused by excessively large files; and avoids loading the entire large file into the memory at one time, which reduces the memory pressure of the client and reduces the requirements for the client's hardware resources, so that large file uploads can be successfully completed on devices with limited resources. The global upload component optimizes the user experience, and users can monitor the upload progress in real time, including the file upload status and completion percentage, which enhances the user's control over the upload process; it is easy to integrate with existing Vue projects, convenient for development and maintenance, and has good versatility and compatibility. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] Figure 1 A flowchart for the global component implementation of the present invention;

[0025] Figure 2 This is a flow chart of MD5 calculation for each slice during the uploading process of the present invention. DETAILED DESCRIPTION

[0026] In order to make the purpose, content and advantages of the present invention more clear, the specific implementation methods of the present invention are further described in detail below in conjunction with the drawings and examples.

[0027] The present invention aims to propose a front-end large file uploading method, which solves the above problems occurring in large file uploading by uploading in pieces, calculating MD5 values ​​in pieces, optimizing the upload page, etc.

[0028] The present invention studies the problem of uploading large files on the front end, and implements a more user-friendly Vue project global upload component based on the vue-simple-loader plug-in. vue-simple-uploader is an upload plug-in encapsulated based on simple-Uploader.js. Compared with the upload component of element-ui, it can meet most file upload requirements. spark-md5.js is a lightweight JavaScript library for quickly calculating the MD5 value of strings and files. The SparkMD5.ArrayBuffer instance object can calculate the MD5 hash value of a large file on the client (such as a browser) without loading the entire file into memory. This method is particularly suitable for processing large files, and can calculate the hash value while reading the file, rather than processing the entire file at one time. The present invention is based on the global upload component implemented by the vue-simple-loader plug-in and the spark-md5.js library, adds the "calculating MD5" state, realizes the sharding calculation of the md5 value, and optimizes the upload page.

[0029] Figure 1 This is a flowchart of the global upload component implemented based on the vue-simple-loader plug-in of the present invention, refer to Figure 1 As shown, it mainly includes the following steps:

[0030] Step S1, install and reference the vue-simple-loader plug-in and spark-md5.js library.

[0031] Step S2, since the md5 value of the file needs to be calculated before uploading the file, the default automatic upload of the vue-simple-loader plug-in is canceled here, and the plug-in attribute auto-start is set to "false".

[0032] Step S3, file sharding and md5 value calculation. At this time, the front-end upload status is displayed as "calculating md5...". Set the shard size, read the file through the FileReader object, combine the slice sharding method of the File object to implement sharded file reading, pass the sharded data to the append method of the SparkMD5.ArrayBuffer object, and calculate the md5 value of the shards. The append method will perform hash calculation on the currently passed in data block and update the internal hash state. By gradually processing each sharded data block, the memory overflow problem can be effectively avoided, and the calculation results of the hash value can be gradually accumulated and updated during the processing process, and finally the md5 value of the entire file is obtained through the end method of the SparkMD5.ArrayBuffer object.

[0033] Step S4, start uploading in pieces. The front end uses the resume method provided by the vue-simple-loader plug-in to start uploading. When the resume method is called, the vue-simple-loader plug-in will start uploading in pieces and send an upload request to the back end. The piece file data is sent in each upload request. In addition, the piece number, the set piece size, the current piece size, the file size, the file name, the total number of pieces, the file md5 value and other parameters are also sent.

[0034] Step S5, the back end receives the information such as the slice file and the slice number, the total number of slices, the file md5 value in the front end request, calculates the md5 value of the temporary file after all the slices have been received, and compares the calculated md5 value with the md5 value sent by the front end to ensure the file integrity and accuracy. Read the stored slice files in the order of the slice numbers, create a new file for storing the merged complete file, and write the data of each slice into the new file.

[0035] Figure 2 This is a flowchart of calculating MD5 for each slice during the upload process of the present invention. Figure 2 As shown, the sharding MD5 calculation process proposed in the present invention mainly includes the following steps:

[0036] Step S31, set the fragment size, the number of fragments is the file size divided by the fragment size rounded up.

[0037] Step S32, read the file slice content in a loop. Use the FileReader object in combination with the File.silce method to read the slice file. Instantiate the SparkMD5.ArrayBuffer object spark, and pass each slice data into the spark.append method for hash calculation.

[0038] Step S33: After all data has been processed, call spark.end() to obtain the final md5 value.

[0039] In the step S31, the file size is filesize, the chunk size is chunksize, and the number of chunks is: chunks=math.ceil(filesize / chunksize).

[0040] In step S32, the segmented file is read using the FileReader object in combination with the File.silce method.

[0041] In the step S32, start=currentchunk*chunksize, end=min(start+chunksize,file.size), filereader.readAsArrayBuffer(blobslice.call(file,start,end)); wherein currentchunk is the current slice, chunksize is the slice size, and start and end are the beginning and end of the slice.

[0042] The step S32 also includes: outputting sharding parsing information to display the current progress of calculating md5.

[0043] The key codes are as follows:

[0044] Template code:

[0045]

[0046]

[0047] Script code:

[0048]

[0049]

[0050]

[0051] The present invention proposes a front-end large file upload method, and implements a global upload component based on the vue-simple-uploader plug-in. The fragmented upload divides the large file into smaller fragments for parallel upload, which significantly reduces the time of a single upload request, greatly improves the overall upload speed, effectively reduces the duplication of work and time waste caused by upload failures, and improves the success rate of file uploads; it can also make full use of network bandwidth to avoid network congestion and transmission delays caused by excessively large files; and avoids loading the entire large file into the memory at one time, which reduces the memory pressure of the client and reduces the requirements for the client's hardware resources, so that large file uploads can be successfully completed on devices with limited resources. The global upload component optimizes the user experience, and users can monitor the upload progress in real time, including the file upload status and completion percentage, which enhances the user's control over the upload process; it is easy to integrate with existing Vue projects, convenient for development and maintenance, and has good versatility and compatibility.

[0052] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the technical principles of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. A front-end large file uploading method, characterized in that: The method comprises the following steps: Step S1, install and reference the vue-simple-loader plug-in and spark-md5.js library; Step S2, cancel the default automatic upload of the vue-simple-loader plug-in, and set the plug-in attribute auto-start to "false"; Step S3, split the file into pieces and calculate the md5 value; at this time, the front-end upload status is displayed as "calculating md5..."; Step S4, start uploading in pieces. The front end uses the resume method provided by the vue-simple-loader plug-in to start uploading. When the resume method is called, the vue-simple-loader plug-in will start uploading in pieces and send an upload request to the back end. Each upload request sends the piece file data, and also sends the piece number, total number of pieces, and file md5 value parameters; Step S5, the back end receives the fragment file and fragment number, total number of fragments, file md5 value and other information in the front end request, calculates the md5 value of the temporary file after all the received fragments are merged, and compares the calculated md5 value with the md5 value sent by the front end; reads the stored fragment files in the order of fragment numbers, creates a new file for storing the merged complete file, and writes the data of each fragment into the new file.

2. The front-end large file uploading method according to claim 1, characterized in that: The S3 includes: setting the shard size, reading the file through the FileReader object, combining the slice sharding method of the File object to implement sharded file reading, passing the sharded data to the append method of the SparkMD5.ArrayBuffer object, and calculating the md5 value of the shards; the append method will perform hash calculation on the currently passed in data block and update the internal hash state; by gradually processing each sharded data block, and gradually accumulating and updating the calculation results of the hash value during the processing process, finally obtaining the md5 value of the entire file through the end method of the SparkMD5.ArrayBuffer object.

3. The front-end large file uploading method according to claim 2, characterized in that: The S4 upload request also includes: a set segment size, a current segment size, a file size, and a file name.

4. The front-end large file uploading method as claimed in claim 2, characterized in that: spark-md5.js is used to quickly calculate MD5 values ​​of strings and files. The SparkMD5.ArrayBuffer instance object calculates the MD5 hash value of large files on the client without loading the entire file into memory.

5. The front-end large file uploading method as claimed in claim 4, characterized in that: The client is a browser.

6. The front-end large file uploading method as claimed in claim 4, characterized in that: The MD5 calculation process specifically includes: Step S31, set the fragment size, the number of fragments is the file size divided by the fragment size and rounded up; Step S32, looping and reading the file slice contents; instantiating the SparkMD5.ArrayBuffer object spark, and passing each slice data into the spark.append method for hash calculation; Step S33: After all data has been processed, call spark.end() to obtain the final md5 value.

7. The front-end large file uploading method according to claim 6, characterized in that: In the step S31, the file size is filesize, the chunk size is chunksize, and the number of chunks is: chunks=math.ceil(filesize / chunksize).

8. The front-end large file uploading method according to claim 7, characterized in that: In step S32, the segmented file is read using the FileReader object in combination with the File.silce method.

9. The front-end large file uploading method as claimed in claim 8, characterized in that: In the step S32, start=currentchunk*chunksize, end=min(start+chunksize,file.size), filereader.readAsArrayBuffer(blobslice.call(file,start,end)); wherein currentchunk is the current slice, chunksize is the slice size, and start and end are the beginning and end of the slice.

10. The front-end large file uploading method according to claim 6, characterized in that: The step S32 also includes: outputting sharding parsing information to display the current progress of calculating md5.

Images