A large file upload method for narrowband network environment

Abstract

The present invention discloses a large file uploading method for a narrowband network environment, which belongs to the technical field of file uploading, and includes S1, loading a large file to be uploaded into a browser supporting HTML5 language to obtain first file summary information; S2, uploading the first file summary information, if the first file summary information already exists in the background database, notifying the browser to complete the upload immediately, if not, executing step S3; S3, calculating the slice size of the large file to be uploaded, and cutting the large file to be uploaded into corresponding block files in real time; S4, uploading the block files to the background server one by one, merging the block files after all the block files are transmitted, and completing the large file upload. Through the collaboration of the browser and the background program, the method realizes the reduction of large files to small files, and ensures the browser performance; it can realize breakpoint resume and second transmission, greatly improve the upload efficiency, make the operation simple and efficient, and also greatly improve the robustness of the system and the narrowband link adaptation ability.

Description

Technical Field

[0001] The invention belongs to the technical field of file uploading, and in particular relates to a large file uploading method used in a narrowband network environment. Background Art

[0002] With the rapid development of Internet technology towards industry and military, the gradual popularization of massive big data applications, and the increasing traffic interactions in the network, the efficient and stable uploading of large and super-large files has become a daily need in industry and military. The existing browser-implemented file upload technology is a streaming upload technology implemented by scripting language, which has the following disadvantages: First, the user may cause abnormal interruption of file upload due to closing the browser or sudden network or power outage when using the terminal, which means that the file needs to be uploaded again; second, the upload is unstable, and large files (larger than 2GB) are prone to upload failure due to network fluctuations and disconnection, and even large file upload is not supported. Third, due to the different performance of the browser and the memory of the terminal, uploading large files can easily cause the browser to freeze and fail to respond. Summary of the invention

[0003] Purpose of the invention: The technical problem to be solved by the present invention is to provide a method for uploading large files in a narrowband network environment in response to the shortcomings of the prior art, so as to improve the efficiency and stability of browser uploading, greatly reduce resource consumption, achieve fast uploading and ensure the integrity of files.

[0004] In order to solve the above technical problems, the present invention discloses a large file uploading method for a narrowband network environment, comprising the following steps:

[0005] S1, loading the large file to be uploaded into a browser supporting HTML5 (HyperText Markup Language 5, Hypertext 5.0) language, and obtaining first file summary information of the large file to be uploaded;

[0006] S2. Upload the first file summary information to the backend server. The backend server queries the backend database whether the first file summary information already exists. If so, the browser is notified to complete the upload immediately. If not, step S3 is executed.

[0007] S3, according to the real-time bandwidth, delay and packet loss rate of the connected narrowband network, calculate the slice size of the large file to be uploaded, and cut the large file to be uploaded into corresponding block files in real time according to the slice size;

[0008] S4. Upload the block files to the backend server one by one through the HTTP / 2 protocol. After all the block files are transferred, merge the block files to complete the large file upload.

[0009] In one implementation, in step S1, obtaining the first file summary information of the large file to be uploaded includes using the BlobBuilder interface in the browser HTML5 protocol to convert the large file to be uploaded into a Blob object, and using the Blob object and the file name of the large file to be uploaded as feature values ​​to be encrypted using MD5 (Message-Digest Algorithm 5), and the obtained encrypted value is the first file summary information.

[0010] In one implementation, in step S3, before uploading the large file, the real-time network speed of the narrowband network is obtained according to the real-time bandwidth, delay and packet loss rate of the connected narrowband network, and the slice size of the large file to be uploaded is calculated according to the real-time network speed, including: when the obtained real-time network speed is greater than 200KB / s, the slice size is set to 2M; when the obtained real-time network speed is greater than 100KB / s and less than or equal to 200KB / s, the slice size is set to 1M; when the obtained real-time network speed is greater than 50KB / s and less than or equal to 100KB / s, the slice size is set to 512K; when the obtained real-time network speed is less than or equal to 50KB / s, the slice size is set to 256K.

[0011] In one implementation, in step S3, according to the real-time bandwidth condition under the narrowband network condition, the entire large file is cut into n block files, and a block mark array is used to record each block file number and a block mark, wherein the block mark identifies the upload status of the block file, and the block file numbers are 0 to n-1 in sequence, where n is a natural number; step S4 includes:

[0012] S4-1, upload the block files to the backend server in sequence. If the block file is uploaded, the block mark of the corresponding block file in the block mark array is set to true. If the upload is not completed, the block mark is set to false;

[0013] S4-2, the backend server converts the block file into a binary stream file, and sends the binary stream file to the file server;

[0014] S4-3, when all block files of the large file have been uploaded, and the block flags of all block files in the block flag array are true, the background server notifies the file server to merge the binary stream files of the block files into a complete large file, and obtains the second file summary information of the merged large file by performing MD5 algorithm processing on the complete large file and the corresponding file name, and records the second file summary information in the background database;

[0015] S4-4, the background server verifies the first file summary information of the large file to be uploaded and the second file summary information of the merged large file. If the two are the same, it returns the information that the large file has been successfully uploaded to the browser; if the two are different, it returns the information that the large file has failed to be uploaded, and then executes step S1 to upload again.

[0016] In one implementation, during the execution of step S4, if the upload of a large file is interrupted due to network fluctuations or other reasons, the block mark array value stops recording, and the block file upload record in the block mark array is retained. When the same large file is uploaded again, the background server detects the block file upload record in the block mark array, directly skips the uploaded block file, and continues to perform steps S4-1 to S4-4 from the block file that has not been uploaded to upload, thereby realizing breakpoint resume of large files.

[0017] In one implementation, during the execution of step S4, if the network bandwidth is always in an unstable state during the upload process, resulting in a timeout or failure in the block file upload, that is, when the block mark of the corresponding block file in the block mark array is false, the block file that failed to be uploaded is rolled back, and step S3 is re-executed to calculate the large file slice size, the large file to be uploaded is cut into corresponding block files, and step 4 is executed again to upload, thereby achieving stable uploading of large files.

[0018] In one implementation, in step S3, the large file to be uploaded is cut into corresponding block files in real time according to the slice size, and the browser plug-in Plupload is used for cutting.

[0019] Beneficial effects:

[0020] First, through the collaborative processing of the browser and the background program, the operation of uploading large files becomes simple and efficient. Second, the use of the newer HTTP / 2 protocol has a faster transmission speed and occupies less bandwidth than the HTTP / 1.1 protocol, making it easier to implement functions such as breakpoint resume, which greatly reduces user waiting time and improves upload efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] The present invention will be further described in detail below in conjunction with the accompanying drawings and specific embodiments, and the above and / or other advantages of the present invention will become more clear.

[0022] Figure 1 is an exemplary system architecture diagram according to an embodiment of the present invention.

[0023] Figure 2 The present invention is a flowchart of a method for uploading large files in a narrowband network environment according to an embodiment of the present invention. DETAILED DESCRIPTION

[0024] In order to make the technical problems, technical solutions and beneficial effects to be solved by the present invention more clearly understood, the present invention is described in detail below in conjunction with embodiments. It should be noted that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0025] The present application provides a large file uploading method for a narrowband network environment, which can be applied to narrowband and unreliable network environment scenarios such as tactical terminal mobile communications and industrial Internet of Things.

[0026] like Figure 1 As shown, the system architecture of the embodiment of the present application includes a browser, a backend server, a backend database and a file server. Among them, the browser supports HTML5 language, and the browser uploads large files to the backend server through the HTTP / 2 protocol. The HTTP / 2 protocol is the first update of the HTTP (Hyper Text Transfer Protocol) protocol since the release of HTTP1.1 in 1999, and is mainly based on the SPDY (An experimental protocol for afaster web, a faster Web protocol) protocol. The next generation Internet communication protocol formed on the basis of the protocol. The HTTP / 2 protocol is a binary transmission protocol, which is easier to implement data packet transmission than the text (string) transmission of HTTP1.x. It reduces network delay by supporting multiplexing of requests and responses, reduces protocol overhead by compressing the HTTPS header field, and increases support for request priority and server push. It avoids the problem of poor performance of bursty and short-term HTTP connections, and supports multiple request-response messages initiated through a single HTTP / 2 connection, which can greatly improve the robustness of transmission and narrowband link adaptation capabilities.

[0027] like Figure 2 FIG. 1 is a flowchart of a method for uploading a large file in a narrowband network environment according to an embodiment of the present application. The method includes the following steps:

[0028] S1. Load the large file to be uploaded into a browser supporting HTML5 language, and obtain the first file summary information of the large file to be uploaded;

[0029] S2. Upload the first file summary information to the backend server. The backend server queries the backend database whether the first file summary information already exists. If it exists, there is no need to upload it again, and the browser is notified to complete the upload immediately. If it does not exist, execute step S3.

[0030] S3. Calculate the slice size of the large file to be uploaded according to the real-time bandwidth, delay and packet loss rate of the connected narrowband network, and cut the large file to be uploaded into corresponding block files in real time according to the slice size, and use the block mark array to record the current block file number and block mark;

[0031] S4. Upload the block files to the backend server one by one through the HTTP / 2 protocol. After all the block files are transferred, merge the block files to complete the large file upload.

[0032] In this embodiment, in the step S1, obtaining the first file summary information of the large file to be uploaded includes using the BlodBuilder interface in the browser HTML5 protocol to convert the large file to be uploaded into a Blob object, and using the Blob object and the file name of the large file to be uploaded as feature values ​​to encrypt using MD5 encoding, and the obtained encrypted value is the first file summary information.

[0033] In this embodiment, in the step S3, before uploading the large file, the real-time bandwidth, delay and packet loss rate of the narrowband network are obtained through code, and the real-time network speed of the narrowband network is obtained according to the real-time bandwidth, delay and packet loss rate of the connected narrowband network. According to the obtained real-time network speed, it is obtained through multiple tests: when the obtained real-time network speed is greater than 200KB / s, the slice size is set to 2M; when the obtained real-time network speed is greater than 100KB / s and less than or equal to 200KB / s, the slice size is set to 1M; when the obtained real-time network speed is greater than 50KB / s and less than or equal to 100KB / s, the slice size is set to 512K; when the obtained real-time network speed is less than or equal to 50KB / s, the slice size is set to 256K.

[0034] In this embodiment, in step S3, according to the real-time bandwidth condition under the narrowband network condition, the entire large file is cut into n block files, and a block mark array is used to record each block file number and block mark, the block mark identifies the upload status of the block file, and the block file numbers are 0 to n-1 in sequence, where n is a natural number; step S4 includes:

[0035] S4-1, upload the block files to the backend server in sequence, and set the block flag of the corresponding block file in the block flag array to true if the block file is uploaded, and set the block flag to false if the upload is not completed;

[0036] S4-2, the backend server converts the block file into a binary stream file, and sends the binary stream file to the file server;

[0037] S4-3, when all block files of the large file have been uploaded, and the block flags of all block files in the block flag array are true, the background server notifies the file server to merge the binary stream files of the block files into a complete large file, and obtains the second file summary information of the merged large file by performing MD5 algorithm processing on the complete large file and the corresponding file name, and records the second file summary information in the background database;

[0038] S4-4, the background server verifies the first file summary information of the large file to be uploaded and the second file summary information of the merged large file. If the two are the same, it returns the information that the large file has been successfully uploaded to the browser; if the two are different, it returns the information that the large file has failed to be uploaded, and then executes step S1 to upload again.

[0039] In this embodiment, during the execution of step S4, if the upload of a large file is interrupted midway, the block file upload record in the block mark array is retained. When the same large file is uploaded again, the background server detects the block file upload record in the block mark array, directly skips the uploaded block file, and continues to execute steps S4-1 to S4-4 from the block file that has not been uploaded to upload.

[0040] In this embodiment, during the execution of step S4, if the network bandwidth is always in an unstable state during the upload process, resulting in a block file upload timeout or failure, that is, when the block mark of the corresponding block file in the block mark array is false, the block file that failed to upload is rolled back, step S3 is re-executed to calculate the large file slice size, the large file to be uploaded is cut into corresponding block files, and step 4 is executed again for uploading. The rollback step includes: the backend server deletes the part that failed to upload, and finds the file slice position through the marked file number, returns the corresponding value of the foreground, recalculates the slice size from the slice position, and continues uploading.

[0041] In this embodiment, in step S3, the large file to be uploaded is cut into corresponding block files in real time according to the slice size, and the browser plug-in Plupload is used for cutting.

[0042] The present invention provides a method for uploading large files in a narrowband network environment. There are many methods and ways to implement the technical solution. The above is only a specific implementation of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principle of the present invention. These improvements and modifications should also be regarded as the protection scope of the present invention. All components not specified in this embodiment can be implemented by existing technologies.

Claims

1. A method for uploading large files in a narrowband network environment, characterized by: The steps include: Step S1, loading the large file to be uploaded into a browser supporting HTML5 language, and obtaining first file summary information of the large file to be uploaded; Step S2, uploading the first file summary information to the backend server, the backend server queries the backend database whether the first file summary information already exists, if so, notifying the browser to immediately complete the upload, if not, executing step S3; Step S3: Calculate the slice size of the large file to be uploaded according to the real-time bandwidth, delay and packet loss rate of the connected narrowband network, and cut the large file to be uploaded into corresponding block files in real time according to the slice size; Step S4: upload the block files to the backend server one by one through the HTTP / 2 protocol. After all the block files are transferred, merge the block files to complete the large file upload. In step S3, according to the real-time bandwidth condition under the narrowband network condition, the entire large file is cut into n block files, and a block mark array is used to record the number and block mark of each block file, wherein the block mark identifies the upload status of the block file, and the block file numbers are 0 to n-1 in sequence, where n is a natural number; The step S4 comprises: S4-1, upload the block files to the backend server in sequence, and if the block file is uploaded, set the block flag of the corresponding block file in the block flag array to true, and if the upload is not completed, set the block flag to false; S4-2, the background server converts the block file into a binary stream file, and sends the binary stream file to the file server; S4-3, when all block files of the large file have been uploaded, and the block flags of all block files in the block flag array are true, the background server notifies the file server to merge the binary stream files of the block files into a complete large file, and obtains the second file summary information of the merged large file by performing MD5 algorithm processing on the complete large file and the corresponding file name, and records the second file summary information in the background database; S4-4, the backend server verifies the first file summary information of the large file to be uploaded and the second file summary information of the merged large file. If the two are the same, the backend server returns the message that the large file has been successfully uploaded to the browser; if the two are different, the backend server returns the message that the large file has been uploaded unsuccessfully, and then executes step S1 to upload again; During the execution of step S4, if the upload of a large file is interrupted, the block file upload record in the block mark array is retained. When the same large file is uploaded again, the background server detects the block file upload record in the block mark array, directly skips the uploaded block file, and continues to execute steps S4-1 to S4-4 from the block file that has not been uploaded to upload; In the process of executing step S4, if the network bandwidth is always in an unstable state during the upload process, resulting in a timeout or failure in the block file upload, that is, when the block mark of the corresponding block file in the block mark array is false, the block file that failed to upload is rolled back, and step S3 is re-executed to calculate the large file slice size, the large file to be uploaded is cut into corresponding block files, and step 4 is executed again for uploading.

2. The method for uploading large files in a narrowband network environment according to claim 1, characterized in that: In the step S1, obtaining the first file summary information of the large file to be uploaded includes using the BlodBuilder interface in the browser HTML5 protocol to convert the large file to be uploaded into a Blob object, and using the Blob object and the file name of the large file to be uploaded as feature values ​​to encrypt using MD5 encoding, and the obtained encrypted value is the first file summary information.

3. The method for uploading large files in a narrowband network environment according to claim 1, characterized in that: In step S3, before uploading the large file, the real-time network speed of the narrowband network is obtained according to the real-time bandwidth, delay and packet loss rate of the connected narrowband network, and the slice size of the large file to be uploaded is calculated according to the real-time network speed, including: when the obtained real-time network speed is greater than 200KB / s, the slice size is set to 2M; when the obtained real-time network speed is greater than 100KB / s and less than or equal to 200KB / s, the slice size is set to 1M; when the obtained real-time network speed is greater than 50KB / s and less than or equal to 100KB / s, the slice size is set to 512K; when the obtained real-time network speed is less than or equal to 50KB / s, the slice size is set to 256K.

4. The method for uploading large files in a narrowband network environment according to claim 1, characterized in that: In step S3, the large file to be uploaded is cut into corresponding block files in real time according to the slice size, and the browser plug-in Plupload is used for cutting.

Images