A method and storage medium for file transfer

Abstract

The application discloses a file transmission method and storage medium, a browser end acquires a target file to be transmitted, the target file is divided into blocks, a file block with a block size is obtained, and transmission of the file block is performed; the browser end periodically calculates a transmission time and an error rate of the file block, calculates a block correction size according to the transmission time and the error rate; the browser end adjusts the block size of the file block to be transmitted in real time according to the block correction size; the application divides a large file according to an initial block size, realizes transmission of the large file through the block division, solves problems existing in the prior large file transmission, and corrects the initial block size according to the transmission time and the error rate, so that the efficiency and stability of the block transmission are ensured.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of file transmission, and in particular to a file transmission method and storage medium. BACKGROUND

[0002] Cloud storage has become an important aspect of modern data storage and management, especially in the context of massive data and large-scale data storage needs, cloud storage plays a key role in providing convenient, efficient and economical data storage solutions. However, in the face of growing data size and complexity, traditional cloud storage solutions are facing serious challenges. The most obvious one is that uploading a super large file from a browser web page to a cloud storage server is a relatively prominent problem.

[0003] In traditional file upload systems, users usually need to upload the entire file to the cloud storage service at one time. However, this approach has serious problems when dealing with super large files. For example, some firewalls, gateways, etc. may limit the maximum upload size of a single file, usually not more than 1GB. This limits the user's ability to upload large files, especially those who may need to upload files larger than 1G. In addition, even if the firewall, gateway and other components support the upload of super large files, there are limitations. For example, if the network fluctuates during the upload process, it may cause the upload to fail, and the user has to start the upload again, which not only consumes time and network resources, but also causes the user to have a bad user experience.

[0004] In the prior art, users uploading large files usually need to download a special client to transfer files: many cloud storage services usually require customers to download a client to upload files, which is very inconvenient to use, such as customers who just want to download files temporarily on a new computer must first install the client. Even if the web provides file transfer functions, there are limitations on file size. For example, some services may limit the maximum upload size of a single file, usually not more than 1GB. For users who need to upload larger files, this limitation obviously affects the user experience.

[0005] The need to download and install the client obviously increases the burden on the user. For example, if the user temporarily needs to download or upload files on a new computer, they first need to download and install the corresponding client software, which may take some time. This feature of relying on a specific client limits the convenience of the software, especially when the user only needs to perform one-time or temporary file transfer.

[0006] Some web platforms provide file transfer functions, but still have limitations on file size, such as the upload size of a single file generally does not exceed 1MB, 10MB or 1GB. Such size limitation is determined by the data transmission technology characteristics between the client and the server. In the traditional web file upload technology, the file upload is usually implemented by the following method:

[0007] <form action＝" upload"method＝"POST"enctype＝"multipart form-data">

[0008] <input type＝"file"name＝"file">

[0009] <input type＝"submit"value＝"上传">

[0010] < / form>

[0011] This way uploads the entire file at a time, which is usually not a problem for small file transmission, but may cause delay or failure in the transmission process for large files. Therefore, in order to ensure the stability of data transmission, service providers usually set a file size limit. However, this limit obviously affects the user experience, especially for users who need to upload 1G, 2G or even 4G files.

[0012] In summary, the technical features in the prior art solution, including file transfer relying on a special client and file size limitation, all lead to the inconvenience and inefficiency of large file upload. These defects limit the ability of the prior art to handle large file upload and storage, resulting in poor user experience during use. SUMMARY

[0013] The technical problem to be solved by the present application is to provide a file transfer method and storage medium to solve the problems encountered in large file transfer.

[0014] In order to solve the above technical problems, the technical solution adopted by the present application is:

[0015] A file transfer method, comprising the steps of:

[0016] S1, the browser end obtains a target file to be transferred, divides the target file into blocks to obtain file blocks of a block size, and transmits the file blocks;

[0017] S2, the browser end periodically calculates the transmission time and error rate of the file blocks, and calculates a block correction size according to the transmission time and error rate;

[0018] S3, the browser end adjusts the block size of the file blocks to be transmitted in real time according to the block correction size.

[0019] In order to solve the above technical problems, another technical solution adopted by the present application is:

[0020] A storage medium having a computer program stored thereon, the computer program being executed to implement the steps of the above-mentioned file transfer method.

[0021] The beneficial effect of the present application is that: the file transmission method and storage medium of the present application, according to the initial block size, the large file is divided into blocks, and the transmission of the large file is realized by the block method, and the problems existing in the transmission of the large file are solved, and the initial block size is corrected according to the transmission time and the error rate, and the efficiency and stability of the block transmission are ensured. BRIEF DESCRIPTION OF DRAWINGS

[0022] Figure 1 The flow chart of the file transmission method of the embodiment of the present application is shown in the figure.

[0023] Figure 2 The flow chart of the file transmission method of the embodiment of the present application is shown in the figure. DETAILED DESCRIPTION

[0024] In order to explain the technical content, the purpose and the effect of the present application in detail, the following will be explained in combination with the embodiments and the drawings.

[0025]

[0026]

[0027] Please refer to Figure 1 A file transmission method, comprising the steps of:

[0028] S1, the browser end obtains the target file to be transmitted, divides the target file into blocks, obtains the block size of the file block, and transmits the file block;

[0029] S2, the browser end periodically calculates the transmission time and the error rate of the file block, and calculates the block correction size according to the transmission time and the error rate;

[0030] S3, the browser end adjusts the block size of the file block to be transmitted according to the block correction size.

[0031] From the above description, the beneficial effect of the present application is that: the file transmission method and storage medium of the present application, according to the initial block size, the large file is divided into blocks, and the transmission of the large file is realized by the block method, and the problems existing in the transmission of the large file are solved, and the initial block size is corrected according to the transmission time and the error rate, and the efficiency and stability of the block transmission are ensured.

[0032] Further, the block correction size calculated according to the transmission time and the error rate is:

[0033] s1=M / t;

[0034] s2=e / c;

[0035]

[0036] Wherein, s1 represents the transmission speed, M represents the current block size, t represents the transmission time, s2 represents the error rate, e represents the transmission error block speed, c represents the total number of blocks transmitted, B represents the block correction size, K and a represent the calibration factor, and are preset values.

[0037] As described above, the block size is optimized and corrected according to the block correction size.

[0038] Further, the transmission of the file blocks in step S1 is encrypted.

[0039] As described above, the encrypted transmission ensures the security of the file transmission.

[0040] Further, the encrypted transmission includes the steps of:

[0041] Requesting a public key from the server, and after receiving the public key returned by the server, generating a key, encrypting the key with the public key, and sending the encrypted key to the server, so that the server decrypts the key with a private key to obtain the key;

[0042] The encrypted file obtained by encrypting the file blocks with the key is transmitted to the server, so that the server decrypts the encrypted file with the key to obtain the block file.

[0043] As described above, the transmission key is encrypted according to the public key of the server and transmitted to the server, and the file blocks are encrypted and transmitted through the key, which further improves the security of the file transmission.

[0044] Further, the key is generated as follows:

[0045] The key is generated by combining the received public key, the time stamp of receiving the public key, and the current user ID.

[0046] As described above, the key is generated according to the received public key, the time stamp of receiving the public key, and the current user ID, so that each generated key is different and has a certain randomness, ensuring the security of the key.

[0047] Further, when transmitting the file blocks, the block information of the target file is transmitted synchronously.

[0048] Further, the method further includes the steps of:

[0049] S4, the server receives each of the file blocks, and according to the block information uploaded synchronously with the file blocks, splices the file blocks to obtain the original target file.

[0050] From the above description, when the block transmission is performed, the synchronization transmission block information is transmitted, and the server splices the blocks to obtain the target file.

[0051] Further, the step S4 is specifically:

[0052] The server receives each of the file blocks and stores the file blocks in the NAS file storage system.

[0053] The file blocks are spliced in the NAS file storage system to obtain the target file.

[0054] From the above description, in order to avoid the block occupying the space of the server, the block is directly stored in the cloud shared NAS server.

[0055] Further, the NAS file storage system adopts a random read-write mechanism and filters the file path where the file block is located through a file name matching mechanism.

[0056] From the above description, the NAS file storage system adopts a random read-write mechanism and filters the file path where the file block is located through a file name matching mechanism, thereby improving the efficiency of storage and management.

[0057] Further, the step S3 is specifically:

[0058] The browser end determines whether the block correction size is greater than a preset upper limit of the block size, and if so, adjusts the block size of the file block to be transmitted according to the upper limit of the block size.

[0059] Determines whether the block correction size is less than a preset lower limit of the block size, and if so, adjusts the block size of the file block to be transmitted according to the lower limit of the block size.

[0060] If the block correction size is greater than the lower limit of the block size and less than the upper limit of the block size, the block size of the file block to be transmitted is adjusted according to the block correction size.

[0061] From the above description, limiting the upper limit and the lower limit of the block size can prevent the block from being too small, causing too many block HTTP requests and causing overhead, thereby ensuring the efficiency and stability of the block transmission.

[0062] A storage medium having a computer program stored thereon, the computer program being executed to implement the steps of the above-mentioned file transmission method.

[0063] The application discloses a file transmission method and storage medium, which is suitable for a large file transmission scenario, in particular to a large file transmission scenario under a Web.

[0064] Please refer to Figure 1 The embodiment one of the application is:

[0065] A file transmission method comprises the following steps:

[0066] S1, a browser end acquires a target file to be transmitted, divides the target file into file blocks with a block size, and transmits the file blocks.

[0067] When the file blocks are transmitted, the block information of the target file is synchronously transmitted.

[0068] In the embodiment, when a user selects to upload a large file, a browser Web page receives a file upload request, and first performs block processing on the file. According to a specified block size (an initial block size), the file content is subjected to a data block operation, so that each block can be independently processed and uploaded.

[0069] The file blocks are uploaded to a cloud sharing server through a js initiated http file upload request. Each file block is independently uploaded and does not affect each other. Even if unstable network environment is encountered, block retransmission is performed, so that the file can be finally successfully uploaded. Meanwhile, since no client needs to be downloaded and installed, the user can perform uploading on any device, and the use convenience is greatly improved.

[0070] S2, the browser end periodically calculates a transmission time of the file blocks and an error rate, and calculates a block correction size according to the transmission time and the error rate.

[0071] The block correction size calculated according to the transmission time and the error rate is specifically:

[0072] s1=M / t;

[0073] s2=e / c;

[0074]

[0075] Wherein, s1 represents a transmission speed, M represents the current block size, t represents a transmission time, s2 represents an error rate, e represents a block speed of a transmission error, c represents a total number of transmission blocks, B represents a block correction size, K and a represent calibration factors, and are preset constant values.

[0076] In this embodiment, when each file chunk is transmitted, the transmission time and error rate of the chunk transmission are counted. According to the transmission time and error rate, the current network condition is determined (for example, if a 1M chunk transmission consumes more than 10s or the error rate of the chunk transmission in the file transmission process is higher than 5%, it indicates that the current network environment is poor, and the size of the chunk needs to be adjusted according to the network environment).

[0077] Specifically embodied as:

[0078] When the average 1MB data transmission time is less than N seconds, then according to the formula

[0079] s1=M / t;

[0080] Wherein, s1 represents the transmission speed, M represents the current chunk size, and t represents the transmission time;

[0081] The transmission speed is calculated as 1MB / N;

[0082] According to the formula

[0083] s2=e / c;

[0084] Wherein, s2 represents the error rate, e represents the transmission error chunk speed, and c represents the total number of transmission chunks;

[0085] The transmission error rate is obtained;

[0086] Using the formula

[0087]

[0088] The target size (chunk correction size) B of the chunk to be adjusted is obtained. (Wherein K and a are a constant value, which is a calibration factor, used to adjust the size of the chunk and the influence degree of the error rate on the size of the chunk). S3, the browser side adjusts the chunk size of the file chunk to be transmitted in real time according to the chunk correction size;

[0089] Step S3 is specifically:

[0090] The browser side determines whether the chunk correction size is greater than the preset upper limit of the chunk size, and if so, adjusts the chunk size of the file chunk to be transmitted according to the upper limit of the chunk size;

[0091] Determine whether the chunk correction size is less than the preset lower limit of the chunk size, and if so, adjust the chunk size of the file chunk to be transmitted according to the lower limit of the chunk size;

[0092] If the chunk correction size is greater than the lower limit of the chunk size and less than the upper limit of the chunk size, then adjust the chunk size of the file chunk to be transmitted according to the chunk correction size.

[0093] In the embodiment, to prevent the block from being too large, the upper limit and the lower limit of the block size are limited, for example, the lower limit is 1 KB and the upper limit is 20 MB. In this way, the block can be prevented from being too small to cause too many block HTTP requests and too much cost, or too large to cause the server to consume too much server resource when processing the received block, thereby ensuring the efficiency and stability of the block transmission.

[0094] S4, the server receives each file block, and splices the file block according to the block information synchronously uploaded with the file block to obtain the original target file.

[0095] In the embodiment, in the cloud sharing server, the system confirms the logical relationship of each file block according to the block information recorded when the block is uploaded, and splices the file blocks to obtain the original large file.

[0096] Embodiment two of the application is:

[0097] A file transmission method, which is different from the first embodiment in that the transmission of the file block in step S1 adopts encrypted transmission.

[0098] The encrypted transmission includes the following steps:

[0099] The public key is requested from the server, and after receiving the public key returned by the server, a secret key is generated, the secret key is encrypted by the public key and sent to the server, so that the server decrypts the secret key by the private key.

[0100] In the embodiment, each block is encrypted when transmitted. The encryption algorithm adopts AES encryption. After the user logs in, the RSA public key is requested from the server. After the front-end webjs receives the public key, the received token+timestamp+userNo information is combined into a 32-byte secret key (userNo takes the first 10 bytes, and the insufficient part is supplemented with zero, and the excess part is discarded; timestamp occupies 10 bytes, of which 7 bytes store timestamp, and the remaining 3 bytes are set as FF; token is truncated to the first ten bytes to occupy the remaining 12 bytes, and then the combined key is further calculated by using the sha-256 algorithm to obtain the AES key), which is encrypted by the public key and sent to the server. Because the public key is used for encryption, only the private key of the server can decrypt it, which can ensure the security of the secret key. In subsequent block transmission, each block is encrypted by the secret key. Even if the block is intercepted, it cannot be decrypted, thereby ensuring the security of the block.

[0101] In this embodiment, the key is generated according to the received public key, the time stamp of receiving the public key and the ID of the current user. In other equivalent embodiments, different algorithms and data can be used for key generation according to actual needs.

[0102] The encrypted file obtained by encrypting the file block with the key is transmitted to the server, so that the server decrypts the encrypted file with the key to obtain the block file.

[0103] Please refer to Figure 2 Embodiment three of the present application is:

[0104] A file transmission method, different from embodiments one or two, is that step S4 is specifically:

[0105] The server receives each file block and stores the file block in the NAS file storage system.

[0106] The file blocks are spliced in the NAS file storage system to obtain the target file.

[0107] The NAS file storage system adopts a random read-write mechanism and filters the file path where the file block is located through a file name matching mechanism.

[0108] In this embodiment, during block uploading, in order to avoid occupying the space of the server by the block, the block is directly stored in the cloud shared NAS server, and the file path where the block is located is filtered through a file name matching mechanism, so as to realize logical hiding for the user.

[0109] During file splicing, the target file to be spliced is directly spliced in the NAS file storage system. Here, the system adopts a file random read-write mechanism and a multi-thread mechanism, and splices the blocks in parallel to improve the efficiency of storage and management.

[0110] Embodiment four of the present application is:

[0111] A file transmission method, different from embodiments one to three, is that in this embodiment, an initial block size is preset, and the browser divides the target file according to the initial block size when the target file is obtained.

[0112] When the block size of the file block to be transmitted is adjusted in real time in step S3, the initial block size is also adjusted, so that the transmission of the next target file can more quickly adapt to the current network state, and the efficiency and stability of block transmission are improved.

[0113] Embodiment five of the present application is:

[0114] A method for file transfer, in this embodiment, a user needs to upload a 4GB video file to a cloud sharing server directly through a browser.

[0115] Step one: the user selects the file in the browser and starts the uploading process.

[0116] Step two: the web page receives the uploaded file, and in this process, the program processes the file according to the written slicing strategy. At the beginning, when there is no file transmission, the file is processed by default according to 15MB per slice, for example, each block is 15MB, and the file block queue is composed in order.

[0117] Step three: create 5 file uploading threads to start loading file blocks from the queue continuously.

[0118] Step four: use the AES algorithm to encrypt the blocks with the key, and after encryption, the blocks are transmitted. Whenever the upload is completed, the transmission speed of the block and the transmission error rate of the block are calculated to obtain the block size to be adjusted. According to the calculated block size, the data at the front of the file is read to continue the transmission of the block until all the data in the file is uploaded. In order to prevent the AES encryption process from causing no file transmission for a period of time, the pipeline technology is adopted to create 5 encryption threads to read the file blocks according to the size of the last block for encryption, and then store them in the sending buffer. The size of the sending buffer is limited to 5 blocks (limited to 5 because the uploading thread is 5, so that 5 threads can perform file transmission at the same time). When the sending buffer is full, no more blocks are encrypted in advance to prevent premature encryption of the blocks from causing the block size adjustment to be not timely.

[0119] Step five: in the file block upload, the system will start to upload these file blocks one by one to the cloud storage service. Each file block is uploaded independently and does not affect each other. In this way, even in the case of unstable network environment, it will not affect the upload of other file blocks. For example, when uploading the 30th file block, network fluctuations occur, causing the 30th block to fail to upload, which will only affect the upload of this file block and will not affect other uploaded or to-be-uploaded file blocks. And the program will automatically re-upload this block without starting to upload the entire file from the beginning.

[0120] Step six: On the cloud sharing server, the server receives these file blocks, first performs a simple check to ensure that the received data size is consistent with the expected size. After completing the size check, the file is decrypted using the AES algorithm to obtain the real block data. The key of each user is obtained from the server memory through USER_ID, so that even if a user's key is leaked, it does not affect the security of other user data, because each user's key is different, thereby improving the security of the data.

[0121] Step seven: When the block is received, the server will splice the file blocks according to the recorded logical relationship of the file blocks. In order to speed up the splicing operation, the RandomFile random read-write method is used to splice the file from any file block; and all operations are directly performed in the NAS file system without copying to the local storage of the cloud sharing server for splicing and then stored in the NAS file system. When splicing is completed, the original 4GB video file is obtained.

[0122] Embodiment six of the present application is:

[0123] A storage medium having a computer program stored thereon, the computer program being executed to implement the steps of the file transmission method of any one of embodiments one to four.

[0124] In summary, the present application provides a file transmission method and storage medium, which divides large files according to the initial block size, transmits the large files through block division, solves the problems existing in the transmission of large files, and corrects the initial block size according to the transmission time and error rate to ensure the efficiency and stability of block transmission. At the same time, the encrypted transmission method is introduced to ensure the security of file transmission. Based on the NAS file system, the data storage and management efficiency is improved.

[0125] 1. Improve the experience of file uploading. Users do not need to download and install any client software, and can directly upload files through a browser on any network-connected device, which greatly improves the convenience of file uploading.

[0126] 2. Eliminate file size restrictions. The present application eliminates the size restrictions on files by using block uploading, allowing users to upload large files exceeding 1GB. This increases the flexibility and convenience of users when handling large files.

[0127] 3. Improve the stability and reliability of file uploading. Through adaptive block size algorithm, error retransmission, block uploading and file checking technology, the present application can ensure the correct and complete uploading of files in unstable network environment. This greatly improves the stability and reliability of file uploading.

[0128] 4. Improve user experience. The technical solution of the present application not only improves the efficiency of file uploading and storage, but also improves the user experience. Users can more conveniently and stably upload large files, and more efficiently manage and access the files stored in the cloud.

[0129] 5. Improve the security of user data. In addition to using traditional SSL certificates to encrypt HTTP requests, the temporary RSA public and private key mechanism is added to ensure the security of user data. The RSA public key obtained by the user each time the user logs in is different, thereby ensuring the security of the synchronization AES key each time. In addition, the AES key is added with a timestamp, which ensures that the key generated each time the user logs in is different, and the sha 256 algorithm is used to protect the security of the key content. Through the above mechanism, the security of the user data is guaranteed, and even if all the data is intercepted, it will not be stolen.

[0130] The above is only an embodiment of the present application, and does not limit the patent scope of the present application. Any equivalent transformation, direct or indirect application in related technical fields using the content of the present application specification and drawings are also included in the patent protection scope of the present application.

Claims

1. A method of file transfer, characterized by, The method comprises the steps of: S1, obtaining a target file to be transmitted at the browser end, dividing the target file into blocks to obtain file blocks of a block size, and transmitting the file blocks; S2, periodically calculating the transmission time and error rate of the file blocks at the browser end, and calculating a block correction size according to the transmission time and error rate; The calculation of the block correction size according to the transmission time and error rate is specifically: ； ； ； Wherein, s1 represents the transmission speed, M represents the current block size, t represents the transmission time, s2 represents the error rate, e represents the transmission error block speed, c represents the total number of blocks transmitted, B represents the block correction size, K and a represent the calibration factor, and are preset constants; S3, adjusting the block size of the file blocks to be transmitted in real time according to the block correction size at the browser end; When transmitting the file blocks, the block information of the target file is transmitted synchronously; The method further comprises the steps of: S4, receiving each file block at the server end, and splicing the file blocks according to the block information uploaded synchronously with the file blocks to obtain the original target file; Step S4 is specifically: The server end receives each file block and stores the file blocks in a NAS file storage system; The file blocks are spliced in the NAS file storage system to obtain the target file.

2. The method of claim 1, wherein, The transmission of the file blocks in step S1 adopts encrypted transmission.

3. The method of claim 2, wherein, The encrypted transmission comprises the steps of: Requesting a public key from the server end, generating a key after receiving the public key returned by the server end, encrypting the key with the public key, and sending the encrypted key to the server end, so that the server end decrypts the key with a private key to obtain the key; The encrypted file obtained by encrypting the file blocks with the key is transmitted to the server end, so that the server end decrypts the encrypted file with the key to obtain the block file.

4. The method of claim 3, wherein, The generation of the key is specifically: The key is generated by combining the received public key, the time stamp of receiving the public key, and the ID of the current user.

5. The method of claim 1, wherein, The NAS file storage system adopts a random read-write mechanism and filters the file path where the file blocks are located through a file name matching mechanism.

6. The method of claim 1, wherein, Step S3 is specifically: The browser end determines whether the block correction size is greater than a preset upper limit of the block size, and adjusts the block size of the file blocks to be transmitted according to the upper limit of the block size if the block correction size is greater than the upper limit of the block size; Determine whether the block correction size is less than a preset lower limit of the block size, and adjust the block size of the file blocks to be transmitted according to the lower limit of the block size if the block correction size is less than the lower limit of the block size; If the block correction size is greater than the lower limit of the block size and less than the upper limit of the block size, adjust the block size of the file blocks to be transmitted according to the block correction size.

7. A storage medium having stored thereon a computer program, characterized in that The computer program is executed to realize the steps in the file transmission method of any one of claims 1-6.

Images