File transmission method, file transmission system, electronic device, and storage medium

Abstract

The present disclosure relates to the technical field of big data, and specifically discloses a file transmission method, a file transmission system, an electronic device and a storage medium. The method comprises the following steps: receiving a plurality of slice files of a first to-be-received file sent by a file sending end and original slice summary information of a plurality of split data segments of the plurality of slice files in the first to-be-received file; generating slice sequence information corresponding to the plurality of slice files according to the slice summary information of the plurality of slice files and the original slice summary information of the plurality of split data segments; converting the plurality of slice files into a second to-be-received file according to the slice sequence information; and if second file summary information of the second to-be-received file matches first file summary information of the first to-be-received file, determining the second to-be-received file as a target received file. The method can effectively avoid the situation of accidental loss of file data content or disorder of slice file sequence in the file transmission process, and ensures the stability of file transmission.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of big data, and in particular to a file transmission method, a file transmission system, an electronic device, a storage medium and a computer program product. BACKGROUND

[0002] In daily file transmission, the file content usually contains various data such as customer information, contract information and system information. The various data described above are mostly key information or related data involving key content. In the file transmission process, a complex network environment needs to be passed through. Therefore, how to ensure the stability of file transmission and the integrity of the file after transmission is particularly important in the process of data or file transmission. If file data is missing or incomplete due to the transmission process, unpredictable resource loss will be caused.

[0003] However, in the file transmission method adopted in the traditional technical solution, when a large data file transmission process is performed, on the one hand, due to the large amount of file data, the file transmission time is long, which increases the instability of file transmission. On the other hand, in the file transmission process, due to various unstable factors in the transmission network environment, in the case of terminal or delay, the data file transmission will be incomplete. In summary, the stability of the file transmission method or method adopted in the traditional technical solution is relatively low. SUMMARY

[0004] Therefore, it is necessary to provide a file transmission method, system, electronic device, computer readable storage medium and computer program product with higher stability for the above technical problems.

[0005] In a first aspect, the present application provides a file transmission method applied to a file receiving end, the method comprising:

[0006] receiving a plurality of slice files of a first to-be-received file and original segment digest information of a plurality of divided data segments corresponding to the plurality of slice files in the first to-be-received file sent by a file sending end;

[0007] generating slice sequence information corresponding to the plurality of slice files according to the slice digest information of the plurality of slice files and the original segment digest information of the plurality of divided data segments;

[0008] converting the plurality of slice files into a second to-be-received file according to the slice sequence information;

[0009] if second file digest information of the second to-be-received file matches first file digest information of the first to-be-received file, determining the second to-be-received file as a target received file.

[0010] In one of the embodiments, the receiving the multiple slice files of the first to-be-received file sent by the file sending end comprises:

[0011] According to the slice file information corresponding to the first to-be-received file and the number of slice files, multiple file receiving tasks are created;

[0012] The multiple file receiving tasks are executed by calling multiple task threads, and the multiple slice files of the first to-be-received file sent by the file sending end are received, wherein the number of the task threads corresponds to the number of unfinished file receiving tasks in the process of receiving the slice files.

[0013] In one of the embodiments, the method further comprises: if the number of threads of the completed task threads corresponds to the number of tasks corresponding to the multiple file receiving tasks, generating slice summary information corresponding to the multiple slice files.

[0014] In one of the embodiments, the generation mode of the second file summary information of the second to-be-received file comprises at least one of the following:

[0015] According to the slice sequence information, the slice summary information corresponding to the multiple slice files is converted into the second file summary information of the second to-be-received file;

[0016] The file data of the second to-be-received file is subjected to summary information operation to obtain the second file summary information of the second to-be-received file.

[0017] In a second aspect, the application provides another file transmission method applied to a file sending end, which comprises:

[0018] Sending, to a file receiving end, multiple slice files of a first to-be-received file and original slice summary information of multiple divided data segments corresponding to the multiple slice files in the first to-be-received file, wherein the file receiving end is configured to convert the multiple slice files into a second to-be-received file according to slice sequence information, and the slice sequence information is generated according to slice summary information of the multiple slice files and the original slice summary information of the multiple divided data segments;

[0019] Sending, to the file receiving end, first file summary information of the first to-be-received file, wherein the file receiving end is further configured to determine the second to-be-received file as a target received file in the case that second file summary information of the second to-be-received file matches the first file summary information of the first to-be-received file.

[0020] In one of the embodiments, the generation mode of the multiple slice files of the first to-be-received file comprises:

[0021] If the file data amount of the first to-be-received file is greater than a preset splitting threshold, the first to-be-received file is split to obtain a plurality of split data segments, and original segment summary information corresponding to the plurality of split data segments is generated;

[0022] According to the original segment summary information, the plurality of split data segments are encapsulated respectively to generate a plurality of slice files of the first to-be-received file.

[0023] In one of the embodiments, the method further comprises:

[0024] determining a file transmission mode corresponding to the first to-be-received file, and determining network element span information of the first to-be-received file in a transmission process according to the file transmission mode;

[0025] determining a splitting threshold corresponding to the first to-be-received file according to the network element span information.

[0026] In a third aspect, the present application further provides a file transmission system. The system comprises:

[0027] a file sending end configured to send, to a file receiving end, a plurality of slice files of a first to-be-received file and original segment summary information of a plurality of split data segments corresponding to the plurality of slice files in the first to-be-received file;

[0028] a file receiving end configured to receive, from a file sending end, a plurality of slice files of a first to-be-received file and original segment summary information of a plurality of split data segments corresponding to the plurality of slice files in the first to-be-received file; generate slice sequence information corresponding to the plurality of slice files according to slice summary information of the plurality of slice files and the original segment summary information of the plurality of split data segments; and convert the plurality of slice files into a second to-be-received file according to the slice sequence information;

[0029] The file sending end is further configured to send first file summary information of the first to-be-received file to the file receiving end.

[0030] The file receiving end is further configured to determine the second to-be-received file as a target received file in a case where second file summary information of the second to-be-received file matches the first file summary information of the first to-be-received file.

[0031] In one of the embodiments, the file receiving end is further configured to create a plurality of file receiving tasks according to the slice file information corresponding to the first to-be-received file and the number of slice files; and invoke a plurality of task threads to execute the plurality of file receiving tasks to receive a plurality of slice files of the first to-be-received file sent by the file sending end, wherein the number of the task threads corresponds to the number of unfinished file receiving tasks in the process of receiving the slice files.

[0032] In one of the embodiments, the file receiving end is further configured to generate slice digest information corresponding to the plurality of slice files if the number of threads of the finished task threads corresponds to the number of tasks corresponding to the plurality of file receiving tasks.

[0033] In one of the embodiments, the file receiving end is further configured to generate the second file digest information of the second to-be-received file in any one of the following manners: converting the slice digest information corresponding to the plurality of slice files into the second file digest information of the second to-be-received file according to the slice sequence information; and performing digest information operation on file data of the second to-be-received file to obtain the second file digest information of the second to-be-received file.

[0034] In one of the embodiments, the file sending end is further configured to perform slicing on the first to-be-received file to obtain a plurality of sliced data segments if the file data amount of the first to-be-received file is greater than a preset slicing threshold, and generate original segment digest information corresponding to the plurality of sliced data segments; and encapsulate the plurality of sliced data segments respectively according to the original segment digest information to generate a plurality of slice files of the first to-be-received file.

[0035] In one of the embodiments, the file sending end is further configured to determine a file transmission mode corresponding to the first to-be-received file, and determine network element span information of the first to-be-received file in the transmission process according to the file transmission mode; and determine a slicing threshold corresponding to the first to-be-received file according to the network element span information.

[0036] In a fourth aspect, the present application further provides an electronic device. The electronic device includes a memory and a processor, the memory stores a computer program, and the processor implements the method provided in the first aspect or the second aspect when executing the computer program.

[0037] In a fifth aspect, the present application further provides a computer readable storage medium. The computer readable storage medium stores a computer program, and the computer program is executed by a processor to implement the method provided in the first aspect or the second aspect.

[0038] In a sixth aspect, the present application also provides a computer program product. The computer program product comprises a computer program which, when executed by a processor, implements the method provided in the first aspect or the second aspect.

[0039] The file transmission method, device, electronic device, storage medium and computer program product described above store the original file in the form of slices. In the file receiving process, a plurality of slice files are first received, and the file transmission efficiency is effectively improved by the slice-based file transmission method. In the process of receiving the slice files, the original slice digest information corresponding to the plurality of split data segments of the plurality of slice files in the first to-be-received file is also received. Then, the current slice digest information of each slice file is compared and verified with the original slice digest information, so as to ensure that the file data in the obtained slice file is not missing and has not been tampered with. Further, the plurality of slice files are combined based on the slice sequence information corresponding to the slice digest information to form a second to-be-received file. The second to-be-received file is verified, and the second file digest information corresponding to the second to-be-received file is compared and verified with the first file digest information of the first to-be-received file. In the case where the two digest information is determined to be the same, the initial file is determined as the target file to be finally obtained according to the file transmission instruction. Through the two integrity verifications based on the digest information of the slice file and the initial file formed by combination, the accidental loss of file data content in the file transmission process can be effectively avoided, and the stability of the file transmission is ensured. In addition, the verification process of the converted file can also avoid the sequence disorder of the slice files in the process of combining and restoring the slice files, and further improve the stability of the file transmission and restoration process. BRIEF DESCRIPTION OF DRAWINGS

[0040] Figure 1 An application environment diagram of the file transmission method in an embodiment;

[0041] Figure 2 A flowchart of the file transmission method executed by the file receiving end in an embodiment;

[0042] Figure 3 A flowchart of the sub-steps of forming a plurality of slice files in an embodiment;

[0043] Figure 4 A flowchart of the file transmission method executed by the file sending end in an embodiment;

[0044] Figure 5 An interaction diagram between the file sending end and the file receiving end in an embodiment;

[0045] Figure 6A structural block diagram of a file transmission system in an embodiment;

[0046] Figure 7 An internal structural diagram of an electronic device in an embodiment. DETAILED DESCRIPTION

[0047] For the purpose, technical solutions and advantages of the present application to be clearer, the present application is further described in detail below in combination with the drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and do not limit the present application.

[0048] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data for analysis, stored data, displayed data, etc.) involved in the present application are all information and data authorized by the user or authorized by all parties, and the collection, use and processing of related data need to comply with relevant laws, regulations and standards of relevant countries and regions.

[0049] As pointed out in the background, the file transmission method or file receiving method used in the traditional technical solution has the following problems: on the one hand, due to the large amount of file data, the file transmission time is long, which increases the instability of file transmission; on the other hand, in the file transmission process, due to various unstable factors in the transmission network environment, in the case of terminal or delay, it will cause the data file transmission to be incomplete.

[0050] In view of the foregoing technical problems or defects, the file transmission method provided by the embodiments of the present application can be applied in the application environment as shown in Figure 1 In the application environment, it mainly includes that the file receiving end 102 communicates with the file sending end 106 and transmits and interacts file data through the service forwarding layer 104. The service forwarding layer 104 is used to forward various request information between the file receiving end 102 and the file sending end 106, and realize data forwarding, exchange and policy control, while providing high available bandwidth to ensure fair sharing between various access devices and optimize network resource utilization. The data storage system is provided in the file sending end 106, which can store various types of file data. The data storage system can be integrated on the file sending end 106, or placed on the cloud or other network servers, or implemented by using an independent server or a server cluster composed of multiple servers. The file receiving end 102 can be, but is not limited to, various personal computers, notebook computers, smart phones, tablet computers, Internet of Things devices and portable wearable devices. The Internet of Things device can be a smart speaker, a smart TV, a smart air conditioner, a smart vehicle device, etc. The portable wearable device can be a smart watch, a smart bracelet, a head-mounted device, etc.

[0051] InFigure 1 In the application environment shown, before the file sending end 106 performs file transmission to the file receiving end 102, the file sending end 106 can slice the file to be sent, i.e., the first to-be-received file, to form a plurality of slice files according to the network condition of the transmission network and the bandwidth occupation state in the service forwarding layer 104; and in the process of encapsulating the plurality of slice data segments formed after slicing to form slice files, the original segment digest information corresponding to each slice data segment is determined through digest information calculation. After the file sending end 106 completes slicing, digest calculation, and file encapsulation of the file, the plurality of slice files are sent to the service forwarding layer 104 in a parallel sending manner.

[0052] On the file receiving end 102 side, after the file information of the first to-be-received file is determined, the file receiving end 102 obtains the plurality of slice files of the first to-be-received file and the original segment digest information corresponding to each slice file in a parallel download receiving manner from the server forwarding layer 104 by calling a file download interface. After receiving the slice files, the file receiving end 102 performs first-stage file data integrity verification, generates slice digest information according to the file data content of the received slice files by using the same digest algorithm as that of the file sending end 106, and compares and verifies the slice digest information with the received original segment digest information. In the case where the two pieces of digest information match, it can be confirmed that the data content of the slice file is complete and has not been tampered with.

[0053] Then, in the case where the file receiving end 102 confirms that the two pieces of digest information match, the file receiving end 102 parses each slice file to obtain slice sequence information corresponding to the slice file, and integrates the file data content in each slice file based on the slice sequence information to obtain an initial complete file, denoted as a second to-be-received file. At the same time, the file receiving end 102 sends information of initial verification success to the file sending end 106 through the service forwarding layer 104 to trigger subsequent actions of the file sending end 106.

[0054] Based on the information of initial verification success, the file sending end 106 sends the first file digest information of the original file, i.e., the first to-be-received file, to the file receiving end 102 through the service forwarding layer 104. After receiving the first file digest information, the file receiving end 102 starts second-stage file data integrity verification, compares the second file digest information of the second to-be-received file with the received first file digest information, and judges whether the file formed by combining the plurality of slice files has omission of file data content and confusion of slice sequence. In the case where the two pieces of digest information match each other, the file formed by combining the plurality of slice files is determined as the final target received file.

[0055] In one embodiment, as Figure 2As shown, a file transmission method is provided, and the method is applied to Figure 1 Taking the file receiving end 102 in the method as an example, the method includes the following steps:

[0056] In step 202, the file receiving end receives the first plurality of slice files of the first to-be-received file and the original segment digest information of the plurality of split data segments corresponding to the plurality of slice files in the first to-be-received file sent by the file sending end.

[0057] In an embodiment, the first to-be-received file is an original file sent by the file sending end and needs to be successfully received by the file receiving end. In the embodiment, the slice file is a file formed by performing data encapsulation processing on the original file after the file data of the original file is split into a plurality of split data segments according to a predetermined file data size. In the embodiment, the original segment digest information is obtained by performing an algorithm on the data content in each segment based on the data content in each segment after the original file is sliced into a plurality of split data segments. More specifically, the generation process of the digest information in the embodiment is a process of converting data of an arbitrary length into a hash value of a fixed length by a specific algorithm. Since the digest information has the characteristics of irreversibility and rapidity, the digest information formed can be used in the process of verifying the integrity of the file data.

[0058] Illustratively, before starting the file transmission, the file sending end and the file receiving end can perform information interaction of instructions or messages. In the process of information interaction, the file receiving end can query all file information stored in the file sending end, and the file sending end can also explicitly indicate the specific download requirements of the file receiving end. Specifically, after the file receiving end explicitly indicates the file information of the to-be-received file through information interaction with the file sending end, the file receiving end initiates a file transmission request to the file receiving end to download and transmit the file. The file sending end sends all slice files associated with the file information to the file receiving end in a parallel transmission manner according to the file transmission request and the file information. When the to-be-received file is split into a plurality of slice files, the MD5 value of each slice file is generated as the original segment digest information of the slice file by performing an MD5 message digest algorithm on the split data segment in each slice file, and the original segment digest information is associated with the corresponding slice file. The original segment digest information is transmitted to the file receiving end together with the slice file in the slice file transmission process, and is received by the file receiving end.

[0059] In step 204, the slice sequence information corresponding to the plurality of slice files is generated according to the slice digest information of the plurality of slice files and the original segment digest information of the plurality of split data segments.

[0060] In the embodiment, the slice summary information is obtained by performing the same information summary algorithm as that in the file sending end on all data contents in each slice file after the file receiving end successfully receives the slice file. In addition, the slice sequence information in the embodiment is used to describe the combination order or arrangement logic when the file data in the slice file is combined and restored. Specifically, the slice sequence information in the embodiment can be determined based on the sequence identification information carried in the slice file; further, the sequence identification information is identification information given to each divided data segment according to the front-back order of each divided data segment in the original file content when the first to-be-received file (original file) is divided, and the identification information is encapsulated in the slice file along with the divided data segment.

[0061] For example, in the embodiment, the integrity of the data contents in the slice file needs to be verified after the slice file is successfully received. First, the slice file needs to be transcoded and parsed to remove the format information and transmission information in the file, and to retain the body data contents in the slice file and the sequence identification information corresponding to the data contents. Then, it is needed to determine whether the body data contents in the slice file are strictly consistent with the divided data segments formed by the file receiving end to determine whether the data contents are missing or tampered during transmission; specifically, the MD5 value of the body data in the slice file is calculated by using the same MD5 algorithm as that in the file sending end, and is taken as the slice summary information to compare with the received original segment summary information; if the contents of the two summary information match or are completely consistent, it is determined that the body data contents in the slice file are completely the same as the divided data segments, that is, there is no missing or tampering of the data contents in the transmission process of the slice file. After the foregoing integrity verification is completed and passed, the sequence identification information in each slice file is further extracted, and the respective sequence identification information of the plurality of slice files is sorted to form the slice sequence information as a reference basis for the file restoration process.

[0062] In addition, if it is determined in the integrity verification stage that the two summary information do not match, the file receiving end re-sends the file download receiving request to the file sending end, so that the file sending end re-sends the slice file with missing or tampered data.

[0063] In step 206, the plurality of slice files are converted into the second to-be-received file according to the slice sequence information.

[0064] In the embodiment, the second to-be-received file is a complete file formed by sorting and integrating the data contents in each slice file according to the sequence based on the slice sequence information.

[0065] Exemplarily, after determining that there is no data omission or tampering in each of the successfully received slice files through the comparison and verification of the slice summary information and the original slice summary information in the embodiment, the parsing processing of the slice files according to the foregoing steps is performed to obtain the sequence identification information corresponding to the data content in each slice file, and based on the arrangement order of each piece of data content in the original file (i.e., the first to-be-received file) described by the sequence identification information, the arrangement order is taken as the slice sequence information in file restoration, and the data contents in all slice files are integrated to obtain a preliminarily complete second to-be-received file.

[0066] In step 208, if the second file summary information of the second to-be-received file matches the first file summary information of the first to-be-received file, the second to-be-received file is determined as the target received file.

[0067] In the embodiment, the first file summary information and the second file summary information are both data characteristic information capable of uniquely representing the data content of the file, and the data characteristic information also has irreversibility, so as to be used as a reference basis for verifying the integrity of the file. More specifically, in the embodiment, the first file summary information is the summary information corresponding to the complete data content of the original file obtained by performing an operation on the original file (the first to-be-received file) stored at the file sending end by using a summary algorithm. The second file summary information in the embodiment can be the summary information obtained by performing an operation on the initial complete file formed by integrating the data content in the slice files by using the same summary algorithm, or can be the summary information formed by combining the slice summary information corresponding to each slice file according to the foregoing slice sequence information.

[0068] Exemplarily, after the second to-be-received file, i.e., the initial complete file, is formed by combining and splicing the data contents in the plurality of slice files, the process of the second-stage integrity verification is entered; the second-stage integrity verification is used to verify whether the combination order in the slice file combination process is correct and whether the complete file formed by the combination exists data loss. Before the verification, the file receiving end first needs to receive the first file digest information of the first to-be-received file sent by the file sending end; at the same time, the file receiving end also needs to generate the second file digest information corresponding to the second to-be-received file based on the second to-be-received file formed by the combination. Since the digest algorithm used in the embodiment at the file sending end is the MD5 algorithm, the first file digest information formed is a specific MD5 value; similarly, the second file digest information generated at the file receiving end is also a specific MD5 value. The two MD5 values are directly compared, if the same, it is determined that the complete file formed by the combination, i.e., the second to-be-received file, is completely consistent with the original file of the file sending end, and there is no situation of sequence disorder and data loss, and the second to-be-received file is determined as the target received file that the file receiving end needs to download and receive.

[0069] On the contrary, if the two MD5 values are not the same, it is determined that the second to-be-received file is not consistent with the original file of the file sending end, and there may be an error operation in the process of forming the second to-be-received file by the combination of the slice files, or the sequence identification information carried in the slice file may exist an error. Therefore, the file receiving end needs to download the slice file again or return to the combination step of the slice file to form the second to-be-received file again.

[0070] In the file transmission method, the original file is stored in the form of slices, and in the file receiving process, a plurality of slice files are first received, and the file transmission efficiency is effectively improved through the slice file transmission method. In the process of receiving the slice files, the original slice summary information corresponding to the plurality of split data segments in the first to-be-received file is also received. Then, the current slice summary information of each slice file is compared and verified with the original slice summary information to ensure that the file data in the obtained slice file is not missing and has not been tampered with. Further, the plurality of slice files are combined based on the slice sequence information corresponding to the slice summary information to form a second to-be-received file. The second to-be-received file formed by the combination is verified, and the second file summary information corresponding to the second to-be-received file is compared and verified with the first file summary information of the first to-be-received file. In the case where the two summary information is determined to be the same, the initial file is determined as the target file to be obtained by the file transmission instruction. Through the two integrity verifications based on the summary information of the slice file and the initial file formed by the combination, the accidental loss of file data content in the file transmission process can be effectively avoided, and the stability of the file transmission is ensured. In addition, the verification process of the converted file can also avoid the sequence disorder of the slice files in the process of combining and restoring the slice files, further improving the stability of the file transmission and restoration process.

[0071] In one embodiment, as shown in Figure 3 the foregoing method can include the following steps in the process of receiving the plurality of slice files of the first to-be-received file sent by the file sending end:

[0072] Step 302, creating a plurality of file receiving tasks according to the slice file information corresponding to the first to-be-received file and the number of slice files.

[0073] In an embodiment, the file receiving task refers to a download task created by calling the file download interface provided by the service forwarding layer according to the number of slice files to be downloaded after the file receiving end determines the complete file information or associated slice file information that needs to be received. In addition, the slice file information in the embodiment is used to describe the specific attribute characteristics of the slice file, and the attribute characteristics include but are not limited to the file name, file source, sequence information, and summary information of each slice file.

[0074] Exemplarily, in the embodiments, the Hyper Text Transfer Protocol (HTTP) can be used to realize data interaction between the file receiving end, the file sending end and the service forwarding layer, to realize file downloading and uploading. In the file receiving end, the HTTP file stream sent by the file sending end and forwarded by the service forwarding layer is received, and the file stream is saved to the local temporary directory of the file receiving end. Further, the file receiving end obtains the slice file information associated with the first to-be-received file and the number of slice files according to the file information in the HTTP file stream. Specifically, the file receiving end performs preliminary analysis and information extraction on the file stream, obtains the file information corresponding to the first to-be-received file (complete original file), further analyzes the file information, and identifies the slice file information associated with the first to-be-received file and the number of slice files. Then, a file receiving task is created based on each slice file, and the number of finally created file receiving tasks is consistent with the number of slice files.

[0075] In step 304, a plurality of task threads are called to execute a plurality of file receiving tasks to receive a plurality of slice files of the first to-be-received file sent by the file sending end.

[0076] In the embodiments, the task thread refers to an execution unit in a process in the file receiving end, which is used to independently execute different tasks. More specifically, the file receiving end in the embodiments is configured with a thread pool, and a plurality of callable threads are configured in the thread pool, which can perform file downloading according to the assigned file receiving tasks.

[0077] Exemplarily, the file receiving end in the embodiments is configured with a task scheduler, which is used to schedule and manage the slice file downloading tasks, i.e., the file receiving tasks. Specifically, the task scheduler can batch query the to-be-processed slice file receiving tasks from the task list of the local cache space of the file receiving end, and then distribute the to-be-processed tasks to the execution threads in the thread pool. In addition, the execution thread pool set in the file receiving end can control the number of task execution threads, receive the executable tasks sent by the task scheduler, start thread execution, and control the same number of threads as the number of unfinished file receiving tasks to run. More specifically, the retry mechanism of the execution thread pool can reassign the available threads in the thread pool to the file receiving task to retransmit the slice files in the case that a specific task thread is interrupted due to an abnormal situation.

[0078] In addition, the file receiving end in the embodiment can also be configured with a task processor, which can process the newly received slice files. The task processor can verify the integrity of the file after the download is completed, and if the file is not complete, a log is recorded, which can provide a basis for troubleshooting. If the file is divided into multiple slice files (for example, the first to-be-received file), the slice files in the large file are verified again, and finally the slice files are merged and restored, the file is moved to the appropriate position, and the storage into the warehouse is automatically triggered.

[0079] In the embodiment, each file or slice is encapsulated into a task, and the file transmission request is executed independently. When the network abnormally causes transmission failure or timeout, the task will be suspended and retried, thereby ensuring the reliability of the file download transmission process.

[0080] In an embodiment, in the process of receiving the multiple slice files of the first to-be-received file sent by the file sending end, the following steps can also be included:

[0081] In step 306, if the number of thread of the completed task thread corresponds to the number of tasks corresponding to the multiple file receiving tasks, the slice digest information corresponding to the multiple slice files is generated.

[0082] In the embodiment, the file receiving end counts the number of completed download tasks by initializing a counter before executing the file receiving task. After the number of slice files associated with the first to-be-received file is determined, the number of completed download tasks in the counter will change after the task thread of each slice file completes the download task. Until the number of download tasks in the counter is consistent with the number of slice files associated with the first to-be-received file, the task manager in the file receiving end executes the integration and integrity verification steps of the slice file. In the embodiment, the number of slice files is verified to effectively avoid the loss or omission of slice files in the file transmission, thereby improving the reliability of the parallel transmission process of the slice files.

[0083] In an embodiment, the generation of the second file digest information of the second to-be-received file includes at least one of the following steps:

[0084] In step one, the slice digest information corresponding to the multiple slice files is converted into the second file digest information of the second to-be-received file according to the slice sequence information.

[0085] Specifically in the embodiment, the sequence identification information can be directly obtained in the process of parsing the slice file, and the slice sequence information is formed based on the plurality of sequence identification information. The slice sequence information can not only be used as a basis for integrating the data content in all slice files, but also be used as a basis for synthesizing the digest information. Specifically, after the sequence information of all slice files is determined, the slice digest information corresponding to all slice files is integrated according to the combination order represented by the file sequence information, and the complete digest information formed by the integration is the initial complete file formed by combination, that is, the second file digest information corresponding to the second to-be-received file.

[0086] Step two, performing digest information operation on the file data of the second to-be-received file to obtain the second file digest information of the second to-be-received file.

[0087] Specifically in the embodiment, after the sequence identification information corresponding to the data content in each slice file is obtained, the slice sequence information in the file restoration is formed based on the arrangement order of each piece of data content in the original file described by the sequence identification information, the data content in all slice files is integrated, and a preliminary complete second to-be-received file is obtained. Then, the same information digest algorithm as the file sending end, for example, MD5 algorithm, is used to calculate the MD5 value corresponding to the second to-be-received file as the second digest information.

[0088] Exemplarily, in the process of file transmission in the embodiment, if the device as the file receiving end has certain data operation processing capability, then the digest information operation can be performed on the file data of the complete file after the plurality of slice files are integrated to obtain the initial complete file, so as to obtain the corresponding file digest information. If the data operation processing capability of the device is relatively weak, then the slice digest information corresponding to each slice file can be directly integrated and converted to form the file digest information corresponding to the initial complete file. In the process of integrating the initial complete file, that is, the second file digest information corresponding to the second to-be-received file, different digest information generation modes are adopted, so as to cope with more complex implementation environments and provide reliable digest information support for the integrity verification of the file.

[0089] In another embodiment, as Figure 4 shown, the application also provides a file transmission method. Taking the file sending end 106 in Figure 1 as an example, the method includes the following steps:

[0090] At step 402, the file receiving end is sent the plurality of slice files of the first to-be-received file and the original segment digest information of the plurality of split data segments corresponding to the plurality of slice files in the first to-be-received file, wherein the file receiving end is configured to convert the plurality of slice files into a second to-be-received file according to slice sequence information, and the slice sequence information is generated according to the slice digest information of the plurality of slice files and the original segment digest information of the plurality of split data segments.

[0091] At step 404, the file receiving end is sent the first file digest information of the first to-be-received file, wherein the file receiving end is further configured to determine the second to-be-received file as a target received file in a case where the second file digest information of the second to-be-received file matches the first file digest information of the first to-be-received file.

[0092] For example, after receiving the file to be transmitted and sent, the file sending end in the embodiment first stores the original file in a designated storage space. After interacting with the file receiving end through instruction information, the original file to be transmitted is determined as the first to-be-received file. Further, the file receiving end encapsulates the file information to be transmitted into a designated data format, and sends the file information to the service notification interface of the receiving end. The data format of the file information includes the file name, source, sequence, and MD5 value of the original file. When the file is large, the file is sliced, and the information further includes the slice name of each slice, the MD5 value of each slice, and the slice sequence.

[0093] During the file transmission process, based on the received file information, the file receiving end initiates a file transmission request to the file sending end to perform file transmission and download. First, the integrity of each slice is verified according to the MD5 value of each slice recorded in the file information. After the integrity verification, the slice files are integrated and restored, and the integrity is verified again based on the MD5 value of the original file to ensure the integrity of the file.

[0094] In one embodiment, the generation manner of the plurality of slice files of the first to-be-received file in the above method specifically includes the following steps:

[0095] Step one, if the file data amount of the first to-be-received file is greater than a preset split threshold, the first to-be-received file is split to obtain a plurality of split data segments, and original segment digest information corresponding to the plurality of slice data segments is generated.

[0096] Step two, the plurality of slice data segments are encapsulated according to the original segment digest information to generate the plurality of slice files of the first to-be-received file.

[0097] In an embodiment, the splitting threshold is a criterion for determining whether a file to be transmitted, i.e., the first to-be-received file, needs to be transmitted in the form of slices. The split data segment in the embodiment refers to an independent data segment formed after splitting the file data of the first to-be-received file.

[0098] For example, the splitting threshold in the embodiment can be equivalent to the data size of each split data segment. For example, the maximum data size of each split data segment is 10 MB, and when a file to be transmitted is 120 MB, before transmission, the file sending end first splits the file as the first to-be-received file to obtain 12 slice files, and the data size of each slice file is 10 MB. In addition, the splitting threshold and the data size of each split data segment in the embodiment can be independent of each other. For another example, the splitting threshold in the embodiment is set to 200 MB, that is, when a file to be transmitted exceeds 200 MB, the file needs to be split. At the same time, the split file size in the embodiment is set to 10 MB, so that the data size of each slice file after splitting the foregoing file does not exceed 10 MB. By setting the splitting threshold, the embodiment can strictly split large files, thereby ensuring the efficiency of the file transmission process.

[0099] In an embodiment, the foregoing method further includes the following steps:

[0100] Step 1: determining the file transmission mode corresponding to the first to-be-received file, and determining the network element span information of the first to-be-received file in the transmission process according to the file transmission mode.

[0101] Step 2: determining the splitting threshold corresponding to the first to-be-received file according to the network element span information.

[0102] In an embodiment, the file transmission mode is used to describe the transmission mode between the file receiving end and the file transmission end, and between other network elements. Further, the network element span information in the embodiment is used to represent the distribution and coverage of network elements (such as file sending end, file receiving end, and service transit layer) in the network architecture.

[0103] Exemplarily, the network element span information can be used in the embodiment to determine whether the network condition of file transmission is good or bad. For example, the network element span information can be used to determine whether the current file is transmitted within a local area network or between local area networks; the network condition is relatively better when the file is transmitted within a local area network, and therefore the splitting threshold in the embodiment can be set to be relatively larger, and vice versa. The splitting of the file based on the network condition can make the splitting judgment more accurate and more suitable for the actual needs of the application scenario, and thus make the file transmission process more reliable.

[0104] The accompanying drawings are incorporated in and constitute a part of the specification. Figure 5 The following describes the completion process of the file transmission method completed by the file sending end and the file receiving end in the technical solution of the present application:

[0105] Step one, at the file sending end, the file data to be transmitted is encapsulated according to a predetermined data format. In the encapsulation process, the file data is also MD5 encrypted to form a file digest corresponding to the file data. In the case where the file data needs to be split for storage, the encapsulated file is split, and each file slice includes a slice name and an MD5 segment.

[0106] More specifically, before sending the file data, the file sending end needs to determine the size of the file data based on a preset threshold to determine whether the file needs to be split. Specifically, the preset threshold in the solution can be changed based on different network environment conditions. For example, it is determined whether the file data is transmitted within a local area network or between local area networks. When the file data is transmitted within a local area network (or the network condition is slightly better), the preset threshold can be set to be larger; when the file data is transmitted between local area networks (or the network condition is slightly worse), the preset threshold can be set to be smaller.

[0107] Step two, at the file receiving end, based on the file information to be downloaded, the multiple file slices corresponding to the file information are downloaded. The multiple file slices to be downloaded are determined according to the file information, each file slice is encapsulated into a task, and the task is distributed to a thread pool for execution.

[0108] Step three, after the downloading of all file slices is completed, the file slices are recombined, and a file slice digest is formed based on the MD5 segment corresponding to each file slice; the file digest corresponding to the file information to be downloaded is verified for integrity, and in the case where the integrity is determined, the target file is restored based on the sequence of the slices.

[0109] The integrity verification process performed at the file receiving end specifically includes two stages: in the first stage, after the file receiving end downloads all the slice files corresponding to the file information, the file receiving end performs file digest information (MD5 algorithm) operation based on each slice file downloaded to obtain first digest information. The first digest information is compared with the slice digest information obtained by performing digest information operation on each slice file of the initial file by the file uploading end when uploading the file, to determine that each slice file received is complete. In the second stage, after the integrity of the single slice file is verified, after the file receiving end integrates multiple slice files to form a complete file, the file digest information operation is performed on the complete file to obtain second digest information. Then, the complete digest information obtained by performing digest information operation on the complete file data of the initial file by the file uploading end when uploading the file is obtained. The second digest information is compared and verified with the complete digest information, to determine whether the complete file formed by integration has missing data content, and whether the order of each slice file in the integration process is correct.

[0110] It should be understood that, although each step in the flowchart involved in each of the above embodiments is displayed in sequence according to the arrow, these steps are not necessarily executed in sequence according to the arrow. Unless otherwise specified herein, the execution of these steps is not strictly limited in sequence, and these steps can be executed in other sequences. Moreover, at least part of the steps in the flowchart involved in each of the above embodiments can include multiple steps or multiple stages, which are not necessarily executed at the same time, but can be executed at different times, and the execution sequence of these steps or stages is not necessarily sequential, but can be executed in rotation or alternation with at least part of other steps or steps or stages in other steps.

[0111] Based on the same inventive concept, the embodiments of the present application also provide a file transmission system for implementing the above-mentioned file transmission method. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme described in the above method, so the specific limitations in one or more file transmission system embodiments provided below can refer to the limitations of the file transmission method in the above text, which will not be repeated here.

[0112] In one embodiment, as shown in Figure 6 a file transmission system 600 is provided, comprising a file sending end 601 and a file receiving end 602, wherein:

[0113] The file sending end 601 is configured to send, to the file receiving end 602, multiple slice files of a first to-be-received file and original slice digest information of multiple split data segments corresponding to the multiple slice files in the first to-be-received file.

[0114] The file receiving end 602 is configured to receive the plurality of slice files of the first to-be-received file and the original slice summary information of the plurality of split data segments corresponding to the plurality of slice files in the first to-be-received file sent by the file sending end 601, generate slice sequence information corresponding to the plurality of slice files according to the slice summary information of the plurality of slice files and the original slice summary information of the plurality of split data segments, and convert the plurality of slice files into a second to-be-received file according to the slice sequence information.

[0115] The file sending end 601 is further configured to send first file summary information of the first to-be-received file to the file receiving end 602.

[0116] The file receiving end 602 is further configured to determine the second to-be-received file as a target received file in a case where the second file summary information of the second to-be-received file matches the first file summary information of the first to-be-received file.

[0117] In an embodiment, the file receiving end 602 is further configured to create a plurality of file receiving tasks according to the slice file information corresponding to the first to-be-received file and the number of slice files, and call a plurality of task threads to execute the plurality of file receiving tasks to receive the plurality of slice files of the first to-be-received file sent by the file sending end, wherein the number of task threads corresponds to the number of unfinished file receiving tasks in the process of receiving the slice files.

[0118] In an embodiment, the file receiving end 602 is further configured to generate the slice summary information corresponding to the plurality of slice files if the number of threads of the completed task threads corresponds to the number of tasks corresponding to the plurality of file receiving tasks.

[0119] In an embodiment, the file receiving end 602 is further configured to generate the second file summary information of the second to-be-received file in any one of the following manners: converting the slice summary information corresponding to the plurality of slice files into the second file summary information of the second to-be-received file according to the slice sequence information, and performing summary information operation on the file data of the second to-be-received file to obtain the second file summary information of the second to-be-received file.

[0120] In an embodiment, the file sending end 601 is further configured to split the first to-be-received file into a plurality of split data segments if the file data amount of the first to-be-received file is greater than a preset split threshold, generate original slice summary information corresponding to the plurality of slice data segments, and encapsulate the plurality of slice data segments respectively according to the original slice summary information to generate the plurality of slice files of the first to-be-received file.

[0121] In an embodiment, the file sending end 601 is further configured to determine a file transmission mode corresponding to the first to-be-received file, and determine network element span information of the first to-be-received file in a transmission process according to the file transmission mode; and determine a splitting threshold corresponding to the first to-be-received file according to the network element span information.

[0122] The modules in the file transmission system can be implemented by software, hardware, or a combination thereof. The modules can be embedded in or independent of a processor in an electronic device in hardware form, or stored in a memory in the electronic device in software form, so as to be invoked and executed by the processor to perform operations corresponding to the modules.

[0123] In an embodiment, an electronic device, which can be a terminal, has an internal structure as shown in Figure 7 The electronic device includes a processor, a memory, an input / output interface, a communication interface, a display unit, and an input device. The processor, the memory, and the input / output interface are connected through a system bus, and the communication interface, the display unit, and the input device are connected to the system bus through the input / output interface. The processor of the electronic device is configured to provide computing and control capabilities. The memory of the electronic device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operating system and the computer program in the non-volatile storage medium to run. The input / output interface of the electronic device is configured to exchange information between the processor and external devices. The communication interface of the electronic device is configured to perform wired or wireless communication with external terminals. The wireless communication can be achieved through WIFI, mobile cellular network, NFC (Near Field Communication), or other technologies. The computer program is executed by the processor to implement a file transmission method. The display unit of the electronic device is configured to form a visually visible picture, which can be a display screen, a projection device, or a virtual reality imaging device. The display screen can be a liquid crystal display screen or an electronic ink display screen. The input device of the electronic device can be a touch layer overlaid on the display screen, or a key, trackball, or touchpad arranged on the housing of the electronic device, or an external keyboard, touchpad, or mouse, etc.

[0124] Those skilled in the art can understand that Figure 7 The structure shown in the above embodiment is only a block diagram of part of the structure related to the scheme of the present application, and does not limit the electronic device to which the scheme of the present application is applied. The specific electronic device can include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0125] In an embodiment, an electronic device is also provided, including a memory and a processor, the memory storing a computer program, and the processor implementing the steps in the above-mentioned method embodiments when executing the computer program.

[0126] In an embodiment, a computer-readable storage medium is provided, storing a computer program, which, when executed by a processor, implements the steps in the above-mentioned method embodiments.

[0127] In an embodiment, a computer program product is provided, including a computer program, which, when executed by a processor, implements the steps in the above-mentioned method embodiments.

[0128] A person of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiments can be completed by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium and, when executed, can include the processes of the above-mentioned embodiments. Any reference to a memory, database, or other medium used in the embodiments provided in the present application can include at least one of a non-volatile and volatile memory. The non-volatile memory can include a read-only memory (ROM), a magnetic tape, a floppy disk, a flash memory, an optical storage, a high-density embedded non-volatile memory, a resistive memory (ReRAM), a magnetoresistive random access memory (MRAM), a ferroelectric memory (FRAM), a phase change memory (PCM), a graphene memory, etc. The volatile memory can include a random access memory (RAM) or an external cache memory, etc. As an illustration but not limitation, the RAM can be in various forms, such as a static random access memory (SRAM) or a dynamic random access memory (DRAM), etc. The database involved in the embodiments provided in the present application can include at least one of a relational database and a non-relational database. The non-relational database can include a distributed database based on a blockchain, etc., without being limited thereto. The processor involved in the embodiments provided in the present application can be a general-purpose processor, a central processing unit, a graphics processing unit, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., without being limited thereto.

[0129] Any combination of the technical features in the above embodiments can be made, and for the sake of brevity, not all possible combinations are described, however, any combination of the technical features is deemed to be within the scope of the present disclosure as long as there is no inconsistency.

[0130] The above embodiments only express several implementation manners of the present application, and the description is relatively specific and detailed, but it should not be understood as a limitation on the patent scope of the present application. It should be pointed out that for ordinary skilled persons in the art, several modifications and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the appended claims.

Claims

1. A file transfer method characterized by, The method applied to a file receiving end comprises: receiving a plurality of slice files of a first to-be-received file and original slice abstract information of a plurality of divided data segments corresponding to the plurality of slice files in the first to-be-received file sent by a file sending end; generating slice sequence information corresponding to the plurality of slice files according to slice abstract information of the plurality of slice files and the original slice abstract information of the plurality of divided data segments; converting the plurality of slice files into a second to-be-received file according to the slice sequence information; if second file abstract information of the second to-be-received file matches first file abstract information of the first to-be-received file, determining the second to-be-received file as a target received file.

2. The method of claim 1, wherein, The plurality of slice files of the first to-be-received file sent by the file sending end comprises: creating a plurality of file receiving tasks according to slice file information corresponding to the first to-be-received file and a slice file quantity; calling a plurality of task threads to execute the plurality of file receiving tasks to receive the plurality of slice files of the first to-be-received file sent by the file sending end, wherein a quantity of the task threads corresponds to a quantity of unfinished file receiving tasks in a slice file receiving process.

3. The method of claim 2, wherein, The method further comprises: if a quantity of threads of the completed task threads corresponds to a quantity of tasks corresponding to the plurality of file receiving tasks, generating slice abstract information corresponding to the plurality of slice files.

4. The method of claim 1, wherein, The generation mode of the second file abstract information of the second to-be-received file at least comprises one of the following: converting slice abstract information corresponding to the plurality of slice files into the second file abstract information of the second to-be-received file according to the slice sequence information; performing abstract information operation on file data of the second to-be-received file to obtain the second file abstract information of the second to-be-received file.

5. A file transfer method characterized by, The method applied to a file sending end comprises: sending a plurality of slice files of a first to-be-received file and original slice abstract information of a plurality of divided data segments corresponding to the plurality of slice files in the first to-be-received file to a file receiving end, wherein the file receiving end is configured to convert the plurality of slice files into a second to-be-received file according to slice sequence information, and the slice sequence information is generated according to slice abstract information of the plurality of slice files and the original slice abstract information of the plurality of divided data segments; sending first file abstract information of the first to-be-received file to the file receiving end, wherein the file receiving end is further configured to determine the second to-be-received file as a target received file in a case that second file abstract information of the second to-be-received file matches the first file abstract information of the first to-be-received file.

6. The method of claim 5, wherein, The generation mode of the plurality of slice files of the first to-be-received file comprises: if a file data quantity of the first to-be-received file is greater than a preset division threshold, dividing the first to-be-received file to obtain a plurality of divided data segments and generating original slice abstract information corresponding to the plurality of divided data segments; According to the original segment summary information, the multiple segmented data segments are encapsulated respectively to generate multiple slice files of the first to-be-received file.

7. The method of claim 6, wherein, The method further comprises: determining a file transmission mode corresponding to the first to-be-received file, and determining network element span information of the first to-be-received file in a transmission process according to the file transmission mode; determining a segmentation threshold corresponding to the first to-be-received file according to the network element span information.

8. A file transfer system characterized by comprising: The system comprises: a file sending end configured to send, to a file receiving end, multiple slice files of a first to-be-received file and original segment summary information of multiple segmented data segments corresponding to the multiple slice files in the first to-be-received file; a file receiving end configured to receive the multiple slice files of the first to-be-received file and the original segment summary information of the multiple segmented data segments corresponding to the multiple slice files in the first to-be-received file sent by the file sending end, generate slice sequence information corresponding to the multiple slice files according to slice summary information of the multiple slice files and the original segment summary information of the multiple segmented data segments, and convert the multiple slice files into a second to-be-received file according to the slice sequence information. The file sending end is further configured to send first file summary information of the first to-be-received file to the file receiving end. The file receiving end is further configured to determine the second to-be-received file as a target received file in a case where second file summary information of the second to-be-received file matches the first file summary information of the first to-be-received file.

9. The system of claim 8, wherein, The file receiving end is further configured to create multiple file receiving tasks according to slice file information corresponding to the first to-be-received file and a number of slice files, and call multiple task threads to execute the multiple file receiving tasks to receive the multiple slice files of the first to-be-received file sent by the file sending end, wherein the number of the task threads corresponds to a number of unfinished file receiving tasks in a receiving process of the slice files.

10. The system of claim 9, wherein, The file receiving end is further configured to generate slice summary information corresponding to the multiple slice files in a case where a number of threads of the completed task threads corresponds to a number of tasks corresponding to the multiple file receiving tasks.

11. The system of claim 8, wherein, The file receiving end is further configured to generate second file summary information of the second to-be-received file in any one of the following manners: convert the slice summary information corresponding to the multiple slice files into the second file summary information of the second to-be-received file according to the slice sequence information; or perform summary information operation on file data of the second to-be-received file to obtain the second file summary information of the second to-be-received file.

12. The system of claim 8, wherein, The file sending end is further configured to perform segmentation on the first to-be-received file to obtain multiple segmented data segments and generate original segment summary information corresponding to the multiple segmented data segments in a case where a file data amount of the first to-be-received file is greater than a preset segmentation threshold, and encapsulate the multiple segmented data segments respectively according to the original segment summary information to generate the multiple slice files of the first to-be-received file.

13. The system of claim 8, wherein, The file sending end is further configured to determine a file transmission mode corresponding to the first to-be-received file, and determine network element span information of the first to-be-received file in a transmission process according to the file transmission mode; and determine a splitting threshold corresponding to the first to-be-received file according to the network element span information.

14. An electronic device comprising a memory and a processor, the memory storing a computer program, characterized in that, The computer program is executed by the processor to implement the steps of the method of any one of claims 1 to 7.

15. A computer readable storage medium having stored thereon a computer program, characterized in that, The computer program is executed by the processor to implement the steps of the method of any one of claims 1 to 7.

16. A computer program product comprising a computer program, characterized in that, The computer program is executed by the processor to implement the steps of the method of any one of claims 1 to 7. The computer program is executed by the processor to implement the steps of the method of any one of claims 1 to 7.

Images