File transmission method and device, electronic equipment, storage medium and program product

Abstract

Embodiments of the present application disclose a file transmission method and device, electronic equipment, storage medium and program product; at least one target file block of a target file can be obtained from received file blocks, block information carried by the target file block includes a file identifier of the target file and a block sequence number of each target file block in the block of the target file, a first mapping relationship corresponding to the file identifier is established based on the block size and the block sequence number of each target file block, the first mapping relationship is a mapping relationship between the byte offset of each target file block and the block sequence number, a second mapping relationship between the block sequence number and each target file block stored in the server is established, and a block merging result of the target file is obtained according to mapping relationship integration performed on the first mapping relationship and the second mapping relationship; the file block merging delay can be reduced, the file transmission efficiency is improved, and the processing capacity for the file block is enhanced.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of communication, in particular to a file transmission method and device, electronic equipment, storage medium and program product. BACKGROUND

[0002] With the rapid development of current technology, users often transmit files through a network, for example, sending files to other users, or remotely storing files, etc. When the file is too large or the network environment is not good, in order to speed up file transmission, a file transmission method can be used to split the file into multiple blocks, and sequentially or concurrently upload each block.

[0003] At present, the main method for merging file blocks after the end of file block uploading is to sequentially scan the temporary index list of the file, reconstruct the formal index according to the temporary index list, and then delete the temporary index. However, using this scheme, since the sequential execution method is used, when the number of file blocks is large, even if the processing time of a single block is short, the overall merging time of the file is still long, the block merging time delay is large, and the file transmission efficiency is affected. SUMMARY

[0004] The embodiments of the present application provide a file transmission method, device, electronic equipment, storage medium and program product, which can reduce the merging time delay of file blocks, improve the file transmission efficiency, and enhance the processing capability of file blocks.

[0005] The embodiments of the present application provide a file transmission method, comprising:

[0006] From the received file blocks, at least one target file block of a target file is obtained, the block information carried by the target file block includes a file identifier of the target file and a block sequence number of each target file block in the block of the target file;

[0007] Based on the block size and the block sequence number of each target file block, a first mapping relationship corresponding to the file identifier is established, and the first mapping relationship is a mapping relationship between the byte offset of each target file block and the block sequence number;

[0008] The block sequence number and each target file block stored in the server are established to form a second mapping relationship;

[0009] According to the mapping relationship integration for the first mapping relationship and the second mapping relationship, a block merging result of the target file is obtained.

[0010] Correspondingly, the embodiments of the present application also provide a file transmission device, comprising:

[0011] The block obtaining unit is configured to obtain at least one target file block of a target file from the received file blocks, wherein the block information carried by the target file block includes a file identifier of the target file and a block sequence number of each target file block in the target file blocks of the target file;

[0012] The first mapping relationship establishing unit is configured to establish a first mapping relationship corresponding to the file identifier based on the block size of each target file block and the block sequence number, wherein the first mapping relationship is a mapping relationship between the byte offset of each target file block and the block sequence number;

[0013] The second mapping relationship establishing unit is configured to establish a second mapping relationship between the block sequence number and each target file block stored in the server.

[0014] The block merging unit is configured to obtain a block merging result of the target file according to the mapping relationship integration performed on the first mapping relationship and the second mapping relationship.

[0015] Optionally, the first mapping relationship establishing unit is configured to determine a block mode of the target file according to the block size of each target file block.

[0016] When the block mode is a fixed-length block, the block size of the target file block is taken as the first mapping relationship corresponding to the file identifier.

[0017] Optionally, the first mapping relationship establishing unit is configured to determine a block mode of the target file according to the block size of each target file block.

[0018] When the block mode is a non-fixed-length block, the byte offset of each target file block is determined according to the block size of each target file block.

[0019] The first mapping relationship corresponding to the file identifier is established based on the byte offset and the block sequence number of the same target file block.

[0020] Optionally, the first mapping relationship establishing unit is configured to take any target file block with an undetermined byte offset as a first file block when the block mode is a non-fixed-length block.

[0021] The second file block with a block sequence number smaller than that of the first file block is determined from the target file blocks.

[0022] The byte offset of the first file block is obtained based on the sum of the block size of each second file block and the block size of the first file block.

[0023] Return to perform the step of chunking any of the target files with an undetermined byte offset as a first file chunk until each of the target files has a determined byte offset.

[0024] Optionally, the file transmission apparatus provided by the embodiment of the present application further comprises a chunk downloading unit, configured to receive a chunk downloading request, wherein the chunk downloading request comprises a target downloading offset of a to-be-downloaded chunk in a target downloading file and a target file identifier of the target downloading file.

[0025] The first mapping relationship and the second mapping relationship corresponding to the target file identifier are determined.

[0026] According to the target downloading offset and the first mapping relationship, a downloading chunk serial number of the to-be-downloaded chunk in the target downloading file is determined.

[0027] According to the downloading chunk serial number and the second mapping relationship, a target downloading file chunk corresponding to the downloading chunk serial number is transmitted.

[0028] Optionally, the file transmission apparatus provided by the embodiment of the present application further comprises a file downloading unit, configured to receive a file downloading request, wherein the file downloading request comprises a target file identifier of a target downloading file.

[0029] The first mapping relationship and the second mapping relationship corresponding to the target file identifier are determined.

[0030] According to the first mapping relationship, downloading chunk serial numbers of all file chunks in the target downloading file are determined.

[0031] According to the downloading chunk serial numbers and the second mapping relationship, target downloading file chunks corresponding to the downloading chunk serial numbers are transmitted.

[0032] Optionally, the file chunk is a fixed-length chunk, and the first mapping relationship comprises a chunk size of the file chunk; the chunk downloading unit is configured to calculate a quotient between the target downloading offset and the chunk size of the file chunk in the first mapping relationship.

[0033] According to the quotient, a chunk serial number of a to-be-downloaded chunk in the target downloading file is determined.

[0034] Optionally, the second mapping relationship corresponds to the first mapping relationship.

[0035] Correspondingly, the chunk downloading unit and the file downloading unit are configured to determine the first mapping relationship corresponding to the target downloading file identifier.

[0036] According to the first mapping relationship, a second mapping relationship corresponding to the first mapping relationship is determined.

[0037] Optionally, the block downloading unit is configured to sort the block sequence numbers according to the byte offsets in the first mapping relationship, to obtain a sorted first mapping relationship.

[0038] According to the target download offset, a binary search is performed in the byte offsets in the first mapping relationship to determine a target byte offset corresponding to the target download offset.

[0039] In the first mapping relationship, the block sequence number corresponding to the target byte offset is determined as a download block sequence number of a block to be downloaded in the target download file.

[0040] Optionally, the block merging unit is configured to determine mapping relationship data corresponding to the first mapping relationship and the second mapping relationship.

[0041] The mapping relationship data is format-converted to obtain mapping relationship data in a target format.

[0042] According to the mapping relationship data in the target format, a block merging result of the target file is obtained.

[0043] Optionally, the block merging unit is configured to acquire mapping relationship data from a storage location where the first mapping relationship and the second mapping relationship are stored.

[0044] The mapping relationship data is copied into a data storage file to obtain mapping relationship data in a target format.

[0045] Correspondingly, an electronic device is also provided, which includes a memory and a processor. The memory stores an application program, and the processor is configured to run the application program in the memory to execute the steps in any of the file transmission methods provided in the embodiments of the present application.

[0046] Correspondingly, a computer readable storage medium is also provided, which stores a plurality of instructions. The instructions are adapted to be loaded by a processor to execute the steps in any of the file transmission methods provided in the embodiments of the present application.

[0047] In addition, a computer program product is also provided, which includes a computer program or instructions. When the computer program or instructions are executed by a processor, the steps in any of the file transmission methods provided in the embodiments of the present application are implemented.

[0048] The scheme of the embodiment of the present application can obtain at least one target file block of a target file from the received file blocks, the block information carried by the target file block includes a file identifier of the target file and a block sequence number of each target file block in the block of the target file, a first mapping relationship corresponding to the file identifier is established based on the block size of each target file block and the block sequence number, the first mapping relationship is a mapping relationship between the byte offset of each target file block and the block sequence number, a second mapping relationship between the block sequence number and each target file block stored in the server is established, and the block merging result of the target file is obtained according to the mapping relationship integration for the first mapping relationship and the second mapping relationship; since the first mapping relationship and the second mapping relationship are reconstructed in real time after the block uploading, it is not necessary to perform the operation of deleting a temporary index and reconstructing an official index in the block merging process, therefore, the merging delay of the file block can be reduced, the file transmission efficiency is improved, and the processing capacity for the file block is enhanced. BRIEF DESCRIPTION OF DRAWINGS

[0049] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed to be used in the embodiment description will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without any creative effort on the basis of these drawings.

[0050] Figure 1 is a scene schematic diagram of the file transmission method provided by the embodiment of the present application;

[0051] Figure 2 is a flowchart of the file transmission method provided by the embodiment of the present application;

[0052] Figure 3 is a mapping relationship design schematic diagram provided by the embodiment of the present application;

[0053] Figure 4 is a delay effect statistical diagram provided by the embodiment of the present application;

[0054] Figure 5 is another flowchart of the file transmission method provided by the embodiment of the present application;

[0055] Figure 6 is still another flowchart of the file transmission method provided by the embodiment of the present application;

[0056] Figure 7 is a structural schematic diagram of the file transmission device provided by the embodiment of the present application;

[0057] Figure 8 is another structural schematic diagram of the file transmission device provided by the embodiment of the present application;

[0058] Figure 9 This is a schematic diagram of the structure of the electronic device provided in an embodiment of the present invention. Detailed Implementation

[0059] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0060] This invention provides a file transfer method, apparatus, electronic device, and computer-readable storage medium. Specifically, this invention provides a file transfer method suitable for a file transfer apparatus that can be integrated into an electronic device.

[0061] The electronic device can be a terminal or other device, including but not limited to mobile terminals and fixed terminals. For example, mobile terminals include but are not limited to smartphones, smartwatches, tablets, laptops, smart vehicles, etc., while fixed terminals include but are not limited to desktop computers, smart TVs, etc.

[0062] The electronic device can also be a server or other similar device. The server can be an independent physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDN (Content Delivery Network), and big data and artificial intelligence platforms, but it is not limited to these.

[0063] The file transfer method of this invention can be implemented by a server or by a terminal and a server together.

[0064] The following example illustrates how a file transfer method can be implemented jointly by a terminal and a server.

[0065] like Figure 1 As shown, the file transfer system provided in this embodiment of the invention includes a terminal 10 and a server 20, etc.; the terminal 10 and the server 20 are connected via a network, such as a wired or wireless network.

[0066] Terminal 10 can serve as a terminal for users to transfer files, and can be used to send target file blocks of the target file to server 20.

[0067] Server 20 can be used to obtain at least one target file block of a target file from the received file blocks. The block information carried by the target file block includes the file identifier of the target file and the block sequence number of each target file block in the blocks of the target file.

[0068] Server 20 can establish a first mapping relationship corresponding to the file identifier based on the block size and block number of each target file block. The first mapping relationship is the mapping relationship between the byte offset of each target file block and the block number. A second mapping relationship is established between the block number and each target file block stored in the server. The block merging result of the target file is obtained by integrating the mapping relationship between the first mapping relationship and the second mapping relationship.

[0069] The following sections provide detailed descriptions of each example. It should be noted that the order in which the embodiments are described is not intended to limit the preferred order of the embodiments.

[0070] The embodiments of the present invention will be described from the perspective of a file transfer device, which may be integrated into a server and / or a terminal.

[0071] like Figure 2 As shown, the specific process of the file transfer method in this embodiment can be as follows:

[0072] 201. From the received file blocks, obtain at least one target file block of the target file. The block information carried by the target file block includes the file identifier of the target file and the block number of each target file block in the blocks of the target file.

[0073] The target file is the file transmitted by the user using the file transfer method provided in this embodiment of the invention. The target file may include, but is not limited to, text documents, images, audio / video files, and compressed files. For example, the target file format may be .doc, .jpg, .zip, etc. This embodiment of the invention does not limit the content and format of the target file.

[0074] In this context, a target file chunk refers to a data block obtained by dividing the target file into chunks. When transmitting a file, it can be divided into fixed-length or variable-length chunks according to upload or storage requirements, resulting in at least one file chunk for easy transmission.

[0075] It is understandable that a file transfer device can receive file chunks from the same file or different files.

[0076] For example, in concurrent processing scenarios, a file transfer device may simultaneously receive at least two file blocks from the same file and / or different files. In this case, it is necessary to determine the target file block of the target file from all the received file blocks.

[0077] For example, the file transmission device can receive only one file chunk from the same file at the same time. In this case, the received file chunk can be regarded as a target file chunk of the target file until the file transmission device stops receiving the file chunk, or the file transmission device starts receiving a file chunk of another file, etc.

[0078] The file identifier can be a transmission number of the target file, or the file identifier can be a file name of the target file, or the file identifier can be generated according to a user identifier and a file name of the target file.

[0079] The chunk sequence number can be obtained according to a chunk order of the target file. For example, the target file can be sequentially divided, and the chunk sequence number is the order in which each target file chunk is divided from the target file.

[0080] Alternatively, the chunk sequence number can be a transmission order of the target file chunk, such as an order in which the terminal starts uploading the target file chunk, or an order in which the server receives the target file chunk.

[0081] 202. Based on the chunk size and the chunk sequence number of each target file chunk, a first mapping relationship corresponding to the file identifier is established, and the first mapping relationship is a mapping relationship between the byte offset of each target file chunk and the chunk sequence number.

[0082] The chunk size refers to the data size of the target file chunk. For example, the target file chunk can be 64 KB, 16 bits, etc. The chunk sizes of the target file chunks of the same target file can be the same or different.

[0083] The byte offset refers to the offset between the target file chunk and the first storage position of the target file. When determining the byte offset, it can be determined based on the storage start address of the target file chunk and / or the chunk size of the target file chunk, or it can be determined based on the storage end address of the target file chunk.

[0084] For example, the first storage position of the target file is 0, and the storage start address of a target file chunk is 8, so the byte offset can be 8; or the storage end address of a target file chunk is 8, so the byte offset can be 8.

[0085] In some optional examples, the first mapping relationship can include a corresponding relationship between the byte offset of each target file chunk and the chunk sequence number. At this time, step 202 can specifically include:

[0086] According to the chunk size of each target file chunk, the byte offset of each target file chunk is determined.

[0087] The first mapping relationship corresponding to the file identifier is established based on the byte offset and the block sequence number of each target file block.

[0088] That is, no matter whether the fixed-length block or the non-fixed-length block is adopted, the first mapping relationship can be established according to the byte offset and the block sequence number of each target file block after the byte offset of each target file block is calculated.

[0089] It should be noted that the first mapping relationship is the mapping relationship between the byte offset and the block sequence number of each target file block, but it does not mean that the first mapping relationship must specifically include the byte offset, the block sequence number and the specific correspondence between the byte offset and the block sequence number. The mapping between the byte offset and the block sequence number can be obtained by mapping through the first mapping relationship.

[0090] For example, for the target file of the fixed-length block, the first mapping relationship thereof can only include the block size of each target file block. That is, the step of “establishing the first mapping relationship corresponding to the file identifier based on the block size and the block sequence number of each target file block” can include:

[0091] determining the block mode of the target file according to the block size of each target file block;

[0092] when the block mode is the fixed-length block, the block size of the target file block is taken as the first mapping relationship corresponding to the file identifier.

[0093] For the first mapping relationship including only the block size of the target file block, the mapping relationship between the byte offset and the block sequence number is actually implied in the block size of each target file block. For example, when the block size part_size of each target file block is known, the byte offset of the target file block with the block sequence number n is byte_offset = part_size * n.

[0094] If the same processing scheme is not adopted for the fixed-length or non-fixed-length block, for the non-fixed-length block, the step of “establishing the first mapping relationship corresponding to the file identifier based on the block size and the block sequence number of each target file block” can specifically include:

[0095] determining the block mode of the target file according to the block size of each target file block;

[0096] when the block mode is the non-fixed-length block, determining the byte offset of each target file block according to the block size of each target file block;

[0097] The first mapping relationship corresponding to the file identifier is established based on the byte offset and the block sequence number of the same target file block.

[0098] The non-fixed-length block refers to that at least one target file block has a block size different from any target file block when the target file is blocked.

[0099] The first mapping relationship and the second mapping relationship of the non-fixed-length block are designed. Even if 10,000 file blocks are used for binary search, the worst-case query number is: log2(10000) + 1 = 14 memory operations. Compared with 10,000 write and delete operations required to establish a formal index in the related art, the time consumption is negligible, which greatly improves the efficiency of block merging.

[0100] In some examples, the blocking manner of the target file can be directly included in the block information of the target file block.

[0101] Specifically, the step of "when the blocking manner is a non-fixed-length block, determining the byte offset of each target file block according to the block size of each target file block" includes:

[0102] When the blocking manner is a non-fixed-length block, any target file block with an undetermined byte offset is taken as a first file block.

[0103] From the target file blocks, a second file block with a block sequence number smaller than that of the first file block is determined.

[0104] Based on the sum of the block size of each second file block and the block size of the first file block, the byte offset of the first file block is obtained.

[0105] Return to execute the step of taking any target file block with an undetermined byte offset as a first file block until the byte offset of each target file block is determined.

[0106] For example, when calculating the byte offset of a target file block, the sum of the file block size of all other target file blocks with a sequence number smaller than the target file block and the file block size of the target file block can be calculated as the byte offset of the target file block.

[0107] 203, a second mapping relationship between the block sequence number and each target file block stored in the server is established.

[0108] Specifically, the second mapping relationship can be in the form of a key-value pair, the key including the block number, and the value including the specific data of the target file block corresponding to the block number. Alternatively, the second mapping relationship can be in the form of a data table, the row or column number being the block number, and the content in the row or column being the specific data of the target file block corresponding to the block number, and the like.

[0109] In actual application, if a user needs to download a certain file block or a certain byte offset interval, the file transmission method provided by the embodiment of the present application can further include:

[0110] receiving a block download request, the block download request including a target download offset of a to-be-downloaded block in a target download file and a target file identifier of the target download file;

[0111] determining the first mapping relationship and the second mapping relationship corresponding to the target file identifier;

[0112] determining, according to the target download offset and the first mapping relationship, a download block number of the to-be-downloaded block in the target download file;

[0113] transmitting, according to the download block number and the second mapping relationship, a target download file block corresponding to the download block number.

[0114] For example, a user can select to download a target file block with a target byte offset of 6-8, and the first mapping relationship includes that 0-4 is block No. 1 and 4-8 is block No. 2. At this time, according to 6-8 and the first mapping relationship, the block number (download block number) of the file block with the byte offset of 6-8 can be determined as 2. Then, according to the second mapping relationship, block No. 2 can be downloaded as the target download file block.

[0115] In some optional embodiments, the file block can be a fixed-length block, and the first mapping relationship can include a block size of the file block.

[0116] Correspondingly, the step of “determining, according to the target download offset and the first mapping relationship, a block number of the to-be-downloaded block in the target download file” can specifically include:

[0117] calculating a quotient between the target download offset and the block size of the file block in the first mapping relationship;

[0118] determining, according to the quotient, the block number of the to-be-downloaded block in the target download file.

[0119] For the file chunk obtained by fixed-length chunking, there is a strict one-to-one correspondence between the chunk sequence number and the byte offset of the chunk during uploading, that is, the byte offset byte_offset = part_size * n, and the specific chunk sequence number can be located according to this calculation method during downloading.

[0120] Therefore, the chunk sequence number can be obtained by calculating the quotient between the target download offset and the chunk size of the file chunk in the first mapping relationship. For example, the user can select the to-be-downloaded chunk with the target download offset of 6-9, and the first mapping relationship includes each chunk size of 4, then 6 ÷ 4 = 1 remainder 2, and 9 ÷ 4 = 2 remainder 1, at this time, it can be determined that the chunk sequence number corresponding to the to-be-downloaded chunk with the target download offset of 6-9 is 2 and 3.

[0121] In some other optional embodiments, the step of "determining the download chunk sequence number of the to-be-downloaded chunk in the target download file according to the target download offset and the first mapping relationship" can specifically include:

[0122] According to the byte offset in the first mapping relationship, the chunk sequence number is offset sorted to obtain the sorted first mapping relationship;

[0123] According to the target download offset, a binary search is performed in the byte offset of the first mapping relationship to determine the target byte offset corresponding to the target download offset;

[0124] In the first mapping relationship, the chunk sequence number corresponding to the target byte offset is determined as the download chunk sequence number of the to-be-downloaded chunk in the target download file.

[0125] It can be understood that in the embodiments of the present application, sequential search, reverse search and the like can also be used, and the skilled in the art can set it according to actual needs, and the embodiments of the present application do not limit it.

[0126] In actual application process, if the user needs to download the complete target file, the file transmission method provided by the embodiments of the present application can further include:

[0127] Receiving a file download request, the file download request including a target file identifier of a target download file;

[0128] Determining the first mapping relationship and the second mapping relationship corresponding to the target file identifier;

[0129] According to the first mapping relationship, determining the download chunk sequence number of all file chunks in the target download file;

[0130] According to the download chunk sequence number and the second mapping relationship, transmitting the target download file chunk corresponding to each download chunk sequence number.

[0131] For example, you can configure the file download request to download the entire target file if it only includes the target file identifier; or, the file download request can include information such as the starting address and / or ending address of the target file.

[0132] In some examples, the second mapping relationship corresponds to the first mapping relationship, and the step "determine the first and second mapping relationships corresponding to the target download file identifier" may specifically include:

[0133] Determine the first mapping relationship corresponding to the target download file identifier;

[0134] Based on the first mapping relationship, determine the second mapping relationship corresponding to the first mapping relationship.

[0135] like Figure 3 As shown, the first mapping relationship and the second mapping relationship can be stored in two sets of key-value pairs respectively. The key of the first set of key-value pairs is the file identifier (main filekey), and the value of the first set of key-value pairs (the value of the main filekey) is the first mapping relationship. The first mapping relationship is the mapping relationship between byte offset and block sequence number. For example, 100 is the block sequence number, and 100MB is the byte offset of the file block with block sequence number 100.

[0136] The second set of key-value pairs can correspond to the first set of key-value pairs. The key of the second set of key-value pairs is the block number, and the value of the second set of key-value pairs is the target file block stored on the server corresponding to the block number.

[0137] In other examples, the second mapping can also be created by mapping the target file identifier.

[0138] For example, the key of the second set of key-value pairs can be a combination of file identifier and block number, and the value of the second set of key-value pairs is the target file block stored on the server corresponding to the block number.

[0139] 204. Based on the mapping relationships performed on the first and second mapping relationships, the block merging result of the target file is obtained.

[0140] It should be noted that for file block merging, the data in each block can be stored in the same file, or simply establishing connections between the blocks corresponding to the same target file can be used. This embodiment of the invention does not limit the method of file block merging.

[0141] In actual application, in order to facilitate storage, the step of obtaining the block merging result of the target file according to the mapping relationship integration for the first mapping relationship and the second mapping relationship can specifically include:

[0142] determining mapping relationship data corresponding to the first mapping relationship and the second mapping relationship;

[0143] performing format conversion on the mapping relationship data to obtain mapping relationship data in a target format;

[0144] obtaining the block merging result of the target file according to the mapping relationship data in the target format.

[0145] For example, the mapping relationship data can be stored in the form of Protobuf structure serialization. Since Profobuf is a binary data format, the mapping relationship data needs to be encoded (format converted) to obtain encoded data.

[0146] ProtoBuff is a method of serializing and deserializing structured data. When the mapping relationship data is stored by Protobuf, a proto file generally needs to be defined, the proto file is compiled and imported, the mapping relationship data is stored in the proto file, and a serialization method is called for serialization.

[0147] Alternatively, a serialization mechanism such as json or xml can also be used, and the embodiments of the present application do not limit this.

[0148] However, if the mapping data is still stored in the form of the built-in array of Protobuf, the serialization of the long array will occupy a long time of CPU calculation, occupy computing resources, and may affect the running of other processes.

[0149] In order to reduce the occupation time of computing resources, specifically, the step of determining the mapping relationship data corresponding to the first mapping relationship and the second mapping relationship can include:

[0150] obtaining the mapping relationship data from a storage location storing the first mapping relationship and the second mapping relationship;

[0151] performing format conversion on the mapping relationship data to obtain mapping relationship data in a target format, including:

[0152] copying the mapping relationship data to a data storage file to obtain the mapping relationship data in the target format.

[0153] That is, in the embodiment of the application, the data in the memory is directly copied to the data storage file. It is tested that the efficiency of directly copying the array in the memory to the byte type of the Protobuf file is higher than that of serialization by using fixed_int64, and the occupied memory is smaller. The test results of 10,000 file blocks are shown in the following table:

[0154]

[0155] As shown in Figure 4 , when uploading the park related data of a certain industrial park, the file transmission method provided by the embodiment of the application (opening the optimization switch) is used, and the time delay is obviously reduced.

[0156] As can be seen from the above, the embodiment of the application can obtain at least one target file block of a target file from the received file blocks, the block information carried by the target file block includes the file identifier of the target file and the block sequence number of each target file block in the block of the target file, the first mapping relationship corresponding to the file identifier is established based on the block size of each target file block and the block sequence number, the first mapping relationship is the mapping relationship between the byte offset of each target file block and the block sequence number, the second mapping relationship between the block sequence number and each target file block stored in the server is established, and the block merging result of the target file is obtained according to the mapping relationship integration for the first mapping relationship and the second mapping relationship; since the first mapping relationship and the second mapping relationship are reconstructed in real time after the block uploading, it is not necessary to perform the operation of deleting the temporary index and reconstructing the formal index in the block merging process, so that the merging time delay of the file blocks can be reduced, the file transmission efficiency can be improved, and the processing capacity for the file blocks is enhanced.

[0157] According to the method described in the foregoing embodiment, the following will be further described by way of example.

[0158] In the embodiment of the application, the target file is taken as an example to be chunked in a non-fixed length manner, and the system of Figure 1 is combined to describe.

[0159] As shown in Figure 5 , the specific process of the file transmission method of the embodiment can be as follows:

[0160] 501. The terminal sends the target file block of the target file to the server.

[0161] Generally, for a target file with a data size greater than a certain threshold, the terminal can complete the file uploading initialization by using a preset file block interface when sending the target file, and the background returns a unique identifier of this uploading, which is used for subsequent file uploading process.

[0162] After initialization, the user can upload the file blocks of the target file using the unique identifier, which can be uploaded one at a time or concurrently. The file block number corresponding to the file block can be provided when uploading.

[0163] 502、The server obtains at least one target file block of the target file from the received file blocks, and the block information carried by the target file block includes the file identifier of the target file and the block number of each target file block in the target file.

[0164] The target file is a file transmitted by the user using the file transmission method provided by the embodiment of the application. The target file can include, but is not limited to, a text document, an image, an audio / video, and a compressed package.

[0165] When transmitting the file, the file can be divided into fixed-length or non-fixed-length blocks according to the uploading or storage requirements to obtain at least one file block for transmission. In the embodiment of the application, a non-fixed-length block method is adopted.

[0166] 503、The server determines the byte offset of each target file block according to the block size of each target file block, and establishes a first mapping relationship corresponding to the file identifier based on the byte offset and the block number of the same target file block.

[0167] The non-fixed-length block refers to the block size of at least one target file block obtained by dividing the target file is different from any target file block. The fixed-length block refers to the block size of all target file blocks obtained by dividing the target file is the same.

[0168] The design of the first mapping relationship and the second mapping relationship for non-fixed-length blocks, even if 10,000 file blocks are subjected to binary search, the worst-case query times: log2(10000) + 1 = 14 memory operations, relative to the 10,000 write and delete operations required to establish a formal index in related technologies, the time consumption is negligible, greatly improving the efficiency of block merging.

[0169] 504、The server establishes a second mapping relationship between the block number and each target file block stored in the server.

[0170] Specifically, the second mapping relationship can be in the form of a key-value pair, the suffix of the key can be the block number, and the value includes the specific data of the target file block corresponding to the block number.

[0171] 505、The server obtains the block merging result of the target file according to the mapping relationship integration of the first mapping relationship and the second mapping relationship.

[0172] It should be noted that for the block merging of the file, a certain connection relationship can be established for each target file block corresponding to the same target file. For example, by establishing the first mapping relationship and the second mapping relationship, it can be considered that the block merging of the target file is realized.

[0173] For example, the first mapping relationship can be defined by the following code:

[0174] message ShaMainRecord{ ...

[0175] int64 part_size = 11; / / 0 represents the old block organization method (compatible), greater than 0 represents fixed length, and less than 0 represents variable length

[0176] int32 npart = 12; / / When the block is variable length (part_size < 0), the number of blocks of Finish is indicated

[0177] bytes parts = 13; / / When the block is variable length (part_size < 0), the block end_offset mapping table of Finish is indicated, and the memory byte sequence of int64[N]

[0178] }

[0179] / / Access example of ShaMainRecord parts

[0180] const int64_t *parts = (const int64_t*)record.parts().c_str();

[0181] int64_t end_offset_n = parts[n - 1]; / / The end_offset of the nth block

[0182] 506、The server receives the block download request sent by the terminal, and the block download request includes the target download offset of the to-be-downloaded block in the target download file and the target file identifier of the target download file.

[0183] For example, the user can select to download the to-be-downloaded block with a target byte offset of 6-8 in the target download file with a target file identifier of 1.

[0184] The server can determine the first mapping relationship corresponding to the target file identifier according to the target file identifier (1).

[0185] 507、The server determines the download block sequence number of the to-be-downloaded block in the target download file according to the target download offset and the first mapping relationship.

[0186] For example, the first mapping relationship includes that 0-4 is block 1 and 4-8 is block 2. At this time, the block sequence number (download block sequence number) of the file block whose byte offset is 6-8 can be determined as 2 according to the target byte offset 6-8 and the first mapping relationship.

[0187] 508、The server transmits the target download file block corresponding to the download block sequence number according to the download block sequence number and the second mapping relationship.

[0188] For example, the server can send block 2 of the file as the target download file block to the terminal according to the second mapping relationship.

[0189] 509、The terminal receives the target download file block transmitted by the server.

[0190] Through the design of the first mapping relationship and the second mapping relationship in the embodiment of the application, the merging delay of block uploading can be greatly reduced, the interface delay in the large block scenario is reduced from several hundred seconds to within hundreds of milliseconds, and the average block delay is reduced from seconds to about 10 milliseconds. The scheme of the embodiment of the application reduces the operations of rebuilding and deleting N block indexes during merging, reduces the system bottom burr request, and improves the availability.

[0191] As can be seen from the above, the embodiment of the application rebuilds the first mapping relationship and the second mapping relationship in real time after block uploading, and in the block merging process, the operations of deleting temporary indexes and rebuilding formal indexes are no longer needed, so that the merging delay of file blocks can be reduced, the file transmission efficiency can be improved, and the processing capacity for file blocks is enhanced.

[0192] According to the method described in the foregoing embodiment, the following will be further described by way of example.

[0193] In the embodiment of the application, the target file will be divided into blocks in a fixed-length manner, and the system shown in Figure 1 will be described.

[0194] As shown in Figure 6 , the file transmission method of the embodiment can have the following specific process:

[0195] 601、The terminal sends a target file block of a target file to a server.

[0196] Generally, for a target file with a data size greater than a threshold, the terminal can complete file uploading initialization through a preset file chunk interface when sending the target file, and the background returns a unique identifier of this uploading, which is used for subsequent file uploading process.

[0197] After initialization, the user can use the unique identifier to upload file chunks of the target file, which can be uploaded at a time or concurrently. The uploading can provide a chunk sequence number corresponding to the file chunk.

[0198] 602、The server obtains at least one target file chunk of the target file from the received file chunks, and the chunk information carried by the target file chunk includes a file identifier of the target file and a chunk sequence number of each target file chunk in the target file.

[0199] The target file is a file transmitted by the user using the file transmission method provided by the embodiment of the application. The target file can include but is not limited to a text document, an image, an audio / video, a compressed package, and the like.

[0200] When transmitting the file, the file can be divided into fixed-length or non-fixed-length chunks according to uploading or storage requirements, and at least one file chunk is obtained for transmission. In the embodiment of the application, a fixed-length chunking method is adopted.

[0201] 603、The server takes the chunk size of each target file chunk as a first mapping relationship corresponding to the file identifier.

[0202] It should be noted that although the first mapping relationship is a mapping relationship between the byte offset and the chunk sequence number of each target file chunk, it does not mean that the first mapping relationship must specifically include the byte offset, the chunk sequence number, and the specific correspondence between the byte offset and the chunk sequence number. The mapping between the byte offset and the chunk sequence number can be obtained through the first mapping relationship.

[0203] For example, for a target file with fixed-length chunking, the first mapping relationship thereof can only include the chunk size of each target file chunk. For the first mapping relationship including only the chunk size of the target file chunk, the mapping relationship between the byte offset and the chunk sequence number is actually implied in the chunk size of each target file chunk. For example, when the chunk size part_size of each target file chunk is known, the byte offset byte_offset of the target file chunk with the chunk sequence number n is byte_offset = part_size * n.

[0204] 604、The server establishes a second mapping relationship between the chunk sequence number and each target file chunk stored in the server.

[0205] Specifically, the second mapping relationship can be in the form of a key-value pair, the suffix of the key can be a block number, and the value includes the specific data of the target file block corresponding to the block number.

[0206] 605、The server obtains the block merging result of the target file according to the mapping relationship integration for the first mapping relationship and the second mapping relationship.

[0207] It should be noted that for the block merging of the file, a certain connection relationship can be constructed for each target file block corresponding to the same target file. For example, by establishing the first mapping relationship and the second mapping relationship, it can be considered that the block merging of the target file is realized.

[0208] For example, the first mapping relationship can be defined by the following code:

[0209] message ShaMainRecord{ ...

[0210] int64 part_size = 11; / / 0 represents the old block organization method (compatible), greater than 0 represents fixed length, and less than 0 represents variable length

[0211] }

[0212] / / Access example of ShaMainRecord parts

[0213] const int64_t *parts = (const int64_t*)record.parts().c_str();

[0214] int64_t end_offset_n = parts[n - 1]; / / The end offset of the nth block

[0215] 606、The server receives the block download request sent by the terminal, and the block download request includes the target download offset of the to-be-downloaded block in the target download file and the target file identifier of the target download file.

[0216] For example, the user can select to download the to-be-downloaded block with a target download offset of 6-9 in the target download file with a target file identifier of 1.

[0217] The server can determine the first mapping relationship corresponding to the target file identifier according to the target file identifier (1).

[0218] 607、The server calculates the quotient between the target download offset and the chunk size of the file chunk in the first mapping relationship, and determines the chunk sequence number of the to-be-downloaded chunk in the target download file according to the quotient.

[0219] The first mapping relationship includes each chunk size of 4, 6 / 4=1 with a remainder of 2, and 9 / 4=2 with a remainder of 1. At this time, it can be determined that the target download offset of 6-9 corresponds to the chunk sequence numbers of 2 and 3 of the to-be-downloaded chunk.

[0220] 608、The server transmits the target download file chunk corresponding to the download chunk sequence number according to the second mapping relationship.

[0221] For example, the server can send the No. 2 and No. 3 file chunks as the target download file chunks to the terminal according to the second mapping relationship.

[0222] 609、The terminal receives the target download file chunk transmitted by the server.

[0223] The embodiment of the application improves the merging performance. For the fixed-length chunk upload scene, the corresponding product capacity is improved from the previous maximum of 10,000 chunks to 100,000 chunks, and the product support capacity is improved.

[0224] As can be seen from the above, the embodiment of the application reconstructs the first mapping relationship and the second mapping relationship in real time after the chunk upload, and does not need to perform operations such as deleting a temporary index and reconstructing an official index in the chunk merging process. Therefore, the file chunk merging delay can be reduced, the file transmission efficiency is improved, and the processing capacity for the file chunk is enhanced.

[0225] In order to better implement the above method, correspondingly, the embodiment of the application also provides a file transmission device.

[0226] Reference Figure 7 The device includes:

[0227] The chunk acquisition unit 701 can be used to acquire at least one target file chunk of a target file from the received file chunk, and the chunk information carried by the target file chunk can include the file identifier of the target file and the chunk sequence number of each target file chunk in the chunk of the target file.

[0228] The first mapping relationship establishing unit 702 can be used to establish the first mapping relationship corresponding to the file identifier based on the chunk size and the chunk sequence number of each target file chunk, and the first mapping relationship is the mapping relationship between the byte offset of each target file chunk and the chunk sequence number.

[0229] The second mapping relationship establishing unit 703 can be used to establish the second mapping relationship between the chunk sequence number and each target file chunk stored in the server.

[0230] The block merging unit 704 can be configured to obtain a block merging result of the target file according to the mapping relationship integration performed on the first mapping relationship and the second mapping relationship.

[0231] In some optional embodiments, the first mapping relationship establishing unit 702 can be configured to determine a block mode of the target file according to the block size of each target file block.

[0232] When the block mode is the fixed-length block, the block size of the target file block is taken as the first mapping relationship corresponding to the file identifier.

[0233] In some optional embodiments, the first mapping relationship establishing unit 702 can be configured to determine a block mode of the target file according to the block size of each target file block.

[0234] When the block mode is the non-fixed-length block, the byte offset of each target file block is determined according to the block size of each target file block.

[0235] The first mapping relationship corresponding to the file identifier is established based on the byte offset and the block sequence number of the same target file block.

[0236] In some optional embodiments, the first mapping relationship establishing unit 702 can be configured to, when the block mode is the non-fixed-length block, take any target file block with an undetermined byte offset as a first file block.

[0237] From the target file blocks, a second file block with a block sequence number smaller than that of the first file block is determined.

[0238] The byte offset of the first file block is obtained based on the sum of the block size of each second file block and the block size of the first file block.

[0239] The step of taking any target file block with an undetermined byte offset as the first file block is returned to be executed until the byte offset of each target file block is determined.

[0240] In some optional embodiments, as shown in Figure 8 The file transmission device provided by the embodiment of the present application can further include a block downloading unit 705 configured to receive a block downloading request, and the block downloading request can include a target downloading offset of a to-be-downloaded block in a target downloading file and a target file identifier of the target downloading file.

[0241] The first mapping relationship and the second mapping relationship corresponding to the target file identifier are determined.

[0242] The downloading block sequence number of the to-be-downloaded block in the target downloading file is determined according to the target downloading offset and the first mapping relationship.

[0243] According to the download block sequence number and the second mapping relationship, the target download file block corresponding to the download block sequence number is transmitted.

[0244] In some optional embodiments, the file transmission apparatus provided by the embodiments of the present application can further include a file download unit 706, which can be configured to receive a file download request, and the file download request can include a target file identifier of a target download file.

[0245] The first mapping relationship and the second mapping relationship corresponding to the target file identifier are determined.

[0246] According to the first mapping relationship, the download block sequence number of all file blocks in the target download file is determined.

[0247] According to the download block sequence number and the second mapping relationship, the target download file block corresponding to the download block sequence number is transmitted.

[0248] In some optional embodiments, the file block is a fixed-length block, and the first mapping relationship can include a block size of the file block; the block download unit 705 can be configured to calculate a quotient between the target download offset and the block size of the file block in the first mapping relationship.

[0249] According to the quotient, the block sequence number of the to-be-downloaded block in the target download file is determined.

[0250] In some optional embodiments, the second mapping relationship corresponds to the first mapping relationship.

[0251] Correspondingly, the block download unit 705 and the file download unit 706 can be configured to determine the first mapping relationship corresponding to the target download file identifier.

[0252] According to the first mapping relationship, the second mapping relationship corresponding to the first mapping relationship is determined.

[0253] In some optional embodiments, the block download unit 705 can be configured to sort the block sequence numbers according to the byte offset in the first mapping relationship, to obtain a sorted first mapping relationship.

[0254] According to the target download offset, a binary search is performed in the byte offset of the first mapping relationship to determine a target byte offset corresponding to the target download offset.

[0255] In the first mapping relationship, the block sequence number corresponding to the target byte offset is determined as the download block sequence number of the to-be-downloaded block in the target download file.

[0256] In some optional embodiments, the block merging unit 704 can be configured to determine the mapping relationship data corresponding to the first mapping relationship and the second mapping relationship.

[0257] perform format conversion on the mapping relationship data to obtain mapping relationship data in a target format;

[0258] obtain the block merging result of the target file according to the mapping relationship data in the target format.

[0259] In some optional embodiments, the block merging unit 704 can be configured to obtain the mapping relationship data from a storage location where the first mapping relationship and the second mapping relationship are stored.

[0260] copy the mapping relationship data into a data storage file to obtain the mapping relationship data in the target format.

[0261] As can be seen, by using the file transmission apparatus, at least one target file block of a target file can be obtained from the received file blocks, the block information carried by the target file block includes a file identifier of the target file and a block sequence number of each target file block in the target file, a first mapping relationship corresponding to the file identifier is established based on the block size and the block sequence number of each target file block, the first mapping relationship is a mapping relationship between the byte offset of each target file block and the block sequence number, a second mapping relationship between the block sequence number and each target file block stored in the server is established, and the block merging result of the target file is obtained according to the mapping relationship integration performed on the first mapping relationship and the second mapping relationship. Since the first mapping relationship and the second mapping relationship are reconstructed in real time after the blocks are uploaded, the operations such as deleting temporary indexes and reconstructing formal indexes do not need to be performed in the block merging process, so that the file block merging delay can be reduced, the file transmission efficiency can be improved, and the processing capability for the file blocks is enhanced.

[0262] In addition, an electronic device is also provided in the embodiments of the present application, which can be a terminal or a server, etc. Figure 9 As shown in the figure, a structure schematic diagram of the electronic device related to the embodiments of the present application is shown, and specifically:

[0263] The electronic device can include a radio frequency (RF, Radio Frequency) circuit 901, a memory 902 including one or more computer readable storage media, an input unit 903, a display unit 904, a sensor 905, an audio circuit 906, a wireless fidelity (WiFi, Wireless Fidelity) module 907, a processor 908 including one or more processing cores, and a power supply 909, etc. Those skilled in the art can understand that Figure 9The electronic device structure shown in the figures does not constitute a limitation on the electronic device, and can include more or fewer components than shown, or combine some components, or arrange the components differently. Among them:

[0264] The RF circuit 901 can be used for receiving and sending signals in the process of information or communication, especially, receiving the downlink information of the base station and handing it over to one or more processors 908 for processing; in addition, sending the data related to the uplink to the base station. Generally, the RF circuit 901 includes but is not limited to an antenna, at least one amplifier, a tuner, one or more oscillators, a subscriber identity module (SIM) card, a transceiver, a coupler, a low noise amplifier (LNA), a duplexer, etc. In addition, the RF circuit 901 can also communicate with the network and other devices through wireless communication. Wireless communication can use any communication standard or protocol, including but not limited to global system for mobile communication (GSM), general packet radio service (GPRS), code division multiple access (CDMA), wideband code division multiple access (WCDMA), long term evolution (LTE), email, short messaging service (SMS), etc.

[0265] The memory 902 can be used to store software programs and modules, and the processor 908 can execute various function applications and data processing by running the software programs and modules stored in the memory 902. The memory 902 can mainly include a program storage area and a data storage area, wherein the program storage area can store an operating system, at least one application program required by a function (such as a sound playing function, an image playing function, etc.), etc.; the data storage area can store data created according to the use of the electronic device (such as audio data, a phone book, etc.), etc. In addition, the memory 902 can include a high-speed random access memory, and can also include a non-volatile memory, such as at least one magnetic disk storage device, a flash memory device, or other volatile solid-state memory device. Accordingly, the memory 902 can also include a memory controller to provide access for the processor 908 and the input unit 903 to the memory 902.

[0266] The input unit 903 can be configured to receive input of numbers or characters, and generate key, mouse, joystick, optical or trackball signals related to user settings and function control. Specifically, in one embodiment, the input unit 903 can include a touch-sensitive surface and other input devices. The touch-sensitive surface, also known as a touch display or touchpad, can collect touch operations (such as a user's operation on or near the touch-sensitive surface using a finger, a stylus, or any suitable object or accessory) on or near the touch-sensitive surface, and drive the corresponding connection device according to the pre-set program. Optionally, the touch-sensitive surface can include two parts of touch detection device and touch controller. Among them, the touch detection device detects the touch position of the user and detects the signal generated by the touch operation, and transmits the signal to the touch controller; the touch controller receives the touch information from the touch detection device, and converts it into touch coordinates and sends it to the processor 908, and can receive the command from the processor 908 and execute it. In addition, the touch-sensitive surface can be implemented in various types such as resistive, capacitive, infrared, and surface acoustic wave. In addition to the touch-sensitive surface, the input unit 903 can also include other input devices. Specifically, the other input devices can include one or more of a physical keyboard, function keys (such as volume control keys, on-off keys, etc.), trackballs, mice, joysticks, etc.

[0267] The display unit 904 can be configured to display information input by the user or information provided to the user, and various graphical user interfaces of the electronic device, which can be composed of graphics, text, icons, video, and any combination thereof. The display unit 904 can include a display panel, which can be configured in the form of a liquid crystal display (LCD), an organic light-emitting diode (OLED), etc. Further, the touch-sensitive surface can cover the display panel, and when the touch-sensitive surface detects a touch operation on or near it, it is transmitted to the processor 908 to determine the type of touch event, and then the processor 908 provides corresponding visual output on the display panel according to the type of touch event. Although in Figure 9 In some embodiments, the touch-sensitive surface and the display panel can be integrated to realize input and output functions.

[0268] The electronic device can also include at least one sensor 905, such as a light sensor, a motion sensor, and other sensors. Specifically, the light sensor can include an ambient light sensor that can adjust the brightness of the display panel according to the brightness of ambient light, and a proximity sensor that can turn off the display panel and / or backlight when the electronic device is moved to the ear. As one of the motion sensors, the gravity acceleration sensor can detect the magnitude of acceleration in each direction (generally three axes), and when at rest, can detect the magnitude and direction of gravity, and can be used for identifying the posture of the mobile phone (such as switching between landscape and portrait screens, related games, magnetometer posture calibration), vibration recognition related functions (such as pedometers, taps), and the like. As for other sensors that the electronic device can also be configured, such as a gyroscope, a barometer, a hygrometer, a thermometer, an infrared sensor, and the like, they will not be described here.

[0269] The audio circuit 906, the speaker, and the microphone can provide an audio interface between the user and the electronic device. The audio circuit 906 can convert the received audio data into an electrical signal and transmit it to the speaker, which converts the electrical signal into a sound signal and outputs it. On the other hand, the microphone collects a sound signal and converts it into an electrical signal, which is received by the audio circuit 906 and converted into audio data. The audio data is output to the processor 908 for processing, and then transmitted to another electronic device via the RF circuit 901, or output to the memory 902 for further processing. The audio circuit 906 can also include a jack for connecting external earphones to the electronic device.

[0270] WiFi is a short-range wireless transmission technology. The WiFi module 907 can help the user to send and receive emails, browse web pages, and access streaming media, and provides the user with wireless broadband Internet access. Although Figure 9 The WiFi module 907 is shown, but it is understood that it is not a necessary component of the electronic device and can be omitted without changing the essence of the application.

[0271] The processor 908 is the control center of the electronic device, which connects all parts of the mobile phone through various interfaces and lines, executes various functions of the electronic device and processes data by running or executing software programs and / or modules stored in the memory 902, and calling data stored in the memory 902, thereby overall detecting the mobile phone. Optionally, the processor 908 can include one or more processing cores; preferably, the processor 908 can integrate an application processor and a modem processor, wherein the application processor mainly processes the operating system, user interface, and application programs, and the modem processor mainly processes wireless communication. It can be understood that the above-mentioned modem processor can also not be integrated into the processor 908.

[0272] The electronic device also includes a power supply 909 (such as a battery) that supplies power to the various components. Preferably, the power supply is logically connected to the processor 908 through a power management system, which enables functions such as charge management, discharge management, and power consumption management through the power management system. The power supply 909 can also include one or more DC or AC power sources, a recharging system, a power failure detection circuit, a power converter or inverter, a power status indicator, and any other components.

[0273] Although not shown, the electronic device can also include a camera, a Bluetooth module, and the like, which are not described here in detail. In the present embodiment, the processor 908 in the electronic device loads one or more executable files corresponding to the processes of one or more application programs into the memory 902 according to the following instructions, and runs the application programs stored in the memory 902 by the processor 908, thereby implementing various functions, such as:

[0274] From the received file blocks, at least one target file block of a target file is obtained, and the block information carried by the target file block includes a file identifier of the target file and a block sequence number of each target file block in the blocks of the target file;

[0275] Based on the block size and the block sequence number of each target file block, a first mapping relationship corresponding to the file identifier is established, and the first mapping relationship is a mapping relationship between the byte offset of each target file block and the block sequence number;

[0276] A second mapping relationship between the block sequence number and each target file block stored in the server is established;

[0277] According to the mapping relationship integration for the first mapping relationship and the second mapping relationship, a block merging result of the target file is obtained.

[0278] Those of ordinary skill in the art can understand that all or part of the steps in the various methods of the above embodiments can be completed by instructions, or by related hardware controlled by the instructions, which can be stored in a computer readable storage medium and loaded and executed by a processor.

[0279] To this end, the embodiment of the present application provides a computer readable storage medium, which stores a plurality of instructions. The instructions can be loaded by a processor to execute the steps in any of the file transmission methods provided by the embodiments of the present application. For example, the instructions can execute the following steps:

[0280] From the received file blocks, at least one target file block of a target file is obtained, and the block information carried by the target file block includes a file identifier of the target file and a block sequence number of each target file block in the blocks of the target file;

[0281] The first mapping relationship corresponding to the file identifier is established based on the block size and the block sequence number of each target file block, and the first mapping relationship is a mapping relationship between the byte offset of each target file block and the block sequence number.

[0282] The second mapping relationship between the block sequence number and each target file block stored in the server is established.

[0283] The block merging result of the target file is obtained according to the mapping relationship integration for the first mapping relationship and the second mapping relationship.

[0284] The specific implementation of each operation can be referred to the foregoing embodiments, and will not be described here.

[0285] The computer readable storage medium can include a read only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0286] The computer readable storage medium stores instructions, and the instructions can execute the steps of any file transmission method provided by the embodiments of the present application, so that the beneficial effects of any file transmission method provided by the embodiments of the present application can be achieved. Details are described in the foregoing embodiments, and will not be described here.

[0287] According to an aspect of the present application, a computer program product or computer program is also provided, which includes computer instructions stored in a computer readable storage medium. The processor of the electronic device reads the computer instructions from the computer readable storage medium, and the processor executes the computer instructions to make the electronic device execute the method provided in various optional implementation manners in the foregoing embodiments.

[0288] The file transmission method, device, electronic device, storage medium and program product provided by the embodiments of the present application are described in detail above, and the principles and implementation manners of the present application are described by applying specific examples. The above embodiment is only used to help understand the method and core idea of the present application; meanwhile, for those skilled in the art, the specific implementation manner and application range can be changed according to the idea of the present application, and the above description should not be understood as a limitation of the present application.

Claims

1. A file transfer method characterized by, The method comprises the following steps: Obtaining at least one target file block of a target file from received file blocks, wherein the block information carried by the target file block comprises a file identifier of the target file and a block sequence number of each target file block in the target file blocks of the target file; Based on the block size and the block sequence number of each target file block, a first mapping relationship corresponding to the file identifier is established, which comprises: determining the block mode of the target file according to the block size of each target file block; when the block mode is fixed-length block, the block size of the target file block is taken as the first mapping relationship corresponding to the file identifier; when the block mode is non-fixed-length block, the byte offset of each target file block is determined according to the block size of each target file block; based on the byte offset and the block sequence number of the same target file block, the first mapping relationship corresponding to the file identifier is established; the first mapping relationship is the mapping relationship between the byte offset of each target file block and the block sequence number; Establishing a second mapping relationship between the block sequence number and each target file block stored in the server; According to the mapping relationship integration for the first mapping relationship and the second mapping relationship, the block merging result of the target file is obtained.

2. The file transfer method of claim 1, wherein, When the block mode is non-fixed-length block, the byte offset of each target file block is determined according to the block size of each target file block, which comprises: When the block mode is non-fixed-length block, any target file block with an undetermined byte offset is taken as a first file block; From the target file blocks, a second file block with a block sequence number smaller than that of the first file block is determined; Based on the sum of the block size of each second file block and the block size of the first file block, the byte offset of the first file block is obtained; The step of taking any target file block with an undetermined byte offset as a first file block is returned until the byte offset of each target file block is determined.

3. The file transfer method of claim 1, wherein, The method further comprises: Receiving a block download request, wherein the block download request comprises a target download offset of a to-be-downloaded block in a target download file and a target file identifier of the target download file; Determining a first mapping relationship and a second mapping relationship corresponding to the target file identifier; According to the target download offset and the first mapping relationship, a download block sequence number of the to-be-downloaded block in the target download file is determined; According to the download block sequence number and the second mapping relationship, a target download file block corresponding to the download block sequence number is transmitted.

4. The file transfer method of claim 1, wherein, The method further comprises: Receiving a file download request, wherein the file download request comprises a target file identifier of a target download file; Determining a first mapping relationship and a second mapping relationship corresponding to the target file identifier; According to the first mapping relationship, a download block sequence number of all file blocks in the target download file is determined; According to the download block sequence number and the second mapping relationship, a target download file block corresponding to each download block sequence number is transmitted.

5. The file transfer method of claim 3, wherein, The file block is a fixed-length block, and the first mapping relationship includes a block size of the file block. The determining of the block sequence number of the to-be-downloaded block in the target download file according to the target download offset and the first mapping relationship includes: calculating a quotient between the target download offset and the block size of the file block in the first mapping relationship; determining the block sequence number of the to-be-downloaded block in the target download file according to the quotient.

6. The file transfer method according to any one of claims 3-4, wherein, The second mapping relationship corresponds to the first mapping relationship. The determining of the first mapping relationship and the second mapping relationship corresponding to the target download file identifier includes: determining the first mapping relationship corresponding to the target download file identifier; determining the second mapping relationship corresponding to the first mapping relationship according to the first mapping relationship.

7. The file transfer method of claim 3, wherein, The determining of the download block sequence number of the to-be-downloaded block in the target download file according to the target download offset and the first mapping relationship includes: performing offset sorting on the block sequence number according to a byte offset in the first mapping relationship to obtain a sorted first mapping relationship; performing binary search on the byte offset in the first mapping relationship according to the target download offset to determine a target byte offset corresponding to the target download offset; in the first mapping relationship, determining a block sequence number corresponding to the target byte offset as the download block sequence number of the to-be-downloaded block in the target download file.

8. The file transfer method of claim 1, wherein, The obtaining of the block merging result of the target file according to the mapping relationship integration performed on the first mapping relationship and the second mapping relationship includes: determining mapping relationship data corresponding to the first mapping relationship and the second mapping relationship; performing format conversion on the mapping relationship data to obtain mapping relationship data in a target format; obtaining the block merging result of the target file according to the mapping relationship data in the target format.

9. The file transfer method of claim 8, wherein, The determining of the mapping relationship data corresponding to the first mapping relationship and the second mapping relationship includes: obtaining mapping relationship data from a storage location storing the first mapping relationship and the second mapping relationship; The format conversion on the mapping relationship data to obtain mapping relationship data in a target format includes: copying the mapping relationship data into a data storage file to obtain mapping relationship data in a target format.

10. A file transfer apparatus characterized by comprising: The method includes: a block obtaining unit configured to obtain at least one target file block of a target file from received file blocks, the block information carried by the target file block including a file identifier of the target file and a block sequence number of each target file block in blocks of the target file; The first mapping relationship establishing unit is configured to establish a first mapping relationship corresponding to the file identifier based on the chunk size of each target file chunk and the chunk sequence number, including: determining a chunking manner of the target file according to the chunk size of each target file chunk; when the chunking manner is fixed-length chunking, taking the chunk size of the target file chunk as the first mapping relationship corresponding to the file identifier; when the chunking manner is non-fixed-length chunking, determining a byte offset of each target file chunk according to the chunk size of each target file chunk; and establishing the first mapping relationship corresponding to the file identifier based on the byte offset and the chunk sequence number of the same target file chunk; the first mapping relationship is a mapping relationship between the byte offset of each target file chunk and the chunk sequence number; The second mapping relationship establishing unit is configured to establish a second mapping relationship between the chunk sequence number and each target file chunk stored in the server. The chunk merging unit is configured to obtain a chunk merging result of the target file according to the mapping relationship integration performed on the first mapping relationship and the second mapping relationship.

11. An electronic device, comprising: The computer readable storage medium stores a plurality of instructions adapted to be loaded by the processor to execute the steps in the file transmission method according to any one of claims 1 to 9.

12. A computer-readable storage medium, characterized in that, The computer readable storage medium stores a plurality of instructions adapted to be loaded by the processor to execute the steps in the file transmission method according to any one of claims 1 to 9.

13. A computer program product comprising computer programs or instructions, characterized in that, The computer program or instructions, when executed by the processor, implement the steps in the file transmission method according to any one of claims 1 to 9.

Images