File transmission method and device, storage medium, and electronic device

Abstract

The present invention discloses a file transfer method and device, a storage medium, and an electronic device, which relate to the field of big data. The method comprises: responding to a file transfer request, obtaining a target file to be transferred, file attribute information, and a file transfer configuration; calculating the total number of concurrent transfer tasks and the maximum file segment interval that each transfer task can transfer based on the file attribute information and the file transfer configuration; segmenting the target file based on the total number of concurrent transfer tasks and the maximum file segment interval to obtain a segmented file set; determining the transfer task corresponding to the segmented file based on the metadata information of each segmented file to obtain a task set; executing each transfer task in the task set based on the number of concurrent tasks to transfer the target file to a server. The present invention solves the technical problem in the related art that when breakpoint resume is used for file transfer, it cannot be applied to a multi-threaded concurrent file upload network scenario.

Description

Technical Field

[0001] The present invention relates to the field of big data technology, and in particular to a file transmission method and device, a storage medium, and an electronic device. Background Art

[0002] In daily work, the system often encounters files that occupy a large amount of memory when uploading files. Due to limitations of factors such as bandwidth and network configuration, uploading large files often takes a long time. If a network problem occurs during the upload process, the uploaded files will need to be retransmitted.

[0003] To address the above issues, a resumable upload strategy is usually used for uploading large files. This strategy means first dividing the large file into several small files, and then uploading each small file separately. After the upload is completed, the several small files are merged and restored into the large file. When the upload is interrupted, only the interrupted small files need to be retransmitted.

[0004] In related technologies, the resumable upload strategy mainly adopts the method of uploading directly based on the offset, usually with different lengths between segments. It can have strong adaptability to different network environments, but is not suitable for scenarios using multi-threaded concurrent uploads.

[0005] To address the above-mentioned problems, no effective solutions have been proposed so far. Summary of the Invention

[0006] The embodiments of the present invention provide a file transmission method and device, a storage medium, and an electronic device to at least solve the technical problem in the related art that when using breakpoint resume for file transmission, it cannot be applied to the multi-threaded concurrent file upload network scenario.

[0007] According to one aspect of an embodiment of the present invention, a file transfer method is provided, which is applied to a client and includes: responding to a file transfer request, obtaining a target file to be transferred, file attribute information and a file transfer configuration, wherein the file transfer configuration includes at least: a maximum number of concurrent transmissions and a file size that can be transferred at a single time; based on the file attribute information and the file transfer configuration, calculating the total number of concurrent transmission tasks and the maximum file segment interval that each transmission task can transmit; based on the total number of concurrent transmission tasks and the maximum file segment interval, segmenting the target file to obtain a segmented file set, wherein the segmented file set contains N segmented files, and N is a positive integer greater than or equal to 1; based on the metadata information of each segmented file, determining the transmission task corresponding to the segmented file to obtain a task set, wherein the metadata information includes at least: a starting offset of the segmented file during transmission, the file attribute information and the number of concurrent tasks to be executed; based on the number of concurrent tasks, executing each of the transmission tasks in the task set to transfer the target file to a server.

[0008] Optionally, the step of responding to a file transfer request and obtaining the target file to be transferred, file attribute information and file transfer configuration includes: responding to the file transfer request, obtaining the target file to be transferred and file application requirements, wherein the file application requirements are used to characterize parameters of interest in the file content and / or file name of the target file; calculating a cryptographic hash value of the target file based on the parameters of interest; sending the cryptographic hash value to the server and receiving the file transfer configuration issued by the server; extracting the file size of the target file, and determining the file attribute information based on the file size and the cryptographic hash value.

[0009] Optionally, after sending the password hash value to the server, it also includes: the server queries the database whether there is file upload information corresponding to the password hash value, wherein the file upload information includes: the number of historical concurrent tasks, the starting offset and the ending offset of the segmented files of the historical transmission; if there is file upload information corresponding to the password hash value in the database, the server sends the file upload information and the file transfer configuration to the client; if there is no file upload information corresponding to the password hash value in the database, the server sends the file transfer configuration to the client.

[0010] Optionally, based on the file attribute information and the file transfer configuration, the step of calculating the total number of concurrent transmission tasks and the maximum file segment interval that each transmission task can transmit includes: calculating the maximum file segment interval that each transmission task can transmit based on the single transmittable file size and the file size of the target file; and determining the total number of concurrent transmission tasks based on the maximum file segment interval.

[0011] Optionally, the step of segmenting the target file based on the total number of concurrent transmission tasks and the maximum file segment interval to obtain a segmented file set includes: evenly segmenting the target file based on the total number of concurrent transmission tasks and the maximum file segment interval to obtain the segmented file set, wherein a parallel strategy is adopted for transmission between each of the segmented files, and a serial strategy is adopted for transmission within the segmented file.

[0012] Optionally, after segmenting the target file to obtain a segmented file set, it also includes: in the case where the file upload information sent by the server is not received, determining the starting offset and ending offset of the segmented file to be transmitted currently based on the maximum file segment interval and the file size of the segmented file; in the case where the file upload information sent by the server is received, based on the starting offset and ending offset of the segmented file transmitted historically, excluding the file segment interval of the segmented file transmitted historically, determining the starting offset and ending offset of the segmented file to be transmitted currently.

[0013] Optionally, after segmenting the target file to obtain a segmented file set, it also includes: when the file upload information sent by the server is not received, determining the number of concurrent tasks to be executed based on the maximum concurrent transmission number; or, when the file upload information sent by the server is received, determining the number of concurrent tasks to be executed based on the historical number of concurrent tasks.

[0014] Optionally, after segmenting the target file based on the total number of concurrent transmission tasks and the maximum file segment interval to obtain a segmented file set, it also includes: obtaining the network communication status and network transmission speed between the client and the server; and adjusting the number of concurrent tasks to be executed and the file size of the segmented file based on the network communication status and the network transmission speed.

[0015] Optionally, after executing each of the transmission tasks in the task set and transmitting the target file to the server, it also includes: receiving an interrupted resume transmission instruction, wherein the interrupted resume transmission instruction means that a transmission interruption state occurs during the transmission of the target file, and the file needs to be uploaded to the server again; responding to the interrupted resume transmission instruction, obtaining the historical file upload information and the file transfer configuration issued by the server; based on the historical file upload information and the file transfer configuration, querying the starting offset of each segmented file in the target file to resume file transmission.

[0016] Optionally, after executing each of the transmission tasks in the task set and transmitting the target file to the server, it also includes: the server saving the metadata information based on the file attribute information; the server writing the segmented file to the specified location of the space storing the target file based on the starting offset and ending offset in the metadata information; after all the segmented files are transferred, the server combines N of the segmented files according to the starting offset of the segmented files at the time of transmission and the file attribute information to obtain the target file.

[0017] According to another aspect of an embodiment of the present invention, a file transfer device is provided, which is applied to a client and includes: a first acquisition unit, configured to respond to a file transfer request and acquire a target file to be transferred, file attribute information, and a file transfer configuration, wherein the file transfer configuration includes at least a maximum number of concurrent transmissions and a file size that can be transferred at a single time; a calculation unit, configured to calculate, based on the file attribute information and the file transfer configuration, the total number of concurrent transmission tasks and the maximum file segment interval that can be transferred by each transmission task; a segmentation unit, configured to segment the target file based on the total number of concurrent transmission tasks and the maximum file segment interval to obtain a segmented file set, wherein the segmented file set includes N segmented files, where N is a positive integer greater than or equal to 1; a determination unit, configured to determine, based on metadata information of each segmented file, a transmission task corresponding to the segmented file to obtain a task set, wherein the metadata information includes at least a starting offset of the segmented file during transmission, the file attribute information, and the number of concurrent tasks to be executed; and a transmission unit, configured to execute each transmission task in the task set based on the number of concurrent tasks to transfer the target file to a server.

[0018] Optionally, the first acquisition unit includes: a first acquisition module, used to respond to a file transfer request, obtain the target file to be transferred and the file application requirement, wherein the file application requirement is used to characterize the parameters of interest in the file content and / or file name of the target file; a first calculation module, used to calculate the cryptographic hash value of the target file based on the parameters of interest; a first receiving module, used to send the cryptographic hash value to the server and receive the file transfer configuration issued by the server; a first determination module, used to extract the file size of the target file, and determine the file attribute information based on the file size and the cryptographic hash value.

[0019] Optionally, the first acquisition unit also includes: a first query module, used by the server to query whether there is file upload information corresponding to the password hash value in the database, wherein the file upload information includes: the number of historical concurrent tasks, the starting offset and the ending offset of the segmented files of the historical transmission; a first sending module, used by the server to send the file upload information and the file transfer configuration to the client if there is file upload information corresponding to the password hash value in the database; a second sending module, used by the server to send the file transfer configuration to the client if there is no file upload information corresponding to the password hash value in the database.

[0020] Optionally, the calculation unit includes: a second calculation module, used to calculate the maximum file segment interval that each transmission task can transmit based on the single transmittable file size and the file size of the target file; a second determination module, used to determine the total number of concurrent transmission tasks based on the maximum file segment interval.

[0021] Optionally, the segmentation unit includes: a third determination module for determining the starting offset and ending offset of the segmented file to be transmitted based on the maximum file segment interval and the file size of the segmented file when the file upload information sent by the server is not received; a fourth determination module for determining the starting offset and ending offset of the segmented file to be transmitted based on the starting offset and ending offset of the segmented file transmitted historically, excluding the file segment interval of the segmented file transmitted historically, when the file upload information sent by the server is received.

[0022] Optionally, the segmentation unit also includes: a segmentation module, which is used to evenly segment the target file based on the total number of concurrent transmission tasks and the maximum file segment interval to obtain the segmented file set, wherein a parallel strategy is adopted for transmission between each of the segmented files, and a serial strategy is adopted for transmission within the segmented file.

[0023] Optionally, the segmentation unit also includes: a fifth determination module, used to determine the number of concurrent tasks to be executed based on the maximum concurrent transmission number when the file upload information sent by the server is not received; or, a sixth determination module, used to determine the number of concurrent tasks to be executed based on the historical number of concurrent tasks when the file upload information sent by the server is received.

[0024] Optionally, the file transfer device also includes: a second acquisition unit, used to obtain the network communication status and network transmission speed between the client and the server; an adjustment unit, used to adjust the number of concurrent tasks to be executed and the file size of the segmented file based on the network communication status and the network transmission speed.

[0025] Optionally, the file transfer device also includes: a receiving unit for receiving an interrupted resume instruction, wherein the interrupted resume instruction means that a transmission interruption state occurs during the transmission of the target file, and the file needs to be uploaded to the server again; a responding unit for responding to the interrupted resume instruction, obtaining the historical file upload information and the file transfer configuration issued by the server; and a query unit for querying the starting offset of each segmented file in the target file based on the historical file upload information and the file transfer configuration, and resuming the file transfer.

[0026] Optionally, the file transfer device also includes: a saving unit, used by the server to save the metadata information based on the file attribute information; a writing unit, used by the server to write the segmented file to a specified location in the space storing the target file based on the starting offset and ending offset in the metadata information; and a combining unit, used by the server to combine N segmented files according to the starting offset of the segmented files during transmission and the file attribute information after all the segmented files are transferred to obtain the target file.

[0027] According to another aspect of an embodiment of the present invention, a computer-readable storage medium is further provided, wherein the computer-readable storage medium includes a stored computer program, wherein when the computer program is running, the device where the computer-readable storage medium is located is controlled to execute any one of the file transfer methods described above.

[0028] According to another aspect of an embodiment of the present invention, an electronic device is also provided, comprising one or more processors and a memory, wherein the memory is used to store one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors implement any one of the file transfer methods described above.

[0029] In the present disclosure, the following steps are adopted: responding to a file transfer request, obtaining the target file to be transferred, file attribute information and file transfer configuration; based on the file attribute information and the file transfer configuration, calculating the total number of concurrent transfer tasks and the maximum file segment interval that each transfer task can transfer, wherein the file transfer configuration at least includes: the maximum number of concurrent transfers and the size of the file that can be transferred at a single time; based on the total number of concurrent transfer tasks and the maximum file segment interval, segmenting the target file to obtain a segmented file set, wherein the segmented file set contains N segmented files, N is a positive integer greater than or equal to 1; based on the metadata information of each segmented file, determining the transfer task corresponding to the segmented file to obtain a task set, wherein the metadata information at least includes: the starting offset of the segmented file during transmission, file attribute information and the number of concurrent tasks to be executed; based on the number of concurrent tasks, executing each transfer task in the task set to transfer the target file to the server.

[0030] In the present disclosure, by calculating the total number of concurrent transmission tasks and the maximum file segment interval that each transmission task can transmit, the file is fixedly segmented, and combined with the offset of the segmented file, large file multi-task concurrent transmission and single task offset-based transmission are realized. For large files that need to be uploaded, a fixed segmentation method is adopted, and multi-task concurrent transmission is used to improve the file transmission speed (that is, multiple segmented files can be transmitted at one time). At the same time, for single task transmission, an offset-based transmission method is adopted, so that a single task can inherit the advantages of offset upload and can dynamically adjust the segment size based on different network conditions (when segmenting, it can be less than the maximum file segment interval), thereby solving the technical problem that when using breakpoint resumption for file transmission in related technologies, it cannot be applied to multi-threaded concurrent file upload network scenarios.

[0031] In the present disclosure, the file is evenly segmented according to the transmission configuration returned by the server; after segmentation, the transmission file size of each segment is adjusted according to the network transmission situation; the segmented files are transmitted in parallel and serially within the segment.

[0032] In the present disclosure, when a breakpoint resuming is required, the transmission configuration of the server can be obtained, the starting point of each segment can be found, and the file can be resuming the transfer.

[0033] In the present disclosure, specifically for a single file transfer task, an offset upload scheme can be adopted, so that a single task can inherit the advantages of offset upload (the lengths between segments can be different, and it can adapt to different network environments and has strong adaptability), can flexibly adapt to different network conditions / network transmission scenarios, and can split a single file segment into larger or smaller segments. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] The drawings described herein are used to provide a further understanding of the present invention and constitute a part of this application. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation of the present invention. In the drawings:

[0035] Figure 1 is a flow chart of an optional file transmission method according to an embodiment of the present invention;

[0036] Figure 2 This is an optional flowchart for obtaining a target file to be transferred, file attribute information, and file transfer configuration according to an embodiment of the present invention;

[0037] Figure 3 This is an optional file transfer system architecture diagram according to an embodiment of the present invention;

[0038] Figure 4 is a schematic diagram of an optional file transmission device according to an embodiment of the present invention;

[0039] Figure 5 The present invention is a block diagram of the hardware structure of an electronic device (or mobile device) according to a file transmission method according to an embodiment of the present invention. DETAILED DESCRIPTION

[0040] In order to enable those skilled in the art to better understand the solutions of the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the embodiments described are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts should fall within the scope of protection of the present invention.

[0041] It should be noted that the terms "first", "second", etc. in the description and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that the numbers used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0042] To facilitate those skilled in the art to understand the present invention, some terms or nouns involved in the embodiments of the present invention are explained below:

[0043] MD5, the abbreviation of Message-Digest Algorithm 5, is a widely used cryptographic hash function that can generate a 128-bit (16-byte) hash value to ensure the integrity and consistency of information transmission. The MD5 value is equivalent to the file ID and its value is unique.

[0044] It should be noted that the file transfer method and device in the present disclosure can be used in the field of big data. When multi-task concurrent transmission and offset-based transmission of large files are adopted, they can also be used in any field other than the field of big data. When multi-task concurrent transmission and offset-based transmission of large files are adopted, the application field of the file transfer method and device in the present disclosure is not limited.

[0045] It should be noted that the relevant information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for display and analysis, etc.) involved in this disclosure are all information and data authorized by the user or fully authorized by all parties. For example, an interface is set up between this system and the relevant user or organization. Before obtaining relevant information, it is necessary to send an acquisition request to the aforementioned user or organization through the interface, and obtain the relevant information after receiving the consent information fed back by the aforementioned user or organization.

[0046] The present invention can be applied to various file transfer devices / apparatuses / products / software. By calculating the total number of concurrent transmission tasks and the maximum file segment interval that can be transmitted by each transmission task, combined with the segmented file offset, the present invention can simultaneously realize the concurrent transmission of large files by multiple tasks and the offset-based transmission of a single task.

[0047] In the present invention, for large file transfer, a fixed segmentation method is adopted to perform multi-task concurrent transmission to improve the file transmission speed. At the same time, specifically for single task transmission, an offset-based transmission method is adopted, so that a single task can inherit the advantages of offset upload, meet different network implementation scenarios, adapt to different network conditions, and realize dynamic adjustment of file segment size.

[0048] The present invention is described in detail below with reference to various embodiments.

[0049] Example 1

[0050] According to an embodiment of the present invention, a file transfer method embodiment is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0051] The present invention will be described below in conjunction with preferred implementation steps. Figure 1 is a flow chart of an optional file transmission method according to an embodiment of the present invention. Figure 1 As shown, the method includes the following steps:

[0052] Step S101: In response to a file transfer request, obtain the target file to be transferred, file attribute information, and file transfer configuration, wherein the file transfer configuration includes at least: the maximum number of concurrent transfers and the size of a file that can be transferred at a time;

[0053] Step S102: Calculate the total number of concurrent transmission tasks and the maximum file segment interval that can be transmitted by each transmission task based on the file attribute information and the file transmission configuration;

[0054] Step S103: Segment the target file based on the total number of concurrent transmission tasks and the maximum file segment interval to obtain a segment file set, wherein the segment file set includes N segment files, where N is a positive integer greater than or equal to 1;

[0055] Step S104: determining the transmission tasks corresponding to the segmented files based on the metadata information of each segmented file to obtain a task set, wherein the metadata information includes at least: the starting offset of the segmented file during transmission, file attribute information, and the number of concurrent tasks to be executed;

[0056] Step S105 , executing each transmission task in the task set based on the number of concurrent tasks, and transmitting the target file to the server.

[0057] Through the above steps, first respond to the file transfer request, obtain the target file to be transferred, file attribute information and file transfer configuration, wherein the file transfer configuration at least includes: the maximum number of concurrent transmissions and the size of the file that can be transferred at a single time; based on the file attribute information and the file transfer configuration, calculate the total number of concurrent transmission tasks and the maximum file segment interval that each transmission task can transmit; based on the total number of concurrent transmission tasks and the maximum file segment interval, segment the target file to obtain a segmented file set, wherein the segmented file set contains N segmented files, and N is a positive integer greater than or equal to 1; based on the metadata information of each segmented file, determine the transmission task corresponding to the segmented file to obtain a task set, wherein the metadata information at least includes: the starting offset of the segmented file during transmission, file attribute information and the number of concurrent tasks to be executed; based on the number of concurrent tasks, execute each transmission task in the task set to transfer the target file to the server.

[0058] In this embodiment, when resuming the transfer of large files, fixed file segmentation is achieved by calculating the total number of concurrent transmission tasks and the maximum file segment interval that each transmission task can transfer. Combined with the segmented file offset, multi-task concurrent transmission of large files and offset-based transmission of a single task are achieved at the same time. This not only improves the transmission speed of large files, but also enables a single transmission task to inherit the advantages of offset upload, and can adapt to multi-threaded concurrent file transfer in different network scenarios, thereby solving the technical problem in related technologies that when using breakpoint resumption for file transfer, it cannot be applied to multi-threaded concurrent file upload network scenarios.

[0059] The file transmission method provided in the embodiment of the present invention is schematically illustrated by taking a client as an implementation subject. The client pre-establishes a network connection with a server to implement interactive operations such as file uploading and file downloading.

[0060] The following describes the above steps in detail.

[0061] Step S101: respond to a file transfer request, obtain a target file to be transferred, file attribute information, and file transfer configuration, wherein the file transfer configuration at least includes: a maximum number of concurrent transfers and a single transferable file size.

[0062] The maximum number of concurrent transfers refers to the maximum number of file transfers that can be processed at a given point in time based on the client's actual processing capabilities. This maximum number of concurrent transfers can be determined based on the network conditions between the client and server, the client's memory size, and the number of available processor threads. The size of a file that can be transferred at a single time can be determined based on the network conditions at the most recent historical point in time.

[0063] Figure 2 FIG. 1 is an optional flowchart of obtaining a target file to be transferred, file attribute information, and file transfer configuration according to an embodiment of the present invention. Figure 2 As shown, the specific implementation steps of step S101 include:

[0064] Step S201 : In response to a file transfer request, a target file to be transferred and a file application requirement are obtained, wherein the file application requirement is used to represent parameters of interest in the file content and / or file name of the target file.

[0065] A file transfer request may be a request sent by a user via a terminal device such as a PC, mobile terminal, panel, or iPad. The file transfer request may be transmitted in the form of a file data packet plus a request instruction. The target file to be transferred may be any file format, such as a Word document, a table, an image, a video, a TXT file, or any combination thereof. The number of files transferred simultaneously may be one or more. The specific file size, file type, and number of files to be transferred are not specifically limited in this embodiment and are subject to the actual file transfer process.

[0066] This embodiment mainly transfers larger files. The larger files / large files are adjusted based on the real-time network conditions of each terminal and the thread processing conditions of the operating memory. For example, files with a size greater than 100M are defined as large files, and files with a size less than 100M are defined as small files.

[0067] Step S202, calculating the cryptographic hash value of the target file based on the parameter of interest;

[0068] It should be noted that in this embodiment, the client can represent the cryptographic hash value by calculating the MD5 value of the file. The cryptographic hash value can be the MD5 value or other types of hash values. This embodiment uses the MD5 value for illustration. The calculation of the cryptographic hash value is selected based on the user's application requirements. The application requirements are obtained by calculating the parameters of interest to the user. For example, if the user is sensitive to file content, only the cryptographic hash value of the file content needs to be calculated. If the user is sensitive to both file content and file name, the cryptographic hash value of both file content and file name needs to be calculated.

[0069] Step S203: Send the password hash value to the server and receive the file transfer configuration sent by the server.

[0070] The client can carry the cryptographic hash value of the target file to query the server for the file upload information and the server's file upload configuration.

[0071] Step S204: extract the file size of the target file, and determine the file attribute information based on the file size and the cryptographic hash value.

[0072] The cryptographic hash value is used as the unique identification of the file. After calculating the cryptographic hash value of the file, the client carries the cryptographic hash value of the file to query the server for the file upload information and the server's upload configuration for the file.

[0073] In an embodiment of the present invention, after sending the password hash value to the server, it also includes: the server queries whether there is file upload information corresponding to the password hash value in the database, wherein the file upload information includes: the number of historical concurrent tasks, the starting offset and the ending offset of the segmented files of the historical transmission; if there is file upload information corresponding to the password hash value in the database, the server sends the file upload information and the file transfer configuration to the client; if there is no file upload information corresponding to the password hash value in the database, the server sends the file transfer configuration to the client.

[0074] It should be noted that the server checks the client-generated cryptographic hash value to see if there is any upload information for the file represented by the cryptographic hash value. If so, the server returns the upload information along with the server's basic file upload configuration information to the client. Otherwise, the server returns only the basic file upload configuration information to the client.

[0075] The server's basic configuration information for file uploads includes the maximum number of concurrent transmissions and the size of the file that can be transmitted at a time. The server's historical file upload information includes the number of historical concurrent tasks, the starting and ending offsets of the segmented files of historical transmissions, and the file's cryptographic hash value.

[0076] Step S102: Based on the file attribute information and the file transmission configuration, the total number of concurrent transmission tasks and the maximum file segment interval that can be transmitted by each transmission task are calculated.

[0077] In an embodiment of the present invention, the steps of calculating the total number of concurrent transmission tasks and the maximum file segment interval that each transmission task can transmit based on file attribute information and file transfer configuration include: calculating the maximum file segment interval that each transmission task can transmit based on the file size that can be transmitted in a single time and the file size of the target file; and determining the total number of concurrent transmission tasks based on the maximum file segment interval.

[0078] Optionally, in this embodiment, when determining the total number of concurrent transmission tasks based on the maximum file segment interval, the specific steps include: calculating the total number of concurrent transmission tasks based on the maximum number of concurrent transmissions, the file size that can be transmitted at a single time, and the maximum file segment interval that each transmission task can transmit.

[0079] It should be noted that the server segments the transferred files by calculating the maximum number of concurrent transfer tasks and the maximum file segment interval. Based on the relevant information returned by the server to the client, if the server has upload information of a file represented by a certain password hash value, then the server returns the historical upload information of the file, and the client needs to reinitialize the upload configuration information at the next time point according to the historical upload information. For example, the client needs to set the number of concurrent transfer tasks for this upload according to the historical concurrent task number set in the returned information; if it does not exist, the client can set the maximum number of tasks not greater than the maximum number configured by the server according to the actual situation, and then calculate the maximum file segment interval that needs to be uploaded for each concurrent transfer task. The calculation formula is:

[0080] [filesize / concurrent]=chunk;

[0081] filesize%concurrent=mod;

[0082] Where filesize represents the total file size, concurrent represents the number of concurrent transfer tasks set by the client, [] indicates rounding the result, chunk represents the file size of each segmented interval obtained by evenly dividing the file size according to the number of concurrent transfer tasks set by the client and rounding it down, and mod represents the remainder obtained by evenly dividing the file size according to the number of concurrent transfer tasks set by the client.

[0083] The remainder mod will fall into the last chunk, that is, the transmission interval of the last chunk is chunk+mod, and then the successfully uploaded file segments returned by the server are sent to the corresponding tasks according to the interval segments.

[0084] Step S103 : segmenting the target file based on the total number of concurrent transmission tasks and the maximum file segment interval to obtain a segment file set, wherein the segment file set includes N segment files, where N is a positive integer greater than or equal to 1.

[0085] In this embodiment, the step of segmenting the target file based on the total number of concurrent transmission tasks and the maximum file segment interval to obtain a segmented file set includes: evenly segmenting the target file based on the total number of concurrent transmission tasks and the maximum file segment interval to obtain a segmented file set, wherein a parallel strategy is adopted for transmission between each segmented file, and a serial strategy is adopted for transmission within the segmented file.

[0086] It should be noted that after the client receives the transmission configuration returned by the server, it evenly segments the file. After segmentation, when transferring the file, parallel transmission is used between the segmented files, and serial transmission is used within each segmented file. In this embodiment, after segmenting the target file to obtain a set of segmented files, it also includes: in the case where the file upload information sent by the server is not received, the starting offset and ending offset of the segmented file to be transmitted are determined based on the maximum file segment interval and the file size of the segmented file; in the case where the file upload information sent by the server is received, the starting offset and ending offset of the segmented file to be transmitted are determined based on the starting offset and ending offset of the segmented file transmitted in the past, excluding the file segment interval of the segmented file transmitted in the past.

[0087] In this embodiment, after the target file is segmented to obtain a segmented file set, it also includes: when the file upload information sent by the server is not received, the number of concurrent tasks to be executed is determined based on the maximum concurrent transmission number; or, when the file upload information sent by the server is received, the number of concurrent tasks to be executed is determined based on the historical concurrent task number.

[0088] In this embodiment, after segmenting the target file based on the total number of concurrent transmission tasks and the maximum file segment interval to obtain a segmented file set, it also includes: obtaining the network communication status and network transmission speed between the client and the server; based on the network communication status and network transmission speed, adjusting the number of concurrent tasks to be executed and the file size of the segmented files.

[0089] It should be noted that during the file transfer process, the client can dynamically adjust the size of each transmitted segment according to the network transmission speed. If the network conditions are good, the segmentation can be set larger (smaller than the maximum file segment interval while meeting the network conditions and reasonably setting the size of each segmented file to be transmitted). If the network conditions are poor, the segmentation can be set smaller.

[0090] Step S104: Based on the metadata information of each segmented file, determine the transmission task corresponding to the segmented file and obtain a task set, wherein the metadata information at least includes: the starting offset of the segmented file during transmission, file attribute information and the number of concurrent tasks to be executed.

[0091] In this embodiment, in addition to transmitting the segmented file stream (specifically including the main data content of the file) to the server each time the client transmits, it also needs to transmit metadata information of other file streams (mainly recording the file location and file transfer status, which is convenient for the subsequent server to use when merging files), including the cryptographic hash value of the file, the starting offset and ending offset of the segmented file, file attribute information (for example, the total size of the file) and the set number of concurrent tasks.

[0092] In an embodiment of the present invention, after the file is segmented, each task of the client calculates the starting offset of the transmission and starts the transmission. If there is no historical transmission information, each task uses the starting offset of the interval as the initial position; otherwise, the file segment that has been successfully transmitted is excluded, and the segmented transmission starts from the next byte as the starting position. In addition to transmitting the segmented file stream to the server each time the client transmits, it also needs to transmit metadata information of other file streams, including but not limited to the file password hash value, the starting offset and ending offset of the file stream, the total size of the file, and the set number of concurrent tasks.

[0093] Step S105 , executing each transmission task in the task set based on the number of concurrent tasks, and transmitting the target file to the server.

[0094] Among them, when the client executes a transmission task, it may execute multiple tasks at the same time and transmit multiple files concurrently. During the transmission, segmented files can be transmitted out of order. When the server subsequently combines files, it can query the location of the target file to which the segmented file belongs through the sequence number and / or offset of the segmented file, and then complete the combination of the target file.

[0095] In this embodiment, after executing each transmission task in the task set and transmitting the target file to the server, it also includes: receiving an interrupted resume transmission instruction, wherein the interrupted resume transmission instruction means that during the process of transmitting the target file, a transmission interruption state occurs and the file needs to be uploaded to the server again; responding to the interrupted resume transmission instruction, obtaining the historical file upload information and file transfer configuration issued by the server; based on the historical file upload information and file transfer configuration, querying the starting offset of each segmented file in the target file, and resuming the file transmission.

[0096] In this embodiment, after executing each transmission task in the task set and transmitting the target file to the server, it also includes: the server saves metadata information based on the file attribute information; the server writes the segmented file to the specified location of the space storing the target file based on the starting offset and ending offset in the metadata information; after all the segmented files are transmitted, the server combines N segmented files according to the starting offset and file attribute information of the segmented files during transmission to obtain the target file.

[0097] In this embodiment, since files transmitted by different clients may be received at the same time, the server needs to use the metadata information transmitted by the client when merging the uploaded files, especially the cryptographic hash value of the file, to query which segmented files belong to the same target file, and then sort and combine the segmented files under the same target file according to the actual offset, cutoff offset, file size and other information of each segmented file, and summarize them into the same target file.

[0098] When determining whether the transmission of the segmented files is complete, it is possible to determine whether the file upload is complete based on the metadata information of the saved files and the total size of the segmented files.

[0099] Through the above embodiment, the total number of concurrent transmission tasks and the maximum file segment interval are calculated to segment the target file to obtain a segmented file set, and then combined with the offset of the segmented file, large file multi-task concurrent transmission and single task offset-based transmission are achieved at the same time.

[0100] In this embodiment, for large file transfers, a fixed segmentation method can be used, and multiple tasks can be transmitted concurrently to improve the file transmission speed; at the same time, for a single transmission task, an offset-based transmission method is adopted, so that a single task can inherit the advantages of offset upload and dynamically adjust the segment size based on different network conditions, thereby solving the technical problem in the related technology of using breakpoint resumption for file transfer, which is not applicable to multi-threaded concurrent file upload network scenarios.

[0101] Figure 3 FIG. 1 is an optional file transfer system architecture diagram according to an embodiment of the present invention. Figure 3 As shown in the figure, the file transfer system consists of two main bodies: the client and the server. The file transfer process is described in detail below:

[0102] Step 1: Calculate the file's MD5 value.

[0103] The client first calculates the md5 value of the file (corresponding to the above password hash value, Figure 3 The MD5 value is used as an example. The calculation of the MD5 value depends on the application's needs. If only the file content is sensitive, the MD5 value only needs to be calculated for the file content. If the file name is also sensitive, the file name can be included in the calculation. The client carries the MD5 value of the file to query the server for file upload information and the server's upload configuration.

[0104] Step 2: Query file upload information and service configuration information.

[0105] The server checks the MD5 value generated by the client to see if there is any upload information for the file represented by the MD5. If so, the server returns the upload information and the basic configuration information for file upload to the client. Otherwise, the server returns only the basic configuration information for file upload to the client.

[0106] The server's basic file upload configuration includes the maximum number of concurrent uploads supported for the same file and the maximum size of a single block. The server's historical file upload information includes the total file size, the number of concurrent tasks, the start and end offsets of the uploaded segments, and the file's MD5 value.

[0107] Step 3: Calculate the maximum number of concurrent upload tasks.

[0108] Step 4: Divide into segments according to the maximum number of tasks.

[0109] The client calculates the maximum number of concurrent upload tasks and segments the file. If the server returns historical upload information for the target file, the client needs to reinitialize the upload configuration according to the previous settings (i.e., the parameters set during the historical file transfer process). If the client needs to set the number of concurrent tasks for this upload according to the concurrent task number set in the returned information, if it does not exist, the client can set the maximum number of tasks according to the actual situation, which is not greater than the maximum number of tasks configured on the server. Then, the client calculates the maximum file segment range that needs to be uploaded for each concurrent task.

[0110] Step 5: Each task calculates the position offset of the file to be transferred within the segment.

[0111] Each task on the client side calculates the starting offset of the transmission and starts the transmission. If there is no historical transmission information, each task uses the starting offset of the interval as the initial position; otherwise, the segment that has been successfully transmitted is excluded, and the segmented transmission starts from the next byte as the starting position. In addition to transmitting the segmented file stream to the server each time the client transmits, it also needs to transmit the metadata information of other file streams. The metadata information may include but is not limited to: the md5 value of the file, the starting offset and ending offset of the file stream, the total size of the file, and the set number of concurrent tasks. During the transmission process, the client can dynamically adjust the size of each transmitted segment according to the network transmission speed. If the network conditions are good, the segment can be set larger. If the network conditions are poor, the segment can be set smaller.

[0112] Step 6: The server saves the file upload information and writes the segment information to the file

[0113] The server uses the uploaded file's MD5 value as a unique identifier for the file. It saves the file's upload metadata and writes the file stream to the file's corresponding offset, as described in the metadata. It then determines whether the file has been uploaded. If so, the server stops.

[0114] Step 7: Client upload completed.

[0115] Step 8: Server-side merging is complete.

[0116] Step 9: End.

[0117] The above implementation scheme combines the advantages of fixed segment transmission and offset transmission. It can not only adapt to the scenario of multi-task concurrent transmission and improve the file transmission speed, but also flexibly set the segment size according to different network conditions to give full play to the advantages of breakpoint resumption.

[0118] The present invention is described below in conjunction with another optional embodiment.

[0119] Example 2

[0120] This embodiment provides a file transmission device, and each implementation unit included in the file transmission device corresponds to each implementation step in the first embodiment.

[0121] Figure 4 FIG. 1 is a schematic diagram of an optional file transmission device according to an embodiment of the present invention. Figure 4 As shown, it includes: a first acquisition unit 41, a calculation unit 42, a segmentation unit 43, a determination unit 44, and a transmission unit 45, wherein:

[0122] The first acquisition unit 41 is configured to respond to a file transfer request and acquire a target file to be transferred, file attribute information, and a file transfer configuration, wherein the file transfer configuration includes at least: a maximum number of concurrent transfers and a single transferable file size;

[0123] A calculation unit 42 is configured to calculate the total number of concurrent transmission tasks and the maximum file segment interval that can be transmitted by each transmission task based on the file attribute information and the file transmission configuration;

[0124] A segmentation unit 43 is configured to segment the target file based on the total number of concurrent transmission tasks and the maximum file segment interval to obtain a segment file set, wherein the segment file set includes N segment files, where N is a positive integer greater than or equal to 1;

[0125] a determining unit 44 configured to determine, based on metadata information of each segmented file, a transmission task corresponding to the segmented file, and obtain a task set, wherein the metadata information includes at least: a starting offset of the segmented file during transmission, file attribute information, and a number of concurrent tasks to be executed;

[0126] The transmission unit 45 is configured to execute each transmission task in the task set based on the number of concurrent tasks, and transmit the target file to the server.

[0127] The above-mentioned file transfer device responds to the file transfer request through the first acquisition unit 41, obtains the target file to be transferred, file attribute information and file transfer configuration, wherein the file transfer configuration at least includes: the maximum number of concurrent transmissions and the size of the file that can be transferred at a single time; the calculation unit 42 calculates the total number of concurrent transmission tasks and the maximum file segment interval that can be transferred by each transmission task based on the file attribute information and the file transfer configuration; the segmentation unit 43 segments the target file based on the total number of concurrent transmission tasks and the maximum file segment interval to obtain a segmented file set, wherein the segmented file set contains N segmented files, and N is a positive integer greater than or equal to 1; the determination unit 44 determines the transmission task corresponding to the segmented file based on the metadata information of each segmented file to obtain a task set, wherein the metadata information at least includes: the starting offset of the segmented file during transmission, file attribute information and the number of concurrent tasks to be executed; the transmission unit 45 executes each transmission task in the task set based on the number of concurrent tasks to transfer the target file to the server.

[0128] In this embodiment, when resuming the transfer of large files, fixed file segmentation is achieved by calculating the total number of concurrent transmission tasks and the maximum file segment interval that each transmission task can transfer. Combined with the segmented file offset, multi-task concurrent transmission of large files and offset-based transmission of a single task are achieved at the same time. This not only improves the transmission speed of large files, but also enables a single transmission task to inherit the advantages of offset upload, and can adapt to multi-threaded concurrent file transfer in different network scenarios, thereby solving the technical problem in related technologies that when using breakpoint resumption for file transfer, it cannot be applied to multi-threaded concurrent file upload network scenarios.

[0129] The file transmission method provided in the embodiment of the present invention is schematically illustrated by taking a client as an implementation subject. The client pre-establishes a network connection with a server to implement interactive operations such as file uploading and file downloading.

[0130] Optionally, the first acquisition unit 41 includes: a first acquisition module, used to respond to a file transfer request, obtain the target file to be transferred and the file application requirement, wherein the file application requirement is used to characterize the parameters of interest in the file content and / or file name of the target file; a first calculation module, used to calculate the cryptographic hash value of the target file based on the parameters of interest; a first receiving module, used to send the cryptographic hash value to the server and receive the file transfer configuration issued by the server; a first determination module, used to extract the file size of the target file, and determine the file attribute information based on the file size and the cryptographic hash value.

[0131] Optionally, the first acquisition unit 41 also includes: a first query module, which is used for the server to query whether there is file upload information corresponding to the password hash value in the database, wherein the file upload information includes: the number of historical concurrent tasks, the starting offset and the ending offset of the segmented files of the historical transmission; a first sending module, which is used for the server to send the file upload information and the file transfer configuration to the client when there is file upload information corresponding to the password hash value in the database; and a second sending module, which is used for the server to send the file transfer configuration to the client when there is no file upload information corresponding to the password hash value in the database.

[0132] Optionally, the calculation unit 42 includes: a second calculation module, used to calculate the maximum file segment interval that each transmission task can transmit based on the file size that can be transmitted in a single time and the file size of the target file; a second determination module, used to determine the total number of concurrent transmission tasks based on the maximum file segment interval.

[0133] Optionally, the segmentation unit 43 includes: a third determination module for determining the starting offset and ending offset of the segmented file to be transmitted based on the maximum file segment interval and the file size of the segmented file when the file upload information sent by the server is not received; a fourth determination module for determining the starting offset and ending offset of the segmented file to be transmitted based on the starting offset and ending offset of the segmented file transmitted historically, excluding the file segment interval of the segmented file transmitted historically, when the file upload information sent by the server is received.

[0134] Optionally, the segmentation unit 43 also includes: a segmentation module, which is used to evenly segment the target file based on the total number of concurrent transmission tasks and the maximum file segment interval to obtain a segmented file set, wherein a parallel strategy is adopted for transmission between each segmented file, and a serial strategy is adopted for transmission within the segmented file.

[0135] Optionally, the segmentation unit 43 also includes: a fifth determination module, used to determine the number of concurrent tasks to be executed based on the maximum concurrent transmission number when the file upload information sent by the server is not received; or, a sixth determination module, used to determine the number of concurrent tasks to be executed based on the historical number of concurrent tasks when the file upload information sent by the server is received.

[0136] Optionally, the file transfer device also includes: a second acquisition unit, used to obtain the network communication status and network transmission speed between the client and the server; an adjustment unit, used to adjust the number of concurrent tasks to be executed and the file size of the segmented file based on the network communication status and network transmission speed.

[0137] Optionally, the file transfer device also includes: a receiving unit for receiving an interrupted resume instruction, wherein the interrupted resume instruction means that during the process of transmitting the target file, a transmission interruption state occurs and the file needs to be uploaded to the server again; a responding unit for responding to the interrupted resume instruction and obtaining the historical file upload information and file transfer configuration issued by the server; a query unit for querying the starting offset of each segmented file in the target file based on the historical file upload information and file transfer configuration to resume the file transfer.

[0138] Optionally, the file transfer device also includes: a saving unit, which is used by the server to save metadata information based on file attribute information; a writing unit, which is used by the server to write the segmented file to a specified location in the space where the target file is stored based on the starting offset and ending offset in the metadata information; and a combining unit, which is used by the server to combine N segmented files according to the starting offset and file attribute information of the segmented files during transmission to obtain the target file after all the segmented files are transferred.

[0139] The above-mentioned file transfer device can also include a processor and a memory. The above-mentioned first acquisition unit 41, calculation unit 42, segmentation unit 43, determination unit 44, transmission unit 45, etc. are all stored in the memory as program units, and the processor executes the above-mentioned program units stored in the memory to realize the corresponding functions.

[0140] The processor includes a kernel that retrieves the corresponding program unit from the memory. One or more kernels can be set. By adjusting kernel parameters, when transferring large files, fixed segment transmission and offset transmission methods can be combined to simultaneously achieve multi-task concurrent transmission of large files and offset-based transmission of a single task. This not only improves the transmission speed of large files, but also enables a single transmission task to inherit the advantages of offset upload, adapt to multiple concurrent transmission tasks in different scenarios, and improve file transmission speed. At the same time, the segment size can be flexibly set according to different network conditions to give full play to the advantages of breakpoint resumption.

[0141] The above-mentioned memory may include non-permanent memory in a computer-readable medium, random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM, and the memory includes at least one memory chip.

[0142] According to another aspect of an embodiment of the present invention, a computer-readable storage medium is provided, which includes a stored computer program, wherein when the computer program is running, the device where the computer-readable storage medium is located is controlled to execute any of the above-mentioned file transfer methods.

[0143] According to another aspect of an embodiment of the present invention, an electronic device is also provided, comprising one or more processors and a memory, wherein the memory is used to store one or more programs, wherein when the one or more programs are executed by one or more processors, the one or more processors implement any one of the above-mentioned file transfer methods.

[0144] The present application also provides a computer program product, which, when executed on a data processing device, is suitable for executing an initialized program having the following method steps: responding to a file transfer request, obtaining a target file to be transferred, file attribute information, and a file transfer configuration, wherein the file transfer configuration includes at least: a maximum number of concurrent transmissions and a file size that can be transferred at a single time; based on the file attribute information and the file transfer configuration, calculating the total number of concurrent transmission tasks and the maximum file segment interval that each transmission task can transmit; based on the total number of concurrent transmission tasks and the maximum file segment interval, segmenting the target file to obtain a segmented file set, wherein the segmented file set contains N segmented files, and N is a positive integer greater than or equal to 1; based on the metadata information of each segmented file, determining the transmission task corresponding to the segmented file to obtain a task set, wherein the metadata information includes at least: a starting offset of the segmented file during transmission, file attribute information, and the number of concurrent tasks to be executed; based on the number of concurrent tasks, executing each transmission task in the task set to transfer the target file to the server.

[0145] Figure 5 FIG. 1 is a hardware structure diagram of an electronic device (or mobile device) according to a file transmission method of an embodiment of the present invention. Figure 5 As shown, the electronic device may include one or more (illustrated as 502a, 502b, ..., 502n in the figure) processors 502 (the processor 502 may include but is not limited to a processing device such as a microprocessor MCU or a programmable logic device FPGA), a memory 504 for storing data. In addition, it may also include: a display, an input / output interface (I / O interface), a universal serial bus (USB) port (which may be included as one of the ports of the I / O interface), a network interface, a keyboard, a power supply and / or a camera. It will be understood by those skilled in the art that Figure 5 The structure shown is only for illustration and does not limit the structure of the above electronic device. Figure 5 More or fewer components than shown, or with Figure 5 Different configurations shown.

[0146] The serial numbers of the above embodiments of the present invention are for description only and do not represent the advantages or disadvantages of the embodiments.

[0147] In the above embodiments of the present invention, the description of each embodiment has its own focus. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0148] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. Among them, the device embodiments described above are only exemplary. For example, the division of units can be a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of units or modules, which can be electrical or other forms.

[0149] Units described as separate components may or may not be physically separate, and components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple units. Some or all of the units may be selected to achieve the purpose of the present embodiment according to actual needs.

[0150] In addition, the functional units in the various embodiments of the present invention may be integrated into a single processing unit, each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or software functional units.

[0151] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, server or network device, etc.) to execute all or part of the steps of the methods of each embodiment of the present invention. The aforementioned storage medium includes: U disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), mobile hard disk, magnetic disk or optical disk, etc. Various media that can store program codes.

[0152] The above are only preferred embodiments of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. A file transmission method, characterized in that: Applicable to scenarios where clients can resume large file transfers, including: Responding to a file transfer request, obtaining a target file to be transferred, file attribute information, and a file transfer configuration, including: responding to the file transfer request, obtaining the target file and file application requirements, the file application requirements being used to characterize parameters of interest in the file content and / or file name of the target file; calculating a cryptographic hash value of the target file based on the parameters of interest; sending the cryptographic hash value to a server, and receiving the file transfer configuration issued by the server; determining the file attribute information based on the file size of the target file and the cryptographic hash value; calculating the total number of concurrent transmission tasks and the maximum file segment interval that can be transmitted by each transmission task based on the file attribute information and the file transfer configuration, wherein the file transfer configuration at least includes: the maximum number of concurrent transmissions and the size of a file that can be transmitted at a single time; Segmenting the target file based on the total number of concurrent transmission tasks and the maximum file segment interval to obtain a segment file set, wherein the segment file set includes N segment files, where N is a positive integer greater than or equal to 1; Adjusting the number of concurrent tasks to be executed and the file size of the segmented files based on the network communication status and network transmission speed between the client and the server; Based on metadata information of each segmented file, determining a transmission task corresponding to the segmented file to obtain a task set, wherein the metadata information at least includes: a starting offset of the segmented file during transmission, the file attribute information, and a number of concurrent tasks to be executed; Based on the number of concurrent tasks, executing each of the transmission tasks in the task set to transmit the target file to the server; The server saves the metadata information based on the file attribute information; the server writes the segmented file to a specified location in the space storing the target file based on the starting offset and the ending offset in the metadata information; after all the segmented files are transferred, the server combines N of the segmented files according to the starting offset of the segmented files during transmission and the file attribute information to obtain the target file.

2. The method according to claim 1, characterized in that After sending the password hash value to the server, the method further includes: The server queries a database for file upload information corresponding to the password hash value, wherein the file upload information includes: a historical number of concurrent tasks, a start offset and an end offset of segmented files transmitted historically; If file upload information corresponding to the password hash value exists in the database, the server sends the file upload information and the file transfer configuration to the client; In a case where the file upload information corresponding to the password hash value does not exist in the database, the server sends the file transfer configuration to the client.

3. The method according to claim 2, characterized in that The step of calculating the total number of concurrent transmission tasks and the maximum file segment interval that can be transmitted by each transmission task based on the file attribute information and the file transmission configuration includes: Calculate the maximum file segment interval that can be transmitted by each transmission task based on the single transmittable file size and the file size of the target file; Based on the maximum file segment interval, the total number of concurrent transmission tasks is determined.

4. The method according to claim 2, characterized in that The step of segmenting the target file based on the total number of concurrent transmission tasks and the maximum file segment interval to obtain a segmented file set includes: Based on the total number of concurrent transmission tasks and the maximum file segment interval, the target file is evenly segmented to obtain the segmented file set, wherein the segmented files are transmitted in parallel and in serial.

5. The method according to claim 2, characterized in that After the target file is segmented to obtain a segmented file set, the method further includes: In the case where the file upload information sent by the server is not received, determining the starting offset and the ending offset of the segmented file to be currently transmitted based on the maximum file segment interval and the file size of the segmented file; When the file upload information sent by the server is received, the starting offset and ending offset of the segmented file to be transmitted are determined based on the starting offset and ending offset of the segmented file transmitted historically, excluding the file segment interval of the segmented file transmitted historically.

6. The method according to claim 2, characterized in that After the target file is segmented to obtain a segmented file set, the method further includes: In the case where the file upload information sent by the server is not received, determining the number of concurrent tasks to be executed based on the maximum number of concurrent transmissions; or When the file upload information sent by the server is received, the number of concurrent tasks to be executed is determined based on the historical number of concurrent tasks.

7. The method according to claim 1, characterized in that After executing each of the transmission tasks in the task set and transmitting the target file to the server, the method further includes: receiving a resume transmission instruction, wherein the resume transmission instruction indicates that a transmission interruption state occurs during the transmission of the target file, and the file needs to be uploaded to the server again; In response to the interrupted and resumed transmission instruction, the server obtains the historical file upload information and the file transmission configuration; Based on the historical file upload information and the file transfer configuration, the starting offset of each segmented file in the target file is queried to resume file transfer.

8. A file transmission device, characterized in that: Applicable to scenarios where clients can resume large file transfers, including: A first acquiring unit is configured to respond to a file transfer request and acquire a target file to be transferred, file attribute information, and a file transfer configuration, wherein the file transfer configuration includes at least: a maximum number of concurrent transfers and a single transferable file size; The first acquisition unit includes: a first acquisition module, configured to respond to a file transfer request and acquire a target file to be transferred and a file application requirement, wherein the file application requirement is used to characterize a parameter of interest in the file content and / or file name of the target file; a first calculation module, configured to calculate a cryptographic hash value of the target file based on the parameter of interest; a first receiving module, configured to send the cryptographic hash value to a server and receive the file transfer configuration issued by the server; and a first determination module, configured to extract a file size of the target file and determine the file attribute information based on the file size and the cryptographic hash value. A calculation unit, configured to calculate the total number of concurrent transmission tasks and the maximum file segment interval that can be transmitted by each transmission task based on the file attribute information and the file transmission configuration; A segmentation unit is configured to segment the target file based on the total number of concurrent transmission tasks and the maximum file segment interval to obtain a segment file set, wherein the segment file set includes N segment files, where N is a positive integer greater than or equal to 1; a second acquiring unit, configured to acquire a network communication status and a network transmission speed between the client and the server; an adjusting unit, configured to adjust the number of concurrent tasks to be executed and the file size of the segmented file based on the network communication status and the network transmission speed; a determining unit, configured to determine, based on metadata information of each segmented file, a transmission task corresponding to the segmented file, and obtain a task set, wherein the metadata information at least includes: a starting offset of the segmented file during transmission, the file attribute information, and a number of concurrent tasks to be executed; A transmission unit, configured to execute each transmission task in the task set based on the number of concurrent tasks, and transmit the target file to a server; The system further includes: a saving unit, configured for the server to save the metadata information based on the file attribute information; a writing unit, configured for the server to write the segmented files to a designated location in the space storing the target file based on the starting offset and the ending offset in the metadata information; and a combining unit, configured for the server to combine N segmented files according to the starting offset of the segmented files during transmission and the file attribute information to obtain the target file after all the segmented files have been transferred.

9. A computer-readable storage medium, characterized in that The computer-readable storage medium includes a stored computer program, wherein when the computer program is executed, the device where the computer-readable storage medium is located is controlled to execute the file transmission method according to any one of claims 1 to 7.

10. An electronic device, characterized in that: The device comprises one or more processors and a memory, wherein the memory is used to store one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors implement the file transfer method according to any one of claims 1 to 7.

Images