Data transmission method and device, electronic equipment and computer readable medium

Abstract

The invention discloses a data transmission method and device, electronic equipment and a computer readable medium, and relates to the technical field of data transmission, one specific implementation mode comprises the steps that a data transmission request is received, and all uploaded data fragments are acquired; in response to completion of uploading of the last data fragment, creating a temporary directory; pre-merging the stored uploaded data fragments into a temporary directory, and further calculating a first hash value of the pre-merged data fragments; acquiring a second hash value of the original data corresponding to each received data fragment, performing data integrity verification based on the first hash value and the second hash value, and in response to passing of the data integrity verification, executing a real merging process of the received data fragments to obtain target data; and setting automatic deletion time for the temporary directory, storing the target data to the target directory, and deleting the temporary directory when the automatic deletion time is up. And the security and accuracy of data transmission are improved.

Description

Technical Field

[0001] The present application relates to the technical field of data transmission, and in particular to a data transmission method, device, electronic device and computer-readable medium. Background Art

[0002] File merging: In some scenarios, you may need to merge multiple files into one file. For example, when transferring a file from one server to another, you can transfer the file in pieces and re-merge it into a single file at the receiving end. In data transmission systems in different business scenarios, files are no longer verified after being merged, resulting in poor data transmission security and accuracy. Summary of the invention

[0003] In view of this, the embodiments of the present application provide a data transmission method, device, electronic device and computer-readable medium, which can solve the problem that in the existing data transmission system in different business scenarios, no verification is performed after the files are merged, resulting in poor data transmission security and accuracy.

[0004] To achieve the above object, according to one aspect of an embodiment of the present application, a data transmission method is provided, including:

[0005] Receive data transmission requests and obtain each uploaded data segment;

[0006] In response to the last data shard being uploaded, a new temporary directory is created;

[0007] Pre-merge the saved uploaded data shards into a temporary directory, and then calculate the first hash value of the pre-merged data shards;

[0008] Obtaining a second hash value of the original data corresponding to each received data shard, performing a data integrity check based on the first hash value and the second hash value, and in response to passing the data integrity check, performing a real merging process on the received data shards to obtain target data;

[0009] Set an automatic deletion time for the temporary directory, store the target data in the target directory, and delete the temporary directory when the automatic deletion time is reached.

[0010] Optionally, before creating the temporary directory, the method further includes:

[0011] Get the unique string corresponding to each data shard, generate a new directory based on the unique string, and save each uploaded data shard to the new directory.

[0012] Optionally, pre-merge the saved uploaded data shards into a temporary directory, including:

[0013] Pre-merge the saved uploaded data segments into a temporary directory in the order of upload time.

[0014] Optionally, obtaining a second hash value of the original data corresponding to each received data shard includes:

[0015] The hash value carried by the last data shard is obtained as the second hash value of the original data.

[0016] Optionally, before creating the temporary directory, the method further includes:

[0017] In response to acquiring the data shard carrying the hash value, the data shard carrying the hash value is determined as the last data shard.

[0018] Optionally, the method further comprises:

[0019] Associating the field attribute data corresponding to each data slice with the target data to obtain associated data;

[0020] Store the associated data to the target directory.

[0021] In addition, the present application also provides a data transmission device, including:

[0022] A receiving unit is configured to receive a data transmission request and obtain each uploaded data segment;

[0023] A new unit is configured to create a new temporary directory in response to the completion of uploading the last data segment;

[0024] A pre-merging unit is configured to pre-merge the saved uploaded data shards into a temporary directory, and then calculate a first hash value of the pre-merged data shards;

[0025] a verification unit configured to obtain a second hash value of the original data corresponding to each received data shard, perform a data integrity check based on the first hash value and the second hash value, and in response to passing the data integrity check, perform a real merging process on the received data shards to obtain target data;

[0026] The deletion unit is configured to set an automatic deletion time for the temporary directory, store the target data in the target directory, and delete the temporary directory when the automatic deletion time is reached.

[0027] Optionally, the device further includes a new directory generating unit configured to:

[0028] Get the unique string corresponding to each data shard, generate a new directory based on the unique string, and save each uploaded data shard to the new directory.

[0029] Optionally, the pre-merging unit is further configured to:

[0030] Pre-merge the saved uploaded data segments into a temporary directory in the order of upload time.

[0031] Optionally, the verification unit is further configured to:

[0032] The hash value carried by the last data shard is obtained as the second hash value of the original data.

[0033] Optionally, the newly created unit is further configured to:

[0034] In response to acquiring the data shard carrying the hash value, the data shard carrying the hash value is determined as the last data shard.

[0035] Optionally, the device further comprises an associating unit configured to:

[0036] Associating the field attribute data corresponding to each data slice with the target data to obtain associated data;

[0037] Store the associated data to the target directory.

[0038] In addition, the present application also provides a data transmission electronic device, including: one or more processors; a storage device for storing one or more programs, when the one or more programs are executed by one or more processors, the one or more processors implement the data transmission method as described above.

[0039] In addition, the present application also provides a computer-readable medium on which a computer program is stored, and when the program is executed by a processor, the data transmission method as described above is implemented.

[0040] To achieve the above objective, according to another aspect of the embodiments of the present application, a computer program product is provided.

[0041] A computer program product according to an embodiment of the present application includes a computer program, and when the program is executed by a processor, the data transmission method provided by the embodiment of the present application is implemented.

[0042] One embodiment of the above invention has the following advantages or beneficial effects: the present application obtains each uploaded data slice by receiving a data transmission request; creates a new temporary directory in response to the completion of uploading the last data slice; pre-merges the saved uploaded data slices into the temporary directory, and then calculates the first hash value of the pre-merged data slice; obtains the second hash value of the original data corresponding to each received data slice, performs a data integrity check based on the first hash value and the second hash value, and in response to passing the data integrity check, performs a real merging process of the received data slices to obtain the target data; sets an automatic deletion time for the temporary directory, stores the target data in the target directory, and deletes the temporary directory when the automatic deletion time is reached. The security and accuracy of data transmission are improved.

[0043] The further effects of the above-mentioned non-conventional optional manner will be described below in conjunction with the specific implementation manner. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] The accompanying drawings are used to better understand the present application and do not constitute an improper limitation on the present application.

[0045] Figure 1 is a schematic diagram of the main process of a data transmission method according to an embodiment of the present application;

[0046] Figure 2 is a schematic diagram of the main process of a data transmission method according to an embodiment of the present application;

[0047] Figure 3 It is a schematic diagram of the main flow of a data transmission method according to an embodiment of the present application;

[0048] Figure 4 is a schematic diagram of main units of a data transmission device according to an embodiment of the present application;

[0049] Figure 5 is an exemplary system architecture diagram to which the embodiments of the present application can be applied;

[0050] Figure 6 It is a structural diagram of a computer system of a terminal device or server suitable for implementing an embodiment of the present application. DETAILED DESCRIPTION

[0051] The following is a description of the exemplary embodiments of the present application in conjunction with the accompanying drawings, including various details of the embodiments of the present application to facilitate understanding, and they should be considered as merely exemplary. Therefore, those of ordinary skill in the art should recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of the present application. Similarly, for the sake of clarity and conciseness, the description of known functions and structures is omitted in the following description. It should be noted that the acquisition, transmission, storage, use, and processing of data in the technical solution of the present application are in compliance with the relevant provisions of national laws and regulations. It should be noted that in the embodiments of the present application, certain software, components, models, and other existing solutions in the industry may be mentioned, which should be considered as exemplary, and their purpose is only to illustrate the feasibility of the implementation of the technical solution of the present application, but it does not mean that the applicant has or must have used the solution. In the technical solution of the present application, the collection, analysis, use, transmission, storage, and other aspects of the user's personal information involved are in compliance with the provisions of relevant laws and regulations, and are used for legal and reasonable purposes, and are not shared, leaked, or sold outside of these legal uses, and are subject to supervision and management by regulatory authorities. Take necessary measures to prevent illegal access to such personal information, ensure that persons with access to personal information comply with relevant laws and regulations, and ensure the security of personal information. Once such personal information is no longer needed, risks should be minimized by limiting or even prohibiting data collection and / or deleting data.

[0052] When used, including in certain related applications, user privacy is protected by de-identifying data, such as by removing specific identifiers when used, controlling the amount or specificity of data stored, controlling how data is stored, and / or other methods of de-identification.

[0053] Figure 1 is a schematic diagram of the main process of a data transmission method according to an embodiment of the present application, such as Figure 1 As shown, the data transmission method mainly includes the following steps S101 to S105.

[0054] Step S101, receiving a data transmission request, and obtaining each uploaded data segment.

[0055] In this embodiment, the execution subject of the data transmission method (for example, a server) can receive a data transmission request through a wired connection or a wireless connection. The data to be transmitted can be, for example, a picture, which is divided into several slices, marked with slice numbers, and fixed in size range to obtain each data slice, and the execution subject obtains each uploaded data slice corresponding to the data transmission request.

[0056] Step S102, in response to the completion of uploading the last data segment, a new temporary directory is created.

[0057] Specifically, before creating a new temporary directory, the method further includes: in response to acquiring a data slice carrying a hash value, determining the data slice carrying the hash value as the last data slice.

[0058] When uploading the last data segment, the hash value of the large file corresponding to the data to be transmitted is uploaded together. When it is detected that the uploaded data segment carries the hash value, it is determined that the uploaded data segment is the last data segment. When the upload of the last data segment is completed, a new temporary directory is created for pre-merging the data segments. By pre-merging the data segments, errors can be avoided when formally merging the data segments, thereby ensuring the accuracy of uploading the data segments.

[0059] Step S103: pre-merge the saved uploaded data slices into a temporary directory, and then calculate the first hash value of the pre-merged data slices.

[0060] Specifically, pre-merging the saved uploaded data segments into the temporary directory includes: pre-merging the saved uploaded data segments into the temporary directory in the order of upload time.

[0061] The data is divided into pieces, such as the piece files, and saved strictly in the order of upload time. After merging, the content before and after is verified to ensure the final quality of the data shard merging and improve the reliability of the data shard merging.

[0062] Step S104, obtaining the second hash value of the original data corresponding to each received data slice, performing data integrity check based on the first hash value and the second hash value, and in response to passing the data integrity check, executing the actual merging process of the received data slices to obtain the target data.

[0063] The second hash value is the hash value of the large file corresponding to the data to be transmitted.

[0064] Specifically, obtaining the second hash value of the original data corresponding to each received data slice includes: obtaining the hash value carried by the last data slice as the second hash value of the original data.

[0065] The obtained second hash value is compared with the first hash value of the pre-merged data shard for consistency. If the comparison is consistent, it is determined that the data integrity check has passed, and the real merging process of the received data shards is performed to quickly and accurately obtain the target data. The target data is the data obtained by truly merging the various data shards. If the comparison is inconsistent, the upload of this data shard fails.

[0066] Step S105, setting an automatic deletion time for the temporary directory, storing the target data in the target directory, and deleting the temporary directory when the automatic deletion time is reached.

[0067] Specifically, the method further includes: associating the field attribute data corresponding to each data slice with the target data to obtain associated data; and storing the associated data in the target directory.

[0068] The field attribute data corresponding to each data slice may include attribute data such as file name, file type, channel, etc. The field attribute data corresponding to each data slice is associated with the merged target data and stored in the target directory of the database.

[0069] By setting an automatic deletion time for the temporary directory, you can reduce the number of database calls and the time spent on data transmission. There is no need to specifically call the database to delete the temporary directory. You only need to set the automatic deletion time and call the deletion program to delete the temporary directory when the automatic deletion time is reached.

[0070] This embodiment receives a data transmission request to obtain each uploaded data slice; in response to the completion of uploading the last data slice, a new temporary directory is created; the saved uploaded data slices are pre-merged into the temporary directory, and then the first hash value of the pre-merged data slice is calculated; the second hash value of the original data corresponding to each received data slice is obtained, and a data integrity check is performed based on the first hash value and the second hash value. In response to passing the data integrity check, a real merging process of the received data slices is performed to obtain the target data; an automatic deletion time is set for the temporary directory, the target data is stored in the target directory, and the temporary directory is deleted when the automatic deletion time is reached. The security and accuracy of data transmission are improved.

[0071] Figure 2 is a schematic diagram of the main flow of a data transmission method according to an embodiment of the present application, such as Figure 2 As shown, the data transmission method mainly includes the following steps S201 to S206.

[0072] Step S201: receiving a data transmission request and obtaining each uploaded data segment.

[0073] The data to be transmitted corresponding to the data transmission request may be text, audio or video, and the embodiment of the present application does not specifically limit the data to be transmitted. After receiving the data transmission request, the execution subject may obtain each data fragment carried by the request.

[0074] Step S202, obtaining a unique string corresponding to each data slice, generating a new directory according to the unique string, and saving each uploaded data slice to the new directory.

[0075] The data segments may also be uploaded with a unique string corresponding to the data segment, which may be an identifier of a large file corresponding to each data segment, and is used to characterize a unique large file. The size of the large file exceeds the common file size, such as hundreds of megabytes or several gigabytes.

[0076] By generating a new directory based on a unique string to temporarily save each uploaded data segment, confusion of the uploaded data segments can be avoided and the accuracy of data transmission can be improved.

[0077] Step S203, in response to the completion of uploading the last data segment, a new temporary directory is created.

[0078] If an upload terminator is obtained in the uploaded data segment, it is determined that the uploaded data segment is the last data segment, and the execution subject can create a new temporary directory to store the uploaded data segment to ensure the accuracy of data transmission.

[0079] Step S204: pre-merge the saved uploaded data slices into a temporary directory, and then calculate the first hash value of the pre-merged data slices.

[0080] The saved uploaded data slices are pre-merged in a preset order, and the pre-merged data slices are saved in a temporary directory, and the hash values ​​of the pre-merged data slices are calculated to obtain a first hash value. The first hash value is used to verify the integrity of the data to ensure the accuracy of data transmission.

[0081] Step S205, obtaining the second hash value of the original data corresponding to each received data slice, performing data integrity verification based on the first hash value and the second hash value, and in response to passing the data integrity verification, executing the actual merging process of the received data slices to obtain the target data.

[0082] When the first hash value is the same as the second hash value, it indicates that the uploaded data fragments are consistent with the large file corresponding to the data to be transmitted after being merged, and there is no error, and it is determined that the data integrity check has passed. After the data integrity check passes, the received data fragments are actually merged to obtain the target data for transmission.

[0083] Step S206, setting an automatic deletion time for the temporary directory, storing the target data in the target directory, and deleting the temporary directory when the automatic deletion time is reached.

[0084] The execution subject can determine the time interval for automatically deleting the temporary directory according to the scenario corresponding to the data transmission request, and determine the automatic deletion time of the temporary directory based on the current time and the time interval. This ensures that the temporary directory does not occupy too much space without affecting data transmission, and does not require frequent access to the database to avoid wasting computing resources. After the target data is stored in the target directory, the temporary directory is deleted when the automatic deletion time is reached to free up space and ensure high data transmission performance.

[0085] Figure 3It is a schematic diagram of the main flow of a data transmission method according to an embodiment of the present application. In the embodiment of the present application, MD5 is a commonly used hash algorithm for generating a unique digest value. When comparing the contents of files, the MD5 algorithm is used to generate the digest values ​​of two files, and then the two digest values ​​are compared to see if they are the same. If the contents of the two files are exactly the same, then their digest values ​​should also be exactly the same. If the digest values ​​are different, it means that the contents of the files are different. The MD5 algorithm is based on the message digest algorithm, and finally generates a 128-bit digest value by performing a series of calculations and operations on the binary data of the file. Since the length of the digest value of the MD5 algorithm is relatively fixed and short, it can be quickly generated and compared. When comparing the contents of files, the MD5 algorithm can be used to generate the digest value of each file first, and then compare whether the two digest values ​​are the same. If they are the same, it can be confirmed that the contents of the files are the same; if they are different, it can be determined that the contents of the files are different. It is more suitable for scenarios where the integrity requirements are not too high. A large file refers to a file in a computer system whose file size exceeds a certain threshold. The specific definition of the threshold can be determined according to different environments and application scenarios, and is based on tens of megabytes or hundreds of megabytes. File size exceeds a certain threshold: The size of a large file exceeds the common file size, such as hundreds of megabytes or several gigabytes. Long read and write time: Due to its large size, it may take a long time to read and write large files. High storage requirements: Large files take up more storage space and therefore require sufficient disk space to store them. Slow transfer speed: Due to the large file size, the transfer process takes longer and requires more bandwidth. File sharding transfer: The steps of file sharding usually include sharding, transfer, and merging. The following is a brief description of these steps: Sharding: First, select an appropriate shard size. Then, use a file sharder to split the original file into multiple smaller shards or blocks. Each shard is given a unique identifier, and this identifier is recorded in the shard index along with the order and location information of the shard. Transfer: Transfer the shard to the target location. This can be achieved through network transfer or other transfer mechanisms. If the file is large, sharding transfer can improve transfer efficiency and support breakpoint resuming so that the transfer can be resumed in the event of a transfer interruption. Merge: Once all shards have been transferred to the target location, the receiver can merge the shards using the shard index. According to the order and position information recorded in the shard index, the shards are reassembled into a complete file in the correct order. The merged file is completely consistent with the original file.

[0086] like Figure 3As shown, the user transmits the slice file (i.e., data slice) at the front end, and transmits the slice index, the number of slices, the unique string uuid, and the hash value MD5 of the large file to the back end program. The back end program creates a temporary directory based on the uuid and saves the received slice files (i.e., data slices). When the last slice file (i.e., data slice) is uploaded, it is determined whether the number of slices in the temporary directory matches the number of slice files (i.e., data slices) transmitted by the user at the front end. If not, the process of transmitting the slice file from the front end to the back end is retried until the number of slices in the temporary directory matches the number of slice files (i.e., data slices) transmitted by the user at the front end. Then, the slice files (i.e., data slices) in the temporary directory are pre-merged into the temporary directory to verify the integrity of the file. If the file integrity verification is successful, the temporary directory is deleted, and the slice files (i.e., data slices) are formally merged into the target directory. The front end transmits the file name, type, channel and other attribute data and stores them in the database. The front end displays the file name as the input name, and the data transmission process ends. If the file integrity verification fails, the file (i.e., data slice) upload fails, and the data transmission process ends.

[0087] For example, upload fragments: the front end first divides the image into several fragments, marks the fragment serial number, and fixes the size range of the fragments. When uploading each fragment, send the index of the fragment, the total number of fragments and an agreed unique string to the back end. When sending the last fragment, pass the MD5 of the large file to the back end. Save fragments: The back end generates a new directory based on the unique string and saves the fragment files in the directory. After receiving all the fragments, pre-merge them into a temporary directory. Compare the MD5 of the merged file with the MD5 of the last upload by the front end. If they are the same, it means that all fragments have been received completely and uploaded successfully. At this time, delete the pre-merged files and temporary directories, merge them in the unique directory, and save the files. File attribute assignment: After confirming that the acceptance is correct, the front end transmits the field attributes facing the customer, such as the actual saved file name, file type, file channel, etc. First, upload the data fragments one by one and save them in a specific directory one by one. When the last data slice is uploaded, a new temporary directory is created, and the saved data slices are first merged into the temporary directory. The MD5 of the merged file in the temporary directory is compared with the MD5 of the last uploaded by the front end. If they are the same, it means that the file has been completely uploaded. The temporary directory is deleted, and the files (i.e., each uploaded data slice) are actually merged in the specified directory. Finally, the front end sends the file name, channel, type and other information, and finally successfully enters the file attributes into the database. The file name and other attributes can be stored in the database or cache first, and then directly assigned to the file after the slice is successfully uploaded. Reliability of merging: The slice files are saved strictly in the order of upload, and the contents before and after the merge are verified to ensure the final quality of the merge. Flexible saving: After the verification is passed, the real file name and other file attributes are obtained from the front end and stored in the database. It provides flexibility for subsequent operations. After the merge, the corresponding field values ​​in the database can be modified according to the front end. The embodiment of the present application can be named according to the final input name of the front end, rather than the original file name. The file integrity is verified together with the front end to ensure that the file is finally merged and successfully uploaded. Ensure that the slice order is correct and the reassembled file content remains unchanged.

[0088] Figure 4 Schematic diagram of the main units of the data transmission device according to an embodiment of the present application. Figure 4 As shown, the data transmission device 400 includes a receiving unit 401, a creating unit 402, a pre-merging unit 403, a checking unit 404 and a deleting unit 405.

[0089] The receiving unit 401 is configured to receive a data transmission request and obtain each uploaded data segment.

[0090] The new creation unit 402 is configured to create a new temporary directory in response to the completion of uploading the last data segment.

[0091] The pre-merging unit 403 is configured to pre-merge the saved uploaded data segments into a temporary directory, and then calculate a first hash value of the pre-merged data segments.

[0092] The verification unit 404 is configured to obtain a second hash value of the original data corresponding to each received data shard, perform a data integrity check based on the first hash value and the second hash value, and in response to passing the data integrity check, perform a real merging process on the received data shards to obtain the target data.

[0093] The deleting unit 405 is configured to set an automatic deletion time for the temporary directory, store the target data in the target directory, and delete the temporary directory when the automatic deletion time is reached.

[0094] In some embodiments, the data transmission device further includes Figure 4 The new directory generation unit not shown in the figure is configured to: obtain a unique string corresponding to each data slice, generate a new directory according to the unique string, and save each uploaded data slice to the new directory.

[0095] In some embodiments, the pre-merging unit 403 is further configured to: pre-merge the saved uploaded data segments into a temporary directory in the order of upload time.

[0096] In some embodiments, the verification unit 404 is further configured to: obtain a hash value carried by the last data fragment as a second hash value of the original data.

[0097] In some embodiments, the creating unit 402 is further configured to: in response to acquiring the data shard carrying the hash value, determine the data shard carrying the hash value as the last data shard.

[0098] In some embodiments, the data transmission device further includes Figure 4 The association unit not shown in the figure is configured to: associate the field attribute data corresponding to each data slice with the target data to obtain the associated data; and store the associated data in the target directory.

[0099] It should be noted that the data transmission method and the data transmission device of the present application have a corresponding relationship in terms of specific implementation content, so the repeated content will not be described again.

[0100] Figure 5 An exemplary system architecture 500 to which the data transmission method or data transmission device according to the embodiment of the present application can be applied is shown.

[0101] like Figure 5As shown, system architecture 500 may include terminal devices 501, 502, 503, a network 504 and a server 505. Network 504 is used to provide a medium for communication links between terminal devices 501, 502, 503 and server 505. Network 504 may include various connection types, such as wired, wireless communication links or optical fiber cables, etc.

[0102] Users can use terminal devices 501, 502, 503 to interact with server 505 through network 504 to receive or send messages, etc. Various communication client applications can be installed on terminal devices 501, 502, 503, such as shopping applications, web browser applications, search applications, instant messaging tools, email clients, social platform software, etc. (only for example).

[0103] The terminal devices 501 , 502 , and 503 may be various electronic devices having a data transmission processing screen and supporting web browsing, including but not limited to smart phones, tablet computers, laptop computers, desktop computers, and the like.

[0104] Server 505 can be a server that provides various services, such as a background management server that supports data transmission requests submitted by users using terminal devices 501, 502, and 503 (only as an example). The background management server can receive data transmission requests and obtain each uploaded data slice; in response to the completion of uploading the last data slice, create a new temporary directory; pre-merge the saved uploaded data slices into the temporary directory, and then calculate the first hash value of the pre-merged data slice; obtain the second hash value of the original data corresponding to each received data slice, perform data integrity verification based on the first hash value and the second hash value, and in response to the data integrity verification passing, perform the actual merging process of the received data slices to obtain the target data; set an automatic deletion time for the temporary directory, store the target data in the target directory, and delete the temporary directory when the automatic deletion time is reached. Improve the security and accuracy of data transmission.

[0105] It should be noted that the data transmission method provided in the embodiment of the present application is generally executed by the server 505 , and accordingly, the data transmission device is generally arranged in the server 505 .

[0106] It should be understood that Figure 5 The number of terminal devices, networks and servers in the embodiment is only for illustration. Any number of terminal devices, networks and servers may be provided according to implementation requirements.

[0107] Reference below Figure 6 , which shows a schematic diagram of the structure of a computer system 600 of a terminal device suitable for implementing an embodiment of the present application. Figure 6The terminal device shown is merely an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.

[0108] like Figure 6 As shown, the computer system 600 includes a central processing unit (CPU) 601, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 602 or a program loaded from a storage part 608 into a random access memory (RAM) 603. In the RAM 603, various programs and data required for the operation of the computer system 600 are also stored. The CPU 601, the ROM 602, and the RAM 603 are connected to each other via a bus 604. An input / output (I / O) interface 605 is also connected to the bus 604.

[0109] The following components are connected to the I / O interface 605: an input section 606 including a keyboard, a mouse, etc.; an output section 607 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage section 608 including a hard disk, etc.; and a communication section 609 including a network interface card such as a LAN card, a modem, etc. The communication section 609 performs communication processing via a network such as the Internet. A drive 610 is also connected to the I / O interface 605 as needed. A removable medium 611, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 610 as needed, so that a computer program read therefrom is installed into the storage section 608 as needed.

[0110] In particular, according to the embodiments disclosed in the present application, the process described above with reference to the flowchart can be implemented as a computer software program. For example, the embodiments disclosed in the present application include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program contains program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from the network through the communication part 609, and / or installed from the removable medium 611. When the computer program is executed by the central processing unit (CPU) 601, the above-mentioned functions defined in the system of the present application are executed.

[0111] It should be noted that the computer-readable medium shown in the present application may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. Computer-readable storage media may include, for example, but are not limited to electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices or devices, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present application, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, device or device. In the present application, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, which carries a computer-readable program code. This propagated data signal may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. Computer-readable signal media may also be any computer-readable medium other than computer-readable storage media, which may send, propagate or transmit a program for use by or in conjunction with an instruction execution system, apparatus or device. The program code contained on the computer-readable medium may be transmitted using any appropriate medium, including but not limited to: wireless, wire, optical cable, RF, etc., or any suitable combination of the above.

[0112] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present application. In this regard, each box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the above-mentioned module, program segment or a part of a code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flow chart, and the combination of the boxes in the block diagram or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0113] The units involved in the embodiments described in the present application may be implemented by software or hardware. The units described may also be set in a processor, for example, it may be described as: a processor includes a receiving unit, a new unit, a pre-merging unit, a checking unit, and a deleting unit. The names of these units do not, in some cases, constitute limitations on the units themselves.

[0114] As another aspect, the present application also provides a computer-readable medium, which may be included in the device described in the above embodiment; or it may exist independently and not be assembled into the device. The above computer-readable medium carries one or more programs. When the above one or more programs are executed by a device, the device receives a data transmission request and obtains each uploaded data slice; in response to the completion of uploading the last data slice, a new temporary directory is created; the saved uploaded data slices are pre-merged into the temporary directory, and then the first hash value of the pre-merged data slice is calculated; the second hash value of the original data corresponding to each received data slice is obtained, and a data integrity check is performed based on the first hash value and the second hash value. In response to the passing of the data integrity check, a real merging process of the received data slice is executed to obtain the target data; an automatic deletion time is set for the temporary directory, and the target data is stored in the target directory. When the automatic deletion time is reached, the temporary directory is deleted.

[0115] The computer program product of the present application includes a computer program, and when the computer program is executed by a processor, the data transmission method in the embodiment of the present application is implemented.

[0116] According to the technical solution of the embodiment of the present application, the security and accuracy of data transmission can be improved.

[0117] The above specific implementations do not constitute a limitation on the protection scope of this application. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions may occur depending on design requirements and other factors. Any modifications, equivalent substitutions and improvements made within the spirit and principles of this application should be included in the protection scope of this application.

Claims

1. A data transmission method, characterized in that: include: Receive data transmission requests and obtain each uploaded data segment; In response to the last data shard being uploaded, a new temporary directory is created; Pre-merge the saved uploaded data shards into the temporary directory, and then calculate the first hash value of the pre-merged data shards; Obtaining a second hash value of the original data corresponding to each of the received data slices, performing a data integrity check based on the first hash value and the second hash value, and in response to passing the data integrity check, performing a real merging process on the received data slices to obtain target data; An automatic deletion time is set for the temporary directory, the target data is stored in the target directory, and when the automatic deletion time is reached, the temporary directory is deleted.

2. The method according to claim 1, characterized in that Before creating the new temporary directory, the method further includes: A unique string corresponding to each data slice is obtained, a new directory is generated according to the unique string, and each uploaded data slice is saved to the new directory.

3. The method according to claim 1, characterized in that The step of pre-merging the saved uploaded data segments into the temporary directory includes: The saved uploaded data segments are pre-merged into the temporary directory in the order of upload time.

4. The method according to claim 1, characterized in that: The obtaining of the second hash value of the original data corresponding to each of the received data fragments includes: The hash value carried by the last data shard is obtained as the second hash value of the original data.

5. The method according to claim 1, characterized in that Before creating the new temporary directory, the method further includes: In response to acquiring the data fragment carrying the hash value, the data fragment carrying the hash value is determined as the last data fragment.

6. The method according to claim 1, characterized in that The method further comprises: Associating the field attribute data corresponding to each data slice with the target data to obtain associated data; The associated data is stored in the target directory.

7. A data transmission device, characterized in that: include: A receiving unit is configured to receive a data transmission request and obtain each uploaded data segment; A new unit is configured to create a new temporary directory in response to the completion of uploading the last data segment; a pre-merging unit configured to pre-merge the saved uploaded data shards into the temporary directory, and then calculate a first hash value of the pre-merged data shards; a verification unit configured to obtain a second hash value of the original data corresponding to each of the received data slices, perform a data integrity check based on the first hash value and the second hash value, and in response to passing the data integrity check, perform a real merging process on the received data slices to obtain target data; The deleting unit is configured to set an automatic deletion time for the temporary directory, store the target data in the target directory, and delete the temporary directory when the automatic deletion time is reached.

8. The device according to claim 7, characterized in that The device also includes a new directory generating unit configured to: A unique string corresponding to each data slice is obtained, a new directory is generated according to the unique string, and each uploaded data slice is saved to the new directory.

9. The device according to claim 7, characterized in that The pre-merging unit is further configured to: The saved uploaded data segments are pre-merged into the temporary directory in the order of upload time.

10. The device according to claim 7, characterized in that The verification unit is further configured to: The hash value carried by the last data shard is obtained as the second hash value of the original data.

11. The device according to claim 7, characterized in that The newly constructed unit is further configured to: In response to acquiring the data fragment carrying the hash value, the data fragment carrying the hash value is determined as the last data fragment.

12. The device according to claim 7, characterized in that The device further comprises an associating unit configured to: Associating the field attribute data corresponding to each data slice with the target data to obtain associated data; The associated data is stored in the target directory.

13. An electronic device for data transmission, characterized in that: include: one or more processors; a storage device for storing one or more programs, When the one or more programs are executed by the one or more processors, the one or more processors implement the method according to any one of claims 1 to 6.

14. A computer readable medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the method according to any one of claims 1 to 6 is implemented.

15. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the method according to any one of claims 1 to 6 is implemented.

Images