Data uploading method and device, computer equipment, readable storage medium and program product

Abstract

The invention relates to a data uploading method and device, computer equipment, a readable storage medium and a program product. The method comprises the steps of determining a first initial fragment corresponding to a random write instruction and a first target fragment located in a local cache space in the first initial fragment; determining a second initial fragment which is uploaded before the receiving moment of the random write instruction and a first serial number of the second initial fragment; in the second initial fragments, the serial numbers of the fragments and the serial numbers before the serial numbers of the fragments are determined as second serial numbers for the fragments with the serial numbers at the most rear in the preset arrangement sequence; determining fragments corresponding to the remaining serial numbers except the first serial number in the second serial numbers as second target fragments; and uploading the first target fragment and the second target fragment to a remote server, and executing a random write instruction based on an uploading result. According to the method provided by the invention, the waiting time of the user can be effectively shortened.

Description

Technical Field

[0001] This application relates to the technical field of object storage, and particularly to a data upload method, device, computer device, readable storage medium, and program product. Background Art

[0002] With the development of cloud computing technology, cloud storage has become an important means for enterprises and individuals to store a large amount of data. Among them, object storage (ZOS, Zettabyte Object Storage) is a cost-effective cloud storage solution that provides data storage services over the Internet, stores data in the form of objects on remote servers, each object has a unique identifier, and can be accessed and operated independently.

[0003] Existing object storage solutions perform well in processing sequential write operations. However, when processing random write operations, if the shards required for the random write operation have been uploaded from the local cache to the remote server, it is necessary to upload all the shards of the file in the local cache to the remote server before performing the random write operation, which results in a long waiting time for the user and a poor user experience. Summary of the Invention

[0004] Based on this, it is necessary to provide a data upload method, device, computer device, readable storage medium, and program product that can reduce the waiting time of users for the above technical problems.

[0005] In a first aspect, this application provides a data upload method, and the method includes:

[0006] During the process of uploading the shards of the target file from the local cache space to the remote server, if a random write instruction sent by the user terminal is received, determine at least one first initial shard corresponding to the random write instruction, and a first target shard among the at least one first initial shards that is located in the local cache space; wherein, the target file is divided into multiple shards with different serial numbers, and the different serial numbers follow a preset arrangement order;

[0007] [[ID=2�]]For the receiving moment when the random write instruction is received, determine the second initial shards that have been uploaded to the remote server before the receiving moment, and the first serial number of the second initial shards;

[0008] For the shard among the second initial shards with the last serial number in the preset arrangement order, determine the serial number of the shard and the serial numbers before the serial number of the shard in the preset arrangement order as the second serial number;

[0009] In the case where the first serial number is different from the second serial number, determine the remaining serial numbers in the second serial number except the first serial number as the third serial number, and determine the shards corresponding to the third serial number as the second target shards;

[0010] Upload the first target shard and the second target shard to the remote server, and execute a random write instruction based on the upload result.

[0011] In one embodiment, executing a random write instruction based on the upload result includes:

[0012] Determine the shards in at least one first initial shard that are located in the remote server as the third target shards;

[0013] Download the first target shard and the third target shard in the remote server to the local cache space;

[0014] Execute a random write instruction on the downloaded shards, and upload the shards after executing the random write instruction to the remote server.

[0015] In one embodiment, the method further includes:

[0016] When all the first initial shards are located in the local cache space, execute a random write instruction on the first initial shards.

[0017] In one embodiment, the method further includes:

[0018] For any shard uploaded from the local cache space to the remote server, after the shard is uploaded to the remote server, delete the shard in the local cache space.

[0019] In one embodiment, the method further includes:

[0020] Sort multiple shards of the target file according to a preset arrangement order of serial numbers;

[0021] Divide multiple shards into different shard intervals based on the sorting result; wherein, the random write instruction corresponds to at least one shard interval.

[0022] In one embodiment, the method further includes:

[0023] When no random write instruction is received, for the shards in the target file, upload the shards that only include valid data to the remote server;

[0024] When the upload of the shards that only include valid data is completed, upload the shards that include both valid data and all-zero data to the remote server;

[0025] When the upload of the shards that include both valid data and all-zero data is completed, upload the shards that only include all-zero data to the remote server.

[0026] In a second aspect, the present application further provides a data upload device, and the device includes:

[0027] A first determination module, configured to determine at least one first initial shard corresponding to the random write instruction and a first target shard located in the local cache space among the at least one first initial shards when receiving a random write instruction during the process of uploading shards of a target file from the local cache space to a remote server; wherein the target file is divided into multiple shards with different serial numbers, and the different serial numbers follow a preset arrangement order;

[0028] A second determination module, configured to determine, for the receiving moment when the random write instruction is received, a second initial shard that has been uploaded to the remote server before the receiving moment and a first serial number of the second initial shard;

[0029] A third determination module, configured to, for the shard with the last serial number in the preset arrangement order among the second initial shards, determine the serial number of the shard and the serial numbers before the serial number of the shard in the preset arrangement order as the second serial numbers;

[0030] A fourth determination module, configured to, when the first serial number is inconsistent with the second serial number, determine the remaining serial numbers except the first serial number in the second serial numbers as the third serial numbers, and determine the shards corresponding to the third serial numbers as the second target shards;

[0031] An upload module, configured to upload the first target shard and the second target shard to the remote server, and execute the random write instruction based on the upload result.

[0032] In a third aspect, the present application further provides a computer device. The computer device includes a memory and a processor. The memory stores a computer program, and when the processor executes the computer program, the steps of the method in any one of the above embodiments are implemented.

[0033] In a fourth aspect, the present application further provides a computer-readable storage medium. The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the method in any one of the above embodiments are implemented.

[0034] In a fifth aspect, the present application further provides a computer program product. The computer program product includes a computer program, and when the computer program is executed by a processor, the steps of the method in any one of the above embodiments are implemented.

[0035] In the above data uploading method, device, computer device, readable storage medium and program product, during the process of uploading the shards of a target file from the local cache space to a remote server, if a random write instruction sent by a user terminal is received, at least one first initial shard corresponding to the random write instruction and a first target shard among the at least one first initial shards that is located in the local cache space are determined; wherein, the target file is divided into multiple shards with different serial numbers, and the different serial numbers follow a preset arrangement order; for the receiving moment when the random write instruction is received, a second initial shard that has been uploaded to the remote server before the receiving moment and a first serial number of the second initial shard are determined; for the shard among the second initial shards whose serial number is the last in the preset arrangement order, the serial number of the shard and the serial numbers before the serial number of the shard in the preset arrangement order are determined as second serial numbers; in the case where the first serial number is inconsistent with the second serial numbers, the remaining serial numbers in the second serial numbers except the first serial number are determined as third serial numbers, and the shards corresponding to the third serial numbers are determined as second target shards; the first target shard and the second target shard are uploaded to the remote server, and the random write instruction is executed based on the upload result. In the method provided by this application, before executing the random write instruction, only the first target shard and the second target shard need to be uploaded to the remote server, which can effectively reduce the waiting time of users and improve the user experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] In order to more clearly illustrate the technical solutions in the embodiments of the present application or related technologies, the following will briefly introduce the drawings required for describing the embodiments of the present application or related technologies. Obviously, the drawings in the following description are only some embodiments of the present application. For those of ordinary skill in the art, other related drawings can be obtained based on these drawings without creative efforts.

[0037] Figure 1 It is a flowchart of the data uploading method in one embodiment;

[0038] Figure 2 It is a flowchart of the random write instruction execution method in one embodiment;

[0039] Figure 3 It is a schematic diagram of a random write operation in the prior art;

[0040] Figure 4 It is a schematic diagram of a random write operation in another embodiment;

[0041] Figure 5 It is a structural block diagram of the data uploading device in one embodiment;

[0042] Figure 6 It is an internal structure diagram of a computer device in one embodiment. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0043] In order to make the objectives, technical solutions and advantages of the present application more clear and understandable, the present application will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0044] In the prior art, when a random write operation occurs, first check whether the shard interval corresponding to the current random write is still local, that is, whether it hits. If the interval hits, directly execute the random write operation. If the corresponding interval has been uploaded to the remote server, that is, the interval does not hit, the set of shards that have not been uploaded in the local cache space of the file is denoted as , and at this time, it is necessary to first pause the ongoing write operation, and upload all the shards in to the remote server, and then download the shards corresponding to the random write operation. After the random write operation is completed, resume the paused write operation and continue to upload data according to the original logic.

[0045] In the prior art, whenever a user performs a random write operation, if the interval does not hit, it is necessary to wait for all the un-uploaded shards to be uploaded before re-downloading the corresponding shards and completing the random write operation, which results in a longer waiting time for the user and a poor experience. Therefore, in this embodiment, by reducing the uploaded shards, the random write operation can be completed faster, the user waiting time can be reduced, and the user experience can be improved.

[0046] In one embodiment, as Figure 1 shown, a data upload method is provided. In this embodiment, an example is given where this method is applied to a terminal. It can be understood that this method can also be applied to a server, and can also be applied to a system including a terminal and a server, and is implemented through the interaction between the terminal and the server. In this embodiment, the method includes the following steps:

[0047] S102. During the process of uploading the shards of the target file from the local cache space to the remote server, if a random write instruction sent by the user terminal is received, determine at least one first initial shard corresponding to the random write instruction, and a first target shard located in the local cache space among the at least one first initial shards; wherein, the target file is divided into multiple shards with different serial numbers, and the different serial numbers follow a preset arrangement order.

[0048] Among them, the random write instruction is used to randomly modify the corresponding shards in the local cache space; uploading the shards of the target file from the local cache space to the remote server is the process of object storage; the serial number of the shard represents the position of the shard in the target file, and by integrating the shards according to the preset arrangement order of the serial numbers, the complete target file can be obtained.

[0049] S104. Determine the second initial shard uploaded to the remote server before the receiving moment and the first sequence number of the second initial shard for the receiving moment when the random write instruction is received.

[0050] Optionally, during the object storage process, the shards are not uploaded strictly in the preset arranged order of sequence numbers. Therefore, the first sequence numbers of different second initial shards may be discontinuous.

[0051] S106. For the shard with the last sequence number in the preset arranged order among the second initial shards, determine the sequence number of the shard and the sequence numbers before the sequence number of the shard in the preset arranged order as the second sequence numbers.

[0052] Optionally, for the shard with the last sequence number among the second initial shards, if there are shards with sequence numbers before the sequence number of the last shard that have not been uploaded to the remote server, there will be shard missing and the complete target file cannot be restored. Therefore, it is necessary to first determine all the second sequence numbers before the sequence number of the last shard in the preset arranged order, and based on the second sequence numbers, determine whether there are missing shards among the uploaded shards.

[0053] S108. When the first sequence number is inconsistent with the second sequence numbers, determine the remaining sequence numbers in the second sequence numbers except the first sequence number as the third sequence numbers, and determine the shards corresponding to the third sequence numbers as the second target shards.

[0054] Optionally, by comparing the first sequence number with the second sequence numbers, if there are missing shards, determine the sequence numbers of the missing shards as the third sequence numbers, and denote the set of the second target shards corresponding to the third sequence numbers as 。

[0055] S110. Upload the first target shards and the second target shards to the remote server, and execute the random write instruction based on the upload result.

[0056] Optionally, the first target shards are the shards in the first initial shards that have not been uploaded to the remote server. Since the shards will be deleted from the local cache space after being uploaded, only the first target shards are left in the local cache space and all the shards corresponding to the random write instruction are not included. At this time, the random write instruction cannot be executed in the local cache space. Therefore, it is necessary to download the remaining shards corresponding to the random write instruction except the first target shards from the remote server. However, since these remaining shards have gone through the upload process, there may be a problem of data inconsistency between the downloaded remaining shards and the first target shards. Therefore, it is necessary to upload the first target shards first, and then download the uploaded first target shards and the remaining shards together.

[0057] Optionally, to avoid missing shards in the remote server, which may lead to an incomplete restored target file and cause file corruption, it is necessary to upload the second target shard to the remote server to ensure the consistency of the data in the remote server and the data in the target file.

[0058] Optionally, the set of the first target shards can be denoted as , and and The sum of the sets is determined as , that is and , so, what is finally uploaded is .

[0059] In the above data upload method, during the process of uploading the shards of the target file from the local cache space to the remote server, if a random write instruction sent by the user terminal is received, at least one first initial shard corresponding to the random write instruction and the first target shards among at least one first initial shards located in the local cache space are determined; among them, the target file is divided into multiple shards with different serial numbers, and different serial numbers follow a preset arrangement order; for the receiving moment when the random write instruction is received, the second initial shards uploaded to the remote server before the receiving moment and the first serial number of the second initial shards are determined; for the shard with the last serial number in the preset arrangement order among the second initial shards, the serial number of the shard and the serial numbers before the serial number of the shard in the preset arrangement order are determined as the second serial number; in the case where the first serial number is inconsistent with the second serial number, the remaining serial numbers in the second serial number except the first serial number are determined as the third serial number, and the shards corresponding to the third serial number are determined as the second target shards; the first target shards and the second target shards are uploaded to the remote server, and the random write instruction is executed based on the upload result. In the method provided by this application, before executing the random write instruction, only the first target shards and the second target shards need to be uploaded to the remote server, which can effectively reduce the waiting time of the user and improve the user experience.

[0060] In some embodiments, as Figure 2 shown, executing the random write instruction based on the upload result includes:

[0061] S202. Determine the shards among at least one first initial shards located in the remote server as the third target shards.

[0062] S204. Download the first target shards and the third target shards in the remote server to the local cache space.

[0063] S206. Execute the random write instruction on the downloaded shards, and upload the shards after executing the random write instruction to the remote server.

[0064] Among them, the third target shard is the remaining shards corresponding to the random write instructions other than the first target shard.

[0065] In this embodiment, the first target shard and the third target shard in the remote server are downloaded to the local cache space, and the random write instructions are executed on the downloaded shards, so as to ensure the consistency of the shards for executing the random write instructions.

[0066] In some embodiments, the method further includes: when all the first initial shards are located in the local cache space, executing the random write instructions on the first initial shards.

[0067] Optionally, if all the first initial shards are located in the local cache space, the local cache space includes all the shards corresponding to the random write instructions, and there is no problem of data inconsistency between these shards, and the random write instructions can be directly executed on these shards.

[0068] In this embodiment, when all the first initial shards are located in the local cache space, executing the random write instructions on the first initial shards can effectively improve the execution efficiency of the random write instructions and reduce the waiting time of the user.

[0069] In some embodiments, the method further includes: for any shard uploaded from the local cache space to the remote server, after the shard is uploaded to the remote server, deleting the shard in the local cache space.

[0070] In this embodiment, after the shard is uploaded to the remote server, deleting the shard in the local cache space can release the local space and avoid excessive local cache pressure.

[0071] In some embodiments, the method further includes: sorting multiple shards of the target file according to a preset sorting order of serial numbers; dividing the multiple shards into different shard intervals based on the sorting result; wherein, the random write instructions correspond to at least one shard interval.

[0072] Optionally, in order to support the reading and writing of large files while reducing the impact on the reading and writing performance of small files, considering the maximum number of shards supported for uploading by the object storage, for example, the maximum number of shards is 10,000, this embodiment adopts a file sharding strategy optimization technology to reasonably shard the file, so that it can support the reading, writing and storage of large files and also take into account the performance requirements for modifying files of different sizes.

[0073] Specifically, the maximum number of shards is divided into n segments according to the preset sorting order of serial numbers, and the serial number of the last shard in each segment is denoted as , then the maximum number of shards are divided into n shard intervals:

[0074]

[0075] Where, is the maximum number of shards, is the number of shards corresponding to each shard interval, .

[0076] Optionally, the size of the shard data within each shard interval is recorded as , then:

[0077]

[0078] in, , according to the requirements of the S3 protocol, .

[0079] Based on this sharding method, the maximum file size that object storage can support is:

[0080]

[0081] In this embodiment, since the shards in the later interval will not be used in small files, the data stored in the shards in the earlier interval has a higher modification frequency in the overall usage scenario. In view of this feature, the file reading and writing performance is optimized by adjusting the size of the shards in the front and back intervals. In addition, a larger later interval can support reading and writing larger files.

[0082] In some embodiments, the method further includes: when no random write instruction is received, for the slices in the target file, uploading the slices that only include valid data to the remote server; when the uploading of the slices that only include valid data is completed, uploading the slices that include both valid data and all-zero data to the remote server; when the uploading of the slices that include both valid data and all-zero data is completed, uploading the slices that only include all-zero data to the remote server.

[0083] Optionally, during object storage, the target file's shards can be uploaded sequentially according to different priorities. Specifically, the target file's shards can be divided into three upload priorities: 1) shards completely filled with valid data and no portion affected by read-modify-write (RMW); 2) shards partially filled with valid data and no portion affected by RMW; 3) shards containing only zero data. Each shard is uploaded sequentially in the order of priorities 1), 2), and 3).

[0084] In this embodiment, different shards are uploaded in sequence according to whether the shards contain valid data, which can more reasonably execute the object storage process.

[0085] In some embodiments, another data upload method is provided, which includes the following:

[0086] Step 1: Reasonably set file sharding. According to the size and modification frequency of the file, it is set that the file is divided into 3 segments in total, and the shard sizes are respectively:

[0087]

[0088] Step 2: Support and optimization of random write operations. Taking the random write operations shown in Figure 3 and Figure 4 as an example, the operation process is described in detail. Among them, Figure 3 is a schematic diagram of the random write operation in the prior art, Figure 4 is a schematic diagram of the random write operation in this embodiment. As shown in Figure 3 , in the first round of operation, the server stores the historical version of the file: shards 1 - 14 are all A, this time shards 1 - 7 and 12 - 14 are modified to B, and B is additionally written in shards 15 - 20. At this time, the local cache is that shards 1 - 7 and 12 - 20 are all B. Since the object storage supports out-of-order upload, 4 shard data of 2 - 4 and 17 are uploaded in this round. In the second round of operation, the data stored on the server changes correspondingly according to the upload result of the previous round. At this time, the user modifies the data of shards 1 - 2 to C. Since there is no data of shard 2 in the local area and the interval is not hit, the subsequent writing is suspended, and the data of shards 1, 5 - 7, 12 - 16, and 18 - 20 that have not been uploaded in is uploaded. In the third round of operation, the data stored on the server is consistent with the state after all the writes and uploads in the first round, and the user's random write operation takes effect, and C is written and uploaded in shards 1 - 2. In the fourth round of operation, the random write operation is completed, and the file writable state is restored, and the process ends.

[0089] The optimized random write operation process is as shown in Figure 4 . Similarly, in the first round of operation, the server stores the historical version of the file: shards 1 - 14 are all A, this time shards 1 - 7 and 12 - 14 are modified to B, and B is additionally written in shards 15 - 20. At this time, the local cache is that shards 1 - 7 and 12 - 20 are all B. In this round, 4 shard data of 2 - 4 and 17 are uploaded. In the second round of operation, the user modifies the data of shards 1 - 2 to C. Since there is no data of shard 2 in the local area and the interval is not hit, the subsequent writing is suspended. According to the method in this technical solution, it is calculated that contains shard 1 in Slices 15 - 16 in the set are selected and the corresponding data is uploaded. In the third operation, the data stored on the server is temporarily in a state where new and old data are interleaved, but it already meets the requirements of random writes. The user's random write operation takes effect, and C is written and uploaded to slices 1 - 2. In the fourth operation, the random write operation is completed, and the data can be written and uploaded again. Slices 5 - 7, 12 - 14, and 17 - 20 in the set are continuously uploaded. In the fifth operation, the file upload operation is completed and the process ends.

[0090] Compare Figure 3 and Figure 4 It can be seen that compared with the original method which requires uploading 12 slices, the optimized method only needs to upload 3 slices to perform random write operations, effectively reducing the user's waiting time. If the slices that are postponed for upload are hit by subsequent random write operations, the repeated upload of the same slice can be avoided, which affects the data upload efficiency. Further, if the postponed slices are at the rear of a file with a larger slice size, the optimization effect is more obvious.

[0091] It should be understood that although each step in the flowcharts involved in the above-described embodiments is shown in sequence according to the arrow indication, these steps are not necessarily executed in the order indicated by the arrows. Unless there is a clear description in this article, the execution of these steps has no strict order limit, and these steps can be executed in other orders. Moreover, at least some of the steps in the flowcharts involved in the above-described embodiments may include multiple steps or multiple stages. These steps or stages are not necessarily executed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be executed alternately or in turn with at least some of the steps or stages in other steps or other steps.

[0092] Based on the same inventive concept, the embodiments of the present application also provide a data upload device for implementing the data upload method described above. The solution provided by this device to solve the problem is similar to the solution described in the above method. Therefore, the specific limitations in one or more embodiments of the data upload device provided below can refer to the limitations on the data upload method in the above text, and will not be repeated here.

[0093] In an exemplary embodiment, as Figure 5 shown, a data upload device 500 is provided, including: a first determination module 501, a second determination module 502, a third determination module 503, a fourth determination module 504, and an upload module 505, where:

[0094] The first determination module 501 is configured to, during the process of uploading the shards of the target file from the local cache space to the remote server, if a random write instruction sent by the user terminal is received, determine at least one first initial shard corresponding to the random write instruction, and a first target shard among the at least one first initial shards that is located in the local cache space; wherein, the target file is divided into multiple shards with different serial numbers, and the different serial numbers follow a preset arrangement order.

[0095] The second determination module 502 is configured to, for the receiving moment when the random write instruction is received, determine a second initial shard that has been uploaded to the remote server before the receiving moment, and a first serial number of the second initial shard.

[0096] The third determination module 503 is configured to, for the shard among the second initial shards with the last serial number in the preset arrangement order, determine the serial number of the shard and the serial number before the serial number of the shard in the preset arrangement order as the second serial number.

[0097] The fourth determination module 504 is configured to, when the first serial number is inconsistent with the second serial number, determine the remaining serial numbers in the second serial number except the first serial number as the third serial number, and determine the shards corresponding to the third serial number as the second target shards.

[0098] The upload module 505 is configured to upload the first target shards and the second target shards to the remote server, and execute the random write instruction based on the upload result.

[0099] In some embodiments, the upload module 505 is further configured to determine the shards among the at least one first initial shards that are located in the remote server as the third target shards; download the first target shards and the third target shards in the remote server to the local cache space; execute the random write instruction on the downloaded shards, and upload the shards after executing the random write instruction to the remote server.

[0100] In some embodiments, the data upload device 500 is specifically configured to execute the random write instruction on the first initial shards when all the first initial shards are located in the local cache space.

[0101] In some embodiments, the data upload device 500 is further configured to, for any shard uploaded from the local cache space to the remote server, delete the shard in the local cache space after the shard is uploaded to the remote server.

[0102] In some embodiments, the data upload device 500 is further configured to sort the multiple shards of the target file according to the preset arrangement order of the serial numbers; divide the multiple shards into different shard intervals based on the sorting result; wherein, the random write instruction corresponds to at least one shard interval.

[0103] In some embodiments, the data uploading device 500 is further configured to, in the case of not receiving a random write instruction, upload the shards that only include valid data to the remote server for the shards in the target file; after the uploading of the shards that only include valid data is completed, upload the shards that include both valid data and all-zero data to the remote server; and after the uploading of the shards that include both valid data and all-zero data is completed, upload the shards that only include all-zero data to the remote server.

[0104] Each module in the above data uploading device can be implemented in whole or in part by software, hardware, and their combination. The above modules can be embedded in the processor in the computer device in hardware form or be independent of the processor, or can be stored in the memory in the computer device in software form, so that the processor can call and execute the operations corresponding to the above respective modules.

[0105] In an exemplary embodiment, a computer device is provided. The computer device can be a terminal, and its internal structure diagram can be as Figure 6 shown. The computer device includes a processor, a memory, an input / output interface, a communication interface, a display unit, and an input device. Among them, the processor, the memory, and the input / output interface are connected through a system bus, and the communication interface, the display unit, and the input device are connected to the system bus through the input / output interface. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The input / output interface of the computer device is used for exchanging information between the processor and external devices. The communication interface of the computer device is used for communicating with external terminals in a wired or wireless manner, and the wireless manner can be implemented through WIFI, a mobile cellular network, near field communication (NFC), or other technologies. When the computer program is executed by the processor, it implements a data uploading method.

[0106] Those skilled in the art can understand that Figure 6 the structure shown in

[0107] is only a block diagram of some structures related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0108] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored, and when the computer program is executed by a processor, the steps in the above method embodiments are implemented.

[0109] In one embodiment, a computer program product is provided, including a computer program, and when the computer program is executed by a processor, the steps in the above method embodiments are implemented.

[0110] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use, and processing of relevant data need to comply with relevant regulations.

[0111] Those of ordinary skill in the art can understand that all or part of the processes in the methods of the above embodiments can be completed by instructing relevant hardware through a computer program. The computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above methods. Among them, any reference to a memory, database, or other medium used in the embodiments provided in the present application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM), etc. The databases involved in the embodiments provided in the present application can include at least one of relational databases and non-relational databases. Non-relational databases can include distributed databases based on blockchain, etc., without limitation. The processors involved in the embodiments provided in the present application can be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, data processing logics based on quantum computing, artificial intelligence (AI) processors, etc., without limitation.

[0112] The technical features of the above embodiments can be combined arbitrarily. For the sake of brevity of description, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, it should be considered as the scope recorded in the present application.

[0113] The above-described embodiments merely represent several implementation manners of the present application. The description thereof is relatively specific and detailed, but it should not be construed as a limitation on the scope of the patent of the present application. It should be noted that for those of ordinary skill in the art, without departing from the concept of the present application, several modifications and improvements can still be made, and these all fall within the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the appended claims.

Claims

1. A data uploading method, characterized in that, The method includes: During the process of uploading shards of a target file from a local cache space to a remote server, if a random write instruction sent by a user terminal is received, determine at least one first initial shard corresponding to the random write instruction, and a first target shard among the at least one first initial shards that is located in the local cache space; wherein, the target file is divided into multiple shards with different serial numbers, and the different serial numbers follow a preset arrangement order. For the receiving moment when the random write instruction is received, determine a second initial shard that has been uploaded to the remote server before the receiving moment, and a first serial number of the second initial shard. For the shard among the second initial shards with the last serial number in the preset arrangement order, determine the serial number of the shard and the serial numbers before the serial number of the shard in the preset arrangement order as a second serial number. In the case where the first serial number is inconsistent with the second serial number, determine the remaining serial numbers in the second serial number except the first serial number as a third serial number, and determine the shards corresponding to the third serial number as second target shards. Upload the first target shard and the second target shards to the remote server, and execute the random write instruction based on the upload result.

2. The method according to claim 1, wherein The executing the random write instruction based on the upload result includes: Determine the shards among the at least one first initial shards that are located in the remote server as third target shards. Download the first target shard and the third target shards in the remote server to the local cache space. Execute the random write instruction on the downloaded shards, and upload the shards after executing the random write instruction to the remote server.

3. The method according to claim 1, characterized in that The method further includes: In the case where all the first initial shards are located in the local cache space, execute the random write instruction on the first initial shards.

4. The method according to claim 1, wherein The method further includes: For any shard uploaded from the local cache space to the remote server, after the shard is uploaded to the remote server, delete the shard in the local cache space.

5. The method according to claim 1, wherein The method further includes: Sort the multiple shards of the target file according to the preset arrangement order of the serial numbers. Based on the sorting result, divide the multiple shards into different shard intervals; wherein, the random write instruction corresponds to at least one shard interval.

6. The method according to claim 1, characterized in that, The method further includes: In the case where the random write instruction is not received, for the shards in the target file, upload the shards that only include valid data to the remote server. In the case where the upload of the shards that only include valid data is completed, upload the shards that include both valid data and all-zero data to the remote server. In the case where the upload of the shards that include both valid data and all-zero data is completed, upload the shards that only include all-zero data to the remote server.

7. A data uploading device, characterized in that, The device includes: A first determination module, configured to, during the process of uploading shards of a target file from a local cache space to a remote server, if a random write instruction sent by a user terminal is received, determine at least one first initial shard corresponding to the random write instruction, and a first target shard among the at least one first initial shards that is located in the local cache space; wherein the target file is divided into multiple shards with different serial numbers, and the different serial numbers follow a preset arrangement order; A second determination module, configured to, for the receiving moment when the random write instruction is received, determine second initial shards that have been uploaded to the remote server before the receiving moment, and a first serial number of the second initial shards; A third determination module, configured to, for the shard among the second initial shards with the last serial number in the preset arrangement order, determine the serial number of the shard and the serial numbers before the serial number of the shard in the preset arrangement order as second serial numbers; A fourth determination module, configured to, when the first serial number is inconsistent with the second serial numbers, determine the remaining serial numbers among the second serial numbers except the first serial number as third serial numbers, and determine the shards corresponding to the third serial numbers as second target shards; An upload module, configured to upload the first target shard and the second target shards to the remote server, and execute the random write instruction based on the upload result.

8. A computer device, comprising a memory and a processor, the memory storing a computer program, characterized in that, When the processor executes the computer program, the steps of the method according to any one of claims 1 to 6 are implemented.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

Images