Data uploading method and device, computer device, readable storage medium and program product

Abstract

The application relates to a data uploading method and device, computer equipment, a readable storage medium and a program product. The method comprises the following steps: determining a first initial shard corresponding to a random write instruction and a first target shard in the first initial shard located in a local cache space; determining a second initial shard uploaded before a receiving time of the random write instruction and a first serial number of the second initial shard; for a shard in the second initial shard and located at the rear end in a preset arrangement sequence, determining a serial number of the shard and a serial number located before the serial number of the shard as a second serial number; determining shards corresponding to remaining serial numbers except the first serial number in the second serial number as second target shards; uploading the first target shard and the second target shards to a remote server, and executing the random write instruction based on an uploading result. The method provided by the application can effectively reduce the waiting time of a user.

Description

Technical Field

[0001] The present application relates to the field of object storage technology, and in particular to a data uploading method, apparatus, computer equipment, readable storage medium, and program product. Background Art

[0002] With the development of cloud computing technology, cloud storage has become an important means for businesses and individuals to store large amounts of data. Zettabyte Object Storage (ZOS) is a cost-effective cloud storage solution that provides data storage services over the internet, storing data as objects on remote servers. Each object has a unique identifier and can be accessed and operated independently.

[0003] Existing object storage solutions perform well when processing sequential write operations. However, when processing random write operations, if the shards required for the random write operation have been uploaded from the local cache to the remote server, all shards of the file in the local cache must be uploaded to the remote server before the random write operation can be performed. This causes users to wait longer and the user experience is poor. Summary of the Invention

[0004] Based on this, it is necessary to provide a data uploading method, apparatus, computer equipment, readable storage medium and program product that can reduce the user's waiting time to address the above technical problems.

[0005] In a first aspect, the present application provides a data uploading method, the method comprising:

[0006] During the process of uploading a shard of a target file from a local cache space to a remote server, if a random write instruction sent by a user terminal is received, determining at least one first initial shard corresponding to the random write instruction and a first target shard located in the local cache space among the at least one first initial shard; wherein the target file is divided into a plurality of shards with different serial numbers, and the different serial numbers follow a preset arrangement order;

[0007] Determining, for a reception time of the random write instruction, a second initial fragment uploaded to the remote server before the reception time, and a first sequence number of the second initial fragment;

[0008] For the fragment whose sequence number is the last in the preset arrangement order in the second initial fragment, the sequence number of the fragment and the sequence number that is before the sequence number of the fragment in the preset arrangement order are determined as the second sequence number;

[0009] If the first sequence number is inconsistent with the second sequence number, the remaining sequence numbers in the second sequence number except the first sequence number are determined as the third sequence number, and the fragment corresponding to the third sequence number is determined as the second target fragment;

[0010] The first target shard and the second target shard are uploaded to a remote server, and a random write instruction is executed based on the uploaded result.

[0011] In one embodiment, executing a random write instruction based on the upload result includes:

[0012] Determine a shard located in a remote server among the at least one first initial shard as a third target shard;

[0013] Download the first target segment and the third target segment from the remote server to the local cache space;

[0014] Execute random write instructions on the downloaded shards, and upload the shards after executing the random write instructions to the remote server.

[0015] In one embodiment, the method further comprises:

[0016] When all first initial fragments are located in the local cache space, a random write instruction is executed on the first initial fragments.

[0017] In one embodiment, the method further comprises:

[0018] For any shard uploaded from the local cache space to the remote server, after the shard is uploaded to the remote server, the shard in the local cache space is deleted.

[0019] In one embodiment, the method further comprises:

[0020] Sort multiple fragments of the target file according to the preset order of sequence numbers;

[0021] Based on the sorting result, multiple shards are divided into different shard intervals; wherein the random write instruction corresponds to at least one shard interval.

[0022] In one embodiment, the method further comprises:

[0023] In the case that no random write instruction is received, for the fragments in the target file, only the fragments containing valid data are uploaded to the remote server;

[0024] When the upload of the fragment containing only valid data is completed, the fragment containing both valid data and all-zero data is uploaded to the remote server;

[0025] When the upload of the segments including both valid data and all-zero data is completed, the segments including only all-zero data will be uploaded to the remote server.

[0026] In a second aspect, the present application further provides a data uploading device, comprising:

[0027] A first determination module is configured to, when receiving a random write instruction sent by a user terminal during uploading of a target file segment from a local cache space to a remote server, determine at least one first initial segment corresponding to the random write instruction and a first target segment in the at least one first initial segment located in the local cache space; wherein the target file is divided into a plurality of segments with different serial numbers, and the different serial numbers follow a preset arrangement order;

[0028] A second determining module is configured to determine, based on a receiving time of the random write instruction, a second initial fragment uploaded to the remote server before the receiving time, and a first sequence number of the second initial fragment;

[0029] a third determining module, configured to determine, for a fragment of the second initial fragment whose sequence number is the last in the preset arrangement order, the sequence number of the fragment and a sequence number preceding the sequence number of the fragment in the preset arrangement order as a second sequence number;

[0030] A fourth determining module is configured to, when the first sequence number is inconsistent with the second sequence number, determine the remaining sequence numbers in the second sequence number except the first sequence number as the third sequence number, and determine the fragment corresponding to the third sequence number as the second target fragment;

[0031] The uploading module is used to upload the first target shard and the second target shard to the remote server, and execute the random write instruction based on the upload result.

[0032] In a third aspect, the present application further provides a computer device, which includes a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of the method in any of the above embodiments when executing the computer program.

[0033] In a fourth aspect, the present application further provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the method in any one of the above embodiments.

[0034] In a fifth aspect, the present application further provides a computer program product, which includes a computer program that implements the steps of the method in any one of the above embodiments when executed by a processor.

[0035] The data uploading method, device, computer device, readable storage medium and program product upload the slices of the target file from the local cache space to the remote server, and if a random write instruction sent by the user terminal is received, at least one first initial slice corresponding to the random write instruction and a first target slice in the at least one first initial slice located in the local cache space are determined; the target file is divided into slices with different serial numbers, and the different serial numbers follow a preset arrangement order; for a receiving time at which the random write instruction is received, a second initial slice uploaded to the remote server before the receiving time and a first serial number of the second initial slice are determined; for a slice in the second initial slice with a serial number at the back end in the preset arrangement order, the serial number of the slice and serial numbers before the serial number of the slice in the preset arrangement order are determined as second serial numbers; in a case where the first serial number is inconsistent with the second serial numbers, remaining serial numbers except the first serial number in the second serial numbers are determined as third serial numbers, and slices corresponding to the third serial numbers are determined as second target slices; the first target slice and the second target slices are uploaded to the remote server, and the random write instruction is executed based on an uploading result. The method provided in the application only needs to upload the first target slice and the second target slices to the remote server before executing the random write instruction, which can effectively reduce the waiting time of the user and improve the user experience. BRIEF DESCRIPTION OF DRAWINGS

[0036] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the related art, the drawings needed to be used in the description of the embodiments of the present application or the related art will be briefly introduced. Obviously, the drawings in the following description only some embodiments of the present application, and for those skilled in the art, other related drawings can be obtained without creative labor on the basis of these drawings.

[0037] Figure 1 A flowchart of a data uploading method in an embodiment;

[0038] Figure 2 A flowchart of a random write instruction execution method in an embodiment;

[0039] Figure 3 A schematic diagram of a random write operation in the prior art;

[0040] Figure 4 A schematic diagram of a random write operation in another embodiment;

[0041] Figure 5 A structural block diagram of a data uploading device in an embodiment;

[0042] Figure 6 An internal structure diagram of a computer device in an embodiment. DETAILED DESCRIPTION

[0043] In order to make the purposes, technical solutions and advantages of the present application clearer, the present application will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only intended to explain the present application, and not to limit the present application.

[0044] In the prior art, when a random write operation occurs, it is first checked whether the shard interval corresponding to this random write is still in the local, i.e. whether it is hit. If the interval is hit, the random write operation is directly performed. If the corresponding interval has been uploaded to the remote server, i.e. the interval is not hit, the set of shards of the file in the local cache space that have not been uploaded is recorded as At this time, the currently ongoing write operation needs to be paused first, and all shards in are uploaded to the remote server, and then the shard corresponding to the random write operation is downloaded. After the random write operation is completed, the paused write operation is resumed, and the data uploading is continued according to the original logic.

[0045] In the prior art, whenever a user performs a random write operation, if the interval is not hit, the user needs to wait until all the un-uploaded shards are completed uploading before the corresponding shard can be downloaded again and the random write operation is completed, which results in a long waiting time for the user and a poor experience. Therefore, in the present embodiment, by reducing the uploaded shards, the random write operation can be completed faster, the user waiting time is reduced, and the user experience is improved.

[0046] In one embodiment, as shown in Figure 1 , a data uploading method is provided, and the present embodiment takes the method applied to a terminal as an example. It should be understood that the method can also be applied to a server, and can also be applied to a system including a terminal and a server, and is realized through the interaction of the terminal and the server. In the present embodiment, the method includes the following steps:

[0047] S102, in the process of uploading the shards of the target file from the local cache space to the remote server, if a random write instruction sent by the user terminal is received, at least one first initial shard corresponding to the random write instruction and a first target shard in the at least one first initial shard located in the local cache space are determined; wherein the target file is divided into a plurality of shards with different serial numbers, and the different serial numbers follow a preset arrangement order.

[0048] Wherein, the random write instruction is used to randomly modify the corresponding shard in the local cache space; the process of uploading the shards of the target file from the local cache space to the remote server is the process of object storage; the serial number of the shard represents the position of the shard in the target file, and the shards are integrated according to the preset arrangement order of the serial numbers, so that the complete target file can be obtained.

[0049] S104, determine, for a receiving time of the received random write instruction, a second initial fragment that has been uploaded to the remote server before the receiving time, and a first sequence number of the second initial fragment.

[0050] Optionally, in the process of object storage, the fragments are not strictly uploaded according to the preset arrangement order of the sequence numbers, and therefore the first sequence numbers of different second initial fragments can be discontinuous.

[0051] S106, for a fragment in the second initial fragment with a sequence number at the last in the preset arrangement order, determine the sequence number of the fragment and the sequence numbers before the sequence number of the fragment in the preset arrangement order as second sequence numbers.

[0052] Optionally, for the fragment with the sequence number at the last in the second initial fragment, if there is a fragment with a sequence number before the sequence number of the last fragment that has not been uploaded to the remote server, there is a missing fragment, and the complete target file cannot be restored, therefore, all the second sequence numbers before the sequence number of the last fragment in the preset arrangement order need to be determined first, and whether there is a missing fragment in the uploaded fragments is determined based on the second sequence numbers.

[0053] S108, in the case where the first sequence number and the second sequence number are inconsistent, determine the remaining sequence numbers except the first sequence number in the second sequence numbers as third sequence numbers, and determine the fragments corresponding to the third sequence numbers as second target fragments.

[0054] Optionally, by comparing the first sequence number and the second sequence number, if there is a missing fragment, the sequence number of the missing fragment is determined as a third sequence number, and the set of second target fragments corresponding to the third sequence number is recorded as .

[0055] S110, upload the first target fragments and the second target fragments to the remote server, and execute the random write instruction based on the uploading result.

[0056] Optionally, the first target fragments are the fragments in the first initial fragments that have not been uploaded to the remote server, since the fragments are deleted from the local cache space after being uploaded, the local cache space only contains the first target fragments, and does not contain all the fragments corresponding to the random write instruction, at this time, the random write instruction cannot be executed in the local cache space, therefore, the remaining fragments corresponding to the random write instruction except the first target fragments need to be downloaded from the remote server, however, since these remaining fragments have undergone the uploading process, there can be a problem of data inconsistency between the downloaded remaining fragments and the first target fragments, therefore, the first target fragments need to be uploaded first, and then the uploaded first target fragments and the remaining fragments are downloaded together.

[0057] Optionally, in order to avoid the loss of the shards in the remote server, resulting in the incomplete target file after restoration, causing file damage, the second target shard needs to be uploaded to the remote server, thereby ensuring the consistency of the data in the remote server and the data in the target file.

[0058] Optionally, the set of the first target shards can be denoted as The sum of the set of and is determined as that is and Therefore, the final uploaded is .

[0059] In the data uploading method, in the process of uploading the shards of the target file from the local cache space to the remote server, if a random write instruction sent by the user terminal is received, at least one first initial shard corresponding to the random write instruction and a first target shard in the at least one first initial shard located in the local cache space are determined; wherein the target file is divided into a plurality of shards with different serial numbers, and the different serial numbers follow a preset arrangement order; for a receiving time at which the random write instruction is received, a second initial shard uploaded to the remote server before the receiving time and a first serial number of the second initial shard are determined; for a shard in the second initial shard with a serial number farthest to the rear in the preset arrangement order, the serial number of the shard and the serial numbers before the serial number of the shard in the preset arrangement order are determined as a second serial number; in the case where the first serial number is inconsistent with the second serial number, the remaining serial numbers except the first serial number in the second serial number are determined as a third serial number, and the shards corresponding to the third serial number are determined as second target shards; the first target shard and the second target shards are uploaded to the remote server, and the random write instruction is executed based on the uploading result. The method provided in the present application only needs to upload the first target shard and the second target shard to the remote server before executing the random write instruction, which can effectively reduce the waiting time of the user and improve the user experience.

[0060] In some embodiments, as shown in Figure 2 executing the random write instruction based on the uploading result includes:

[0061] S202, determining a shard in the at least one first initial shard located in the remote server as a third target shard.

[0062] S204, downloading the first target shard and the third target shard in the remote server to the local cache space.

[0063] S206, executing the random write instruction on the downloaded shard, and uploading the shard after executing the random write instruction to the remote server.

[0064] The third target slice is the remaining slice corresponding to the random write instruction except the first target slice.

[0065] In this embodiment, the first target segment and the third target segment in the remote server are downloaded to the local cache space, and random write instructions are executed on the downloaded segments, which can ensure the consistency of the segments on which the random write instructions are executed.

[0066] In some embodiments, the method further includes: executing a random write instruction on the first initial fragments when all the first initial fragments are located in the local cache space.

[0067] Optionally, if all first initial shards are located in the local cache space, the local cache space includes all shards corresponding to the random write instruction, and there is no data inconsistency problem between these shards, and the random write instruction can be directly executed on these shards.

[0068] In this embodiment, when all first initial fragments are located in the local cache space, executing random write instructions on the first initial fragments can effectively improve the execution efficiency of the random write instructions and reduce the waiting time of the user.

[0069] In some embodiments, the method further includes: for any shard uploaded from the local cache space to the remote server, after the shard is uploaded to the remote server, deleting the shard in the local cache space.

[0070] In this embodiment, after the fragments are uploaded to the remote server, the fragments in the local cache space are deleted, which can free up local space and avoid excessive pressure on the local cache.

[0071] In some embodiments, the method further includes: sorting multiple shards of the target file according to a preset arrangement order of serial numbers; dividing the multiple shards into different shard intervals based on the sorting results; wherein the random write instruction corresponds to at least one shard interval.

[0072] Optionally, in order to support the reading and writing of large files while reducing the impact on the reading and writing performance of small files, the maximum number of shards supported by the object storage for upload is comprehensively considered. For example, the maximum number of shards is 10,000. This embodiment adopts file sharding strategy optimization technology to reasonably shard the files, so that it supports the reading, writing and storage of large files while taking into account the performance requirements when modifying files of different sizes.

[0073] Specifically, the maximum number of fragments is divided into n segments according to the preset sequence of sequence numbers, and the last fragment sequence number of each segment is recorded as , then the maximum number of shards is divided into n shard intervals:

[0074]

[0075] Where, is the maximum number of shards, is the number of shards corresponding to each shard interval, .

[0076] Optionally, the size of the shard data within each shard interval is recorded as , then:

[0077]

[0078] in, , according to the requirements of the S3 protocol, .

[0079] Based on this sharding method, the maximum file size that object storage can support is:

[0080]

[0081] In this embodiment, since the shards in the later interval will not be used in small files, the data stored in the shards in the earlier interval has a higher modification frequency in the overall usage scenario. In view of this feature, the file reading and writing performance is optimized by adjusting the size of the shards in the front and back intervals. In addition, a larger later interval can support reading and writing larger files.

[0082] In some embodiments, the method further includes: when no random write instruction is received, for the slices in the target file, uploading the slices that only include valid data to the remote server; when the uploading of the slices that only include valid data is completed, uploading the slices that include both valid data and all-zero data to the remote server; when the uploading of the slices that include both valid data and all-zero data is completed, uploading the slices that only include all-zero data to the remote server.

[0083] Optionally, during object storage, the target file's shards can be uploaded sequentially according to different priorities. Specifically, the target file's shards can be divided into three upload priorities: 1) shards completely filled with valid data and no portion affected by read-modify-write (RMW); 2) shards partially filled with valid data and no portion affected by RMW; 3) shards containing only zero data. Each shard is uploaded sequentially in the order of priorities 1), 2), and 3).

[0084] In this embodiment, different shards are uploaded in sequence according to whether the shards contain valid data, which can more reasonably execute the object storage process.

[0085] In some embodiments, another data uploading method is provided, which includes the following contents:

[0086] Step one: reasonable setting of file fragments. According to the size and modification frequency of the file, the file fragments are set into 3 segments, and the sizes of the fragments are:

[0087]

[0088] Step two: support and optimization of random write operation. Taking the random write operation shown in Figure 3 and Figure 4 as an example, the operation process is described in detail, wherein, Figure 3 is a schematic diagram of the random write operation in the prior art, Figure 4 is a schematic diagram of the random write operation in the present embodiment. As shown in Figure 3 , in the first round of operation, the server stores the historical version of the file: 1-14 fragments are A, 1-7 and 12-14 fragments are B in this modification, and B is additionally written in 15-20 fragments, at this time, the local cache is 1-7 and 12-20 fragments, which are all B. Since the object storage supports unordered uploading, 2-4 and 17 fragments of data are uploaded in this round. In the second round of operation, the server storage data changes according to the uploading result of the last round. At this time, the user modifies the data of fragments 1-2 to C. Since the local data does not have fragment 2, the interval is not hit, so the subsequent write is suspended, and the data of fragments 1, 5-7, 12-16 and 18-20 in which have not been uploaded is uploaded. In the third round of operation, the server storage data remains the same as the state after all the write uploads in the first round, and the user's random write operation takes effect, writing C in 1-2 fragments and uploading. In the fourth round of operation, the random write operation is completed, and the file is restored to the writable state, and the process ends.

[0089] The optimized random write operation process is shown in Figure 4 . Similarly, in the first round of operation, the server stores the historical version of the file: 1-14 fragments are A, 1-7 and 12-14 fragments are B in this modification, and B is additionally written in 15-20 fragments, at this time, the local cache is 1-7 and 12-20 fragments, which are all B. 2-4 and 17 fragments of data are uploaded in this round. In the second round of operation, the user modifies the data of fragments 1-2 to C. Since the local data does not have fragment 2, the interval is not hit, so the subsequent write is suspended, and the data of fragments 1 and 5-7 in and are calculated according to the method in the present technical solution. The shards 15-16 in the collection are collected and corresponding data is uploaded. In the third round of operations, the data stored in the server is temporarily in a state of interleaving new and old data, but the requirement of random writing is met, the user's random writing operation takes effect, and the data is written in C and uploaded in 1-2 shards. In the fourth round of operations, the random writing operation is completed, the data is restored to a writable and uploadable state, and the data is written and uploaded in 3-4 shards. The shards 5-7, 12-14 and 17-20 in the collection continue to upload. In the fifth round of operations, the file uploading operation is completed, and the process ends.

[0090] Comparison Figure 3 And Figure 4 As can be seen, compared with the original method of uploading 12 shards, the optimized method only needs to upload 3 shards to perform the random writing operation, effectively reducing the user waiting time. If the shard that is temporarily suspended for uploading is hit by a subsequent random writing operation, repeated uploading of the same shard can be avoided, affecting the data uploading efficiency. Further, if the shard that is temporarily suspended for uploading is located at the rear of a file with a larger shard size, the optimization effect is more obvious.

[0091] It should be understood that, although each step in the flowchart involved in each embodiment as described above is displayed in sequence according to the arrow, these steps are not necessarily executed in sequence according to the arrow. Unless otherwise specified herein, the execution of these steps is not strictly limited in sequence, and these steps can be executed in other sequences. Moreover, at least part of the steps in the flowchart involved in each embodiment as described above can include multiple steps or stages, which are not necessarily executed at the same time, but can be executed at different times, and the execution sequence of these steps or stages is not necessarily sequential, but can be executed alternately or alternately with at least part of other steps or steps or stages in other steps.

[0092] Based on the same inventive concept, the embodiments of the present application also provide a data uploading device for implementing the above-mentioned data uploading method. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme described in the above method, so the specific limitations in one or more data uploading device embodiments provided below can refer to the limitations of the data uploading method described above, which will not be repeated here.

[0093] In an exemplary embodiment, as shown in Figure 5 A data uploading device 500 is provided, comprising a first determination module 501, a second determination module 502, a third determination module 503, a fourth determination module 504, and an uploading module 505, wherein:

[0094] The first determination module 501 is used to determine at least one first initial fragment corresponding to the random write instruction and a first target fragment located in the local cache space in the at least one first initial fragment if a random write instruction sent by a user terminal is received during the process of uploading the fragments of the target file from the local cache space to the remote server; wherein the target file is divided into multiple fragments with different serial numbers, and the different serial numbers follow a preset arrangement order.

[0095] The second determining module 502 is configured to determine, based on a receiving time of the random write instruction, a second initial fragment uploaded to the remote server before the receiving time and a first sequence number of the second initial fragment.

[0096] The third determining module 503 is configured to determine, for the fragment whose sequence number is the last in the preset arrangement sequence in the second initial fragment, the sequence number of the fragment and the sequence number preceding the sequence number of the fragment in the preset arrangement sequence as the second sequence number.

[0097] The fourth determining module 504 is configured to determine the remaining sequence numbers in the second sequence number except the first sequence number as the third sequence number when the first sequence number is inconsistent with the second sequence number, and determine the fragment corresponding to the third sequence number as the second target fragment.

[0098] The upload module 505 is configured to upload the first target segment and the second target segment to a remote server, and execute a random write instruction based on the upload result.

[0099] In some embodiments, the upload module 505 is further used to determine a shard located in a remote server in at least one first initial shard as a third target shard; download the first target shard and the third target shard in the remote server to a local cache space; execute a random write instruction on the downloaded shard, and upload the shard after executing the random write instruction to the remote server.

[0100] In some embodiments, the data upload device 500 is specifically configured to execute a random write instruction on the first initial fragments when all the first initial fragments are located in the local cache space.

[0101] In some embodiments, the data uploading device 500 is further configured to delete any shard uploaded from the local cache space to the remote server after the shard is uploaded to the remote server.

[0102] In some embodiments, the data upload device 500 is also used to sort multiple fragments of the target file according to a preset arrangement order of the serial numbers; divide the multiple fragments into different fragment intervals based on the sorting results; wherein the random write instruction corresponds to at least one fragment interval.

[0103] In some embodiments, the data uploading apparatus 500 is further configured to, in the case that no random write instruction is received, upload, to the remote server, a fragment including only valid data in the target file; in the case that the uploading of the fragment including only valid data ends, upload, to the remote server, a fragment including both valid data and all-zero data; and in the case that the uploading of the fragment including both valid data and all-zero data ends, upload, to the remote server, a fragment including only all-zero data.

[0104] The modules in the data uploading apparatus described above can be implemented by software, hardware, or a combination thereof, in whole or in part. The modules can be embedded in or independent of a processor in a computer device in hardware form, or stored in a memory in a computer device in software form, so as to be invoked and executed by a processor to perform the operations corresponding to the modules.

[0105] In one example embodiment, a computer device is provided, which can be a terminal, and an internal structure diagram of the computer device can be as shown in Figure 6 The computer device includes a processor, a memory, an input / output interface, a communication interface, a display unit, and an input device. The processor, the memory, and the input / output interface are connected through a system bus, and the communication interface, the display unit, and the input device are connected to the system bus through the input / output interface. The processor of the computer device is configured to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operating system and the computer program in the non-volatile storage medium to run. The input / output interface of the computer device is configured to exchange information between the processor and external devices. The communication interface of the computer device is configured to perform wired or wireless communication with external terminals, and the wireless communication can be achieved through WIFI, mobile cellular network, near field communication (NFC), or other technologies. The computer program is executed by the processor to implement a data uploading method.

[0106] Those skilled in the art can understand that Figure 6 The structure shown in the above

[0107] In one embodiment, a computer device is also provided, which includes a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the steps in the above method embodiments.

[0108] In an embodiment, a computer readable storage medium is provided, having stored thereon a computer program which, when executed by a processor, implements the steps of any of the above method embodiments.

[0109] In an embodiment, a computer program product is provided, comprising a computer program which, when executed by a processor, implements the steps of any of the above method embodiments.

[0110] It should be noted that the user information (including but not limited to user equipment information, user personal information, etc.) and data (including but not limited to data for analysis, stored data, displayed data, etc.) involved in the present application are all information and data authorized by the user or authorized by all parties, and the collection, use and processing of related data need to comply with relevant regulations.

[0111] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a non-volatile computer readable storage medium, and when executed, can include the processes of the above-mentioned embodiment methods. Any reference to memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile memory and volatile memory. The non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical storage, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. The volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration but not limitation, the RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM), etc. The database involved in the embodiments provided in the present application can include at least one of a relational database and a non-relational database. The non-relational database can include a distributed database based on a block chain, etc., without being limited thereto. The processor involved in the embodiments provided in the present application can be a general-purpose processor, a central processing unit, a graphics processing unit, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, an artificial intelligence (AI) processor, etc., without being limited thereto.

[0112] The technical features of the above embodiments can be combined in any manner. To make the description concise, all possible combinations of the technical features in the above embodiments are not described, but as long as the combinations of the technical features do not exist contradictions, they should be considered as the scope of the present application.

[0113] The above-described embodiments are merely illustrative of several embodiments of the present application, and the description is relatively specific and detailed, but should not be understood as a limitation on the scope of the patent. It should be noted that for those skilled in the art, without departing from the concept of the present application, a number of modifications and improvements can be made, which are all within the scope of the present application. Therefore, the scope of protection of the present application should be subject to the appended claims.

Claims

1. A data uploading method, characterized in that: The method comprises: During uploading of a target file segment from a local cache space to a remote server, if a random write instruction is received from a user terminal, determining at least one first initial segment corresponding to the random write instruction and a first target segment located in the local cache space among the at least one first initial segment; wherein the target file is divided into a plurality of segments with different sequence numbers, and the different sequence numbers follow a preset arrangement order; Determining, for a reception time of the random write instruction, a second initial fragment uploaded to the remote server before the reception time, and a first sequence number of the second initial fragment; For the fragment whose sequence number is the last in the preset arrangement order among the second initial fragments, determine the sequence number of the fragment and the sequence number that precedes the sequence number of the fragment in the preset arrangement order as the second sequence number; If the first sequence number is inconsistent with the second sequence number, determining the remaining sequence numbers in the second sequence number except the first sequence number as a third sequence number, and determining the fragment corresponding to the third sequence number as the second target fragment; The first target shard and the second target shard are uploaded to the remote server, and the random write instruction is executed based on the upload result.

2. The method according to claim 1, characterized in that The executing the random write instruction based on the upload result includes: Determine a shard in the at least one first initial shard and located in the remote server as a third target shard; Downloading the first target segment and the third target segment from the remote server to the local cache space; The random write instruction is executed on the downloaded segments, and the segments after the random write instruction is executed are uploaded to the remote server.

3. The method according to claim 1, characterized in that The method further comprises: In a case where all first initial fragments are located in the local cache space, the random write instruction is executed on the first initial fragments.

4. The method according to claim 1, wherein The method further comprises: For any fragment uploaded from the local cache space to the remote server, after the fragment is uploaded to the remote server, the fragment in the local cache space is deleted.

5. The method according to claim 1, wherein The method further comprises: Sorting the multiple fragments of the target file according to the preset arrangement order of the sequence numbers; The multiple shards are divided into different shard intervals based on the sorting result; wherein the random write instruction corresponds to at least one shard interval.

6. The method according to claim 1, characterized in that The method further comprises: In the case where the random write instruction is not received, uploading the fragments in the target file that only include valid data to the remote server; When the uploading of the fragments including only valid data is completed, uploading the fragments including both valid data and all-zero data to the remote server; When the uploading of the segments including both valid data and all-zero data is completed, the segments including only all-zero data are uploaded to the remote server.

7. A data uploading device, characterized in that: The device comprises: A first determination module is configured to, when receiving a random write instruction sent by a user terminal during uploading of a target file segment from a local cache space to a remote server, determine at least one first initial segment corresponding to the random write instruction and a first target segment in the at least one first initial segment located in the local cache space; wherein the target file is divided into a plurality of segments with different serial numbers, and the different serial numbers follow a preset arrangement order; A second determining module is configured to determine, based on a time at which the random write instruction is received, a second initial fragment uploaded to the remote server before the time at which the random write instruction is received, and a first sequence number of the second initial fragment; a third determining module, configured to determine, for a fragment of the second initial fragment whose sequence number is the last in the preset arrangement order, the sequence number of the fragment and a sequence number preceding the sequence number of the fragment in the preset arrangement order as a second sequence number; a fourth determining module, configured to, when the first sequence number is inconsistent with the second sequence number, determine the remaining sequence numbers in the second sequence number except the first sequence number as a third sequence number, and determine the fragment corresponding to the third sequence number as a second target fragment; An uploading module is used to upload the first target segment and the second target segment to the remote server, and execute the random write instruction based on the upload result.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 6 are implemented.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

10. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

Images