Data writing method and data reading method

Abstract

The invention provides a data writing method and a data reading method, and the method comprises the steps: obtaining target data and a target file in response to a data writing request, and writing the target data into a target block of the target file; when the target block is fully written, generating a block identifier of the target block and an offset variable of the target block based on the target data; first data information is generated based on the block identifier and the offset variable and written to a first index.

Description

Technical Field

[0001] This application relates to the field of data storage technology, and in particular to a method for writing and reading data. Background Technology

[0002] This section is intended to provide background or context for the embodiments of this application as set forth in the claims. The description herein is not to be construed as prior art simply because it is included in this section.

[0003] In distributed system consensus protocols, Raft is one of the most common schemes. When the Raft master node receives a data request from a client, it synchronizes this data to all other nodes in the system (followers). Once a majority of nodes confirm the data, the master node considers the data committable and synchronizes this commit information to all nodes to ensure that all nodes confirm the commit, completing a final data confirmation. To ensure data integrity and prevent data loss in the event of a system crash, committable data needs to be written to disk for persistence. This is typically achieved using a disk storage scheme based on an LSM-tree (Log-Structured Merge Tree).

[0004] However, although LSM-tree-based persistence schemes perform well in many scenarios, they have the following two main drawbacks in the application of the Raft consensus protocol: Redundant sorting operations: In the implementation of LSM-tree, data is first written to a memtable in memory, and sorted during this process. However, in the Raft protocol's log commit mechanism, log entries are generated and processed in a strict order. Therefore, for an already ordered Raft log, the sorting step of LSM-tree is redundant, leading to unnecessary computational overhead.

[0005] The high resource consumption of merge operations: LSM-tree structures perform file merge operations from layer L0 to layer L6. This merging can involve hundreds of MB or even tens of GB of data, placing high demands on disk I / O and CPU resources. However, in Raft protocol log persistence scenarios, this merging operation is actually unnecessary because Raft itself ensures data consistency and order, eliminating the need for an additional merging process to maintain data order. Therefore, such merging not only wastes valuable system resources but can also lead to performance bottlenecks; for example... Figure 7 The diagram shown is a schematic of the data storage architecture in the prior art.

[0006] Therefore, this application urgently needs to propose a data writing method and a data reading method that can solve the above-mentioned technical problems. Summary of the Invention

[0007] This application provides a data writing method and a data reading method to solve the technical problem of low data reading and writing efficiency in the prior art.

[0008] One aspect of this application proposes a method for writing data, the method comprising: In response to a write data request, the target data and target file are obtained, and the target data is written to the target block of the target file; Once the target block is full, a block identifier and an offset variable for the target block are generated based on the target data. First data information is generated based on the block identifier and the offset variable, and the first data information is written into the first index.

[0009] Further, after generating first data information based on the block identifier and the offset variable and writing the first data information into the first index, the method includes: obtaining the maximum block identifier and the minimum block identifier from the block identifiers of the target blocks of any target file; generating second data information based on the maximum block identifier and the minimum block identifier, and writing the second data information into the first index.

[0010] Furthermore, the method further includes: when the target file is full, obtaining the file identifier of the target file and the maximum block identifier of the target block of the target file; generating third data information based on the file identifier and the maximum block identifier, and writing the third data information into the second index.

[0011] Further, the target data includes a data identifier and a target value; the step of generating a block identifier and an offset variable of the target block based on the target data after the target block is full includes: obtaining the data identifier of the target data written to the target block, determining the data identifier of the target data as the block identifier of the target block; and determining the offset variable of the target block based on the block identifier.

[0012] Further, determining the offset variable of the target block based on the block identifier includes: when there are multiple block identifiers, determining the offset variable based on the difference between the largest block identifier and the smallest block identifier among the block identifiers; when there is only one block identifier, determining the offset variable to be 0.

[0013] Furthermore, the step of responding to a write data request, obtaining target data and a target file, and writing the target data to a target block of the target file includes: responding to a write data request, obtaining a data identifier of the target data; and sequentially writing the target data to a target block of the target file based on the data identifier of the target data.

[0014] Furthermore, the method further includes: generating a data identifier to be deleted in response to a data deletion request; retrieving a data identifier of target data stored in the target file; determining a data identifier not greater than the data identifier to be deleted as the target data identifier; and deleting the target data corresponding to the target data identifier and the target data identifier stored in the target block.

[0015] Furthermore, the method further includes: when the target data identifier includes a maximum block identifier and / or a minimum block identifier, determining the maximum block identifier and minimum block identifier of the target data that has not been deleted in the target block where the target data corresponding to the target data identifier is located, and updating the generated second data information based on the maximum block identifier and minimum block identifier of the target data that has not been deleted.

[0016] Furthermore, the method further includes: when the target data identifier includes a maximum block identifier, then determining the maximum block identifier of the undeleted target data in the target block where the target data corresponding to the target data is located, and the file identifier of the target file where the target data corresponding to the target data is located; updating the generated third data information based on the file identifier and the maximum block identifier of the undeleted target data.

[0017] Furthermore, the method also includes: when the target file is full, marking the read / write status of the target file as read-only; otherwise, marking the read / write status of the target file as read / write.

[0018] In another aspect, this application provides a method for reading data, the method comprising: Responding to a read data request, obtain the data identifier of the data to be read; Based on the data identifier, the block to be read is located using a global index, and the value of the data to be read is read based on the block to be read; The data to be read is written to the read block of the file based on the data writing method described above.

[0019] Further, the step of locating the block to be read based on the data identifier through a global index, and reading the value of the data to be read based on the block to be read, includes: determining a file identifier from a second index based on the data identifier, determining the file to be read based on the file identifier, and reading the value of the data to be read from the block to be read in the file to be read based on the data identifier.

[0020] This application proposes a data writing method and a data reading method. The data writing method includes: in response to a data writing request, acquiring target data and a target file, and writing the target data to a target block of the target file; when the target block is full, generating a block identifier and an offset variable of the target block based on the target data; generating first data information based on the block identifier and the offset variable, and writing the first data information into a first index. The data writing method proposed in this application eliminates the step of sorting data during the data writing process due to the ordered nature of the target data (eliminating the memtable in the original data storage architecture from a data storage architecture perspective), improving data writing efficiency while reducing system resource consumption. Furthermore, this application creatively proposes generating a block identifier based on a data identifier, generating an offset variable based on the block identifier, and then generating the first data information to construct a first index. This provides convenience for data reading without increasing the difficulty of data writing or reducing data writing efficiency. Specifically, during data reading, the system can quickly locate and read the target data based on the data identifier included in the data reading request according to a binary search rule. Furthermore, this application proposes that when the target file is full, the file identifier of the target file and the maximum block identifier of the target block of the target file are obtained; third data information is generated based on the file identifier and the maximum block identifier, and the third data information is written into the second index. By establishing the second index, an inter-file index is constructed, enabling the location of target data across files, and the combination of the second index and the first index further improves the data query efficiency. Attached Figure Description

[0021] To more clearly illustrate the technical solutions in the embodiments of this application, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0022] Other features, objects, and advantages of this application will become more apparent from the following detailed description of non-limiting embodiments with reference to the accompanying drawings: Figure 1A flowchart illustrating a data writing method provided in an embodiment of this application; Figure 2 A schematic diagram of a data storage architecture provided for another embodiment of this application; Figure 3 A flowchart illustrating a data reading method provided in yet another embodiment of this application; Figure 4 This is a schematic diagram of the structure of a data writing device provided in another embodiment of this application; Figure 5 This is a schematic diagram of the structure of a data reading device provided in an embodiment of this application; Figure 6 This is a schematic diagram of the structure of an electronic device suitable for implementing the solutions in the embodiments of this application; Figure 7 This is a schematic diagram of the data storage architecture of the prior art described in this application; The same or similar reference numerals in the accompanying drawings represent the same or similar parts. Detailed Implementation

[0023] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0024] In a typical configuration of this application, the terminal and the service network devices each include one or more processors (CPUs), input / output interfaces, network interfaces, and memory.

[0025] Memory may include non-persistent storage in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.

[0026] Computer-readable media include permanent and non-permanent, removable and non-removable media, which can store information by any method or technology. Information can be computer program instructions, data structures, program modules, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, read-only optical disc (CD-ROM), digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other non-transfer medium that can be used to store information accessible by a computing device.

[0027] This application provides a method for writing data, the method comprising at least the following steps: Step S101: In response to the write data request, obtain the target data and the target file, and write the target data to the target block of the target file; Step S102: When the target block is full, generate the block identifier of the target block and the offset variable of the target block based on the target data; Step S103: Generate first data information based on the block identifier and the offset variable, and write the first data information into the first index.

[0028] In practical scenarios, the execution entity of this method can be a user device, or a device formed by integrating a user device and a network device through a network, or it can be an application running on the aforementioned devices. The user device includes, but is not limited to, various terminal devices such as computers, mobile phones, tablets, smartwatches, and wristbands. The network device includes, but is not limited to, network hosts, single network servers, multiple network server sets, or cloud computing-based computer sets, and can be used to implement some processing functions when setting an alarm clock. Here, the cloud consists of a large number of hosts or network servers based on cloud computing. Cloud computing is a type of distributed computing, consisting of a virtual computer composed of a group of loosely coupled computer sets.

[0029] Example 1

[0030] Figure 1 The present application illustrates a data writing method processing flow according to an embodiment of the present application, the method including at least the following processing steps: Step S101: In response to the write data request, obtain the target data and the target file, and write the target data to the target block of the target file; Step S102: When the target block is full, generate the block identifier of the target block and the offset variable of the target block based on the target data; Step S103: Generate first data information based on the block identifier and the offset variable, and write the first data information into the first index.

[0031] In one implementation, obtaining the target file in response to a write data request includes: if the write data request includes a file identifier, the target file can be determined according to that file identifier; alternatively, a file with a read / write status can be randomly selected as the target file.

[0032] It should be understood that the solution described in this application implements Raft log persistence (e.g., storing Raft logs to the target disk, and the target data described in this application can be understood as Raft logs); and compared with the prior art, this application does not rely on memtable, but writes the target data into the target file in order based on the order of the target data itself. In this way, on the one hand, it ensures that the target data is ordered in the target file, and on the other hand, it eliminates the sorting operation of the target data in memtable.

[0033] like Figure 2 As shown, based on the data storage architecture for implementing the data writing method and data reading scheme described in this application, the architecture includes a disk area (i.e., the target file described in this application). The disk area includes at least one disk and a global index storage unit. The global index storage unit is used to store a second index. Each disk corresponds to an entry file and a first index. The entry file includes several blocks for storing several target data entries.

[0034] The first index includes several index areas for storing data information of any block. The data information of a block includes a maximum block identifier, a minimum block identifier, and an offset variable. Further, the minimum block identifier is the smallest data identifier among the data identifiers of the target data written to the block (which can also be understood as the data identifier corresponding to the first target data of the block), and the maximum block identifier is the largest data identifier among the data identifiers of the target data written to the block (which can also be understood as the data identifier corresponding to the last target data of the block). The offset variable is the absolute value of the difference between the maximum block identifier and the minimum block identifier. The block identifier, data identifier, and file identifier mentioned in this application are all positive integers.

[0035] In one embodiment, after generating first data information based on the block identifier and the offset variable and writing the first data information into the first index, the method includes: obtaining the maximum block identifier and the minimum block identifier from the block identifiers of the target blocks of any target file; generating second data information based on the maximum block identifier and the minimum block identifier, and writing the second data information into the first index.

[0036] In one embodiment, the first index further includes a target index area for storing second data information of the first index. The second data information of the first index is the maximum block identifier and the minimum block identifier among all block identifiers existing in the first index. For example, if the first index includes data information of three blocks A, B, and C, where the minimum block identifier corresponding to A is 1, the minimum block identifier corresponding to B is 2, and the minimum block identifier corresponding to C is 3, then the minimum block identifier included in the data information of the first index is 1. Correspondingly, assuming that the maximum block identifier corresponding to A is 4, the maximum block identifier corresponding to B is 5, and the maximum block identifier corresponding to C is 6, then the maximum block identifier included in the data information of the first index is 6.

[0037] It should be understood that the data identifier described in this application can be understood as the key of the target data, and the target value of the target data is the value; that is, the data form of the target data described in this application is key-value pair form.

[0038] In one embodiment, the method further includes: when the target file is full, obtaining the file identifier of the target file and the maximum block identifier of the target block of the target file; generating third data information based on the file identifier and the maximum block identifier, and writing the third data information into the second index.

[0039] Specifically, the second index is equivalent to integrating the isolated target files, which is equivalent to building an inter-file index. The global index covers all files in the entire data storage system, providing a unified view to quickly locate the file where the target data is located.

[0040] Building a second index has several advantages: First, it enables cross-file location, allowing direct identification of the target file containing the target data without traversing each file. Second, it reduces unnecessary I / O operations, as a single lookup determines which file to access, avoiding reading unrelated files and thus reducing disk I / O. Third, it improves query efficiency, enabling rapid location of target data even in large-scale distributed environments, significantly increasing query speed.

[0041] In one embodiment, the target data includes a data identifier and a target value; the step of generating a block identifier and an offset variable of the target block based on the target data after the target block is full includes: obtaining the data identifier of the target data written to the target block, determining that the data identifier of the target data is the block identifier of the target block; and determining the offset variable of the target block based on the block identifier.

[0042] In one embodiment, determining the offset variable of the target block based on the block identifier includes: when there are multiple block identifiers, determining the offset variable based on the difference between the largest block identifier and the smallest block identifier among the block identifiers; when there is only one block identifier, determining the offset variable to be 0.

[0043] Specifically, this application does not limit the amount of data stored in the block, and similarly, this application does not limit the number of blocks included in the target file. For example, the number of target blocks can be one or more, or the target data stored in the target block can be one or more.

[0044] In one embodiment, the step of obtaining target data and target file in response to a write data request, and writing the target data to a target block of the target file, includes: obtaining a data identifier of the target data in response to a write data request; and sequentially writing the target data to a target block of the target file based on the data identifier of the target data.

[0045] Specifically, when the target data is a Raft log, the target data has the characteristic of being ordered, that is, the data identifiers of the target data are arranged sequentially.

[0046] In one embodiment, the method further includes: generating a data identifier to be deleted in response to a data deletion request; retrieving a data identifier of target data stored in the target file; determining a data identifier not greater than the data identifier to be deleted as the target data identifier; and deleting the target data corresponding to the target data identifier and the target data identifier stored in the target block.

[0047] Specifically, if the target file stores 100,000 data entries, then the maximum data identifier corresponding to the target file is determined to be 100,000 (assuming the data identifier of the first target data entry is 1, and the data identifiers of subsequent target data entries are arranged sequentially according to the rule of a difference of 1; the meaning of this example is that because log data is ordered, the maximum data identifier corresponding to the target file can be determined). If 50% of the data in the target file is to be deleted, then the range of data identifiers of the data to be deleted / or the maximum value of the data identifiers of the data to be deleted can be determined based on the data identifiers of the target data. Therefore, based on the data identifiers of the data to be deleted and the data identifiers of the target data, batch deletion of the target data can be achieved. The batch deletion of target data can be performed using an asynchronous task.

[0048] In one embodiment, the method further includes: when the target data identifier includes a maximum block identifier and / or a minimum block identifier, determining the maximum block identifier and minimum block identifier of the target data that has not been deleted in the target block where the target data corresponding to the target data identifier is located, and updating the generated second data information based on the maximum block identifier and minimum block identifier of the target data that has not been deleted.

[0049] When deleting data, the data writing method described in this application only requires deleting the target data according to the identifier information such as the target data identifier, block identifier, and file identifier. Since the data written to the target block and target file in this application does not require or need to be ordered, the data writing and deletion process does not require the frequent comparison and sorting of updated data as in existing data writing and deletion schemes (data writing and deletion schemes based on LSM-Tree structure). Because it is not necessary to frequently read and write large amounts of data, the index will not increase the burden on the target file (such as disk). Furthermore, in the prior art, there is a situation where the actual amount of data written far exceeds the amount of data requested for writing due to multiple rewritings of the same data to complete the merging. However, the data writing and deletion scheme proposed in this application does not have the above-mentioned drawbacks of the prior art.

[0050] In one embodiment, the method further includes: when the target data identifier includes a maximum block identifier, determining the maximum block identifier of the undeleted target data in the target block where the target data corresponding to the target data is located, and determining the file identifier of the target file where the target data corresponding to the target data is located; updating the generated third data information based on the file identifier and the maximum block identifier of the undeleted target data.

[0051] In this application, the third data information can be in key-value pair format, i.e., the file identifier is the key and the maximum block identifier is the value. Updating the generated third data information based on the file identifier and the maximum block identifier of the undeleted target data includes: after locking the file identifier, updating the original maximum block identifier included in the generated third data information to the maximum block identifier of the undeleted target data.

[0052] In one embodiment, the method further includes: when the target file is full, marking the read / write status of the target file as read-only; otherwise, marking the read / write status of the target file as read / write.

[0053] It's important to understand that a target file whose read / write status is read-only can have its read / write status changed to read / write. For example, when you need to delete target data in a target file, you can change its read / write status to read / write. However, it's important to note that a target file whose read / write status has been changed from read-only to read / write but whose target data has not yet been deleted is considered to be in a state of being full. At this point, it's necessary to be careful that the target file does not mistakenly receive write data requests.

[0054] The data writing method proposed in this embodiment has significant innovations compared to existing technologies: Firstly, in existing LSM-tree implementations, all data is first written to the memtable, and each write requires merging with existing data. However, the data writing method proposed in this application eliminates the need for comparison operations during the writing of target data to the target file, saving CPU overhead associated with data comparison and writing. Secondly, in existing LSM-tree implementations, file merging occurs intermittently, including merging data from any two or more levels from the first to the nth level; and existing data merging rewrites and generates new files, resulting in disk I / O and CPU consumption. The data writing method proposed in this application avoids file merging operations, only requiring deletion at appropriate times, thus saving CPU consumption from disk I / O read / write and CPU consumption from comparing the text identifiers of multiple files. Therefore, compared to existing technologies, the CPU and disk I / O consumption is lower when writing data using the method described in this application.

[0055] Example 2

[0056] Figure 3 The present application illustrates a data reading method processing flow according to an embodiment of the present application. The method includes at least the following processing steps: Step S201: In response to the read data request, obtain the data identifier of the data to be read; Step S202: Locate the block to be read using the global index based on the data identifier, and read the value of the data to be read based on the block to be read.

[0057] The data to be read is written to the read block of the file based on the data writing method described above.

[0058] In one embodiment, the step of locating the block to be read based on the data identifier through a global index and reading the value of the data to be read based on the block to be read includes: determining a file identifier from a second index based on the data identifier, determining the file to be read based on the file identifier, and reading the value of the data to be read from the block to be read in the file to be read based on the data identifier.

[0059] In one implementation, determining the file descriptor from the second index based on the data identifier can be achieved using a binary search rule. Similarly, reading the value of the data to be read from the block to be read in the file to be read based on the data identifier can also be achieved using a binary search rule.

[0060] Based on the data writing method proposed in Embodiment 1, a first index and a second index are established. The first index is equivalent to an intra-file index, and the second index is equivalent to an inter-file index. The inter-file index enables cross-file data retrieval or, in other words, enables fast data location at irregular intervals within a file. Intra-file retrieval enables accurate retrieval and fast location of target data within a file.

[0061] The data reading method based on the data writing method proposed in Embodiment 1 not only accelerates data query efficiency but also reduces system resource consumption and enhances system scalability. Specifically, the accelerated data query efficiency is achieved by first narrowing the search scope to specific target files using the second index, and then precisely locating the target data based on the first index; the combination of these two approaches significantly improves query efficiency. Reduced system resource consumption is primarily reflected in the reduction of unnecessary file readings and memory usage (reading files consumes memory), enabling the system to handle more requests with limited resources. Enhanced system scalability is mainly reflected in the fact that as the amount of target data increases, this two-level indexing strategy of the first and second indexes can still maintain high data indexing performance and is suitable for deployment in large-scale distributed systems.

[0062] Example 3

[0063] Figure 4 This application illustrates a data writing apparatus according to an embodiment of the present application, the apparatus comprising: A response unit is used to respond to a write data request, obtain target data and a target file, and write the target data to a target block of the target file; A generation unit is used to generate a block identifier and an offset variable of the target block based on the target data when the target block is full. A writing unit is used to generate first data information based on the block identifier and the offset variable and write the first data information into a first index.

[0064] Example 4

[0065] like Figure 5 As shown, this application proposes a data reading device, the device comprising: A response unit for obtaining a data identifier for the data to be read in response to a read data request; A determining unit for locating the block to be read using a global index based on the data identifier; A reading unit for reading the value of the data to be read based on the block to be read; The data to be read is written to the read block of the file based on the writing data device described above.

[0066] Example 5

[0067] Based on the same inventive concept, this application also provides an electronic device. The method corresponding to the electronic device can be the data writing method and data reading method in the foregoing embodiments, and its problem-solving principle is similar to that method. The electronic device provided in this application includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to execute the methods and / or technical solutions of the foregoing embodiments of this application.

[0068] The electronic device can be a user device, or a device formed by integrating user devices and network devices through a network, or it can be an application running on the aforementioned devices. The user device includes, but is not limited to, various terminal devices such as computers, mobile phones, tablets, smartwatches, and wristbands. The network device includes, but is not limited to, network hosts, single network servers, multiple network server sets, or cloud computing-based computer sets, and can be used to implement some processing functions when setting an alarm clock. Here, the cloud consists of a large number of hosts or network servers based on cloud computing. Cloud computing is a type of distributed computing, consisting of a virtual computer composed of a group of loosely coupled computer sets.

[0069] Figure 6 The diagram illustrates the structure of an electronic device suitable for implementing the methods and / or technical solutions in the embodiments of this application. The device 500 includes a Central Processing Unit (CPU) 501, which can perform various appropriate actions and processes based on a program stored in a Read-Only Memory (ROM) 502 or a program loaded from a storage portion 508 into a Random Access Memory (RAM) 503. The RAM 503 also stores various programs and data required for system operation. The CPU 501, ROM 502, and RAM 503 are interconnected via a bus 504. An Input / Output (I / O) interface 505 is also connected to the bus 504.

[0070] The following components are connected to I / O interface 505: an input section 506 including a keyboard, mouse, touchscreen, microphone, infrared sensor, etc.; an output section 507 including a cathode ray tube (CRT), liquid crystal display (LCD), LED display, OLED display, etc., and speakers, etc.; a storage section 508 including one or more computer-readable media such as hard disk, optical disk, magnetic disk, semiconductor memory, etc.; and a communication section 509 including a network interface card such as a LAN (Local Area Network) card, modem, etc. The communication section 509 performs communication processing via a network such as the Internet.

[0071] In particular, the methods and / or embodiments in this application can be implemented as computer software programs. For example, the embodiments disclosed in this application include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowchart. When the computer program is executed by the central processing unit (CPU) 501, it performs the functions defined in the methods of this application.

[0072] Example 6

[0073] Another embodiment of this application provides a computer-readable storage medium having computer program instructions stored thereon, which can be executed by a processor to implement the methods and / or technical solutions of any one or more embodiments of this application described above.

[0074] Specifically, this embodiment may employ any combination of one or more computer-readable media. A computer-readable medium may be a computer-readable signal medium or a computer-readable storage medium. A computer-readable storage medium may be, for example—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or any combination thereof. More specific examples of computer-readable storage media (a non-exhaustive list) include: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this document, a computer-readable storage medium may be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, device, or apparatus.

[0075] Computer-readable signal media may include data signals propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals may take various forms, including—but not limited to—electromagnetic signals, optical signals, or any suitable combination thereof. Computer-readable signal media may also be any computer-readable medium other than computer-readable storage media, capable of sending, propagating, or transmitting programs for use by or in connection with an instruction execution system, device, or apparatus.

[0076] The program code contained on a computer-readable medium may be transmitted using any suitable medium, including—but not limited to—wireless, wire, optical fiber, RF, etc., or any suitable combination thereof.

[0077] Computer program code for performing the operations of this application can be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, and C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0078] The flowcharts or block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of devices, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-specific system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0079] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0080] In the several embodiments provided in this application, it should be understood that the disclosed systems, devices, and methods can be implemented in other ways. For example, the device embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or page components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the displayed or discussed mutual couplings or direct couplings or communication connections may be through some interfaces, or indirect couplings or communication connections between devices or units, and may be electrical, mechanical, or other forms.

[0081] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0082] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or in a combination of hardware and software functional units.

[0083] The integrated units implemented as software functional units described above can be stored in a computer-readable storage medium. These software functional units, stored in a storage medium, include several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) or processor to execute some steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0084] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application.

[0085] Furthermore, it is clear that the word "comprising" does not exclude other units or steps, and the singular does not exclude the plural. Multiple units or devices recited in a device claim may also be implemented by a single unit or device through software or hardware. The terms "first," "second," etc., are used to indicate names and do not indicate any specific order.

Claims

1. A method for writing data, characterized in that, The method includes: In response to a write data request, the target data and target file are obtained, and the target data is written to the target block of the target file; Once the target block is full, a block identifier and an offset variable for the target block are generated based on the target data. First data information is generated based on the block identifier and the offset variable, and the first data information is written into the first index.

2. The data writing method according to claim 1, characterized in that, After generating first data information based on the block identifier and the offset variable and writing the first data information into the first index, the method includes: Based on any of the target files, obtain the maximum and minimum block identifiers from the block identifiers of the target blocks of the target files; Second data information is generated based on the maximum block identifier and the minimum block identifier, and the second data information is written into the first index; And / or, the method further includes: When the target file is full, obtain the file identifier of the target file and the maximum block identifier of the target block of the target file; Third data information is generated based on the file identifier and the maximum block identifier, and the third data information is written into the second index.

3. The data writing method according to claim 2, characterized in that, The target data includes a data identifier and a target value; When the target block is full, generating the block identifier and offset variable of the target block based on the target data includes: Obtain the data identifier of the target data to be written to the target block, and determine that the data identifier of the target data is the block identifier of the target block; The offset variable of the target block is determined based on the block identifier.

4. The data writing method according to claim 3, characterized in that, The step of determining the offset variable of the target block based on the block identifier includes: When there are multiple block identifiers, the offset variable is determined based on the difference between the largest block identifier and the smallest block identifier among the block identifiers; When there is only one block identifier, the offset variable is determined to be 0.

5. The data writing method according to claim 1 or 4, characterized in that, The step of responding to a write data request, obtaining target data and a target file, and writing the target data to a target block of the target file includes: In response to a write data request, obtain the data identifier of the target data; Based on the data identifiers of the target data, the target data is sequentially written into the target block of the target file.

6. The data writing method according to claim 5, characterized in that, The method further includes: In response to a data deletion request, a data identifier to be deleted is generated, and the data identifier of the target data stored in the target file is retrieved; A data identifier not greater than the data identifier to be deleted is determined as the target data identifier, and the target data corresponding to the target data identifier and the target data identifier stored in the target block are deleted.

7. The data writing method according to claim 6, characterized in that, The method further includes: When the target data identifier includes a maximum block identifier and / or a minimum block identifier, the maximum block identifier and minimum block identifier of the target data that has not been deleted in the target block where the target data corresponding to the target data identifier is located are determined, and the generated second data information is updated based on the maximum block identifier and minimum block identifier of the target data that has not been deleted.

8. The data writing method according to claim 7, characterized in that, The method further includes: When the target data identifier includes the maximum block identifier, the maximum block identifier of the target data that has not been deleted in the target block where the target data is located, and the file identifier of the target file where the target data is located are determined. The generated third data information is updated based on the file identifier and the largest block identifier of the undeleted target data; And / or, the method further includes: When the target file is full, the read / write status of the target file is marked as read-only; Conversely, the read / write status of the target file is marked as read / write.

9. A method for reading data, characterized in that, The method includes: Responding to a read data request, obtain the data identifier of the data to be read; Based on the data identifier, the block to be read is located using a global index, and the value of the data to be read is read based on the block to be read; The data to be read is written to the read block of the file according to the data writing method described in any one of claims 1 to 8.

10. The data reading method according to claim 9, characterized in that, The step of locating the block to be read using a global index based on the data identifier, and reading the value of the data to be read based on the block to be read, includes: The file identifier is determined from the second index based on the data identifier, and the file to be read is determined based on the file identifier; The value of the data to be read is read from the block to be read in the file to be read based on the data identifier.

Images