Data writing method and data reading method

Abstract

The invention provides a data writing method and a data reading method.The data writing method comprises the steps that in response to a data writing request, target data and a data type, a target key and data information of the target data are determined; obtaining a data retrieval rule corresponding to the data type of the target data, performing retrieval in a preset data storage area based on the data information of the target data, and determining a retrieval result; and obtaining a Hash mapping rule corresponding to the data type of the target data, and writing the data information into the preset data storage area based on the retrieval result and a target key of the target data corresponding to the Hash mapping rule.

Description

Technical Field

[0001] This application relates to the field of data storage technology, and in particular to a data writing method and a data reading method. Background Technology

[0002] This section is intended to provide background or context for the embodiments of this application as set forth in the claims. The description herein is not to be construed as prior art simply because it is included in this section.

[0003] Redis (RemoteDictionaryServer, a high-performance key-value store) is widely used in many scenarios due to its rich and convenient data structure design. However, Redis's all-in-memory data storage method has obvious drawbacks. Because it relies entirely on physical memory, its storage capacity is limited by the size of the hardware memory. When the amount of data continues to grow beyond the memory's capacity, it will face a storage bottleneck.

[0004] To overcome this memory limitation, many solutions choose to write data to disk storage. A large portion of disk storage solutions are based on key-value (KV) storage using LSM trees (Log-Structured Merge-Tree). The core design principle of LSM trees is to transform the originally random write operation into ordered data writing. This ordered structure ensures the ordered output of data when performing range queries, while maintaining the time complexity of write operations and point queries at O(logN).

[0005] Therefore, this application urgently needs to propose a data writing method and a data reading method that can solve the above-mentioned technical problems. Summary of the Invention

[0006] This application provides a data writing method and a data reading method to improve the efficiency of data writing and reduce the complexity of data writing and data reading.

[0007] One aspect of this application proposes a data writing method, the method comprising: In response to a data write request, determine the target data and the data type, target key, and data information of the target data; Obtain the data retrieval rules corresponding to the data type of the target data, perform a retrieval in a preset data storage area based on the data information of the target data, and determine the retrieval results; Obtain the hash mapping rule corresponding to the data type of the target data, and write the data information to the preset data storage area based on the search result and the target key of the target data corresponding to the hash mapping rule.

[0008] Furthermore, the data type of the target data includes single-value data and multi-field composite data; the data information of the target data includes at least the target value and target metadata.

[0009] Furthermore, the step of obtaining the data retrieval rules corresponding to the data type of the target data, and retrieving the data information of the target data in a preset data storage area to determine the retrieval result, includes: when the data type of the target data is single-value data, retrieving the target key of the target data in a preset memory to determine a first result, the first result including existing and non-existent; when the data type of the target data is multi-field composite and the first result is non-existent, retrieving the target key of the target data in a preset disk to determine a second result, the second result including existing and non-existent; wherein, the preset data storage area includes the preset memory and the preset disk.

[0010] Furthermore, the step of obtaining the hash mapping rule corresponding to the data type of the target data, and writing the data information to the preset data storage area based on the retrieval result and the target key of the target data corresponding to the hash mapping rule, includes: when the target data is single-value data and the first result does not exist, then writing the target data to the preset memory; when the target data is single-value data and the first result already exists, then obtaining the initial value and initial metadata corresponding to the target key, and overwriting the initial value and initial metadata based on the target value and target metadata corresponding to the target key.

[0011] Further, the step of obtaining the hash mapping rule corresponding to the data type of the target data, and writing the data information to the preset data storage area based on the retrieval result and the target key of the target data corresponding to the hash mapping rule, includes: when the target data is a multi-field composite and the first result already exists, determining whether the target data has expired; if the target data has expired, obtaining the initial value and initial metadata corresponding to the target key, and overwriting the initial value and initial metadata based on the target value and target metadata corresponding to the target key; if the target data has not expired, obtaining the current version number of the target data, generating a target version number based on the current version number, and storing the target value and target metadata in the preset memory based on the target version number corresponding to the target key of the target data; wherein, the target version number and the current version number are unique.

[0012] Further, the step of obtaining the hash mapping rule corresponding to the data type of the target data, and writing the data information to the preset data storage area based on the retrieval result and the target key of the target data corresponding to the hash mapping rule, includes: when the target data is a multi-field composite and the first result does not exist, then obtaining the second result; when the second result does not exist, then obtaining all version numbers in the preset data storage area, the version numbers including the current version number and the historical version number; generating a target version number based on the current version number and the historical version number, and storing the target value and the target metadata to the preset memory based on the target key of the target data corresponding to the target version number.

[0013] Furthermore, the method further includes: obtaining initial length information of the target data; generating target length information based on the target data; and updating the initial length information based on the target length information.

[0014] Furthermore, the method also includes: when the storage space of the preset memory is less than a preset value, the data in the preset memory is written to the preset disk based on a preset hash rule.

[0015] Another aspect of this application provides a data reading method, the method being used to read data from a preset data storage area, the data being written to a preset memory and a preset disk based on the data writing method described above, the method comprising: In response to a data read request, determine the key of the data to be read; The data to be read is read from a preset memory based on the key of the data to be read, and a reading result is generated; If the read result is a read failure, the data to be read is read from the preset disk.

[0016] Another aspect of this application provides a data writing device, the device comprising: A response unit is used to determine the target data and the data type, target key, and data information of the target data in response to a data write request; The determining unit is used to obtain the data retrieval rules corresponding to the data type of the target data, and to perform a retrieval in a preset data storage area based on the data information of the target data to determine the retrieval results; The writing unit is used to obtain the hash mapping rule corresponding to the data type of the target data, and write the data information to the preset data storage area based on the retrieval result and the target key of the target data corresponding to the hash mapping rule.

[0017] In another aspect, this application provides an electronic device, the electronic device comprising: At least one processor; and A memory that is communicatively connected to the at least one processor; The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the method as described above.

[0018] Current Redis implementations based on LSM-trees write data to both memory and disk. During data retrieval, an ordered memtable is used for data storage, and binary search is employed, resulting in a time complexity of O(logN). For example, reading 1 million data entries requires approximately 20 memory accesses. Disk data retrieval is based on an ordered SST file, and a similar method is used, also with a time complexity of O(logN). For example, reading 1 million data entries requires approximately 23 disk accesses.

[0019] The proposed solution in this application is "based on a hash structure to write data into memory and disk, and during the data reading process, data is read from memory and disk based on the hash structure". The data in memory and disk is read, and both the meta data in memory and disk are stored using a hashmap structure. The time complexity of data reading is reduced to O(1), and theoretically, only one read is needed to locate the required data.

[0020] In summary, the data reading method proposed in this application has the following beneficial effects: On the one hand, it significantly reduces the number of data reads: compared to the LSM-tree scheme, which requires multiple comparisons for data reading (taking 1 million data as an example, the number of times data is read in memory is about 20 times and the number of times data is read on disk is about 23 times), the method proposed in this application uses the characteristics of hashmap to reduce the number of data reads to almost a constant one; this means faster response speed and higher efficiency.

[0021] On the other hand, it improves read and write performance: because the data reading time is greatly reduced, the system can complete tasks more quickly when performing data reading and writing operations, which is especially important for application scenarios that require frequent data access.

[0022] On the other hand, it simplifies the design: using a hashmap instead of a complex tree structure simplifies the design and maintenance of the system, reduces the possibility of errors, and reduces the need for specific algorithm optimizations. Attached Figure Description

[0023] To more clearly illustrate the technical solutions in the embodiments of this application, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0024] Other features, objects, and advantages of this application will become more apparent from the following detailed description of non-limiting embodiments with reference to the accompanying drawings: Figure 1 A schematic flowchart illustrating a data writing method provided in an embodiment of this application; Figure 2 This is a schematic diagram of the data storage architecture in the existing technology; Figure 3 This is a schematic diagram of the data storage architecture proposed in this application; Figure 4 This is a schematic flowchart of a data reading method proposed in an embodiment of this application; Figure 5 This is a schematic diagram of the structure of a data writing device provided in an embodiment of this application; Figure 6 This is a schematic diagram of the structure of a data reading device provided in an embodiment of this application; Figure 7 This is a schematic diagram of the structure of an electronic device suitable for implementing the solutions in the embodiments of this application; The same or similar reference numerals in the accompanying drawings represent the same or similar parts. Detailed Implementation

[0025] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0026] In a typical configuration of this application, the terminal and the service network devices each include one or more processors (CPUs), input / output interfaces, network interfaces, and memory.

[0027] Memory may include non-persistent storage in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.

[0028] Computer-readable media include permanent and non-permanent, removable and non-removable media, which can store information by any method or technology. Information can be computer program instructions, data structures, program modules, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, read-only optical disc (CD-ROM), digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other non-transfer medium that can be used to store information accessible by a computing device.

[0029] This application provides a data writing method, which includes the following steps: Step S101: In response to the data write request, determine the target data and the data type, target key, and data information of the target data; Step S102: Obtain the data retrieval rules corresponding to the data type of the target data, and perform a retrieval in a preset data storage area based on the data information of the target data to determine the retrieval results; Step S103: Obtain the hash mapping rule corresponding to the data type of the target data, and write the data information to the preset data storage area based on the search result and the target key of the target data corresponding to the hash mapping rule.

[0030] In practical scenarios, the execution entity of this method can be a user device, or a device formed by integrating a user device and a network device through a network, or it can be an application running on the aforementioned devices. The user device includes, but is not limited to, various terminal devices such as computers, mobile phones, tablets, smartwatches, and wristbands. The network device includes, but is not limited to, network hosts, single network servers, multiple network server sets, or cloud computing-based computer sets, and can be used to implement some processing functions when setting an alarm clock. Here, the cloud consists of a large number of hosts or network servers based on cloud computing. Cloud computing is a type of distributed computing, consisting of a virtual computer composed of a group of loosely coupled computer sets.

[0031] Example 1

[0032] Figure 1 The present application illustrates a data writing method according to an embodiment of the present application, the method comprising at least the following processing steps: Step S101: In response to the data write request, determine the target data and the data type, target key, and data information of the target data; Step S102: Obtain the data retrieval rules corresponding to the data type of the target data, and perform a retrieval in a preset data storage area based on the data information of the target data to determine the retrieval results; Step S103: Obtain the hash mapping rule corresponding to the data type of the target data, and write the data information to the preset data storage area based on the search result and the target key of the target data corresponding to the hash mapping rule.

[0033] Redis command usage reveals that almost all commands carry key information. This means that current LSM-tree-based storage solutions do not fully utilize the advantage of keys as precise query conditions, failing to further explore the potential for improving data read and write performance; especially for discretized data, existing data writing methods only increase the complexity of data writing and reduce its efficiency. Based on this, this application proposes abandoning the original ordered structure of LSM-tree and using a hash structure for data writing, reducing the time complexity of data writing and data reading from OlogN to O(1).

[0034] like Figure 2 The image shows a data storage architecture in the existing technology, such as... Figure 3 The data storage architecture proposed in this application is shown. Compared with the data writing structure in the prior art, the data storage architecture proposed in this application uses a hashmap in memory instead of a memtable in the prior art to achieve faster writing and reading of a single key using a hashmap. The data storage architecture proposed in this application uses a hash map instead of an SST in the prior art on the disk, so that the discretized data does not necessarily have to meet the condition of ordered writing or reading.

[0035] In one embodiment, the data type of the target data includes single-value data and multi-field composite data; the data information of the target data includes at least the target value and target metadata.

[0036] Specifically, single-value data includes string data (String type data), and the write command is setkeyvalue. For this type of data, you only need to look up the value by the key. Taking the hash structure as an example, for kkv composite data (such as hash / list / set / zset), the write command is hsetkeyfieldvalue. Each key can contain multiple fields, and each field corresponds to a value. The read command is: hgetallkey, which retrieves all field and value information under the key.

[0037] In one embodiment, the step of obtaining the data retrieval rule corresponding to the data type of the target data, and retrieving the data information of the target data in a preset data storage area to determine the retrieval result, includes: when the data type of the target data is single-value data, retrieving the target key of the target data in a preset memory to determine a first result, the first result including existing and non-existent; when the data type of the target data is multi-field composite and the first result is non-existent, retrieving the target key of the target data in a preset disk to determine a second result, the second result including existing and non-existent; wherein, the preset data storage area includes the preset memory and the preset disk.

[0038] In one embodiment, obtaining the hash mapping rule corresponding to the data type of the target data and writing the data information to the preset data storage area based on the retrieval result and the target key of the target data corresponding to the hash mapping rule includes: when the target data is single-value data and the first result does not exist, then writing the target data to the preset memory; when the target data is single-value data and the first result already exists, then obtaining the initial value and initial metadata corresponding to the target key, and overwriting the initial value and initial metadata based on the target value and target metadata corresponding to the target key.

[0039] Specifically, for single-value data, during the data writing process, the system first checks if the target data to be written exists in the hashmap. If it is determined that the target data exists in the hashmap, the value corresponding to the target data is modified. It's important to understand that checking the hashmap for the target data before writing can involve first checking if a matching key exists in the hashmap based on the target data's key (hereinafter defined as the target key). If the target key exists in the hashmap, the initial value corresponding to the target key is obtained, and then the initial value is overwritten according to the target value corresponding to the target key. This completes one data write operation.

[0040] In general, the data writing process for single-value data is as follows: In response to a data write request, the target data to be written to the preset data storage area is determined according to the data write request. The target data includes the target key and the target value; based on the target key, the data is retrieved from the hashmap to determine whether the target key already exists in the hashmap. If it is determined that the target key already exists in the hashmap, the initial value corresponding to the target key that exists in the hashmap is determined; the initial value is updated based on the target value.

[0041] The data structure of single-value data is relatively simple. For single-value data, the target value and expiration time are specified during the process of writing the data to the data storage space. Therefore, when it is determined that the target key already exists in the hashmap, the target key can be updated directly with the target value.

[0042] In one implementation, if the target key is not found in the hashmap, the search continues to check if the target key exists on the disk. In the data storage architecture, the storage capacity of memory is smaller than that of the disk. Therefore, when the memory is full, data is transferred from memory to the disk so that new data can be written to memory. Thus, if the target key is not found in memory, it is also necessary to check if it exists on the disk.

[0043] In one embodiment, obtaining the hash mapping rule corresponding to the data type of the target data, and writing the data information to the preset data storage area based on the retrieval result and the target key of the target data corresponding to the hash mapping rule, includes: when the target data is a multi-field composite and the first result already exists, determining whether the target data has expired; if the target data has expired, obtaining the initial value and initial metadata corresponding to the target key, and overwriting the initial value and initial metadata based on the target value and target metadata corresponding to the target key; if the target data has not expired, obtaining the current version number of the target data, generating a target version number based on the current version number, and storing the target value and target metadata in the preset memory based on the target version number corresponding to the target key of the target data; wherein, the target version number and the current version number are unique.

[0044] Specifically, when the target data to be written is a KKV composite data type, the hashmap is retrieved based on the target key, and the meta data of the KKV composite data is obtained from disk. Querying the hashmap by the target key involves attempting to find the target value matching the provided target key from the hash table (hashmap) in memory. A hash table is an efficient data structure that allows for fast access to the corresponding value via a key. When the data type is KKV composite data, the target key can be understood as the first or second level key in the KKV data structure; the hashmap is where the key-value pairs are stored, typically located in memory for fast access.

[0045] Specifically, retrieving meta data of the kkv composite data from disk based on the target key of the target data means that in addition to checking the hashmap in memory, it is necessary to further retrieve the meta data on disk to obtain more data information corresponding to the target key. The meta data may contain the following information: version number: identifying different versions of the data and helping to track update history; creation time / modification time: recording when the data was created or last modified; expiration time: specifying the validity period of the data; status: such as whether it has been marked for deletion, etc.

[0046] In one implementation, when the data type is kkv composite data, if the target key cannot be retrieved in the hashmap in memory, or if it is necessary to verify the validity of the data (e.g., to confirm whether the data related to the target key has expired), then a deeper search is required based on the meta data stored on disk.

[0047] When the target data is KKV data, data retrieval is performed simultaneously or sequentially based on memory and disk. This approach ensures data consistency and integrity: relying solely on memory data may not guarantee that all operations are based on the latest and most valid data state. By checking meta data on disk, important information can be avoided from being overwritten or outdated data used. Furthermore, it handles complex data updates and management: complex KKV structures may involve multiple write and modification operations. In such cases, meta data helps determine whether the current operation should be based on a specific version of the data, thus preventing data loss or inconsistency issues caused by concurrent operations. Finally, it supports persistent storage: while memory offers the advantage of fast access, it is not persistent. Data in memory will be lost in the event of a system crash or restart. Meta data on disk ensures that even in such a situation, the data can be restored to the most recent state. This approach combines the speed advantage of memory with the persistence capabilities of disk, ensuring efficient and reliable data management.

[0048] In one embodiment, obtaining the hash mapping rule corresponding to the data type of the target data, and writing the data information to the preset data storage area based on the retrieval result and the target key of the target data corresponding to the hash mapping rule, includes: when the target data is a multi-field composite and the first result does not exist, obtaining a second result; when the second result does not exist, obtaining all version numbers in the preset data storage area, the version numbers including the current version number and historical version numbers; generating a target version number based on the current version number and the historical version number, and storing the target value and the target metadata to the preset memory based on the target key of the target data corresponding to the target version number.

[0049] In this embodiment, the current version number includes the version number currently existing in the data storage structure, and the historical version number includes the version number that once existed in the data storage structure.

[0050] Specifically, generating a target version number based on the current version number and historical version numbers includes: generating a target version number that is different from the current version number and historical version numbers; that is, the target version number is the same as any current version number and historical version number.

[0051] In one embodiment, the method further includes: obtaining initial length information of the target data; generating target length information based on the target data; and updating the initial length information based on the target length information.

[0052] Specifically, the initial length information can be the number of bytes before the target data is written, and the target length information can be the number of bytes after the target data is written.

[0053] In one embodiment, the method further includes: when the storage space of the preset memory is less than a preset value, writing the data in the preset memory to the preset disk based on a preset hash rule.

[0054] In one embodiment, the method further includes: encoding the target data to be written to generate a target code; and matching the target data to a target region based on the target code.

[0055] Specifically, the encoding of String type data is relatively simple: first, the target key corresponding to the target data is converted into a unique slotid using a hash algorithm, and then the slotid is precisely matched to the target shard or slot to determine the location where the target data should be written; therefore, the encoding of String only needs to complete the mapping from key to slotId.

[0056] For multi-field composite data, taking Hash type data as an example, firstly, the hash data key is encoded as slotId to determine the shard to which it belongs; then, a new, globally unique version (64-bit unsigned integer) is assigned to the key of the corresponding hash data; then, this version is concatenated with the field in the hash to form a new key; finally, the value corresponding to this new key is the actual value specified in the user command.

[0057] By encoding different data types, not only is precise location of data achieved, but single-value data and multi-field composite data are further separated; thereby improving the maintainability, consistency and scalability of the system, it is especially suitable for high-concurrency, multi-version, and distributed data management scenarios.

[0058] Example 2

[0059] Figure 4 This application illustrates a data reading method according to an embodiment of the present application. The method is used to read data from a preset data storage area. The data is written to preset memory and a preset disk using the data writing method described in Embodiment 1. The method includes: Step S201: In response to the data read request, determine the key of the data to be read; Step S202: Read the data to be read from a preset memory based on the key of the data to be read, and generate a reading result; Step S203: When the reading result is a reading failure, the data to be read is read from the preset disk.

[0060] Example 3

[0061] Figure 5 This application illustrates a data writing apparatus according to an embodiment of the present application, the apparatus comprising: A response unit is used to determine the target data and the data type, target key, and data information of the target data in response to a data write request; The determining unit is used to obtain the data retrieval rules corresponding to the data type of the target data, and to perform a retrieval in a preset data storage area based on the data information of the target data to determine the retrieval results; The writing unit is used to obtain the hash mapping rule corresponding to the data type of the target data, and write the data information to the preset data storage area based on the retrieval result and the target key of the target data corresponding to the hash mapping rule.

[0062] Example 4

[0063] Figure 6According to an embodiment of this application, a data reading device is provided, the device comprising: A response unit is used to determine the key of the data to be read in response to a data read request; The generation unit reads the data to be read from a preset memory based on the key of the data to be read, and generates a reading result; The reading unit reads the data to be read from the preset disk when the reading result is a reading failure.

[0064] Example 5

[0065] Based on the same inventive concept, this application also provides an electronic device. The method corresponding to the electronic device can be the data writing method and data reading method in the foregoing embodiments, and its problem-solving principle is similar to that method. The electronic device provided in this application includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to execute the methods and / or technical solutions of the foregoing embodiments of this application.

[0066] The electronic device can be a user device, or a device formed by integrating user devices and network devices through a network, or it can be an application running on the aforementioned devices. The user device includes, but is not limited to, various terminal devices such as computers, mobile phones, tablets, smartwatches, and wristbands. The network device includes, but is not limited to, network hosts, single network servers, multiple network server sets, or cloud computing-based computer sets, and can be used to implement some processing functions when setting an alarm clock. Here, the cloud consists of a large number of hosts or network servers based on cloud computing. Cloud computing is a type of distributed computing, consisting of a virtual computer composed of a group of loosely coupled computer sets.

[0067] Figure 7The diagram illustrates the structure of an electronic device suitable for implementing the methods and / or technical solutions in the embodiments of this application. The device 500 includes a Central Processing Unit (CPU) 501, which can perform various appropriate actions and processes based on a program stored in a Read-Only Memory (ROM) 502 or a program loaded from a storage portion 508 into a Random Access Memory (RAM) 503. The RAM 503 also stores various programs and data required for system operation. The CPU 501, ROM 502, and RAM 503 are interconnected via a bus 504. An Input / Output (I / O) interface 505 is also connected to the bus 504.

[0068] The following components are connected to I / O interface 505: an input section 506 including a keyboard, mouse, touchscreen, microphone, infrared sensor, etc.; an output section 507 including a cathode ray tube (CRT), liquid crystal display (LCD), LED display, OLED display, etc., and speakers, etc.; a storage section 508 including one or more computer-readable media such as hard disk, optical disk, magnetic disk, semiconductor memory, etc.; and a communication section 509 including a network interface card such as a LAN (Local Area Network) card, modem, etc. The communication section 509 performs communication processing via a network such as the Internet.

[0069] In particular, the methods and / or embodiments in this application can be implemented as computer software programs. For example, the embodiments disclosed in this application include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowchart. When the computer program is executed by the central processing unit (CPU) 501, it performs the functions defined in the methods of this application.

[0070] Example 6

[0071] Another embodiment of this application provides a computer-readable storage medium having computer program instructions stored thereon, which can be executed by a processor to implement the methods and / or technical solutions of any one or more embodiments of this application described above.

[0072] Specifically, this embodiment may employ any combination of one or more computer-readable media. A computer-readable medium may be a computer-readable signal medium or a computer-readable storage medium. A computer-readable storage medium may be, for example—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or any combination thereof. More specific examples of computer-readable storage media (a non-exhaustive list) include: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this document, a computer-readable storage medium may be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, device, or apparatus.

[0073] Computer-readable signal media may include data signals propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals may take various forms, including—but not limited to—electromagnetic signals, optical signals, or any suitable combination thereof. Computer-readable signal media may also be any computer-readable medium other than computer-readable storage media, capable of sending, propagating, or transmitting programs for use by or in connection with an instruction execution system, device, or apparatus.

[0074] The program code contained on a computer-readable medium may be transmitted using any suitable medium, including—but not limited to—wireless, wire, optical fiber, RF, etc., or any suitable combination thereof.

[0075] Computer program code for performing the operations of this application can be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, and C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0076] The flowcharts or block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of devices, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-specific system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0077] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0078] In the several embodiments provided in this application, it should be understood that the disclosed systems, devices, and methods can be implemented in other ways. For example, the device embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or page components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the displayed or discussed mutual couplings or direct couplings or communication connections may be through some interfaces, or indirect couplings or communication connections between devices or units, and may be electrical, mechanical, or other forms.

[0079] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0080] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or in a combination of hardware and software functional units.

[0081] The integrated units implemented as software functional units described above can be stored in a computer-readable storage medium. These software functional units, stored in a storage medium, include several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) or processor to execute some steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0082] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application.

[0083] Furthermore, it is clear that the word "comprising" does not exclude other units or steps, and the singular does not exclude the plural. Multiple units or devices recited in a device claim may also be implemented by a single unit or device through software or hardware. The terms "first," "second," etc., are used to indicate names and do not indicate any specific order.

Claims

1. A data writing method, characterized in that, The method includes: In response to a data write request, determine the target data and the data type, target key, and data information of the target data; Obtain the data retrieval rules corresponding to the data type of the target data, perform a retrieval in a preset data storage area based on the data information of the target data, and determine the retrieval results; Obtain the hash mapping rule corresponding to the data type of the target data, and write the data information to the preset data storage area based on the search result and the target key of the target data corresponding to the hash mapping rule.

2. The data writing method according to claim 1, characterized in that, The target data includes single-value data and multi-field composite data. The target data information includes at least the target value and target metadata.

3. The data writing method according to claim 2, characterized in that, The process of obtaining the data retrieval rules corresponding to the data type of the target data, and retrieving the data from a preset data storage area based on the data information of the target data to determine the retrieval results includes: When the data type of the target data is single-value data, a search is performed in a preset memory based on the target key of the target data to determine a first result, which includes existing and non-existent results; When the data type of the target data is a multi-field composite and the first result is non-existent, a search is performed on a preset disk based on the target key of the target data to determine the second result, which includes both existing and non-existent results; The preset data storage area includes the preset memory and the preset disk.

4. The data writing method according to claim 3, characterized in that, The step of obtaining the hash mapping rule corresponding to the data type of the target data, and writing the data information to the preset data storage area based on the retrieval result and the target key of the target data corresponding to the hash mapping rule, includes: When the target data is a single-value data and the first result does not exist, the target data is written to the preset memory. When the target data is single-value data and the first result already exists, the initial value and initial metadata corresponding to the target key are obtained, and the initial value and initial metadata are overwritten based on the target value and target metadata corresponding to the target key.

5. The data writing method according to claim 3, characterized in that, The step of obtaining the hash mapping rule corresponding to the data type of the target data, and writing the data information to the preset data storage area based on the retrieval result and the target key of the target data corresponding to the hash mapping rule, includes: When the target data is a composite of multiple fields and the first result indicates that it already exists, it is determined whether the target data has expired. If the target data has expired, then obtain the initial value and initial metadata corresponding to the target key, and overwrite the initial value and initial metadata based on the target value and target metadata corresponding to the target key; If the target data has not expired, the current version number of the target data is obtained, a target version number is generated based on the current version number, and the target value and the target metadata are stored in the preset memory based on the target key of the target data corresponding to the target version number. The target version number is unique to the current version number.

6. The data writing method according to claim 5, characterized in that, The step of obtaining the hash mapping rule corresponding to the data type of the target data, and writing the data information to the preset data storage area based on the retrieval result and the target key of the target data corresponding to the hash mapping rule, includes: When the target data is a composite of multiple fields and the first result does not exist, then the second result is obtained; If the second result is not found, then all version numbers in the preset data storage area are obtained, including the current version number and historical version numbers; A target version number is generated based on the current version number and the historical version number. The target value and the target metadata are stored in a preset memory based on the target key of the target data corresponding to the target version number.

7. The data writing method according to claim 6, characterized in that, The method further includes: Obtain the initial length information of the target data; Generate target length information based on the target data; The initial length information is updated based on the target length information; And / or, the method further includes: When the storage space of the preset memory is less than the preset value, the data in the preset memory is written to the preset disk based on the preset hash rule.

8. A data reading method, characterized in that, The method is used to read data from a preset data storage area, the data being written to a preset memory and a preset disk based on the data writing method according to any one of claims 1 to 7, the method comprising: In response to a data read request, determine the key of the data to be read; The data to be read is read from a preset memory based on the key of the data to be read, and a reading result is generated; If the read result is a read failure, the data to be read is read from the preset disk.

9. A data writing device, characterized in that, The device includes: A response unit is used to determine the target data and the data type, target key, and data information of the target data in response to a data write request; The determining unit is used to obtain the data retrieval rules corresponding to the data type of the target data, and to perform a retrieval in a preset data storage area based on the data information of the target data to determine the retrieval results; The writing unit is used to obtain the hash mapping rule corresponding to the data type of the target data, and write the data information to the preset data storage area based on the retrieval result and the target key of the target data corresponding to the hash mapping rule.

10. An electronic device, the electronic device comprising: At least one processor; as well as A memory communicatively connected to the at least one processor; characterized in that, The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the method of any one of claims 1 to 8.

Images