A data processing method and apparatus

Abstract

The application provides a data processing method and device. The method comprises: receiving a data write request; performing serialization processing on to-be-written data in response to the data write request; writing the serialized to-be-written data into a non-volatile memory, the non-volatile memory being accessible by a byte unit; obtaining a storage address of a key-value pair in the serialized to-be-written data in the non-volatile memory; and inserting the key in the key-value pair and the storage address into a multi-way tree structure in the non-volatile memory. The data processing method and device provided by the embodiment of the application can improve the data write performance.

Description

TECHNICAL FIELD

[0001] The present application relates to artificial intelligence technology and big data technology, and in particular to a data processing method and device. BACKGROUND

[0002] With the development of big data technology, the demand for data storage is also increasing. Further optimizing the write performance of data is the goal that data storage technology has always pursued. SUMMARY

[0003] The embodiments of the present application provide a data processing method and device, which can improve the write performance of data.

[0004] The technical scheme of the embodiments of the present application is implemented as follows:

[0005] The embodiments of the present application provide a data processing method, comprising: receiving a data write request;

[0006] In response to the data write request, performing serialization processing on the data to be written;

[0007] Writing the serialized data to be written into a non-volatile memory, the non-volatile memory being accessible by byte units;

[0008] Obtaining the storage address of the key-value pair in the non-volatile memory;

[0009] Inserting the key in the key-value pair and the storage address into the multi-way tree structure in the non-volatile memory.

[0010] In some embodiments, the method further comprises: in response to the data update request, updating the first key-value pair in the non-volatile memory.

[0011] In some embodiments, the method further comprises:

[0012] Searching for the information corresponding to the first key-value pair in the multi-way tree structure;

[0013] If the information corresponding to the first key-value pair exists in the multi-way tree structure, obtaining the first key in the updated first key-value pair and the first storage address of the updated first key-value pair in the non-volatile memory;

[0014] Updating the information corresponding to the first key-value pair in the multi-way tree structure to the first key and the first storage address.

[0015] In some embodiments, the method further comprises: deleting the information corresponding to the first key-value pair before updating in the multi-way tree structure.

[0016] In some embodiments, the method further comprises: if the first key-value pair does not exist in the multi-way tree structure, obtaining an updated first key in the first key-value pair and a first storage address of the updated first key-value pair in the non-volatile memory;

[0017] adding the first key and the first storage address to the multi-way tree structure.

[0018] In some embodiments, the writing the serialized to-be-written data into the non-volatile memory comprises:

[0019] obtaining a storage offset of the serialized to-be-written data in the non-volatile memory;

[0020] writing the serialized to-be-written data into the non-volatile memory based on the storage offset.

[0021] In some embodiments, the writing the serialized to-be-written data into the non-volatile memory based on the storage offset comprises:

[0022] determining a second storage location of the serialized to-be-written data in the non-volatile memory based on the storage offset and a size of an identifier corresponding to the serialized to-be-written data;

[0023] writing the serialized to-be-written data into the second storage location and writing the identifier into a third storage location corresponding to the storage offset.

[0024] Embodiments of the present application provide a data processing apparatus, which comprises:

[0025] a receiving module configured to receive a data write request;

[0026] a processing module configured to perform serialization processing on to-be-written data in response to the data write request;

[0027] a writing module configured to write the serialized to-be-written data into a non-volatile memory, wherein the non-volatile memory can be accessed by byte units;

[0028] an obtaining module configured to obtain a storage address of a key-value pair in the serialized to-be-written data in the non-volatile memory;

[0029] an inserting module configured to insert a key in the key-value pair and the storage address into a multi-way tree structure in the non-volatile memory.

[0030] In some embodiments, the receiving module is further configured to receive a data update request;

[0031] The write module is further configured to update the first key-value pair in the nonvolatile memory in response to the data update request.

[0032] In some embodiments, the obtaining module is further configured to search for the information corresponding to the first key-value pair in the multiway tree structure, and if the information corresponding to the first key-value pair exists in the multiway tree structure, obtain a first key in the updated first key-value pair and a first storage address of the updated first key-value pair in the nonvolatile memory.

[0033] The insertion module is further configured to update the information corresponding to the first key-value pair in the multiway tree structure as the first key and the first storage address.

[0034] In some embodiments, the processing module is further configured to delete the information corresponding to the first key-value pair before the update in the multiway tree structure.

[0035] In some embodiments, the obtaining module is further configured to, if the information corresponding to the first key-value pair does not exist in the multiway tree structure, obtain a first key in the updated first key-value pair and a first storage address of the updated first key-value pair in the nonvolatile memory.

[0036] The insertion module is further configured to add the first key and the first storage address to the multiway tree structure.

[0037] In some embodiments, the write module is configured to obtain a storage offset of the serialized data to be written in the nonvolatile memory.

[0038] The serialized data to be written is written into the nonvolatile memory based on the storage offset.

[0039] In some embodiments, the write module is configured to determine a second storage position of the serialized data to be written in the nonvolatile memory based on the storage offset and a size of an identifier corresponding to the serialized data to be written.

[0040] The serialized data to be written is written into the second storage position, and the identifier is written into a third storage position corresponding to the storage offset.

[0041] Embodiments of the present application provide an electronic device, comprising:

[0042] A memory configured to store executable instructions.

[0043] A processor configured to execute the executable instructions stored in the memory to implement the data processing method provided by the embodiments of the present application.

[0044] The embodiment of the present application provides a computer readable storage medium, which stores executable instructions, and when the executable instructions are executed by a processor, the data processing method provided by the embodiment of the present application is implemented.

[0045] The data processing method provided by the embodiment of the present application comprises the following steps: receiving a data write request; performing serialization processing on to-be-written data in response to the data write request; writing the to-be-written data after serialization processing into a nonvolatile memory, the nonvolatile memory being accessible by a byte unit; obtaining a storage address of a key-value pair in the to-be-written data after serialization processing in the nonvolatile memory; and inserting the key in the key-value pair and the storage address into a multi-way tree structure in the nonvolatile memory. In the embodiment of the present application, the key in the key-value pair and the storage address of the key-value pair in the nonvolatile memory are inserted into the multi-way tree structure, and the storage address of the key-value pair in the nonvolatile memory is taken as a value on the multi-way tree structure, thereby avoiding the problem of write amplification caused by inserting the key-value pair into the multi-way tree structure, and optimizing the performance of writing data. BRIEF DESCRIPTION OF DRAWINGS

[0046] Figure 1 is a schematic diagram of writing data in a nonvolatile memory in the related art;

[0047] Figure 2 is a schematic diagram of updating data in a nonvolatile memory in the related art;

[0048] Figure 3 is a schematic diagram of an optional flow of the data processing method provided by the embodiment of the present application;

[0049] Figure 4 is a schematic diagram of writing to-be-written data in the form of a LOG log into a nonvolatile memory by the embodiment of the present application;

[0050] Figure 5 is a schematic diagram of writing data into a nonvolatile memory based on the data processing method provided by the embodiment of the present application;

[0051] Figure 6 is another optional processing flow schematic diagram of the data processing method provided by the embodiment of the present application;

[0052] Figure 7 is a schematic diagram of updating data in a nonvolatile memory provided by the embodiment of the present application;

[0053] Figure 8 is an optional component structure schematic diagram of the data processing apparatus provided by the embodiment of the present application;

[0054] Figure 9 is a hardware component structure schematic diagram of the electronic device provided by the embodiment of the present application. DETAILED DESCRIPTION

[0055] In order to make the purposes, technical solutions and advantages of the present application clearer, the present application will be further described in detail below with the accompanying drawings. The described embodiments should not be regarded as limitations to the present application. All other embodiments obtained by those of ordinary skill in the art without creative work fall within the scope of protection of the present application.

[0056] In the following description, "some embodiments" are described, which describe a subset of all possible embodiments, but it can be understood that "some embodiments" can be the same subset or different subsets of all possible embodiments, and can be combined with each other without conflict.

[0057] In the following description, the terms "first\second\third" are only to distinguish similar objects, and do not represent a specific order of the objects. It can be understood that "first\second\third" can be interchanged in a specific order or sequence as allowed, so that the embodiments of the present application described herein can be implemented in an order other than that illustrated or described herein. In the following description, the term "a plurality of" refers to at least two.

[0058] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which the present application belongs. The terms used herein are only for the purpose of describing the embodiments of the present application and are not intended to limit the present application.

[0059] In the related art, a schematic diagram of writing data in a non-volatile memory is shown in FIG. 1. The process of writing data can include two stages. The first stage is to serialize the data to be written and then write it into the non-volatile memory. Specifically, the key-value pairs in the data to be written are serialized into binary logs and written into the non-volatile memory. As shown in FIG. 2, the key-value pairs A:B, C:D, and E:F are written into the non-volatile memory in the first stage. The second stage is to insert the key-value pairs into the multi-way tree structure of the non-volatile memory. Since the non-volatile memory can only guarantee atomic writing of 8 bytes, when inserting the key-value pairs into the multi-way tree structure, it will cause the nodes in the multi-way tree structure to move. The content that causes the nodes in the multi-way tree structure to move needs to be guaranteed by the transaction writing mechanism to ensure the safety of the data. Figure 1 Figure 1 Figure 1 ​​As shown, three key-value pairs are inserted into different nodes in the multi-way tree structure respectively. However, due to the atomic write feature of the non-volatile memory, the Key-Value is inserted into the multi-way tree structure of the non-volatile memory again in the second stage, and the Key-Value is executed again for a data write process, which will cause the problem of write amplification and reduce the performance of writing data. The schematic diagram of updating data in the non-volatile memory in the related art is as shown in Figure 2 As shown, the Key-Value updated again in the second stage needs to be inserted into the multi-way tree structure of the non-volatile memory, for example, the key-value pair E:F is updated to E':F', and E:F needs to be updated to E':F' in the multi-way tree structure. Since the Key-Value needs to be stored in the first stage and the second stage, the problem of data write amplification is caused, and the performance of writing data is reduced.

[0060] The data processing method provided in the embodiments of the present application can at least solve the above problems.

[0061] The data processing method provided in the embodiments of the present application will be described below in combination with an exemplary application and implementation of an electronic device provided by the embodiments of the present application.

[0062] Referring to Figure 3 , Figure 3 is a flowchart of the data processing method provided in the embodiments of the present application, which will be described in combination with the steps shown in Figure 3 .

[0063] In step S101, a data write request is received.

[0064] In some embodiments, the data write request is used to request to write the to-be-written data into the non-volatile memory. The non-volatile memory can be accessed by byte units, or one or more bytes of data are stored in one unit of the non-volatile memory. As an example, the non-volatile memory can be an Optane memory.

[0065] In step S102, in response to the data write request, a serialization process is performed on the to-be-written data.

[0066] In some embodiments, the serialization process on the to-be-written data can mean that a binary serialization process is performed on a string corresponding to the to-be-written data, and the to-be-written data after the serialization process can include two parts of a data identifier and a data body. The size of the data identifier can be 8 bytes, and the data identifier is used to identify the data body. The data body can be in the form of a key-value pair, and the key-value pair includes a key and a value.

[0067] In some embodiments, when performing the serialization processing on the to-be-written data, the start position of the data body can be aligned. Specifically, the start position of the data body can be aligned according to the size of the data identifier; if the size of the data identifier is 8 bytes, the start position of the data body is 8-byte aligned.

[0068] In step S103, the serialized to-be-written data is written into the non-volatile memory.

[0069] In some embodiments, the serialized to-be-written data is written into the non-volatile memory in the form of a LOG log, or the serialized to-be-written data is stored into a LOG file.

[0070] In some embodiments, the specific implementation process of writing the serialized to-be-written data into the non-volatile memory can be: obtaining a storage offset of writing the serialized to-be-written data into the non-volatile memory; and writing the serialized to-be-written data into the non-volatile memory based on the storage offset.

[0071] The storage offset can refer to the offset of the to-be-written data relative to the start position of the LOG file, or can refer to the position of the to-be-written data in the LOG file.

[0072] The specific implementation process of writing the serialized to-be-written data into the non-volatile memory based on the storage offset can be: determining a second storage position of the serialized to-be-written data in the non-volatile memory based on the storage offset and the size of the identifier corresponding to the serialized to-be-written data; writing the serialized to-be-written data into the second storage position, and writing the identifier into a third storage position corresponding to the storage offset. Specifically, first, the data identifier corresponding to the to-be-written data (data body) is stored in the non-volatile memory, and the to-be-written data is stored in the storage position after the data identifier. As an example, if the storage offset is the i th bit in the LOG file, and the size of the data identifier is 8 bytes, the second storage position is the (i+64) th bit. The third storage position is the i th bit in the LOG file. The schematic diagram of writing the to-be-written data into the non-volatile memory in the form of a LOG log is shown in Figure 4 The key-value pair corresponding to the to-be-written data is {Key: 1000, Value: XXXXX}, and the data identifier corresponding to the to-be-written data is 007, so the key-value pair is written in the second storage position in the form of a LOG log, and the data identifier is written in the third storage position.

[0073] In some embodiments, the serialized to-be-written data and the data identifier can be written in a non-transactional manner; that is, the writing position of the data identifier in the LOG file is before the writing position of the to-be-written data in the LOG file. If a data writing feedback message is received, it is considered that the data written in the non-volatile memory has been persisted, and if the read data identifier is correct, the data corresponding to the data identifier is also valid.

[0074] In some embodiments, the process of writing the serialized to-be-written data into the non-volatile memory is the first phase of writing data.

[0075] In step S104, the storage address of the key-value pair in the serialized to-be-written data in the non-volatile memory is obtained.

[0076] In some embodiments, after the serialized to-be-written data is written into the non-volatile memory, the key-value pair in the to-be-written data corresponds to a storage address of the non-volatile memory. Through the storage address, the key-value pair stored in the non-volatile memory in the form of a LOG log can be indexed.

[0077] In step S105, the key in the key-value pair and the storage address are inserted into the multi-way tree structure in the non-volatile memory.

[0078] In some embodiments, the Key in the key-value pair and the storage address of the Value in the key-value pair in the non-volatile memory are inserted into the multi-way tree structure in the non-volatile memory.

[0079] In some embodiments, the multi-way tree structure can be a btree.

[0080] In some embodiments, the process of inserting the key in the key-value pair and the storage address into the multi-way tree structure in the non-volatile memory is the second phase of writing data. Since the size of the storage space occupied by the storage address is much smaller than the size of the storage space occupied by the Value in the key-value pair, in the embodiments of the present application, by inserting the Key and the storage address of the Value in the non-volatile memory into the multi-way tree structure, the problem of write amplification caused by inserting Key-Value into the multi-way tree structure can be avoided, and the write performance of data can be improved.

[0081] The schematic diagram of writing data into the non-volatile memory based on the data processing method provided in the embodiments of the present application is as follows: Figure 5As shown, the data includes first data and second data, wherein the data identifier of the first data is "005", the size of the data identifier is 8 bytes, the data body of the first data is "99 / ZZ", wherein "99" is the Key in the key-value pair, "ZZ" is the Value in the key-value pair, and the position of ZZ in the LOG file in the non-volatile memory is represented by ZZ_addr. The data identifier of the second data is "006", the size of the data identifier is 8 bytes, the data body of the second data is "1000 / XX", wherein "1000" is the Key in the key-value pair, "XX" is the Value in the key-value pair, and the position of XX in the LOG file in the non-volatile memory is represented by XX_addr. The key in the data body and the address of the value in the key-value pair in the LOG file are stored in the multi-way tree structure of the non-volatile memory. Therefore, for the first data, the Key = 99 and the position "ZZ_addr" of the Value in the LOG file are stored on one node of the multi-way tree structure. For the first data, the Key = 1000 and the position "XX_addr" of the Value in the LOG file are stored on another node of the multi-way tree structure. In the multi-way tree structure, the historically stored Key = 1 and the address "YY_addr" of the Value in the LOG file are also included.

[0082] Based on Figure 3 As shown in the data processing method, the data to be written can be written into the non-volatile memory. In some embodiments, if the data written into the non-volatile memory is updated, another optional processing flow diagram of the data processing method provided by the embodiments of the present application is as shown in Figure 6 As shown, the data processing method can include at least the following steps:

[0083] In step S201, in response to a data update request, a first key-value pair is updated in the non-volatile memory.

[0084] In some embodiments, the first key-value pair is stored in the form of a LOG log in the non-volatile memory, and the Key corresponding to the first key-value pair and the position of the Value corresponding to the first key-value pair in the LOG log are inserted into the multi-way tree structure. If a data update request for the first key-value pair is received, the first key-value pair stored in the form of a LOG log in the non-volatile memory is updated.

[0085] In step S202, information corresponding to the first key-value pair is searched in the multi-way tree structure.

[0086] In some embodiments, the specific process of searching for the information corresponding to the first key-value pair in the multiway tree structure can be searching for a data identifier corresponding to the first key-value pair in the multiway tree structure, and if the data identifier corresponding to the first key-value pair is found in the multiway tree structure, it is confirmed that the multiway tree structure includes the information corresponding to the first key-value pair; the node in which the data identifier is stored stores the information corresponding to the first key-value pair. If the data identifier corresponding to the first key-value pair is not found in the multiway tree structure, it is confirmed that the multiway tree structure does not include the information corresponding to the first key-value pair.

[0087] In step S203, if the multiway tree structure includes the information corresponding to the first key-value pair, the first key in the updated first key-value pair and the first storage address of the updated first key-value pair in the non-volatile memory are obtained; and the information corresponding to the first key-value pair in the multiway tree structure is updated to the first key and the first storage address.

[0088] In some embodiments, if the multiway tree structure includes the information corresponding to the first key-value pair, the Key in the updated first key-value pair stored in the form of a LOG log and the position (first storage position) of the Value in the updated first key-value pair in the LOG log are searched for in the non-volatile memory. The information corresponding to the first key-value pair before updating in the multiway tree structure is modified to the information corresponding to the updated first key-value pair; for example, the information corresponding to the first key-value pair in the multiway tree structure is modified to the Key in the updated first key-value pair and the first storage address.

[0089] Since the size of the storage space occupied by the storage address is much smaller than the size of the storage space occupied by the Value in the key-value pair, in the embodiments of the present application, by updating the Key and the storage address of the Value in the non-volatile memory in the multiway tree structure, the problem of write amplification caused by re-inserting the Key-Value in the multiway tree structure can be avoided, and the write performance of data can be improved.

[0090] In some embodiments, after the information corresponding to the first key-value pair in the multiway tree structure is updated to the first key and the storage address, the method can further include deleting the information corresponding to the first key-value pair before updating in the multiway tree structure. Specifically, the Key in the first key-value pair before updating and the address of the Value in the first key-value pair before updating in the LOG log are deleted in the multiway tree structure.

[0091] As shown in the data in Figure 5 As an example of the data shown in Figure 7As shown, if the key-value pair "99 / ZZ" stored in the form of a LOG log in the non-volatile memory is updated to "99' / ZZ'", the position of the Value in the LOG log is updated from YY to YY'. In the multi-way tree structure, the information corresponding to the key-value pair "99 / ZZ" is updated to "99' / ZZ'_Addr". Thus, when the Key-Value is updated one or more times, no transactional write of data is needed in the multi-way tree structure, and only the address of the Value in the LOG file needs to be atomically updated, thereby improving the performance of inserting data in the multi-way tree structure.

[0092] In step S204, if the information corresponding to the first key-value pair does not exist in the multi-way tree structure, the first key in the updated first key-value pair and the first storage address of the updated first key-value pair in the non-volatile memory are obtained; and the first key and the first storage address are added to the multi-way tree structure.

[0093] In some embodiments, if the multi-way tree structure includes the information corresponding to the first key-value pair, the Key in the updated first key-value pair stored in the form of a LOG log in the non-volatile memory and the position (first storage position) of the Value in the updated key-value pair in the LOG log are found. The Key in the updated first key-value pair and the first storage position are written in a node in the multi-way tree structure.

[0094] In the embodiments of the present application, when data is written to the non-volatile memory, an atomic write mode can be used, that is, the data written to the non-volatile memory is all written or all not written.

[0095] In the embodiments of the present application, when data is stored, the corresponding Key-Value of the data is first stored in the non-volatile memory in the form of a LOG log; and then the Key and the storage position of the Value in the LOG log are inserted in the multi-way tree structure. Correspondingly, when the data is updated, the Key-Value stored in the form of a LOG log in the non-volatile memory is first updated; and then the updated Key and the storage position of the updated Value in the LOG log are modified in the multi-way tree structure. In this way, by storing the Key and the storage position of the Value in the LOG log in the multi-way tree structure, the write amplification problem caused by storing Key-Value in the multi-way tree structure can be avoided, and the write amplification problem caused by modifying Key-Value in the multi-way tree structure when the data in the LOG log is updated can also be avoided; and the write performance of the data is optimized.

[0096] The embodiments of the present application also provide a data processing apparatus, and an optional component structure diagram of the data processing apparatus is shown in Figure 8 As shown, the data processing apparatus comprises:

[0097] The receiving module 301 is configured to receive a data write request.

[0098] The processing module 302 is configured to perform serialization processing on the data to be written in response to the data write request.

[0099] The writing module 303 is configured to write the data to be written after serialization processing into a non-volatile memory, wherein the non-volatile memory can be accessed by byte units.

[0100] The obtaining module 304 is configured to obtain a storage address of a key-value pair in the data to be written after serialization processing in the non-volatile memory.

[0101] The inserting module 305 is configured to insert the key in the key-value pair and the storage address into a multi-way tree structure in the non-volatile memory.

[0102] In some embodiments, the receiving module 301 is further configured to receive a data update request.

[0103] The writing module 303 is further configured to update a first key-value pair in the non-volatile memory in response to the data update request.

[0104] In some embodiments, the obtaining module 304 is further configured to search for information corresponding to the first key-value pair in the multi-way tree structure, and if the information corresponding to the first key-value pair exists in the multi-way tree structure, obtain a first key in the first key-value pair after update and a first storage address of the first key-value pair after update in the non-volatile memory.

[0105] The inserting module 305 is further configured to update the information corresponding to the first key-value pair in the multi-way tree structure as the first key and the first storage address.

[0106] In some embodiments, the processing module 302 is further configured to delete information corresponding to the first key-value pair before update in the multi-way tree structure.

[0107] In some embodiments, the obtaining module 304 is further configured to, if the information corresponding to the first key-value pair does not exist in the multi-way tree structure, obtain a first key in the first key-value pair after update and a first storage address of the first key-value pair after update in the non-volatile memory.

[0108] The inserting module 305 is further configured to add the first key and the first storage address to the multi-way tree structure.

[0109] In some embodiments, the writing module 303 is configured to obtain a storage offset of the data to be written after serialization processing for writing into the non-volatile memory.

[0110] write the serialized to-be-written data into the non-volatile memory based on the storage offset.

[0111] In some embodiments, the writing module 303 is configured to determine a second storage location of the serialized to-be-written data in the non-volatile memory based on the storage offset and a size of an identifier corresponding to the serialized to-be-written data.

[0112] write the serialized to-be-written data into the second storage location and write the identifier into a third storage location corresponding to the storage offset.

[0113] In some embodiments, the data processing method provided by the embodiments of the present application can be implemented by a terminal device or a server. Specifically, the terminal device or the server can implement the data processing method provided by the embodiments of the present application by running a computer program. For example, the computer program can be a native program or a software module in an operating system; can be a native application program (APP), that is, a program that needs to be installed in an operating system to run; can be a small program, that is, a program that only needs to be downloaded into a browser environment to run; and can also be a small program that can be embedded into any APP. In summary, the above computer program can be any form of application program, module or plug-in.

[0114] In some embodiments, the server can be a standalone physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server providing cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDNs, and basic cloud computing services such as big data and artificial intelligence platforms. Cloud technology refers to a hosting technology that unifies a series of resources such as hardware, software and network in a wide area network or local area network to realize data calculation, storage, processing and sharing. The terminal device can be a smart phone, a tablet computer, a notebook computer, a desktop computer, etc., but is not limited thereto. The terminal device and the server can be connected directly or indirectly through wired or wireless communication, which is not limited in the embodiments of the present application.

[0115] The data processing method provided by the embodiments of the present application is taken as an example of an electronic device. The hardware composition structure diagram of the electronic device 700 provided by the embodiments of the present application is shown in FIG. 7. Figure 9The illustrated electronic device 700 includes at least one processor 701, a memory 702, and at least one network interface 704. The various components of electronic device 700 are coupled together by a bus system 705, which is used for the communication of information among the components and to relay house the various buses throughout electronic device 700. It is understood that the bus system 705 is used to implement connections to send and receive electrical signals to each of the components mentioned in electronic device 700. The bus system 705 includes, among other things, a data bus, a power bus, a control bus, and a state signal bus. However, for the sake of clarity, only the data bus is shown in Figure 9 FIG. 7.

[0116] It is to be understood that the memory 702 can be volatile or nonvolatile memory, or both. In one embodiment, the nonvolatile memory can be ROM, programmable ROM (PROM), erasable programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), ferroelectric random access memory (FRAM), flash memory, a magnetic storage disk, a magnetic tape, or a compact disk (CD-ROM). The magnetic storage disk can be a magnetic hard disk or a magnetic tape. The volatile memory can be random access memory (RAM) used as an external cache. By way of example, and not limitation, many forms of RAM are available, such as Static Random Access Memory (SRAM), Synchronous Static Random Access Memory (SSRAM), Dynamic Random Access Memory (DRAM), Synchronous Dynamic Random Access Memory (SDRAM), Double Data Rate Synchronous Dynamic Random Access Memory (DDR SDRAM), Enhanced Synchronous Dynamic Random Access Memory (ESDRAM), Sync Link Dynamic Random Access Memory (SLDRAM), and Direct Rambus Random Access Memory (DRRAM). The memory 702 described herein is intended to include, without being limited to, these and any other suitable types of memory.

[0117] The memory 702 in the embodiments of the present application is configured to store various types of data to support the operation of the electronic device 700. Examples of the data include any computer programs, such as the application programs 7022, for operating on the electronic device 700. The programs for implementing the method of the embodiments of the present application can be included in the application programs 7022.

[0118] The method disclosed in the embodiments of the present application can be applied to the processor 701 or implemented by the processor 701. The processor 701 can be an integrated circuit chip having a processing capability of signals. In the implementation process, each step of the above method can be completed by the integrated logic circuit of hardware in the processor 701 or the instruction in the form of software. The processor 701 described above can be a general processor, a digital signal processor (DSP), or other programmable logic device, discrete gate or transistor logic device, discrete hardware component, etc. The processor 701 can implement or execute the methods, steps and logic block diagrams disclosed in the embodiments of the present application. The general processor can be a microprocessor or any conventional processor, etc. In combination with the steps of the method disclosed in the embodiments of the present application, the execution can be directly completed by a hardware decoding processor or a combination of hardware and software modules in the decoding processor. The software module can be located in a storage medium, and the storage medium is located in the memory 702. The processor 701 reads the information in the memory 702 and combines the hardware to complete the steps of the above method.

[0119] In the exemplary embodiments, the electronic device 700 can be implemented by one or more application specific integrated circuits (ASICs), DSPs, programmable logic devices (PLDs), complex programmable logic devices (CPLDs), FPGAs, general-purpose processors, controllers, MCUs, MPUs, or other electronic elements, for executing the above method.

[0120] The embodiments of the present application provide a computer program product or computer program, which includes computer instructions stored in a computer readable storage medium. The processor of the computer device reads the computer instructions from the computer readable storage medium, and the processor executes the computer instructions, so that the computer device executes the data processing method described above in the embodiments of the present application.

[0121] The embodiment of the present application provides a computer readable storage medium storing executable instructions, wherein the executable instructions are stored, and when the executable instructions are executed by a processor, the processor executes a data processing method provided by the embodiment of the present application, for example, as shown in the following. Figures 3 to 7 The data processing method is shown.

[0122] In some embodiments, the computer readable storage medium can be FRAM, ROM, PROM, EPROM, EEPROM, flash memory, magnetic surface memory, optical disc, or CD-ROM memory, and the like; and can also be various devices including one or any combination of the above storage.

[0123] In some embodiments, the executable instructions can be in the form of programs, software, software modules, scripts or codes, written in any form of programming language (including compiled or interpreted languages, or declarative or procedural languages), and can be deployed in any form, including being deployed as independent programs or being deployed as modules, components, subroutines or other units suitable for use in a computing environment.

[0124] As an example, the executable instructions can correspond to files in a file system, but are not necessarily, can be stored in part of a file storing other programs or data, for example, stored in one or more scripts in a HyperText Markup Language (HTML) document, stored in a single file dedicated to the program in question, or stored in multiple cooperative files (for example, files storing one or more modules, subroutines or code portions).

[0125] As an example, the executable instructions can be deployed to execute on one computing device, or on multiple computing devices located in one place, or on multiple computing devices distributed in multiple places and interconnected through a communication network.

[0126] The above is only an embodiment of the present application, and is not used to limit the protection scope of the present application. Any modification, equivalent replacement and improvement made within the spirit and scope of the present application shall be included in the protection scope of the present application.

Claims

1. A data processing method, comprising: receiving a data write request; performing serialization processing on to-be-written data in response to the data write request; writing the serialized to-be-written data into a non-volatile memory, the non-volatile memory being accessible by byte units; wherein the serialized to-be-written data comprises a data identifier and a data body, the data identifier being used to identify the data body; the data body being a key-value pair in the serialized to-be-written data; obtaining a storage address of the key-value pair in the non-volatile memory; inserting a key in the key-value pair and the storage address into a multi-way tree structure in the non-volatile memory; wherein the writing of the serialized to-be-written data into the non-volatile memory comprises writing the serialized data body and the data identifier in reverse order in a non-transactional manner. 2.The method of claim 1, further comprising: updating a first key-value pair in the non-volatile memory in response to the data update request. 3.The method of claim 2, further comprising: searching for information corresponding to the first key-value pair in the multi-way tree structure; if the information corresponding to the first key-value pair exists in the multi-way tree structure, obtaining a first key in the updated first key-value pair and a first storage address of the updated first key-value pair in the non-volatile memory; updating the information corresponding to the first key-value pair in the multi-way tree structure to the first key and the first storage address. 4.The method of claim 2, further comprising: deleting the information corresponding to the first key-value pair before updating in the multi-way tree structure. 5.The method of claim 3, further comprising: if the information corresponding to the first key-value pair does not exist in the multi-way tree structure, obtaining a first key in the updated first key-value pair and a first storage address of the updated first key-value pair in the non-volatile memory; adding the first key and the first storage address to the multi-way tree structure. 6.The method of claim 1, wherein the writing of the serialized to-be-written data into the non-volatile memory comprises: obtaining a storage offset of the serialized to-be-written data written into the non-volatile memory; writing the serialized to-be-written data into the non-volatile memory based on the storage offset. 7.The method of claim 6, wherein the writing of the serialized to-be-written data into the non-volatile memory based on the storage offset comprises: determining a second storage location of the serialized to-be-written data in the non-volatile memory based on the storage offset and a size of an identifier corresponding to the serialized to-be-written data; writing the serialized to-be-written data into the second storage location and writing the identifier into a third storage location corresponding to the storage offset. 8.A data processing apparatus, comprising: a receiving module configured to receive a data write request; The processing module is configured to perform serialization processing on the data to be written in response to the data write request; The writing module is configured to write the serialized data to be written into the non-volatile memory, the non-volatile memory being accessible by byte units; wherein the serialized data to be written includes data identifiers and data bodies, the data identifiers being used to identify the data bodies; the data bodies being key-value pairs in the serialized data to be written; The obtaining module is configured to obtain a storage address of the key-value pairs in the serialized data to be written in the non-volatile memory; The inserting module is configured to insert the keys in the key-value pairs and the storage address into the multi-way tree structure in the non-volatile memory. The writing of the serialized data to be written into the non-volatile memory includes writing the data bodies and the data identifiers in a reverse order in a non-transactional manner.

9. The apparatus of claim 8, wherein the receiving module is further configured to receive a data update request; The writing module is further configured to update a first key-value pair in the non-volatile memory in response to the data update request.

10. The apparatus of claim 9, wherein the obtaining module is further configured to search for information corresponding to the first key-value pair in the multi-way tree structure; and if the information corresponding to the first key-value pair exists in the multi-way tree structure, obtain a first key in the updated first key-value pair and a first storage address of the updated first key-value pair in the non-volatile memory; The inserting module is further configured to update the information corresponding to the first key-value pair in the multi-way tree structure to the first key and the first storage address.

Images