Data processing method and device and related equipment

By obtaining object key values ​​and node numbers in the key-value storage system, and determining the meta information of the data link list and the target data block number, the problem of low data processing efficiency in the prior art is solved, and more efficient data processing is achieved.

CN119066072APending Publication Date: 2024-12-03XIAOHONGSHU TECH CO LTD
View PDF 6 Cites 0 Cited by

Patent Information

Application Number
CN202411202432.0
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-08-29
Publication Date
2024-12-03

AI Technical Summary

Technical Problem

When implementing the redis linked list protocol layer, existing key-value storage systems that support the redis protocol cannot be fully aligned with the linked list capabilities provided by the redis protocol, resulting in low data processing efficiency.

Method used

By obtaining the object key value and node number in the data processing request, the meta information of the pending data link list is determined based on the data link list information in the key value engine, the target data block number and in-block index are further determined, and the response processing is carried out to improve data processing efficiency.

Benefits of technology

It realizes faster and more accurate determination of key-value information to be processed, improves data processing efficiency in data link lists, and reduces data processing complexity.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119066072A_ABST
    Figure CN119066072A_ABST
Patent Text Reader

Abstract

The embodiment of the invention discloses a data processing method and device and related equipment, and the method comprises the steps: obtaining a data processing request which carries an object key value and a node number; based on key value information of each data chain table recorded in a key value engine, determining meta-information of a to-be-processed data chain table corresponding to the object key value; determining each data block number indicated by the meta-information of the to-be-processed data chain table, and determining a target data block number and a target block internal index corresponding to the node number in each data block number; and according to the target data block number and the index in the target block, performing response processing on the key value information of the to-be-processed data chain table recorded in the key value engine. In this way, the to-be-processed key value information can be determined more quickly and more accurately, and the efficiency of data processing in the data chain table is improved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a data processing method, apparatus and related equipment. Background Art

[0002] At present, enterprise storage products often use key-value storage systems that support the redis protocol (the redis protocol is a simple text protocol used to operate and transmit data between clients and servers) to perform data reading, writing, and updating. Such key-value storage systems that support the redis protocol can usually support internal enterprise data search, advertising recommendations, business recommendations, etc. However, the current method of implementing the redis linked list protocol layer on the key-value storage engine cannot fully align with the linked list capabilities provided by the redis protocol, resulting in low data processing efficiency in the key-value storage system that supports the redis protocol. Summary of the invention

[0003] The embodiments of the present application provide a data processing method, apparatus and related equipment, which can determine the key value information to be processed more quickly and accurately, and improve the efficiency of data processing in a data linked list.

[0004] In a first aspect, an embodiment of the present application provides a data processing method, including:

[0005] Obtaining a data processing request, wherein the data processing request carries an object key value and a node number;

[0006] Based on the key value information of each data linked list recorded in the key value engine, determining the meta information of the to-be-processed data linked list corresponding to the object key value;

[0007] Determine each data block number indicated by the meta information of the to-be-processed data linked list, and determine the target data block number and the target block index corresponding to the node number from among the data block numbers;

[0008] According to the target data block number and the index within the target block, response processing is performed on the key value information of the to-be-processed data linked list recorded in the key value engine.

[0009] In a second aspect, an embodiment of the present application provides a data processing device, including:

[0010] An acquisition unit, used for acquiring a data processing request, wherein the data processing request carries an object key value and a node number;

[0011] A first determining unit, configured to determine the meta information of the to-be-processed data linked list corresponding to the object key value based on the key value information of each data linked list recorded in the key value engine;

[0012] A second determining unit is used to determine the data block numbers indicated by the meta information of the to-be-processed data linked list, and determine the target data block number and the target block index corresponding to the node number from the data block numbers;

[0013] A response unit is used to respond to the key value information of the to-be-processed data linked list recorded in the key value engine according to the target data block number and the index within the target block.

[0014] In a third aspect, an embodiment of the present application provides a computing device, comprising: a processor and a memory, wherein the processor is used to execute the method described in the first aspect above.

[0015] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores program instructions, and when the program instructions are executed, the method described in the first aspect above is implemented.

[0016] In a fifth aspect, an embodiment of the present application provides a computer program product, wherein the computer program product includes program instructions, and when the program instructions are executed by a processor, the method described in the first aspect is implemented.

[0017] The embodiment of the present application first determines the metadata of the data linked list to be processed corresponding to the object key value based on the key value information of each data linked list recorded in the key value engine, and then determines the data block numbers indicated by the metadata of the data linked list to be processed, and determines the target data block number and the target block index corresponding to the node number in each data block number. According to the target data block number and the target block index, the key value information of the data linked list to be processed recorded in the key value engine is responded to and processed, which can determine the key value information to be processed more quickly and accurately, and improve the efficiency of data processing in the data linked list. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0019] Figure 1 It is a schematic diagram of a data linked list recorded in a key-value engine provided in an embodiment of the present application;

[0020] Figure 2 It is a structural diagram of a mapping relationship of meta information provided in an embodiment of the present application;

[0021] Figure 3It is a schematic diagram of an application scenario of a data processing method provided in an embodiment of the present application;

[0022] Figure 4 It is a flowchart of a data processing method provided in an embodiment of the present application;

[0023] Figure 5 It is a schematic diagram of the structure of a data block inserted by a node provided in an embodiment of the present application;

[0024] Figure 6 It is a structural diagram of an update key-value engine provided in an embodiment of the present application;

[0025] Figure 7 This is a schematic diagram of the structure of another data block inserted by a node provided in an embodiment of the present application;

[0026] Figure 8 is a structural diagram of another update key-value engine provided in an embodiment of the present application;

[0027] Fig. 9 This is a structural diagram of a data block splitting provided by an embodiment of the present application;

[0028] Fig.10 This is a schematic diagram of the structure of a data block for node deletion provided in an embodiment of the present application;

[0029] Fig.11 This is a schematic diagram of the structure of another data block deleted by a node provided in an embodiment of the present application;

[0030] Fig.12 is a structural schematic diagram of a data processing device provided in an embodiment of the present application;

[0031] Fig.13 It is a structural diagram of a computing device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0032] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0033] It should be understood that when used in this specification and the appended claims, the terms "include" and "comprises" indicate the presence of described features, integers, steps, operations, elements and / or components, but do not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or combinations thereof.

[0034] It should also be understood that the terms used in this application specification are only for the purpose of describing specific embodiments and are not intended to limit the application. As used in this application specification and the appended claims, unless the context clearly indicates otherwise, the singular forms "a", "an" and "the" are intended to include plural forms.

[0035] It should be further understood that the term “and / or” used in the specification and appended claims refers to any combination and all possible combinations of one or more of the associated listed items, and includes these combinations.

[0036] In a specific implementation, the computing device described in the embodiments of the present application includes, but is not limited to, other portable devices such as mobile phones, laptop computers, or tablet computers with a touch surface (e.g., a touch screen display and / or a touch pad). It should also be understood that in some embodiments, the device is not a portable communication device, but a desktop computer with a touch surface (e.g., a touch screen display and / or a touch pad).

[0037] Reference to "embodiments" in this application means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present application. The appearance of the phrase in various locations in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0038] The embodiments of the present application provide a data processing method, an apparatus, and related equipment. The data processing method can be applied to scenarios such as data reading, data writing, data insertion, data deletion, node insertion, and node deletion in a data linked list.

[0039] The embodiment of the present application introduces a key-value engine, in which key-value information of multiple data linked lists is recorded, and the key-value information includes an object key value, a node number and meta information, and the meta information includes a data block number, an intra-block index and a node value. Among them, the object key value is used to identify a data linked list, the node number is used to identify a node in a data linked list, the data block number is used to identify a data block, a data block is used to indicate a node set in a data linked list, a node set includes multiple consecutive nodes, each intra-block index is used to identify a node in a node set in a data linked list indicated by a data block, and the node value refers to the data stored in a node in a data linked list. It can be understood that the data stored in the node may include but is not limited to data identifiers used to indicate images, text, notes, URLs, etc. In some embodiments, the meta-information may also include a meta-information value, which is used to indicate information of a data linked list. The meta-information value may include but is not limited to the length of the linked list, the version value of the linked list, the survival time of the linked list, the current length of each data block corresponding to the data linked list, the data block number of each data block, the maximum length of each data block, the values ​​of the pointers before and after each data block (i.e., the head and tail), etc.

[0040] In one example, specifically Figure 1 Take the key value information of a data linked list recorded in the key value engine as an example to explain. Figure 1 is a schematic diagram of a data linked list recorded in a key-value engine provided in an embodiment of the present application, such as Figure 1In the data linked list 11 shown, each rectangular grid corresponds to a node. The data linked list 11 includes a node 111 for identifying the meta-information of the data linked list 11. The data linked list 11 also includes a node set 112. The key value information of the node set 112 includes the meta-information of multiple consecutive nodes in the corresponding node set. The meta-information of each node includes the data block number corresponding to each node, such as blk_id_x, which is used to identify the data block number is x, the block index, such as inner_m, which is used to identify the value of an index in a block corresponding to blk_id_x, which is m, and inner_n, which is used to identify the value of an index in a block corresponding to blk_id_x, which is n. Node values, such as node_val, are used to identify the corresponding node value. Each node value is used to record the data stored in the corresponding node. The data linked list 11 also includes a node set 113, a node set 114, and a node set 115, wherein the key value information of the node set 113 includes the meta information of a plurality of consecutive nodes in the corresponding node set, and the meta information of each node includes the data block number corresponding to each node, such as blk_id_y, which is used to identify the data block number as y, the block index such as inner_m, which is used to identify the value of an index in a block corresponding to blk_id_y as m, and inner_n, which is used to identify the value of an index in a block corresponding to blk_id_y as n, and the node value such as node_val, which is used to identify the corresponding node value, and each node value is used to record the data stored in the corresponding node. The data block number corresponding to the node set 114 is blk_id_z, and the block index such as i nner_m is used to identify that the value of an intra-block index in the data block corresponding to blk_id_z is m, inner_n is used to identify that the value of an intra-block index in the data block corresponding to blk_id_z is n, node values ​​such as node_val are used to identify the corresponding node value, and each node value is used to record the data stored in the corresponding node. The data block number corresponding to the node set 115 is blk_id_k, the intra-block index such as inner_m is used to identify that the value of an intra-block index in the data block corresponding to blk_id_k is m, inner_n is used to identify that the value of an intra-block index in the data block corresponding to blk_id_k is n, and the node value such as node_val is used to identify the corresponding node value, and each node value is used to record the data stored in the corresponding node.

[0041] The embodiment of the present application determines, when a data processing request is obtained, the meta-information of the data linked list to be processed corresponding to the object key value carried in the data processing request based on the key value information of each data linked list recorded in the key value engine; determines each data block number indicated by the meta-information of the data linked list to be processed, and determines the target data block number and the target block index corresponding to the node number carried in the data processing request from each data block number; and responds to the key value information of the data linked list to be processed recorded in the key value engine according to the target data block number and the target block index.

[0042] In some embodiments, the response processing includes but is not limited to performing any one or more of data query processing, data insertion processing, and data deletion processing on the node value corresponding to the target data block number and the target block index in the linked list of data to be processed recorded in the key-value engine. In some embodiments, the response processing may also include but is not limited to performing any one or more of node insertion processing, node deletion processing, etc. in the node set corresponding to the target data block in the linked list of data to be processed according to the target data block number and the target block index in the linked list of data to be processed recorded in the key-value engine.

[0043] When the data processing request is obtained, the embodiment of the present application can determine the metadata of the data linked list to be processed corresponding to the object key value from the key value engine according to the object key value carried by the data processing request, and read the metadata of the data linked list to be processed into the memory (such as redis memory), and determine the target data block number and the target block internal index corresponding to the node number according to the preset key value mapping relationship in the memory. By determining the target data block number and the target block internal index, the data block number set and / or the block internal index set to be adjusted can be determined based on the data processing request for the target data block number and the target block internal index, and each data block number in the data block number set to be adjusted and / or each block internal index in the block internal index set to be adjusted are encoded based on the target block internal index, and updated to the key value engine based on the encoding result, and the processing requested by the data processing request is performed in the key value engine, and the processing includes but is not limited to data query processing, data write processing, data deletion processing, node insertion processing, node deletion processing, etc.

[0044] In some embodiments, the key-value mapping relationship preset in the memory can be represented by a two-layer tree structure, where the first layer structure is used to indicate the metadata of the data linked list to be processed corresponding to the object key value, and the second layer structure includes each data block identified by multiple data block numbers corresponding to the metadata, wherein each data block includes one or more intra-block indexes for identifying nodes.

[0045] In one example, specifically Figure 2 Take the example to illustrate the mapping relationship of meta information. Figure 2 Schematic diagram of a structure of a mapping relationship of meta information provided in an embodiment of the present application. Figure 2 As shown, the mapping relationship of the meta information can be represented by a two-sided tree structure, wherein the first layer structure is used to indicate the meta information 20, and the second layer structure includes data block 21, data block 22, data block 23, and data block 24. Figure 1 , the data block number of data block 21 corresponds to Figure 1 The data block number x in blk_id_x corresponds to the data block number of data block 22. Figure 1 The data block number y in blk_id_y corresponds to the data block number of data block 23. Figure 1 The data block number z in blk_id_z corresponds to the data block number of data block 24. Figure 1 The data block number k in blk_id_k. Each data block corresponds to multiple block indexes ( Figure 2 (not shown in the figure), each intra-block index corresponds to a node in the linked list of data to be processed, the data block numbers of each data block are different, the intra-block indexes in each data block are different, and the intra-block indexes in different data blocks may be the same or different.

[0046] The present application stores the key value information of multiple data linked lists in a key value engine, and can respond to and process the data linked list to be processed corresponding to the index in the target block in the target data block according to the key value information of each data linked list recorded in the key value engine, so as to avoid processing all nodes in the data linked list to be processed, and can reduce the complexity of data processing of the data linked list. Furthermore, when responding to and processing each node in the node set in the data linked list to be processed corresponding to the index in the target block in the target data block according to the key value information of each data linked list recorded in the key value engine, a two-layer tree structure for representing the mapping relationship of the meta-information of the data linked list to be processed can be determined according to the key value information of each data linked list recorded in the key value engine, and the mapping relationship of the meta-information of the two-layer tree structure is used to implement the response processing of the key value information of the data linked list to be processed requested by the data processing request, which can determine the key value information to be responded to more quickly and accurately, and help improve the data processing efficiency of the data linked list.

[0047] The data processing method provided in the embodiments of the present application can be applied to a data processing device, which can be set in a computing device. In some embodiments, the computing device can include but is not limited to smart phones, tablet computers, laptop computers, desktop computers, vehicle-mounted smart terminals, smart watches and other terminal devices; in other embodiments, the computing device can include but is not limited to independent servers, cluster servers or cloud servers, etc.

[0048] The following is combined with Figure 3-Figure 11 The data processing method provided in the embodiment of the present application is schematically illustrated.

[0049] For details, please see Figure 3 , Figure 3 is a schematic diagram of an application scenario of a data processing method provided in an embodiment of the present application, the application scenario includes a terminal device 31 ( Figure 3 The scenario shown takes the case where the terminal device 31 is a desktop computer as an example).

[0050] In actual applications, the terminal device 31 can obtain a data processing request sent by a user, which carries an object key value and a node number; based on the key value information of each data linked list recorded in the key value engine, determine the metadata of the data linked list to be processed corresponding to the object key value; determine the data block numbers indicated by the metadata of the data linked list to be processed, and determine the target data block number and the target block index corresponding to the node number in each data block number; and respond to the key value information of the data linked list to be processed recorded in the key value engine according to the target data block number and the target block index.

[0051] It should be understood that Figure 3 The application scenario shown is only an example. In actual applications, in addition to the terminal device 31 independently executing the data processing method provided in the embodiment of the present application, the server can also independently execute the data processing method provided in the embodiment of the present application; the terminal device and the server can also collaborate to execute the data processing method provided in the embodiment of the present application. For example, the terminal device receives a data processing request sent by a user, and the data processing request carries an object key value and a node number. The terminal device sends the object key value and the node number carried in the data processing request to the server, so that the server determines the metadata of the data linked list to be processed corresponding to the object key value based on the key value information of each data linked list recorded in the key value engine; determines each data block number indicated by the metadata of the data linked list to be processed, and determines the target data block number and the target block index corresponding to the node number in each data block number; according to the target data block number and the target block index, the key value information of the data linked list to be processed recorded in the key value engine is responded to and processed, and the result data of the response processing is sent to the terminal device. The terminal device can display the result data of the response processing on the screen of the terminal device in the data processing scenario.

[0052] See also Figure 4 , Figure 4 is a flow chart of a data processing method provided in an embodiment of the present application, such as Figure 4 The data processing method in the embodiment shown can be applied to a data processing device, which is arranged in a computing device. The method may at least include the following steps.

[0053] S401: Obtain a data processing request, which carries an object key value and a node number.

[0054] In an embodiment of the present application, a computing device may obtain a data processing request, which carries an object key value and a node number. In some embodiments, the data processing request may be sent by a user through a user interface of a user terminal. Further, the user may send a data processing request to a computing device by triggering a control on the user interface of the user terminal for sending a data processing request, wherein the control may include but is not limited to a data query control, a data read-write control, a data deletion control, a node insertion control, a node deletion control, and the like. In some embodiments, when a node insertion control is triggered, the user interface of the user terminal may be triggered to jump to another user interface, on which a folder for inserting a node may be selected, and after triggering the folder for inserting a node, the user may generate a node insertion request.

[0055] In some embodiments, the key value information of each data linked list recorded in the key value engine includes: object key value, node number and meta information, the meta information includes data block number, block index and node value, wherein the object key value is used to identify a data linked list, the node number is used to identify a node in a data linked list, the data block number is used to identify a data block, a data block is used to indicate a node set in a data linked list, a node set includes multiple consecutive nodes, each block index is used to identify a node in the data linked list indicated by a data block, and the node value refers to the data stored in a node in a data linked list.

[0056] In some embodiments, the object key value, node number and meta information in the key value information may include but are not limited to any one or more characters such as text, letters, numbers, symbols, etc., wherein the data block numbers in the key value information of the same data linked list are different, the indexes in the blocks corresponding to the same data block number are different, and the indexes in the blocks corresponding to different data block numbers may be the same or different. The key value information of the data linked list recorded in the key value engine is as follows: Figure 1 In the corresponding embodiment, the mapping relationship between the data block number, the index within the block, and the node value in the meta information is as follows: Figure 2 The corresponding embodiments are described above and will not be repeated here.

[0057] S402: Based on the key value information of each data linked list recorded in the key value engine, determine the meta information of the to-be-processed data linked list corresponding to the object key value.

[0058] In the embodiment of the present application, the computing device can determine the metadata of the to-be-processed data list corresponding to the object key value based on the key value information of each data list recorded in the key value engine. The key value information also includes a key value mapping relationship, which includes a mapping relationship between a node number, a data block number, and an index within a block. The key value mapping relationship is pre-stored in a memory (such as a redis memory) and can be represented by a two-layer tree structure, as shown in FIG. Figure 2 The corresponding embodiments are described, and will not be described here in detail. In some embodiments, the key value information also includes a mapping relationship between an object key value, a data linked list, and meta information.

[0059] The embodiment of the present application helps to determine the corresponding data block numbers from the key value information of the data linked list to be processed by the key value engine by determining the meta information of the data linked list to be processed corresponding to the object key value.

[0060] S403: Determine the data block numbers indicated by the meta-information of the to-be-processed data linked list, and determine the target data block number and the target block index corresponding to the node number from among the data block numbers.

[0061] In an embodiment of the present application, the computing device may determine the data block numbers indicated by the metadata of the linked list of data to be processed, and determine the target data block number and the index within the target block corresponding to the node number from among the data block numbers.

[0062] In one embodiment, when determining the data block numbers indicated by the metadata of the data linked list to be processed, the computing device can read the metadata of the data linked list to be processed into a memory (such as a redis memory) and determine the data block numbers corresponding to the metadata according to a preset key-value mapping relationship in the memory.

[0063] In one embodiment, when the computing device determines the target data block number and the target block index corresponding to the node number in each data block number, it can determine the target data block number and the target block index corresponding to the node number based on the key value mapping relationship included in the key value information, and the key value mapping relationship includes the mapping relationship between the node number, the data block number and the block index.

[0064] In an optional implementation, the computing device may obtain the address information of each node in the data linked list to be processed, and each address in the address information is used to indicate each node in the data linked list to be processed; the nodes in the data linked list to be processed may be divided into multiple data blocks according to the address information, and different data blocks correspond to different data block numbers, and each data block includes a block index for indicating different nodes; further, the target address corresponding to the node number may be determined according to a preset mapping relationship between the number and the address, and the target data block number and the target block index corresponding to the target address may be determined as the target data block number and the target block index corresponding to the node number. In some embodiments, the address information may include, but is not limited to, any one or more characters such as numbers, letters, words, and symbols.

[0065] In an optional implementation, when the computing device can divide the nodes of the data linked list to be processed into multiple data blocks according to the address information, it can be evenly divided into multiple data blocks according to the number of nodes in the data linked list to be processed, that is, the number of corresponding nodes in each data block is the same. In other optional implementations, the number of corresponding nodes in each data block obtained by division may also be different, and the embodiments of the present application do not specifically limit this.

[0066] Furthermore, when the computing device determines that the target data block number and the target block index corresponding to the target address are the target data block number and the target block index corresponding to the node number, it can determine the target data block number where the target address is located based on the mapping relationship between each address and the data block number, and further determine the target block index corresponding to the target address based on the mapping relationship between the address information and the block index in the target data block number.

[0067] For example, assuming that the node number is 1500, the address information of each node in the to-be-processed data linked list is 0-3000, and the nodes of the to-be-processed linked list are divided into three data blocks according to the address information. The data block numbers of the three data blocks are x, y, and z, respectively. The address information of the node corresponding to data block x is 0-1000, the address information of the node corresponding to data block y is 1001-2000, and the address information of the node corresponding to data block z is 2001-3000. If the preset mapping relationship between the number and the address is that the starting number and the starting address are the same, then it can be determined that the target address corresponding to the node number 1500 is 1500, and further, the target address corresponding to the node number 1500 can be determined according to the mapping relationship between the starting number and the starting address. The mapping relationship between the address information 1001-2000 in data block y and the data block number determines that the target data block number corresponding to the target address 1500 is y, that is, the node number 1500 is in the data block y; further, assuming that the numbering of the block index used to indicate each node in the data block y starts from the starting block index of the data block y as inner_a, inner_b, ..., inner_n, the target block index corresponding to the address information 1500 in the data block y can be determined to be inner_t according to the mapping relationship between the address information 1001-2000 in the data block y and the block indexes inner_a, inner_b, ..., inner_n of the data block y.

[0068] The embodiment of the present application determines the meta-information of the data linked list to be processed corresponding to the object key value by using the key value information of the data linked list to be processed recorded in the key value engine, reads the meta-information into the memory, determines the data block numbers corresponding to the meta-information from the key value mapping relationship preset in the memory, and determines the target data block number and the target block index corresponding to the node number from the data block numbers. The key value information of the data linked list to be processed can be determined more quickly and efficiently from the key value information of each data linked list recorded in the key value engine according to the target data block number and the target block index.

[0069] S404: According to the target data block number and the index in the target block, the key value information of the to-be-processed data linked list recorded in the key value engine is responded to and processed.

[0070] In the embodiment of the present application, the computing device can respond to the key value information of the linked list of data to be processed recorded in the key value engine according to the target data block number and the index within the target block.

[0071] In some embodiments, the response processing includes: performing any one or more of data query processing, data insertion processing, and data deletion processing on the node value corresponding to the target data block number and the index in the target block in the linked list of data to be processed recorded in the key-value engine.

[0072] In one embodiment, when the computing device responds to and processes the key value information of the linked list of data to be processed recorded in the key value engine according to the target data block number and the index within the target block, when the data processing request is used to request data query processing of the linked list of data to be processed, the data processing request is a data query request, and the data query request may also carry the data to be queried. The computing device may determine, according to the target data block number and the index within the target block, the data to be queried in the node corresponding to the target data block number and the index within the target block in the linked list of data to be processed, and read the data to be queried from the node corresponding to the target data block number and the index within the target block in the linked list of data to be processed.

[0073] In one implementation, the computing device may obtain a data query request sent by the terminal device, the data query request may be generated by a user inputting the data to be queried on the user interface of the terminal device and triggering a control for sending a data query request carrying the data to be queried, and the terminal device may send the data query request to the computing device after generating the data query request. In some embodiments, the way to trigger the control may include but is not limited to pressing, sliding, clicking, etc.

[0074] In one embodiment, when the computing device responds to and processes the key value information of the linked list of data to be processed recorded in the key value engine according to the target data block number and the index within the target block, when the data processing request is used to request data insertion processing in the linked list of data to be processed, the data processing request is a data insertion request, and the data insertion request may also carry the data to be inserted. The computing device may determine, according to the target data block number and the index within the target block, the node in the linked list of data to be processed that corresponds to the target data block number and the index within the target block, and add the data to be inserted in the node in the linked list of data to be processed that corresponds to the target data block number and the index within the target block.

[0075] In one embodiment, a computing device may obtain a data insertion request sent by a user terminal. The data insertion request may be generated by a user inputting data to be inserted on a user interface of the user terminal and triggering a control for sending a data insertion request carrying the data to be inserted. The user terminal may send the data insertion request to the computing device after generating the data insertion request.

[0076] In one embodiment, when the computing device responds to and processes the key value information of the data linked list to be processed recorded in the key value engine according to the target data block number and the index within the target block, when the data processing request is used to request data deletion processing of the data linked list to be processed, the data processing request is a data deletion request, and the data deletion request may also carry the data to be deleted. The computing device may determine, according to the target data block number and the index within the target block, that the data to be inserted is in the node corresponding to the target data block number and the index within the target block in the data linked list to be processed, and delete the data to be deleted in the node corresponding to the target data block number and the index within the target block in the data linked list to be processed.

[0077] In one embodiment, a computing device may obtain a data deletion request sent by a user terminal. The data deletion request may be generated by a user inputting data to be deleted on a user interface of the user terminal and triggering a control for sending a data deletion request carrying the data to be deleted. The user terminal may send the data deletion request to the computing device after generating the data deletion request.

[0078] In the embodiment of the present application, the key-value information recorded in the key-value engine and the key-value mapping relationship stored in the memory (such as the redis memory) can quickly and efficiently respond to the acquired data processing request.

[0079] In some embodiments, the response processing also includes: based on the target data block number and the index within the target block, performing node insertion processing, node deletion processing, etc. on the linked list of data to be processed recorded in the key-value engine.

[0080] In one embodiment, when the computing device responds to the key value information of the to-be-processed data linked list recorded in the key value engine according to the target data block number and the target block internal index, when the data processing request is used to request node insertion processing on the to-be-processed data linked list, the block capacity of the target data block corresponding to the target data block number after inserting the block internal index of the node to be inserted can be estimated; according to the target data block number and the target block internal index, the insertion position of the node to be inserted in the multiple nodes identified by the target data block number in the to-be-processed data linked list is determined; if the block capacity meets the first capacity condition, the key value information of the to-be-processed data linked list recorded in the key value engine is responded to according to the insertion position of the node to be inserted, the target data block number and the target block internal index; if the block capacity meets the second capacity condition, the target data block is split to obtain the first data block and the second data block, and according to the first data block, the second data block, the target data block number and the target block internal index, the key value information of the to-be-processed data linked list recorded in the key value engine is responded to.

[0081] In one embodiment, when a data processing request is used to request node insertion processing for a linked list of data to be processed, the data processing request is a node insertion request, and the node insertion request may also carry a node to be inserted. The computing device may obtain the node insertion request sent by the user terminal, and the node insertion request may be generated by the user inputting a file of a node to be inserted on the user interface of the user terminal, and triggering a control for sending a node insertion request for a file carrying the node to be inserted. The user terminal may send the node insertion request for the file carrying the node to be inserted to the computing device after generating the node insertion request. In some embodiments, the user interface where the control for sending the node insertion request for a folder carrying the node to be inserted is located may be the same as or different from the user interface for inputting the folder for inserting the node to be inserted. In some embodiments, if the user interface where the control for sending the node insertion request for a folder carrying the node to be inserted is located is different from the user interface for inputting the folder for inserting the node to be inserted, when the user touches the control for sending the node insertion request on the first user interface of the user terminal, it triggers a jump from the first user interface to the second user interface of the user terminal, and the second user interface is used to input the file of the node to be inserted. After the file of the node to be inserted is input, the node insertion request is generated.

[0082] In one embodiment, when estimating the block capacity of the target data block corresponding to the target data block number after the intra-block index of the node to be inserted is inserted, the computing device can obtain the block capacity of the target data block and determine the capacity corresponding to the intra-block index of the node to be inserted, and determine the block capacity of the target data block corresponding to the target data block number after the intra-block index of the node to be inserted based on the block capacity of the target data block and the capacity corresponding to the intra-block index of the node to be inserted.

[0083] In some embodiments, the block capacity of the target data block may be determined based on the sum of the capacities corresponding to the in-block indexes in the target data block, and the capacities corresponding to the in-block indexes may be determined based on the target data block number and the storage capacities of the nodes corresponding to the in-block indexes. In some embodiments, the block capacity of the target data block may be determined based on the number of in-block indexes in the target data block, wherein the representation of the block capacity may be the same as the number of in-block indexes, for example, if the number of in-block indexes in the target data block is 4, then the block capacity of the target data block may be determined to be 4. The representation of the block capacity may also be different from the number of in-block indexes and the representation of the block capacity may be in a mapping relationship with the number of in-block indexes, for example, if the number of in-block indexes in the target data block is 4, then the block capacity of the target data block may be determined to be A, and if the number of in-block indexes in the target data block is 5, then the block capacity of the target data block may be determined to be B, and B may be equal to A+1 or B may be determined based on other mapping relationships.

[0084] In one embodiment, when the computing device determines the insertion position of the node to be inserted in the multiple nodes identified by the target data block number in the linked list of data to be processed according to the target data block number and the target block internal index, it can obtain the node set identified by the target data block number, the node set includes multiple nodes, each node is continuous with each other, each node corresponds to a different block internal index, and the insertion position of the node to be inserted in each continuous node of the node set is determined according to the target block internal index. In some embodiments, the insertion position includes but is not limited to the target position or internal position of each node in the node set, wherein the target position includes the head position or tail position of each node, and the internal position is other positions in the node set except the head position and the tail position.

[0085] In some embodiments, the block capacity satisfying the first capacity condition may be that the block capacity is less than or equal to a preset capacity threshold, and the block capacity satisfying the second capacity condition may be that the block capacity is greater than the preset capacity threshold.

[0086] In one embodiment, when the block capacity satisfies the first capacity condition and when the position of the in-block index of the node to be inserted is determined to be the internal position of the target data block based on the insertion position, the computing device can generate and add the in-block index of the node to be inserted in the target data block based on the insertion position when responding to the key value information of the linked list of data to be processed recorded in the key value engine according to the insertion position of the node to be inserted, the target data block number and the in-block index of the target data block; determine the first in-block index set to be adjusted in the target data block after adding the in-block index of the node to be inserted based on the insertion position, and adjust the value of each in-block index in the first in-block index set; encode the in-block index of the node to be inserted and each in-block index in the first in-block index set after the value is adjusted, and update the encoding result to the key value engine; in the updated key value engine, write the data of the node to be inserted into the node value corresponding to the target data block number and the in-block index of the node to be inserted in the key value engine. In some embodiments, the computing device may also adjust a head pointer or a tail pointer of the target data block to adjust the number of intra-block indexes in the target data block, wherein the head pointer and the tail pointer are used to indicate the position of the intra-block index in the target data block (i.e., indicated as the number of intra-block indexes).

[0087] In one embodiment, when the computing device determines the first set of in-block indexes to be adjusted in the target data block after adding the in-block index of the node to be inserted based on the insertion position, the computing device may determine the number of in-block indexes on both sides of the position of the in-block index of the node to be inserted in the target data block after adding the in-block index of the node to be inserted based on the insertion position; and determine the first set of in-block indexes to be adjusted in the target data block after adding the in-block index of the node to be inserted based on the number of in-block indexes on both sides of the position of the in-block index of the node to be inserted.

[0088] Furthermore, when the computing device determines the first in-block index set to be adjusted in the target data block after adding the in-block index of the node to be inserted based on the number of in-block indexes on both sides of the position of the in-block index of the node to be inserted, the number of in-block indexes on both sides of the position of the in-block index of the node to be inserted can be compared to determine the in-block index corresponding to the side with the smaller number as the first in-block index set.

[0089] In one example, specifically Figure 5 Take this as an example to illustrate: Figure 5 is a schematic diagram of a data block inserted by a node provided in an embodiment of the present application, such as Figure 5 As shown, the target data block number of the target data block 50 is blk_x, the target block index is inner_a, the block indexes in the target data block 50 include inner_m, inner_a, inner_b, inner_n, the block index of the node to be inserted is inner_c, the position of the block index of the node to be inserted determined based on the insertion position is the internal position 501 of the target data block 50, and it is determined that the number of block indexes on the left side of the position 501 of the block index of the node to be inserted in the target data block 50 obtained after adding the block index inner_c of the node to be inserted is 1 (i.e., the block index inner_m), and the number of block indexes on the right side of the position 501 of the block index of the node to be inserted is 3 (including: inner_a, inner_b, inner_n), then it can be determined that the block index inner_m on the left side of the position 501 of the block index of the node to be inserted is the first block index set.

[0090] Furthermore, when the computing device encodes the in-block index of the node to be inserted and each in-block index in the first in-block index set after value adjustment, and updates the encoding result to the key-value engine, the computing device may specifically combine Figure 1 , Figure 5 and Figure 6 Explain the update of the key-value engine. Figure 6 This is a schematic diagram of a structure of an update key value engine provided by an embodiment of the present application, based on Figure 5 The in-block index inner_c of the node to be inserted and the in-block index inner_m in the first in-block index set after the value is adjusted are added in , and the in-block index of the node to be inserted and each in-block index in the first in-block index set after the value is adjusted are encoded, and the encoding result is updated to Figure 1 After the data to be processed in the key value engine is linked in the table, the following is obtained: Figure 6 The linked list of data to be processed in the key-value engine is shown.

[0091] The embodiment of the present application determines the first set of in-block indexes to be adjusted based on the internal position of the in-block index to be inserted in the target data block, so as to adjust some in-block indexes (i.e., the first set of in-block indexes) and avoid adjusting all in-block indexes in the target data block, which helps to reduce the complexity of data processing and improve the efficiency of data processing.

[0092] In one embodiment, when the block capacity meets the first capacity condition, the computing device responds to the key value information of the linked list of data to be processed recorded in the key value engine according to the target data block number and the target block index. When the position of the block index of the node to be inserted is determined as the target position of the target data block based on the insertion position, the block index of the node to be inserted is generated and added in the target data block based on the insertion position; the block index of the node to be inserted is encoded, and the encoding result is updated to the key value engine; in the updated key value engine, the data of the node to be inserted is written to the node value in the key value engine corresponding to the target data block number and the block index of the node to be inserted, wherein the target position may include the head position or the tail position of the target data block.

[0093] In one example, specifically Figure 7 Take this as an example to illustrate: Figure 7 is a schematic diagram of the structure of another data block inserted by a node provided in an embodiment of the present application, such as Figure 7 As shown, the target data block 70 is numbered blk_x, the target block index is inner_a, the block indexes in the target data block 70 include inner_m, inner_a, inner_b, and inner_n, the block index of the node to be inserted is inner_c, the position of the block index inner_c of the node to be inserted determined based on the insertion position is the head position 701 of the target data block 70, the block index inner_c of the node to be inserted is generated in the target data block and added to the head position 701 of the target data block 70, and the target data block after the block index inner_c of the node to be inserted is added is obtained.

[0094] Furthermore, it can be combined with Figure 1 , Figure 7 and Figure 8 Explain the update of the key-value engine. Figure 8 is a schematic diagram of another structure of an update key value engine provided by an embodiment of the present application. Figure 7 Encode the block index inner_c of the node to be inserted added in , and update the encoding result to Figure 1 After the data to be processed in the key value engine is linked in the table, the following is obtained: Figure 8 The linked list of data to be processed in the key-value engine is shown.

[0095] The embodiment of the present application can avoid adjusting the block index in the target data block by directly adding the block index to be inserted at the head position or the tail position of the target data block based on the head position or the tail position of the block index to be inserted in the target data block, which helps to reduce the complexity of data processing and improve the efficiency of data processing.

[0096] In one embodiment, when the block capacity meets the second capacity condition, when the computing device splits the target data block to obtain the first data block and the second data block, it can determine the insertion position of the node to be inserted in the multiple nodes identified by the target data block number in the linked list of data to be processed; based on the position of the in-block index of the node to be inserted, the target data block with the in-block index of the node to be inserted added is split to obtain the first data block and the second data block.

[0097] In one embodiment, when the computing device responds to the key value information of the to-be-processed data linked list recorded in the key value engine according to the first data block, the second data block, the target data block number and the target block intra-index, it can determine the set of data block numbers to be adjusted in the first data block and the second data block obtained by the splitting process based on the position of the intra-block index of the node to be inserted, and adjust each data block number in the data block number set; encode the intra-block index of the node to be inserted and the adjusted data block number set, and update the encoding result to the key value engine; in the updated key value engine, write the data of the node to be inserted into the node value in the key value engine corresponding to the set of data block numbers and the intra-block index of the node to be inserted.

[0098] In one embodiment, when the computing device splits the target data block after adding the in-block index of the node to be inserted based on the position of the in-block index of the node to be inserted, and obtains the first data block and the second data block, the computing device may add the in-block index of the node to be inserted based on the position of the in-block index of the node to be inserted, and split the target data block after adding the in-block index of the node to be inserted based on the position of the in-block index of the node to be inserted, and determine the in-block index of the node to be inserted and each in-block index to the left of the in-block index of the node to be inserted in the target data block after adding the in-block index of the node to be inserted as the in-block index of the first data block, and determine each in-block index to the right of the in-block index of the node to be inserted as the in-block index of the second data block. Similarly, the in-block index of the node to be inserted and each in-block index to the right of the in-block index of the node to be inserted in the target data block after adding the in-block index of the node to be inserted may be determined as the in-block index of the first data block, and each in-block index to the left of the in-block index of the node to be inserted may be determined as the in-block index of the second data block.

[0099] Specific can Fig. 9Take the target data block as an example to illustrate the splitting. Fig. 9 is a structural diagram of a data block splitting provided by an embodiment of the present application, such as Fig. 9 As shown, the target data block 90 is numbered blk_x, the target block index is inner_a, the target data block 90 includes inner_m, inner_a, inner_b, inner_k, inner_n, the block index of the node to be inserted is inner_c, and the target data block obtained after adding the block index inner_c of the node to be inserted is split based on the insertion position 901, and the block index inner_c of the node to be inserted in the target data block after adding the block index inner_c of the node to be inserted and each block index to the left of the block index inner_c of the node to be inserted (i.e., inner_m) are determined as the block index of the first data block 91, and each block index to the right of the block index inner_c of the node to be inserted (i.e., inner_a, inner_b, inner_k, inner_n) is determined as the block index of the second data block 92.

[0100] In one embodiment, when the computing device determines the set of data block numbers to be adjusted in the first data block and the second data block obtained by splitting processing based on the position of the index within the block of the node to be inserted, the data block number of the first data block and / or the data block number of the second data block can be determined as the set of data block numbers to be adjusted, wherein the data block number of the first data block is different from the data block number of the second data block, and when the data block number of the first data block is the set of data block numbers to be adjusted, the data block number of the second data block can use the target data block number; when the data block number of the second data block is the set of data block numbers to be adjusted, the data block number of the first data block can use the target data block number.

[0101] In one embodiment, when the computing device determines that the position of the in-block index of the node to be inserted is at the internal position of the first data block, it can determine that the data block number set to be adjusted includes the data block number of the first data block, determine the third in-block index set to be adjusted in the first data block based on the position of the in-block index of the node to be inserted at the internal position of the first data block, and adjust the data block number of the first data block in the data block number set and each in-block index in the third in-block index set, and encode the in-block index of the node to be inserted, each in-block index in the third in-block index set after the value adjustment, and the adjusted data block number set, and update the encoding result to the key-value engine. Among them, the method of determining the third in-block index set is the same as the method of determining the combination of the first in-block index, which will not be repeated here.

[0102] In one embodiment, when the computing device determines that the position of the in-block index of the node to be inserted is at the internal position of the second data block, it can determine that the data block number set to be adjusted includes the data block number of the second data block, determine the fourth in-block index set to be adjusted in the second data block based on the position of the in-block index of the node to be inserted at the internal position of the second data block, and adjust the data block number of the second data block in the data block number set and each in-block index in the fourth in-block index set, and encode the in-block index of the node to be inserted, each in-block index in the fourth in-block index set after the value adjustment, and the adjusted data block number set, and update the encoding result to the key-value engine. Among them, the method of determining the fourth in-block index set is the same as the method of determining the combination of the first in-block index, which will not be repeated here.

[0103] The embodiment of the present application limits the block capacity of the data block. When the block capacity of the target data block is insufficient to add the in-block index to be inserted, the target data block is split. This helps to perform corresponding splitting processing on the to-be-processed data linked list in the key-value engine according to the first data block and the second data block obtained by the splitting processing, thereby reducing the situation where the addition of the node to be inserted is caused by insufficient capacity of the node set corresponding to the target data block in the to-be-processed data linked list of the key-value engine.

[0104] In one embodiment, when the computing device responds to and processes the key value information of the to-be-processed data linked list recorded in the key value engine according to the target data block number and the target block index, when the data processing request is used to request node deletion processing of the to-be-processed data linked list, it can determine to delete the target block index in the target data block corresponding to the target data block number according to the target data block number and the target block index; determine the second block index set to be adjusted in the target data block after deleting the target block index according to the position of the target block index in the target data block, and adjust the value of each block index in the second block index set; delete the key value information corresponding to the target data block number and the target block index in the key value engine; encode each block index in the second block index set after the value adjustment, and update the encoding result to the key value engine.

[0105] In one embodiment, when the data processing request is used to request a node deletion process for the data linked list to be processed, specifically Fig.10 Take the example of deleting a data block corresponding to a node to explain. Fig.10 is a schematic diagram of a structure of a data block for node deletion provided in an embodiment of the present application, such as Fig.10As shown, the target data block number of the target data block 100 is blk_x, the target block internal index is inner_a, and the internal block indexes in the target data block 100 include inner_m, inner_a, inner_b, inner_k, and inner_n. Then, the target block internal index inner_a can be deleted from the target data block.

[0106] In one embodiment, when the data processing request is used to request multiple nodes to be deleted from the data linked list to be processed, the specific method may be: Fig.11 Take the example to explain how to delete the data blocks corresponding to multiple nodes. Fig.11 is a schematic diagram of the structure of another data block for node deletion provided in an embodiment of the present application, such as Fig.11 As shown, the target data block 110 is numbered blk_x, the target block indexes are inner_a, inner_b, inner_k, and inner_f, and the block indexes in the target data block 110 include inner_m, inner_a, inner_b, inner_k, inner_e, inner_h, inner_g, inner_i, inner_f, and inner_n. The target block indexes inner_a, inner_b, inner_k, and inner_f can be deleted in the target data block 110.

[0107] In one embodiment, if the computing device determines that there is no intra-block index in the target data block after deleting the intra-block index based on the target data block number and the position of the intra-block index in the target data block, the target data block in the metadata in the memory can be deleted and the node set corresponding to the target data block in the key-value engine can be deleted.

[0108] The embodiment of the present application can more quickly and efficiently delete the node corresponding to the target data block number and the target block index carried by the data processing request when a data processing request for requesting to delete a node is obtained, by using the data block number and the block index in the key value information of the linked list of data to be processed recorded in the key value engine.

[0109] The embodiment of the present application obtains a data processing request, which carries an object key value and a node number; determines the metadata of the data linked list to be processed corresponding to the object key value based on the key value information of each data linked list recorded in the key value engine; determines each data block number indicated by the metadata of the data linked list to be processed, and determines the target data block number and the target block index corresponding to the node number in each data block number; responds to the key value information of the data linked list to be processed recorded in the key value engine according to the target data block number and the target block index, so as to determine the key value information to be processed more quickly and accurately and improve the efficiency of data processing.

[0110] See also Fig.12 , Fig.12 1 is a structural diagram of a data processing device provided in an embodiment of the present application. Specifically, the data processing device is set in a computing device, and the data processing device includes: an acquisition unit 1201, a first determination unit 1202, a second determination unit 1203 and a response unit 1204;

[0111] The acquisition unit 1201 is used to acquire a data processing request, where the data processing request carries an object key value and a node number;

[0112] The first determining unit 1202 is used to determine the meta information of the to-be-processed data linked list corresponding to the object key value based on the key value information of each data linked list recorded in the key value engine;

[0113] The second determining unit 1203 is used to determine the data block numbers indicated by the meta information of the to-be-processed data linked list, and determine the target data block number and the target block index corresponding to the node number from the data block numbers;

[0114] The response unit 1204 is used to respond to the key value information of the to-be-processed data linked list recorded in the key value engine according to the target data block number and the index in the target block.

[0115] Furthermore, the key value information of each data linked list recorded in the key value engine includes: object key value, node number and meta information, the meta information includes data block number, intra-block index and node value, wherein the object key value is used to identify a data linked list, the node number is used to identify a node in a data linked list, the data block number is used to identify a data block, a data block is used to indicate a node set in a data linked list, a node set includes multiple consecutive nodes, each intra-block index is used to identify a node in the data linked list indicated by a data block, and the node value refers to the data stored in a node in a data linked list.

[0116] Further, when the second determining unit 1203 determines the target data block number and the target block index corresponding to the node number from the data block numbers, it is specifically used to:

[0117] Determine the target data block number and the target block index corresponding to the node number according to the key-value mapping relationship included in the key-value information, wherein the key-value mapping relationship includes a mapping relationship among the node number, the data block number and the block index;

[0118] The response processing includes: performing any one or more of data query processing, data insertion processing, and data deletion processing on the target data block number and the node value corresponding to the index in the target block in the linked list of data to be processed recorded in the key-value engine.

[0119] Further, when the second determining unit 1203 determines the target data block number and the target block index corresponding to the node number according to the key value mapping relationship included in the key value information, it is specifically used to:

[0120] Acquire address information of each node in the linked list of data to be processed, where each address in the address information is used to indicate each node in the linked list of data to be processed;

[0121] Dividing the nodes of the to-be-processed data linked list according to the address information to obtain a plurality of data blocks, where different data blocks correspond to different data block numbers, and each data block includes an intra-block index for indicating different nodes;

[0122] According to the preset mapping relationship between the number and the address, the target address corresponding to the node number is determined, and the target data block number and the target block index corresponding to the target address are determined to be the target data block number and the target block index corresponding to the node number.

[0123] Further, when the response unit 1204 performs response processing on the key value information of the to-be-processed data linked list recorded in the key value engine according to the target data block number and the index in the target block, it is specifically used to:

[0124] When the data processing request is used to request node insertion processing on the to-be-processed data linked list, estimating the block capacity of the target data block corresponding to the target data block number after inserting the intra-block index of the to-be-inserted node;

[0125] Determine, according to the target data block number and the target block internal index, an insertion position of the to-be-inserted node in a plurality of nodes identified by the target data block number in the to-be-processed data linked list;

[0126] If the block capacity meets the first capacity condition, responding to the key value information of the to-be-processed data linked list recorded in the key value engine according to the insertion position of the to-be-inserted node, the target data block number and the index within the target block;

[0127] If the block capacity meets the second capacity condition, the target data block is split to obtain a first data block and a second data block, and the key value information of the data linked list to be processed recorded in the key value engine is responded to based on the first data block, the second data block, the target data block number and the index within the target block.

[0128] Further, when the response unit 1204 performs response processing on the key value information of the to-be-processed data linked list recorded in the key value engine according to the insertion position of the to-be-inserted node, the target data block number and the index within the target block, it is specifically used to:

[0129] Generate and add the intra-block index of the node to be inserted in the target data block based on the insertion position;

[0130] Determine, based on the insertion position, a first set of in-block indexes to be adjusted in the target data block after adding the in-block index of the node to be inserted, and adjust the value of each in-block index in the first set of in-block indexes;

[0131] Encode the in-block index of the node to be inserted and each in-block index in the first in-block index set after value adjustment, and update the encoding result to the key-value engine;

[0132] In the updated key-value engine, the data of the node to be inserted is written into the node value in the key-value engine corresponding to the target data block number and the intra-block index of the node to be inserted.

[0133] Further, when the response unit 1204 performs response processing on the key value information of the to-be-processed data linked list recorded in the key value engine according to the insertion position of the to-be-inserted node, the target data block number and the index within the target block, it is specifically used to:

[0134] When the position of the in-block index of the node to be inserted is determined to be the target position of the target data block based on the insertion position, generating and adding the in-block index of the node to be inserted in the target data block based on the insertion position;

[0135] Encode the intra-block index of the node to be inserted, and update the encoding result to the key value engine;

[0136] In the updated key-value engine, the data of the node to be inserted is written into the node value in the key-value engine corresponding to the target data block number and the intra-block index of the node to be inserted.

[0137] Further, when the response unit 1204 determines the first in-block index set to be adjusted in the target data block after adding the in-block index of the node to be inserted based on the insertion position, it is specifically used to:

[0138] Determine, based on the insertion position, the number of in-block indexes on both sides of the position of the in-block index of the node to be inserted in the target data block after adding the in-block index of the node to be inserted;

[0139] Based on the number of in-block indexes on both sides of the position of the in-block index of the node to be inserted, the first in-block index set to be adjusted in the target data block after adding the in-block index of the node to be inserted is determined.

[0140] Further, when the response unit 1204 splits the target data block to obtain the first data block and the second data block, it is specifically used to:

[0141] Determine the insertion position of the node to be inserted in the plurality of nodes identified by the target data block number in the linked list of data to be processed;

[0142] Based on the position of the intra-block index of the node to be inserted, splitting the target data block after adding the intra-block index of the node to be inserted, to obtain the first data block and the second data block;

[0143] When the response unit 1204 performs response processing on the key value information of the to-be-processed data linked list recorded in the key value engine according to the first data block, the second data block, the target data block number and the target block internal index, it is specifically used to:

[0144] Based on the position of the index in the block of the node to be inserted, determine a set of data block numbers to be adjusted in the first data block and the second data block obtained by the splitting process, and adjust each data block number in the set of data block numbers;

[0145] Encode the block index of the node to be inserted and the adjusted data block number set, and update the encoding result to the key value engine;

[0146] In the updated key-value engine, the data of the node to be inserted is written into the node value in the key-value engine corresponding to the data block number set and the intra-block index of the node to be inserted.

[0147] Further, when the response unit 1204 performs response processing on the key value information of the to-be-processed data linked list recorded in the key value engine according to the target data block number and the index in the target block, it is specifically used to:

[0148] When the data processing request is used to request node deletion processing on the to-be-processed data linked list, determining, according to the target data block number and the target block index, to delete the target block index in the target data block corresponding to the target data block number;

[0149] Determine, according to the position of the target block intra-index in the target data block, a second block intra-index set to be adjusted in the target data block after deleting the target block intra-index, and adjust the value of each block intra-index in the second block intra-index set;

[0150] Deleting the key value information corresponding to the target data block number and the index in the target block in the key value engine;

[0151] Encode each intra-block index in the second intra-block index set after value adjustment, and update the encoding result to the key-value engine.

[0152] In an embodiment of the present application, a data processing device obtains a data processing request, which carries an object key value and a node number; determines the metadata of the data linked list to be processed corresponding to the object key value based on the key value information of each data linked list recorded in the key value engine; determines each data block number indicated by the metadata of the data linked list to be processed, and determines the target data block number and the target block index corresponding to the node number in each data block number; responds to the key value information of the data linked list to be processed recorded in the key value engine according to the target data block number and the target block index, so that the key value information to be processed can be determined more quickly and accurately, thereby improving the efficiency of data processing.

[0153] See also Fig.13 , Fig.13 1 is a schematic diagram of the structure of a computing device provided in an embodiment of the present application. As shown in the figure, the computing device in this embodiment may include: one or more processors 1301, memory 1302, bus 1303 and communication interface 1304, the processor 1301, communication interface 1304 and memory 1302 are connected through bus 1303; memory 1302 is used to store programs; processor 1301 is used to call the program stored in the above memory through bus 1303, and perform the following steps:

[0154] Obtaining a data processing request, wherein the data processing request carries an object key value and a node number;

[0155] Based on the key value information of each data linked list recorded in the key value engine, determining the meta information of the to-be-processed data linked list corresponding to the object key value;

[0156] Determine each data block number indicated by the meta information of the to-be-processed data linked list, and determine the target data block number and the target block index corresponding to the node number from among the data block numbers;

[0157] According to the target data block number and the index within the target block, response processing is performed on the key value information of the to-be-processed data linked list recorded in the key value engine.

[0158] Furthermore, the key value information of each data linked list recorded in the key value engine includes: object key value, node number and meta information, the meta information includes data block number, intra-block index and node value, wherein the object key value is used to identify a data linked list, the node number is used to identify a node in a data linked list, the data block number is used to identify a data block, a data block is used to indicate a node set in a data linked list, a node set includes multiple consecutive nodes, each intra-block index is used to identify a node in the data linked list indicated by a data block, and the node value refers to the data stored in a node in a data linked list.

[0159] Further, when the processor 1301 determines the target data block number and the target block index corresponding to the node number in each data block number, it is specifically configured to:

[0160] Determine the target data block number and the target block index corresponding to the node number according to the key-value mapping relationship included in the key-value information, wherein the key-value mapping relationship includes a mapping relationship among the node number, the data block number and the block index;

[0161] The response processing includes: performing any one or more of data query processing, data insertion processing, and data deletion processing on the target data block number and the node value corresponding to the index in the target block in the linked list of data to be processed recorded in the key-value engine.

[0162] Further, when the processor 1301 determines the target data block number and the target block index corresponding to the node number according to the key-value mapping relationship included in the key-value information, it is specifically configured to:

[0163] Acquire address information of each node in the linked list of data to be processed, where each address in the address information is used to indicate each node in the linked list of data to be processed;

[0164] Dividing the nodes of the to-be-processed data linked list according to the address information to obtain a plurality of data blocks, where different data blocks correspond to different data block numbers, and each data block includes an intra-block index for indicating different nodes;

[0165] According to the preset mapping relationship between the number and the address, the target address corresponding to the node number is determined, and the target data block number and the target block index corresponding to the target address are determined to be the target data block number and the target block index corresponding to the node number.

[0166] Further, when the processor 1301 performs response processing on the key value information of the to-be-processed data linked list recorded in the key value engine according to the target data block number and the index in the target block, it is specifically used to:

[0167] When the data processing request is used to request node insertion processing on the to-be-processed data linked list, estimating the block capacity of the target data block corresponding to the target data block number after inserting the intra-block index of the to-be-inserted node;

[0168] Determine, according to the target data block number and the target block internal index, an insertion position of the to-be-inserted node in a plurality of nodes identified by the target data block number in the to-be-processed data linked list;

[0169] If the block capacity meets the first capacity condition, responding to the key value information of the to-be-processed data linked list recorded in the key value engine according to the insertion position of the to-be-inserted node, the target data block number and the index within the target block;

[0170] If the block capacity meets the second capacity condition, the target data block is split to obtain a first data block and a second data block, and the key value information of the data linked list to be processed recorded in the key value engine is responded to based on the first data block, the second data block, the target data block number and the index within the target block.

[0171] Further, when the processor 1301 performs response processing on the key value information of the to-be-processed data linked list recorded in the key value engine according to the insertion position of the to-be-inserted node, the target data block number and the index within the target block, it is specifically used to:

[0172] Generate and add the intra-block index of the node to be inserted in the target data block based on the insertion position;

[0173] Determine, based on the insertion position, a first set of in-block indexes to be adjusted in the target data block after adding the in-block index of the node to be inserted, and adjust the value of each in-block index in the first set of in-block indexes;

[0174] Encode the in-block index of the node to be inserted and each in-block index in the first in-block index set after value adjustment, and update the encoding result to the key-value engine;

[0175] In the updated key-value engine, the data of the node to be inserted is written into the node value in the key-value engine corresponding to the target data block number and the intra-block index of the node to be inserted.

[0176] Further, when the processor 1301 performs response processing on the key value information of the to-be-processed data linked list recorded in the key value engine according to the insertion position of the to-be-inserted node, the target data block number and the index within the target block, it is specifically used to:

[0177] When the position of the in-block index of the node to be inserted is determined to be the target position of the target data block based on the insertion position, generating and adding the in-block index of the node to be inserted in the target data block based on the insertion position;

[0178] Encode the intra-block index of the node to be inserted, and update the encoding result to the key value engine;

[0179] In the updated key-value engine, the data of the node to be inserted is written into the node value in the key-value engine corresponding to the target data block number and the intra-block index of the node to be inserted.

[0180] Further, when the processor 1301 determines, based on the insertion position, a first set of in-block indexes to be adjusted in the target data block after adding the in-block index of the node to be inserted, it is specifically configured to:

[0181] Determine, based on the insertion position, the number of in-block indexes on both sides of the position of the in-block index of the node to be inserted in the target data block after adding the in-block index of the node to be inserted;

[0182] Based on the number of in-block indexes on both sides of the position of the in-block index of the node to be inserted, the first in-block index set to be adjusted in the target data block after adding the in-block index of the node to be inserted is determined.

[0183] Further, when the processor 1301 splits the target data block to obtain the first data block and the second data block, it is specifically configured to:

[0184] Determine the insertion position of the node to be inserted in the plurality of nodes identified by the target data block number in the linked list of data to be processed;

[0185] Based on the position of the intra-block index of the node to be inserted, splitting the target data block after adding the intra-block index of the node to be inserted, to obtain the first data block and the second data block;

[0186] When the processor 1301 performs response processing on the key value information of the to-be-processed data linked list recorded in the key value engine according to the first data block, the second data block, the target data block number and the target block internal index, it is specifically used to:

[0187] Based on the position of the index in the block of the node to be inserted, determine a set of data block numbers to be adjusted in the first data block and the second data block obtained by the splitting process, and adjust each data block number in the set of data block numbers;

[0188] Encode the block index of the node to be inserted and the adjusted data block number set, and update the encoding result to the key value engine;

[0189] In the updated key-value engine, the data of the node to be inserted is written into the node value in the key-value engine corresponding to the data block number set and the intra-block index of the node to be inserted.

[0190] Further, when the processor 1301 performs response processing on the key value information of the to-be-processed data linked list recorded in the key value engine according to the target data block number and the index in the target block, it is specifically used to:

[0191] When the data processing request is used to request node deletion processing on the to-be-processed data linked list, determining, according to the target data block number and the target block index, to delete the target block index in the target data block corresponding to the target data block number;

[0192] Determine, according to the position of the target block intra-index in the target data block, a second block intra-index set to be adjusted in the target data block after deleting the target block intra-index, and adjust the value of each block intra-index in the second block intra-index set;

[0193] Deleting the key value information corresponding to the target data block number and the index in the target block in the key value engine;

[0194] Encode each intra-block index in the second intra-block index set after value adjustment, and update the encoding result to the key-value engine.

[0195] In an embodiment of the present application, a computing device obtains a data processing request, which carries an object key value and a node number; determines the metadata of the data linked list to be processed corresponding to the object key value based on the key value information of each data linked list recorded in the key value engine; determines each data block number indicated by the metadata of the data linked list to be processed, and determines the target data block number and the target block index corresponding to the node number in each data block number; responds to the key value information of the data linked list to be processed recorded in the key value engine according to the target data block number and the target block index, so that the key value information to be processed can be determined more quickly and accurately, thereby improving the efficiency of data processing.

[0196] It should be understood that in the embodiment of the present application, the processor 1301 may be a central processing unit (CPU), and the processor may also be other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.

[0197] The memory 1302 may include a read-only memory and a random access memory, and provide instructions and data to the processor 1301. A portion of the memory 1302 may also include a non-volatile random access memory. For example, the memory 1302 may also store information about the device type.

[0198] In a specific implementation, the processor 1301 described in the embodiment of the present application can execute the implementation method described in the data processing method or text data processing method provided in the embodiment of the present application, and can also execute the implementation method of the data processing device or text data processing device described in the embodiment of the present application, which will not be repeated here.

[0199] An embodiment of the present application also provides a computer-readable storage medium, in which program instructions are stored, and when the program instructions are executed, any of the data processing methods described above is implemented.

[0200] The computer-readable storage medium may be an internal storage unit of the device described in any of the foregoing embodiments, such as a hard disk or memory of the device. The computer-readable storage medium may also be an external storage device of the device, such as a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), etc. equipped on the device. Furthermore, the computer-readable storage medium may also include both an internal storage unit of the device and an external storage device. The computer-readable storage medium is used to store the computer program and other programs and data required by the mobile device. The computer-readable storage medium may also be used to temporarily store data that has been output or is to be output.

[0201] The embodiments of the present application also provide a computer program product or a computer program, which includes computer instructions stored in a computer-readable storage medium. The processor of the computing device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computing device executes the methods provided in the above various embodiments.

[0202] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in terms of function in the above description. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0203] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the terminals and units described above can refer to the corresponding processes in the aforementioned method embodiments, and will not be repeated here.

[0204] In the several embodiments provided in the present application, it should be understood that the disclosed computing devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or units, or it can be an electrical, mechanical or other form of connection.

[0205] The steps in the method of the embodiment of the present application can be adjusted in order, combined and deleted according to actual needs.

[0206] The units in the terminal in the embodiment of the present application can be merged, divided and deleted according to actual needs.

[0207] The units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the embodiments of the present application.

[0208] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0209] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program codes.

[0210] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of various equivalent modifications or replacements within the technical scope disclosed in the present application, and these modifications or replacements should be included in the protection scope of the present application. Therefore, the protection scope of the present application shall be based on the protection scope of the claims.

Claims

1. A data processing method, characterized in that: include: Obtaining a data processing request, wherein the data processing request carries an object key value and a node number; Based on the key value information of each data linked list recorded in the key value engine, determining the meta information of the to-be-processed data linked list corresponding to the object key value; Determine each data block number indicated by the meta information of the to-be-processed data linked list, and determine the target data block number and the target block index corresponding to the node number from among the data block numbers; According to the target data block number and the index within the target block, response processing is performed on the key value information of the to-be-processed data linked list recorded in the key value engine.

2. The method according to claim 1, characterized in that The key value information of each data linked list recorded in the key value engine includes: object key value, node number and meta information, the meta information includes data block number, block index and node value, wherein the object key value is used to identify a data linked list, the node number is used to identify a node in a data linked list, the data block number is used to identify a data block, a data block is used to indicate a node set in a data linked list, a node set includes multiple consecutive nodes, each block index is used to identify a node in the data linked list indicated by a data block, and the node value refers to the data stored in a node in a data linked list.

3. The method according to claim 2, characterized in that The step of determining the target data block number and the target block index corresponding to the node number from among the data block numbers includes: Determine the target data block number and the target block index corresponding to the node number according to the key-value mapping relationship included in the key-value information, wherein the key-value mapping relationship includes a mapping relationship among the node number, the data block number and the block index; The response processing includes: performing any one or more of data query processing, data insertion processing, and data deletion processing on the target data block number and the node value corresponding to the index in the target block in the linked list of data to be processed recorded in the key-value engine.

4. The method according to claim 3, characterized in that The determining, according to the key-value mapping relationship included in the key-value information, a target data block number and an index within the target block corresponding to the node number, comprises: Acquire address information of each node in the linked list of data to be processed, where each address in the address information is used to indicate each node in the linked list of data to be processed; Dividing the nodes of the to-be-processed data linked list according to the address information to obtain a plurality of data blocks, where different data blocks correspond to different data block numbers, and each data block includes an intra-block index for indicating different nodes; According to the preset mapping relationship between the number and the address, the target address corresponding to the node number is determined, and the target data block number and the target block index corresponding to the target address are determined to be the target data block number and the target block index corresponding to the node number.

5. The method according to claim 2, characterized in that: The step of responding to the key value information of the to-be-processed data linked list recorded in the key value engine according to the target data block number and the index in the target block includes: When the data processing request is used to request node insertion processing on the to-be-processed data linked list, estimating the block capacity of the target data block corresponding to the target data block number after inserting the intra-block index of the to-be-inserted node; Determine, according to the target data block number and the target block internal index, an insertion position of the to-be-inserted node in a plurality of nodes identified by the target data block number in the to-be-processed data linked list; If the block capacity meets the first capacity condition, responding to the key value information of the to-be-processed data linked list recorded in the key value engine according to the insertion position of the to-be-inserted node, the target data block number and the index within the target block; If the block capacity meets the second capacity condition, the target data block is split to obtain a first data block and a second data block, and the key value information of the data linked list to be processed recorded in the key value engine is responded to based on the first data block, the second data block, the target data block number and the index within the target block.

6. The method according to claim 5, characterized in that The step of responding to the key value information of the to-be-processed data linked list recorded in the key value engine according to the insertion position of the to-be-inserted node, the target data block number and the index within the target block includes: Generate and add the intra-block index of the node to be inserted in the target data block based on the insertion position; Determine, based on the insertion position, a first set of in-block indexes to be adjusted in the target data block after adding the in-block index of the node to be inserted, and adjust the value of each in-block index in the first set of in-block indexes; Encode the in-block index of the node to be inserted and each in-block index in the first in-block index set after value adjustment, and update the encoding result to the key-value engine; In the updated key-value engine, the data of the node to be inserted is written into the node value in the key-value engine corresponding to the target data block number and the intra-block index of the node to be inserted.

7. The method according to claim 5, characterized in that The step of responding to the key value information of the to-be-processed data linked list recorded in the key value engine according to the insertion position of the to-be-inserted node, the target data block number and the index within the target block includes: When the position of the in-block index of the node to be inserted is determined to be the target position of the target data block based on the insertion position, generating and adding the in-block index of the node to be inserted in the target data block based on the insertion position; Encode the intra-block index of the node to be inserted, and update the encoding result to the key value engine; In the updated key-value engine, the data of the node to be inserted is written into the node value in the key-value engine corresponding to the target data block number and the intra-block index of the node to be inserted.

8. The method according to claim 6, characterized in that The determining, based on the insertion position, of a first set of in-block indexes to be adjusted in the target data block after adding the in-block index of the node to be inserted comprises: Determine, based on the insertion position, the number of in-block indexes on both sides of the position of the in-block index of the node to be inserted in the target data block after adding the in-block index of the node to be inserted; Based on the number of in-block indexes on both sides of the position of the in-block index of the node to be inserted, the first in-block index set to be adjusted in the target data block after adding the in-block index of the node to be inserted is determined.

9. The method according to claim 5, characterized in that The step of splitting the target data block to obtain a first data block and a second data block includes: Determine the insertion position of the node to be inserted in the plurality of nodes identified by the target data block number in the linked list of data to be processed; Based on the position of the intra-block index of the node to be inserted, splitting the target data block after adding the intra-block index of the node to be inserted, to obtain the first data block and the second data block; The step of responding to the key value information of the to-be-processed data linked list recorded in the key value engine according to the first data block, the second data block, the target data block number and the target block internal index includes: Based on the position of the index in the block of the node to be inserted, determine a set of data block numbers to be adjusted in the first data block and the second data block obtained by the splitting process, and adjust each data block number in the set of data block numbers; Encode the block index of the node to be inserted and the adjusted data block number set, and update the encoding result to the key value engine; In the updated key-value engine, the data of the node to be inserted is written into the node value in the key-value engine corresponding to the data block number set and the intra-block index of the node to be inserted.

10. The method according to claim 2, characterized in that The step of responding to the key value information of the to-be-processed data linked list recorded in the key value engine according to the target data block number and the index in the target block includes: When the data processing request is used to request node deletion processing on the to-be-processed data linked list, determining, according to the target data block number and the target block index, to delete the target block index in the target data block corresponding to the target data block number; Determine, according to the position of the target block intra-index in the target data block, a second block intra-index set to be adjusted in the target data block after deleting the target block intra-index, and adjust the value of each block intra-index in the second block intra-index set; Deleting the key value information corresponding to the target data block number and the index in the target block in the key value engine; Encode each intra-block index in the second intra-block index set after value adjustment, and update the encoding result to the key-value engine.

11. A data processing device, characterized in that: include: An acquisition unit, used for acquiring a data processing request, wherein the data processing request carries an object key value and a node number; A first determining unit, configured to determine the meta information of the to-be-processed data linked list corresponding to the object key value based on the key value information of each data linked list recorded in the key value engine; A second determining unit is used to determine the data block numbers indicated by the meta information of the to-be-processed data linked list, and determine the target data block number and the target block index corresponding to the node number from the data block numbers; A response unit is used to respond to the key value information of the to-be-processed data linked list recorded in the key value engine according to the target data block number and the index within the target block.

12. A computing device, characterized in that: The method comprises a processor and a memory, wherein the processor and the memory are connected to each other, wherein the memory is used to store a computer program, and the processor is configured to call the computer program to execute the method according to any one of claims 1 to 10.

13. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores program instructions, which, when executed, implement the method according to any one of claims 1 to 10.

14. A computer program product, characterized in that The computer program product comprises program instructions, and when the program instructions are executed by a processor, the method according to any one of claims 1 to 10 is implemented.

Citation Information

Patent Citations

  • Realization method and apparatus for data structure of key value storage system

    CN107918612A

  • Space keyword indexing method and platform and storage medium

    CN108628969A

  • Data processing method and device, data node and storage medium

    CN112214175A

  • Ordered linked list quick query method and device based on range diagram

    CN112632087A

  • Data message storage method and device and computer equipment

    CN114489505A