A data acquisition method, apparatus, device, and storage medium

Abstract

This invention discloses a data acquisition method, apparatus, device, and storage medium. The method includes: upon receiving a first request, traversing the parent nodes of a tree structure based on a first data identifier carried by the first request to obtain the physical address of the target child node corresponding to the first request; traversing the metadata of the target child node based on the physical address of the target child node and the first data identifier to obtain the physical address of the target data row; and reading the data of the target data row based on the physical address of the target data row. This invention solves the problem that accessing the data content of a data row requires reading multiple layers of logical and physical addresses, increasing the overhead of indirect address calculations during memory access, making the entire access path very complex, leading to memory access latency and a decrease in overall system performance. This invention reduces the complexity of the memory access path.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and in particular to a data acquisition method, apparatus, device and storage medium. Background Technology

[0002] To enhance parallel processing capabilities under diverse database load scenarios, most database systems currently employ multi-version technology. By maintaining multiple versions of physical data rows for each logical record in the database, read and write operations on the same logical record can run under their respective snapshot versions of data without blocking each other.

[0003] For organizing and storing multiple versions of physical data rows, most existing in-memory database systems use a heap-based data organization method. Generally, all data rows, including both new and old versions of the record, are stored in the same storage space. Whenever a data update request is made, the system first requests storage space for a new data row in the data table, then copies the latest version of the data row to the new data row space, and finally writes the changes to this new data row. A new-to-old-version approach is commonly used to maintain multiple versions of each record; that is, a linked list is used to arrange the version data rows from newest to oldest, and the required version data can be accessed by traversing the linked list.

[0004] The data organization form based on heap structure has the following drawbacks: when accessing the data content of a data row, it is necessary to read through multiple layers of logical and physical addresses, which greatly increases the overhead of indirect address calculation during memory access, making the entire access path very complex, thus leading to memory access latency and a decrease in overall system performance. Summary of the Invention

[0005] This invention provides a data acquisition method, apparatus, device, and storage medium that can solve the problem that when accessing the data content of a data row, it is necessary to read multiple layers of logical and physical addresses, which increases the overhead of indirect address calculation during memory access, making the entire access path very complex, resulting in memory access latency and overall system performance degradation. By storing the data of the data row in the child nodes of a tree structure, the complexity of the memory access path is reduced.

[0006] According to one aspect of the present invention, a data acquisition method is provided, comprising:

[0007] When the first request is received, the parent nodes of the tree structure are traversed according to the first data identifier carried by the first request to obtain the physical address of the target child node corresponding to the first request.

[0008] The physical address of the target data row is obtained by traversing the metadata of the target child node based on the physical address of the target child node and the first data identifier.

[0009] Read the data of the target data row based on its physical address.

[0010] According to another aspect of the present invention, a data acquisition apparatus is provided, the data acquisition apparatus comprising:

[0011] The first acquisition module is used to, when receiving a first request, traverse the parent nodes of the tree structure according to the first data identifier carried by the first request to obtain the physical address of the target child node corresponding to the first request.

[0012] The second acquisition module is used to traverse the metadata of the target child node according to the physical address of the target child node and the first data identifier to obtain the physical address of the target data row.

[0013] The reading module is used to read the data of the target data row according to the physical address of the target data row.

[0014] According to another aspect of the present invention, an electronic device is provided, the electronic device comprising:

[0015] At least one processor; and

[0016] A memory communicatively connected to the at least one processor; wherein,

[0017] The memory stores a computer program that can be executed by the at least one processor, the computer program being executed by the at least one processor to enable the at least one processor to perform the data acquisition method according to any embodiment of the present invention.

[0018] According to another aspect of the present invention, a computer-readable storage medium is provided, the computer-readable storage medium storing computer instructions for causing a processor to execute and implement the data acquisition method described in any embodiment of the present invention.

[0019] This invention addresses the problem that accessing data in a data row requires traversing multiple layers of logical and physical addresses, increasing the overhead of indirect address calculations during memory access, making the entire access path complex, and thus causing memory access latency and overall system performance degradation. By storing the data in the child nodes of the tree structure when a first request is received, the complexity of the memory access path is reduced.

[0020] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of the present invention, nor is it intended to limit the scope of the invention. Other features of the invention will become readily apparent from the following description. Attached Figure Description

[0021] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of the present invention and should not be regarded as a limitation on the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0022] Figure 1 This is a flowchart of a data acquisition method according to an embodiment of the present invention;

[0023] Figure 2 This is a schematic diagram of the update and query operations in an embodiment of the present invention;

[0024] Figure 3 This is a schematic diagram of the structure of a data acquisition device according to an embodiment of the present invention;

[0025] Figure 4 This is a schematic diagram of the structure of an electronic device according to an embodiment of the present invention. Detailed Implementation

[0026] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0027] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0028] Example 1

[0029] Figure 1 This is a flowchart illustrating a data acquisition method provided in an embodiment of the present invention. This embodiment is applicable to data acquisition situations. The method can be executed by a data acquisition device in this embodiment, which can be implemented in software and / or hardware, such as... Figure 1 As shown, the method specifically includes the following steps:

[0030] S110, when the first request is received, the parent nodes of the tree structure are traversed according to the first data identifier carried by the first request to obtain the physical address of the target child node corresponding to the first request.

[0031] The tree structure includes parent nodes and child nodes. The structure of the parent node is identical to that of the child nodes. The parent node stores data identifiers and the physical addresses of the child nodes. The child nodes store node attribute information, metadata of the data rows, and record data for each data row. The tree structure is built based on the target table. It should be noted that after building the tree structure based on the target table, it can be updated via insertion requests. Specifically, the tree structure can be built as follows: based on the target table, according to the insertion request, the data row to be inserted is inserted into the corresponding child node, and simultaneously, the attribute information of the child node, the metadata of the data rows in the child node, and the record data of each data row are updated. During the data insertion process, the update flag in the metadata needs to be modified to the first value.

[0032] The first request can be a query request, a deletion request, or an update request; this embodiment of the invention does not impose any restrictions on this. Specifically, if the first request is a query request, it carries a first data identifier and query range information; if the first request is an update request, it carries a first data identifier and target data; and if the first request is a deletion request, it carries a first data identifier.

[0033] Optionally, based on the first data identifier carried in the first request, the parent nodes of the tree structure are traversed to obtain the physical address of the target child node corresponding to the first request, including:

[0034] If the first data identifier is less than or equal to the data identifier corresponding to the parent node, then query the data identifier corresponding to the left child node of the parent node based on the first data identifier;

[0035] If the data identifier corresponding to the left child node of the parent node is the same as the first data identifier, then obtain the physical address of the child node corresponding to the data identifier that is the same as the first data identifier;

[0036] The physical address of the child node corresponding to the data identifier that is the same as the first data identifier is determined as the physical address of the target child node corresponding to the first request.

[0037] The parent node includes: a data identifier, a data identifier corresponding to the left child node of the parent node, a physical address of the left child node of the parent node, a data identifier corresponding to the right child node of the parent node, and a physical address of the right child node of the parent node. The data identifier corresponding to the left child node of the parent node is less than or equal to the data identifier corresponding to the parent node, and the data identifier corresponding to the right child node of the parent node is greater than the data identifier corresponding to the parent node.

[0038] For example, if the parent node's key value is 4, and the first data identifier carried in the first request is key=7, the parent node includes: the left child node A of the parent node with key value 3, the physical address corresponding to the left child node A is X; the left child node B of the parent node with key value 2, the physical address corresponding to the left child node A is y; the right child node C of the parent node with key value 5, the physical address corresponding to the left child node A is z; the left child node D of the parent node with key value 6, the physical address corresponding to the left child node A is r; and the left child node E of the parent node with key value 7, the physical address corresponding to the left child node A is u. Since 7 is greater than 4, the right child node of the parent node is queried, and the left child node E of the parent node has key value 7, the physical address corresponding to the left child node A is u. Therefore, the target child node corresponding to the first request is determined to be child node E, and the physical address of the target child node is physical address u.

[0039] S120: Based on the physical address of the target child node and the first data identifier, traverse the metadata of the target child node to obtain the physical address of the target data row.

[0040] It should be noted that in a multi-version database system, one logical record in a data table corresponds to the storage of multiple physical data rows.

[0041] The child nodes include: node attribute information, metadata of data rows and record data of each data row. For example, the child nodes may include: (1) child node metadata header (total space size of child nodes and current space size used); (2) data row metadata array, which stores the meta of each data row and the physical address pointing to the previous version. The meta is a 64-bit integer value, which includes: 1 bit indicating whether the current data row is being updated, 1 bit indicating whether the current data row is visible, 14 bits storing the length of the key value, 16 bits storing the offset of the data row, and 32 bits storing the transaction ID that created the data row; (3) the entire data content of each data row.

[0042] The data organization method provided in this invention stores the latest data row content of a data table record in leaf nodes, reducing memory access path complexity. Different versions of data rows are categorized and stored separately using pointers, reducing access contention in centralized storage structures. Furthermore, each data row corresponds to metadata, maintaining different metadata information for different categories of data rows, reducing version storage overhead.

[0043] Specifically, the method for obtaining the physical address of the target data row by traversing the metadata of the target child node based on the physical address of the target child node and the first data identifier can be as follows: read the metadata of the target child node based on the physical address of the target node, query the target data row corresponding to the data identifier that is the same as the first data identifier in the metadata of the target child node based on the first data identifier, and obtain the physical address of the target data row.

[0044] In a specific example, if the first request is a query request: find all field values ​​of the employee with ID=7. First, traverse the parent nodes of the tree structure based on the ID value in the query request to find the physical address of the target child node; then, traverse the meta tags of the data rows in the target child node using a binary search method; next, calculate the physical location of the data row content based on the offset in the meta tag, thereby reading the ID in the data row content and comparing it with ID=7, until the target data row with ID=7 is found, and obtain the physical address of the target data row.

[0045] By leveraging the ordered nature of the data row storage, the entire data row can be directly read from its child nodes simply by traversing the index based on the data identifier, without requiring multiple layers of indirect address access. Therefore, this data organization method proposed in this embodiment of the invention can save significant unnecessary memory indirect addressing costs, thus providing a substantial performance advantage during retrieval.

[0046] S130, Read the data of the target data row according to the physical address of the target data row.

[0047] The data in the target data row refers to the record data of the target data row. For example, if the target data row is the data row with ID=7, then all field values ​​of the employee with ID=7 are read.

[0048] Optionally, the first request is a query request;

[0049] After reading the data of the target data row according to its physical address, the process further includes:

[0050] The data from the target data row will be returned as the query result.

[0051] Specifically, the data of the target data row is returned as the query result. For example, if the target data row is the data row with ID=7, then all field values ​​of the employee with ID=7 are read and all field values ​​of the employee with ID=7 are returned as the query result.

[0052] Optionally, before obtaining the physical address of the target child node corresponding to the first request by traversing the parent nodes of the tree structure according to the first data identifier carried by the first request upon receiving the first request, the method further includes:

[0053] Obtain the target table;

[0054] A tree structure is constructed based on the data identifiers in the target table. The tree structure includes parent nodes and child nodes. The child nodes include node attribute information, metadata of the data rows, and record data of each data row. The parent nodes include data identifiers and the physical addresses of the child nodes.

[0055] The space size of each child node can be a fixed value, for example, 64KB per child node. The child nodes are divided into three areas: 1) Leaf node metadata header; 2) Data row metadata array, storing the meta tag of each data row and the physical address pointing to the previous version. The meta tag is a 64-bit integer value containing: 1 bit indicating whether the current data row is being updated, 1 bit indicating whether the current data row is visible, 14 bits storing the length of the key value, 16 bits storing the offset of the data row, and 32 bits storing the transaction ID that created the data row; 3) The entire data content of each data row.

[0056] In a specific example, an employee data table named "employee" contains attribute fields such as employee ID (length 4), employee name (length 20), salary (length 4), and address (length 32). The latest version of all records in the table is stored in a tree structure constructed from employee IDs. The parent node of the tree structure stores the ID value and address pointer, while the child nodes store the entire row of data.

[0057] Optionally, the first request is an update request, and the target table includes: attribute field length information;

[0058] Accordingly, after reading the data of the target data row based on its physical address, the process further includes:

[0059] Create a temporary space;

[0060] The target data row is stored in the temporary space;

[0061] Add a write lock to the data in the target data row, and modify the pointer of the previous version in the metadata of the target child node to point to a temporary space address;

[0062] Update the data in the target data row according to the attribute field length information and the target data carried in the update request;

[0063] Release the write lock on the data for the target data row.

[0064] The attribute fields are elements in the target table, and the length information of the attribute fields is the length of each element. For example, if the target table is an employee data table, it contains attribute fields such as employee ID, employee name, salary, and address. The length of employee ID is 4, the length of employee name is 20, the length of salary is 4, and the length of address is 32.

[0065] The temporary space is a temporary version buffer. The pointer in the metadata is initially null. After updating the data stored in the child nodes, the pointer is modified to the address of the temporary space where the data before the update was stored.

[0066] One method for adding a write lock to the data in the target data row is to change the update flag in the metadata to a first value using an atomic instruction operation.

[0067] The first value can be a preset value, and the first value is different from the second value. For example, the first value can be 1.

[0068] One method to release the write lock on the target data row is to change the update flag in the metadata to a second value using an atomic instruction operation.

[0069] The second value is a preset value, for example, it can be 0.

[0070] Specifically, the method for updating the target data row based on the attribute field length information and the target data carried in the update request can be as follows: determine the offset of the data to be changed based on the attribute field length information, and update the data to be changed to the target data based on the offset of the data to be changed. For example, the update request could be: change the salary of the employee with ID=5 to 5200. The target table is an employee data table, which contains attribute fields such as employee ID (length is 4), employee name (length is 20), salary (length is 4), and address (length is 32). Based on the attribute field length information, the offset of the data to be changed is determined to be 24, and the original salary field value is modified to 5200.

[0071] In a specific example, if the first request is an update request, such as changing the salary of the employee with ID=5 to 5200.

[0072] 1) First, following the same query request process as before, find the data content of data row ID=5;

[0073] 2) Secondly, the entire record data content is migrated to a temporary version buffer, and the metadata of concurrent read transactions' operations on it during write operations is tracked and maintained for serialization verification when subsequent transactions are committed.

[0074] 3) Next, by using an atomic instruction to change the update flag in the data row's meta to 1, a write lock is acquired on the data content of that data row. After that, other concurrent write operations will read the metadata flag and return failure.

[0075] 4) Then, based on the length of the attribute field, calculate the offset of the changed content as 24, and change the salary field value to 5200 in place.

[0076] 5) Finally, after the update transaction is successfully committed, an atomic instruction operation is used to change the update flag in the data row's meta tag to 0, thereby releasing the write lock on that data row. Afterward, the data row becomes a new version and is visible to other transactions. Before this, a migration operation for the old data row needs to be completed. That is, the data content of the data row in the buffer is migrated to the retired version data block using memory copying, and the pointer of the previous version (old version) of the data row is set to the retired version.

[0077] Throughout the process, the update operation does not affect other concurrent transactions' access to the latest version of the record's data row. When there is a concurrent read request, the previous version pointer of the data row can be used to direct access to the latest version of the cached data placed in the buffer. Additionally, a timed thread is used to clean up the cache to speed up the retrieval of data items in the buffer. Therefore, although the data organization method proposed in this embodiment of the invention may introduce some migration costs to the system, actual experimental results show that it can improve overall performance under diverse database load scenarios.

[0078] Optional, also includes:

[0079] When an insertion request is received, the parent nodes of the tree structure are traversed according to the second data identifier of the data row to be inserted carried in the insertion request to obtain the physical address of the first child node corresponding to the insertion request.

[0080] If there is no data row in the first child node that is identified by the second data identifier, then a data item is constructed, wherein the data item is used to store the metadata of the data row corresponding to the second data identifier;

[0081] Modify the update flag bit in the metadata of the data row corresponding to the second data identifier to the first value;

[0082] The offset of the data row to be inserted is determined based on the node attribute information of the first child node;

[0083] Insert the data corresponding to the data row to be inserted into the first child node according to the offset of the data row to be inserted, and update the metadata according to the offset of the data row to be inserted.

[0084] Modify the update flag bit in the metadata of the data row corresponding to the second data identifier to the second value.

[0085] Specifically, if there is no data row with the second data identifier in the first child node, the data item can be constructed as follows: traverse all data rows in the child node, determine whether the child node already has a data row with the second data identifier, and if it already exists, return an insertion failure because the data is duplicated; if the child node does not already have a data row with the second data identifier, then obtain a new data item from the metadata array of the child node.

[0086] In a specific example, when an insert request is received, for instance: insert employee information with ID=7.

[0087] First, traverse the tree structure based on the ID value of the query request to find the physical address of the child node to be inserted;

[0088] Then, iterate through all the data rows in the child node and check if the child node already has a data row with ID=7. If it already exists, return an insertion failure because the data is duplicated.

[0089] If the child node does not have a record with ID=7, a new data item is retrieved from the child node's metadata array to store the metadata of the data row with ID=7. The update flag in the metadata is initialized to 1, a write lock is acquired on the data row, and the data row is temporarily not visible to other transactions.

[0090] Next, obtain the total space size and currently used space size of the node from the node header, and calculate the offset of the data row to be inserted;

[0091] Next, copy the content of the data row with ID=7 to the offset, and update the offset value of the data row to the metadata;

[0092] Finally, after the insert transaction is successfully committed, an atomic instruction operation modifies the update flag in the metadata to 0, thereby releasing the write lock on the data row. After that, the data row becomes the latest version and is visible to other transactions.

[0093] Optionally, the offset of the data row to be inserted is determined based on the node attribute information of the first child node, including:

[0094] The total space size and used space size corresponding to the first child node are determined based on the node attribute information of the first child node;

[0095] The offset of the data row to be inserted is determined based on the total space size and the used space size corresponding to the first child node.

[0096] The node attribute information of the first child node includes: the total space size and the used space size corresponding to the first child node.

[0097] Specifically, the method for determining the offset of the data row to be inserted based on the total space size and the used space size corresponding to the first child node can be as follows: determine the offset of the data row to be inserted based on the difference between the total space size and the used space size corresponding to the first child node.

[0098] Optionally, the first request is a deletion request;

[0099] After reading the data of the target data row according to its physical address, the process further includes:

[0100] Create a temporary space;

[0101] The target data row is stored in the temporary space;

[0102] Add a write lock to the data in the target data row, and modify the pointer of the previous version in the metadata of the target child node to point to a temporary space address;

[0103] Modify the visibility flag in the metadata of the target data row to the second value;

[0104] Release the write lock on the data for the target data row.

[0105] The temporary space is a temporary version buffer.

[0106] The pointer in the metadata is initially null. After modifying the data stored in the child node, the pointer is changed to the temporary space address where the data before the modification was stored.

[0107] One method for adding a write lock to the data in the target data row is to change the update flag in the metadata to a first value using an atomic instruction operation.

[0108] The first value can be a preset value, and the first value is different from the second value. For example, the first value can be 1.

[0109] The second value is a preset value, for example, it can be 0.

[0110] One method to release the write lock on the target data row is to change the update flag in the metadata to a second value using an atomic instruction operation.

[0111] In a specific example, if the first request is a delete request, such as deleting employee information with ID=6.

[0112] First, following the same query request process as before, the data content with data row ID=6 is found;

[0113] Secondly, the data content of the entire data row is migrated to a temporary version buffer, and the metadata of concurrent read transactions' operations on it during the deletion operation is tracked and maintained for serialization verification when subsequent transactions are committed.

[0114] Next, an atomic instruction is used to change the update flag in the metadata to 1, and a write lock is acquired on the data content of the data row. After that, other concurrent write operations will read the metadata flag and return failure. The pointer of the metadata pointing to the previous version is modified to point to a temporary space address. Then, for read operation requests on the data row before the update operation is successfully committed, the data content stored in the temporary space will be returned. At the same time, the transaction information of these read operations will be recorded to provide serialization verification for the subsequent transaction commit phase.

[0115] Finally, after the update transaction is successfully committed, the metadata (meta) is modified to 0 through an atomic instruction operation, thereby logically deleting the data row.

[0116] When the active version tree is merging, if the metadata (meta) of the data row is read as 0, the memory space occupied by the data row will be cleaned up (i.e., physically deleted). When the system memory space reclamation thread detects that the transaction of the data row in the version buffer has expired, it will clean up the memory space occupied by the data row (i.e., physically delete).

[0117] This invention employs a performance-optimized Bztree structure to organize and store the latest version data rows of all records in a data table. Each leaf node of the tree is a fixed-size data block. Within each block, data rows are organized according to a slot format, including fixed-byte node metadata, a dynamically adjusted array to store the metadata of each data row, and the entire record data of each data row placed according to an offset. Secondly, for data rows currently being updated, they are stored in a global version cache, and their associated transaction read / write operation information is maintained. Then, for data rows that will no longer require updates, a sequential append method is used to store the data rows in multiple fixed-size contiguous memory blocks, maintaining simple snapshot timestamp information for version retrieval. Therefore, each time a data row is updated, the latest data row stored in the leaf node is moved to the buffer, and the changed content is directly modified in-place on the leaf node. Although this method incurs some memory copying overhead, it does not affect other concurrent transactions' access to the data row.

[0118] This invention categorizes data rows based on the table version lifecycle and stores them separately in multiple storage spaces, thereby reducing data access contention. Simultaneously, it maintains applicable metadata information for different categories of data rows according to the diverse needs of database workloads, saving memory space. Furthermore, the execution engine can directly read the entire data content of a data row by traversing the leaf nodes, greatly simplifying the memory data access path and thus improving the overall performance of the system in handling diverse database workloads.

[0119] In a specific example, if the first request includes: a query request (query all fields of the employee with ID=7) and an update request (update the salary of the employee with ID=5 to 5200). Figure 2 As shown, the parent node's ID is 4. The left child nodes of the parent node include: child nodes with ID=4, ID=3, ID=2, and ID=1. The right child nodes of the parent node include: child nodes with ID=5, ID=6, and ID=7. Since 7 is greater than 4, the right child nodes of the parent node are queried to obtain the physical address of the child node with ID=7, thus retrieving all fields of the employee with ID=7. Since 5 is greater than 4, the right child nodes of the parent node are queried to obtain the physical address of the child node with ID=5, thus retrieving all fields of the employee with ID=5. All fields of the employee with ID=5 are then stored in the version buffer, and the salary of the employee with ID=5 in the child node is updated to 5200. The retired version data block stores retired version data, for example... Figure 2The retired version data block stores all fields of employee ID=5 and all fields of employee ID=7.

[0120] The technical solution of this embodiment solves the problem that when a first request is received, the parent nodes of the tree structure are traversed according to the first data identifier carried by the first request to obtain the physical address of the target child node corresponding to the first request; the metadata of the target child node is traversed according to the physical address of the target child node and the first data identifier to obtain the physical address of the target data row; and the data of the target data row is read according to the physical address of the target data row. This solves the problem that when accessing the data content of a data row, it is necessary to read multiple layers of logical and physical addresses, which increases the overhead of indirect address calculation during memory access, making the entire access path very complex, resulting in memory access latency and overall system performance degradation. By storing the data of the data row in the child nodes of the tree structure, the complexity of the memory access path is reduced.

[0121] Example 2

[0122] Figure 3 This is a schematic diagram of a data acquisition device provided in an embodiment of the present invention. This embodiment is applicable to data acquisition applications. The device can be implemented using software and / or hardware, and can be integrated into any device that provides data acquisition functionality, such as… Figure 3 As shown, the data acquisition device specifically includes: a first acquisition module 310, a second acquisition module 320, and a reading module 330.

[0123] The first acquisition module is used to, when receiving the first request, traverse the parent nodes of the tree structure according to the first data identifier carried by the first request to obtain the physical address of the target child node corresponding to the first request.

[0124] The second acquisition module is used to traverse the metadata of the target child node according to the physical address of the target child node and the first data identifier to obtain the physical address of the target data row.

[0125] The reading module is used to read the data of the target data row according to the physical address of the target data row.

[0126] The above-described products can perform the methods provided in any embodiment of the present invention, and have the corresponding functional modules and beneficial effects for performing the methods.

[0127] The technical solution of this embodiment solves the problem that when a first request is received, the parent nodes of the tree structure are traversed according to the first data identifier carried by the first request to obtain the physical address of the target child node corresponding to the first request; the metadata of the target child node is traversed according to the physical address of the target child node and the first data identifier to obtain the physical address of the target data row; and the data of the target data row is read according to the physical address of the target data row. This solves the problem that when accessing the data content of a data row, it is necessary to read multiple layers of logical and physical addresses, which increases the overhead of indirect address calculation during memory access, making the entire access path very complex, resulting in memory access latency and overall system performance degradation. By storing the data of the data row in the child nodes of the tree structure, the complexity of the memory access path is reduced.

[0128] Example 3

[0129] Figure 4 A schematic diagram of an electronic device 10 that can be used to implement embodiments of the present invention is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices (e.g., helmets, glasses, watches, etc.), and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the invention described and / or claimed herein.

[0130] like Figure 4 As shown, the electronic device 10 includes at least one processor 11 and a memory, such as a read-only memory (ROM) 12 or a random access memory (RAM) 13, communicatively connected to the at least one processor 11. The memory stores computer programs executable by the at least one processor. The processor 11 can perform various appropriate actions and processes based on the computer program stored in the ROM 12 or loaded from storage unit 18 into the RAM 13. The RAM 13 may also store various programs and data required for the operation of the electronic device 10. The processor 11, ROM 12, and RAM 13 are interconnected via a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0131] Multiple components in electronic device 10 are connected to I / O interface 15, including: input unit 16, such as keyboard, mouse, etc.; output unit 17, such as various types of displays, speakers, etc.; storage unit 18, such as disk, optical disk, etc.; and communication unit 19, such as network card, modem, wireless transceiver, etc. Communication unit 19 allows electronic device 10 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0132] Processor 11 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, digital signal processors (DSPs), and any suitable processor, controller, microcontroller, etc. Processor 11 performs the various methods and processes described above, such as data acquisition methods.

[0133] In some embodiments, the data acquisition method may be implemented as a computer program tangibly contained in a computer-readable storage medium, such as storage unit 18. In some embodiments, part or all of the computer program may be loaded and / or mounted on electronic device 10 via ROM 12 and / or communication unit 19. When the computer program is loaded into RAM 13 and executed by processor 11, one or more steps of the data acquisition method described above may be performed. Alternatively, in other embodiments, processor 11 may be configured to perform the data acquisition method by any other suitable means (e.g., by means of firmware).

[0134] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), payload-programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0135] Computer programs used to implement the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, such that when executed by the processor, the computer programs cause the functions / operations specified in the flowcharts and / or block diagrams to be performed. The computer programs may be executed entirely on a machine, partially on a machine, or as a standalone software package, partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0136] In the context of this invention, a computer-readable storage medium can be a tangible medium that may contain or store a computer program for use by or in conjunction with an instruction execution system, apparatus, or device. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination thereof. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0137] To provide interaction with a user, the systems and techniques described herein can be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the electronic device. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0138] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as data servers), or computing systems that include middleware components (e.g., application servers), or computing systems that include frontend components (e.g., user computers with graphical user interfaces or web browsers through which users can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., communication networks). Examples of communication networks include local area networks (LANs), wide area networks (WANs), blockchain networks, and the Internet.

[0139] A computing system can include clients and servers. Clients and servers are generally located far apart and typically interact through communication networks. The client-server relationship is created by computer programs running on the respective computers and having a client-server relationship with each other. The server can be a cloud server, also known as a cloud computing server or cloud host, which is a hosting product within the cloud computing service system to address the shortcomings of traditional physical hosts and VPS services, such as high management difficulty and weak business scalability.

[0140] It should be understood that the various forms of processes shown above can be used, with steps reordered, added, or deleted. For example, the steps described in this invention can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution of this invention can be achieved, and this is not limited herein.

[0141] The specific embodiments described above do not constitute a limitation on the scope of protection of this invention. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this invention should be included within the scope of protection of this invention.

Claims

1. A data acquisition method, characterized in that, include: When a first request is received, the parent nodes of the tree structure are traversed according to the first data identifier carried by the first request to obtain the physical address of the target child node corresponding to the first request; wherein, the tree structure includes parent nodes and child nodes, the structure of the parent node is the same as the structure of the child node, and the content stored in the parent node includes: data identifier and physical address of child node; the content stored in the child node includes: node attribute information, metadata of data row and record data of each data row; The metadata of the target child node is read according to the physical address of the target child node. The target data row corresponding to the data identifier that is the same as the first data identifier in the metadata of the target child node is queried according to the first data identifier, and the physical address of the target data row is obtained. Read the data of the target data row according to the physical address of the target data row, wherein the data of the target data row is the record data of the target data row; The method further includes: When an insertion request is received, the parent nodes of the tree structure are traversed according to the second data identifier of the data row to be inserted carried in the insertion request to obtain the physical address of the first child node corresponding to the insertion request. If there is no data row in the first child node that is identified by the second data identifier, then a data item is constructed, wherein the data item is used to store the metadata of the data row corresponding to the second data identifier; Modify the update flag bit in the metadata of the data row corresponding to the second data identifier to the first value; The offset of the data row to be inserted is determined based on the node attribute information of the first child node, wherein the node attribute information of the first child node includes: the total space size and the used space size corresponding to the first child node; Insert the data corresponding to the data row to be inserted into the first child node according to the offset of the data row to be inserted, and update the metadata according to the offset of the data row to be inserted. Modify the update flag bit in the metadata of the data row corresponding to the second data identifier to the second value.

2. The method according to claim 1, characterized in that, The first request is a query request; After reading the data of the target data row according to its physical address, the process further includes: The data from the target data row will be returned as the query result.

3. The method according to claim 1, characterized in that, Before obtaining the physical address of the target child node corresponding to the first request by traversing the parent nodes of the tree structure according to the first data identifier carried by the first request upon receiving the first request, the process further includes: Obtain the target table; Construct a tree structure based on the data identifiers in the target table.

4. The method according to claim 3, characterized in that, The first request is an update request, and the target table includes: attribute field length information; Accordingly, after reading the data of the target data row based on its physical address, the process further includes: Create a temporary space; The target data row is stored in the temporary space; Add a write lock to the data in the target data row, and modify the pointer of the previous version in the metadata of the target child node to point to a temporary space address; Update the data in the target data row according to the attribute field length information and the target data carried in the update request; Release the write lock on the data for the target data row.

5. The method according to claim 1, characterized in that, The offset of the data row to be inserted is determined based on the node attribute information of the first child node, including: The total space size and used space size corresponding to the first child node are determined based on the node attribute information of the first child node; The offset of the data row to be inserted is determined based on the total space size and the used space size corresponding to the first child node.

6. The method according to claim 1, characterized in that, The first request is a deletion request; After reading the data of the target data row according to its physical address, the process further includes: Create a temporary space; The target data row is stored in the temporary space; Add a write lock to the data in the target data row, and modify the pointer of the previous version in the metadata of the target child node to point to a temporary space address; Modify the visibility flag in the metadata of the target data row to the second value; Release the write lock on the data for the target data row.

7. A data acquisition device, characterized in that, include: The first acquisition module is used to, upon receiving a first request, traverse the parent nodes of a tree structure according to the first data identifier carried by the first request to obtain the physical address of the target child node corresponding to the first request; wherein, the tree structure includes parent nodes and child nodes, the structure of the parent node is the same as the structure of the child node, and the content stored in the parent node includes: data identifier and physical address of the child node; the content stored in the child node includes: node attribute information, metadata of data rows, and record data of each data row; The second acquisition module is used to read the metadata of the target child node according to the physical address of the target child node, query the target data row corresponding to the data identifier that is the same as the first data identifier in the metadata of the target child node according to the first data identifier, and obtain the physical address of the target data row; The reading module is used to read the data of the target data row according to the physical address of the target data row, wherein the data of the target data row is the record data of the target data row; The data acquisition device is further configured to: when receiving an insertion request, traverse the parent nodes of the tree structure according to the second data identifier of the data row to be inserted carried in the insertion request, and obtain the physical address of the first child node corresponding to the insertion request; If there is no data row in the first child node that is identified by the second data identifier, then a data item is constructed, wherein the data item is used to store the metadata of the data row corresponding to the second data identifier; Modify the update flag bit in the metadata of the data row corresponding to the second data identifier to the first value; The offset of the data row to be inserted is determined based on the node attribute information of the first child node, wherein the node attribute information of the first child node includes: the total space size and the used space size corresponding to the first child node; Insert the data corresponding to the data row to be inserted into the first child node according to the offset of the data row to be inserted, and update the metadata according to the offset of the data row to be inserted. Modify the update flag bit in the metadata of the data row corresponding to the second data identifier to the second value.

8. An electronic device, characterized in that, The electronic device includes: At least one processor; and A memory communicatively connected to the at least one processor; wherein, The memory stores a computer program that can be executed by the at least one processor, the computer program being executed by the at least one processor to enable the at least one processor to perform the data acquisition method according to any one of claims 1-6.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions that are used to cause a processor to execute the data acquisition method according to any one of claims 1-6.

Images