Data processing method, electronic device and storage medium

Abstract

The present invention provides a data processing method, electronic device, and storage medium. In the present invention, an index tree scan method is used in conjunction with a mechanism for randomly sampling data pages of a database table. This method eliminates the need for a full table scan of the target database table when obtaining randomly sampled data from the target database table. This effectively reduces the number of disk I / O accesses, conserves disk I / O resources, and enhances the query performance of the database system, particularly in distributed database systems with massive amounts of data.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a data processing method, electronic equipment, and storage medium. Background Art

[0002] Database systems commonly use disks to store data, using pages (also called data pages) as the basic unit of exchange between disk and memory. Typically, a data page stores several rows from a database table. Because database tables typically contain large amounts of data, all rows in a table may be stored across multiple data pages on disk. In other words, a single data page stores several records from a database table.

[0003] In some application scenarios, to meet client data requirements, the database system may need to randomly sample database tables and obtain randomly sampled data from them. This typically involves performing multiple disk I / O (Input / Output) operations to load multiple data pages corresponding to the database table from disk into memory. In memory, all rows in the database table are read from these multiple pages using a full table scan. Finally, random sampling is performed on all read rows to obtain the randomly sampled data from the database table.

[0004] However, the random sampling method of the database table is prone to frequent disk access, which consumes a lot of disk I / O resources and reduces the query performance of the database system, especially when the distributed database system has massive data. Summary of the Invention

[0005] Various aspects of the present application provide a data processing method, an electronic device, and a storage medium, which are used to reduce the number of I / O accesses to a disk, save disk I / O resources, and enhance the query performance of a database system.

[0006] An embodiment of the present application provides a data processing method, including: traversing the index tree corresponding to the target database table to determine the last non-leaf layer of the index tree, each leaf node in the leaf layer of the index tree is associated with a data page on the disk, and different data pages store different row records in the target database table; for each non-leaf node on the last non-leaf layer, reading the current index item from the index page associated with the non-leaf node in turn, and randomly generating a random number for the current index item; if the random number meets the random data sampling condition, loading the target data page of the leaf node pointed to by the current index item from the disk into the memory, and reading and storing the row records in the target data page in the memory.

[0007] An embodiment of the present application also provides a data processing method, including: receiving an SQL statement submitted by a client for a target database table; obtaining randomly sampled data of the target database table; converting the SQL statement into an abstract syntax tree, and generating a final execution plan according to the abstract syntax tree based on the randomly sampled data; converting the final execution plan into a physical execution plan, and querying the database where the target database table is located based on the physical execution plan, obtaining query results, and outputting the query results to the client.

[0008] An embodiment of the present application also provides an electronic device, comprising: a memory and a processor; the memory is used to store a computer program; the processor is coupled to the memory and is used to execute the computer program to perform the steps in the data processing method.

[0009] An embodiment of the present application further provides a computer storage medium storing a computer program, which, when executed by a processor, enables the processor to implement the steps in the data processing method.

[0010] In an embodiment of the present application, an index tree scanning method is adopted in combination with a mechanism of randomly sampling data pages of a database table. In the process of obtaining randomly sampled data of a target database table, there is no need to perform a full table scan of the target database table, which effectively reduces the number of I / O accesses to the disk, saves disk I / O resources, and enhances the query performance of the database system, especially when a distributed database system has massive data. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0012] Figure 1 A schematic diagram of the structure of a database system provided in an embodiment of the present application;

[0013] Figure 2 is an exemplary data structure of a data page;

[0014] Figure 3 are exemplary parent nodes and child nodes;

[0015] Figure 4 is an exemplary index tree;

[0016] Figure 5 A flowchart of a data processing method provided in an embodiment of the present application;

[0017] Figure 6 A flowchart of another data processing method provided in an embodiment of the present application;

[0018] Figure 7 Record changes for the exemplary rows;

[0019] Figure 8 A flowchart of another data processing method provided in an embodiment of the present application;

[0020] Figure 9 A schematic diagram of the structure of a data processing device provided in an embodiment of the present application;

[0021] Figure 10 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0022] To make the purpose, technical solutions, and advantages of this application more clear, the technical solutions of this application will be clearly and completely described below in conjunction with the specific embodiments of this application and the corresponding drawings. Obviously, the embodiments described are only part of the embodiments of this application, not all of them. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0023] The existing random sampling method for database tables is prone to frequent disk access, consuming more disk I / O resources, and reducing the query performance of the database system, especially when the distributed database system has massive data. To this end, the embodiments of the present application provide a data processing method, electronic device, and storage medium. In the embodiments of the present application, an index tree scanning method is adopted and combined with a mechanism for randomly sampling data pages of a database table. In the process of obtaining randomly sampled data of a target database table, there is no need to perform a full table scan on the target database table, which effectively reduces the number of I / O accesses to the disk, saves disk I / O resources, and enhances the query performance of the database system, especially when the distributed database system has massive data.

[0024] The following describes in detail the technical solutions provided by various embodiments of the present application in conjunction with the accompanying drawings.

[0025] Figure 1 This is a schematic diagram of the structure of a database system provided in an embodiment of the present application. Figure 1 A database system (DBS) consists of a database server and disks. The database server is responsible for database management functions, including system configuration and management, data access and update management, data integrity management, and data security management. It also performs database query and modification tasks, as well as database maintenance functions, but is not limited to these.

[0026] In this embodiment, the database server responds to a query request sent by a client, queries the database, obtains the query results, and returns them to the client. The general process for the database server to obtain the query results is as follows: the database server accesses the disk to load the data page containing the query results from the disk into the memory, accesses the memory to read the query results from the data page, and returns the query results to the client.

[0027] In this embodiment, the database system uses a disk to store data, and all kinds of data in the database (including various database tables) are stored in disk files on the disk. Each disk file includes multiple data pages, and a data page is the smallest unit for reading data from the disk. Usually, all row records in each database table (referring to a row of data in a database table) are stored in multiple data pages, and each data page includes one or more row records in the database table. Each row of data in a data page is stored sequentially according to the size of the primary key, and each row of data has a pointer pointing to the location of the next row of data, forming a one-way linked list, wherein the primary key uniquely identifies the row data, wherein, in addition to storing the row records in the database table, the data page also stores other row data according to actual application requirements. See Figure 2 A large number of data pages are stored sequentially, one by one. Adjacent data pages reference each other using a doubly linked list format. One pointer in the doubly linked list points to the access information of the previous data page, while the other pointer points to the access information of the next data page. The access information can be the page number of the data page, and the data page can be searched based on the page number.

[0028] In this embodiment, unlike a full table scan, an index tree scan is used to scan a database table. Examples of index trees include, but are not limited to, Btree trees (also known as B-trees), B-trees (also known as B-trees), and B+trees (also known as B+trees). B+trees have the following characteristics: 1. Non-leaf nodes store only key values. 2. Linked list pointers are present between all leaf nodes. 3. All data records are stored in leaf nodes.

[0029] For ease of understanding, the index tree is B+tree as an example. Determine the primary key of the database table. The primary key can uniquely identify the attribute or attribute group of a row in the database table. After determining the primary key of the database table, create a primary key index in the form of a B+tree index tree. The B+tree index tree is composed of a root node layer, one or more non-leaf layers, and a leaf layer from top to bottom. Among them, the root node layer includes a root node, the non-leaf layer includes one or more non-leaf nodes, and the leaf layer includes one or more leaf nodes.

[0030] In this embodiment, the root node and non-leaf nodes are each associated with an index page, which maintains index information for the database table. Leaf nodes are associated with a data page, which contains rows from the database table indexed by one or more index trees. In other words, the B+tree index maintains database table rows in the data pages corresponding to leaf nodes, while index pages corresponding to nodes other than leaf nodes maintain index information but not database table rows.

[0031] In this embodiment, the index page maintains index information in the form of index items. Specifically, an index page may include one or more index items, and optionally, each index item includes a primary key and access information pointing to a child node. The access information pointing to a child node may be the page number, storage location information, or access address information of the index page or data page associated with the child node, but is not limited thereto. It should be understood that when a non-leaf node is a node in the last non-leaf layer in the index tree, and the child node of the non-leaf node is a leaf node, then the index item of the non-leaf node includes the access information of the data page associated with the child node. When a non-leaf node is a node in a non-leaf layer other than the last non-leaf layer in the index tree, and the child node of the non-leaf node is also a non-leaf node, then the index item of the non-leaf node includes the access information of the index page associated with the child node.

[0032] See also Figure 3 Each index entry includes a primary key and the page number of the index page or data page associated with the child node. The node where the index entry is located is the parent node. The primary key reflects the unique row identifier of the row record in the database table. The primary key in the index entry can be used to determine whether to continue searching the index page or data page associated with the child node. Specifically, if the primary key to be searched is smaller than the primary key in the index entry, it means that the primary key to be searched is not on the index page or data page associated with the child node. If the primary key to be searched is greater than or equal to the primary key in the index entry, it means that the primary key to be searched is likely to exist on the index page or data page associated with the child node, and the search continues on the index page or data page associated with the child node of the parent node.

[0033] For ease of understanding, assume that there is a database table with records (ID, name) as shown in Table 1, and establish the following with ID as the primary key: Figure 4 The B+tree index tree in the form of the primary key index is shown.

[0034] Table 1

[0035] ID name 1 Li …… …… 100 Zhang 101 Du …… …… 200 Zhao …… …… 300 Bai 301 Sun …… …… 400 Che 401 Huang …… …… 500 Lu 501 Zhang …… …… 600 Wu

[0036] exist Figure 4In Table 1, the page number of the index page corresponding to the root node is 100. An index entry (401, 102) is recorded in the index page with page number 100. In this index entry, 401 is the primary key, that is, the unique row identifier of the row with ID 401 in Table 1. In this index entry, 102 is the page number of the index page associated with the child node of the root node.

[0037] The page number of the index page corresponding to a non-leaf node is 101. Three index entries are recorded in the index page with page number 101: (101, 501), (201, 502), and (301, 503). The 101 in the (101, 501) index entry is the primary key, that is, the unique row identifier of the row record with ID 101 in Table 1. 501 is the page number of the data page associated with the child node of the non-leaf node (which is a leaf node). Similarly, the representational meaning of the index entries (201, 502) and (301, 503) will not be repeated here.

[0038] Based on the index tree, two search operations can be performed. One is a range search and page search within the primary key, and the other is a random search starting from the root node. In this embodiment, when randomly sampling a database table, data is randomly sampled from any data page corresponding to the database table.

[0039] In this embodiment, the primary key range of each index item can be determined based on the primary keys in adjacent index items. Specifically, based on the primary keys in two adjacent index items in an index page, the primary key range of the index item with the smaller primary key is determined. The minimum value in the primary key range of the last index item in an index page is the primary key recorded in the last index item, and the maximum value is the maximum value that can be covered by the index page. The maximum value that can be covered by each index page is associated with the parent node corresponding to the index page.

[0040] For example, the three index items on index page 101 are arranged in ascending order of primary key: index item (101, 501), index item (201, 502), and index item (301, 503). Comparing the primary keys of the adjacent index items (101, 501) and (201, 502), the primary key range of the index item (101, 501) with the smaller primary key is determined to be 101 to 200. Comparing the primary keys of the adjacent index items (201, 502) and (301, 503), the primary key range of the index item (201, 502) with the smaller primary key is determined to be 201 to 300. Regarding the primary key range of index item (301, 503), based on the parent node corresponding to index page 101, the maximum value that index page 101 can cover is determined to be 400. Therefore, the primary key range of index item (301, 503) is 301 to 400.

[0041] In this embodiment, the index item to which the primary key to be searched belongs can be determined based on the primary key range corresponding to which index item the primary key to be searched falls, and the child nodes pointed to by the index item can be traversed to continue the query in the index page or data page associated with the child node.

[0042] In this embodiment, the index item to which the primary key to be searched belongs can also be determined based on a binary search method. Specifically, for the currently traversed index page, one of the index items with smaller primary keys is sequentially used as the current index item, and the primary key to be searched is compared with the current index item in sequence. If the primary key to be searched is larger than the primary key in the current index item, the next index item of the current index item is used as the new current index item, and the step of comparing the primary key to be searched with the current index item in sequence is repeated until the primary key to be searched is smaller than or equal to the primary key in the current index item. If the primary key to be searched is equal to the primary key in the current index item, the index item to which the primary key to be searched belongs is the current index item. If the index item to which the primary key to be searched belongs is smaller than the current index item, the index item to which the primary key to be searched belongs is the previous index item of the current index item.

[0043] For example, when searching for an index entry that matches the primary key to be searched in index page 101 of page number, index entry (101, 501), index entry (201, 502), and index entry (301, 503) are used as the current index entry in sequence. When the current index entry is index entry (101, 501), primary key 260 is compared with primary key 101 in index entry (101, 501). Since primary key 260 is greater than primary key 101, primary key 260 is compared with primary key 101 in index entry (201, 502). Since primary key 260 is greater than primary key 201, primary key 260 is compared with primary key 301 in index entry (301, 503). Since primary key 260 is less than primary key 301, it is indicated that the previous index entry of index entry (301, 503), that is, (201, 502), is the index entry to which primary key 260 belongs.

[0044] In actual applications, all index items or row records that are smaller than the primary key in the first index item in the root node or non-leaf node will also be stored in an index page or data page, but there is no index item pointing to the index page or data page in the index page. In order to save the page number of the index page or data page, it can be saved in the page header of its parent node (or other predefined location). When traversing the index tree to search for a smaller primary key later, the corresponding data page can still be located. In other words, the page number of the first child node of the index page corresponding to the root node or non-leaf node has no corresponding index item pointing to it, but is stored in the page header of the index page (or other predefined location), and the key values of all index items or row records stored in the first child node are smaller than the key value contained in the first index item on its parent node. See Figure 4 , the dotted arrow points to the first child node of the index page corresponding to the root node or non-leaf node.

[0045] For example, the page header of the index page with page number 100 corresponding to the root node stores page number 101. Index page 101 stores a primary key with a key value less than 401, and the index item corresponding to primary key 401 is the first index item in the index page with page number 100. The page header of the index page with page number 101 corresponding to the first non-leaf node stores page number 500. The data page with page number 500 stores a primary key with a key value less than 101, and the index item corresponding to primary key 101 is the first index item in the index page with page number 101. The page header of the index page with page number 102 corresponding to the second non-leaf node stores page number 504. The data page with page number 504 stores a primary key with a key value less than 501, and the index item corresponding to primary key 501 is the first index item in the index page with page number 10.

[0046] For example, if the primary key to be searched is 90, we start traversing from the root node. Since 90 is smaller than the primary key 401 contained in the first index item (401, 100) corresponding to the root node, we need to read the page number 101 corresponding to the first child node from the page header of the index page with page number 100. Figure 4 The dotted arrow in the figure points from the index page with page number 100 to the index page with page number 101. Next, a search is performed on index page 101. Since 90 is less than the primary key value 101 contained in the first index item (101, 500), it is necessary to read the page number 500 corresponding to the first child node of the index page with page number 101 from the page header, and finally find the row record corresponding to the primary key 90 in the data page with page number 500.

[0047] The following combination Figure 5 , which explains in detail how to scan a database table using an index tree scan method. Figure 5 A flowchart of a data processing method provided in an embodiment of the present application. The method can be executed by a data processing device, which can be implemented by software and / or hardware and can generally be integrated into a database system, for example, a database server.

[0048] See also Figure 5 , the method may include the following steps:

[0049] 501. Traverse the index tree corresponding to the target database table to determine the last non-leaf layer of the index tree. Each leaf node in the leaf layer of the index tree is associated with a data page in the disk. Different data pages store different row records in the target database table.

[0050] 502. For each non-leaf node on the last non-leaf layer, read the current index entry from the index page associated with the non-leaf node in turn, and randomly generate a random number for the current index entry;

[0051] 503. If the random number satisfies the random data sampling condition, the target data page of the leaf node pointed to by the current index item is loaded from the disk into the memory.

[0052] 504. Read and store row records in the target data page in memory.

[0053] In this embodiment, the target database table is any database table maintained in the database. In actual applications, it may be necessary to scan data from multiple database tables in the database, and each database table that needs to be scanned will be used as a target database table.

[0054] In this embodiment, an index tree for index scanning of the target database table is pre-established. Taking the B+tree in the form of a primary key index as an example, the index page associated with the root node and the non-leaf node includes one or more index items. Optionally, each index item includes a primary key and access information pointing to the child node. The access information pointing to the child node can be the page number and storage location information of the index page or data page associated with the child node, but is not limited to this. It is worth noting that the storage location information of the index page or data page associated with the child node is pre-associated with its page number. Therefore, the corresponding index page or data page can also be accessed based on the page number. The storage location information can be, for example, storage location information on a disk or in memory, but is not limited to this.

[0055] In this embodiment, for the target database table that needs to be randomly sampled, the nodes of each layer are traversed starting from the root node of the index tree until the last non-leaf layer of the index tree is traversed.

[0056] After traversing to the last non-leaf layer, traverse each non-leaf node in the non-leaf layer in turn. For the non-leaf node currently traversed, read an index item from the index page associated with the non-leaf node as the current index item, and randomly generate a random number for the current index item; if the random number meets the random data sampling condition, then load the target data page of the leaf node pointed to by the current index item from the disk into the memory, and read and store the row records in the target data page in the memory to complete the data collection of the target data page. If the random number does not meet the random data sampling condition, give up collecting the data of the target data page of the leaf node pointed to by the current index item.

[0057] In this embodiment, the random data sampling conditions are flexibly set based on actual application requirements. For example, the random data sampling condition is that the random number is less than or equal to a specified value. In other words, if the random number is less than or equal to the specified value, sampling is performed. If the random number is greater than the specified value, sampling is abandoned.

[0058] The percentage of the specified value is the set sampling rate. The set sampling rate is selected between the minimum sampling rate and the maximum sampling rate. The minimum sampling rate and the maximum sampling rate are sampling rates flexibly set according to actual application requirements. For example, if the target database table has 10,000 rows, the minimum sampling rate is 1% and the maximum sampling rate is 99%. That is, at least 100 rows and a maximum of 9,900 rows can be sampled from the target database table. The set sampling rate can be any value in the numerical range of [1%, 99%], and the specified value is any value in the numerical range of [1, 99]. If the set sampling rate is 10%, the specified value is recorded as 10, that is, if the generated random number is less than or equal to 10, sampling is performed; if the generated random number is greater than 10, sampling is abandoned.

[0059] by Figure 4 For example, when traversing to the index item (101, 501) in the index page with page number 101, if the currently generated random number is less than or equal to the specified value, data collection is performed on the data page with page number 501. If the currently generated random number is greater than the specified value, data collection on the data page with page number 501 is abandoned.

[0060] In this embodiment, for a target data page to be collected, the disk is accessed to load the target data page from the disk into memory. Next, the memory is accessed to read the row records in the loaded target data page and the read row records are stored, thus completing the data collection for the target data page. After traversing all non-leaf nodes in the last non-leaf layer of the index tree, the entire task of randomly sampling the data pages in the target database table is completed. At this point, the row records of each target data page read are the randomly sampled data of the target database table.

[0061] In this embodiment, the row records in the target data page read by accessing the memory can be stored in a designated storage location, cached in a dedicated buffer pool, or directly sent to the client, without limitation. Of course, after reading and storing the row records in the target data page in the memory, the stored row records can also be sent to the client.

[0062] In some application scenarios, the database system needs to perform data statistics on the data maintained in the database (specifically, the row records in the database table). The statistical information includes, but is not limited to, statistical information such as data volume, data distribution, and unique values. Therefore, in some embodiments, information statistics are performed on the stored row records, and the statistical result information is sent to the client. It is worth noting that after completing the random sampling task of a target database table, information statistics can be started for all stored row records. It is also possible to start information statistics for all stored row records after completing the random sampling tasks of multiple target database tables, without specific restrictions.

[0063] For a target database table with 1,000,000,000 rows, if the sampling rate is 50%, half of the rows need to be sampled from the target database table. Assuming the size of each data page is 16KB (kilobyte), if the existing full table scan method is used, the number of disk I / Os consumed is 6,250,000; if the index tree scan method used in this case is used, the number of disk I / Os consumed is 3,128,906. It can be seen that under ideal conditions, the number of disk I / Os consumed by the index tree scan method in this case is half that of the full table scan method. In addition, the smaller the sampling rate, the more disk I / O can be saved, and the more obvious the performance advantage.

[0064] The technical solution provided by the embodiment of the present application adopts an index tree scanning method and combines it with a mechanism of randomly sampling data pages of a database table. In the process of obtaining randomly sampled data of a target database table, there is no need to perform a full table scan of the target database table, which effectively reduces the number of I / O accesses to the disk, saves disk I / O resources, and enhances the query performance of the database system, especially when a distributed database system has massive data.

[0065] In actual applications, a data page stores multiple row records. To accurately read the row records of the data page one by one from the memory, a leaf cursor is used to control the row record reading process. Therefore, in some embodiments, the implementation process of reading and storing the row records in the target data page in the memory can be: in response to a row record collection event triggered by calling a row record collection interface, read the row record following the row record last saved in the leaf cursor in the target data page loaded into the memory, and store the currently read row record; save the currently read row record in the leaf cursor; repeat the above steps until all row records in the target data page are read.

[0066] In this embodiment, a row record collection interface is provided for returning the next piece of sampled data, for example, the row record collection interface is recorded as InnodbSampler::next(); an initialization interface is also provided for initializing sampling, for example, the initialization interface is recorded as InnodbSampler::init(); and a cleanup interface is also provided for cleaning up resources after sampling is completed, for example, the cleanup interface is recorded as InnodbSampler::end(). The above-mentioned interface can be an SDK (Software Development Kit) interface or an API (Application Programming Interface), without limitation.

[0067] In this embodiment, before starting data sampling on a data page, an initialization interface can be called to perform sampling initialization operations. After starting data sampling on a data page, a row record collection interface can be called to obtain random sampled data one by one. After completing data sampling on a data page, a cleanup interface can be called to perform resource cleanup operations.

[0068] In this embodiment, in addition to calling the row record collection interface to trigger the row record collection event, a leaf cursor is also used to save each row record read to facilitate locating the next row record to be read. Thus, before the next row record is read, the row record last saved in the leaf cursor is used to accurately locate the previous row record read on the data page. The row record after the previous row record is located as the next row record to be read.

[0069] It's worth noting that a leaf cursor not only stores each row read, but also the page's Log Sequence Number (LSN). If the data on a page is modified, the page's LSN changes. If the page's data remains unchanged, the page's LSN remains unchanged. Furthermore, to accurately read rows one by one, the page's LSN is used to determine how to read the next row.

[0070] Based on the above, the present application also provides a data processing method, such as Figure 6 As shown, the method may include the following steps:

[0071] 601. Traverse the index tree corresponding to the target database table to determine the last non-leaf layer of the index tree. Each leaf node in the leaf layer of the index tree is associated with a data page in the disk. Different data pages store different row records in the target database table.

[0072] 602. For each non-leaf node on the last non-leaf layer, read the current index entry from the index page associated with the non-leaf node in turn, and randomly generate a random number for the current index entry;

[0073] 603. If the random number meets the random data sampling condition, the target data page of the leaf node pointed to by the current index item is loaded from the disk into the memory.

[0074] 604. In response to the row record collection event triggered by calling the row record collection interface, determine whether the log logical sequence number LSN of the target data page has changed.

[0075] 605. Determine whether the log logical sequence number LSN of the target data page has changed. If so, execute step 608; if not, execute step 606.

[0076] 606. Read a row record following the row record last saved in the leaf cursor in the target data page loaded into the memory, and store the currently read row record.

[0077] 607. Save the currently read row record to the leaf cursor.

[0078] After step 607 , the process returns to step 604 until all row records in the target data page are read.

[0079] 608. Determine whether the LSN of the index page has changed. If so, execute step 610; if not, execute step 609.

[0080] 609. Relocate the target data page in memory based on the last saved index entry in the non-leaf cursor.

[0081] After step 609, the process returns to step 606 and subsequent steps.

[0082] 610. Obtain the first primary key corresponding to the last saved row record in the leaf cursor; traverse the index tree from the root node until the target leaf node is traversed, and the data page corresponding to the target leaf node includes at least one row record whose primary key is less than or equal to the first primary key.

[0083] 611. Read the next row record that is located after at least one row record in the data page of the target leaf node loaded into the memory, and store the currently read row record.

[0084] After step 611 , the process returns to step 607 and subsequent steps.

[0085] In an embodiment of the present application, a leaf cursor is used to save each row record read from a data page, and a non-leaf cursor is used to save each index item read from an index page, so as to facilitate the subsequent rapid positioning of the next row record to be read. Therefore, in some embodiments, after reading the current index item from the index pages associated with the non-leaf nodes in sequence, the current index item is also saved in the non-leaf cursor; after all row records in the target data page are read, the last saved index item is obtained from the non-leaf cursor; the index item after the last saved index item is searched in the index page associated with the non-leaf node, and the index item after the last saved index item is used as the new current index item, until all index items in the index pages associated with the non-leaf nodes are processed.

[0086] In this embodiment, if the LSN of the target data page or the LSN of the index page has not changed, the next returned row record can be read at a lower cost, thereby reducing unnecessary disk I / O overhead. If both the LSN of the target data page and the LSN of the index page have changed, the next row record to be read is quickly located in the primary key range search method. Specifically, the primary key corresponding to the last saved row record in the leaf cursor is obtained as the first primary key; starting from the root node, the index tree is traversed until the target leaf node is traversed, and the data page corresponding to the target leaf node includes at least one row record whose primary key is less than or equal to the first primary key. Read the next row record located after at least one row record in the data page of the target leaf node loaded into the memory.

[0087] It is worth noting that if there are multiple rows with a primary key less than or equal to the first primary key, the next row following the row with the largest primary key is read.

[0088] For easier understanding, see Figure 7 Assuming that row 3 in the target database table is the last row read, after row 3 is read from memory, no matter what changes occur to the data page including row 3, calling the InnodbSampler::next() interface can correctly return the next row 4 to be sampled.

[0089] Case 1: Before and after row record 3 is read, the row record on the data page where it is located remains unchanged.

[0090] In the above situation, InnodbSampler::next() is first called to read row record 3 from the data page, and the read row record 3 is sent to the client and saved in the leaf cursor. Then, InnodbSampler::next() is called to trigger the reading of the next row record. When reading the next row record, the last read row record 3 is located in the data page according to row record 3 in the leaf cursor. Finally, the row record after row record 3, that is, row record 4, is read, and the read row record 4 is sent to the client and saved in the leaf cursor.

[0091] The second case: After row record 3 is read, row record 3 on the data page where it is located is deleted.

[0092] In the above situation, InnodbSampler::next() is first called to read row 3 from the data page, send it to the client, and save it to the leaf cursor. Then, row 3 is deleted from the data page. Next, InnodbSampler::next() is called to trigger the read of the next row. Specifically, the index tree is traversed within the primary key range less than or equal to that of row 3 until row 2 is found on the data page. Since row 3 has been deleted, the next row after row 2 is row 4. Row 4 is read, sent to the client, and saved to the leaf cursor.

[0093] The third case: After row record 3 is read, row records 1, 2, and 3 on the data page where it is located are all deleted.

[0094] It is worth noting that each data page stores two special row records: infimum and supreme. Infimum represents the logical minimum row record on the current data page, while supreme represents the logical maximum row record on the current data page. These two row records are only used to mark the search boundaries and are not returned to the client.

[0095] In the above situation, InnodbSampler::next() is first called to read row record 3 from the data page, send the read row record 3 to the client, and save row record 3 to the leaf cursor. Then, row records 1, 2, and 3 are all deleted from the data page. Next, InnodbSampler::next() is called to trigger the reading of the next row record. Specifically, the index tree is traversed within the primary key range less than or equal to the corresponding row record 3 until row record infimum is found on the data page. Since row records 1, 2, and 3 have been deleted, the row record after row record infimum is row record 4. Row record 4 is read and sent to the client, and row record 4 is saved to the leaf cursor.

[0096] Traversing the index tree based on the first primary key is a method for searching within a primary key range. As an example, traversing the index tree from the root node until reaching a target leaf node includes: for nodes other than the target leaf node currently traversed, determining index entries in index pages associated with other nodes that match the first primary key; and traversing the child nodes of the other nodes pointed to by the index entries that match the first primary key.

[0097] It is worth noting that if the index page associated with the other node includes only one index item, then the index item is the index item that matches the first primary key. If the index page associated with the other node includes multiple index items, the primary key range that each index item can index is analyzed, and the index item corresponding to the primary key range to which the first primary key belongs is the index item that matches the first primary key. It is worth noting that when determining the primary key range of each index item, it can be determined based on the primary key in each index item. Among them, the primary key recorded in each index item is the minimum primary key, and the primary key range of each index item can be determined based on the minimum primary key recorded in each index item. For example, the primary key range corresponding to the index item (101, 501) in index page 101 is 101 to 200; the primary key range corresponding to the index item (201, 502) in index page 101 is 201 to 300; and the primary key range corresponding to the index item (301, 503) in index page 101 is 301 to 400.

[0098] by Figure 4Taking the index tree shown as an example, after reading the row record (180, Mu) from the data page with page number 501, the row record (180, Mu) is returned to the client, and the row record (180, Mu) is saved to the leaf cursor. Then, before reading the row record (181, Jia), it is determined that the LSNs of the data page with page number 501 and the index page with page number 101 have both changed. Then, within the primary key range less than or equal to 181, the index tree is traversed from the root node. Since the primary key 181 is less than the primary key 401 in the index entry (401, 102) corresponding to the root node, the page number 101 of the index page with page number 100 that is less than the primary key 400 is obtained from the page header of the index page with page number 100. At this time, based on the page number 101 of the found index page, the index page with page number 101 is traversed. Among the three index items of index page 101, the index item (101, 501) matches the primary key 181, so the data page with page number 501 is traversed.

[0099] In this embodiment, a locking mechanism can also be introduced to ensure data consistency in the database. Thus, in some embodiments, before reading the row record following the last row record saved in the leaf cursor of the target data page loaded into memory, the target data page is locked; after reading the row record following the last row record saved in the leaf cursor of the target data page loaded into memory, the lock on the target data page is released. It is worth noting that after the target data page is locked, only specified operations are allowed on the target data page, such as reading rows in the target data page.

[0100] In some embodiments, when traversing the index tree of the target database table, when traversing to the parent node, the index page corresponding to the parent node is locked, for example, a shared lock (S lock) is added; when traversing to the child node of the parent node, the index page or data page of the child node is locked, and the lock on the index page corresponding to the parent node is released.

[0101] The technical solution provided by the embodiments of the present application utilizes an index tree scan approach combined with a mechanism for randomly sampling database table data pages. This eliminates the need for a full table scan of the target database table when acquiring randomly sampled data from the target database table. This effectively reduces the number of disk I / O accesses, conserves disk I / O resources, and enhances the query performance of the database system, particularly in distributed database systems with massive amounts of data. Furthermore, by utilizing leaf and non-leaf cursors to control the reading of row records, row records of data pages can be more accurately read from memory one by one.

[0102] Typically, a database system executes SQL statements as follows: First, a syntax check is performed on the SQL (Structured Query Language) statement submitted by the client. Next, a semantic check is performed on the SQL statement. Next, the SQL statement is parsed to generate a syntax tree. A preliminary execution plan is generated based on the syntax tree. Next, the preliminary execution plan is optimized to obtain the final execution plan. Finally, the final execution plan is converted into a physical execution plan, which is used to query the database for the SQL statement's query results and return the results to the client. The quality of the execution plan directly impacts the query performance of the database system. The generation or optimization of the execution plan relies on database data information, such as data size, data distribution, and unique values. Currently, a full table scan is commonly used to perform random sampling in the database to obtain the data information required for generating or optimizing the execution plan. However, this approach consumes a large amount of disk I / O, particularly in distributed database systems with massive amounts of data, reducing the query performance of the database system.

[0103] To this end, the present application also provides a data processing method, such as Figure 8 As shown, the data processing method may include the following steps:

[0104] 801. Receive an SQL statement for a target database table submitted by a client.

[0105] 802. Obtain randomly sampled data from a target database table.

[0106] 803. Convert the SQL statement into an abstract syntax tree, and generate a final execution plan according to the abstract syntax tree based on the randomly sampled data;

[0107] 804. Convert the final execution plan into a physical execution plan, query the database where the target database table is located based on the physical execution plan, obtain the query results, and output the query results to the client.

[0108] Among them, further optionally, generating a final execution plan according to the abstract syntax tree based on the random sampling data includes: generating a preliminary execution plan according to the abstract syntax tree based on the random sampling data; optimizing the preliminary execution plan based on the random sampling data to obtain the final execution plan.

[0109] In this embodiment, the data processing method provided in the aforementioned method embodiment may be used to obtain random sampling data of the target database table, which will not be described in detail.

[0110] The technical solution provided by the embodiments of the present application utilizes an index tree scan approach combined with a mechanism for randomly sampling database table data pages to obtain randomly sampled data from a target database table during the processing of SQL statements initiated by a client. This effectively reduces the number of disk I / O accesses required to obtain randomly sampled data from the target database table, conserving disk I / O resources and enhancing the query performance of the database system, particularly in distributed database systems with massive amounts of data.

[0111] Figure 9 This is a structural diagram of a data processing device provided in an embodiment of the present application. Figure 9 The device may include: a traversal module 91 and a processing module 92.

[0112] The traversal module 91 is used to traverse the index tree corresponding to the target database table to determine the last non-leaf layer of the index tree. Each leaf node in the leaf layer of the index tree is associated with a data page on the disk. Different data pages store different row records in the target database table.

[0113] Processing module 92 is configured to read the current index item from the index page associated with each non-leaf node on the last non-leaf layer in turn, and randomly generate a random number for the current index item;

[0114] The processing module 92 is also used to load the target data page of the leaf node pointed to by the current index item from the disk into the memory if the random number meets the random data sampling condition, and read and store the row records in the target data page in the memory.

[0115] Further optionally, after reading and storing the row records in the target data page in the memory, the processing module 92 is further configured to: send the stored row records to the client; or perform information statistics on the stored row records and send the statistical result information to the client.

[0116] Further optionally, when the processing module 92 reads and stores row records in the target data page in the memory, it is specifically used to: respond to the row record collection event triggered by calling the row record collection interface, read a row record following the row record last saved in the leaf cursor in the target data page loaded into the memory, and store the currently read row record; save the currently read row record in the leaf cursor; repeat the above steps until all row records in the target data page are read.

[0117] Further optionally, before the processing module 92 reads a row record following the last row record saved in the leaf cursor in the target data page loaded into the memory, it is also used to: determine whether the log logical sequence number LSN of the target data page has changed; if the LSN of the target data page has not changed, then execute the operation of reading a row record following the last row record saved in the leaf cursor in the target data page loaded into the memory.

[0118] Further optionally, the processing module 92 is also used to determine whether the LSN of the index page has changed if the LSN of the target data page has changed; if the LSN of the index page has not changed, read the last saved index item in the non-leaf cursor; based on the last saved index item in the non-leaf cursor, relocate the target data page in the memory, and execute the operation of reading a row record following the row record last saved in the leaf cursor in the target data page loaded into the memory.

[0119] Further optionally, after reading the current index item from the index pages associated with the non-leaf nodes in sequence, the processing module 92 is also used to: save the current index item to the non-leaf cursor; after all row records in the target data page are read, obtain the last saved index item from the non-leaf cursor; search for the index item after the last saved index item in the index page associated with the non-leaf node, and use the index item after the last saved index item as the new current index item, until the processing of all index items in the index pages associated with the non-leaf nodes is completed.

[0120] Further optionally, the processing module 92 is also used to: if the LSN of the index page changes, obtain the first primary key corresponding to the last saved row record in the leaf cursor; traverse the index tree starting from the root node until the target leaf node is traversed, and the data page corresponding to the target leaf node includes at least one row record whose primary key is less than or equal to the first primary key; read the next row record located after at least one row record in the data page of the target leaf node loaded into the memory, and store the currently read row record; return to execute the step of saving the currently read row record in the leaf cursor.

[0121] Further optionally, the processing module 92 traverses the index tree starting from the root node until it traverses the target leaf node, and is specifically used to: for other nodes currently traversed except the target leaf node, determine the index items in the index pages associated with other nodes that match the first primary key; traverse the child nodes of other nodes pointed to by the index items that match the first primary key.

[0122] Further optionally, the processing module 92 is also used to: lock the target data page before reading a row record following the last row record saved in the leaf cursor in the target data page loaded into the memory; and after reading a row record following the last row record saved in the leaf cursor in the target data page loaded into the memory, the processing module 92 is also used to: release the lock on the target data page.

[0123] Further optionally, the index tree is a B+tree index tree.

[0124] about Figure 9 The specific manner in which each module in the data processing device shown performs operations can be found in the relevant description in the aforementioned method embodiment, which will not be repeated here.

[0125] The detailed implementation process of the processor executing each action can be found in the relevant description in the aforementioned method embodiment or device embodiment, and will not be repeated here.

[0126] It should be noted that the execution entity of each step of the method provided in the above embodiment can be the same device, or the method can be executed by different devices. For example, the execution entity of steps 501 to 504 can be device A; for another example, the execution entity of steps 501 and 502 can be device A, and the execution entity of steps 503 and 504 can be device B; and so on.

[0127] In addition, in some of the processes described in the above embodiments and the accompanying drawings, multiple operations that appear in a specific order are included, but it should be clearly understood that these operations may not be executed in the order in which they appear in this article or may be executed in parallel. The sequence numbers of the operations, such as 501, 502, etc., are only used to distinguish between different operations, and the sequence numbers themselves do not represent any execution order. In addition, these processes may include more or fewer operations, and these operations may be executed in sequence or in parallel. It should be noted that the descriptions of "first", "second", etc. in this article are used to distinguish different messages, devices, modules, etc., and do not represent the order of precedence, nor do they limit "first" and "second" to be different types.

[0128] Figure 10 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application. Figure 10 As shown, the electronic device includes: a memory 101 and a processor 102;

[0129] The memory 101 is used to store computer programs and can be configured to store various other data to support operations on the computing platform. Examples of such data include instructions for any application or method operating on the computing platform, contact data, phone book data, messages, pictures, videos, etc.

[0130] The memory 101 can be implemented by any type of volatile or non-volatile memory device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk.

[0131] The processor 102 is coupled to the memory 101 and is used to execute a computer program in the memory 101, which is used to: traverse the index tree corresponding to the target database table to determine the last non-leaf layer of the index tree, each leaf node in the leaf layer of the index tree is associated with a data page in the disk, and different data pages store different row records in the target database table; for each non-leaf node on the last non-leaf layer, read the current index item from the index page associated with the non-leaf node in turn, and randomly generate a random number for the current index item; if the random number meets the random data sampling condition, load the target data page of the leaf node pointed to by the current index item from the disk into the memory, and read and store the row records in the target data page in the memory.

[0132] Alternatively, the processor 102 is coupled to the memory 101 and is configured to execute a computer program in the memory 101 to: receive an SQL statement submitted by a client for a target database table; obtain randomly sampled data of the target database table; convert the SQL statement into an abstract syntax tree, and generate a final execution plan according to the abstract syntax tree based on the randomly sampled data; convert the final execution plan into a physical execution plan, query the database where the target database table is located based on the physical execution plan, obtain query results, and output the query results to the client.

[0133] Further, if Figure 10 As shown, the electronic device further includes: a communication component 103, a display 104, a power component 105, an audio component 106 and other components. Figure 10 Only some components are shown schematically, which does not mean that the electronic device only includes Figure 10 In addition, Figure 10 The components in the dotted box are optional components, not mandatory components, and the specific components may depend on the product form of the electronic device. The electronic device of this embodiment can be implemented as a terminal device such as a desktop computer, a laptop computer, a smart phone or an IOT device, or a server device such as a conventional server, a cloud server or a server array. If the electronic device of this embodiment is implemented as a terminal device such as a desktop computer, a laptop computer, a smart phone, etc., it may include Figure 10 If the electronic device of this embodiment is implemented as a conventional server, cloud server or server array and other server-side devices, it may not include Figure 10Components within the dotted box.

[0134] The detailed implementation process of the processor executing each action can be found in the relevant description in the aforementioned method embodiment or system embodiment, and will not be repeated here.

[0135] Accordingly, an embodiment of the present application further provides a computer-readable storage medium storing a computer program, which, when executed, can implement the steps that can be performed by the electronic device in the above method embodiment.

[0136] Accordingly, an embodiment of the present application also provides a computer program product, including a computer program / instruction. When the computer program / instruction is executed by a processor, the processor is enabled to implement the steps in the above method embodiment that can be performed by an electronic device.

[0137] The above-mentioned communication component is configured to facilitate wired or wireless communication between the device where the communication component is located and other devices. The device where the communication component is located can access a wireless network based on a communication standard, such as WiFi, 2G, 3G, 4G / LTE, 5G and other mobile communication networks, or a combination thereof. In an exemplary embodiment, the communication component receives a broadcast signal or broadcast-related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component also includes a near field communication (NFC) module to facilitate short-range communication. For example, the NFC module can be implemented based on radio frequency identification (RFID) technology, infrared data association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology and other technologies.

[0138] The above-mentioned display includes a screen, which may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touch screen to receive input signals from a user. The touch panel includes one or more touch sensors to sense touches, slides, and gestures on the touch panel. The touch sensor can not only sense the boundaries of a touch or slide action, but also detect the duration and pressure associated with the touch or slide operation.

[0139] The power supply assembly provides power to various components of the device in which the power supply assembly is located. The power supply assembly may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power to the device in which the power supply assembly is located.

[0140] The above-mentioned audio component can be configured to output and / or input audio signals. For example, the audio component includes a microphone (MIC), and when the device where the audio component is located is in an operating mode, such as call mode, recording mode, and voice recognition mode, the microphone is configured to receive external audio signals. The received audio signal can be further stored in a memory or sent via a communication component. In some embodiments, the audio component also includes a speaker for outputting audio signals.

[0141] Those skilled in the art will appreciate that the embodiments of the present application can be provided as methods, systems, or computer program products. Therefore, the present application can adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application can adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program code.

[0142] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the steps in the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0143] These computer program instructions may also be stored in a computer readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0144] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing the instructions executed on the computer or other programmable device for implementing the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0145] In a typical configuration, a computing device includes one or more processors (CPUs), input / output interfaces, network interfaces, and memory.

[0146] Memory may include non-permanent storage in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. Memory is an example of a computer-readable medium.

[0147] Computer-readable media includes permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. The information can be computer-readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassettes, magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer-readable media does not include transitory computer-readable media (transitory media), such as modulated data signals and carrier waves.

[0148] It should also be noted that the terms "comprises," "includes," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, method, commodity, or apparatus that includes a series of elements includes not only those elements but also other elements not explicitly listed, or includes elements inherent to such process, method, commodity, or apparatus. In the absence of further limitations, an element defined by the phrase "comprises a ..." does not exclude the presence of other identical elements in the process, method, commodity, or apparatus that includes the element.

[0149] The above are merely embodiments of the present application and are not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present application should all be included within the scope of the claims of the present application.

Claims

1. A data processing method, characterized in that: include: Traversing the index tree corresponding to the target database table to determine the last non-leaf layer of the index tree, where each leaf node in the leaf layer of the index tree is associated with a data page on the disk, and different data pages store different row records in the target database table; For each non-leaf node on the last non-leaf layer, read the current index item from the index page associated with the non-leaf node in turn, and randomly generate a random number for the current index item; If the random number satisfies the random data sampling condition, the target data page of the leaf node pointed to by the current index item is loaded from the disk into the memory, and the row record in the target data page in the memory is read and stored; Reading and storing row records in the target data page in the memory includes: In response to a row record collection event triggered by calling a row record collection interface, determine whether the log logical sequence number (LSN) of the target data page has changed; if the LSN of the target data page has not changed, read a row record following the row record last saved in the leaf cursor in the target data page loaded into the memory, and store the currently read row record; save the currently read row record in the leaf cursor; repeat the above steps until all row records in the target data page are read; If the LSN of the target data page changes, determine whether the LSN of the index page changes; if the LSN of the index page does not change, read the last saved index item in the non-leaf cursor; based on the last saved index item in the non-leaf cursor, relocate the target data page in the memory, and execute the operation of reading a row record following the row record last saved in the leaf cursor in the target data page loaded into the memory.

2. The method according to claim 1, characterized in that After reading and storing the row records in the target data page in the memory, the method further includes: Send the stored row records to the client; Alternatively, statistics are performed on the stored row records, and the statistical result information is sent to the client.

3. The method according to claim 1, characterized in that After reading the current index items from the index pages associated with the non-leaf nodes in sequence, the method further includes: Saving the current index item to a non-leaf cursor; After all row records in the target data page are read, the last saved index item is obtained from the non-leaf cursor; Search the index page associated with the non-leaf node for the index item after the last saved index item, and use the index item after the last saved index item as the new current index item until all index items in the index page associated with the non-leaf node are processed.

4. The method according to claim 1, wherein Also includes: If the LSN of the index page changes, obtain the first primary key corresponding to the last row record saved in the leaf cursor; Traversing the index tree from the root node until a target leaf node is reached, wherein the data page corresponding to the target leaf node includes at least one row record whose primary key is less than or equal to the first primary key; Reading a next row record following the at least one row record in the data page of the target leaf node loaded into the memory, and storing the currently read row record; Return to the step of saving the currently read row record to the leaf cursor.

5. The method according to claim 4, characterized in that Traversing the index tree from the root node until reaching the target leaf node includes: For other nodes currently traversed except the target leaf node, determine the index items matching the first primary key in the index pages associated with the other nodes; and traverse the child nodes of the other nodes pointed to by the index items matching the first primary key.

6. The method according to claim 2, characterized in that Before reading a row record following the row record last saved in the leaf cursor in the target data page loaded into the memory, the method further includes: Locking the target data page; After reading a row record following the row record last saved in the leaf cursor in the target data page loaded into the memory, the method further includes: Release the lock on the target data page.

7. The method according to any one of claims 1 to 5, characterized in that The index tree is a B+tree index tree.

8. A data processing method, characterized in that: include: Receive SQL statements submitted by the client for the target database table; Obtaining randomly sampled data from the target database table according to the method according to any one of claims 1 to 6; Converting the SQL statement into an abstract syntax tree, and generating a final execution plan according to the abstract syntax tree based on the randomly sampled data; The final execution plan is converted into a physical execution plan, and the database where the target database table is located is queried according to the physical execution plan to obtain a query result, and the query result is output to the client.

9. The method according to claim 8, characterized in that Generating a final execution plan according to the abstract syntax tree based on the randomly sampled data includes: generating a preliminary execution plan according to the abstract syntax tree based on the randomly sampled data; The preliminary execution plan is optimized based on the randomly sampled data to obtain a final execution plan.

10. An electronic device, characterized in that: include: memory and processor; The memory is used to store computer programs; The processor is coupled to the memory and configured to execute the computer program to perform the steps of the method according to any one of claims 1 to 7.

11. A computer storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the processor is enabled to implement the steps of the method according to any one of claims 1 to 7.

Images