Paging query method, device and equipment and readable medium

Abstract

The embodiment of the invention provides a paging query method and device, equipment and a readable medium. The method comprises the following steps: determining an index database for a database; in response to a received paging query request for the database, determining an index configuration and a query initial position matched with the paging query request by using the index database; based on the index configuration and the query initial position, determining a query statement for the database; and based on the query statement, determining a query result corresponding to the paging query request from a database. In this way, the query result can be determined from the database through the index database, and the paging query efficiency can be improved.

Description

Technical Field

[0001] The exemplary embodiments disclosed herein generally relate to the field of computer technology, and more specifically, to methods, apparatus, devices, and computer-readable storage media for paginated queries. Background Technology

[0002] In paginated query scenarios, when the amount of data in a database table is too large, or when there are multiple table joins, deep pagination may cause slow queries. Summary of the Invention

[0003] In a first aspect of this disclosure, a method for paginated queries is provided. The method includes: determining an index database for a database; in response to receiving a paginated query request for the database, using the index database, determining an index configuration matching the paginated query request and a query start position; determining a query statement for the database based on the index configuration and the query start position; and determining query results corresponding to the paginated query request from the database based on the query statement.

[0004] In a second aspect of this disclosure, an apparatus for paginated queries is provided. The apparatus includes: a determining module configured to determine an index database for a database; a receiving module configured to, in response to receiving a paginated query request for the database, determine, using the index database, an index configuration matching the paginated query request and a query start position; an executing module configured to determine a query statement for the database based on the index configuration and the query start position; and a querying module configured to determine, based on the query statement, a query result corresponding to the paginated query request from the database.

[0005] In a third aspect of this disclosure, an electronic device is provided. The electronic device includes at least one processing unit; and at least one memory coupled to the at least one processing unit and storing instructions for execution by the at least one processing unit, the instructions causing the electronic device to perform the method of the first aspect of this disclosure when executed by the at least one processing unit.

[0006] In a fourth aspect of this disclosure, a computer-readable storage medium is provided. This computer-readable storage medium stores a computer program that can be executed by a processor to perform the method according to a first aspect of this disclosure.

[0007] In a fifth aspect of this disclosure, a computer program product is provided, including computer-executable instructions, wherein the computer-executable instructions can be executed by a processor to perform a method according to a first aspect of this disclosure.

[0008] It should be understood that the description in the Summary of the Invention section is not intended to limit the key or essential features of the embodiments of this disclosure, nor is it intended to restrict the scope of this disclosure. Other features of this disclosure will become readily apparent from the following description. Attached Figure Description

[0009] In the following detailed description, in conjunction with the accompanying drawings, the above and other features, advantages, and aspects of the various implementations of this disclosure will become more apparent. In the accompanying drawings, the same or similar reference numerals denote the same or similar elements, wherein:

[0010] Figure 1 A schematic diagram of an example environment in which embodiments of the present disclosure can be implemented is shown;

[0011] Figure 2 A flowchart illustrating a pagination query process according to some embodiments of the present disclosure is shown;

[0012] Figure 3 A flowchart illustrating the process of determining an index database according to some embodiments of the present disclosure is shown;

[0013] Figure 4 A flowchart illustrating a process for determining query results according to some embodiments of the present disclosure is shown;

[0014] Figure 5 A schematic diagram illustrating the process of determining the skip list structure according to some embodiments of the present disclosure is shown;

[0015] Figure 6 A schematic structural block diagram of an apparatus for pagination query according to certain embodiments of the present disclosure is shown; and

[0016] Figure 7 A block diagram of an electronic device in which one or more embodiments of the present disclosure may be implemented is shown. Detailed Implementation

[0017] Embodiments of this disclosure will now be described in more detail with reference to the accompanying drawings. While some embodiments of this disclosure are shown in the drawings, it should be understood that this disclosure can be implemented in various forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided to provide a more thorough and complete understanding of this disclosure. It should be understood that the accompanying drawings and embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of protection of this disclosure.

[0018] In the description of embodiments of this disclosure, the term "comprising" and similar terms should be understood as open-ended inclusion, i.e., "including but not limited to". The term "based on" should be understood as "at least partially based on". The term "one embodiment" or "the embodiment" should be understood as "at least one embodiment". The term "some embodiments" should be understood as "at least some embodiments". Other explicit and implicit definitions may also be included below.

[0019] It should be noted that the acquisition, storage, and application of user personal information involved in the technical solution disclosed herein comply with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0020] It is understood that before using the technical solutions disclosed in the various embodiments of this disclosure, users should be informed of the types, scope of use, and usage scenarios of the personal information involved in this disclosure through appropriate means in accordance with relevant laws and regulations, and user authorization should be obtained.

[0021] For example, in response to receiving a user's active request, a prompt message is sent to the user to clearly inform the user that the requested operation will require the acquisition and use of the user's personal information, thereby enabling the user to choose whether to provide personal information to the software or hardware such as electronic devices, applications, servers or storage media that perform the operation of the technical solution disclosed herein, based on the prompt message.

[0022] As an optional but non-restrictive implementation, in response to a user's active request, a prompt message can be sent to the user, for example, via a pop-up window, where the prompt message can be presented in text format. Furthermore, the pop-up window can also include a selection control allowing the user to choose whether to "agree" or "disagree" to provide personal information to the electronic device.

[0023] Pagination queries refer to returning the query results in pages when there are too many results in a database query. Traditional pagination solutions generally use two methods. The first is using the LIMIT and OFFSET keywords. The LIMIT keyword indicates the number of records to return per page, and the OFFSET keyword indicates the starting position. The second method uses subqueries to achieve pagination, using the ROW_NUMBER() function to sort the results, appending a row number to each row, and then using the row numbers to filter and achieve pagination.

[0024] However, when a single table in a database contains a large amount of data, or when there are multiple table joins, slow queries due to deep pagination can occur. As the number of pages queried increases, the amount of data that needs to be retrieved and skipped also increases exponentially, consuming database resources and causing problems. Furthermore, even with indexes, data skew can occur, where the hash value is much lower than the data volume, causing the index to become ineffective.

[0025] In view of this, embodiments of this disclosure provide a scheme for paginated queries. The scheme includes: determining an index database for a database; in response to receiving a paginated query request for the database, using the index database, determining an index configuration matching the paginated query request and a query start position; determining a query statement for the database based on the index configuration and the query start position; and determining the query results corresponding to the paginated query request from the database based on the query statement. In this way, the efficiency and accuracy of paginated queries can be improved.

[0026] Figure 1 A schematic diagram of an example environment 100 in which embodiments of the present disclosure can be implemented is shown.

[0027] like Figure 1 As shown, the query system 110 may include a database 101 and an index database 102. The query system 110 may determine the corresponding query result 120 based on the received pagination query request 105.

[0028] In some embodiments, index database 102 may be determined based on database 101. In some examples, database 101 may include multiple database tables (also referred to as data tables). In other examples, index database 102 may include index configuration information and query start position information. In many more examples, query result 120 may indicate multiple pages of data. Furthermore, query results may be returned in the form of pages.

[0029] The query system 110 may include any computing system with computing capabilities, such as various computing devices / systems, terminal devices, server devices, etc. Terminal devices may be any type of mobile terminal, fixed terminal, or portable terminal, including mobile phones, desktop computers, laptop computers, notebook computers, netbook computers, tablet computers, media computers, multimedia tablets, handheld computers, portable gaming terminals, VR / AR devices, personal communication system (PCS) devices, personal navigation devices, personal digital assistants (PDAs), audio / video players, digital cameras / camcorders, positioning devices, television receivers, radio receivers, e-book devices, gaming devices, or any combination of the foregoing, including accessories and peripherals of these devices or any combination thereof.

[0030] It should be understood that the structure and function of the various elements in environment 100 are described for illustrative purposes only and do not imply any limitation on the scope of this disclosure.

[0031] The following description will continue with reference to the accompanying drawings, which will provide some exemplary embodiments of this disclosure.

[0032] Figure 2 A flowchart of a pagination query process 200 according to some embodiments of the present disclosure is shown. Process 200 can be implemented at query system 110. For ease of discussion, reference will be made to... Figure 1 The environment 100 is used to describe the process 200.

[0033] In box 210, query system 110 determines the index database for the database.

[0034] In some embodiments, the query system 110 can divide business data in the database into at least one data block. Further, the query system 110 can determine the data node corresponding to the at least one data block, whereby the data node stores the data associated with the at least one data block. In some embodiments, the query system 110 can determine the data node corresponding to the at least one data block by determining the starting data value of the at least one data block and its storage address in the database. In some examples, the database may include at least one data table. In this way, the index database can be determined based on the data blocks, which can improve the efficiency of paginated queries.

[0035] Furthermore, the query system 110 can determine the join structure used to connect at least one data block. Even further, the query system 110 can determine the index database based on the at least one data block and the join structure. In some examples, the join structure can indicate a connection via pointers. Specifically, a down pointer can be used. This is understood to be merely exemplary and not limiting. In this way, the index database can be determined based on the data blocks, which can improve the efficiency of pagination queries.

[0036] In some embodiments, if the amount of business data in the database is greater than or equal to a data threshold, the query system 110 synchronizes the business data to the index database. In some examples, because database queries may be slow when the amount of business data is large, a dynamic index can be created and the business data synchronized to the index database when the data amount is greater than or equal to the data threshold. Furthermore, a sorting method can be set for the data during synchronization. For example, suppose the database includes a customer table, and a dynamic index is created only when the amount of data corresponding to a certain value in the customer_type field of the dynamic index table of the customer table is greater than 10,000. This is merely exemplary and not restrictive. In other examples, when the amount of business data is small (less than the data threshold), a corresponding dynamic index may not be created, and the database query can be performed directly. This is also merely exemplary and not restrictive. This approach can improve the efficiency of paginated queries.

[0037] In some embodiments, the query system 110 uses a scheduled task to synchronize data changes in the database to the index database. In some embodiments, the query system 110 can retrieve the corresponding data block from the index table based on the location of the changed data table in the database. Further, the query system 110 can determine the processing method for the data block based on a first data volume, a first data volume threshold, and a second data volume threshold. Further, the query system 110 processes the first database based on the determined processing method to obtain processed data blocks. Even further, the query system 110 can synchronize data changes between processed and unprocessed data blocks to the index database. In this way, the accuracy of paginated queries can be improved.

[0038] In some embodiments, if the first data volume is less than a first data volume threshold, the method for processing the data blocks is determined to instruct data block merging. In some examples, after data block merging is completed, the node value corresponding to the data block is also reduced accordingly based on the number of data blocks reduced. In this way, the accuracy of paginated queries can be improved.

[0039] In other embodiments, if the first data volume is greater than a second data volume threshold, the processing method for the data block is determined by instructing the data block to be split, where the second data volume threshold is greater than the first data volume threshold. In some examples, after the data block split is completed, the node value corresponding to the data block is also increased accordingly based on the number of data blocks added. In other examples, data retrieval or data insertion into the data block can be performed using multiple threads. In many more examples, the corresponding dynamic index is also updated after splitting or merging data blocks. In this way, the accuracy of paginated queries can be improved.

[0040] In other embodiments, in response to detecting data changes in the database, the detected data changes are synchronized to the index database. This improves the accuracy of paginated queries.

[0041] In some examples, a server can detect data changes in the database and synchronize these changes to the index database. Alternatively, the server could be a canal server. In other examples, Binlog (Binary Log) messages obtained from a canal server can be converted based on the index configuration in the index configuration table. A Binlog is a binary log file used to record all database change operations. Specifically, the Binlog message format uses "afterChangeOfColumns" to represent the list of fields before a change and "beforeChangeOfColumns" to represent the list of fields after a change. Furthermore, Binlog messages can be filtered to retain messages updating fields configured in the index configuration table. Even further, scheduled tasks for inserting and deleting data operations can be generated and communicated via messages. Moreover, based on the aforementioned update messages, the changes can be synchronized to the dynamic index data structure in the index database, and identifiers (IDs) in data blocks within the table can be inserted and deleted.

[0042] In some embodiments, the structure of the index database indicates an item from a skip list structure, a tree structure, or an ordered set. This improves the accuracy of paginated queries.

[0043] In box 220, in response to receiving a pagination query request for the database, query system 110 uses the index database to determine the index configuration that matches the pagination query request and the starting position of the query. This improves the accuracy and efficiency of pagination queries.

[0044] In some embodiments, query system 110 may perform data filtering on paginated query requests based on filtering conditions to determine a filtering ratio (filtered value) between the amount of filtered data corresponding to the paginated query request and the total amount of data. In some examples, a higher filtering ratio indicates that more data is filtered out. In other examples, the filtering ratio may also indicate the ratio of the number of rows returned in the execution plan to the total number of rows in the table. In still other examples, if the filtering ratio is less than a ratio threshold, query system 110 may directly determine the query results by querying the database. Specifically, the query results may be returned in the form of pages. Further, if the filtering ratio is greater than or equal to the ratio threshold, query system 110 may determine an index configuration that matches the paginated query request. In some examples, after the above index configuration, the number of pages to be queried in the paginated query request and the amount of data per page can be converted into an initial order value. The initial value calculation formula is as follows:

[0045] S=(P-1)×W (1)

[0046] Where S represents the starting value, P represents the page number, and W represents the amount of data per page.

[0047] Furthermore, the query system 110 can retrieve data nodes in the dynamic index using the starting value, and then find the corresponding data block through the data node. Since the value in the data node is the starting index of the data block, the Nth data in that data block is the ID of the initial query in the query request. The offset calculation formula is as follows:

[0048] N = SK (2)

[0049] Where N represents the offset, S represents the starting value, and K represents the data node value.

[0050] In some embodiments, the index configuration is determined based on an index configuration table, which includes data entries corresponding to at least one index configuration. Each data entry includes at least one of the following: sequence number (id), table name (table_name), index name (index_name), index type (dynamic_index_type), data threshold (threshold), and filtered value (filtered value). The index configuration table is shown in Table 1 below:

[0051] Table 1 Index Configuration Table

[0052]

[0053] In box 230, query system 110 determines the query statement against the database based on index configuration and the query start position. In some examples, the query statement may indicate Structured Query Language (SQL). In some examples, the query statement can be determined by dynamically concatenating the query start position with the LIMIT keyword and the amount of data per page using a plugin and SQL. In some examples, the plugin may be the MyBatis plugin. Specifically, the LIMIT keyword can be used to limit the number of records returned in the query results. In some examples, the LIMIT keyword can limit the return of the first N records. In other examples, the LIMIT keyword can limit the return of the N records starting from the i-th record. It should be understood that this is merely exemplary and not restrictive. In this way, the accuracy and efficiency of paginated queries can be improved.

[0054] In box 240, query system 110 determines the query results corresponding to the pagination query request from the database based on the query statement. In some examples, the query results may be multi-page data. In more examples, the query results may be returned as a single page. This approach improves the accuracy and efficiency of pagination queries.

[0055] The following is a reference. Figure 3 The example process for determining the index database is further illustrated below. It should be understood that this is merely exemplary and not restrictive.

[0056] Figure 3 A flowchart of a process 300 for determining an index database according to some embodiments of the present disclosure is shown. Process 300 may be implemented at query system 110. For ease of discussion, reference will be made to... Figure 1 The environment 100 is used to describe the process 300.

[0057] In box 310, query system 110 creates a dynamic index.

[0058] In some examples, dynamic indexes are created by inserting the table names from the business database that need optimization, along with the index information used in the business database, into the index configuration table. Furthermore, the created dynamic indexes can be used to determine or update the index database.

[0059] In box 320, query system 110 determines the data threshold.

[0060] In some examples, the data threshold can be determined through window configuration by adding a data threshold. Specifically, the data threshold can indicate the amount of data used to create a dynamic index.

[0061] In box 330, query system 110 reads data changes of business data in the database.

[0062] Specifically, changes in business data in the database can be monitored through scheduled tasks or servers.

[0063] In box 340, query system 110 to start a scheduled task.

[0064] In other examples, scheduled tasks can synchronize data changes in the business database to dynamic indexes by inserting or deleting data.

[0065] In box 350, query system 110 listens to the database through the server and returns asynchronous messages.

[0066] In some examples, a Canal server can listen to the database and return asynchronous messages. In other examples, Binlog messages received from the server can be transformed based on the index configuration in the index configuration table. Furthermore, Binlog messages can be filtered to retain messages that update fields configured in the index configuration table. Even further, the insert and delete operations required by scheduled tasks can be generated and communicated via messages. Moreover, based on the aforementioned update messages, the changes can be synchronized to the dynamic index data structure in the index database, and IDs in the table can be inserted and deleted from data blocks.

[0067] After executing box 340 or box 350, you can query system 110 to synchronize data changes in box 360. In some examples, query system 110 synchronizes data changes to index database 102.

[0068] The following is a reference. Figure 4 The example process for determining the query results is further illustrated below. It should be understood that this is merely exemplary and not restrictive.

[0069] Figure 4 A flowchart of a process 400 for determining query results according to some embodiments of the present disclosure is shown. Process 400 can be implemented at query system 110. For ease of discussion, reference will be made to... Figure 1 The environment 100 is used to describe the process 400.

[0070] In box 410, query system 110 receives pagination query requests.

[0071] After receiving a pagination query request, you can perform a query plan analysis in box 420, query system 110.

[0072] In some examples, execution plan analysis may include a two-stage scan. Specifically, assuming the database includes a customer table (customers) and an order table (orders), the first stage is a full table scan of the customer table, during which a specific field (customer_level) can be selected for filtering. Further, the second stage is an index scan of the order table, using a specific field (e.g., customer_id) as an index, and then filtering based on the selected criteria. This is merely exemplary and not restrictive. In other examples, the execution plan analysis may also determine a filtering ratio. In many more examples, the filtering ratio may indicate the ratio of the number of rows returned in the execution plan to the total number of rows in the table.

[0073] In box 430, query system 110 to verify whether the filter ratio is less than the ratio threshold.

[0074] In some examples, the filter ratio indicates the ratio of the amount of filtered data to the total amount of data. In other examples, a higher filter ratio means more data is filtered out.

[0075] If the filtering ratio is less than the ratio threshold, the query system 110 can directly determine the query result from the database 101 in box 435.

[0076] If the filter ratio is greater than or equal to the ratio threshold, in box 440, query system 110 to determine the matching index configuration from the index configuration table.

[0077] In some examples, the index configuration table can be determined from the index database 102. If a matching index configuration can be determined from the index configuration table, then in box 450, query system 110 can determine that the index configuration exists. If a matching index configuration cannot be determined from the index configuration table, then in box 450, query system 110 can determine that the index configuration does not exist.

[0078] In box 450, query system 110 to determine if the index configuration exists.

[0079] Furthermore, if query system 110 determines that the index configuration does not exist, then execute the above box 435.

[0080] If the index configuration is confirmed to exist, a query statement can be generated in box 460, query system 110.

[0081] In some examples, the query statement can refer to an SQL statement. In other examples, the query statement can be determined by dynamically concatenating the starting position of the query with the LIMIT keyword and the amount of data per page using the MyBatis plugin and SQL. In many more examples, in MySQL databases, pagination is typically performed using the LIMIT and OFFSET keywords. Specifically, the LIMIT keyword can be used to indicate the number of rows to return in the result set, and the OFFSET keyword can be used to indicate from which row in the result set the data should be returned.

[0082] In box 470, query system 110 determines the query results from the database based on the query statement.

[0083] In some examples, query results may be returned as pages. Specifically, this could be one page or multiple pages of data. This is merely illustrative and not restrictive.

[0084] The following is a reference. Figure 5 The example process for determining the skip list structure is further illustrated. It should be understood that this is merely exemplary and not restrictive.

[0085] Figure 5 A schematic diagram of a process 500 for determining a skip list structure according to some embodiments of the present disclosure is shown. Process 500 can be implemented at query system 110. For ease of discussion, reference will be made to... Figure 1 The environment 100 is used to describe the process 500.

[0086] like Figure 5 As shown, multiple data blocks can be determined from the database based on the data logical range. Specifically, the data in the database can be divided into data blocks 501, 510, 520, 530, 540, 550, 570, 580, and 590, respectively, corresponding to the data logical ranges 0-10W, 10W-20W, 20W-30W, 30W-40W, 40W-50W, 50W-50W, 50W-70W, 70W-80W, 80W-90W, and 90W-100W.

[0087] Furthermore, the data nodes corresponding to data block 501 (505), 510 (515), 520 (525), 530 (535), 540 (545), 550 (555), 560 (565), 570 (575), 580 (585), and 590 (595) can be identified. Specifically, each data node includes the starting data value of at least one data block and the storage address of at least one data block in the database.

[0088] Furthermore, the skip list structure can include a three-level index and a raw ordered linked list. Specifically, the first-level index can include data nodes 505, 545, and 585; the second-level index can include data nodes 505, 545, and 585; and the third-level index can include data nodes 505, 525, 545, 555, and 585. Additionally, the raw ordered linked list can include data nodes 505, 515, 525, 535, 545, 555, 565, 575, 585, and 595.

[0089] More specifically, suppose we want to find data node 535. First, we traverse the first, second, and third level indices of the skip list structure. While traversing the first level index, when we reach data node 505, we find that the next node is 545, so we determine that data node 535 is between data nodes 505 and 545. Further, while traversing the second level index, when we reach data node 505, we find that the next node is 545, so we determine that data node 535 is between data nodes 505 and 545. Further, while traversing the third level index, when we reach data node 525, we find that the next node is 545, so we determine that data node 535 is between data nodes 525 and 545. More specifically, using pointers, we descend to the next level of the original ordered linked list and continue traversing. At this point, we find that we only need to traverse two data nodes (data nodes 525 and 535) to find data node 535. This saves two data nodes compared to directly traversing the original ordered linked list.

[0090] In summary, the embodiments of this disclosure can utilize the index database to determine query results from the database, thereby improving the efficiency of paginated queries. Furthermore, they can synchronously update the index database based on data changes in the database, thereby improving the accuracy of paginated queries.

[0091] Embodiments of this disclosure also provide corresponding apparatus for implementing the above methods or processes.

[0092] Figure 6 A schematic structural block diagram of a paginated query apparatus 600 according to certain embodiments of the present disclosure is shown. Apparatus 600 may be implemented as or included in query system 110. Various modules / components in apparatus 600 may be implemented by hardware, software, firmware, or any combination thereof.

[0093] like Figure 6 As shown, the apparatus 600 includes a determining module 610 configured to determine an index database for the database. The apparatus 600 also includes a receiving module 620 configured to, in response to receiving a pagination query request for the database, use the index database to determine an index configuration matching the pagination query request and a query start position. The apparatus 600 further includes an execution module 630 configured to determine a query statement for the database based on the index configuration and the query start position. The apparatus 600 also includes a query module 640 configured to determine the query results corresponding to the pagination query request from the database based on the query statement.

[0094] In some embodiments, the receiving module 620 is further configured to perform data filtering on the pagination query request based on filtering conditions to determine the filtering ratio of the filtered data corresponding to the pagination query request to the total data volume; and to determine an index configuration that matches the pagination query request in response to the filtering ratio being greater than a ratio threshold.

[0095] In some embodiments, the index configuration is determined based on an index configuration table, which includes data entries corresponding to at least one index configuration. In other embodiments, each data entry includes at least one of a sequence number, table name, index name, index type, data threshold, and filter ratio.

[0096] In some embodiments, the determining module 610 is further configured to divide business data in the database into at least one data block; determine a data node corresponding to the at least one data block, the data node storing data related to the at least one data block; determine a connection structure for connecting the at least one data block; and determine an index database based on the at least one data block and the connection structure.

[0097] In some embodiments, the determining module 610 is further configured to determine the starting data value of at least one data block and the storage address of at least one data block in the database.

[0098] In some embodiments, the determining module 610 is further configured to synchronize the business data to the index database in response to the data volume of business data in the database being greater than or equal to a data threshold.

[0099] In some embodiments, the determining module 610 is further configured to synchronize data changes in the database to the index database using a scheduled task; and / or to synchronize the detected data changes to the index database in response to detecting data changes in the database.

[0100] In some embodiments, the database includes at least one data table, and the determining module 610 is further configured to retrieve the corresponding data block from the index data table based on the location of the changed data table in the database; determine the processing method for the data block based on a first data volume, a first data volume threshold, and a second data volume threshold; process the first database based on the determined processing method to obtain the processed data block; and synchronize the data changes between the processed data block and the unprocessed data block to the index database.

[0101] In some embodiments, the determining module 610 is further configured to determine, in response to a first data volume being less than a first data volume threshold, to indicate data block merging as a processing method for the data block; and to determine, in response to a first data volume being greater than a second data volume threshold, to indicate data block splitting as a processing method for the data block, wherein the second data volume threshold is greater than the first data volume threshold.

[0102] In some embodiments, the structure of the index database indicates one of a skip list structure, a tree structure, or an ordered set.

[0103] The units and / or modules included in device 600 can be implemented in various ways, including software, hardware, firmware, or any combination thereof. In some embodiments, one or more units and / or modules can be implemented using software and / or firmware, such as machine-executable instructions stored on a storage medium. In addition to or as an alternative to machine-executable instructions, some or all of the units and / or modules in device 600 can be implemented at least partially by one or more hardware logic components. By way of example and not limitation, exemplary types of hardware logic components that can be used include field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), complex programmable logic devices (CPLDs), and so on.

[0104] It should be understood that one or more steps in the above methods can be performed by suitable electronic devices or combinations of electronic devices. Such electronic devices or combinations of electronic devices may include, for example, […]. Figure 1 The query system 110 in the middle.

[0105] Figure 7 A block diagram of an electronic device 700 in which one or more embodiments of the present disclosure may be implemented is shown. It should be understood that... Figure 7The electronic device 700 shown is merely exemplary and should not be construed as limiting the functionality and scope of the embodiments described herein. Figure 7 The electronic device 700 shown can be used to achieve Figure 1 The query system 110 or Figure 6 Device 600.

[0106] like Figure 7 As shown, electronic device 700 is in the form of a general-purpose electronic device. Components of electronic device 700 may include, but are not limited to, one or more processors or processing units 710, memory 720, storage device 730, one or more communication units 740, one or more input devices 750, and one or more output devices 760. Processing unit 710 may be a physical or virtual processor and is capable of performing various processes according to programs stored in memory 720. In a multiprocessor system, multiple processing units execute computer-executable instructions in parallel to improve the parallel processing capability of electronic device 700.

[0107] Electronic device 700 typically includes multiple computer storage media. Such media can be any available media accessible to electronic device 700, including but not limited to volatile and non-volatile media, removable and non-removable media. Memory 720 can be volatile memory (e.g., registers, cache, random access memory (RAM)), non-volatile memory (e.g., read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory), or some combination thereof. Storage device 730 can be removable or non-removable media and can include machine-readable media, such as flash drives, disks, or any other media capable of storing information and / or data and accessible within electronic device 700.

[0108] Electronic device 700 may further include additional removable / non-removable, volatile / non-volatile storage media. Although not explicitly stated... Figure 7 As shown, disk drives for reading from or writing to removable, non-volatile disks (e.g., "floppy disks") and optical disk drives for reading from or writing to removable, non-volatile optical disks can be provided. In these cases, each drive can be connected to a bus (not shown) via one or more data media interfaces. Memory 720 may include computer program product 725 having one or more program modules configured to perform various methods or actions of various embodiments of this disclosure.

[0109] The communication unit 740 enables communication with other electronic devices via a communication medium. Additionally, the functionality of the components of the electronic device 700 can be implemented using a single computing cluster or multiple computing machines capable of communicating via communication connections. Therefore, the electronic device 700 can operate in a networked environment using logical connections to one or more other servers, network personal computers (PCs), or another network node.

[0110] Input device 750 can be one or more input devices, such as a mouse, keyboard, trackball, etc. Output device 760 can be one or more output devices, such as a monitor, speaker, printer, etc. Electronic device 700 can also communicate with one or more external devices (not shown) via communication unit 740 as needed. These external devices include storage devices, display devices, etc., and can communicate with one or more devices that enable user interaction with electronic device 700, or with any device that enables electronic device 700 to communicate with one or more other electronic devices (e.g., network card, modem, etc.). Such communication can be performed via input / output (I / O) interface (not shown).

[0111] According to an exemplary implementation of this disclosure, a computer-readable storage medium is provided that stores computer-executable instructions thereon, wherein the computer-executable instructions are executed by a processor to implement the methods described above. According to an exemplary implementation of this disclosure, a computer program product is also provided, which is tangibly stored on a non-transitory computer-readable medium and includes computer-executable instructions, which are executed by a processor to implement the methods described above.

[0112] Various aspects of this disclosure are described herein with reference to flowchart illustrations and / or block diagrams of methods, apparatuses, devices, and computer program products implemented according to this disclosure. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer-readable program instructions.

[0113] These computer-readable program instructions can be provided to a processing unit of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus to produce a machine such that, when executed by the processing unit of the computer or other programmable data processing apparatus, they create means for implementing the functions / actions specified in one or more blocks of the flowchart and / or block diagram. These computer-readable program instructions can also be stored in a computer-readable storage medium that causes a computer, programmable data processing apparatus, and / or other device to operate in a particular manner. Thus, the computer-readable medium storing the instructions comprises an article of manufacture that includes instructions for implementing aspects of the functions / actions specified in one or more blocks of the flowchart and / or block diagram.

[0114] Computer-readable program instructions can be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions that execute on the computer, other programmable data processing apparatus, or other device to perform the functions / actions specified in one or more boxes of a flowchart and / or block diagram.

[0115] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of an instruction, which contains one or more executable instructions for implementing the specified logical function. In some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, may be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions.

[0116] Various implementations of this disclosure have been described above. These descriptions are exemplary and not exhaustive, nor are they limited to the disclosed implementations. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described implementations. The terminology used herein is chosen to best explain the principles, practical applications, or improvements to technology in the market, or to enable others skilled in the art to understand the various implementations disclosed herein.

Claims

1. A method for paginated querying, comprising: Determine the index database for the database; In response to receiving a pagination query request for the database, the index database is used to determine the index configuration that matches the pagination query request and the query start position; Based on the index configuration and the query start position, determine the query statement for the database; as well as Based on the query statement, the query result corresponding to the pagination query request is determined from the database.

2. The method according to claim 1, wherein determining the index configuration matching the pagination query request using the index database includes: Based on the filtering conditions, data filtering is performed on the pagination query request to determine the filtering ratio of the filtered data volume to the total data volume corresponding to the pagination query request. as well as In response to the filtering ratio being greater than or equal to the ratio threshold, the index configuration matching the pagination query request is determined.

3. The method according to claim 2, wherein the index configuration is determined based on an index configuration table, the index configuration table including data entries corresponding to at least one index configuration, wherein each data entry includes at least one of a sequence number, table name, index name, index type, data threshold, and filter ratio.

4. The method of claim 1, wherein determining the index database comprises: Divide the business data in the database into at least one data block; Determine a data node corresponding to the at least one data block, wherein the data node stores data associated with the at least one data block; Determine the connection structure used to connect the at least one data block; as well as The index database is determined based on the at least one data block and the connection structure.

5. The method of claim 4, wherein determining the data node comprises: Determine the starting data value of the at least one data block and the storage address of the at least one data block in the database.

6. The method according to claim 1, further comprising: In response to a data volume in the database that is greater than or equal to a data threshold, the business data is synchronized to the index database.

7. The method according to claim 1, further comprising: Use scheduled tasks to synchronize data changes in the database to the index database; and / or In response to detecting data changes in the database, the detected data changes are synchronized to the index database.

8. The method of claim 7, wherein the database comprises at least one data table, and wherein synchronizing data changes in the database to the index database using a scheduled task comprises: Based on the location of the changed data table in the database, the corresponding data block is retrieved from the index data table; Based on the first data volume, the first data volume threshold, and the second data volume threshold of the data block, the processing method for the data block is determined; The first database is processed according to the determined processing method to obtain processed data blocks; The data changes between the processed data blocks and the unprocessed data blocks are synchronized to the index database.

9. The method according to claim 8, wherein determining the processing method for the data block includes: In response to the first data volume being less than the first data volume threshold, a processing method for the data block is determined, indicating that the data blocks should be merged. as well as In response to the first data volume being greater than the second data volume threshold, a processing method for the data block is determined, indicating that the data block is split, wherein the second data volume threshold is greater than the first data volume threshold.

10. The method of claim 1, wherein the structure of the index database indicates one of a skip list structure, a tree structure, or an ordered set.

11. An apparatus for paginated queries, comprising: The module is configured to determine the index database for the database. The receiving module is configured to, in response to receiving a pagination query request for a database, use the index database to determine the index configuration that matches the pagination query request and the query start position; The execution module is configured to determine the query statement for the database based on the index configuration and the query start position; as well as The query module is configured to determine the query results corresponding to the pagination query request from the database based on the query statement.

12. An electronic device, comprising: At least one processing unit; as well as At least one memory, coupled to the at least one processing unit and storing instructions for execution by the at least one processing unit, the instructions causing the electronic device to perform the method according to any one of claims 1 to 10 when executed by the at least one processing unit.

13. A computer-readable storage medium having a computer program stored thereon, the computer program being executable by a processor to implement the method according to any one of claims 1 to 10.

14. A computer program product comprising computer-executable instructions, wherein the computer-executable instructions, when executed by a processor, implement the method according to any one of claims 1 to 10.

Images