A method and apparatus for data query
By storing the association between query conditions and partition information in the cache, data can be retrieved directly from the partition table, solving the problem of low data query efficiency and achieving efficient data query.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2021-11-08
- Publication Date
- 2026-03-24
AI Technical Summary
As the amount of data in the data table increases, the data query efficiency becomes low, and high concurrency access places high demands on database thread resources, resulting in limited improvement in query efficiency.
By storing the association between query conditions and partition information in the cache, business data that meets the query conditions can be retrieved directly from the partition table, avoiding traversing the entire business data table.
It improves the speed and efficiency of data querying, especially when querying large datasets, and reduces the reliance on database thread resources.
Smart Images

Figure CN114020790B_ABST
Abstract
Description
TECHNICAL FIELD
[0001] The present application relates to the technical field of database, and particularly relates to a data query method and device. BACKGROUND
[0002] With the development of Internet technology, the amount of data in a data table also increases exponentially, resulting in lower and lower efficiency of data table query. One solution is to improve query efficiency by supporting high concurrency access. However, high concurrency access has a high requirement for database thread resources, and database thread resources are limited, resulting in limited improvement of query efficiency. SUMMARY
[0003] Embodiments of the present application provide a data query method and device to solve the problem of slow response speed when querying data.
[0004] In a first aspect, the embodiments of the present application provide a data query method, which can be applied to a database server. The method can include: receiving a first query request sent by a client, the first query request carrying a first query condition; obtaining first partition information matching the first query condition from a cache according to the first query condition; wherein the cache includes first partition information corresponding to the first query condition, and the first partition information includes identifiers of N partition tables that are hit by the first query condition from a business data table before the first query request is received, the business data table is divided into a plurality of partition tables, and N is a positive integer; querying business data satisfying the first query condition from partition tables indicated by the identifiers of the N partition tables; and sending the business data to the client.
[0005] Based on the above scheme, when a large data table is queried, partition information satisfying the same query condition is queried in the cache, and then business data satisfying the query condition is directly obtained according to the partition information, without traversing the entire business data table, thereby greatly improving the query speed.
[0006] In some embodiments, the method further includes: receiving a second query request, the second query request carrying a second query condition; when no partition information matching the first query condition is obtained from the cache according to the second query condition, querying business data satisfying the second query condition from the business data table, and associatively saving the second query condition and second partition information in the cache, the second partition information including a partition identifier of a partition table in which the business data satisfying the second query condition is located.
[0007] Based on the above scheme, when there is no partition information matching the second query condition in the cache, after the business data satisfying the second query condition is queried from the plurality of partition tables included in the business data table, the partition information where the queried business data is located and the second query condition are associated and saved in the cache. If a request containing the second query condition is received subsequently, the partition information can be obtained from the cache without traversing all the partition tables again, and the query efficiency can be improved.
[0008] In some embodiments, the first partition information further includes a data quantity of the N partition tables satisfying the first query condition; the first query request further carries page information of a page to be jumped; and querying the business data satisfying the first query condition from the partition tables respectively indicated by the identifiers of the at least one partition table includes: determining at least one partition table of the N partition tables including target data of a page to be jumped indicated by the page information according to the data quantity of the N partition tables satisfying the first query condition; and obtaining the target data from the at least one partition table.
[0009] Based on the above scheme, the partition where the target data of the current display page is located and the starting position in the partition are determined through the page information, the partition and the starting position in the partition are directly used to obtain the target data in the data table, full table scanning is avoided, and the query speed is improved.
[0010] In some embodiments, the page information includes a quantity of business data displayed per page and a page code of a page to be jumped; and determining at least one partition table of the N partition tables including target data of a page to be jumped indicated by the page information according to the data quantity of the N partition tables respectively satisfying the first query condition includes: determining an index range of the target data to be displayed in the page to be jumped in the business data satisfying the first query condition according to the page code of the page to be jumped and the quantity of business data displayed per page.
[0011] At least one partition table of the N partition tables including the target data to be displayed in the current page to be jumped is determined according to the data quantity corresponding to the N partition tables respectively and the index range.
[0012] Based on the above scheme, when a user performs page turning or another user sends a query request with the same query condition through a client, only the position of the target data to be queried in the partition table needs to be calculated again, and then the corresponding business data is accurately found from the business data table according to the position of the partition table, so that the response speed of the query is improved.
[0013] In some embodiments, a start index position of the index range satisfies a condition shown in the following formula:
[0014] start = (Pagenum - 1) x Pagesize + 1;
[0015] The end index position of the index range satisfies the condition shown in the following formula:
[0016] end = Pagenum x Pagesize;
[0017] Wherein, Pagesize represents the number of business data displayed per page, Pagenum represents the page number of the page to be jumped, start represents the start index position of the index range, and end represents the end index position of the index range.
[0018] In some embodiments, the at least one partition table includes a first partition table, and the at least one partition table including the target data to be displayed on the page to be jumped is determined according to the data quantity corresponding to each of the N partition tables and the index range, including: determining, according to the query order of the N partition tables, that the data quantity included in N1 partition tables located before the first partition table in the query order is less than the start index position of the index range, and the sum of the data quantity included in the N1 partition tables and the data quantity included in the first partition table is greater than the start index position.
[0019] In some embodiments, the target data is obtained from the at least one partition table, including: obtaining business data satisfying the first query condition from the at least one partition table; when the sum of the data quantity included in the N1 partition tables and the data quantity included in the first partition table is greater than or equal to the end index position, determining the start position and the end position of the target data in the business data satisfying the first query condition included in the first partition table according to the start index position and the end index position; or,
[0020] When the sum of the data quantity included in the N1 partition tables and the data quantity included in the first partition table is less than the end index position, determining the start position of the target data in the business data satisfying the first query condition included in the first partition table according to the start index position; the at least one partition table further includes a second partition table, and the query order of the second partition table in the N partition tables is located after the first partition table; and the end position of the target data in the business data satisfying the first query condition included in the second partition table is determined according to the end index position.
[0021] Based on the above scheme, when data query is performed, only the start position and the end position of the business data in the partition that meets the first query condition and that is involved by the target data displayed in the page to be jumped are calculated, and the business data is directly obtained from the partition, thereby improving the response speed of data query.
[0022] In a second aspect, an embodiment of the present application provides a data query device, including:
[0023] A receiving module is configured to receive a first query request sent by a client, the first query request carrying a first query condition;
[0024] A processing module is configured to obtain first partition information matched with the first query condition from a cache according to the first query condition;
[0025] The cache includes the first partition information corresponding to the first query condition, and the first partition information includes identifiers of N partition tables that are hit by the first query condition from a business data table before the first query request is received, the business data table is divided into a plurality of partition tables, and N is a positive integer;
[0026] The processing module is further configured to query business data meeting the first query condition from the partition tables indicated by the identifiers of the N partition tables respectively;
[0027] A sending module is configured to send the business data to the client.
[0028] In some embodiments, the receiving module is further configured to receive a second query request, the second query request carrying a second query condition; and the processing module is further configured to query business data meeting the second query condition from partition tables included in the business data table according to the second query condition when no partition information matched with the first query condition is obtained from the cache according to the second query condition, and to save the second query condition and second partition information in association in the cache, the second partition information including a partition identifier of a partition table in which the business data meeting the second query condition is located.
[0029] In some embodiments, the first partition information further includes a data quantity of the N partition tables meeting the first query condition; and the first query request further carries page information of a page to be jumped;
[0030] When the processing module queries business data meeting the first query condition from the partition tables indicated by the identifiers of the at least one partition table respectively, the processing module is specifically configured to: determine at least one partition table of the N partition tables that includes target data of the page to be jumped indicated by the page information according to the data quantity of the N partition tables meeting the first query condition; and obtain the target data from the at least one partition table.
[0031] In some embodiments, the page information includes a number of business data displayed per page and a page number of a page to be jumped;
[0032] When the processing module determines at least one partition table including target data of the page to be jumped indicated by the page information from the N partition tables according to the number of data satisfying the first query condition in the N partition tables, the processing module is specifically configured to: determine an index range of the target data to be displayed in the page to be jumped in the business data satisfying the first query condition according to the page number of the page to be jumped and the number of business data displayed per page.
[0033] According to the number of data corresponding to the N partition tables respectively and the index range, at least one partition table including the target data to be displayed in the current page to be jumped in the N partition tables is determined.
[0034] In some embodiments, a start index position of the index range satisfies a condition shown in the following formula:
[0035] start = (Pagenum-1) x Pagesize + 1;
[0036] An end index position of the index range satisfies a condition shown in the following formula:
[0037] end = Pagenum x Pagesize;
[0038] Wherein, Pagesize represents the number of business data displayed per page, Pagenum represents the page number of the page to be jumped, start represents the start index position of the index range, and end represents the end index position of the index range.
[0039] In some embodiments, the at least one partition table includes a first partition table, and the processing module is specifically configured to: determine, according to a query order of the N partition tables, that N1 partition tables located before the first partition table in the query order include data whose number is less than the start index position of the index range, and a sum of the number of data included by the N1 partition tables and the number of data included by the first partition table is greater than the start index position.
[0040] In some embodiments, the processing module, when obtaining the target data from the at least one partition table, is specifically configured to: obtain service data satisfying the first query condition from the at least one partition table; when the sum of the number of data included in the N1 partition tables and the number of data included in the first partition table is greater than or equal to the end index position, determine the start position and the end position of the target data in the service data satisfying the first query condition included in the first partition table according to the start index position and the end index position; or,
[0041] when the sum of the number of data included in the N1 partition tables and the number of data included in the first partition table is less than the end index position, determine the start position of the target data in the service data satisfying the first query condition included in the first partition table according to the start index position; the at least one partition table further includes a second partition table, the second partition table is located after the first partition table in the query order in the N partition tables; and the end position of the target data in the service data satisfying the first query condition included in the second partition table is determined according to the end index position.
[0042] In a third aspect, an embodiment of the present application provides a data query device, including:
[0043] a memory and a processor;
[0044] a memory configured to store program instructions;
[0045] a processor configured to invoke the program instructions stored in the memory and execute the method according to any one of the embodiments of the first aspect.
[0046] In a fourth aspect, an embodiment of the present application provides a computer readable storage medium storing computer instructions, when the computer instructions are run on a computer, the computer is caused to execute the method according to the first aspect or any one of the embodiments of the first aspect.
[0047] In addition, the technical effects brought by any one of the implementation manners of the second aspect to the fourth aspect can refer to the technical effects brought by the different implementation manners of the first aspect, which will not be described here. BRIEF DESCRIPTION OF DRAWINGS
[0048] In order to more clearly illustrate the technical solutions of the embodiments of the present application or the prior art, the following will briefly introduce the drawings needed to be used in the embodiments or the prior art description. Obviously, the drawings in the following description are some embodiments of the present application, and those skilled in the art can obtain other drawings according to these drawings without creative labor.
[0049] Figure 1 A flowchart illustrating a data query method provided in an embodiment of this application;
[0050] Figure 2A A schematic diagram illustrating that the page to be redirected to in this application embodiment includes only business data from a single partition table;
[0051] Figure 2B A schematic diagram illustrating the business data of two partition tables provided in this embodiment of the application;
[0052] Figure 2C A schematic diagram illustrating the business data of three partition tables provided in this embodiment of the application;
[0053] Figure 3 A schematic diagram of a data query device provided in an embodiment of this application;
[0054] Figure 4 This is a schematic diagram of the structure of a database server provided in an embodiment of this application;
[0055] Figure 5 This is a schematic diagram of the architecture of a database system provided in an embodiment of this application. Detailed Implementation
[0056] To make the objectives, implementation methods and advantages of this application clearer, the exemplary implementation methods of this application will be clearly and completely described below with reference to the accompanying drawings of the exemplary embodiments of this application. Obviously, the described exemplary embodiments are only some embodiments of this application, and not all embodiments.
[0057] Based on the exemplary embodiments described in this application, all other embodiments obtained by those skilled in the art without inventive effort are within the scope of protection of the appended claims. Furthermore, although the disclosures in this application are presented by way of one or more exemplary examples, it should be understood that each aspect of these disclosures can also constitute a complete implementation on its own.
[0058] It should be noted that the brief descriptions of terms in this application are only for the convenience of understanding the embodiments described below, and are not intended to limit the embodiments of this application. Unless otherwise stated, these terms should be understood in their ordinary and common meaning.
[0059] Redis caching solutions cache frequently accessed data. When querying frequently accessed data, it can be retrieved directly from the cache without accessing the entire table. Each access request to the table uses one I / O (Input / Output) thread. If several access requests retrieve frequently accessed data, it can be retrieved directly from the cache without using an I / O thread to access the table. This solution addresses the problem of slow queries due to insufficient I / O caused by high concurrency. However, Redis caches cannot store large amounts of data, and their use is limited when accessing large tables. Redis database queries typically use Structured Query Language (SQL). SQL optimization improves query speed by joining many tables. However, when querying large tables, low SQL execution efficiency can lead to connection timeouts, preventing normal data returns and significantly limiting query speed.
[0060] This application proposes a data query method and apparatus. By associating query conditions with the partitioned table containing the business data that meets the query conditions, and storing this as partition information in a cache, when a query request with the same query conditions is received, the partition information can be directly retrieved from the database cache. Then, the business data can be retrieved based on the partition information, avoiding traversing the entire business data table and thus improving query speed. Compared to Redis caching solutions, this application does not store the data itself, avoiding the problem of insufficient caching when querying large data tables. Compared to SQL-optimized queries, this application can quickly locate the partitioned table containing the query data, further improving query speed.
[0061] The following is a brief description of the technical terms used in this application. Redis is an open-source, high-performance key-value database, and a fast in-memory NoSQL database. Keys can be stored as strings. Values can be stored as strings, lists, sets, hash tables, or sorted sets. Redis supports many features, such as persisting data from memory to disk, using replication to improve read performance, and using sharding to improve write performance.
[0062] In Redis, a List is a linked list of strings ordered by insertion. Redis allows adding new elements to the head (left) and tail (right) of the linked list corresponding to a key, which then serve as a value matching that key. Redis creates different linked lists for different keys. If all elements in the linked list corresponding to a key are deleted, the key corresponding to that linked list will also be deleted from the database.
[0063] The database tables involved in this application, which can be simply referred to as data tables or tables, are one of the most important components of a database. A database is a framework, while data tables are the actual content within it. For example, in a "teaching management system," the teaching management database contains four tables, each revolving around a specific theme: a "Teacher" table, a "Course" table, a "Grade" table, and a "Student" table, used to manage information related to students, teachers, and courses during the teaching process. These independent data tables are linked together through relationships, forming a database that can be cross-referenced and is easily understood. Each data table in the database corresponds to a physically existing table file. The content of the data tables is stored on the physical disk in the form of table files.
[0064] The partition table involved in this application is obtained by dividing a business data table. In some embodiments, a business data table can be divided into multiple partition tables based on the date. Each partition table corresponds to a specific date. Of course, other partitioning methods can also be used, and this application does not specifically limit them.
[0065] This application provides a method for data querying. Figure 1 An exemplary flow of a data query method is illustrated, which can be executed by a database server. The database server can be deployed using Redis technology. The data query method flow includes:
[0066] 101. The database server receives a query request.
[0067] In some embodiments, the query request may originate from a client. For example, a user queries business data in the database server through a browser. The browser may use a paginated query method, in which case the query request may be a request to retrieve business data for a specific page.
[0068] For example, a query request may include query conditions. Query conditions may include the identifier of the business data table to be queried, and the conditions that the queried data must meet. For example, in a vehicle management database, query conditions may include license plates, vehicle logos, etc. For ease of description, this query request will be referred to as the first query request. The query conditions carried by this query request will be referred to as the first query conditions.
[0069] 102. Based on the first query condition, match the cache to see if it includes partition information that satisfies the first query condition.
[0070] In some embodiments, when the cache does not include partition information that meets the first query condition, business data that meets the first query condition is queried from each partition included in the business data table according to the first query condition.
[0071] In one possible example, when business data that meets the first query condition is found, the identifiers of the partition tables to which each piece of business data belongs, along with the first query condition, can be saved in the cache. The partition information can be understood as the identifiers of the partition tables associated with and the first query condition stored in the cache.
[0072] For example, when associating and saving the identifiers of partition tables with the first query condition, a key-value storage method can be used. For instance, the first query condition can be used as the key, and the identifiers of each partition table that satisfies the first query condition can be used as the value. Optionally, the value can be in list format.
[0073] In another possible example, the identifiers of the partition tables to which each piece of business data belongs, the number of business data in each partition table that meets the first query condition, and the first query condition itself can be stored together. The partition information can be understood as the identifiers of the partition tables stored in the cache, the number of business data in each partition table that meets the first query condition, and the first query condition itself.
[0074] For example, the identifier of the partition table, the number of business data in the partition table that satisfy the first query condition, and the first query condition can be stored using a key-value storage method. For instance, the first query condition can be used as the key, and the identifiers of each partition table that satisfies the first query condition, as well as the number of business data in the partition table that satisfies the first query condition, can be used as the value. Optionally, the value can be in list format. In this embodiment, a row in the business data table can be understood as a single data entry, and the number of business data entries that satisfy the first query condition can be understood as the number of entries in the partition table that satisfy the first query condition.
[0075] In other embodiments, when the database server determines that the cache includes partition information that satisfies the first query condition, it executes step 103. For ease of description, the partition information in the cache that satisfies the first query condition is referred to as the first partition information. In some embodiments, the first partition information includes the identifiers of N partition tables that were accessed and hit from the business data table using the first query condition before the first query request was received, and the business data table is divided into multiple partition tables. This can be understood as follows: before receiving the first query request, the database server receives other requests to query business data using the first query condition. After querying the business data table based on the request and finding business data that satisfies the first query condition, it can associate and store the identifiers of the N partition tables containing the business data that satisfies the first query condition, along with the query condition, as the first partition information in the cache. Therefore, when the first query request is received, the N partition tables that satisfy the first query condition can be directly retrieved from the cache.
[0076] For example, the first partition information may also include the number of data in each of the N partition tables that satisfy the first query condition.
[0077] 103. Query business data that meets the first query condition from the partition tables indicated by the identifiers of the N partition tables respectively.
[0078] The above approach prioritizes querying the cache for partition tables that meet certain query conditions. This allows the retrieved business data that satisfies the query conditions from the retrieved partition tables, eliminating the need to traverse the entire business table and thus improving query efficiency.
[0079] Optionally, the data query method may also include step 104:
[0080] 104. The database server sends the retrieved business data to the client.
[0081] In one possible implementation, when partitioning the business data table by date, one date corresponds to one partition table. In this case, if the query condition includes one or more dates, when the database server receives the query condition, it can first determine the corresponding partition table based on those dates, and then retrieve the business data that satisfies the other query parameters from the corresponding partition table. If the query condition only includes one date, then that date corresponds to one partition table, and only that partition table needs to be queried, significantly improving query speed.
[0082] In one possible scenario, a user can query business data in the database server through a browser. The browser can use a paginated query method, in which the query request can be a request to query business data for a specific page. For example, if the business data that meets the query conditions needs to be displayed on multiple pages, the user can trigger the query request for that page when submitting a pagination request through the browser. For example, the first query request can also carry pagination information for the page to be navigated to. The first partition information includes the identifier of the partition table, the number of business data in the partition table that meet the first query conditions, and the first query conditions.
[0083] In some embodiments, when querying business data that meets the first query condition from N partition tables, it can be achieved in the following way:
[0084] Based on the number of data in the N partition tables that satisfy the first query condition, at least one partition table is determined, which includes the target data to be paged as indicated by the pagination information; then, the target data is obtained from the at least one partition table.
[0085] In some embodiments, after obtaining the target data, the target data can be returned to the client, and the received target data can be displayed on the browser's page to be redirected.
[0086] In some embodiments, administrators can configure a cleanup cycle for cached partition information in the database server, allowing the data server to periodically clean up the cached partition information according to the cleanup cycle. In one possible approach, if a query condition in the cache is not matched by a query request at the cleanup time point of the cleanup cycle, the partition information corresponding to that query condition can be deleted from the cache. For example, if a query request in the cache is matched, a match indication can be added to the query condition. Thus, at the cleanup time point of the cleanup cycle, the indication determines whether the partition information needs to be deleted from the cache. At the time point, the cleanup cycle ends, and the next cleanup cycle begins. At the start of the next cleanup cycle, the indication for query conditions corresponding to partition information that has not been deleted can be updated, indicating that the query condition corresponding to that partition information was not matched. Before the end of the cleanup cycle, if the query condition corresponding to that partition information is matched, the indication corresponding to that query condition is updated so that the indication indicates that the query condition was matched. For example, the cleanup cycle can be one hour, one week, one day, etc.
[0087] In other embodiments, administrators can set a threshold for the number of times a partition information in the cache is hit. When the database server determines the cleanup time point of a certain cleanup cycle, if the number of times a query condition corresponding to a certain partition information in the cache is hit is less than the threshold, the partition information is deleted from the cache. The database server can update the hit count of the partition information corresponding to the query condition each time the query condition is hit. At the beginning of each cleanup cycle, the hit count of the partition information can be reset to zero so that the count can be restarted in the cleanup cycle.
[0088] In some embodiments, the first query request described above may also carry pagination information, which includes the number of business data entries displayed per page and the page number to be navigated to. When determining at least one partition table from N partition tables that contains the target data to be paged, this can be achieved as follows:
[0089] Based on the page number of the page to be navigated to and the number of business data items displayed on each page, determine the index range of the target data to be displayed in the business data that satisfies the first query condition. Then, based on the data quantity corresponding to the N partition tables and the index range, determine at least one partition table among the N partition tables that includes the target data to be displayed in the current page to be navigated to.
[0090] For example, the number of business data entries displayed per page can be set to Pagesize, and the page number of the page to be navigated to can be set to Pagenum. The index range can be represented by the start index position and the end index position. For example, if the start index position is represented by start, then the start index position of the index range satisfies the condition shown in the following formula: start = (Pagenum - 1) × Pagesize + 1.
[0091] For example, if the end index position is represented by end, then the end index position of the index range satisfies the condition shown in the following formula: end = Pagenum × Pagesize.
[0092] In some embodiments, when retrieving the target data from the at least one partition table, the start and end positions of the target data in the at least one partition table can be determined based on the number of business data corresponding to the N partition tables respectively and the index range of the target data of the page to be redirected in the business data that meets the first query condition. Thus, the target data can be retrieved from the business data that meets the query included in the at least one partition table based on the start and end positions.
[0093] Through the solution provided by the embodiments of the present application, when there are query requests with the same query conditions, that is, when the user uses a browser to perform paging or other users send query requests with the same query conditions through the client, there is no need to re-determine the partition table where the business data that meets the query conditions is located. Only the position of the target data to be queried in the partition table needs to be recalculated, and then the corresponding business data can be accurately found from the business data table according to the partition position where it is located, without traversing the entire data table, which improves the query speed.
[0094] Taking the first partition table as an example in at least one partition table including the target data of the page to be jumped. It can be understood that the partition table including the target data of the page to be jumped can be one or multiple. The present application does not make specific limitations on this. Cache N partition tables indicating business data that meets the first query condition. All of the N partition tables include business data that meets the first query condition. When using a browser to query, the number of business data displayed on each page is fixed. Therefore, a page may display the business data in one or several of the N partition tables. In some embodiments, the N partition tables are configured with a query order. The order can be determined according to the date or other rules can be used to determine.
[0095] Exemplarily, when determining the first partition table from the N partition tables including business data that meets the first query condition, the sum of the data quantities included in N1 partition tables located before the query order of the first partition table can be determined to be less than the start index position of the index range, and the sum of the data quantities included in the N1 partition tables and the data quantity included in the first partition table is greater than the start index position.
[0096] For example, the number of business data that meets the first query condition included in the i-th partition table among the N partition tables can be represented by a(i), then the total number s(i) of business data that meets the first query condition in the first i partition tables can be expressed as: s(i)=a(1)+a(2)+a(3)……+a(i).
[0097] In a possible example, when the total number of business data that meets the first query condition in the first i-1 partition tables is less than the start index position of the target data displayed by the page number to be jumped, that is, s(i-1)<start, then there is no target data displayed by the page number to be jumped in the i-th partition table. The total number of business data that meets the first query condition included in the first i partition tables can be calculated continuously. If s(i)>start. Then the i-th partition table is the partition table including the target data displayed by the page number to be jumped. For example, the i-th partition table can be understood as the first partition table.
[0098] In another possible example, the total number of business data that meet the first query condition in the current i partition tables is equal to the start index position of the target data displayed by the page number of the page to be jumped, that is, s(i) = start. Then, the last piece of business data in the business data that meet the first query condition in the i-th partition table is the target data displayed by the page number of the page to be jumped. For example, the i-th partition table can be understood as the first partition table.
[0099] Further, after determining at least one partition table, obtain the target data from at least one partition table. For example, first obtain the business data that meet the first query condition from at least one partition table. Then, determine the start position and the end position of the target data included in this partition table according to the number of data included in the first partition table.
[0100] In one example, when the sum of the number of data included in the N1 partition tables and the number of data included in the first partition table is greater than or equal to the end index position, determine the start position and the end position of the target data in the business data that meet the first query condition included in the first partition table according to the start index position and the end index position.
[0101] In another example, when the sum of the number of data included in the N1 partition tables and the number of data included in the first partition table is less than the end index position, determine the start position of the target data in the business data that meet the first query condition included in the first partition table according to the start index position; at least one partition table further includes a second partition table, and the query order of the second partition table in the N partition tables is after the first partition table; determine the end position of the target data in the business data that meet the first query condition included in the second partition table according to the end index position.
[0102] For example, continuing with the above example, s(i - 1) < start. If the total number of business data that meet the first query condition in the first i partition tables is greater than the start index position of the target data displayed by the page number of the page to be jumped, that is, s(i) > start, and the total number of business data that meet the first query condition in the first i partition tables is greater than or equal to the end index position of the target data displayed by the page number of the page to be jumped, that is, s(i) > end, then the target data displayed by the page number of the page to be jumped only includes the business data in the i-th partition. The start position and the end position of the target data displayed by the page number of the page to be jumped in the business data that meet the first query in the i-th partition table are:
[0103] The starting position is: start - (s(i - 1) – used(0)), where used(0) is the number of target data displayed as the page number to be jumped to among the service data that meet the first query condition in the previous i - 1 partition tables, that is, the target data displayed in the page number to be jumped to includes the service data after the (start - (s(i - 1) – used(0)))-th service data among the service data that meet the first query condition in the i-th partition table.
[0104] The ending position is: end - s(i - 1), that is, the target data displayed in the page number to be jumped to includes the service data before the (end - s(i - 1))-th service data among the service data that meet the first query condition in the i-th partition table.
[0105] Since s(i - 1) < start, the number of service data that meet the first query condition in the (i - 1)-th partition as the target data displayed in the page number to be jumped to is zero, that is, used(0) = 0.
[0106] In addition, s(i) > end, that is, the ending index position of the target data displayed in the page number to be jumped to is among the service data that meet the first query condition in the i-th partition table. At this time, the target data displayed in the page number to be jumped to has been fetched completely, and there is no need to calculate the number of service data that meet the first query condition in the previous i partition tables as the target data displayed in the page number to be jumped to.
[0107] For example, as shown in Figure 2A it can be seen that the number of service data displayed per page can be set to 7. Taking the page number to be jumped to as 4, the starting index position of the index range is represented by start, start = 22, and the ending index position of the index range is represented by end, end = 28. When i = 4, a(4) = 10, s(3) = 20, s(4) = 30, s(3) < start, and s(4) > end. At this time, the number of service data that meet the first query condition in the 3rd partition table as the target data displayed in the page number to be jumped to is: used(0) = 0, and the starting position of the target data displayed in the page number to be jumped to among the service data that meet the first query condition in the 4th partition is: start - (s(i - 1) – used(0)) = 22 - (20 - 0) = 2, and the ending position is: end - s(i - 1) = 28 - 20 = 8, that is, the target data displayed in the page number to be jumped to includes the 2nd to 8th service data among the service data that meet the first query condition in the 4th partition table.
[0108] For another example, if s(i - 1) < start, and the total number of business data that meet the first query condition in the first i partition tables is greater than the start index position of the target data displayed on the page number to be jumped, that is, s(i) > start, and the total number of business data that meet the first query condition in the first i partition tables is less than the end index position of the target data displayed on the page number to be jumped, that is, s(i) < end, then the target data displayed on the page number to be jumped includes the business data in the i-th partition table and also includes the business data in the (i + 1)-th partition table. If it is determined that the total number of business data that meet the first query condition in the first i + 1 partition tables is greater than or equal to the end index position of the target data displayed on the page number to be jumped, the target data displayed on the page number to be jumped is part of the business data that meet the first query condition in the (i + 1)-th partition.
[0109] The start position and end position of the target data displayed on the page number to be jumped in the business data that meet the first query condition in the i-th partition table are:
[0110] The start position is: start - (s(i - 1) – used(0)), where used(0) is the number of target data displayed in the page number to be jumped among the business data that meet the first query condition in the first i - 1 partition tables. That is, the target data displayed on the page number to be jumped includes the business data after the start - (s(i - 1) – used(0))-th business data among the business data that meet the first query condition in the i-th partition table.
[0111] The end position is: t, where t is the value of the number of business data that meet the first query condition in the i-th partition table, t = a(i). The target data displayed on the page number to be jumped includes the business data before the t-th business data among the business data that meet the first query condition in the i-th partition table.
[0112] The start position and end position of the target data displayed on the page number to be jumped in the business data that meet the first query condition in the (i + 1)-th partition table are:
[0113] The start position is: start - (s(i) – used(1)), where used(1) is the number of target data displayed in the page number to be jumped among the business data that meet the first query condition in the first i partition tables. That is, the target data displayed on the page number to be jumped includes the business data after the start - (s(i) – used(1))-th business data among the business data that meet the first query condition in the (i + 1)-th partition table.
[0114] The ending position is: end-s(i), which means that the target data displayed in the page number of the page to be redirected includes the business data before the end-s(i)th business data in the business data that meets the first query condition in the (i+1)th partition table.
[0115] At this point, the number of business data in the first i-1 partition tables that satisfy the first query condition, and the number of target data displayed in the page number to be redirected, satisfies the following formula:
[0116] used(0) = 0.
[0117] The number of business data in the first i partition tables that satisfy the first query condition, and the number of target data displayed in the page number to be redirected to, satisfy the following formula:
[0118] used(1)=finish(1)-begin(1)+1;
[0119] Here, begin(1) and finish(1) are the start and end positions of the target data in the business data of the i-th partition table that meets the first query condition, respectively, for the page number to be displayed on the page to be redirected.
[0120] For example, see Figure 2BAs shown, set the number of business data displayed per page to 7 and set the page number of the page to be jumped to 4. Then, the start index position of the index range is represented by start, start = 22, and the end index position of the index range is represented by end, end = 28. When i = 4, a(4) = 7, s(4) = 25, and s(3) = 18, s(3) < start, s(4) > start and s(4) < end. At this time, the target data displayed in the page number of the page to be jumped includes the business data that meets the first query condition in the 4th and 5th partition tables. The number of business data that meets the first query condition in the first 3 partition tables as the target data displayed in the page number of the page to be jumped is: used(0) = 0. The start position of the business data that meets the first query interval in the 4th partition of the target data displayed in the page number of the page to be jumped is: start - (s(i - 1) – used(0)) = 22 - (18 - 0) = 4, and the end position is: t = a(i) = a(4) = 7, that is, the target data displayed in the page number of the page to be jumped includes the 4th to 7th business data in the business data that meets the first query condition in the 4th partition table. After that, it is also necessary to calculate the position of the target data displayed in the page number of the page to be jumped in the business data that meets the first query condition in the 5th partition table. When a(5) = 6, that is, s(5) = 31, the number of business data that meets the first query condition in the first 4 partition tables as the target data displayed in the page number of the page to be jumped is: used(1) = finish(1) - begin(1) + 1 = 7 - 4 + 1 = 4. The start position of the business data that meets the first query interval in the 5th partition of the target data displayed in the page number of the page to be jumped is: start - (s(i) – used(1)) = 22 - (25 - 4) = 1, and the end position is: end - s(i) = 28 - 25 = 3, that is, the target data displayed in the page number of the page to be jumped also includes the 1st to 3rd business data in the business data that meets the first query condition in the 5th partition table.
[0121] For another example, if s(i - 1) < start, and the total number of service data that meet the first query condition in the first i partition tables is greater than the start index position of the target data displayed by the page number of the page to be jumped, that is, s(i) > start, and the total number of service data that meet the first query condition in the first i partition tables is less than the end index position of the target data displayed by the page number of the page to be jumped, that is, s(i) < end, then the target data displayed by the page number of the page to be jumped includes the service data in the i-th partition table. It also includes the service data in the (i + 1)-th partition table. After that, calculate the number of service data that meet the first query condition in the first i partition tables as the number of target data displayed by the page number of the page to be jumped. If it is determined that the total number of service data that meet the first query condition in the first (i + 1) partition tables is less than the end index position of the target data displayed by the page number of the page to be jumped, the target data displayed by the page number of the page to be jumped also includes the service data in the (i + 2)-th partition table. After that, calculate the number of service data that meet the first query condition in the first (i + 1) partition tables as the number of target data displayed by the page number of the page to be jumped. And so on. If it is determined that the total number of service data that meet the first query condition in the first (i + r) partition tables is greater than or equal to the end index position of the target data displayed by the page number of the page to be jumped, then the target data displayed by the page number of the page to be jumped is the data among the service data that meet the first query condition in the (i + r)-th partition table. At this time, the target data displayed by the page number to be jumped is in the i-th partition table to the (i + r)-th partition table.
[0122] The start position and end position of the target data displayed by the page number of the page to be jumped among the service data that meet the first query condition in the i-th partition table are:
[0123] The start position is: start - (s(i - 1) – used(0)), where used(0) is the number of target data displayed by the page number of the page to be jumped among the service data that meet the first query condition in the first i - 1 partition tables. That is, the target data displayed by the page number of the page to be jumped includes the service data after the (start - (s(i - 1) – used(0)))-th service data among the service data that meet the first query condition in the i-th partition table.
[0124] The end position is: t, where t is the value of the number of service data that meet the first query condition in the i-th partition table, t = a(i). The target data displayed by the page number of the page to be jumped includes the service data before the t-th service data among the service data that meet the first query condition in the i-th partition table.
[0125] The start position and end position of the target data displayed by the page number of the page to be jumped among the service data that meet the first query condition in the (i + 1)-th partition table are:
[0126] The starting position is: start-(s(i)–used(1)), where used(1) is the number of business data in the first i partition tables that meet the first query condition and are displayed as the target data in the page number of the page to be redirected. That is, the target data displayed in the page number of the page to be redirected includes the business data after the first-(s(i)–used(1)) business data in the (i+1)th partition table that meet the first query condition.
[0127] The ending position is t, where t is the number of business data in the (i+1)th partition table that meets the first query condition, t = a(i+1). The target data displayed in the page number of the page to be redirected includes the business data before the tth business data in the (i+1)th partition table that meets the first query condition.
[0128] Similarly, the start and end positions of the target data displayed on the page to be redirected to within the business data that satisfies the first query condition in the (i+r)th partition table are:
[0129] The starting position is: start-(s(i+r-1)–used(r)), where used(r) is the number of business data in the first i+r-1 partition tables that meet the first query condition and are displayed as the target data in the page number of the page to be redirected. That is, the target data displayed in the page number of the page to be redirected includes the business data after the start-(s(i+r-1)–used(r)) business data in the i+r partition table that meet the first query condition.
[0130] The end position is: end-s(i+r-1), which means that the target data displayed in the page number of the page to be redirected includes the business data before the end-s(i+r-1)th business data in the business data that meets the first query condition in the i+rth partition table.
[0131] At this point, the number of business data in the first i-1 partition tables that satisfy the first query condition, and the number of target data displayed in the page number to be redirected, satisfies the following formula:
[0132] used(0) = 0;
[0133] The number of business data in the first i partition tables that satisfy the first query condition, and the number of target data displayed in the page number to be redirected to, satisfy the following formula:
[0134] used(1)=finish(1)-begin(1)+1;
[0135] Here, begin(1) and finish(1) are the start and end positions of the target data in the business data of the i-th partition table that meets the first query condition, respectively, for the page number to be displayed on the page to be redirected.
[0136] The number of business data in the first i+r-1 partition tables that satisfy the first query condition, and which are displayed as target data in the page number to be redirected, satisfies the following formula:
[0137] At this point, used(r) = finish(r) – begin(r) + 1 + used(r-1);
[0138] Where begin(r-1) and finish(r-1) are the start and end positions of the target data displayed in the page number of the page to be redirected, respectively, in the business data of the i+r-1th partition table that meets the first query condition. used(r-1) is the number of business data in the first i+r-2 partition tables that meet the first query condition, which are displayed as the target data in the page number of the page to be redirected.
[0139] For example, see Figure 2CAs shown, the number of business data displayed per page can be set to 7, and the page number of the page to be jumped to can be set to 4. Then, the start index position of the index range is represented by start, start = 22, and the end index position of the index range is represented by end, end = 28. When i = 3, s(2) = 19, a(3) = 4, a(4) = 3, a(5) = 4, then s(3) = 23, s(4) = 26, s(5) = 30. At this time, s(2) < start, s(3) > start, s(3) < end, s(4) < end and s(5) > end. At this time, the target data displayed in the page number of the page to be jumped to includes the business data in the 3rd, 4th, and 5th partition tables that meet the first query condition. The number of business data in the 2nd partition table that meets the first query condition and is used as the target data displayed in the page number of the page to be jumped to is: used(0) = 0, and the start position of the target data displayed in the page number of the page to be jumped to among the business data in the 3rd partition that meets the first query condition is: start - (s(i - 1) – used(0)) = 22 - (19 - 0) = 3, and the end position is: t = a(i) = a(3) = 4, that is, the target data displayed in the page number of the page to be jumped to includes the 3rd and 4th business data among the business data in the 3rd partition table that meets the first query condition. Then calculate the position of the target data displayed in the page number of the page to be jumped to among the business data in the 4th partition table that meets the first query condition. The number of business data in the first 3 partition tables that meets the first query condition and is used as the target data displayed in the page number of the page to be jumped to is: used(1) = finish(1) - begin(1) + 1 = 4 - 3 + 1 = 2, and the start position of the target data displayed in the page number of the page to be jumped to among the business data in the 4th partition table that meets the first query condition is: start - (s(i) – used(1)) = 22 - (23 - 2) = 1, and the end position is: t = a(i + 1) = a(4) = 3, that is, the target data displayed in the page number of the page to be jumped to also includes the 1st to 3rd business data among the business data in the 4th partition table that meets the first query condition. Then calculate the position of the target data displayed in the page number of the page to be jumped to among the business data in the 5th partition table that meets the first query condition.The number of target data displayed in the page number of the page to be redirected is: used(2) = finish(2) - begin(2) + 1 + used(1) = 3 - 1 + 1 + 2 = 5. Then, the starting position of the target data displayed in the page number of the page to be redirected is: start - (s(i+1) – used(2)) = 22 - (26 - 5) = 1, and the ending position is: end - s(i+1) = 28 - 26 = 2. That is, the target data displayed in the page number of the page to be redirected includes the first to second business data in the business data of the fifth partition table that meet the first query condition.
[0140] The application embodiment provides a data query device 300. Figure 3 An exemplary schematic diagram of a data query apparatus is shown, the apparatus 300 including a receiving module 301, a processing module 302, and a sending module 303.
[0141] The receiving module 301 is used to receive a first query request sent by the client, wherein the first query request carries a first query condition;
[0142] Processing module 302 is used to obtain first partition information matching the first query condition from the cache according to the first query condition;
[0143] The cache includes first partition information corresponding to the first query condition. The first partition information includes the identifiers of N partition tables that were accessed and hit from the business data table with the first query condition before the first query request was received. The business data table is divided into multiple partition tables, and N is a positive integer.
[0144] The processing module 302 is further configured to query business data that satisfies the first query condition from the partition tables indicated by the identifiers of the N partition tables respectively;
[0145] The sending module 303 is used to send the service data to the client.
[0146] In some embodiments, the receiving module 301 is further configured to receive a second query request, the second query request carrying a second query condition;
[0147] The processing module 302 is further configured to, when the partition information matching the first query condition is not obtained from the cache according to the second query condition, query the business data table including the partition table that satisfies the second query condition, and save the second query condition and the second partition information in the cache, wherein the second partition information includes the partition identifier of the partition table where the business data that satisfies the second query condition is located.
[0148] In some embodiments, the first partition information also includes the number of data in the N partition tables that satisfy the first query condition; the first query request also carries pagination information for which a page needs to be navigated to.
[0149] When the processing module 302 queries business data that satisfies the first query condition from the partition tables indicated by the identifiers of the at least one partition table, it is specifically used for:
[0150] Based on the number of data in the N partition tables that satisfy the first query condition, at least one partition table in the N partition tables is determined to include the target data to be paged as indicated by the pagination information;
[0151] The target data is obtained from the at least one partition table.
[0152] In some embodiments, the pagination information includes the number of business data items displayed per page and the page number to be navigated to;
[0153] When the processing module 302 determines at least one partition table in the N partition tables that includes the target data to be paged as indicated by the pagination information, based on the number of data in the N partition tables that satisfy the first query condition, it is specifically used for:
[0154] The index range of the target data to be displayed in the page to be jumped to is determined based on the page number of the page to be jumped to and the number of business data items displayed on each page;
[0155] Based on the data quantity corresponding to the N partition tables and the index range, determine at least one partition table that includes the target data to be displayed in the current page to be navigated to.
[0156] In some embodiments, the starting index position of the index range satisfies the condition shown in the following formula;
[0157] start=(Pagenum-1)×Pagesize+1;
[0158] The ending index position of the index range satisfies the condition shown in the following formula:
[0159] end = Pagenum × Pagesize;
[0160] Where Pagesize represents the number of business data entries displayed per page, Pagenum represents the page number to be navigated to, start represents the starting index position of the index range, and end represents the ending index position of the index range.
[0161] In some embodiments, the at least one partition table includes a first partition table, and the processing module 302 is specifically used for:
[0162] Based on the query order of the N partition tables, determine the number of data in the N1 partition tables that precede the query order of the first partition table and the starting index position that is less than the index range, and the sum of the number of data in the N1 partition tables and the number of data in the first partition table is greater than the starting index position.
[0163] In some embodiments, when the processing module 302 obtains the target data from the at least one partition table, it is specifically used for:
[0164] Retrieve business data that satisfies the first query condition from at least one partition table;
[0165] When the sum of the number of data items in the N1 partition tables and the number of data items in the first partition table is greater than or equal to the end index position, the start and end positions of the target data within the business data satisfying the first query condition in the first partition table are determined based on the start and end index positions; or,
[0166] When the sum of the number of data in the N1 partition tables and the number of data in the first partition table is less than the end index position, the starting position of the target data in the business data that satisfies the first query condition in the first partition table is determined according to the start index position; the at least one partition table also includes a second partition table, and the query order of the second partition table in the N partition tables is after the first partition table; the ending position of the target data in the business data that satisfies the first query condition in the second partition table is determined according to the end index position.
[0167] See Figure 4 The diagram shown is a structural schematic of a database server 400 provided in an embodiment of this application. The database server 400 includes a memory 410 and a processor 420, and the database server 400 can be deployed in the form of a server cluster.
[0168] The memory 410 is used to store program instructions.
[0169] The processor 420 is used to call the program instructions stored in the memory 410 and execute the above data query method according to the obtained program.
[0170] In some embodiments, the database server 400 further includes a communication interface. Figure 4It is not shown in the figure. The communication interface is used to receive query requests sent by the client and to send the queried business data to the client. In this embodiment of the application, the specific connection medium between the memory 410, the processor 420 and the communication interface is not limited, such as a bus. The bus can be divided into address bus, data bus, control bus, etc.
[0171] In the embodiments of this application, the processor may be a general-purpose processor, a digital signal processor, an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components, capable of implementing or executing the methods, steps, and logic block diagrams disclosed in the embodiments of this application. The general-purpose processor may be a microprocessor or any conventional processor. The steps of the methods disclosed in the embodiments of this application can be directly manifested as being executed by a hardware processor, or executed by a combination of hardware and software modules within the processor.
[0172] In the embodiments of this application, the memory can be non-volatile memory, such as a hard disk drive (HDD) or a solid-state drive (SSD), or it can be volatile memory, such as random-access memory (RAM). The memory can also be any other medium capable of carrying or storing desired program code in the form of instructions or data structures, and accessible by a computer, but is not limited thereto. The memory in the embodiments of this application can also be a circuit or any other device capable of implementing storage functions, used to store program instructions and / or data.
[0173] In some embodiments, processor 420 may include a cache. Figure 4 Not shown in the diagram. Of course, the cache can also be deployed outside the processor 420, connected to the processor 420.
[0174] Database server 400 can be deployed within a database system. The database system may include storage unit 500, which comprises multiple storage units for storing business data tables. See [link to relevant documentation]. Figure 5 , Figure 5 Taking M memory units as an example, where M is a positive integer.
[0175] Based on the same technical concept, embodiments of this application also provide a computer-readable non-volatile storage medium, including computer-readable instructions, which, when read and executed by a computer, cause the computer to perform the above-mentioned data query method.
[0176] The data query method provided in this application avoids the problem of insufficient caching when querying large data tables by storing query conditions and the partition information of data that meets the query conditions in the database cache, rather than storing the data itself. During the query, the partition table and its position in the partition table where the target data of the page to be redirected are determined by the pagination information, thereby directly retrieving the target data from the partition table, avoiding a full table scan and improving query speed.
[0177] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.
[0178] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to this application. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.
[0179] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.
[0180] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 Figure 1 The steps of the function specified in one or more boxes.
[0181] Obviously, those skilled in the art can make various modifications and variations to this application without departing from the spirit and scope of this application. Therefore, if such modifications and variations fall within the scope of the claims of this application and their equivalents, this application also intends to include such modifications and variations.
Claims
1. A method for data querying, characterized in that, include: The client sends a first query request, which carries a first query condition and pagination information to be redirected to. The pagination information includes the number of business data items displayed on each page and the page number to be redirected to. The first partition information matching the first query condition is obtained from the cache. The cache includes first partition information corresponding to the first query condition. The first partition information includes the first query condition, the identifiers of N partition tables associated with the first query condition, the number of times the first query condition is hit in the current cleanup cycle, and the number of data in the N partition tables that satisfy the first query condition. The N partition tables are partition tables that were accessed and hit from the business data table with the first query condition before the first query request was received. The business data table is divided into multiple partition tables, and N is a positive integer. The process involves: querying business data that satisfies the first query condition from the partition tables indicated by the identifiers of the N partition tables; determining the index range of the target data to be displayed in the business data that satisfies the first query condition based on the page number of the page to be navigated to and the number of business data items displayed per page; determining at least one partition table that includes the target data to be displayed in the current page to be navigated to based on the data quantity corresponding to each of the N partition tables and the index range; and obtaining the target data from the at least one partition table, including: determining the start and end positions of the target data in the at least one partition table based on the index range and the data quantity corresponding to each of the N partition tables; and obtaining the target data from the business data that satisfies the first query condition included in the at least one partition table based on the start and end positions. Send the service data to the client and update the hit count in the first partition information; If the number of hits in the first partition information is less than the threshold during the current cleanup cycle, then the first partition information will be deleted from the cache.
2. The method as described in claim 1, characterized in that, The method further includes: Receive a second query request, which carries a second query condition; If no matching partition information matching the first query condition is obtained from the cache according to the second query condition, the business data that meets the second query condition is queried from the business data table. The second query condition and the second partition information are associated and stored in the cache. The second partition information includes the partition identifier of the partition table where the business data that meets the second query condition is located.
3. The method as described in claim 1, characterized in that, The starting index position of the index range satisfies the condition shown in the following formula: start=(Pagenum-1)×Pagesize+1; The ending index position of the index range satisfies the condition shown in the following formula: end = Pagenum × Pagesize; Where Pagesize represents the number of business data entries displayed per page, Pagenum represents the page number to be navigated to, start represents the starting index position of the index range, and end represents the ending index position of the index range.
4. The method as described in claim 1, characterized in that, The at least one partition table includes a first partition table. Based on the data quantity corresponding to each of the N partition tables and the index range, at least one partition table is determined that includes the target data to be displayed in the current page to be navigated to, including: Based on the query order of the N partition tables, determine the number of data in the N1 partition tables that precede the query order of the first partition table and the starting index position that is less than the index range, and the sum of the number of data in the N1 partition tables and the number of data in the first partition table is greater than the starting index position.
5. The method as described in claim 4, characterized in that, Obtaining the target data from the business data that satisfies the first query condition included in the at least one partition table includes: Retrieve business data that satisfies the first query condition from at least one partition table; When the sum of the number of data items in the N1 partition tables and the number of data items in the first partition table is greater than or equal to the end index position of the index range, the start and end positions of the target data in the business data that satisfy the first query condition included in the first partition table are determined according to the start and end index positions; or, When the sum of the number of data in the N1 partition tables and the number of data in the first partition table is less than the end index position, the starting position of the target data in the business data that satisfies the first query condition in the first partition table is determined according to the start index position; the at least one partition table also includes a second partition table, and the query order of the second partition table in the N partition tables is after the first partition table; the ending position of the target data in the business data that satisfies the first query condition in the second partition table is determined according to the end index position.
6. A data query apparatus, characterized in that, include: The receiving module is used to receive a first query request sent by the client. The first query request carries a first query condition and pagination information to be redirected to. The pagination information includes the number of business data items displayed on each page and the page number to be redirected to. The processing module is used to retrieve the first partition information that matches the first query condition from the cache according to the first query condition; The cache includes first partition information corresponding to the first query condition. The first partition information includes the first query condition, the identifiers of N partition tables associated with the first query condition, the number of times the first query condition is hit in the current cleanup cycle, and the number of data in the N partition tables that satisfy the first query condition. The N partition tables are partition tables that were accessed and hit from the business data table with the first query condition before the first query request was received. The business data table is divided into multiple partition tables, and N is a positive integer. The processing module is further configured to query business data that satisfies the first query condition from the partition tables indicated by the identifiers of the N partition tables: determining the index range of the target data to be displayed in the business data that satisfies the first query condition based on the page number of the page to be paged and the number of business data items displayed on each page; determining at least one partition table that includes the target data to be displayed in the current page to be paged based on the number of data items corresponding to the N partition tables and the index range; and obtaining the target data from the at least one partition table, specifically including: determining the start and end positions of the target data in the at least one partition table based on the index range and the number of data items corresponding to the N partition tables, and obtaining the target data from the business data that satisfies the first query condition included in the at least one partition table based on the start and end positions; The sending module is used to send the service data to the client; The processing module is further configured to: update the hit count in the first partition information; if the hit count in the first partition information is less than the count threshold during the current cleanup cycle, then delete the first partition information from the cache.
7. The apparatus as claimed in claim 6, characterized in that, The receiving module is further configured to receive a second query request, the second query request carrying a second query condition; The processing module is further configured to, when the partition information matching the first query condition is not obtained from the cache according to the second query condition, query the business data table including the partition table that satisfies the second query condition, and save the second query condition and the second partition information in the cache, wherein the second partition information includes the partition identifier of the partition table where the business data that satisfies the second query condition is located.
8. The apparatus as claimed in claim 6, characterized in that, The starting index position of the index range satisfies the condition shown in the following formula; start=(Pagenum-1)×Pagesize+1; The ending index position of the index range satisfies the condition shown in the following formula: end = Pagenum × Pagesize; Where Pagesize represents the number of business data entries displayed per page, Pagenum represents the page number to be navigated to, start represents the starting index position of the index range, and end represents the ending index position of the index range.
9. The apparatus as claimed in claim 6, characterized in that, The at least one partition table includes a first partition table, and the processing module is specifically used for: Based on the query order of the N partition tables, determine the number of data in the N1 partition tables that precede the query order of the first partition table and the starting index position that is less than the index range, and the sum of the number of data in the N1 partition tables and the number of data in the first partition table is greater than the starting index position.
10. The apparatus as claimed in claim 9, characterized in that, The processing module, when retrieving the target data from the business data satisfying the first query condition included in the at least one partition table, is specifically configured to: Retrieve business data that satisfies the first query condition from at least one partition table; When the sum of the number of data items in the N1 partition tables and the number of data items in the first partition table is greater than or equal to the end index position of the index range, the start and end positions of the target data in the business data that satisfy the first query condition included in the first partition table are determined according to the start and end index positions; or, When the sum of the number of data in the N1 partition tables and the number of data in the first partition table is less than the end index position, the starting position of the target data in the business data that satisfies the first query condition in the first partition table is determined according to the start index position; the at least one partition table also includes a second partition table, and the query order of the second partition table in the N partition tables is after the first partition table; the ending position of the target data in the business data that satisfies the first query condition in the second partition table is determined according to the end index position.
11. A data query device, characterized in that, include: Memory and processor; Memory, used to store program instructions; A processor is configured to invoke program instructions stored in the memory and execute the method described in any one of claims 1 to 5 according to the obtained program.
12. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions that, when executed on a computer, cause the computer to perform the method of any one of claims 1 to 5.
Citation Information
Patent Citations
Data processing method, device and system
CN103294702A
Back-end data processing method and device in front-end and back-end decoupling scene
CN110837517A
Storage method and device for network community, electronic equipment and readable medium
CN112422668A