Data query control method and related equipment

Abstract

The invention discloses a data query control method and related equipment, and relates to the field of databases, and the method comprises the following steps: obtaining to-be-queried page number information and data size information of each page of a user; based on the to-be-queried page number information and the data size information of each page, determining sorting sequence number information of the to-be-queried data; to-be-queried data is obtained from a cache unit and / or an original storage unit of a target database according to the sorting sequence number information, all data information of the target database is stored in the cache unit and the original storage unit, and the cache unit and the original storage unit do not store data repeatedly; the target database comprises a plurality of storage table units, each storage table unit comprises at least one cache unit and at least one original storage unit, and data stored in the cache units are generated based on data history query information.

Description

A data query control method and related equipment Technical Field

[0001] This specification relates to the field of databases, and more specifically, this disclosure relates to a data query control method and related equipment. Background Technology

[0002] As the system scales up, the amount of data grows exponentially, increasing the storage and query pressure on the database. In this situation, data storage needs to be rationally planned to ensure that system performance is not affected. Data sharding and partitioning techniques have thus emerged, with the core purpose of improving system performance, enhancing scalability, increasing high availability, and decoupling business logic by rationally dividing the database and tables.

[0003] While database sharding and table partitioning can bring significant benefits, they also increase the complexity of the system, especially in terms of data pagination queries, where related query methods suffer from problems such as excessively long response times and excessive resource consumption. Summary of the Invention

[0004] The summary section introduces a series of simplified concepts, which will be further explained in detail in the detailed description section. This summary section is not intended to limit the key features and essential technical features of the claimed technical solutions, nor is it intended to determine the scope of protection of the claimed technical solutions.

[0005] Firstly, this disclosure proposes a data query control method, which includes:

[0006] Obtain the user's page number to be queried and the data size information per page, wherein the data size information per page is the number of data items included in each query page;

[0007] Based on the page number information to be queried and the data size information of each page, the sorting sequence information of the data to be queried is determined;

[0008] The data to be queried is obtained from the cache unit and / or original storage unit of the target database according to the sorting sequence information mentioned above. The cache unit and the original storage unit jointly store all the data information of the target database. There is no duplicate data stored between the cache unit and the original storage unit. The target database includes multiple storage table units. Each storage table unit includes at least one cache unit and at least one original storage unit. The data stored in the cache unit is generated based on the data historical query information.

[0009] In one feasible implementation, the process of retrieving the data to be queried from the cache unit and / or original storage unit of the target database based on the sorting sequence information includes:

[0010] In cases where the aforementioned cached data unit does not completely include all the data corresponding to the aforementioned sorting sequence information, the aforementioned original storage unit is filtered based on the missing data in the cache unit and the aforementioned cache unit is expanded to obtain the filtered original storage unit and the expanded cache unit.

[0011] Retrieve the query data from the expanded cache unit; and / or,

[0012] If the aforementioned cached data unit contains all the data corresponding to the aforementioned sorting sequence information, then the data to be queried is retrieved from the aforementioned cached unit.

[0013] In one feasible implementation, where each storage table unit includes a cache unit and the aforementioned original storage unit...

[0014] In the case that the cached data unit includes all the data corresponding to the sorting sequence information, retrieving the data to be queried from the cached unit includes:

[0015] Obtain the larger of the first cached data quantity in the above cache unit and the sequence number corresponding to the above sorting sequence number information;

[0016] The redundancy of the first cache data is determined based on the larger value of the above sequence number and the above quantity of the first cache data.

[0017] For each cache unit corresponding to each storage table unit, a size comparison operation is performed on the first element of the queue, and a data movement operation is performed on the cached data with the larger identity sequence number to move it to the first column unit corresponding to the original storage unit in the same storage table. The aforementioned identity sequence number is determined based on the time information of data storage in the target database.

[0018] Based on the aforementioned first cached data redundancy quantity, the aforementioned size comparison operation and the aforementioned data movement operation are repeatedly executed to obtain the first updated cache unit;

[0019] In the first update cache unit mentioned above, the corresponding number of the above-mentioned data to be queried are obtained based on the data size information of each page.

[0020] In one feasible implementation, where each storage table unit includes at least two cache units and one of the aforementioned original storage units...

[0021] In the case that the cached data unit includes all the data corresponding to the sorting sequence information, retrieving the data to be queried from the cached unit includes:

[0022] Based on the above sorting sequence information, the corresponding target cache unit is determined in at least two cache units in each storage table unit, wherein different cache units in each storage table unit are determined based on different sorting sequence ranges;

[0023] Obtain the larger of the number of second cached data in the target cache unit and the sequence number corresponding to the sorting sequence number information;

[0024] The amount of redundancy in the second cache data is determined based on the larger value of the above sequence number and the amount of the second cache data.

[0025] For each target cache unit corresponding to each storage table unit, a size comparison operation is performed on the first element of the queue, and a data movement operation is performed on the cached data with the larger identity sequence number to move it to the first column unit corresponding to the original storage unit in the same storage table. The aforementioned identity sequence number is determined based on the time information of data storage in the target database.

[0026] Based on the aforementioned second cache data redundancy quantity, the aforementioned size comparison operation and the aforementioned data movement operation are repeatedly performed to obtain the second updated cache unit;

[0027] The second update cache unit retrieves the corresponding number of the above-mentioned data to be queried based on the data size information of each page.

[0028] In one feasible implementation, where each storage table unit includes a cache unit and the aforementioned original storage unit...

[0029] In the case where the cached data unit does not completely include all the data corresponding to the sorting sequence information, the original storage unit is filtered based on the missing data in the cache unit, and the cache unit is expanded to obtain the filtered original storage unit and the expanded cache unit, including:

[0030] Obtain the larger of the number of third cached data in the above cache unit and the sequence number corresponding to the above sorting sequence number information;

[0031] The number of missing data in the first cache is determined based on the larger of the above sequence numbers and the number of third cache data.

[0032] For each original storage unit corresponding to each storage table unit, a size comparison operation is performed on the first element of the queue, and a data movement operation is performed on the original storage data with the smaller identity sequence number to move it to the first column unit corresponding to the cache unit of the same storage table. The aforementioned identity sequence number is determined based on the time information of data storage in the target database.

[0033] Based on the number of missing first cached data, the above size comparison operation and the above data movement operation are repeatedly performed to obtain the third updated cache unit;

[0034] In the third update cache unit mentioned above, the corresponding number of the above-mentioned data to be queried are obtained based on the data size information of each page.

[0035] In one feasible implementation, where each storage table unit includes at least two cache units and one of the aforementioned original storage units...

[0036] In the case where the cached data unit does not completely include all the data corresponding to the sorting sequence information, the original storage unit is filtered based on the missing data in the cache unit, and the cache unit is expanded to obtain the filtered original storage unit and the expanded cache unit, including:

[0037] Based on the above sorting sequence information, the corresponding target cache unit is determined in at least two cache units in each storage table unit, wherein different cache units in each storage table unit are determined based on different sorting sequence ranges;

[0038] Obtain the larger of the fourth cache data quantity in the target cache unit and the sequence number corresponding to the sorting sequence information;

[0039] The number of missing data in the second cache is determined based on the larger of the above sequence numbers and the amount of data in the fourth cache.

[0040] For each storage table unit, a size comparison operation is performed on the first element of the queue of the original storage unit, and a data movement operation is performed on the original storage data with the smaller identity number to move it to the first column of the target cache unit corresponding to the same storage table. The aforementioned identity number is determined based on the time information of the data being stored in the target database.

[0041] Based on the number of missing second cached data, the above size comparison operation and the above data movement operation are repeatedly performed to obtain the fourth updated cache unit;

[0042] The fourth update cache unit above retrieves the corresponding number of the above-mentioned data to be queried based on the data size information of each page.

[0043] In one feasible implementation, it further includes:

[0044] If the current query is the first query, the data to be queried is obtained from the original storage unit based on the secondary query method.

[0045] In one feasible implementation, it further includes:

[0046] If the difference between the sorting sequence number of the data to be queried and the sorting sequence number of the data stored in the current cache unit is greater than a preset difference, the data to be queried is obtained from the original storage unit based on the secondary query method.

[0047] In one feasible implementation, it further includes:

[0048] If any one or more of the following conditions are met: query conditions change, user status information expires, session expires, and expiration date expires, the cached data in the above cache unit will be cleaned up.

[0049] In one feasible implementation, it further includes:

[0050] When a data addition operation is performed on the aforementioned target database, the newly added data will be stored in the aforementioned original storage unit; and / or,

[0051] In the event of data deletion operations in the aforementioned target database, the corresponding data in the aforementioned cache unit and / or the aforementioned original storage unit will be deleted.

[0052] Secondly, embodiments of this disclosure provide a data query control device, comprising:

[0053] The first acquisition unit is used to acquire the user's page number information to be queried and the data size information per page, wherein the data size information per page is the number of data included in each query page;

[0054] The determining unit is used to determine the sorting sequence number information of the data to be queried based on the above-mentioned page number information and the above-mentioned data size information per page;

[0055] The second acquisition unit is used to acquire the data to be queried from the cache unit and / or the original storage unit of the target database according to the sorting sequence information. The cache unit and the original storage unit jointly store all the data information of the target database. There is no duplicate data stored between the cache unit and the original storage unit. The target database includes multiple storage table units. Each storage table unit includes at least one cache unit and at least one original storage unit. The data stored in the cache unit is generated based on the data historical query information.

[0056] Thirdly, an electronic device includes: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program stored in the memory to implement the steps of the data query control method as described in any of the first aspects above.

[0057] Fourthly, this disclosure also proposes a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the data query control method of any one of the first aspects.

[0058] In summary, frequent data access in paginated queries of related technologies can put significant pressure on the database, especially during secondary queries. This disclosed solution employs caching technology to cache historical query information, reducing direct database access through caching units. This transforms secondary queries into single queries, significantly reducing database pressure and improving query efficiency. The caching units and original storage units in this disclosed solution do not store duplicate data, ensuring efficient and consistent data storage. This separation design also allows for better data query control, improving query flexibility and response speed. When a database deletion operation occurs, the data ID remains unchanged, while the sorting sequence number is automatically updated based on existing data, ensuring the continuity of the sequence number during paginated queries. This dynamic adjustment mechanism ensures the accuracy and stability of data queries, avoiding sequence number confusion caused by data deletion. Through database sharding and table partitioning techniques combined with the caching mechanism, the amount of data in each database query is effectively reduced, improving the system's concurrent processing capabilities. Furthermore, the asynchronous establishment of the cache avoids a large number of I / O operations during queries, allowing the system to maintain high query performance even under high concurrency. The introduction of caching reduces the frequency of direct database access in multi-query scenarios, especially when users frequently access the same type of data. Caching significantly reduces database load, thereby improving system stability and response speed. Through the design of cache units combined with a sharded database structure, this solution enables global pagination. In multiple databases or table units, the caching mechanism unifies the display of dispersed data, enhancing the user experience. This disclosed solution, by introducing caching and dynamic sorting mechanisms, not only improves system performance but also significantly reduces database query pressure while ensuring data continuity and query accuracy.

[0059] The data query control method disclosed herein, along with other advantages, objectives, and features of this disclosure, will be partly apparent from the following description and partly understood by those skilled in the art through study and practice of this disclosure. Attached Figure Description

[0060] Various other advantages and benefits will become apparent to those skilled in the art upon reading the following detailed description of preferred embodiments. The accompanying drawings are for illustrative purposes only and are not intended to limit this specification. Furthermore, the same reference numerals denote the same parts throughout the drawings. In the drawings:

[0061] Figure 1 is a flowchart illustrating a data query control method provided in an embodiment of this disclosure;

[0062] Figure 2 is a schematic diagram illustrating the principle of data querying in a cache unit and a raw storage unit according to an embodiment of this disclosure.

[0063] Figure 3 is a schematic diagram illustrating the principle of data querying between another cache unit and the original storage unit provided in an embodiment of this disclosure;

[0064] Figure 4 is a schematic diagram illustrating the principle of data query in an embodiment of this disclosure, where the cached data unit includes all data corresponding to the above-mentioned sorting sequence information.

[0065] Figure 5 is a schematic diagram illustrating the principle of data query in another case where the cached data unit includes all data corresponding to the above sorting sequence information, according to an embodiment of this disclosure.

[0066] Figure 6 is a schematic diagram illustrating the principle of data query in another embodiment of this disclosure, where the cached data unit includes all data corresponding to the above-mentioned sorting sequence information.

[0067] Figure 7 is a schematic diagram illustrating the principle of data query in an embodiment of this disclosure when the cached data unit does not completely include all the data corresponding to the above sorting sequence information.

[0068] Figure 8 is a schematic diagram illustrating the principle of data query in another case where the cached data unit does not completely include all the data corresponding to the above sorting number information, according to an embodiment of this disclosure.

[0069] Figure 9 is a schematic diagram illustrating the principle of data query in another case where the cached data unit does not completely include all the data corresponding to the above sorting number information, according to an embodiment of this disclosure.

[0070] Figure 10 is a schematic diagram illustrating the principle of a secondary query method provided in an embodiment of this disclosure;

[0071] Figure 11 is a schematic diagram illustrating the principle of another secondary query method provided in an embodiment of this disclosure;

[0072] Figure 12 is a schematic diagram illustrating the principle of another secondary query method provided in an embodiment of this disclosure;

[0073] Figure 13 is a schematic diagram illustrating the principle of another secondary query method provided in an embodiment of this disclosure;

[0074] Figure 14 is a schematic diagram illustrating the principle of data addition according to an embodiment of this disclosure;

[0075] Figure 15 is a schematic diagram illustrating another principle of data addition provided in an embodiment of this disclosure;

[0076] Figure 16 is a schematic diagram of a data deletion and addition principle provided in an embodiment of this disclosure;

[0077] Figure 17 is a schematic diagram illustrating another data deletion principle provided in an embodiment of this disclosure;

[0078] Figure 18 is a structural schematic diagram of a data query control electronic device provided in an embodiment of this disclosure;

[0079] Figure 19 is a structural schematic diagram of a display device provided in an embodiment of this disclosure. Detailed Implementation

[0080] The terms "first," "second," "third," "fourth," etc. (if present) in this disclosure, claims, and accompanying drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments described herein can be implemented in a sequence other than that illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus. The technical solutions of the embodiments of this disclosure will now be clearly and completely described in conjunction with the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this disclosure, and not all of them.

[0081] Please refer to Figure 1, which is a flowchart illustrating a data query control method provided in an embodiment of this disclosure. Specifically, it may include:

[0082] S110. Obtain the user's page number information and the data size information per page, wherein the data size information per page is the number of data items included in each query page;

[0083] For example, the page number information to be queried and the data size information per page are query conditions that are entered by the user or automatically generated. For example, the query page number information is the third page, and the data size information per page is 5 records, that is, the number of data records for each query is 5 records.

[0084] S120. Based on the above page number information to be queried and the above data size information for each page, determine the sorting sequence information of the data to be queried;

[0085] For example, the sorting sequence information of the data to be queried can be determined based on the page number information and the data size information of each page. Taking the page number information as the third page and the data size information of each page as 5, the sorting sequence information corresponding to the first page is {1, 2, 3, 4, 5}, the sorting sequence information corresponding to the second page is {6, 7, 8, 9, 10}, then the sorting sequence information of the data to be queried on the third page is {11, 12, 13, 14, 15}.

[0086] It should be noted that the sorting sequence information mentioned in this disclosure is not the data ID information mentioned later. The data ID information is generated based on the order in which data is stored. For example, the data stored as [ID1, ID2, ID3, ID4, ID5] initially corresponds to sorting sequence numbers {1, 2, 3, 4, 5}. That is, the initial ID information and sorting sequence numbers are one-to-one. However, there may be data deletion operations in the database. For example, if the data ID3 is deleted, the data in the database after deleting ID3 will be [ID1, ID2, ID4, ID5], and the corresponding sorting sequence numbers will be {1, 2, 3, 4}. That is, the data ID information in the database does not change with the deletion of data. The data ID information always corresponds to the stored data, while the sorting sequence number changes to ensure that the sequence number is always continuous.

[0087] S130. Obtain the data to be queried from the cache unit and / or original storage unit of the target database according to the sorting sequence information. The cache unit and the original storage unit jointly store all the data information of the target database. There is no duplicate data stored between the cache unit and the original storage unit. The target database includes multiple storage table units. Each storage table unit includes at least one cache unit and at least one original storage unit. The data stored in the cache unit is generated based on the data history query information.

[0088] For example, in the secondary query process of related technologies, each page of data requires a secondary query. When data viewing is too frequent, it will put a lot of pressure on the database. This disclosure uses caching to reduce the database access frequency while meeting business requirements. As shown in Figures 2 and 3, DB1, DB2, and DB3 are three storage table units, which together constitute the target database. Each storage table unit of the target database includes a cache unit (Behind) and a raw storage unit (Front). In this disclosure, the first query is performed according to the corresponding conditions, and a data cache is established asynchronously. The target database without a cache is shown in Figure 2. At the moment the initial cache is established, the first page of data will be displayed according to the default query rules, so the cache will change to the situation shown in Figure 2. The two queues shown in Figures 2 and 3 are used to represent the data IDs currently owned in the DB. The entire cache object must maintain the uniqueness of the key, such as user_name- <condition>-DB1-Front、user_name- <condition>-DB1-Behind represent the Front and Behind queue results obtained by the user from querying DB1 based on specified conditions, respectively.

[0089] All cache units and all original storage units constitute the complete target database, and there is no duplicate data in each cache unit and original storage unit. This storage method leverages queue attributes and caching to successfully convert a secondary query into a single query, thus enabling global pagination of sharded and partitioned data.

[0090] In summary, frequent data access in paginated queries of related technologies can put significant pressure on the database, especially during secondary queries. This disclosed solution employs caching technology to cache historical query information, reducing direct database access through caching units. This transforms secondary queries into single queries, significantly reducing database pressure and improving query efficiency. The caching units and original storage units in this disclosed solution do not store duplicate data, ensuring efficient and consistent data storage. This separation design also allows for better data query control, improving query flexibility and response speed. When a database deletion operation occurs, the data ID remains unchanged, while the sorting sequence number is automatically updated based on existing data, ensuring the continuity of the sequence number during paginated queries. This dynamic adjustment mechanism ensures the accuracy and stability of data queries, avoiding sequence number confusion caused by data deletion. Through database sharding and table partitioning techniques combined with the caching mechanism, the amount of data in each database query is effectively reduced, improving the system's concurrent processing capabilities. Furthermore, the asynchronous establishment of the cache avoids a large number of I / O operations during queries, allowing the system to maintain high query performance even under high concurrency. The introduction of caching reduces the frequency of direct database access in multi-query scenarios, especially when users frequently access the same type of data. Caching significantly reduces database load, thereby improving system stability and response speed. Through the design of cache units combined with a sharded database structure, this solution enables global pagination. In multiple databases or table units, the caching mechanism unifies the display of dispersed data, enhancing the user experience. This disclosed solution, by introducing caching and dynamic sorting mechanisms, not only improves system performance but also significantly reduces database query pressure while ensuring data continuity and query accuracy.

[0091] In some examples, retrieving the data to be queried from the cache unit and / or raw storage unit of the target database based on the aforementioned sorting sequence information includes:

[0092] In cases where the aforementioned cached data unit does not completely include all the data corresponding to the aforementioned sorting sequence information, the aforementioned original storage unit is filtered based on the missing data in the cache unit and the aforementioned cache unit is expanded to obtain the filtered original storage unit and the expanded cache unit.

[0093] Retrieve the data to be queried from the expanded cache unit; and / or

[0094] If the cached data unit contains all the data corresponding to the sorting sequence information, the data to be queried is retrieved from the cached unit.

[0095] For example, when the cache unit does not completely include all the data corresponding to the sorting sequence information, that is, when the cache only contains part of the data corresponding to the sorting sequence information or when all the data of the sorting sequence information is not in the cache unit, the system will perform a filtering operation in the original storage unit and expand the filtered data into the cache unit to complete the expansion operation of the cache unit.

[0096] Specifically, the system identifies missing data items in the cache based on sorting sequence information, then filters these missing data items from the original storage units and adds them to the cache units to form expanded cache units. After the cache units are expanded, user query data will be retrieved directly from the expanded cache units. This reduces future query needs on the original storage units and improves the system's query efficiency.

[0097] In another scenario, the cache unit already contains all the data for the user's query page. In this case, there is no need to access the original storage unit, and the system can directly retrieve the user's query data from the cache.

[0098] Specifically, when a user requests data for page 3, the system checks and finds that the cache unit only contains data for pages 1 and 2, while the data for page 3 is partially or completely absent. In this case, the system will filter out the data for page 3 from the original storage unit and add it to the cache unit for the user to query.

[0099] If the data requested by the user for page 3 already exists entirely in the cache unit, the system does not need to access the original storage unit; it can directly retrieve the required data from the cache and return it to the user.

[0100] The method proposed in this disclosure prioritizes data retrieval from cache units, reducing the frequency of database queries. When the cache is insufficient, it expands the cache by filtering data from the original storage units, ensuring that users can quickly obtain the data they need. If the cache is sufficient, the system requires no further operation; if the cached data is incomplete, the system can automatically filter and expand the cache.

[0101] In some examples, where each storage table unit includes a cache unit and the aforementioned original storage unit,

[0102] In the case that the cached data unit includes all the data corresponding to the sorting sequence information, retrieving the data to be queried from the cached unit includes:

[0103] Obtain the larger of the first cached data quantity in the above cache unit and the sequence number corresponding to the above sorting sequence number information;

[0104] The redundancy of the first cache data is determined based on the larger value of the above sequence number and the above quantity of the first cache data.

[0105] For each cache unit corresponding to each storage table unit, a size comparison operation is performed on the first element of the queue, and a data movement operation is performed on the cached data with the larger identity sequence number to move it to the first column unit corresponding to the original storage unit in the same storage table. The aforementioned identity sequence number is determined based on the time information of data storage in the target database.

[0106] Based on the aforementioned first cached data redundancy quantity, the aforementioned size comparison operation and the aforementioned data movement operation are repeatedly executed to obtain the first updated cache unit;

[0107] In the first update cache unit mentioned above, the corresponding number of the above-mentioned data to be queried are obtained based on the data size information of each page.

[0108] For example, as shown in Figure 4-6, in each storage table unit, the cache unit (Front) stores recently accessed data, and the original storage unit (Behind) stores all data in the database. When the amount of data in the cache unit exceeds a certain threshold, the system needs to move some data from the cache unit to the original storage unit to ensure that the amount of data in the cache unit remains within a reasonable range.

[0109] In the current cache (Front queue), the first cache contains 15 data entries, while the system needs to display data from the second page (entries 5-10). To satisfy the paging rules, the system first needs to reduce the amount of data in the cache unit to the 10th data entry corresponding to the largest sort order number of 10. Therefore, the redundancy in the first cache is 15-10=5 entries. The system achieves this by moving the largest 5th data entry from the cache unit into the original storage unit.

[0110] Specifically, the cache unit sequentially compares the first element in the cache unit (Front queue) of each storage table unit, selects the largest element, and moves it. The selected largest data is then moved from the cache unit to the original storage unit (Behind queue) of the corresponding storage table unit.

[0111] For example: During the first comparison, the data at the head of the cache unit is 25, 23, and 24. The maximum value, 25, is selected and moved from the front queue of DB1 to the behind queue of DB1.

[0112] During the second comparison, the data at the front of the queue are 19, 23, and 24. 24 is selected and moved from the front queue of DB3 to the behind queue of DB3.

[0113] Repeat this process until the top 5 elements are moved into the Behind queue, ensuring that the amount of data remaining in the cache meets the data requirements of the second page.

[0114] Based on the total amount of data in the current cache unit (15 records) and the target page size (10 records), the number of records to be removed is calculated, i.e., the redundancy is 5 records. The system continues to compare the data at the head of the Front queue and moves the largest element to the Behind queue in turn until the amount of data in the cache unit is reduced to the required 10 records.

[0115] After removing redundant data, the remaining data in the cache unit conforms to the pagination rules (i.e., it contains data items 5-10). At this point, the system can directly retrieve the final paginated data from the Behind queue. Based on the example above, the five elements with the largest IDs are ultimately extracted from the Behind queue as the dataset for the second page; these data IDs are {11, 14, 16, 17, 18}.

[0116] This embodiment achieves an optimized query scheme when cached data overflows by dynamically adjusting the data balance between the cache unit and the original storage unit. Through size comparison and data movement operations, the system can retrieve the correct paginated data from the cache and the original storage unit without increasing the database load, improving query efficiency and reducing the frequency of direct database access. This method is particularly suitable for scenarios requiring frequent pagination queries.

[0117] In some examples, where each storage table unit includes at least two cache units and one of the aforementioned original storage units,

[0118] In the case that the cached data unit includes all the data corresponding to the sorting sequence information, retrieving the data to be queried from the cached unit includes:

[0119] Based on the above sorting sequence information, the corresponding target cache unit is determined in at least two cache units in each storage table unit, wherein different cache units in each storage table unit are determined based on different sorting sequence ranges;

[0120] Obtain the larger of the number of second cached data in the target cache unit and the sequence number corresponding to the sorting sequence number information;

[0121] The amount of redundancy in the second cache data is determined based on the larger value of the above sequence number and the amount of the second cache data.

[0122] For each target cache unit corresponding to each storage table unit, a size comparison operation is performed on the first element of the queue, and a data movement operation is performed on the cached data with the larger identity sequence number to move it to the first column unit corresponding to the original storage unit in the same storage table. The aforementioned identity sequence number is determined based on the time information of data storage in the target database.

[0123] Based on the aforementioned second cache data redundancy quantity, the aforementioned size comparison operation and the aforementioned data movement operation are repeatedly performed to obtain the second updated cache unit;

[0124] The second update cache unit retrieves the corresponding number of the above-mentioned data to be queried based on the data size information of each page.

[0125] For example, each storage table unit includes at least two cache units (e.g., cache unit 1 and cache unit 2) and one original storage unit. Different cache units store data within different sorting ranges. For example, cache unit 1 can store the first 100 data entries, while cache unit 2 can store data after the 100th entry.

[0126] The underlying storage unit holds all data in the target database. When the cache unit is insufficient to return the queried data, the system will use a cache expansion mechanism to obtain the required data.

[0127] When a user requests a query for data on a specific page, the system determines which cache unit should retrieve the data from based on the sort sequence number. Different sort sequence number ranges correspond to different cache units. For example, if the queried data sequence number is between 100 and 200, the system will select the cache unit storing data within that range as the target cache unit.

[0128] After identifying the target cache unit, the system checks the amount of data already stored in that unit and the larger of the sort order number corresponding to the queried data. If the amount of data in the target cache unit exceeds the paging requirements, the system calculates the number of data items that need to be moved, i.e., the amount of cached data redundancy. For example, the system might only need to retrieve 5 data items from the cache unit, but the cache unit contains 10 data items, so 5 data items need to be moved out. Within each storage table unit, the system performs a size comparison operation on the data in the target cache unit, selecting the larger data item for the move operation and moving it to the original storage unit (Behind queue) to ensure that the amount of data in the cache meets the requirements.

[0129] The system compares the data at the head of each cache unit queue. For example, if there are data 19 and 23 in cache unit 1 and cache unit 2 of DB1 respectively, the system will select the data with the larger sequence number, 23, and move it to the original storage unit of DB1. This process continues until the amount of data in the cache unit meets the requirements.

[0130] After determining the amount of redundant cached data, the system continuously performs size comparison and data movement operations, moving data exceeding the paging requirements from the cache unit to the original storage unit. In this way, the system ensures that the data in the cache unit is just enough to meet the requirements of paging queries.

[0131] When the amount of data in a cache unit is adjusted, the system will retrieve data that meets the pagination requirements. At this point, the system will extract the requested page data from the cache unit without accessing the original storage unit. For example, if the system needs to query from page 3 onwards, with sorting numbers 11-15, the system will determine which cache unit within the storage table contains the data in that sequence range, perform a data movement operation to ensure that there are 5 matching records in the cache unit, and finally return that data.

[0132] Each piece of data has a unique identifier, typically generated based on the time the data was stored in the database. Using these identifiers, the system can sort the data and determine which data should be prioritized for moving or remaining in the cache.

[0133] It is understood that the above example uses a single storage table unit in the target database. If multiple table units exist, records 1-100 can be allocated to multiple cache units within those table units. The specific values ​​for the sorting sequence range, the number of storage table units, and the number of cache units mentioned above are for illustrative purposes only and do not represent the actual protected values.

[0134] This embodiment of the disclosure achieves efficient optimization of pagination queries by managing multiple cache units and one original storage unit and flexibly utilizing data sorting information. Through dynamic size comparison and data movement operations, the system can balance the data volume of the cache units, ensuring that each pagination query is completed efficiently, reducing the pressure of direct database access, and improving the system's response speed.

[0135] In some examples, where each storage table unit includes a cache unit and the aforementioned original storage unit,

[0136] In the case where the cached data unit does not completely include all the data corresponding to the sorting sequence information, the original storage unit is filtered based on the missing data in the cache unit, and the cache unit is expanded to obtain the filtered original storage unit and the expanded cache unit, including:

[0137] Obtain the larger of the number of third cached data in the above cache unit and the sequence number corresponding to the above sorting sequence number information;

[0138] The number of missing data in the first cache is determined based on the larger of the above sequence numbers and the number of third cache data.

[0139] For each original storage unit corresponding to each storage table unit, a size comparison operation is performed on the first element of the queue, and a data movement operation is performed on the original storage data with the smaller identity sequence number to move it to the first column unit corresponding to the cache unit of the same storage table. The aforementioned identity sequence number is determined based on the time information of data storage in the target database.

[0140] Based on the number of missing first cached data, the above size comparison operation and the above data movement operation are repeatedly performed to obtain the third updated cache unit;

[0141] In the third update cache unit mentioned above, the corresponding number of the above-mentioned data to be queried are obtained based on the data size information of each page.

[0142] For example, as shown in Figures 7 to 9, each storage table unit (e.g., DB1, DB2, DB3) contains a cache unit (Front) and a raw storage unit (Behind). The cache unit stores frequently queried or commonly used data, while the raw storage unit stores complete data. When the cache unit is insufficient to return the paginated data requested by the user, the cache is dynamically expanded to ensure the integrity of data retrieval.

[0143] According to the pagination rules, assuming each page contains 5 records, querying the third page requires returning records 11-15. First, calculate the total amount of data N in the third cache across the three storage table units. As shown in the diagram, N=5, meaning that 5 more records need to be found from the original storage unit (Behind queue) to satisfy the query requirement; that is, the number of missing records in the first cache is 5.

[0144] When the cache unit lacks sufficient data, the system filters from the original storage units. The specific steps are as follows: compare the head data of the Behind queue in each storage unit, find the smallest data, and move it to the corresponding cache unit (Front queue). For example, in the first comparison, if the head data of the Behind queue are 11, 16, and 18, the system selects the smallest, 11, and moves it from the DB2 Behind queue to the DB2 Front queue. This comparison and data movement is repeated until the required amount of data is met.

[0145] For each storage table unit (such as DB1, DB2, DB3), the first element of the queue (the first element of the Behind queue) of the original storage unit is compared, the smallest value is selected, and this value is moved to the corresponding cache unit (Front queue). For example, in the second comparison, the first data of the queue are 14, 16, and 18 respectively, so 14 is selected and placed into the Front queue of DB2.

[0146] The system needs to add enough data from the original storage unit to ensure that the data in the cache unit can meet the needs of pagination queries. By calculating the difference between the current cached data volume and the target data volume, the system determines the amount of data that needs to be added. By repeatedly performing the above size comparison and data movement operations, the system continuously expands the data in the cache unit until enough data is found. For example, in the 15th comparison, the data at the head of the queue is 25, 26, and 27. The smallest one, 25, is selected and moved into the Front queue of DB1.

[0147] When the amount of data in a cache unit reaches the target amount, the system retrieves data from the cache unit. The final result is the 5 data items with the largest IDs from the cache unit, which are returned as the third page. In the example, the IDs for the third page are {19, 21, 23, 24, 25}.

[0148] The method proposed in this disclosure solves the problem of insufficient cached data in paginated queries by dynamically expanding the cache. Through size comparison operations, missing data is gradually filtered from the original storage units and moved to the cache units, thereby reducing the frequency of direct database access and improving query performance and system response speed.

[0149] In some examples, where each storage table unit includes at least two cache units and one of the aforementioned original storage units,

[0150] In the case where the cached data unit does not completely include all the data corresponding to the sorting sequence information, the original storage unit is filtered based on the missing data in the cache unit, and the cache unit is expanded to obtain the filtered original storage unit and the expanded cache unit, including:

[0151] Based on the above sorting sequence information, the corresponding target cache unit is determined in at least two cache units in each storage table unit, wherein different cache units in each storage table unit are determined based on different sorting sequence ranges;

[0152] Obtain the larger of the fourth cache data quantity in the target cache unit and the sequence number corresponding to the sorting sequence information;

[0153] The number of missing data in the second cache is determined based on the larger of the above sequence numbers and the amount of data in the fourth cache.

[0154] For each storage table unit, a size comparison operation is performed on the first element of the queue of the original storage unit, and a data movement operation is performed on the original storage data with the smaller identity number to move it to the first column of the target cache unit corresponding to the same storage table. The aforementioned identity number is determined based on the time information of the data being stored in the target database.

[0155] Based on the number of missing second cached data, the above size comparison operation and the above data movement operation are repeatedly performed to obtain the fourth updated cache unit;

[0156] The fourth update cache unit above retrieves the corresponding number of the above-mentioned data to be queried based on the data size information of each page.

[0157] For example, each storage table unit includes at least two cache units and one original storage unit. The cache units store partial data, and different cache units store data based on different sort number ranges. For example, cache unit 1 may store sort numbers 1-50, and cache unit 2 may store sort numbers 51-100.

[0158] Based on the sort order information of the query, the system will first determine which cache unit (cache unit 1 or cache unit 2) corresponds to the data in that sort range. For example, if the queried data sort order is between 60 and 80, the system will select cache unit 2 (which stores data with sort order of 51-100) as the target cache unit.

[0159] After identifying the target cache unit, the system checks the amount of data currently stored in that unit, i.e., the fourth cache data quantity. The system calculates the larger of the sequence numbers of the data to be retrieved based on the sort order information. By comparing the larger sort order number with the amount of data in the current cache unit, the system determines the amount of missing data, i.e., the second cache data missing quantity. This step ensures that the system knows how much data needs to be replenished from the original storage unit.

[0160] To fill in missing data in the target cache unit, the system sequentially compares the first element of the original storage unit (Behind queue) in each storage table unit. Based on the data's sequence number (generated from the time the data was stored in the database), the system moves smaller data items to the target cache unit. The sequence number identifies the time order of the data; therefore, comparing these numbers ensures that data is moved into the cache unit in the correct order. Smaller data items from the original storage unit are moved to the target cache unit. For example, if the first elements of the queues in the original storage units are 60, 55, and 70, the system will select the element with the largest sequence number, 55, and move it into the cache unit.

[0161] To replenish the data in the target cache unit, the system repeatedly performs size comparisons and data movement operations until the amount of data in the target cache unit meets the requirements of pagination queries. During each filtering step, smaller data items are moved to the target cache unit to ensure that data is populated in the correct order.

[0162] Through multiple filtering and data movement operations, the system fills the missing data in the original storage unit into the target cache unit, ultimately generating an updated cache unit. After the data volume in the cache unit is updated, the system extracts the corresponding number of data from the cache unit based on the size information of each page, as the final query result.

[0163] Suppose the system needs to retrieve data with sorted sequence numbers 60-80, but the data in cache unit 2 is incomplete. The system first identifies cache unit 2 as the target cache unit, then filters data from the original storage unit, moving data items with smaller sequence numbers (such as 70, 75, 80, etc.) into cache unit 2. By repeatedly performing this data movement operation, the system eventually ensures that cache unit 2 contains the data within the required sorted range and returns the paginated data needed by the user.

[0164] It is understood that the above example uses a single storage table unit in the target database. If multiple table units exist, records 1-100 can be allocated to multiple cache units within those table units. The specific values ​​for the sorting sequence range, the number of storage table units, and the number of cache units mentioned above are for illustrative purposes only and do not represent the actual protected values.

[0165] The method proposed in this disclosure dynamically determines the target cache unit based on sorting sequence information and supplements missing data from the original storage unit, ensuring that the system can efficiently handle pagination query requests. By supplementing and expanding the cache unit, the system can complete queries without frequent database access, improving query performance. By using identity sequence numbers and size comparison operations, the system ensures the orderly movement of data, guaranteeing the correctness and continuity of data during pagination queries. Through cache expansion and original storage unit filtering mechanisms, efficient querying is achieved even when cached data is incomplete. The system can dynamically filter and move data to ensure that the data in the target cache unit meets the query requirements and quickly return the pagination query results through the updated cache unit. This method is suitable for high-frequency pagination query scenarios, effectively improving system performance and response speed.

[0166] In some examples, it also includes:

[0167] If the current query is the first query, the data to be queried is obtained from the original storage unit based on the secondary query method.

[0168] In some examples, it also includes:

[0169] If the difference between the sorting sequence number of the data to be queried and the sorting sequence number of the data stored in the current cache unit is greater than a preset difference, the data to be queried is obtained from the original storage unit based on the secondary query method.

[0170] For example, in some scenarios, when a user performs a data query for the first time, the cache unit may not yet contain any data. In this case, the system needs to retrieve the data the user needs directly from the original storage unit using a "secondary query method." The specific steps are as follows:

[0171] The system first determines if the current query is the first query. If it is the first query, the cache unit has not yet been created, so data cannot be retrieved from the cache unit. In the case of the first query, the system uses a secondary lookup method to directly retrieve data from the original storage unit (Behind queue). The secondary lookup method first searches for all the required data in the original storage unit. This data is then temporarily stored in the cache unit so that subsequent queries can retrieve data from the cache more efficiently.

[0172] If a cached unit already contains some data, but the sort order of this data differs significantly from the sort order of the currently queried data, the system will consider retrieving the required data directly from the original storage unit. This is to avoid invalid data in the cache causing a decrease in query efficiency. The system compares the sort order of the data stored in the cached unit with the sort order of the data to be queried. If the difference between the sort order orders exceeds a preset threshold (called the "preset difference"), the system determines that the data in the cache is no longer suitable for the current query. When the difference is too large, the system uses a secondary query method to directly retrieve the data to be queried from the original storage unit, instead of retrieving it from the cache. This avoids using invalid data in the cache, thereby improving query efficiency. The system calculates the difference between the largest or smallest sort order in the cached unit and the sort order of the data to be queried. For example, if the largest sort order in the cached unit is 30, and the user is querying the 60th data item, then the difference is 30. The system compares this difference with a preset threshold (such as 20). If the difference exceeds 20, it considers the difference between the cached data and the data to be queried to be too large, requiring a secondary query. While retrieving data from the original storage unit using the secondary query method, the system also stores the new query data in the cache unit for easy subsequent queries.

[0173] The secondary lookup method can retrieve data directly from the original storage unit when the cached data is insufficient or the difference is too large, ensuring that users can obtain the required query results in a timely manner. Each time data is retrieved using the secondary lookup method, the system automatically stores this data in the cache unit, improving the efficiency of subsequent queries and reducing the frequency of direct database access. Whether it's the initial query or a cache invalidation due to a large difference between the cached and the data to be queried, the secondary lookup method can flexibly adapt, guaranteeing the system's query efficiency and data accuracy.

[0174] The second-order query method is shown in Figures 10-13. The specific steps are as follows:

[0175] 1. First query:

[0176] Data for each database shard / table is obtained based on the global offset. Assuming we need to retrieve the data for the third page, with 5 records per page, the first two pages have already returned 10 records, and the current global offset is 10.

[0177] Assuming the data is evenly distributed across 3 tables (N=3), based on the total offset of 10, a modulo operation is performed on the 3 databases / tables. The offset for each table is: offset = 10 / 3. Taking the integer part, the offset is 3.

[0178] Execute an SQL query in each database / table to retrieve 5 records starting from offset 3 for each database:

[0179] Perform the operation based on the following code:

[0180] select*from DB1 order by id asc limit 5offset 3;

[0181] select*from DB2 order by id asc limit 5offset 3;

[0182] select*from DB3 order by id asc limit 5offset 3;

[0183] The first query result was:

[0184] The dataset obtained by DB1 is: {25,28,31,34,37}

[0185] The dataset obtained by DB2 is: {17,23,26,29,32}

[0186] The dataset obtained by DB3 is: {18,21,24,26,33}

[0187] From the datasets returned by each database shard / table, compare the first record of each database and select the smallest id. In this example, the smallest id is 17 (from DB2).

[0188] 2. Second query step: Query based on the id range

[0189] In the second query, the query range is constructed based on the minimum ID obtained from the first query (id=17) and the maximum ID returned by each shard. The query range for each shard / table is as follows:

[0190] Further filtering is performed based on the following code:

[0191] select*from DB1 order by id asc where id between 17and 37;

[0192] select*from DB3 order by id asc where id between 17and 33;

[0193] The minimum ID in DB2 is 17, so DB2 queries can be omitted.

[0194] The query results are:

[0195] DB1: {19,25,28,31,34,37}

[0196] DB2: {17,23,26,29,32}

[0197] DB3: {18,21,24,27,33}

[0198] Analyzing the result set, we determine the global offset. By analyzing the results of the second query, we can determine the offset of `idmin` (the minimum `id` = 17) in each database. In DB1, `id` = 19 is the minimum value of the query condition, therefore the offset of the virtual `idmin` is 2 (meaning the offset in DB1 is 2). In DB2, `id` = 17 remains unchanged, so the offset is 4 (because `id` = 17 is the first record in DB2). In DB3, `id` = 18 is the minimum value of the query condition, therefore the offset of the virtual `idmin` is 3.

[0199] The global offset is calculated by adding the offsets of the virtual IDmin values ​​from each database shard:

[0200] Global offset = DB1_offset + DB2_offset + DB3_offset = 2 + 4 + 3 = 9.

[0201] Based on the global offset = 9, the required pagination data is calculated. Since the current requirement is to retrieve data from the third page, the offset is 10 (meaning that values ​​are retrieved starting from global offset = 10, with a maximum of 5 data entries).

[0202] Merge and sort all query results: id={17,18,19,21,23,24,25,26,27,28,29,31,32,33,34,37}

[0203] Starting from global offset=10, the first 5 data are retrieved, and the final result is: id={19,21,23,24,25}.

[0204] In some examples, it also includes:

[0205] If any one or more of the following conditions are met: query conditions change, user status information expires, session expires, and expiration date expires, the cached data in the above cache unit will be cleaned up.

[0206] For example, data cleanup operations are typically triggered under one or more of the following conditions:

[0207] Query condition changes: If a user's query conditions change, the cached data may no longer be applicable to the new query. For example, if a user changes the filter conditions when performing a paginated query (e.g., changing from sorting by date to sorting by price), the cached data will no longer match the new query conditions and needs to be cleared.

[0208] User status information expires: User status information in the system (such as user login status, permission information, etc.) may expire after a period of time. If a user's status expires, and the cache still returns data based on expired user information, it may lead to data inconsistency. In this case, the cached data should be cleared to ensure that the query results for data with the new status are correct.

[0209] Session expiration: A user's session information may expire or become invalid after a prolonged period of inactivity. If a user logs in again or re-initiates a request after a session has expired, the system needs to clear the old data in the cache to ensure that the new session can return the latest data related to the current session state.

[0210] Expiration Date: Cached data typically has an expiration date. When cached data exceeds this expiration date, the system considers the data outdated and should be cleared to ensure that the data returned to the user is up-to-date. This mechanism is often used to periodically refresh the data cache.

[0211] The method provided in this disclosure ensures that the system cache stores only the latest and most valid data by clearing data in the cache unit under specific conditions. By clearing expired or invalid cache, the system's query performance can be improved, invalid data storage can be reduced, and erroneous data returns due to inconsistent states can be prevented. This plays a crucial role in ensuring the system's efficiency and correctness.

[0212] In some examples, it also includes:

[0213] When a data addition operation is performed on the aforementioned target database, the newly added data will be stored in the aforementioned original storage unit; and / or,

[0214] In the event of data deletion operations in the aforementioned target database, the corresponding data in the aforementioned cache unit and / or the aforementioned original storage unit will be deleted.

[0215] For example, when new data is inserted into the database, the system needs to store the new data in the original storage unit and update the relevant information of the new data in the cache unit according to the specific caching strategy to ensure that future queries can retrieve this new data in a timely manner. All newly inserted data is first written to the original storage unit (Behind queue). The new data is placed in the correct location according to the database's storage rules, and its position in the database is determined based on the new ID or other storage rules. New data does not immediately affect the current pagination query results. If a pagination query is currently in progress, the query results are not affected by the new data because the cached data is not automatically refreshed due to new data. The new data will be included in the query results during the next query or cache refresh.

[0216] As shown in Figures 14 and 15, when new data is inserted into the database, the system appends the newly added data ID information to the cache unit. For example, assuming the newly added data IDs in the database are {47,48,49}, these IDs will be recorded in the cache unit, but will not immediately affect the currently cached query results. If a user queries the last page of data, the system will directly include the last newly added data in the result set according to the rules of the cache and original storage units. For example, if the IDs appended to the cache are {47,48,49}, these data will be directly displayed on the last page, and the returned result set will be {47,48,49}. In this case, the last page may contain fewer data entries than the specified number per page because the previous pages have already displayed the complete dataset.

[0217] When data is deleted from the database, the system needs to promptly update the data in both the original storage unit and the cache unit to ensure that subsequent queries reflect the latest database state. When data is deleted from the database, the system removes the corresponding data from the original storage unit (Behind queue). The system checks the data currently stored in the cache unit; if the deleted data also exists in the cache unit, the cache needs to be updated to remove the corresponding IDs and data content. Data deletion does not immediately affect currently executing queries. If a user has already queried a page of data, and the current page contains deleted data, the data displayed on the current page will remain unchanged and unaffected.

[0218] As shown in Figures 16 and 17, once a user initiates a new query request, the system will refresh the cache to ensure that deleted data is no longer included in the query results. For example, the data the user sees on the first page of query results is {1,2,3,6,9}, but after the data with ID 6 is deleted, the system will update the cache on the next query, and the new query result will return {1,2,3,9,11}.

[0219] Please refer to Figure 18, a structural schematic diagram of a data query control device provided in this embodiment of the present disclosure, which may include:

[0220] The first acquisition unit 21 is used to acquire the user's page number information to be queried and the data size information per page, wherein the data size information per page is the number of data included in each query page;

[0221] The determining unit 22 is used to determine the sorting sequence information of the data to be queried based on the above-mentioned page number information and the above-mentioned data size information per page;

[0222] The second acquisition unit 23 is used to acquire the data to be queried from the cache unit and / or the original storage unit of the target database according to the sorting sequence information. The cache unit and the original storage unit jointly store all the data information of the target database. There is no duplicate data stored between the cache unit and the original storage unit. The target database includes multiple storage table units. Each storage table unit includes at least one cache unit and at least one original storage unit. The data stored in the cache unit is generated based on the data history query information.

[0223] The aforementioned data query control device can also perform the following steps:

[0224] In some examples, retrieving the data to be queried from the cache unit and / or raw storage unit of the target database based on the aforementioned sorting sequence information includes:

[0225] If the aforementioned cached data unit includes all data corresponding to the aforementioned sorting sequence information, then the aforementioned data to be queried is retrieved from the aforementioned cached unit; and / or,

[0226] In cases where the aforementioned cached data unit does not completely include all the data corresponding to the aforementioned sorting sequence information, the aforementioned original storage unit is filtered based on the missing data in the cache unit and the aforementioned cache unit is expanded to obtain the filtered original storage unit and the expanded cache unit.

[0227] The data to be queried is retrieved from the expanded cache unit.

[0228] In some examples, where each storage table unit includes a cache unit and the aforementioned original storage unit,

[0229] In the case that the cached data unit includes all the data corresponding to the sorting sequence information, retrieving the data to be queried from the cached unit includes:

[0230] Obtain the larger of the first cached data quantity in the above cache unit and the sequence number corresponding to the above sorting sequence number information;

[0231] The redundancy of the first cache data is determined based on the larger value of the above sequence number and the above quantity of the first cache data.

[0232] For each cache unit corresponding to each storage table unit, a size comparison operation is performed on the first element of the queue, and a data movement operation is performed on the cached data with the larger identity sequence number to move it to the first column unit corresponding to the original storage unit in the same storage table. The aforementioned identity sequence number is determined based on the time information of data storage in the target database.

[0233] Based on the aforementioned first cached data redundancy quantity, the aforementioned size comparison operation and the aforementioned data movement operation are repeatedly executed to obtain the first updated cache unit;

[0234] In the first update cache unit mentioned above, the corresponding number of the above-mentioned data to be queried are obtained based on the data size information of each page.

[0235] In some examples, where each storage table unit includes at least two cache units and one of the aforementioned original storage units,

[0236] In the case that the cached data unit includes all the data corresponding to the sorting sequence information, retrieving the data to be queried from the cached unit includes:

[0237] Based on the above sorting sequence information, the corresponding target cache unit is determined in at least two cache units in each storage table unit, wherein different cache units in each storage table unit are determined based on different sorting sequence ranges;

[0238] Obtain the larger of the number of second cached data in the target cache unit and the sequence number corresponding to the sorting sequence number information;

[0239] The amount of redundancy in the second cache data is determined based on the larger value of the above sequence number and the amount of the second cache data.

[0240] For each target cache unit corresponding to each storage table unit, a size comparison operation is performed on the first element of the queue, and a data movement operation is performed on the cached data with the larger identity sequence number to move it to the first column unit corresponding to the original storage unit in the same storage table. The aforementioned identity sequence number is determined based on the time information of data storage in the target database.

[0241] Based on the aforementioned second cache data redundancy quantity, the aforementioned size comparison operation and the aforementioned data movement operation are repeatedly performed to obtain the second updated cache unit;

[0242] The second update cache unit retrieves the corresponding number of the above-mentioned data to be queried based on the data size information of each page.

[0243] In some examples, where each storage table unit includes a cache unit and the aforementioned original storage unit,

[0244] In the case where the cached data unit does not completely include all the data corresponding to the sorting sequence information, the original storage unit is filtered based on the missing data in the cache unit, and the cache unit is expanded to obtain the filtered original storage unit and the expanded cache unit, including:

[0245] Obtain the larger of the number of third cached data in the above cache unit and the sequence number corresponding to the above sorting sequence number information;

[0246] The number of missing data in the first cache is determined based on the larger of the above sequence numbers and the number of third cache data.

[0247] For each original storage unit corresponding to each storage table unit, a size comparison operation is performed on the first element of the queue, and a data movement operation is performed on the original storage data with the smaller identity sequence number to move it to the first column unit corresponding to the cache unit of the same storage table. The aforementioned identity sequence number is determined based on the time information of data storage in the target database.

[0248] Based on the number of missing first cached data, the above size comparison operation and the above data movement operation are repeatedly performed to obtain the third updated cache unit;

[0249] In the third update cache unit mentioned above, the corresponding number of the above-mentioned data to be queried are obtained based on the data size information of each page.

[0250] In some examples, where each storage table unit includes at least two cache units and one of the aforementioned original storage units,

[0251] In the case where the cached data unit does not completely include all the data corresponding to the sorting sequence information, the original storage unit is filtered based on the missing data in the cache unit, and the cache unit is expanded to obtain the filtered original storage unit and the expanded cache unit, including:

[0252] Based on the above sorting sequence information, the corresponding target cache unit is determined in at least two cache units in each storage table unit, wherein different cache units in each storage table unit are determined based on different sorting sequence ranges;

[0253] Obtain the larger of the fourth cache data quantity in the target cache unit and the sequence number corresponding to the sorting sequence information;

[0254] The number of missing data in the second cache is determined based on the larger of the above sequence numbers and the amount of data in the fourth cache.

[0255] For each storage table unit, a size comparison operation is performed on the first element of the queue of the original storage unit, and a data movement operation is performed on the original storage data with the smaller identity number to move it to the first column of the target cache unit corresponding to the same storage table. The aforementioned identity number is determined based on the time information of the data being stored in the target database.

[0256] Based on the number of missing second cached data, the above size comparison operation and the above data movement operation are repeatedly performed to obtain the fourth updated cache unit;

[0257] The fourth update cache unit above retrieves the corresponding number of the above-mentioned data to be queried based on the data size information of each page.

[0258] In some examples, it also includes:

[0259] If the current query is the first query, the data to be queried is obtained from the original storage unit based on the secondary query method.

[0260] In some examples, it also includes:

[0261] If the difference between the sorting sequence number of the data to be queried and the sorting sequence number of the data stored in the current cache unit is greater than a preset difference, the data to be queried is obtained from the original storage unit based on the secondary query method.

[0262] In some examples, it also includes:

[0263] If any one or more of the following conditions are met: query conditions change, user status information expires, session expires, and expiration date expires, the cached data in the above cache unit will be cleaned up.

[0264] In some examples, it also includes:

[0265] When a data addition operation is performed on the aforementioned target database, the newly added data will be stored in the aforementioned original storage unit; and / or,

[0266] In the event of data deletion operations in the aforementioned target database, the corresponding data in the aforementioned cache unit and / or the aforementioned original storage unit will be deleted.

[0267] As shown in FIG19, this embodiment of the present disclosure also provides an electronic device 300, including a memory 310, a processor 320, and a computer program 311 stored in the memory 310 and executable on the processor. When the processor 320 executes the computer program 311, it implements the steps of any of the above-described data query control methods.

[0268] As shown in FIG7, this embodiment of the present disclosure also provides a display device 30, including an electronic device 300. The electronic device includes a memory 310, a processor 320, and a computer program 311 stored in the memory 310 and executable on the processor. When the processor 320 executes the computer program 311, it implements the steps of any of the above-described data query control methods.

[0269] Since the electronic device described in this embodiment is a device used to implement a data query control device in the embodiments of this disclosure, those skilled in the art can understand the specific implementation method and various variations of the electronic device in this embodiment based on the method described in the embodiments of this disclosure. Therefore, how the electronic device implements the method in the embodiments of this disclosure will not be described in detail here. Any device used by those skilled in the art to implement the method in the embodiments of this disclosure is within the scope of protection of this disclosure.

[0270] In practice, when the computer program 311 is executed by the processor, it can implement any of the embodiments corresponding to the first aspect.

[0271] It should be noted that the descriptions of each embodiment in the above embodiments have different focuses. For parts that are not described in detail in a certain embodiment, please refer to the relevant descriptions in other embodiments.

[0272] Those skilled in the art will understand that embodiments of this disclosure can be provided as methods, systems, or computer program products. Therefore, this disclosure can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this disclosure can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0273] This disclosure is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this disclosure. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded computer, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in one or more flowchart illustrations and / or one or more block diagrams.

[0274] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means that implement the functions specified in one or more flowcharts and / or one or more block diagrams.

[0275] These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process, such that the instructions, which execute on the computer or other programmable apparatus, provide steps for implementing the functions specified in one or more flowcharts and / or one or more block diagrams.

[0276] This disclosure also provides a computer program product including computer software instructions that, when executed on a processing device, cause the processing device to execute the data query control process described in the corresponding embodiment.

[0277] A computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the flow or function according to embodiments of this disclosure is generated. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, computer instructions may be transmitted from one website, computer, server, or data center to another via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium may be any available medium that a computer can store or a data storage device such as a server or data center that integrates one or more available media. The available medium may be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid-state disk (SSD)).

[0278] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0279] In the several embodiments provided in this disclosure, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces, indirect coupling or communication connection between apparatuses or units, and may be electrical, mechanical, or other forms.

[0280] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0281] Furthermore, the functional units in the various embodiments of this disclosure can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0282] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this disclosure, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this disclosure. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0283] The above embodiments are only used to illustrate the technical solutions of this disclosure, and are not intended to limit it. Although this disclosure has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this disclosure.< / condition> < / condition>

Claims

1. A data query control method, characterized in that, include: The system obtains the user's page number to be queried and the data size information per page, wherein the data size information per page is the number of data items included in each query page; based on the page number to be queried and the data size information per page, it determines the sorting sequence information of the data to be queried; according to the sorting sequence information, it retrieves the data to be queried from the cache unit and / or the original storage unit of the target database, wherein the cache unit and the original storage unit jointly store all data information of the target database, and there is no duplicate data stored between the cache unit and the original storage unit. The target database includes multiple storage table units, and each storage table unit includes at least one cache unit and at least one original storage unit. The data stored in the cache unit is generated based on historical data query information.

2. The data query control method according to claim 1, characterized in that, The step of retrieving the data to be queried from the cache unit and / or original storage unit of the target database according to the sorting sequence information includes: when the cache data unit does not completely include all the data corresponding to the sorting sequence information, performing a filtering operation on the original storage unit based on the missing data in the cache unit and expanding the cache unit to obtain a filtered original storage unit and an expanded cache unit; retrieving the data to be queried from the expanded cache unit; and / or, when the cache data unit includes all the data corresponding to the sorting sequence information, retrieving the data to be queried from the cache unit.

3. The data query control method according to claim 2, characterized in that, When each storage table unit includes a cache unit and the original storage unit, the step of obtaining the data to be queried in the cache unit when the cache data unit includes all the data corresponding to the sorting sequence information includes: obtaining the larger value of the first cache data quantity and the sequence number corresponding to the sorting sequence information in the cache unit; determining the first cache data redundancy quantity based on the larger sequence number and the first cache data quantity; performing a size comparison operation on the first element of the queue of each cache unit corresponding to each storage table unit, and performing a data movement operation on the cache data with the larger identity sequence number to move it to the column head unit corresponding to the original storage unit of the same storage table, wherein the identity sequence number is determined based on the time information of data storage in the target database; repeatedly performing the size comparison operation and the data movement operation based on the first cache data redundancy quantity to obtain a first updated cache unit; and obtaining the corresponding number of the data to be queried in the first updated cache unit based on the data size information per page.

4. The data query control method according to claim 2, characterized in that, When each storage table unit includes at least two cache units and one original storage unit, the step of obtaining the data to be queried from the cache unit when the cache data unit includes all the data corresponding to the sorting sequence information includes: determining the target cache unit corresponding to the at least two cache units in each storage table unit according to the sorting sequence information, wherein different cache units in each storage table unit are determined based on different sorting sequence ranges; obtaining the second cache data quantity in the target cache unit and the larger value of the sequence number corresponding to the sorting sequence information; determining the second cache data redundancy quantity based on the larger value of the sequence number and the second cache data quantity; performing a size comparison operation on the queue head element of each target cache unit corresponding to each storage table unit, and performing a data movement operation on the cache data with the larger identity sequence number to move it to the column head unit corresponding to the original storage unit of the same storage table, wherein the identity sequence number is determined based on the time information of data storage in the target database; repeatedly performing the size comparison operation and the data movement operation based on the second cache data redundancy quantity to obtain a second updated cache unit; and obtaining the corresponding number of data to be queried in the second updated cache unit based on the data size information per page.

5. The data query control method according to claim 2, characterized in that, In the case where each storage table unit includes a cache unit and the original storage unit, and the cache data unit does not completely include all the data corresponding to the sorting sequence information, the original storage unit is filtered based on the missing data in the cache unit and expanded to obtain a filtered original storage unit and an expanded cache unit. This includes: obtaining the larger value between the third cache data quantity in the cache unit and the sequence number corresponding to the sorting sequence information; determining the first cache data missing quantity based on the larger sequence number and the third cache data quantity; performing a size comparison operation on the first element of the queue of each original storage unit corresponding to each storage table unit, and performing a data movement operation on the original storage data with the smaller identity sequence number to move it to the column head unit corresponding to the cache unit of the same storage table, wherein the identity sequence number is determined based on the time information of data storage in the target database; repeatedly performing the size comparison operation and the data movement operation based on the first cache data missing quantity to obtain a third updated cache unit; and obtaining the corresponding number of the data to be queried in the third updated cache unit based on the data size information per page.

6. The data query control method according to claim 2, characterized in that, When each storage table unit includes at least two cache units and one original storage unit, and when the cached data unit does not completely include all the data corresponding to the sorting sequence information, the original storage unit is filtered based on the missing data in the cache unit and the cache unit is expanded to obtain the filtered original storage unit and the expanded cache unit. This includes: determining the corresponding target cache unit in at least two cache units in each storage table unit according to the sorting sequence information, wherein different cache units in each storage table unit are determined based on different sorting sequence ranges; obtaining the number of fourth cached data in the target cache unit and the sorting sequence information. The larger of the corresponding sequence number is used to determine the number of missing second cached data. A size comparison operation is performed on the first element of the queue of the original storage unit corresponding to each storage table unit, and a data movement operation is performed on the original storage data with the smaller identity sequence number to move it to the first column unit corresponding to the target cache unit in the same storage table. The identity sequence number is determined based on the time information of the data being stored in the target database. The size comparison operation and the data movement operation are repeated based on the number of missing second cached data to obtain the fourth updated cache unit. The corresponding number of data to be queried is obtained in the fourth updated cache unit based on the data size information per page.

7. The data query control method according to any one of claims 1 to 6, characterized in that, Also includes: If the current query is the first query, the data to be queried is obtained from the original storage unit based on the secondary query method.

8. The data query control method according to any one of claims 1 to 6, characterized in that, Also includes: If the difference between the sorting sequence number of the data to be queried and the sorting sequence number of the data stored in the current cache unit is greater than a preset difference, the data to be queried is obtained from the original storage unit based on the secondary query method.

9. The data query control method according to any one of claims 1 to 6, characterized in that, Also includes: If any one or more of the following conditions are met: query condition change, user status information expiration, session expiration, and expiration date expiration, the cached data in the cache unit is cleaned up.

10. The data query control method according to any one of claims 1 to 6, characterized in that, Also includes: When a data addition operation is performed on the target database, the newly added data is stored in the original storage unit; And / or, in the event of a data deletion operation in the target database, the corresponding data in the cache unit and / or the original storage unit will be deleted.

11. A data query control device, characterized in that, include: The first acquisition unit is used to acquire the user's page number information to be queried and the data size information per page, wherein the data size information per page is the number of data included in each query page; the determination unit is used to determine the sorting sequence number information of the data to be queried based on the page number information to be queried and the data size information per page; the second acquisition unit is used to acquire the data to be queried from the cache unit and / or the original storage unit of the target database according to the sorting sequence number information, wherein the cache unit and the original storage unit jointly store all the data information of the target database, there is no duplicate data stored between the cache unit and the original storage unit, the target database includes multiple storage table units, each storage table unit includes at least one cache unit and at least one original storage unit, and the data stored in the cache unit is generated based on the data historical query information.

12. An electronic device, comprising: A memory and a processor, characterized in that the processor, when executing a computer program stored in the memory, implements the steps of the data query control method as described in any one of claims 1-10.

13. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the steps of the data query control method as described in any one of claims 1-10.

Images