A paging query method, device and server of an OLAP database

Abstract

The application discloses a kind of OLAP database's paging query method, device and server, is related to database technical field, the method includes: server obtains the interface paging query request corresponding to the query data set of client device sent;Determine the paging query file piece corresponding to interface paging query request in the target file piece of storage;Wherein, paging query file piece includes one or two target file piece;From paging query file piece, the paging data corresponding to interface paging query request is obtained;Paging data is sent to client device, to make client device display paging data;The application can reduce the data query times of OLAP database by the pre-storage of the query data set corresponding to interface paging query request, reduce the consumption of OLAP database's paging query to database performance;Query data set is stored in the form of file piece, reduces the data reading quantity of paging query, can improve paging query rate.

Description

Technical Field

[0001] This invention relates to the field of database technology, and in particular to a pagination query method, apparatus, and server for OLAP databases. Background Technology

[0002] Currently, OLAP (Online Analytical Processing) databases implement pagination using the commonly used offset scheme. For example, given a table with 1 million records, if the interface displays 10 records per page, the maximum page number (e.g., ...) is... Figure 1 If N is 100,000, then when a user wants to view the last page of data, they send the corresponding SQL (Structured Query Language) to the database, such as "select * from table limit 999990,10". The database will read all the data, discard the first 999990 records, and return the remaining 10 results. This means that with this pagination method, the later the page number requested, the greater the pagination query pressure. Furthermore, since most OLAP databases are distributed storage, the performance consumption of the database increases with the cost of the number of partitions.

[0003] Therefore, how to reduce the performance consumption of pagination queries in OLAP databases and improve the pagination query speed is an urgent problem to be solved. Summary of the Invention

[0004] The purpose of this invention is to provide a pagination query method, apparatus, and server for OLAP databases, thereby reducing the performance consumption of pagination queries in OLAP databases and improving pagination query speed.

[0005] To solve the above technical problems, the present invention provides a pagination query method for OLAP databases, comprising:

[0006] The server obtains the interface pagination query request corresponding to the query dataset sent by the client device; wherein, the query dataset is the dataset corresponding to the target data query request in the OLAP database, and the target data query request is the data query request corresponding to the interface pagination query request;

[0007] Determine the pagination query file segment corresponding to the interface pagination query request in the stored target file segment; wherein, each target file segment stores data in the query dataset that is less than or equal to a preset data volume, and the pagination query file segment includes one or two target file segments;

[0008] The pagination data corresponding to the interface pagination query request is obtained from the pagination query file fragment;

[0009] The pagination data is sent to the client device so that the client device can display the pagination data.

[0010] In some embodiments, determining the pagination query file slice corresponding to the interface pagination query request in the stored target file slice includes:

[0011] Based on the page size and requested page in the interface pagination query request, the offset range of the query dataset is determined; wherein, if the remainder when the data volume of the query dataset is divided by the preset data volume is 0, then the number of target file pieces is the quotient when the data volume of the query dataset is divided by the preset data volume; if the remainder when the data volume of the query dataset is divided by the preset data volume is not 0, then the number of target file pieces is the quotient when the data volume of the query dataset is divided by the preset data volume plus 1;

[0012] The pagination offset range of the pagination query file slice is determined based on the offset range and the preset data volume;

[0013] The data corresponding to the pagination offset range in the pagination query file slice is determined as the pagination data.

[0014] In some embodiments, determining the pagination offset range of the pagination query file slice based on the offset range and the preset data volume includes:

[0015] Based on the offset range and the preset data volume, the number identifier of the paginated query file piece and the pagination offset range are determined; wherein, the preset data volume is an integer multiple of the page size, and the paginated query file piece includes a target file piece.

[0016] In some embodiments, the number identifier is the quotient of the minimum value of the offset range divided by the preset data amount plus 1, and the paging offset range is the remainder of the minimum value of the offset range divided by the preset data amount to the remainder of the maximum value of the offset range divided by the preset data amount.

[0017] In some embodiments, retrieving the pagination data corresponding to the interface pagination query request from the pagination query file slice includes:

[0018] Determine whether the content of the paginated query file slice is stored in the server's cache;

[0019] If so, the pagination data is retrieved from the contents of the cached pagination query file slice;

[0020] If not, the pagination data is retrieved from the pagination query file slice stored on the disk or in the cloud, and the content of the pagination query file slice is stored in the cache; wherein the target file slice is stored on the server's disk and / or in the cloud.

[0021] In some embodiments, the method further includes:

[0022] After the unqueried time of the target file fragment reaches a preset cleanup time, the target file fragment stored on the local disk is deleted.

[0023] In some embodiments, before the server obtains the interface pagination query request corresponding to the query dataset sent by the client device, the method further includes:

[0024] The server obtains the target data query request sent by the client device;

[0025] The query dataset corresponding to the target data query request is obtained from the OLAP database;

[0026] The query dataset is segmented according to the preset data volume to obtain the target file fragments;

[0027] The target file fragments are stored locally on disk and / or in the cloud.

[0028] In some embodiments, obtaining the query dataset corresponding to the target data query request from the OLAP database includes:

[0029] The query dataset is read from the OLAP database in a streaming manner.

[0030] This invention also provides a pagination query device for an OLAP database, applied to a server, comprising:

[0031] The pagination acquisition module is used to acquire the interface pagination query request corresponding to the query dataset sent by the client device; wherein, the query dataset is the dataset corresponding to the target data query request in the OLAP database, and the target data query request is the data query request corresponding to the interface pagination query request;

[0032] The file determination module is used to determine the pagination query file slice corresponding to the interface pagination query request in the stored target file slice; wherein, each target file slice stores data in the query dataset that is less than or equal to a preset data volume, and the pagination query file slice includes one or two target file slices;

[0033] The data query module is used to retrieve the pagination data corresponding to the interface pagination query request from the pagination query file fragment;

[0034] A data sending module is used to send the pagination data to the client device so that the client device can display the pagination data.

[0035] In addition, the present invention also provides a server, comprising:

[0036] Memory, used to store computer programs;

[0037] A processor is configured to execute the computer program to implement the steps of the pagination query method for an OLAP database as described above.

[0038] The present invention provides a pagination query method for an OLAP database, comprising: a server obtaining an interface pagination query request corresponding to a query dataset sent by a client device; wherein the query dataset is a dataset corresponding to a target data query request in the OLAP database, and the target data query request is a data query request corresponding to the interface pagination query request; determining a pagination query file segment corresponding to the interface pagination query request in a stored target file segment; wherein each target file segment stores data in the query dataset that is less than or equal to a preset data volume, and the pagination query file segment includes one or two target file segments; querying the pagination data corresponding to the interface pagination query request from the pagination query file segment; and sending the pagination data to the client device so that the client device displays the pagination data.

[0039] As can be seen, by pre-storing the query dataset corresponding to the interface pagination query request, this invention can reduce the number of data queries in the OLAP database and reduce the performance consumption of OLAP database pagination queries. Storing the query dataset in the form of file fragments allows the corresponding pagination query file fragment in the stored target file fragment to be determined according to the interface pagination query request, thereby retrieving the pagination data corresponding to the interface pagination query request from the pagination query file fragment, reducing the number of data reads, improving the pagination query speed, and reducing server power consumption. Furthermore, this invention also provides an OLAP database pagination query device and server, which also have the above-mentioned beneficial effects. Attached Figure Description

[0040] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0041] Figure 1 This is a diagram illustrating a paginated data display interface in existing technology.

[0042] Figure 2 A flowchart illustrating a pagination query method for an OLAP database provided in an embodiment of the present invention;

[0043] Figure 3 This is a flowchart illustrating another OLAP database pagination query method provided in an embodiment of the present invention;

[0044] Figure 4 A flowchart of the file slice storage process for another OLAP database pagination query method provided in an embodiment of the present invention;

[0045] Figure 5 This is a structural block diagram of an OLAP database pagination query device provided in an embodiment of the present invention;

[0046] Figure 6 This is a schematic diagram of the structure of a server provided in an embodiment of the present invention;

[0047] Figure 7 This is a schematic diagram of a specific structure of a server provided in an embodiment of the present invention. Detailed Implementation

[0048] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0049] Please refer to Figure 2 , Figure 2 This is a flowchart illustrating a pagination query method for an OLAP database provided in an embodiment of the present invention. The method may include:

[0050] Step 101: The server obtains the interface pagination query request corresponding to the query dataset sent by the client device; wherein, the query dataset is the dataset corresponding to the target data query request in the OLAP database, and the target data query request is the data query request corresponding to the interface pagination query request.

[0051] It is understood that, in this embodiment, the query dataset can be the dataset containing the data of the corresponding page (i.e., the paginated data) to be queried by the interface pagination query request, that is, the dataset corresponding to the data in all paginated pages; the target data query request can be a request to query and read the query dataset from an OLAP (Online Analytical Processing) database, that is, the query dataset can be the data set corresponding to the target data query request in the OLAP database. The interface pagination query request can be a request to query and display any page in all paginated pages corresponding to the query dataset.

[0052] Correspondingly, in this embodiment, by setting up a server between the client device and the OLAP database, the server stores the query dataset, avoiding the need to query the corresponding pagination data from the OLAP database every time a pagination query request is received, thus reducing the consumption of database performance.

[0053] In this embodiment, the interface pagination query request can be a request for a portion of the query dataset (i.e., pagination data) corresponding to a specific page to be displayed on the client device's interface; for example, the user can display on the client device... Figure 1 Click the "2" in the lower right corner of the interface shown to make the client device generate and send a page 2 page 2 page 2 query request to the server, and wait for the server to return 10 data items for page 2 (i.e. page data) for page 2 display.

[0054] Correspondingly, the specific request type for interface pagination query requests can be set by the designer; for example, interface pagination query requests can be SQL requests. Figure 3 The interface pagination query request can be an SQL request (pagination SQL) that includes pageSize (page size) and pageNum (requested page). The interface pagination query request can also be in other database query languages, and this embodiment does not impose any restrictions on this.

[0055] Furthermore, the method provided in this embodiment may also include a storage procedure for querying the dataset. For example, prior to this step, the server may obtain the target data query request sent by the client device; query the Query Data Set corresponding to the target data query request from the OLAP database; divide the Query Data Set according to a preset data volume to obtain target file slices; and store the target file slices on local disk and / or in the cloud.

[0056] Step 102: Determine the pagination query file slice corresponding to the interface pagination query request in the stored target file slice; wherein, each target file slice stores data in the query dataset that is less than or equal to the preset data volume, and the pagination query file slice includes one or two target file slices.

[0057] It is understood that in this embodiment, the target file slice can be a file slice obtained by dividing the query dataset in the form of text segmentation (file slice form), that is, multiple file slices obtained by dividing the query dataset in the form of text, and each target file slice can include a part of the data in the query dataset.

[0058] Correspondingly, the specific content of the target file slices in this embodiment, i.e., the specific storage content of the query dataset, can be set by the designer according to the practical scenario and user needs. For example, the maximum storage data volume of each file slice can be a preset data volume, that is, each target file slice can store data in the query dataset that is less than or equal to the preset data volume; the data in each target file slice can be different, that is, all the data in the query dataset can be divided into each target file slice for storage. For example, each file slice can store a maximum of a preset data volume. When the preset data volume is a pre-set fixed value, if the data volume of the query dataset is divisible by the preset data volume, then each target file slice can store the preset data volume of the query dataset; if the data volume of the query dataset is not divisible by the preset data volume, then the first p target file slices can store the preset data volume of the query dataset, and the last target file slice can store q data in the query dataset, where p is the quotient of the data volume of the query dataset divided by the preset data volume, q is the remainder of the data volume of the query dataset divided by the preset data volume, and the number of target file slices is p+1. When the number of target file slices corresponding to each query dataset is a pre-set fixed value m, if the data volume of the query dataset is divisible by m, then each target file slice can store the quotient of the data volume of the query dataset and m. This embodiment does not impose any limitations on this.

[0059] In other words, the target file slice can include the stored data corresponding to the data in the query dataset. The stored data in the target file slice corresponding to each piece of data in the query dataset can include the data type of the corresponding data in the query dataset, so as to facilitate the reading and restoration of the data in the query dataset corresponding to the stored data. For example, the data header of the stored data can include the data type and data information (such as field values) of the corresponding data in the query dataset. For example, the stored data in the target file slice can be in CSV (Comma-Separated Values) format, that is, the data in the query dataset can be stored in the target file slice in CSV format. One row of CSV data (i.e., stored data) corresponds to one piece of data in the OLAP database. The field format (i.e., data type) of the OLAP database table can be written into the header (i.e., data header) of the CSV data. For example, the field value of the user's name is name, and the data type is varchar(32). The data header of the CSV data can correspond to user name &name &java.lang.String. By using the header to record the data format, the data can be restored when reading, preventing the loss of data format.

[0060] This embodiment does not limit the specific storage location of the target file fragment. For example, the target file fragment can be stored locally on the server, such as on the server's local disk; or it can be stored in the cloud on the corresponding cloud server, such as by uploading the target file fragment to a cloud-based COS (Object Storage) repository. In other words, the target file fragment can reside on the server's local disk and / or a cloud server. This embodiment does not impose any restrictions as long as the target file fragment can be stored in a server-queryable location outside the OLAP database.

[0061] Furthermore, when the target file fragments can be stored in the cloud, the server cluster can be divided into a read cluster and a write cluster. Servers in the write cluster can be used to receive data query requests (such as target data query requests), retrieve the corresponding dataset from the OLAP database, and store it in the cloud as file fragments (such as cloud COS servers). Servers in the read cluster can be used to receive interface pagination query requests, retrieve the corresponding paginated data from the cloud, and return it to the client device. That is, the server in this embodiment can be a server in the read cluster (i.e., a read server).

[0062] Correspondingly, in this embodiment, the pagination query file slice can be the target file slice containing the data (i.e., pagination data) to be queried by the interface pagination query request. For example, if the pagination data is stored in one target file slice, the pagination query file slice can include that target file slice; if the pagination data is stored in two target file slices (two adjacent target file slices), the pagination query file slice can include both target file slices. In this embodiment, before determining the pagination query file slice corresponding to the interface pagination query request in the stored target file slices, the server can also determine whether a target file slice is stored. If yes, proceed to step 102 to complete the pagination query. If no, the process can end; or a pagination query failure message can be returned to the client device; or a target data query request can be generated, the query dataset can be retrieved from the OLAP database, and the target file slice corresponding to the query dataset can be stored, before proceeding to step 102 to complete the pagination query. For example, the server can determine whether the target file fragment is stored on the local disk; if the target file fragment is stored on the local disk, it proceeds to step 102 to determine the pagination query file fragment corresponding to the interface pagination query request in the target file fragment stored locally; if the target file fragment is not stored on the local disk, it determines whether the target file fragment is stored in the cloud; if the target file fragment is stored in the cloud, it proceeds to step 102 to determine the pagination query file fragment corresponding to the interface pagination query request in the target file fragment stored in the cloud; if the target file fragment is not stored in the cloud, a target data query request is generated.

[0063] It should be noted that the specific process by which the server retrieves the pagination query file slice corresponding to the interface pagination query request from the target file slice to be stored in this step can be set by the designer according to the practical scenario and user needs. For example, the server can determine the pagination query file slice based on the page size and requested page in the interface pagination query request; for instance, if each file slice can store a maximum of a preset amount of data, and the preset amount of data is a pre-set fixed value, the server can determine the offset range of the query dataset based on the page size and requested page in the interface pagination query request; and determine the pagination query file slice based on the offset range and the preset amount of data. The server can also determine the offset range of the query dataset based on the page size and requested page in the pagination query request; determine the pagination offset range of the pagination query file pieces based on the offset range and the preset data volume; and determine the data corresponding to the pagination offset range in the pagination query file pieces as pagination data to facilitate subsequent pagination data query and reading; wherein, if the remainder when the data volume of the query dataset is divided by the preset data volume is 0, the number of target file pieces is the quotient when the data volume of the query dataset is divided by the preset data volume; if the remainder when the data volume of the query dataset is divided by the preset data volume is not 0, the number of target file pieces is the quotient when the data volume of the query dataset is divided by the preset data volume plus 1.

[0064] Correspondingly, the server determines the pagination offset range of the paginated query file slice based on the offset range and the preset data volume. This can include: determining the number identifier and pagination offset range of the paginated query file slice based on the offset range and the preset data volume; wherein, the preset data volume is an integer multiple of the page size, and the paginated query file slice includes a target file slice. For example, when the server receives a paginated query request from the interface, it can calculate the offset range based on the pageSize (page size) and pageNum (requested page) in the paginated query request; for example, if the target file slice's number identifier is a positive integer starting from 1, pageSize = 10, pageNum = 5000, then offset = 50001~50010. Assuming the maximum data size for each file segment is m = 1000 (i.e., the preset data size), the file index (i.e., the number identifier) ​​of the paging query file segment can be calculated using offset. This is calculated by dividing the minimum value in offset by m and adding 1, or dividing the maximum value in offset by m and adding 1. In other words, the paging data corresponding to pageSize = 10 and pageNum = 5000 is stored in the target file segment (i.e., the paging query file segment) with number identifier 51. The paging offset of the paging data in the paging query file segment ranges from 1 to 10. That is, the paging offset range is the remainder of the minimum value of the offset range divided by m to the remainder of the maximum value of the offset range divided by m, or offset - 50m. In other words, the paging offset range is the minimum value of the offset range minus the product value to the maximum value of the offset range minus the product value. The product value is the difference between the paging data stored in the number identifier minus 1 and the preset data size. When the preset data volume is not an integer multiple of the page size, the paginated query file slice can be a single target file slice or two consecutive target file slices; that is, when the quotient of the minimum value in offset divided by m plus 1 is different from the quotient of the maximum value in offset divided by m plus 1, the identifier of the paginated query file slice can be the identifier of the two consecutive target file slices: the quotient of the minimum value in offset divided by m plus 1 and the quotient of the maximum value in offset divided by m plus 1. This embodiment does not impose any restrictions on this.

[0065] Step 103: Retrieve the pagination data corresponding to the interface pagination query request from the pagination query file.

[0066] In this embodiment, the server can retrieve the pagination data corresponding to the interface pagination query request from the stored pagination query file slice; that is, the server can open the pagination query file slice and retrieve the pagination data from it.

[0067] Specifically, regarding the specific method by which the server retrieves the pagination data corresponding to the interface pagination query request from the pagination query file slice in this embodiment, the server can directly retrieve the pagination data from the pagination query file slice stored on the local disk or in the cloud. To reduce the situation where the same file slice is opened multiple times, the server can utilize a cache to store the content of the pagination query file slice in this embodiment; correspondingly, in this step, the server can retrieve the pagination data from the content of the cached pagination query file slice; for example, in this step, the server can determine whether the content of the pagination query file slice is stored in the server's cache; if so, the pagination data is retrieved from the content of the cached pagination query file slice; if not, the pagination data is retrieved from the pagination query file slice stored on the disk or in the cloud, and the content of the pagination query file slice is stored in the cache, such as by caching the content of the pagination query file slice using Redis (a high-performance in-memory database), to improve response speed; wherein, the target file slice is stored on the server's disk and / or in the cloud.

[0068] Correspondingly, when the target file fragment is stored in the cloud, to reduce the occurrence of the same file fragment being retrieved multiple times, the paginated query file fragment in the cloud can be stored in a cache. Accordingly, in this step, the server can retrieve pagination data from the cached paginated query file fragment. For example, in this step, the server can determine whether the paginated query file fragment is stored in the server's cache; if so, it retrieves the pagination data from the cached paginated query file fragment; if not, it retrieves the pagination data from the cloud-stored paginated query file fragment and stores the paginated query file fragment in the cache, such as by caching the paginated query file fragment using a local cache (FileCache). This embodiment does not impose any limitations on this.

[0069] Step 104: Send the pagination data to the client device so that the client device can display the pagination data.

[0070] It is understood that in this embodiment, the server can return the pagination data corresponding to the interface pagination query request to the client device, so that the client device can display the pagination data on the interface, that is, display the content of the corresponding page on the interface.

[0071] Furthermore, in this embodiment, the server can also periodically clean up the stored target file pieces to reduce the pressure on local and / or cloud storage capacity. For example, when the target file pieces are stored on the server's local disk, the server can delete the target file pieces stored on the local disk after the unqueried time of the target file pieces reaches a preset cleanup time to reduce the capacity pressure on the local disk; the server can also delete the file pieces to be cleaned stored on the local disk at a first preset time interval; wherein, the file pieces to be cleaned are file pieces (such as target file pieces) whose unqueried time has reached the preset cleanup time. When the target file pieces are stored in the cloud, the server can delete the target file pieces stored in the cloud after the unqueried time of the target file pieces reaches a preset cleanup time to reduce the capacity pressure on the cloud; the server can also delete the file pieces to be cleaned stored in the cloud at a second preset time interval; wherein, the file pieces to be cleaned are file pieces (such as target file pieces) whose unqueried time has reached the preset cleanup time; the second preset time can be greater than the first preset time.

[0072] In this embodiment, by pre-storing the query dataset corresponding to the interface pagination query request, the present invention can reduce the number of data queries in the OLAP database and reduce the performance consumption of the OLAP database pagination query. By storing the query dataset in the form of file fragments, the corresponding pagination query file fragment in the stored target file fragment can be determined according to the interface pagination query request, so as to retrieve the pagination data corresponding to the interface pagination query request in the pagination query file fragment, thereby reducing the number of data reads, improving the pagination query speed, and reducing the power consumption of the server.

[0073] Based on the above embodiments, embodiments of the present invention may further include a storage procedure for the target file fragment, to ensure that the server can use the stored target file fragment for pagination queries. For details, please refer to... Figure 4 , Figure 4 A flowchart illustrating the file slice storage process of another OLAP database pagination query method provided in an embodiment of the present invention. The method may include:

[0074] Step 201: The server obtains the target data query request sent by the client device.

[0075] It is understood that this embodiment demonstrates the ability of a server to store file fragments and perform paginated queries. For a server cluster setup where the read server performs paginated queries and the write server performs file fragment storage, the write server can use the same or similar method as the target file fragment storage process provided in this embodiment to complete the cloud storage of the file fragments. This embodiment does not impose any restrictions on this.

[0076] In this embodiment, the target data query request can be a request to retrieve a query dataset from an OLAP database. The specific request type of the target data query request, i.e., the specific request type of the data query request, can be set by the designer. For example, it can be set in a manner corresponding to the pagination query request in the interface. Figure 3 As shown, the target data query request can be an SQL request; the interface pagination query request can also be in other database query languages, and this embodiment does not impose any restrictions on this.

[0077] Step 202: Retrieve the query dataset corresponding to the target data query request from the OLAP database.

[0078] In this step, the server can retrieve the query dataset corresponding to the target data query request from the OLAP database. For example... Figure 3 The server can submit SQL requests (i.e., target data query requests) to the OLAP database to obtain the query dataset returned by the OLAP database.

[0079] Correspondingly, before this step, the server can also determine whether the local disk or cloud storage stores the query dataset corresponding to the target data query request; if so, the process ends or the corresponding query result is returned to the client device; if not, the process can proceed to step 202 to retrieve the query dataset corresponding to the target data query request from the OLAP database.

[0080] Furthermore, in this embodiment, after receiving the data query request sent by the client device, the server can first store the tasks corresponding to the data query request in the form of a queue, and then use a thread pool with a fixed number of threads to process the data query request execution and file writing tasks. That is, before this step, the server can store the target data query request in the task queue. In this step, the server can use the target thread to query the query dataset corresponding to the target data query request from the OLAP database, so that when the target data query request in the task queue needs to be processed, the target thread can be used to complete the data query and storage of the target data query request. The target thread can be any preset thread, and the preset thread can be a preset number of threads in the thread pool.

[0081] Furthermore, in this step, the server streams the query dataset corresponding to the target data query request from the OLAP database to prevent the query dataset from becoming too large and exhausting the server's memory. Specifically, the server can stream the query dataset from the OLAP database by setting the minimum number of reads for the connection to the OLAP database. For example, when the server communicates with the OLAP database via JDBC (Java Database Connectivity), the server can set the JDBC fetchSize (number of reads) to MIN_VALUE = 0x80000000 to stream the query dataset.

[0082] Step 203: Divide the query dataset into target file slices according to the preset data volume.

[0083] Understandably, in this step, the server can segment the query dataset in the form of text splitting to obtain file pieces (i.e. target file pieces) corresponding to the query dataset. The maximum data storage capacity of each file piece can be a preset data volume, that is, each target file piece can store data in the query dataset that is less than or equal to the preset data volume.

[0084] Specifically, the specific method by which the server divides the query dataset into target file pieces according to the preset data volume in this embodiment can be set by the designer according to the practical scenario and user needs. For example, it can be set to correspond to the specific content configuration method of the target file pieces in the previous embodiment to ensure that the data of the query dataset can be stored in the target file pieces. This embodiment does not impose any restrictions on this.

[0085] Step 204: Store the target file slices on local disk and / or in the cloud.

[0086] It is understood that in this embodiment, the server can store the target file fragments on a local disk and / or in the cloud (such as a cloud COS server) to store the target file fragments, which facilitates paginated querying of the query dataset corresponding to the target file fragments.

[0087] In this embodiment, the query dataset is divided into target file slices according to a preset data volume. The query dataset is stored in the form of file slices, so that pagination queries can be completed by opening a portion of the target file slices, reducing the amount of data read, thereby improving the subsequent pagination query speed and reducing the power consumption of the server.

[0088] Corresponding to the above method embodiments, this invention also provides a pagination query device for an OLAP database. The pagination query device for an OLAP database described below and the pagination query method for an OLAP database described above can be referred to in correspondence.

[0089] Please refer to Figure 5 , Figure 5 This is a structural block diagram of a pagination query device for an OLAP database provided in an embodiment of the present invention. The device is applied to a server and may include:

[0090] The pagination acquisition module 10 is used to acquire the interface pagination query request corresponding to the query dataset sent by the client device; wherein, the query dataset is the dataset corresponding to the target data query request in the OLAP database, and the target data query request is the data query request corresponding to the interface pagination query request.

[0091] The file determination module 20 is used to determine the pagination query file slice corresponding to the interface pagination query request in the stored target file slice; wherein, each target file slice stores data in the query dataset that is less than or equal to a preset data volume, and the pagination query file slice includes one or two target file slices;

[0092] The data query module 30 is used to retrieve the pagination data corresponding to the interface pagination query request from the pagination query file slice;

[0093] The data sending module 40 is used to send pagination data to the client device so that the client device can display the pagination data.

[0094] In some embodiments, the file determination module 20 may include:

[0095] The data offset determination submodule is used to determine the offset range of the query dataset based on the page size and requested page in the pagination query request. Specifically, if the remainder when the data volume of the query dataset is divided by the preset data volume is 0, the number of target file pieces is the quotient when the data volume of the query dataset is divided by the preset data volume; if the remainder when the data volume of the query dataset is divided by the preset data volume is not 0, the number of target file pieces is the quotient when the data volume of the query dataset is divided by the preset data volume plus 1.

[0096] The pagination offset determination submodule is used to determine the pagination offset range of the pagination query file slice based on the offset range and the preset data volume.

[0097] The pagination data submodule is used to determine the data corresponding to the pagination offset range in the pagination query file slice as pagination data.

[0098] In some embodiments, the pagination offset determination submodule can be specifically used to determine the number identifier of the pagination query file slice and the pagination offset range based on the offset range and the preset data volume; wherein, the preset data volume is an integer multiple of the page size, and the pagination query file slice includes a target file slice.

[0099] In some embodiments, the number identifier is the quotient of the minimum value of the offset range divided by the preset data amount plus 1, and the paging offset range is the remainder of the minimum value of the offset range divided by the preset data amount to the remainder of the maximum value of the offset range divided by the preset data amount.

[0100] In some embodiments, the data query module 30 may include:

[0101] The content cache judgment submodule is used to determine whether the content of the paginated query file slice is stored in the server's cache;

[0102] The cache query submodule is used to retrieve paginated data from the cached paginated query file content if the content of the paginated query file is stored in the server's cache.

[0103] The storage query submodule is used to retrieve the paginated data from the paginated query file slice stored on disk or in the cloud if the content of the paginated query file slice is not stored in the server's cache, and then store the content of the paginated query file slice in the cache; wherein, the target file slice is stored on the server's disk and / or in the cloud.

[0104] In some embodiments, the device may further include:

[0105] The cleanup module is used to delete target file pieces stored on the local disk after the unqueried time of the target file pieces reaches a preset cleanup time.

[0106] In some embodiments, the device may further include:

[0107] The query and acquisition module is used to acquire the target data query request sent by the client device;

[0108] The database query module is used to retrieve the query dataset corresponding to the target data query request from the OLAP database;

[0109] The data segmentation module is used to segment the query dataset according to a preset data volume to obtain target file slices;

[0110] The data storage module is used for local disk storage and / or cloud storage of target file fragments.

[0111] In some embodiments, the database query module may be specifically used to stream query datasets from an OLAP database.

[0112] In this embodiment, by pre-storing the query dataset corresponding to the interface pagination query request, the present invention can reduce the number of data queries in the OLAP database and reduce the performance consumption of the OLAP database pagination query. By storing the query dataset in the form of file fragments, the file determination module 20 can determine the corresponding pagination query file fragment in the stored target file fragment according to the interface pagination query request, so that the data query module 30 can query the pagination data corresponding to the interface pagination query request in the pagination query file fragment, reducing the number of data reads, improving the pagination query speed, and reducing the power consumption of the server.

[0113] Corresponding to the above method embodiments, this invention also provides a server. The server described below and the pagination query method for an OLAP database described above can be referred to in correspondence.

[0114] Please refer to Figure 6 , Figure 6 This is a schematic diagram of a server provided in an embodiment of the present invention. The server may include:

[0115] Memory D1 is used to store computer programs;

[0116] Processor D2 is used to execute computer programs to implement the steps of the OLAP database pagination query method provided in the above method embodiments.

[0117] For details, please refer to Figure 7 , Figure 7 This is a schematic diagram of a specific structure of a server provided in an embodiment of the present invention. The server 310 can vary significantly due to different configurations or performance. It may include one or more central processing units (CPUs) 322 (e.g., one or more processors) and a memory 332, and one or more storage media 330 (e.g., one or more mass storage devices) for storing application programs 342 or data 344. The memory 332 and storage media 330 can be temporary or persistent storage. The program stored in the storage media 330 may include one or more modules (not shown in the diagram), each module including a series of instruction operations on the data processing device. Furthermore, the central processing unit 322 may be configured to communicate with the storage media 330 and execute the series of instruction operations in the storage media 330 on the server 310.

[0118] Server 310 may also include one or more power supplies 326, one or more wired or wireless network interfaces 350, one or more input / output interfaces 358, and / or one or more operating systems 341. Examples include Windows Server™, Mac OS X™, Unix™, Linux™, FreeBSD™, etc.

[0119] The steps in the pagination query method for OLAP databases described above can be implemented by the server's architecture.

[0120] Corresponding to the above method embodiments, this invention also provides a computer-readable storage medium. The computer-readable storage medium described below corresponds to and can be referred to in relation to the pagination query method for an OLAP database described above.

[0121] A computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps of the OLAP database pagination query method provided in the above-described method embodiments.

[0122] The computer-readable storage medium can specifically be a USB flash drive, a portable hard drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk, or any other readable storage medium capable of storing program code.

[0123] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the apparatus, server, and computer-readable storage medium disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the descriptions are relatively simple; relevant details can be found in the method section.

[0124] The foregoing has provided a detailed description of the OLAP database pagination query method, apparatus, and server provided by this invention. Specific examples have been used to illustrate the principles and implementation methods of this invention. The descriptions of the embodiments above are merely for the purpose of helping to understand the method and core ideas of this invention. It should be noted that those skilled in the art can make various improvements and modifications to this invention without departing from its principles, and these improvements and modifications also fall within the protection scope of the claims of this invention.

Claims

1. A pagination query method for an OLAP database, characterized in that, include: The server obtains the interface pagination query request corresponding to the query dataset sent by the client device; wherein, the query dataset is the dataset corresponding to the target data query request in the OLAP database, and the target data query request is the data query request corresponding to the interface pagination query request; Determine the pagination query file segment corresponding to the interface pagination query request in the stored target file segment; wherein, each target file segment stores data in the query dataset that is less than or equal to a preset data volume, and the pagination query file segment includes one or two target file segments; The pagination data corresponding to the interface pagination query request is obtained from the pagination query file fragment; The pagination data is sent to the client device so that the client device displays the pagination data. The determination of the pagination query file slice corresponding to the interface pagination query request in the stored target file slice includes: Based on the page size and requested page in the interface pagination query request, the offset range of the query dataset is determined; wherein, if the remainder when the data volume of the query dataset is divided by the preset data volume is 0, then the number of target file pieces is the quotient when the data volume of the query dataset is divided by the preset data volume; if the remainder when the data volume of the query dataset is divided by the preset data volume is not 0, then the number of target file pieces is the quotient when the data volume of the query dataset is divided by the preset data volume plus 1; Based on the offset range and the preset data volume, the number identifier and pagination offset range of the paginated query file slice are determined; wherein, the preset data volume is an integer multiple of the page size, and the paginated query file slice includes a target file slice; The data corresponding to the pagination offset range in the pagination query file slice is determined as the pagination data.

2. The pagination query method for an OLAP database according to claim 1, characterized in that, The number identifier is the quotient of the minimum value of the offset range divided by the preset data amount plus 1, and the pagination offset range is the remainder of the minimum value of the offset range divided by the preset data amount to the remainder of the maximum value of the offset range divided by the preset data amount.

3. The pagination query method for an OLAP database according to claim 1, characterized in that, The step of retrieving the pagination data corresponding to the interface pagination query request from the pagination query file fragment includes: Determine whether the content of the paginated query file slice is stored in the server's cache; If so, the pagination data is retrieved from the contents of the cached pagination query file slice; If not, the pagination data is retrieved from the pagination query file slice stored on the disk or in the cloud, and the content of the pagination query file slice is stored in the cache; wherein the target file slice is stored on the server's disk and / or in the cloud.

4. The pagination query method for an OLAP database according to claim 1, characterized in that, Also includes: After the unqueried time of the target file fragment reaches a preset cleanup time, the target file fragment stored on the local disk is deleted.

5. The pagination query method for an OLAP database according to any one of claims 1 to 4, characterized in that, Before the server obtains the interface pagination query request corresponding to the query dataset sent by the client device, it also includes: The server obtains the target data query request sent by the client device; The query dataset corresponding to the target data query request is obtained from the OLAP database; The query dataset is segmented according to the preset data volume to obtain the target file fragments; The target file fragments are stored on local disks and / or in the cloud.

6. The pagination query method for an OLAP database according to claim 5, characterized in that, The step of retrieving the query dataset corresponding to the target data query request from the OLAP database includes: The query dataset is read from the OLAP database in a streaming manner.

7. A pagination query device for an OLAP database, characterized in that, Applied to servers, including: The pagination acquisition module is used to acquire the interface pagination query request corresponding to the query dataset sent by the client device; wherein, the query dataset is the dataset corresponding to the target data query request in the OLAP database, and the target data query request is the data query request corresponding to the interface pagination query request; The file determination module is used to determine the pagination query file slice corresponding to the interface pagination query request in the stored target file slice; wherein, each target file slice stores data in the query dataset that is less than or equal to a preset data volume, and the pagination query file slice includes one or two target file slices; The data query module is used to retrieve the pagination data corresponding to the interface pagination query request from the pagination query file fragment; A data sending module is used to send the pagination data to the client device so that the client device can display the pagination data; The file determination module includes: The data offset determination submodule is used to determine the offset range of the query dataset based on the page size and requested page in the interface pagination query request; wherein, if the remainder when the data volume of the query dataset is divided by the preset data volume is 0, then the number of target file pieces is the quotient when the data volume of the query dataset is divided by the preset data volume; if the remainder when the data volume of the query dataset is divided by the preset data volume is not 0, then the number of target file pieces is the quotient when the data volume of the query dataset is divided by the preset data volume plus 1; The pagination offset determination submodule is used to determine the number identifier and pagination offset range of the pagination query file slice based on the offset range and the preset data volume; wherein, the preset data volume is an integer multiple of the page size, and the pagination query file slice includes a target file slice; The pagination data submodule is used to determine the data corresponding to the pagination offset range in the pagination query file slice as the pagination data.

8. A server, characterized in that, include: Memory, used to store computer programs; A processor, configured to execute the computer program to implement the steps of the pagination query method for an OLAP database as described in any one of claims 1 to 6.

Images