Big data retrieval method and device

Abstract

The invention discloses a big data retrieval method. The method comprises the steps that data in a data summary table corresponding to the data in a database are counted according to the data types, an index statistic table is built based on the statistical results; a page data cache list is built, the page data cache is specified as a prefix page data cache; statistical information satisfying the retrieval condition is searched out of the index statistic table; data from the data summary table satisfying the retrieval condition are filled in the page data cache list; when the space of the prefix page data cache is filled, the data and the statistical information in the prefix page data cache are returned to the retrieval client end. The invention further discloses a big data retrieval device.

Description

technical field [0001] The present invention relates to the technical field of big data, in particular to a big data retrieval method and device. Background technique [0002] The existing data retrieval method is: the client sends a request to the data server, and the data server retrieves a qualified data set and returns it to the client; the client processes the obtained data set, counts some relevant information of the data set, and then The foreground interface is displayed to the user in pages. This method is easy to manage, easy to maintain, and has a mature technical system, but the disadvantage is that if the retrieved data set is too large, such as more than 100,000 data records, then when the retrieval is displayed to the user, the retrieval time on the data server will be long It will be too long, and the amount of network data transmission is too large, which will cause a large delay in displaying to users, and users will have to wait for a long time to see the...

AI Technical Summary

Problems solved by technology

This method is easy to manage, easy to maintain, and has a mature technical system, but the disadvantage is that if the retrieved data set is too large, such as more than 100,000 data records, then when the retrieval is displayed to the user, the retrieval time on the data server will be long It will be too long, and the amount of network data transmission is too large, which will cause a large delay in displaying to users, and users will have to wait for a long time to see the search results

[0006] For the method of indexing by page number, because this method cannot retrieve the full amount of data sets each time it is retrieved, it is impossible to obtain some statistical information of the data set; taking GPRS bills as an example, if this method is used, each retrieval time It is impossible to obtain statistical information such as the total number of user GPRS entries, total traffic, and total cost; if the front-end interface requires synchronous display of these information retrievals, this method has great defects and cannot be applied; even if the cache technology is used to retrieve the full amount of results For caching, you will also face problems such as the data set is too large and the response time for the first retrieval is too long

[0007] How to overcome the problems of long retrieval response time and no statistical information in retrieval results in the retrieval of large amounts of data needs to be solved urgently

Images