Data inquiry method and device

Abstract

The invention relates to a data inquiry method and a data inquiry device. The data inquiry method comprises the following steps: receiving inquiry conditions provided by a user, wherein the inquiry conditions comprise inquiry keywords, an edit distance threshold value and a sliding window width; extracting characteristic values of the inquiry keywords to form a keyword characteristic index; extracting characteristics values of basic windows in a current sliding window to form a characteristic index of the sliding window, wherein the sliding window comprises a set number of basic windows and the characteristic index of the sliding window is a queue consisting of characteristic indexes of all the basic windows in the sliding window; when a preset inquiry trigger condition is reached, triggering inquiry for the current sliding window; and according to the keyword characteristic index and the edit distance threshold value, filtering the characteristic index of the current sliding window to obtain a candidate character string set which meets the filtering lower limit. According to the data inquiry method and the data inquiry device, which are disclosed by the invention, the defect of carrying out accurate inquiry of the keywords in a data stream scene can be effectively made up; and the data inquiry method and the data inquiry device have good inquiry fault-tolerant capability.

Description

technical field [0001] The invention relates to the field of information technology, in particular to a data query method and device. Background technique [0002] The string fuzzy query problem, also known as the string similarity query problem, has always been a research hotspot in the field of data query and processing, and has been widely used in cross-research fields, such as: in databases and data warehouses, between tables through strings Similarity connection to complete data integration and cleaning; approximate pattern matching of DNA or protein sequences in bioinformatics; search engine's "do you mean" prompt function for user input errors; spell checking and error correction of application software, etc. [0003] String similarity is measured by the calculation result of "similarity function" or "distance function". Commonly used similarity functions are: Overlap similarity, Jaccard similarity, Cosine similarity, Dice similarity, etc. Similarity functions are o...

AI Technical Summary

Problems solved by technology

[0034] Under the framework of filtering and verification, many studies are based on fixed-length q-gram feature extraction and filtering, and there are three shortcomings as follows:1. For short strings, high-quality features cannot be guaranteed to be extracted. If the value of q is small, it may cause many strings to share features and generate a large number of candidate strings; if the value of q is large, it may be filtered out strings that might resemble

2. If the value of q is small, the lower limit of the filtering threshold is higher and a larger index is introduced; 3. The support for updates is not efficient, especially when prefix filtering is used. When some data is updated, the entire IDF-based global order may change, which will inevitably lead to re-selection of features, rebuilding of indexes, etc.

[0038] The method based on Trie has the following two disadvantages: 1. The efficiency of long strings is low, because the construction process of Trie needs to compare each character in the string with the existing nodes of the Trie tree one by one, so Trie is not efficient for long strings. The insertion and query efficiency is not high

2. Trie takes a long time to preprocess and index the string set, and the index takes up a lot of space

For the data stream environment, it is obviously not feasible to first obtain all the data for indexing. Even if it is possible to build an index on a part of the data on the data stream, the size of the query and the time to build the index are also due to limited memory. And query real-time requirements, there are strict restrictions

Because if the index is too large, it may not be able to fit into the memory; if the index creation time is too long, the data in the data flow sliding window may expire before the index is created

[0051] At present, keyword queries based on data streams are mostly based on exact matching methods, and exact matching has no fault tolerance and cannot deal with errors in strings

Images