Information query method, device and electronic device
Through the methods of bit vector storage and data string transformation, the problem of ultra-long data string occupying a large amount of storage space and low query efficiency in massive data storage and query is solved, and the effect of saving memory space and reducing the error judgment rate is achieved.
Patent Information
- Application Number
- CN202111239758.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2021-10-25
- Publication Date
- 2025-02-21
- Estimated Expiration
- 2041-10-25
AI Technical Summary
During the massive data storage and query process, ultra-long data strings occupy a large storage space and need to be compared in turn during query, resulting in large data processing volume and low efficiency.
Use bit vectors to store data strings, and only needs to store bit vector information with a very small amount of data to save memory space. By converting the first data string into a third data string, the difference between it and the second data string is greater than the difference between the first data string and the second data string, and based on the bit vector information without the data string and the second data string, the bit vector information and the second data string are The bit vector information of the three data strings conflicts, and it is determined that the stored first data string is queried.
It realizes saving memory space and reducing the misjudgment rate, and improves the efficiency and accuracy of information query.
Smart Images

Figure CN113886656B_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of information technology, and more specifically, to an information query method, device and electronic device. Background Art
[0002] During data storage, each number occupies one character.
[0003] When there is an extremely long data string, such as a data string containing 10, 20 or even more digits, a large storage space is required. Therefore, when there is a large amount of data transmission that needs to be stored, a large amount of storage space is required. Correspondingly, when querying the data string, it needs to be compared one by one, and the amount of data processing is large. Summary of the invention
[0004] In view of this, the present application provides an information query method as follows:
[0005] An information query method, comprising:
[0006] Obtain a first data string to be queried, wherein the first data string includes M bits of data, where M is an integer greater than 1;
[0007] Analyze and determine the first bit vector information corresponding to the first data string in the preset bit vector;
[0008] If the first bit vector information conflicts with the bit vector information of the second data string, transforming the first data string into a third data string, the second data string is different from the first data string, and the difference between the third data string and the second data string is greater than the difference between the first data string and the second data string;
[0009] The second bit vector information corresponding to the third data string in the preset bit vector is analyzed and determined, and based on the fact that the bit vector information of the non-data string conflicts with the second bit vector information of the third data string, it is determined that the first stored data string is queried.
[0010] Optionally, in the above method, the analyzing and determining the first bit vector information corresponding to the first data string includes:
[0011] Processing the first data string through a first group of hash sets to obtain at least two first target positions, where the first target positions are positions in a preset bit vector, and the first group of hash sets includes N hash functions, where the value of N is an integer greater than 1;
[0012] Based on the fact that the values of the at least two first target positions satisfy a preset condition, it is determined that the values of the at least two first target positions represent the first bit vector information corresponding to the first data string.
[0013] Optionally, in the above method, the converting the first data string into a third data string includes:
[0014] Raising the first data string to a power of K to obtain a third data string, where K is an integer greater than 1;
[0015] or
[0016] The third data string is obtained by multiplying the first data string by S, where S is an integer greater than 1.
[0017] Optionally, in the above method, the analyzing and determining the second bit vector information corresponding to the third data string includes:
[0018] Processing the third data string through a second group of hash sets to obtain at least two second target positions, where the second target positions are positions in a preset bit vector, and the second group of hash sets includes N hash functions, where the value of N is an integer greater than 1;
[0019] Based on the fact that the values of the at least two second target positions satisfy a preset condition, it is determined that the values of the at least two second target positions represent the second bit vector information corresponding to the third data string.
[0020] Optionally, the above method, presetting the bit vector, includes:
[0021] Initializing a value of each position in a bit vector of a preset length, wherein the bit vector includes at least two positions;
[0022] Obtaining a first preset data string;
[0023] transforming the first preset data string into a second preset data string;
[0024] Processing the first preset data string through a first group of hash sets to obtain at least two first positions to be updated, where the first positions to be updated are positions in the initialized bit vector, and updating the values of the at least two first positions to be updated to preset values;
[0025] The second preset data string is processed through a second group of hash sets to obtain at least two second positions to be updated, where the second positions to be updated are positions in the initialized bit vector, and the values of the at least two second positions to be updated are updated to preset values.
[0026] Optionally, the above method further includes:
[0027] Based on the conflict between the first bit vector information corresponding to the first data string and the bit vector information of the second data string, determining that a misjudgment occurs when the first data string is processed using the first group of hash sets;
[0028] Counting the preset number of processing times, whether the error rate of using the first group of hash sets to process the data string is greater than a preset threshold;
[0029] Based on the false positive rate being greater than a preset threshold, the number of hash functions included in the first group of hash sets is adjusted to W, where W is an integer greater than N.
[0030] An information query device, comprising:
[0031] An acquisition module, used for acquiring a first data string to be queried, wherein the first data string includes M bits of data, where M is an integer greater than 1;
[0032] A first analysis module, used for analyzing and determining the first bit vector information corresponding to the first data string in a preset bit vector;
[0033] a transformation module, configured to transform the first data string into a third data string if the first bit vector information conflicts with the bit vector information of the second data string, the second data string being different from the first data string, and the difference between the third data string and the second data string being greater than the difference between the first data string and the second data string;
[0034] The second analysis module is used to analyze and determine the second bit vector information corresponding to the third data string in the preset bit vector, and determine to query the stored first data string based on the conflict between the bit vector information of the non-data string and the second bit vector information of the third data string.
[0035] Optionally, the above device further includes:
[0036] A pre-storage module is used to initialize the value of each position in a bit vector of a preset length, wherein the bit vector includes at least two positions; obtain a first preset data string; transform the first preset data string into a second preset data string based on the first preset data string; process the first preset data string through a first group of hash sets to obtain at least two first positions to be updated, wherein the first positions to be updated are positions in the initialized bit vector, and the values of the at least two first positions to be updated are updated to preset values; process the second preset data string through a second group of hash sets to obtain at least two second positions to be updated, wherein the second positions to be updated are positions in the initialized bit vector, and the values of the at least two second positions to be updated are updated to preset values.
[0037] An electronic device, comprising: a memory and a processor;
[0038] Wherein, the memory stores a program;
[0039] The processor is used to load and execute the program stored in the memory to implement each step of the information query method as described in any one of the above items.
[0040] A readable storage medium stores a computer program, wherein the computer program is called and executed by a processor to implement each step of the information query method as described in any one of the above items.
[0041] It can be known from the above technical scheme that the present application provides an information query method, including: obtaining a first data string to be queried, the first data string includes multiple bits of data, analyzing and determining the first bit vector information corresponding to the first data string in a preset bit vector, when the first bit vector information conflicts with the bit vector information of the second data string, transforming the first data string into a third data string, the difference between the third data string and the second data string is greater than the difference between the first data string and the second data string; analyzing and determining the second bit vector information corresponding to the third data string in the preset bit vector, based on the conflict between the bit vector information of the non-data string and the second bit vector information of the third data string, determining that the first data string stored is queried. In this scheme, the bit vector is used to store the data string, and only the bit vector information with a small amount of data needs to be stored, which saves memory space. Moreover, the changed data string is used to expand the difference with other data strings, and bidirectional storage is performed based on the bit vectors corresponding to the data string and the changed data string, respectively, so as to realize bidirectional judgment during query and reduce the misjudgment rate. BRIEF DESCRIPTION OF THE DRAWINGS
[0042] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying any creative work.
[0043] Figure 1 A flowchart of an information query method embodiment 1 provided by the present application;
[0044] Figure 2 A flowchart of an information query method embodiment 2 provided by the present application;
[0045] Figure 3 A flowchart of Example 3 of an information query method provided by this application;
[0046] Figure 4 A flowchart of an information query method embodiment 4 provided by the present application;
[0047] Figure 5 A schematic diagram of the relationship between the value of a hash function and the misjudgment rate in Example 4 of an information query method provided by the present application;
[0048] Figure 6A flowchart of Example 5 of an information query method provided by the present application;
[0049] Figure 7 A schematic diagram of an application scenario of an information query method provided in this application;
[0050] Figure 8 A schematic diagram of a barcode in an application scenario of an information query method provided in this application;
[0051] Fig. 9 A schematic diagram of the process of data processing performed by a data processing center in an application scenario of an information query method provided in this application;
[0052] Fig.10 A schematic diagram of the structure of an information query device embodiment provided in this application. DETAILED DESCRIPTION
[0053] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.
[0054] like Figure 1 The figure is a flow chart of an information query method embodiment 1 provided by the present application. The method is applied to an electronic device and includes the following steps:
[0055] Step S101: obtaining a first data string to be queried;
[0056] The first data string includes M bits of data, where M is an integer greater than 1.
[0057] The first data string may be obtained from other devices, and the other devices may specifically be devices for reading data strings, such as a barcode scanner or a card reader.
[0058] Specifically, the device for reading the data string reads the data string carried in the barcode or card.
[0059] The first data string may include various information, such as cargo information, employee information or other information.
[0060] In a specific implementation, the first data string may include multiple bits of data, such as 5, 10, 11 or even more bits of data. The present application does not impose any limitation on the number of data bits included in the first data string.
[0061] Step S102: analyzing and determining the first bit vector information corresponding to the first data string in the preset bit vector;
[0062] There is a preset bit vector, which is a one-dimensional bit vector, and a one-dimensional array. There are only two values, 0 and 1, in the array, and the required storage space is small.
[0063] The bit vector information of each data string is stored in the bit vector.
[0064] Specifically, it is analyzed to determine whether the first data string has corresponding bit vector information in the preset bit vector.
[0065] If yes, it is determined that the first bit vector information corresponding to the first data string exists in the preset bit vector.
[0066] Specifically, if the first bit vector information corresponding to the first data string is stored in the preset bit vector, it is represented that it is preliminarily determined that the first data string is stored.
[0067] In a specific implementation, a Bloom filter may be used to analyze and process the first data string to obtain a corresponding first bit vector.
[0068] Step S103: if the first bit vector information conflicts with the bit vector information of the second data string, transforming the first data string into a third data string;
[0069] The second data string is different from the first data string, and the difference between the third data string and the second data string is greater than the difference between the first data string and the second data string.
[0070] Specifically, since there is a second data string similar to the first data string, there will be a bit vector information conflict between the first data string and the second data string, that is, in the preset bit vector, the first bit vector information corresponding to the first data string is the same as the bit vector information corresponding to the second data string, and the two conflict, which indicates that an error occurs in the query result based only on the first data string, and therefore, further query is needed.
[0071] Specifically, the first data string is transformed into a third data string to increase the difference with the second data string.
[0072] The step of transforming the first data string into a third data string includes:
[0073] Raising the first data string to a power of K to obtain a third data string, where K is an integer greater than 1;
[0074] or
[0075] The third data string is obtained by multiplying the first data string by S, where S is an integer greater than 1.
[0076] Specifically, the data string is transformed into the third data string by raising it to multiple powers or increasing its multiples.
[0077] For example, the first data string is 1234, and the second data string is 1235. The difference between the two is very small. Then, the third data string obtained by squaring the first data string is 1522756, which is very different from the second data string 1235.
[0078] For example, the first data string is 1234, the second data string is 1235, and the difference between the two is very small. Then, the third data string obtained by multiplying the first data string by 2 is 2468, and the difference between the third data string and the second data string 1235 also becomes larger.
[0079] Specifically, if the bit vector information obtained by processing the data string using the first set of hash sets does not conflict with the bit vector information of other data strings, it is determined that the first stored data string is queried, and there is no need to perform subsequent steps S103-104, that is, the result obtained by processing the first data string using the first set of hash sets does not have an error, and no further query is required.
[0080] Step S104: Analyze and determine the second bit vector information corresponding to the third data string in the preset bit vector, and determine to query the stored first data string based on the conflict between the bit vector information of the non-data string and the second bit vector information of the third data string.
[0081] Specifically, it is analyzed to determine whether the third data string has corresponding bit vector information in the preset bit vector.
[0082] If yes, it is determined that second bit vector information corresponding to the third data string exists in the preset bit vector.
[0083] Specifically, if the second bit vector information corresponding to the third data string is stored in the preset bit vector, and no other data string conflicts with the third data string, it indicates that the first data string is stored.
[0084] It should be noted that the information query method in the embodiment of the present application only needs to query whether there is corresponding bit vector information from the one-dimensional bit vector. The time complexity of the search is one-dimensional, and the complexity is relatively low. Moreover, by expanding the difference in data strings, bidirectional storage is performed based on the bit vectors corresponding to the data string and the changed data string, thereby realizing bidirectional judgment during query and reducing the misjudgment rate.
[0085] In summary, the present embodiment provides an information query method, including: obtaining a first data string to be queried, the first data string including multiple bits of data, analyzing and determining the first bit vector information corresponding to the first data string in a preset bit vector, when the first bit vector information conflicts with the bit vector information of the second data string, transforming the first data string into a third data string, the difference between the third data string and the second data string being greater than the difference between the first data string and the second data string; analyzing and determining the second bit vector information corresponding to the third data string in a preset bit vector, and determining that the first stored data string is queried based on the conflict between the bit vector information of the non-data string and the second bit vector information of the third data string. In this scheme, the bit vector is used to store the data string, and only the bit vector information with a small amount of data needs to be stored, saving memory space. Moreover, the changed data string is used to expand the difference with other data strings, and bidirectional storage is performed based on the bit vectors corresponding to the data string and the changed data string, respectively, so as to realize bidirectional judgment during query and reduce the misjudgment rate.
[0086] like Figure 2 The figure is a flow chart of Example 2 of an information query method provided by the present application, and the method comprises the following steps:
[0087] Step S201: obtaining a first data string to be queried;
[0088] Among them, step S201 is consistent with step S101 in embodiment 1, and is not described in detail in this embodiment.
[0089] Step S202: Processing the first data string through a first set of hash sets to obtain at least two first target locations;
[0090] The first target position is a position in a preset position vector.
[0091] The first group of hash sets includes N hash functions, and the value of N is an integer greater than 1.
[0092] The preset bit vector is specifically a hash table (Hash table, also called hash table).
[0093] In this embodiment, multiple hash functions are used to process the first data string in sequence, and the calculated positions correspond to the positions in the hash table.
[0094] Specifically, a hash table contains multiple locations, and the value in each location is 1 or 0.
[0095] The first data string is processed by a first set of hash sets to obtain a plurality of first target positions, which are corresponding positions in the hash table.
[0096] It should be noted that the number of hash functions included in the first hash set is adjustable.
[0097] It should be noted that the smaller the value of the data string, the closer to the front the corresponding target position in the preset bit vector is, and the larger the value of the data string, the closer to the back the corresponding target position in the preset bit vector is. Correspondingly, the first target position is located in the first several bits of the preset bit vector.
[0098] Step S203: determining that the values of the at least two first target positions represent first bit vector information corresponding to the first data string based on that the values of the at least two first target positions meet a preset condition;
[0099] If the first data string is stored in the preset bit vector, the values in at least two first target positions in the preset bit vector are 1.
[0100] Wherein, based on the fact that the values of the at least two first target positions satisfy a preset condition, that is, the values of the at least two first target positions are both 1, it is determined that the first bit vector information corresponding to the first data string is stored in the preset bit vector.
[0101] For example, the first data string is 1234, and after being processed by the first group of hash sets, the target positions obtained are n, n+1, n+10 and n+12, where n is an integer greater than or equal to 1, and the values of the target positions are all 1, and it is determined that the multiple first target positions are the first bit vector information corresponding to the first data string.
[0102] Step S204: if the first bit vector information conflicts with the bit vector information of the second data string, transforming the first data string into a third data string;
[0103] Step S205: Analyze and determine the second bit vector information corresponding to the third data string in the preset bit vector, and determine to query the stored first data string based on the conflict between the bit vector information of the non-data string and the second bit vector information of the third data string.
[0104] Among them, steps S204-205 are consistent with steps S103-104 in Example 1 and are not described in detail in this embodiment.
[0105] In summary, in an information query method provided by this embodiment, a first data string is processed by a first hash set composed of multiple hash functions to obtain multiple first target positions, the first target position is a position in the preset bit vector, and based on the value of the position corresponding to the first target position in the preset bit vector meeting the condition, it is preliminarily determined that the first data string is stored, and the first data string is found. Using a bit vector to store a data string only requires storing bit vector information with a very small amount of data, saving memory space, and only requires multiple hash functions for processing, utilizing the high-speed characteristics of the Bloom filter to achieve fast query of the first data string.
[0106] like Figure 3 The figure is a flowchart of Embodiment 3 of an information query method provided by the present application, and the method comprises the following steps:
[0107] Step S301: obtaining a first data string to be queried;
[0108] Step S302: analyzing and determining the first bit vector information corresponding to the first data string in the preset bit vector;
[0109] Step S303: if the first bit vector information conflicts with the bit vector information of the second data string, transforming the first data string into a third data string;
[0110] Among them, steps S301-303 are consistent with steps S101-103 in Example 1, and are not described in detail in this embodiment.
[0111] Step S304: performing a second set of hash processing on the third data string to obtain at least two second target positions;
[0112] Wherein, the second target position is a position in a preset bit vector;
[0113] The second group of hash processing includes N hash functions, and the value of N is an integer greater than 1.
[0114] In the theoretical analysis of Bloom filters, it is always assumed that the hash function is perfect and uniformly distributed. However, in reality, perfect hash functions do not exist. Whether the hash function is based on addition, g-bit multiplication, or shift, there may be some clustering of hash values, which will lead to more conflicts, especially when there are more approximate values, conflicts are more likely to occur.
[0115] Specifically, the classic hash function is as follows
[0116]
[0117] The above is a classic hash function, which is based on the shift hash function.
[0118] Because the gap between the low and high bits is caused by the difference, and the low and high bits are discarded during the shifting process, which will inevitably lead to a loss of precision.
[0119] For similar elements, the difference between them is small, and the difference bits between similar elements may be discarded by the shift operation, which causes the difference between similar elements to be ignored. So this is a source of false positives in the actual use of Bloom filters. The more similar elements are input, the greater the possibility of false positives.
[0120] Therefore, in this solution, after a conflict occurs, the difference is enlarged to achieve another analysis and determination, so as to improve the accuracy of the final query result.
[0121] Correspondingly, in this embodiment, multiple hash functions are used to process the third data string in sequence, and the calculated positions are corresponding positions in the hash table.
[0122] Specifically, a hash table contains multiple locations, and the value in each location is 1 or 0.
[0123] The third data string is processed by the second set of hash sets to obtain multiple second target positions, which are corresponding positions in the hash table.
[0124] It should be noted that the number of hash functions included in the second group of hash sets is adjustable, and the number of hash functions included in the second group of hash sets is the same as that in the first group of hash sets.
[0125] It should be noted that the smaller the value of the data string, the closer to the front the corresponding target position in the preset bit vector, and the larger the value of the data string, the closer to the back the corresponding target position in the preset bit vector. Accordingly, the first target position is closer to the front of the preset bit vector relative to the second target position.
[0126] The present application improves the existing Bloom filter by adding differential amplification and rehashing to increase the accuracy of query processing.
[0127] Step S305: based on the values of the at least two second target positions satisfying a preset condition, determining that the values of the at least two second target positions represent second bit vector information corresponding to the third data string;
[0128] If the third data string is stored in the preset bit vector, the values in at least two second target positions in the preset bit vector are 1.
[0129] Wherein, based on the fact that the values of the at least two second target positions satisfy a preset condition, that is, the values of the at least two second target positions are both 1, it is determined that the preset bit vector stores the second bit vector information corresponding to the third data string.
[0130] For example, the third data string is 2468, and after being processed by the second set of hash sets, the target positions obtained are p, p+5, p+8 and p+21, where p is an integer greater than or equal to 1, and the values of p and n are different, and the values of the target positions are all 1, and it is determined that the multiple second target positions are the second bit vector information corresponding to the third data string.
[0131] Step S306: Based on the conflict between the bit vector information of the non-data string and the second bit vector information of the third data string, it is determined that the first stored data string is queried.
[0132] In summary, in an information query method provided by this embodiment, when the first bit vector information corresponds to the first data string, and the first bit vector information has the bit vector information of the second data string that conflicts with it, the third data string obtained by further converting the first data string is processed based on multiple hash functions to obtain the second bit vector information, and it is analyzed to determine that there is no second bit vector information that conflicts with it, thereby realizing two-way judgment, improving the accuracy of the query, and reducing the misjudgment rate.
[0133] like Figure 4 The figure is a flowchart of Example 4 of an information query method provided by the present application, and the method comprises the following steps:
[0134] Step S401: obtaining a first data string to be queried;
[0135] Step S402: Processing the first data string through a first set of hash sets to obtain at least two first target locations;
[0136] Step S403: based on the values of the at least two first target positions satisfying a preset condition, determining that the values of the at least two first target positions represent first bit vector information corresponding to the first data string;
[0137] Step S404: if the first bit vector information conflicts with the bit vector information of the second data string, transforming the first data string into a third data string;
[0138] Step S405: analyzing and determining the second bit vector information corresponding to the third data string in the preset bit vector, and determining that the first data string is queried and stored based on the conflict between the bit vector information of the non-data string and the second bit vector information of the third data string;
[0139] Among them, steps S401-405 are consistent with steps S201-205 in Example 2, and are not repeated in this embodiment.
[0140] Step S406: determining that a misjudgment occurs when the first group of hash sets is used to process the first data string based on a conflict between the first bit vector information corresponding to the first data string and the bit vector information of the second data string;
[0141] Among them, if the first bit vector information corresponding to the first data string conflicts with the bit vector information of other data strings, and the same set of bit vector information corresponds to multiple data strings, a hash conflict occurs. It can be known that the use of the first set of hash sets to process the first data string has resulted in a misjudgment.
[0142] Step S407: Counting whether the error rate of processing the data string using the first set of hash sets is greater than a preset threshold value during the preset processing times;
[0143] When querying each data string, statistics are collected on the query process.
[0144] Specifically, if the bit vector information obtained by using the first group of hash sets to process the data string does not conflict with the bit vector information of other data strings, there is no need to perform the subsequent process of changing the first data string into a third data string and re-determining the corresponding second bit vector information. That is, if the result of using the first group of hash sets to process the data string does not result in a hash conflict, there is no need for further query, and it can be recorded at this time that no misjudgment has occurred.
[0145] Specifically, in the process of continuously querying the data string for several times, the number of misjudgments of the data string processed by the first group of hash sets is counted, and the misjudgment rate of the data string processed by the first group of hash sets is calculated.
[0146] The preset threshold value may be set according to circumstances, such as 40%, 50%, etc.
[0147] Step S408: Based on the false positive rate being greater than a preset threshold, the number of hash functions included in the first group of hash sets is adjusted to W, where W is an integer greater than N.
[0148] If the error rate of the data processed by the first group of hash sets is greater than a preset threshold, it indicates that the processing accuracy of the hash functions in the current first group of hash sets is low. In this case, the number of hash functions included in the first group of hash sets is increased.
[0149] Specifically, the number of hash functions in the second hash set is the same as the number of hash functions in the first hash set. When the number of hash functions included in the first hash set is adjusted, the number of hash functions in the second hash set can be adjusted accordingly.
[0150] It should be noted that the more hash functions there are in a hash set, the higher the accuracy of the processing will not be.
[0151] like Figure 5 The figure shows the relationship between the value of the hash function and the false positive rate. In the figure, the vertical axis represents the false positive rate, the horizontal axis represents the average value of the data string carried by each bit in the hash table, and the four lines represent the values of the number K of the hash function as 4, 8, 12 and 32 respectively. Figure 5 It can be seen that as the mean value of the data string carried by each bit in the hash table increases, the false positive rate increases, and as the number of hash functions increases, the false positive rate gradually decreases. However, the decrease in the false positive rate is not linearly related to the increase in the number of hash functions. Figure 5 In the , the error rates of using 16 functions and 32 functions are very close.
[0152] In summary, in an information query method provided by this embodiment, when the error rate of the result obtained by processing the data string based on the first hash set is high, the error rate is reduced by increasing the number of hash functions in the first hash set.
[0153] like Figure 6 The figure is a flowchart of Example 5 of an information query method provided by the present application, and the method comprises the following steps:
[0154] Step S601: Initialize the value of each position in a bit vector of a preset length;
[0155] The bit vector includes at least two positions.
[0156] The length of the bit vector may be a larger value, such as 30, 50 or even more.
[0157] Specifically, the default value in the initialized bit vector is 0, that is, the value of each position is 0.
[0158] In a specific implementation, the length of the bit vector may be greater than or equal to the sum of the positions obtained by processing the two sets of hash sets. For example, if one set of positions obtained by processing the hash sets involves 10, then the length of the bit vector is greater than or equal to 20.
[0159] In order to ensure that after each hash set adjusts the number of hash functions, the bit vector can carry the bit vector information corresponding to the data string, the length of the bit vector is greater than the sum of the positions obtained by processing the two hash sets.
[0160] Step S602: Obtain a first preset data string;
[0161] The first preset data string is a data string to be stored.
[0162] In a specific implementation, the process steps S602-604 are the process of storing data strings. The steps S602-604 can be executed in a loop, and the bit vector of the preset length can carry multiple data strings.
[0163] like Figure 5 As described in, a bit vector of a preset length can carry 10, 20 ... or even 70 or more data strings.
[0164] Step S603: transforming the first preset data string into a second preset data string;
[0165] Wherein, based on changing the first preset data string into the second preset data string, the difference between the preset data string and other data strings is enlarged.
[0166] Specifically, the transformation process can be: raising the first preset data string to the power of K to obtain the second preset data string, where K is an integer greater than 1; or multiplying the first preset data string by S to obtain the second preset data string, where S is an integer greater than 1.
[0167] Step S604: performing a first set of hash set processing on the first preset data string to obtain at least two first positions to be updated, and updating the values of the at least two first positions to be updated to preset values;
[0168] The first position to be updated is a position in the initialized bit vector.
[0169] Specifically, the first preset data string is processed based on the first group of hash sets to obtain multiple first positions to be updated, and the values of the first positions to be updated are updated to the preset value 1.
[0170] Step S605: performing a second set of hash processing on the second preset data string to obtain at least two second positions to be updated, and updating the values of the at least two second positions to be updated to preset values;
[0171] The second position to be updated is a position in the initialized bit vector.
[0172] Specifically, the second preset data string is processed based on the second group of hash sets to obtain multiple second positions to be updated, and the values of the second positions to be updated are updated to the preset value 1.
[0173] It should be noted that, when the value of the first position to be updated is 1 before the update, the value of the position is updated to 1 or is not updated so that the value of the first position to be updated is 1. In this case, the bit vector carries multiple data strings.
[0174] Step S606: Obtain the first data string to be queried;
[0175] Step S607: analyzing and determining the first bit vector information corresponding to the first data string in the preset bit vector;
[0176] Step S608: if the first bit vector information conflicts with the bit vector information of the second data string, transforming the first data string into a third data string;
[0177] Step S609: Analyze and determine the second bit vector information corresponding to the third data string in the preset bit vector, and determine to query the stored first data string based on the conflict between the bit vector information of the non-data string and the second bit vector information of the third data string.
[0178] Among them, steps S606-609 are consistent with steps S101-104 in Example 1 and are not described in detail in this embodiment.
[0179] In summary, in an information query method provided by this embodiment, a specific process of presetting a bit vector is also provided, including: initializing the value of each position in a bit vector of a preset length, transforming a first preset data string into a second preset data string, processing the first preset data string through a first group of hash sets to obtain at least two first positions to be updated, and updating the values of at least two first positions to be updated in the preset bit vector to preset values; processing the second preset data string through a second group of hash sets to obtain at least two second positions to be updated, and updating the values of at least two second positions to be updated in the preset bit vector to preset values. This process implements the process of storing the first preset data string. Using bit vectors to store data strings only requires storing bit vector information with a very small amount of data, saving memory space
[0180] Corresponding to the above-mentioned embodiment of an information query method provided by the present application, the present application also provides an example of an application scenario for applying the information query method.
[0181] In this scenario, barcode scanning is used to store and query data.
[0182] like Figure 7 The figure shows an application scenario diagram of the information query method, specifically a barcode processing scenario, which includes a data processing center and a query center. The barcode is pre-entered in the data processing center and stored in the form of a bit vector. The query center (such as a barcode scanner) scans the barcode to obtain a scanned barcode to be detected, which is composed of a string of numbers. The query center sends the data string to the data processing center for query. The data processing center queries whether the scanned barcode to be detected is stored and returns the query result. If the barcode is stored, the subsequent steps are executed. If the barcode is not stored, the query ends.
[0183] like Figure 8The barcode diagram is shown in FIG. 1 . The barcode contains multiple bars of different widths, each representing a different number. Figure 8 The data string 690021904622454016 corresponding to the barcode is shown in FIG.
[0184] like Fig. 9 The figure shows a schematic diagram of the process of data processing by the data processing center. Fig. 9 The upper part represents the process of data storage, and the lower part represents the process of data query.
[0185] The process of storing data is as follows: xi represents the barcode element to be stored (i.e., the data string), which is hashed by hf1(xi), hf2(xi)…hfn(xi) to obtain the corresponding position in the bit vector, and the value of the corresponding position is updated to 1, and the data string is hashed by the function f(xi)=x * (take * power), after hd1(x * )、hd2(x * )…hdn(x * ) gets the position in the corresponding bit vector and updates the value of the corresponding position to 1.
[0186] The process of querying data: xj represents the barcode element to be queried (i.e., data string), and the data string is hashed by hf1(xj), hf2(xj)…hfn(xj) to obtain the position in the corresponding bit vector. The value of the position in the corresponding bit vector is 1, indicating that xj is stored. Further, the data string is hashed by the function f(xj)=xj. * (take * power), after hd1(xj * )、hd2(xj * )…hdn(xj * ) obtains the position in the corresponding bit vector, and based on the value of the position in the corresponding bit vector being 1, determines that xj is stored.
[0187] Corresponding to the above-mentioned information query method embodiment provided by the present application, the present application also provides an apparatus embodiment applying the information query method.
[0188] like Fig.10 The structure diagram of an information query device embodiment provided by the present application is shown, and the device includes the following structures: an acquisition module 1001, a first analysis module 1002, a transformation module 1003 and a second analysis module 1004;
[0189] The acquisition module 1001 is used to acquire a first data string to be queried, wherein the first data string includes M bits of data, where M is an integer greater than 1;
[0190] The first analysis module 1002 is used to analyze and determine the first bit vector information corresponding to the first data string in the preset bit vector;
[0191] The transformation module 1003 is used for transforming the first data string into a third data string if the first bit vector information conflicts with the bit vector information of the second data string, the second data string is different from the first data string, and the difference between the third data string and the second data string is greater than the difference between the first data string and the second data string;
[0192] Among them, the second analysis module 1004 is used to analyze and determine the second bit vector information corresponding to the third data string in the preset bit vector, and determine to query the stored first data string based on the conflict between the bit vector information of the non-data string and the second bit vector information of the third data string.
[0193] Optionally, also include:
[0194] A pre-storage module is used to initialize the value of each position in a bit vector of a preset length, wherein the bit vector includes at least two positions; obtain a first preset data string; transform the first preset data string into a second preset data string based on the first preset data string; process the first preset data string through a first group of hash sets to obtain at least two first positions to be updated, wherein the first positions to be updated are positions in the initialized bit vector, and the values of the at least two first positions to be updated are updated to preset values; process the second preset data string through a second group of hash sets to obtain at least two second positions to be updated, wherein the second positions to be updated are positions in the initialized bit vector, and the values of the at least two second positions to be updated are updated to preset values.
[0195] Optionally, the first analysis module is specifically used for:
[0196] Processing the first data string through a first group of hash sets to obtain at least two first target positions, where the first target positions are positions in a preset bit vector, and the first group of hash sets includes N hash functions, where the value of N is an integer greater than 1;
[0197] Based on the fact that the values of the at least two first target positions satisfy a preset condition, it is determined that the values of the at least two first target positions represent the first bit vector information corresponding to the first data string.
[0198] Transformation module, specifically used for:
[0199] Raising the first data string to a power of K to obtain a third data string, where K is an integer greater than 1;
[0200] or
[0201] The third data string is obtained by multiplying the first data string by S, where S is an integer greater than 1.
[0202] Optionally, the second analysis module is specifically used for:
[0203] Processing the third data string through a second group of hash sets to obtain at least two second target positions, where the second target positions are positions in a preset bit vector, and the second group of hash sets includes N hash functions, where the value of N is an integer greater than 1;
[0204] Based on the fact that the values of the at least two second target positions satisfy a preset condition, it is determined that the values of the at least two second target positions represent the second bit vector information corresponding to the third data string.
[0205] Optionally, also include:
[0206] An adjustment module is used to determine whether an error occurs when using the first group of hash sets to process the first data string based on a conflict between the first bit vector information corresponding to the first data string and the bit vector information of the second data string; whether the error rate of using the first group of hash sets to process the data string is greater than a preset threshold value in a preset number of processing times; and based on the error rate being greater than the preset threshold value, adjust the number of hash functions included in the first group of hash sets to W, where W is an integer greater than N.
[0207] In summary, the present application provides an information query device that uses bit vectors to store data strings. It only needs to store bit vector information with a very small amount of data, saving memory space. Moreover, the changed data string is used to expand the difference with other data strings, and bidirectional storage is performed based on the bit vectors corresponding to the data string and the changed data string, respectively, to achieve bidirectional judgment during query and reduce the misjudgment rate.
[0208] Corresponding to the above-mentioned embodiment of an information query method provided by the present application, the present application also provides an electronic device and a readable storage medium corresponding to the information query method.
[0209] The electronic device includes: a memory and a processor;
[0210] Wherein, the memory stores a program;
[0211] The processor is used to load and execute the program stored in the memory to implement each step of the information query method as described in any one of the above items.
[0212] For the specific implementation and control method of the electronic device, please refer to the aforementioned information query method embodiment.
[0213] The readable storage medium stores a computer program thereon, and the computer program is called and executed by a processor to implement each step of the information query method as described in any one of the above claims.
[0214] Specifically, the computer program stored in the readable storage medium executes the information query method, and reference may be made to the aforementioned information query method embodiment.
[0215] In this specification, each embodiment is described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the embodiments can be referred to each other. For the device provided in the embodiment, since it corresponds to the method provided in the embodiment, the description is relatively simple, and the relevant parts can be referred to the method part.
[0216] The above description of the provided embodiments enables professionals and technicians in the field to implement or use the present application. Various modifications to these embodiments will be apparent to professionals and technicians in the field, and the general principles defined herein can be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application will not be limited to the embodiments shown herein, but will conform to the widest range consistent with the principles and novel features provided herein.
Claims
1. An information query method, comprising: Obtain a first data string to be queried, wherein the first data string includes M bits of data, where M is an integer greater than 1; Analyze and determine the first bit vector information corresponding to the first data string in the preset bit vector; If the first bit vector information conflicts with the bit vector information of the second data string, transforming the first data string into a third data string, the second data string is different from the first data string, and the difference between the third data string and the second data string is greater than the difference between the first data string and the second data string; Analyze and determine the second bit vector information corresponding to the third data string in the preset bit vector, and determine to query the stored first data string based on the conflict between the bit vector information of the non-data string and the second bit vector information of the third data string; The analyzing and determining the first bit vector information corresponding to the first data string includes: Processing the first data string through a first group of hash sets to obtain at least two first target positions, where the first target positions are positions in a preset bit vector, and the first group of hash sets includes N hash functions, where the value of N is an integer greater than 1; Based on the values of the at least two first target positions satisfying a preset condition, determining that the values of the at least two first target positions represent first bit vector information corresponding to the first data string; The analyzing and determining the second bit vector information corresponding to the third data string includes: Processing the third data string through a second group of hash sets to obtain at least two second target positions, where the second target positions are positions in a preset bit vector, and the second group of hash sets includes N hash functions, where the value of N is an integer greater than 1; Based on the fact that the values of the at least two second target positions satisfy a preset condition, it is determined that the values of the at least two second target positions represent the second bit vector information corresponding to the third data string.
2. The method according to claim 1, wherein the step of transforming the first data string into a third data string comprises: Raising the first data string to a power of K to obtain a third data string, where K is an integer greater than 1; or The third data string is obtained by multiplying the first data string by S, where S is an integer greater than 1.
3. The method according to claim 1, presetting the bit vector comprises: Initializing a value of each position in a bit vector of a preset length, wherein the bit vector includes at least two positions; Obtaining a first preset data string; transforming the first preset data string into a second preset data string; Processing the first preset data string through a first group of hash sets to obtain at least two first positions to be updated, where the first positions to be updated are positions in the initialized bit vector, and updating the values of the at least two first positions to be updated to preset values; The second preset data string is processed through a second group of hash sets to obtain at least two second positions to be updated, where the second positions to be updated are positions in the initialized bit vector, and the values of the at least two second positions to be updated are updated to preset values.
4. The method according to claim 1, further comprising: Based on the conflict between the first bit vector information corresponding to the first data string and the bit vector information of the second data string, determining that a misjudgment occurs when the first data string is processed using the first group of hash sets; Counting the preset number of processing times, whether the error rate of using the first group of hash sets to process the data string is greater than a preset threshold; Based on the false positive rate being greater than a preset threshold, the number of hash functions included in the first group of hash sets is adjusted to W, where W is an integer greater than N.
5. An information query device, comprising: An acquisition module, used for acquiring a first data string to be queried, wherein the first data string includes M bits of data, where M is an integer greater than 1; A first analysis module, used for analyzing and determining the first bit vector information corresponding to the first data string in a preset bit vector; a transformation module, configured to transform the first data string into a third data string if the first bit vector information conflicts with the bit vector information of the second data string, the second data string being different from the first data string, and the difference between the third data string and the second data string being greater than the difference between the first data string and the second data string; A second analysis module, configured to analyze and determine second bit vector information corresponding to the third data string in a preset bit vector, and determine to query the stored first data string based on the conflict between the bit vector information of the non-data string and the second bit vector information of the third data string; The first analysis module analyzes and determines the first bit vector information corresponding to the first data string, including: Processing the first data string through a first group of hash sets to obtain at least two first target positions, where the first target positions are positions in a preset bit vector, and the first group of hash sets includes N hash functions, where the value of N is an integer greater than 1; Based on the values of the at least two first target positions satisfying a preset condition, determining that the values of the at least two first target positions represent first bit vector information corresponding to the first data string; The second analysis module analyzes and determines second bit vector information corresponding to the third data string, including: Processing the third data string through a second group of hash sets to obtain at least two second target positions, where the second target positions are positions in a preset bit vector, and the second group of hash sets includes N hash functions, where the value of N is an integer greater than 1; Based on the fact that the values of the at least two second target positions satisfy a preset condition, it is determined that the values of the at least two second target positions represent the second bit vector information corresponding to the third data string.
6. The apparatus according to claim 5, further comprising: A pre-storage module, used to initialize the value of each position in a bit vector of a preset length, wherein the bit vector includes at least two positions; Obtaining a first preset data string; Based on the first preset data string, the second preset data string is transformed; the first preset data string is processed by a first group of hash sets to obtain at least two first positions to be updated, the first positions to be updated are positions in the initialized bit vector, and the values of the at least two first positions to be updated are updated to preset values; the second preset data string is processed by a second group of hash sets to obtain at least two second positions to be updated, the second positions to be updated are positions in the initialized bit vector, and the values of the at least two second positions to be updated are updated to preset values.
7. An electronic device comprising: Memory, processor; Wherein, the memory stores a program; The processor is used to load and execute the program stored in the memory to implement each step of the information query method according to any one of claims 1 to 4.
8. A readable storage medium having a computer program stored thereon, wherein the computer program is called and executed by a processor to implement the steps of the information query method according to any one of claims 1 to 4.
Citation Information
Patent Citations
Two-dimensional filter generation method, query method and device
CN104317795A
Hash search method for reducing hash collision
CN104504038A