Data query method and related device
By using multiple Bloom filters to verify data query requests in the server, the resource consumption and server crash problems caused by a large number of invalid query requests are solved, and efficient query filtering and resource management are achieved.
Patent Information
- Application Number
- CN202311590662.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2023-11-24
- Publication Date
- 2025-05-27
AI Technical Summary
A large number of invalid query requests lead to increased server resource consumption and server crash.
By receiving the data query request, the target Bloom filter for verification is determined from the multiple Bloom filters according to the target indication information, perform verification and reject invalid requests to avoid cache penetration.
Effectively filter invalid query requests to avoid increased resource consumption and server crashes, while improving query efficiency.
Smart Images

Figure CN120045585A_ABST
Abstract
Description
Technical Field
[0001] This application relates to the technical field of data processing, and in particular, to a data query method and related devices. Background Art
[0002] Currently, in order for a client to obtain data stored in a server's database, it generally sends a query request to the server. After receiving the query request, the server verifies the query request. If the verification result indicates that the query request is valid, it queries the required data according to the query conditions indicated by the query request and sends the data to the client.
[0003] When the server stores data, it stores the data stored in the database in the cache to reduce the pressure on the database and improve the performance of the database. Among them, when the required data cannot be found in the cache according to the query request, it is necessary to directly obtain the required data from the database.
[0004] However, when the server receives a large number of invalid query requests, on the one hand, it will occupy the processing power and network bandwidth of the server, resulting in increased resource consumption; on the other hand, when a large number of invalid query requests cannot find the required data in the cache, it will "penetrate" the cache and query the database. That is, due to the server receiving a large number of invalid requests, cache penetration may occur, resulting in server crashes. Summary of the Invention
[0005] Embodiments of this application provide a data query method and related devices, aiming to solve the problems of increased server resource consumption and server crashes caused by a large number of invalid query requests.
[0006] The first aspect of this application provides a data query method, which includes:
[0007] Receiving a data query request; the data query request includes target indication information for querying data to be obtained;
[0008] Determining a target Bloom filter for verifying the data query request from multiple Bloom filters according to the target indication information in the data query request; the multiple Bloom filters respectively correspond to different data sets; the Bloom filter is used to represent the data included in the corresponding data set;
[0009] Verifying the data query request through the target Bloom filter to obtain a verification result; the verification result indicates that the data query request is valid or invalid;
[0010] If the verification result indicates that the data query request is valid, query the data to be obtained according to the data query request;
[0011] If the verification result indicates that the data query request is invalid, reject the data query request.
[0012] The second aspect of the present application provides a data query device, which includes:
[0013] A receiving module, configured to receive a data query request; the data query request includes target indication information for querying data to be obtained;
[0014] A target Bloom filter determination module, configured to determine a target Bloom filter for verifying the data query request from multiple Bloom filters according to the target indication information in the data query request; the multiple Bloom filters respectively correspond to different data sets; the Bloom filter is used to characterize the existence status of the data in the corresponding data set;
[0015] A verification module, configured to verify the data query request through the target Bloom filter to obtain a verification result; the verification result indicates that the data query request is valid or invalid;
[0016] A query module, configured to query the data to be obtained according to the data query request if the verification result indicates that the data query request is valid;
[0017] A rejection query module, configured to reject the data query request if the verification result indicates that the data query request is invalid.
[0018] Optionally, the multiple Bloom filters are obtained by the following method:
[0019] A data set division module, configured to divide the data stored in the database into multiple data sets according to data characteristics;
[0020] A calculation and configuration module, configured to perform calculation processing on the indication information corresponding to each data set to obtain multiple calculation results; and configure the Bloom filter corresponding to the data set based on the multiple calculation results.
[0021] Optionally, the calculation and configuration module includes:
[0022] A determination sub-module, configured to determine a set of hash functions corresponding to each data set; the set of hash functions includes multiple hash functions;
[0023] A calculation and configuration sub-module, configured to calculate the indication information corresponding to each data set by using the corresponding set of hash functions to obtain multiple hash values corresponding to each data set; and configure the corresponding Bloom filter based on the multiple hash values corresponding to each data set.
[0024] Optionally, the calculation and configuration sub-module includes:
[0025] A first determination unit, configured to determine the length of the bit array in the Bloom filter corresponding to each data set; the length indicates the number of binary bits included in the bit array;
[0026] A second determination unit, configured to determine the position identifier of each binary bit in the bit array according to the length of the bit array; the values of all binary bits in the bit array are initial values;
[0027] A matching unit, configured to determine the binary bits corresponding to the position identifiers that match multiple hash values in the bit array as target binary bits;
[0028] A configuration unit, configured to modify the initial value corresponding to each of the multiple target binary bits to a preset value to obtain a Bloom filter corresponding to the corresponding data set.
[0029] Optionally, the first determination unit includes:
[0030] A first determination subunit, configured to determine a first required value of the corresponding bit array according to the data volume of the indication information corresponding to each data set;
[0031] A second determination subunit, configured to determine a second required value of the corresponding bit array according to the importance level of each data set;
[0032] A third determination subunit, configured to determine the length of the bit array corresponding to each data set according to the first required value and the second required value.
[0033] Optionally, the verification module includes:
[0034] A processing sub-module, configured to process the target indication information in the data query request by using the hash function set corresponding to the target Bloom filter to obtain multiple to-be-verified hash values;
[0035] A matching sub-module, configured to determine the binary bits corresponding to the position identifiers that match multiple to-be-verified hash values in the bit array of the target Bloom filter as to-be-verified binary bits;
[0036] A verification sub-module, configured to determine whether the values corresponding to the multiple to-be-verified binary bits are preset values; if so, the verification result indicates that the data query request is valid; if not, the verification result indicates that the data query request is invalid.
[0037] Optionally, the data query request further includes a query condition, and the query module includes:
[0038] A screening sub-module, configured to perform screening in the cache by using the query condition to obtain a screening result; multiple data are stored in the cache; the screening result is that data exists or data does not exist;
[0039] A first query sub-module, configured to use the data screened out in the cache as the data to be obtained when the screening result is that data exists;
[0040] A second query sub-module, configured to, when the screening result indicates no data, perform screening in the database using the query condition, and use the data screened out in the database as the data to be obtained.
[0041] Optionally, the device is oriented to a blockchain, and multiple Bloom filters are obtained in the following manner:
[0042] A partitioning module, configured to partition the data stored in the blockchain database into a transaction data set, a block data set, and an event data set;
[0043] A processing and configuration module, configured to perform calculation processing on a first indication information of the transaction data set, a second indication information of the block data set, and a third indication information of the event data set respectively to obtain a first calculation result, a second calculation result, and a third calculation result; and configure the Bloom filters respectively corresponding to the transaction data set, the block data set, and the event data set based on the first calculation result, the second calculation result, and the third calculation result.
[0044] A third aspect of this application provides a computer device, which includes a processor and a memory:
[0045] The memory is used to store program code and transmit the program code to the processor;
[0046] The processor is used to execute the steps provided in the first aspect according to the instructions in the program code.
[0047] A fourth aspect of this application provides a computer-readable storage medium, which is used to store program code, and the program code is used to execute the steps provided in the first aspect.
[0048] A fifth aspect of this application provides a computer program product, including a computer program or instruction, and when the computer program or instruction is executed, the steps provided in the first aspect are implemented.
[0049] It can be seen from the above technical solutions that the embodiments of this application have the following advantages:
[0050] The data query method provided by this application determines a target Bloom filter for verifying a data query request from multiple Bloom filters according to the target indication information in the data query request; where multiple Bloom filters respectively correspond to different data sets, and the Bloom filter is used to represent the data included in the corresponding data set; verify the data query request through the target Bloom filter, if the verification result indicates that the data query request is valid, query the data to be obtained according to the data query request; if the verification result indicates that the data query request is invalid, reject the data query request. Among them, by setting the Bloom filter, the server can be replaced to quickly filter invalid data query requests, avoiding cache penetration caused by a large number of invalid data query requests, and further causing server crashes; at the same time, it can avoid the increase in resource consumption caused by a large number of invalid data query requests. In addition, compared with a single Bloom filter, multiple Bloom filters have lower computational complexity and lower resource occupancy, which can improve the query efficiency. BRIEF DESCRIPTION OF THE DRAWINGS
[0051] Figure 1 FIG. is a schematic diagram of a subscription process of a client based on blockchain provided by the related art;
[0052] Figure 2 FIG. is a schematic diagram of a scenario of a data query method provided by an embodiment of this application;
[0053] Figure 3A FIG. is a schematic diagram of the structure of a data sharing system provided by an embodiment of this application;
[0054] Figure 3B FIG. is a schematic diagram of a blockchain structure provided by an embodiment of this application;
[0055] Figure 3C FIG. is a schematic diagram of a block generation process provided by an embodiment of this application;
[0056] Figure 4 FIG. is a flowchart of a data query method provided by an embodiment of this application;
[0057] Figure 5 FIG. is a schematic diagram of configuring a Bloom filter provided by an embodiment of this application;
[0058] Figure 6a FIG. is a schematic diagram of a data query method for blockchain provided by an embodiment of this application;
[0059] Figure 6b FIG. is a schematic diagram of multiple Bloom filters corresponding to a block provided by an embodiment of this application;
[0060] Figure 6c FIG. is a schematic diagram of a subscription process of a client based on blockchain provided by an embodiment of this application;
[0061] Figure 7 A structural schematic diagram of a data query device provided by an embodiment of the present application;
[0062] Figure 8 A structural schematic diagram of a server in an embodiment of the present application;
[0063] Figure 9 A structural schematic diagram of a terminal device in an embodiment of the present application. Detailed implementation manners
[0064] In order to enable those skilled in the art to better understand the solutions of the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present application.
[0065] The terms "first", "second", "third", "fourth", etc. (if any) in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that such data used can be interchanged under appropriate circumstances so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "comprising" and "having" and any variations thereof are intended to cover non-exclusive inclusion. For example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units not clearly listed or inherent to these processes, methods, products or devices.
[0066] Currently, in order for a client to obtain data stored in a database of a server, it generally sends a query request to the server. After receiving the query request, the server will verify the query request. If the verification result indicates that the query request is valid, it will query the required data according to the query conditions indicated by the query request and send the data to the client. Taking blockchain as an example, see Figure 1 , which shows the subscription process of the client based on blockchain, including:
[0067] Step 1: The client sends a subscription request to a blockchain node of the blockchain.
[0068] A subscription request is a request to subscribe to the data required by the client in the blockchain. The subscription request may include the conditions for subscription, such as the contract address, event or feature information of interest, etc. The client can specify relevant parameters such as the starting block and event topic.
[0069] Step 2: The blockchain node receives the subscription request and establishes a subscription.
[0070] It should be understood that after the blockchain node receives the subscription request, it needs to verify the validity of the subscription request, that is, it needs to verify whether the data indicated by the subscription request exists in the cache or database corresponding to the blockchain node. As an example, assuming that the subscription request Q indicates that the data "ABC" needs to be subscribed, then when the blockchain node receives the subscription request Q, it needs to query whether the data "ABC" exists in the cache or database. If so, the subscription request is a valid subscription request; otherwise, it is an invalid subscription request.
[0071] Step 3: Filter the data required for the subscription request from the cache or database as the subscription information, and push it to the client that initiated the subscription request.
[0072] It should be understood that after determining that the subscription request is valid, the required data can be filtered from the cache or database according to the filtering conditions in the subscription request and pushed to the client in real time. After receiving the data pushed by the blockchain node, the client can process and analyze the data.
[0073] However, when a blockchain node receives a large number of invalid subscription requests, a large number of invalid subscription requests will occupy the processing power and network bandwidth of the blockchain node, resulting in increased resource consumption. At the same time, when the server stores data, it will store the data stored in the database into the cache to reduce the pressure on the database. However, when a large number of invalid query requests cannot find the required data in the cache, they will "penetrate" the cache and directly query the database. That is, because the server receives a large number of invalid requests, it may cause cache penetration and cause the server to crash.
[0074] That is, in the related art, when a server receives a large number of invalid query requests, it may cause problems such as increased server resource consumption and server crash.
[0075] To solve the above technical problems, the present application provides a data query method and related device. According to the target indication information in the data query request, a target Bloom filter for verifying the data query request is determined from multiple Bloom filters; the multiple Bloom filters respectively correspond to different data sets; the Bloom filter is used to represent the data included in the corresponding data set; the data query request is verified through the target Bloom filter. If the verification result indicates that the data query request is valid, the data to be obtained is queried according to the data query request; if the verification result indicates that the data query request is invalid, the data query request is rejected.
[0076] In this way, through the provided Bloom filter, the server can be replaced to quickly filter invalid data query requests, that is, the invalid data query requests are directly filtered by the Bloom filter and will not be processed by the server, which can avoid cache penetration caused by a large number of invalid data query requests and further server crashes, and at the same time can avoid increased resource consumption. In addition, due to the large amount of stored data, if a Bloom filter is determined according to an entire data set, it may lead to a large occupation of resource space and a large computational complexity. In the present application, by dividing the entire data set into multiple data sets and setting multiple Bloom filters according to the multiple data sets, the computational complexity and resource occupancy rate can be reduced, and the query efficiency can be further improved.
[0077] See Figure 2 , which is a schematic diagram of the scenario of a data query method provided by an embodiment of the present application, and may include a client 201, a server 202, and multiple Bloom filters 203 provided in the server 202.
[0078] The client 201 sends a data query request to the server 202. The data query request may include target indication information for querying the data to be obtained, for example, the target indication information is ABC.
[0079] The server 202 receives the data query request sent by the client 201, and determines a target Bloom filter X for verifying the data query request from the multiple Bloom filters 203 according to the target indication information in the data query request. Among them, the multiple Bloom filters respectively correspond to different data sets; the Bloom filter is used to represent the data included in the corresponding data set. As an example, assuming that the target indication information is A and the Bloom filters include A, B, and C, then the target Bloom filter A can be determined from the Bloom filters A, B, and C according to the target indication information being A.
[0080] The target Bloom filter X verifies the data query request to obtain a verification result. If the verification result indicates that the data query request is valid, the data to be obtained is queried according to the data query request. If the verification result indicates that the data query request is invalid, the data query request is rejected. It should be understood that the Bloom filter is used to represent the data included in the corresponding data set. Therefore, by verifying the data query request through the target Bloom filter, it can be determined whether the data set contains the data to be obtained. If it does, the verification result indicates that the data query request is valid; otherwise, the data query result is invalid.
[0081] In this way, by setting the Bloom filter, the server can quickly filter out invalid data query requests, avoid cache penetration caused by a large number of invalid data query requests, and further avoid server crashes. At the same time, it can avoid increased resource consumption. In addition, in this application, by dividing the entire data set into multiple data sets and setting multiple Bloom filters according to the multiple data sets, the computational complexity and resource occupancy rate can be reduced, and the query efficiency can be further improved.
[0082] The data query method provided in the embodiments of this application can be applied to a server with data processing capabilities. The server can be an independent physical server, a server cluster or a distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms.
[0083] The data query method provided in the embodiments of this application involves blockchain technology.
[0084] See Figure 3A the data sharing system shown. The data sharing system 300 refers to a system used for data sharing between nodes. The data sharing system may include multiple nodes 301, and the multiple nodes 301 may refer to each client in the data sharing system. Each node 301 can receive input information during normal operation and maintain the shared data in the data sharing system based on the received input information. To ensure information interconnection within the data sharing system, there may be information connections between each node in the data sharing system, and nodes can transmit information through the above information connections. For example, when any node in the data sharing system receives input information, other nodes in the data sharing system obtain the input information according to the consensus algorithm and store the input information as data in the shared data, so that the data stored on all nodes in the data sharing system is consistent.
[0085] For each node in the data sharing system, there is a corresponding node identifier, and each node in the data sharing system can store the node identifiers of other nodes in the data sharing system, so that subsequently, according to the node identifiers of other nodes, the generated block can be broadcast to other nodes in the data sharing system. Each node can maintain a node identifier list as shown in the following table, and store the node name and the node identifier corresponding to each other in this node identifier list. Among them, the node identifier can be an IP (Internet Protocol) address and any other information that can be used to identify the node. In Table 1, only the IP address is used as an example for illustration.
[0086] Table 1: Relationship Table between Node Name and Node Identifier
[0087]
[0088]
[0089] Each node in the data sharing system stores an identical blockchain. The blockchain consists of multiple blocks. See Figure 3B , the blockchain consists of multiple blocks. The genesis block includes a block header and a block body. The block header stores the input information feature value, version number, timestamp, and difficulty value. The block body stores the input information; the next block of the genesis block uses the genesis block as the parent block. The next block also includes a block header and a block body. The block header stores the input information feature value of the current block, the block header feature value of the parent block, version number, timestamp, and difficulty value, and so on. In this way, the block data stored in each block in the blockchain is associated with the block data stored in the parent block, ensuring the security of the input information in the block.
[0090] When generating each block in the blockchain, see Figure 3C , when the node where the blockchain is located receives the input information, it verifies the input information. After the verification is completed, it stores the input information in the memory pool and updates the hash tree used to record the input information; then, it updates the timestamp to the time when the input information is received, and tries different random numbers, and performs eigenvalue calculations multiple times, so that the calculated eigenvalue can satisfy the following formula:
[0091] SHA 256(SHA 256(version+prev_hash+merkle_root+ntime+nbits+x))<TARGET
[0092] Among them, SHA256 is the eigenvalue algorithm used to calculate the eigenvalue; version (version number) is the version information of the relevant block protocol in the blockchain; prev_hash is the block header eigenvalue of the parent block of the current block; merkle_root is the eigenvalue of the input information; ntime is the update time for updating the timestamp; nbits is the current difficulty, which is a fixed value within a certain period of time and is determined again after exceeding the fixed time period; x is a random number; TARGET is the eigenvalue threshold, and this eigenvalue threshold can be determined according to nbits.
[0093] In this way, when a random number that satisfies the above formula is calculated, the information can be stored correspondingly, the block header and the block body can be generated, and the current block can be obtained. Subsequently, the node where the blockchain is located sends the newly generated block to other nodes in the data sharing system where it is located according to the node identifiers of other nodes in the data sharing system. Other nodes verify the newly generated block and add the newly generated block to the blockchain they store after the verification is completed.
[0094] In this application, the collection and processing of relevant data should strictly comply with the requirements of relevant national laws and regulations when applied in practice, obtain the informed consent or separate consent of the subject of personal information, and carry out subsequent data use and processing behaviors within the scope authorized by laws and regulations and the personal information subject.
[0095] See Figure 4 , which is a flowchart of a data query method provided by an embodiment of this application.
[0096] Combined with Figure 4 shown, the data query method provided by the embodiment of this application may include:
[0097] S401: Receive a data query request.
[0098] Among them, the data query request refers to a request for querying the data to be obtained, and this data query request includes target indication information for querying the data to be obtained. The target indication information is information indicating the data to be obtained, such as index data for querying the data to be obtained, custom query data, etc. There is a mapping relationship between the index data and the custom query data and the data to be obtained. The target indication information may be the ID (Identity document) corresponding to the data to be obtained, a custom field, and so on.
[0099] S402: Determine a target Bloom filter for verifying the data query request from multiple Bloom filters according to the target indication information in the data query request.
[0100] A Bloom Filter is a very long binary vector and a series of random mapping functions. The Bloom Filter can be used to retrieve whether a piece of data is in a data set. That is, the Bloom Filter is used to characterize the data included in the corresponding data set, and multiple Bloom Filters respectively correspond to different data sets.
[0101] It should be understood that in the embodiments of the present application, multiple Bloom Filters are set, which respectively characterize the data included in different data sets. Therefore, the target Bloom Filter for verifying the data query request can be determined through the target indication information in the data query request. As an example, assume that Bloom Filters A, B, and C are respectively used to characterize data sets a, b, and c, and the target indication information indicates the information of the data in data set b. Then the target Bloom Filter B can be determined according to the target indication information.
[0102] S403: Verify the data query request through the target Bloom Filter to obtain a verification result.
[0103] It should be understood that since the target Bloom Filter characterizes the data in the corresponding data set, the target Bloom Filter can match the data query request with the content characterized by the target Bloom Filter. If a matching result exists, it is considered that the verification result indicates that the data query request is valid; otherwise, the verification result indicates that the data query request is invalid.
[0104] S404: If the verification result indicates that the data query request is valid, query the data to be obtained according to the data query request.
[0105] It should be understood that when the verification result indicates that the data query request is valid, it can be considered that there is data in the data set corresponding to the data query request.
[0106] In some possible implementation manners, the data query request further includes a query condition, and step S405 may include:
[0107] A1: Use the query condition to filter in the cache to obtain a filtering result.
[0108] A cache refers to a buffer for data exchange (referred to as Cache), which is a temporary place for storing data (frequently used data). When querying data using an object, it can first query in the cache. If found, it is directly obtained; if not found, it goes to the database to search. The cache temporarily replaces the latest data read from the database with the data in the server memory, which can reduce the database load.
[0109] Among them, the cache can store data from multiple data sets, including but not limited to the data in the data set corresponding to the target Bloom filter. However, since the data in the cache needs to be updated regularly, there may be no data required by the data query request in the cache, that is, there may also be no data in the data set corresponding to the target Bloom filter in the cache.
[0110] The query condition refers to the condition for filtering the data to be queried from the data set. For example, if the query condition is to select the data with the data name of AAA, then the data that meets the condition can be filtered out from the data set.
[0111] The screening result refers to the result of whether the data to be obtained in the target data set is screened out from the data set stored in the cache. When there is a data set corresponding to the target Bloom filter in the cache and the data to be obtained can be screened out from the data set according to the query condition, the screening result is that there is data; otherwise, the screening result is that there is no data.
[0112] A2: When the screening result is that there is data, the data screened out from the cache is used as the data to be obtained.
[0113] It should be understood that if the screening result is that there is data, the required data can be directly screened out from the cache according to the query condition and used as the data to be obtained.
[0114] A3: When the screening result is that there is no data, the query condition is used to screen in the database, and the data screened out from the database is used as the data to be obtained.
[0115] If the candidate result is that there is no data, the query condition needs to be used to screen in the database to obtain the data to be obtained.
[0116] In this embodiment, using the data stored in the cache instead of the data in the database can reduce the load on the database.
[0117] S405: If the verification result indicates that the data query request is invalid, the data query request is rejected.
[0118] It should be understood that if the verification result indicates that the data query request is invalid, it is considered that there is no data in the data set corresponding to the target Bloom filter that matches the data query request, then the data query request can be directly rejected for query.
[0119] In the embodiments of the present application, since the Bloom filter can quickly filter invalid data query requests, when there are a large number of invalid data query requests, the Bloom filter can quickly filter out a large number of invalid query requests on behalf of the server, and the server can normally process valid data query requests, reducing the resource consumption of the server and improving the efficiency of data query.
[0120] In addition, due to the computational complexity and resource space occupancy of the Bloom filter being positively correlated with the stored data, that is, the more data stored, the greater the computational complexity and resource space occupancy of the corresponding Bloom filter. Therefore, in the embodiments of the present application, the entire data set is split into multiple data sets, and corresponding Bloom filters are set according to the multiple data sets to reduce the computational complexity and resource space occupancy of the Bloom filter, further reducing the server resource consumption and improving the efficiency of data query.
[0121] Based on the data query method provided in the above embodiments, in some possible implementation manners, the multiple Bloom filters can be obtained through the following methods:
[0122] B1: Divide the data stored in the database into multiple data sets according to data characteristics.
[0123] Data characteristics refer to the characteristics of the data, such as the type of data, the timeliness of the data, the importance of the data, the amount of data, and so on.
[0124] It should be understood that since the amount of data stored in the database is large and there is a positive correlation between the set Bloom filter and the amount of data, in order to reduce the computational complexity and resource space occupancy of the Bloom filter, the data stored in the database can be divided into different data sets according to data characteristics.
[0125] As an example, assuming that the data characteristic is the type of data, and the data stored in the database includes three types: X, Y, and Z, then the data in the database can be divided into a data set corresponding to type X, a data set corresponding to type Y, and a data set corresponding to type Z.
[0126] B2: Perform computational processing on the indication information corresponding to each data set to obtain multiple calculation results; and configure the Bloom filter corresponding to the data set based on the multiple calculation results.
[0127] The indication information means information characterizing the data in the data set, and there is at least one indication information. The indication information can be the index data or custom query data corresponding to each data in the data set.
[0128] It should be understood that the indication information is used to characterize the information of the data in the data set. By performing computational processing, multiple calculation results are obtained, and a Bloom filter corresponding to the data set is configured using the multiple calculation results. Then, the configured Bloom filter can characterize the existing data in the data set.
[0129] In the embodiments of the present application, when the Bloom filter receives a data query request, the Bloom filter can quickly verify the target indication information in the data query request to determine whether the data to be obtained indicated by the target indication information exists in the data set corresponding to the Bloom filter. If not, the request is directly rejected, realizing fast filtering of invalid data query requests, avoiding cache penetration, and reducing the resource consumption of the server.
[0130] Based on the determination method of the Bloom filter provided in the above embodiments, in some possible implementation manners, step B2 may include:
[0131] C1: Determine the set of hash functions corresponding to each data set.
[0132] Hash is generally translated as "hash", and there is also a direct transliteration as "hash". It is to transform an input of any length (also called pre - mapping, pre - image) into a fixed - length output through a hashing algorithm, and this output is the hash value.
[0133] It should be understood that a Bloom filter is essentially a very long binary vector and a series of random mapping functions. Therefore, when configuring the Bloom filter, it is necessary to map the data in the data set to this very long binary vector, that is, the indication information corresponding to the data in the data set can be calculated through the set of hash functions to obtain the hash value representing the data.
[0134] The set of hash functions includes multiple hash functions, and the multiple hash functions are used to calculate the target indication information to obtain multiple hash values corresponding to the indication information. It should be understood that the number of hash functions included in the set of hash functions corresponding to different data sets can be the same or different.
[0135] It should be noted that since the data set includes multiple data, and each data may have corresponding indication information, it is necessary to calculate each of the multiple indication information respectively using the set of hash functions to obtain multiple hash values corresponding to each indication information.
[0136] C2: For the indication information corresponding to each data set, calculate using the corresponding set of hash functions to obtain multiple hash values corresponding to each data set; and configure the corresponding Bloom filter based on the multiple hash values corresponding to each data set.
[0137] It should be understood that the multiple hash values corresponding to the indication information are used to represent the data corresponding to the indication information. That is, by using the multiple hash values to configure the Bloom filter, the Bloom filter can represent the data in the data set. When the Bloom filter receives a data query request, the Bloom filter can quickly verify the target indication information in the data query request, so as to quickly filter out invalid data query requests.
[0138] In some possible implementation manners, to further illustrate the configuration process of the Bloom filter, refer to Figure 5 , which is a schematic diagram of configuring a Bloom filter provided in an embodiment of this application.
[0139] Combined with Figure 5 shown, configuring the corresponding Bloom filter based on the multiple hash values corresponding to each data set may include:
[0140] D1: Determine the length of the bit array in the Bloom filter corresponding to each data set.
[0141] The data structure of the Bloom filter is a bit array (bitmap) including multiple binary bits. The bit array includes multiple binary bits, and the value of each binary bit is an initial value, such as 0. Specifically, refer to Figure 5 shown.
[0142] Among them, the length indicates the number of binary bits included in the bit array. The length of the bit array is related to the data volume of the indication information and the importance of the data set corresponding to the bit array. Therefore, in some possible implementation manners, D1 may specifically include:
[0143] E1: Determine the first required value of the corresponding bit array according to the data volume of the indication information corresponding to each data set.
[0144] The first required value means a value related to the data volume of the indication information, such as the expected data volume. It should be understood that before configuring the Bloom filter, the data volumes of the data in different data sets are different, that is, the data volumes of the indication information corresponding to different data sets are different. Therefore, it is necessary to determine the expected data volume of the indication information, that is, estimate the expected data volume according to the data volume of the indication information. Generally, the expected data volume will be a little larger than the actual data volume to avoid the length of the bit array of the Bloom filter being insufficient.
[0145] E2: Determine the second required value of the corresponding bit array according to the importance of each data set.
[0146] The second required value refers to a value related to the importance level of the data set, such as the expected false positive rate. It should be understood that in actual requirements and application scenarios, the importance levels of the data in different data sets are different. Therefore, it is necessary to determine the tolerable false positive rate. The lower the false positive rate, the longer the length of the bit array corresponding thereto.
[0147] It should be understood that as a large amount of indication information is added to the Bloom filter, when an indication information not in the Bloom filter is calculated, after determining the position identifier of the bit array according to the obtained hash value, the value of the binary bit corresponding to this position identifier may already be the preset value, and then it will be misjudged that the indication information exists in the Bloom filter. As an example, assume that indication information a and indication information b have the same hash value, and assume that 3 hash calculations are performed, and the hash values are 1, 2, and 3; the indication information a is inserted into the Bloom filter, for example, a[1]=a[2]=a[3]=1; however, the indication information b is not inserted into the Bloom filter. When querying the indication information b, after hash calculation, the obtained array subscripts are 1, 2, and 3. Since the indication information a sets the values of these three binary bits to 1, it will be judged that the indication information b exists in the Bloom filter, that is, a misjudgment occurs.
[0148] E3: Determine the length of the bit array corresponding to each data set according to the first required value and the second required value.
[0149] It should be understood that by comprehensively determining the length of the bit array corresponding to each data set from two dimensions of the first required value and the second required value, the length of the bit array can be made more in line with the actual requirements.
[0150] In a possible implementation manner, the length of the bit array can be determined by the following method:
[0151]
[0152] Among them, m represents the length of the bit array, n represents the expected data volume (that is, the first required value), and p represents the expected false positive rate (that is, the second required value).
[0153] D2: Determine the position identifier of each binary bit in the bit array according to the length of the bit array.
[0154] The values of all binary bits in the bit array are initial values, such as 0. The position identifier refers to the identifier of the position of each binary bit in the bit array. As Figure 5 shown, the position identifiers of each binary bit in the bit array from left to right can be 0, 1, 2, 3,..., 15.
[0155] D3: Determine the binary bits corresponding to the position identifiers that match multiple hash values in the bit array as the target binary bits.
[0156] Among them, by calculating using the hash functions in the set of hash functions, multiple hash values can be obtained. The multiple hash values can be matched with the position identifiers of multiple binary bits in the bit array, and the determined binary bits are used as target binary bits. As an example, assume the hash values are 1, 5, 15, and the position identifiers of the multiple binary bits of the bit array are from 0 to 15. Then the binary bits with position identifiers 1, 5, and 15 are the target binary bits.
[0157] D4: Modify the initial value corresponding to each of the multiple target binary bits to a preset value to obtain the Bloom filter corresponding to the corresponding data set.
[0158] It should be understood that by modifying the initial value of the target binary bit to a preset value, the indication information can be represented in the bit array, that is, the corresponding Bloom filter can represent the data corresponding to the indication information. Combining Figure 5 As shown, as an example, assume that the target binary bits corresponding to indication information 1 are 2, 5, 14, and the target binary bits corresponding to indication information 2 are 5, 8, 13; then the initial value 0 of the binary bits 2, 5, 8, 13, and 14 is respectively modified to the preset value 1. Among them, both indication information 1 and indication information 2 have the target binary bit 5, that is, the misjudgment situation existing in the Bloom filter.
[0159] In the embodiment of the present application, the set Bloom filter can quickly verify the target indication information in the data query request to determine whether the data to be obtained indicated by the target indication information exists in the data set corresponding to the Bloom filter. If not, the request is directly rejected, realizing the quick filtering of invalid data query requests, avoiding cache penetration, and reducing the resource consumption of the server.
[0160] Based on the data query method provided in the above embodiment, in some possible implementation manners, step S404 may include:
[0161] F1: Process the target indication information in the data query request by using the set of hash functions corresponding to the target Bloom filter to obtain multiple hash values to be verified.
[0162] It should be understood that the target indication information indicates the information of the data to be obtained. By processing the target indication information by using the set of hash functions corresponding to the target Bloom filter, multiple hash values to be verified corresponding to the target indication information can be obtained. As an example, assume the target indication information is aaa, and by calculating the target indication information aaa using the target Bloom filter, 3 hash values obtained are 1, 2, and 3 respectively.
[0163] F2: Determine the binary bits corresponding to the position identifiers in the bit array of the target Bloom filter that match multiple hash values to be verified as the binary bits to be verified.
[0164] It should be understood that after determining multiple hash values to be verified corresponding to the target indication information, the position identifiers that match in the bit array of the target Bloom filter can be determined through the multiple hash values to be verified, and the binary bits corresponding to the matching position identifiers are used as the binary bits to be verified. As an example, if the 3 hash values corresponding to the target indication information aaa are 1, 2, and 3 respectively, then the binary bits with position identifiers 1, 2, and 3 can be used as the binary bits to be verified.
[0165] F3: Determine whether the values corresponding to the multiple binary bits to be verified are preset values; if so, the verification result indicates that the data query request is valid; if not, the verification result indicates that the data query request is invalid.
[0166] It should be understood that if the data to be obtained corresponding to the target indication information exists in the database, the indication information of the data to be obtained will be stored in the Bloom filter when configuring the Bloom filter. Therefore, when determining multiple binary bits to be verified through the target indication information, the values corresponding to the multiple binary bits should be preset values, which proves that the data to be obtained corresponding to the target indication information exists, that is, the data query request is valid; otherwise, the data to be obtained does not exist, that is, the data query request is invalid.
[0167] Based on the data query method provided in the above embodiments, the embodiments of the present application also provide a data query method for a blockchain. As shown in Figure 6a This method may include:
[0168] Step 1: In the blockchain, a new block is generated after consensus by blockchain nodes.
[0169] Among them, the new block may include transaction data, block data, event data, etc. Among them, transaction data, block data, and event data include their respective corresponding index data or quick query data, such as block information, transaction information, etc.
[0170] It should be noted that after consensus is completed, it is also necessary to process and verify transaction data, block data, and event data to ensure the correctness of the data.
[0171] Step 2: Divide the data stored in the block into a transaction data set, a block data set, and an event data set.
[0172] It should be understood that the new block may include multiple transaction data, multiple block data, and multiple event data. Therefore, each data needs to be divided into different data sets respectively to facilitate the subsequent configuration of corresponding Bloom filters.
[0173] Step 3: Perform calculation processing on the first indication information of the transaction data set, the second indication information of the block data set, and the third indication information of the event data set respectively to obtain a first calculation result, a second calculation result, and a third calculation result; and based on the first calculation result, the second calculation result, and the third calculation result, configure the corresponding Bloom filters for the transaction data set, the block data set, and the event data set respectively.
[0174] Among them, the first indication information, the second indication information, and the third indication information respectively indicate the index data or quick query data of the data corresponding to the transaction data set, the block data set, and the event data set. By processing the first indication information, the second indication information, and the third indication information, the Bloom filters of the corresponding data sets can be configured.
[0175] Step 4: Finally, store the business data (such as the detailed content of block information and the detailed content of transaction information) corresponding to the first indication information, the second indication information, and the third indication information into the database.
[0176] After setting multiple Bloom filters through Steps 1 to 4, different data query requests can be filtered according to the Bloom filters corresponding to each data set, as Figure 6b shown. For example, define the Bloom filter block_bitmap to store the indication information related to block data, the Bloom filter tx_bitmap to store the indication information related to transaction data, and the Bloom filter event_bitmap to store the indication information corresponding to event data.
[0177] Taking block data as an example, block data may include block header data, block size data, block height data, transaction data, event data, etc. The indication information corresponding to the block header data may be block header information, the indication information corresponding to the block size data may be block size information, and the indication information corresponding to the transaction data may be transaction information, etc.
[0178] As an example, assume that the data query request sent by the client includes the target indication information as tx_ID: 111222. Then, the corresponding target Bloom filter can be determined as tx_bitmap based on the target indication information. The target Bloom filter tx_bitmap can verify the target indication information tx_ID: 111222 to determine whether the data request is valid. Among some possible implementation manners, the target indication information can be a 64-bit, hexadecimal string.
[0179] In some possible implementation manners, for the Bloom filter block_bitmap, refer to Figure 6c , which shows the process of the client subscribing to block information in the blockchain and may include:
[0180] S601: The client sends a block subscription request to the target blockchain node.
[0181] The target blockchain node refers to the node in which the corresponding block includes the required block information.
[0182] The block subscription request refers to the request for subscribing to block information.
[0183] Among them, the block subscription request sent by the client to the target blockchain node may include target block indication information and block query conditions.
[0184] S602: The target blockchain node sends the block subscription request to the Bloom filter block_bitmap.
[0185] Among them, the Bloom filter block_bitmap is used to represent the indication information related to block data, and the target block information can be verified whether it exists through the Bloom filter block_bitmap.
[0186] S603: The Bloom filter block_bitmap verifies whether the target block information in the block subscription request exists. If it exists, a subscription is established based on the block subscription request.
[0187] S604: Use the query conditions in the block subscription request to filter the required data in the database as the query result.
[0188] Among them, in this embodiment, if the block subscription request is valid, the query can be directly performed in the database. In other implementation manners, the block subscription request can also be sent to the cache for query, which is not specifically limited here.
[0189] As an example, assume that the query condition is to filter the block data from 10:00 to 12:00 within a week. Then, the data that meets the conditions can be filtered out from multiple block data according to this query condition.
[0190] S605: Return the query result to the target blockchain node, and the target blockchain node sends the subscription result to the client.
[0191] It should be understood that after obtaining the query result, the target blockchain node can push the query result to the client in real time, and after the query data in the query result is updated, push the new query data to the client.
[0192] Based on the data query method provided in the above embodiments, refer to Figure 7 , this figure is a schematic structural diagram of a data query device provided in an embodiment of the present application. Combining Figure 7 As shown, the data query device 700 provided in the embodiment of the present application may include:
[0193] A receiving module 701, configured to receive a data query request; the data query request includes target indication information for querying data to be obtained;
[0194] A target Bloom filter determination module 702, configured to determine a target Bloom filter for verifying the data query request from a plurality of Bloom filters according to the target indication information in the data query request; the plurality of Bloom filters respectively correspond to different data sets; the Bloom filter is used to represent the existence state of the data in the corresponding data set;
[0195] A verification module 703, configured to verify the data query request through the target Bloom filter to obtain a verification result; the verification result indicates that the data query request is valid or invalid;
[0196] A query module 704, configured to query the data to be obtained according to the data query request if the verification result indicates that the data query request is valid;
[0197] A reject query module 705, configured to reject the data query request if the verification result indicates that the data query request is invalid.
[0198] As an example, the plurality of Bloom filters are obtained in the following manner:
[0199] A data set partitioning module, configured to partition the data stored in the database into a plurality of data sets according to data characteristics;
[0200] A calculation and configuration module, configured to perform calculation processing on the indication information corresponding to each data set to obtain a plurality of calculation results; and configure the Bloom filter corresponding to the data set based on the plurality of calculation results.
[0201] As an example, the calculation and configuration module includes:
[0202] Determination sub-module, configured to determine a set of hash functions corresponding to each data set; the set of hash functions includes multiple hash functions;
[0203] Calculation and configuration sub-module, configured to perform calculations on the indication information corresponding to each data set by using the corresponding set of hash functions to obtain multiple hash values corresponding to each data set; and configure a corresponding Bloom filter based on the multiple hash values corresponding to each data set.
[0204] As an example, the calculation and configuration sub-module includes:
[0205] First determination unit, configured to determine the length of the bit array in the Bloom filter corresponding to each data set; the length indicates the number of binary bits included in the bit array;
[0206] Second determination unit, configured to determine the position identifier of each binary bit in the bit array according to the length of the bit array; the values of all binary bits in the bit array are initial values;
[0207] Matching unit, configured to determine the binary bits corresponding to the position identifiers that match the multiple hash values in the bit array as target binary bits;
[0208] Configuration unit, configured to modify the initial value corresponding to each of the multiple target binary bits to a preset value to obtain the Bloom filter corresponding to the corresponding data set.
[0209] As an example, the first determination unit includes:
[0210] First determination sub-unit, configured to determine a first required value of the bit array corresponding to each data set according to the data volume of the indication information corresponding to each data set;
[0211] Second determination sub-unit, configured to determine a second required value of the bit array corresponding to each data set according to the importance level of each data set;
[0212] Third determination sub-unit, configured to determine the length of the bit array corresponding to each data set according to the first required value and the second required value.
[0213] As an example, the verification module includes:
[0214] Processing sub-module, configured to process the target indication information in the data query request by using the set of hash functions corresponding to the target Bloom filter to obtain multiple hash values to be verified;
[0215] Matching sub-module, configured to determine the binary bits corresponding to the position identifiers that match the multiple hash values to be verified in the bit array of the target Bloom filter as binary bits to be verified;
[0216] A verification sub-module is used to determine whether the values corresponding to multiple binary bits to be verified are preset values; if so, the verification result indicates that the data query request is valid; if not, the verification result indicates that the data query request is invalid.
[0217] As an example, the data query request further includes a query condition. The query module 704 includes:
[0218] A filtering sub-module is used to filter in the cache using the query condition to obtain a filtering result; multiple data are stored in the cache; the filtering result is that data exists or data does not exist;
[0219] A first query sub-module is used to, when the filtering result is that data exists, use the data filtered out in the cache as the data to be obtained;
[0220] A second query sub-module is used to, when the filtering result is that data does not exist, filter in the database using the query condition and use the data filtered out in the database as the data to be obtained.
[0221] As an example, the device is oriented to a blockchain. Multiple Bloom filters are obtained in the following way:
[0222] A partitioning module is used to partition the data stored in the blockchain database into a transaction data set, a block data set, and an event data set;
[0223] A processing and configuration module is used to respectively perform calculation processing on the first indication information of the transaction data set, the second indication information of the block data set, and the third indication information of the event data set to obtain a first calculation result, a second calculation result, and a third calculation result; and based on the first calculation result, the second calculation result, and the third calculation result, respectively configure the Bloom filters corresponding to the transaction data set, the block data set, and the event data set.
[0224] The data query device provided by the embodiments of the present application has the same beneficial effects as the data query method provided by the above embodiments, and thus will not be elaborated herein.
[0225] The structures will be introduced below in the forms of a server and a terminal device respectively.
[0226] Figure 8It is a schematic diagram of a server structure provided by an embodiment of the present application. The server 900 may vary greatly due to different configurations or performances, and may include one or more central processing units (CPUs) 922 (for example, one or more processors) and a memory 932, and one or more storage media 930 (for example, one or more mass storage devices) that store application programs 942 or data 944. Among them, the memory 932 and the storage media 930 may be transient storage or persistent storage. The program stored in the storage media 930 may include one or more modules (not shown in the figure), and each module may include a series of instruction operations on the server. Further, the central processing unit 922 may be configured to communicate with the storage media 930 and execute a series of instruction operations in the storage media 930 on the server 900.
[0227] The server 900 may further include one or more power supplies 926, one or more wired or wireless network interfaces 950, one or more input / output interfaces 958, and / or one or more operating systems 941, such as Windows Server TM , Mac OS X TM , Unix TM , Linux TM , FreeBSD TM and so on.
[0228] Among them, the CPU 922 is used to execute the following steps:
[0229] Receive a data query request; the data query request includes target indication information for querying the data to be obtained;
[0230] Determine a target Bloom filter for verifying the data query request from multiple Bloom filters according to the target indication information in the data query request; the multiple Bloom filters respectively correspond to different data sets; the Bloom filter is used to represent the data included in the corresponding data set;
[0231] Verify the data query request through the target Bloom filter to obtain a verification result; the verification result indicates that the data query request is valid or invalid;
[0232] If the verification result indicates that the data query request is valid, query the data to be obtained according to the data query request;
[0233] If the verification result indicates that the data query request is invalid, reject the data query request.
[0234] The embodiment of the present application also provides another terminal device, such as Figure 9As shown, for the sake of convenience of description, only the parts related to the embodiments of the present application are shown. For the specific technical details not disclosed, please refer to the method part of the embodiments of the present application. The terminal may be any terminal device including a mobile phone, a tablet computer, a personal digital assistant (English full name: Personal Digital Assistant, English abbreviation: PDA), a point of sales (English full name: Point of Sales, English abbreviation: POS), an in-vehicle computer, etc. Taking the terminal as a mobile phone as an example:
[0235] Figure 9 The block diagram of a part of the structure of the mobile phone related to the terminal provided by the embodiments of the present application is shown. Refer to Figure 9 , the mobile phone includes: a radio frequency (English full name: Radio Frequency, English abbreviation: RF) circuit 1010, a memory 1020, an input unit 1030, a display unit 1040, a sensor 1050, an audio circuit 1060, a wireless fidelity (English full name: wirelessfidelity, English abbreviation: WiFi) module 1070, a processor 1080, and a power supply 1090 and other components. Those skilled in the art can understand that Figure 9 the structure of the mobile phone shown in
[0236] does not limit the mobile phone, and may include more or fewer components than shown in the figure, or combine some components, or arrange different components. Figure 9 The following specifically introduces each component of the mobile phone:
[0237] The RF circuit 1010 can be used for receiving and transmitting information or signals during communication. Specifically, after receiving the downlink information from the base station, it is sent to the processor 1080 for processing. Additionally, the uplink data is sent to the base station. Generally, the RF circuit 1010 includes, but is not limited to, an antenna, at least one amplifier, a transceiver, a coupler, a low noise amplifier (full English name: Low Noise Amplifier, English abbreviation: LNA), a duplexer, etc. In addition, the RF circuit 1010 can also communicate with the network and other devices through wireless communication. The above wireless communication can use any communication standard or protocol, including but not limited to the Global System of Mobile communication (full English name: Global System of Mobile communication, English abbreviation: GSM), General Packet Radio Service (GPRS), Code Division Multiple Access (full English name: Code Division Multiple Access, English abbreviation: CDMA), Wideband Code Division Multiple Access (full English name: Wideband Code Division Multiple Access, English abbreviation: WCDMA), Long Term Evolution (full English name: Long Term Evolution, English abbreviation: LTE), email, Short Messaging Service (full English name: Short Messaging Service, SMS), etc.
[0238] The memory 1020 can be used to store software programs and modules. The processor 1080 executes various functional applications and data processing of the mobile phone by running the software programs and modules stored in the memory 1020. The memory 1020 mainly includes a program storage area and a data storage area. Among them, the program storage area can store the operating system, application programs required for at least one function (such as the sound playback function, image playback function, etc.); the data storage area can store the data created according to the use of the mobile phone (such as audio data, phone book, etc.). In addition, the memory 1020 can include high-speed random access memory and can also include non-volatile memory, such as at least one magnetic disk storage device, a flash memory device, or other non-volatile solid-state storage devices.
[0239] The input unit 1030 can be used to receive input digital or character information, and generate key signal inputs related to the user settings and function controls of the mobile phone. Specifically, the input unit 1030 can include a touch panel 1031 and other input devices 1032. The touch panel 1031, also known as a touch screen, can collect touch operations of the user thereon or nearby (such as operations of the user using any suitable object or accessory such as a finger, a stylus, etc. on or near the touch panel 1031), and drive corresponding connection devices according to a preset program. Optionally, the touch panel 1031 can include two parts: a touch detection device and a touch controller. Among them, the touch detection device detects the touch orientation of the user, detects the signal brought by the touch operation, and transmits the signal to the touch controller; the touch controller receives the touch information from the touch detection device, converts it into contact coordinates, and then sends it to the processor 1080, and can receive and execute commands sent by the processor 1080. In addition, various types such as resistive, capacitive, infrared, and surface acoustic wave can be used to implement the touch panel 1031. In addition to the touch panel 1031, the input unit 1030 can also include other input devices 1032. Specifically, the other input devices 1032 can include, but are not limited to, one or more of a physical keyboard, function keys (such as volume control keys, power on / off keys, etc.), a trackball, a mouse, a joystick, etc.
[0240] The display unit 1040 can be used to display information input by the user or provided to the user, as well as various menus of the mobile phone. The display unit 1040 can include a display panel 1041. Optionally, the display panel 1041 can be configured in the form of a liquid crystal display (full English name: Liquid Crystal Display, English abbreviation: LCD), an organic light-emitting diode (full English name: Organic Light-Emitting Diode, English abbreviation: OLED), etc. Further, the touch panel 1031 can cover the display panel 1041. When the touch panel 1031 detects a touch operation thereon or nearby, it transmits it to the processor 1080 to determine the type of touch event. Subsequently, the processor 1080 provides corresponding visual output on the display panel 1041 according to the type of touch event. Although in Figure 9 the touch panel 1031 and the display panel 1041 are implemented as two independent components to realize the input and input functions of the mobile phone, in some embodiments, the touch panel 1031 and the display panel 1041 can be integrated to realize the input and output functions of the mobile phone.
[0241] The mobile phone may further include at least one sensor 1050, such as a light sensor, a motion sensor, and other sensors. Specifically, the light sensor may include an ambient light sensor and a proximity sensor. Among them, the ambient light sensor can adjust the brightness of the display panel 1041 according to the brightness of the ambient light, and the proximity sensor can turn off the display panel 1041 and / or the backlight when the mobile phone is moved to the ear. As a kind of motion sensor, the accelerometer sensor can detect the magnitude of acceleration in various directions (generally three axes), and can detect the magnitude and direction of gravity when stationary, and can be used for applications that identify the posture of the mobile phone (such as horizontal and vertical screen switching, related games, magnetometer posture calibration), vibration recognition related functions (such as pedometer, tapping), etc.; as for other sensors such as gyroscopes, barometers, hygrometers, thermometers, and infrared sensors that the mobile phone can also be configured with, they will not be elaborated here.
[0242] The audio circuit 1060, the speaker 1061, and the microphone 1062 can provide an audio interface between the user and the mobile phone. The audio circuit 1060 can transmit the electrical signal converted from the received audio data to the speaker 1061, and the speaker 1061 converts it into a sound signal for output; on the other hand, the microphone 1062 converts the collected sound signal into an electrical signal, which is received by the audio circuit 1060 and then converted into audio data. After the audio data is output to the processor 1080 for processing, it is sent through the RF circuit 1010 to, for example, another mobile phone, or the audio data is output to the memory 1020 for further processing.
[0243] WiFi belongs to short - range wireless transmission technology. The mobile phone can help users send and receive emails, browse the web, and access streaming media through the WiFi module 1070, which provides users with wireless broadband Internet access. Although Figure 9 the WiFi module 1070 is shown, it can be understood that it does not belong to the essential components of the mobile phone and can be omitted completely within the scope of not changing the essence of the invention according to needs.
[0244] The processor 1080 is the control center of the mobile phone, connecting various parts of the entire mobile phone through various interfaces and lines. By running or executing software programs and / or modules stored in the memory 1020, and by calling the data stored in the memory 1020, it executes various functions of the mobile phone and processes data, thereby collecting overall data and information of the mobile phone. Optionally, the processor 1080 may include one or more processing units; preferably, the processor 1080 may integrate an application processor and a modem processor. Among them, the application processor mainly processes the operating system, user interface, and application programs, etc., and the modem processor mainly processes wireless communication. It can be understood that the above - mentioned modem processor may not be integrated into the processor 1080 either.
[0245] The mobile phone further includes a power supply 1090 (such as a battery) for supplying power to each component. Preferably, the power supply can be logically connected to the processor 1080 through a power management system, so as to implement functions such as charging management, discharging management, and power consumption management through the power management system.
[0246] Although not shown, the mobile phone may further include a camera, a Bluetooth module, etc., which will not be elaborated here.
[0247] In the embodiment of the present application, the processor 1080 included in the terminal further has the following functions:
[0248] Receiving a data query request; the data query request includes target indication information for querying data to be obtained;
[0249] Determining a target Bloom filter for verifying the data query request from a plurality of Bloom filters according to the target indication information in the data query request; the plurality of Bloom filters respectively correspond to different data sets; the Bloom filter is used to represent the data included in the corresponding data set;
[0250] Verifying the data query request through the target Bloom filter to obtain a verification result; the verification result indicates that the data query request is valid or invalid;
[0251] If the verification result indicates that the data query request is valid, querying the data to be obtained according to the data query request;
[0252] If the verification result indicates that the data query request is invalid, rejecting the data query request.
[0253] The embodiment of the present application further provides a computer-readable storage medium for storing program codes, and the program codes are used to execute any one of the implementation manners of a data query method described in the foregoing various embodiments.
[0254] The embodiment of the present application further provides a computer program product including instructions, and when it runs on a computer, it causes the computer to execute any one of the implementation manners of a data query method described in the foregoing various embodiments.
[0255] Those skilled in the art can clearly understand that for the convenience and conciseness of description, the specific working processes of the systems and devices described above can refer to the corresponding processes in the foregoing method embodiments, which will not be elaborated here.
[0256] In several embodiments provided in this application, it should be understood that the disclosed systems and methods can be implemented in other ways. For example, the system embodiments described above are merely illustrative. For example, the division of the system is only a logical function division. In actual implementation, there may be other division methods. For example, multiple systems can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the displayed or discussed couplings or direct couplings or communication connections to each other can be indirect couplings or communication connections through some interfaces, devices or units, and can be in electrical, mechanical or other forms.
[0257] The systems described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they can be located in one place or distributed to multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.
[0258] In addition, each functional unit in various embodiments of this application can be integrated in a processing unit, or each unit can exist physically alone, or two or more units can be integrated in one unit. The above-mentioned integrated units can be implemented in the form of hardware or in the form of software functional units.
[0259] If the above-mentioned integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of this technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in various embodiments of this application. And the aforementioned storage medium includes: USB flash drives, mobile hard disks, read-only memories (English full name: Read-Only Memory, English abbreviation: ROM), random access memories (English full name: Random Access Memory, English abbreviation: RAM), magnetic disks or optical discs and other various media that can store program codes.
[0260] The above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit it; although the present application has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that: they can still modify the technical solutions described in the foregoing embodiments, or perform equivalent replacements on some of the technical features; and these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present application.
Claims
1. A data query method, characterized in that, the method includes: Receiving a data query request; the data query request includes target indication information for querying data to be obtained; According to the target indication information in the data query request, determining a target Bloom filter for verifying the data query request from multiple Bloom filters; the multiple Bloom filters respectively correspond to different data sets; the Bloom filter is used to represent the data included in the corresponding data set; Verifying the data query request through the target Bloom filter to obtain a verification result; the verification result indicates that the data query request is valid or invalid; If the verification result indicates that the data query request is valid, querying the data to be obtained according to the data query request; If the verification result indicates that the data query request is invalid, rejecting the data query request.
2. The method according to claim 1, characterized in that, the multiple Bloom filters are obtained by the following method: Dividing the data stored in the database into multiple data sets according to data characteristics; Performing calculation processing on the indication information corresponding to each data set to obtain multiple calculation results; and configuring the Bloom filter corresponding to the data set based on the multiple calculation results.
3. The method according to claim 2, characterized in that, the performing calculation processing on the indication information corresponding to each data set to obtain multiple calculation results, and configuring the Bloom filter corresponding to the data set based on the multiple calculation results includes: Determining a hash function set corresponding to each data set; the hash function set includes multiple hash functions; Calculating the indication information corresponding to each data set by using the corresponding hash function set to obtain multiple hash values corresponding to each data set; and configuring the corresponding Bloom filter based on the multiple hash values corresponding to each data set.
4. The method according to claim 3, characterized in that, the configuring the corresponding Bloom filter based on the multiple hash values corresponding to each data set includes: Determining the length of the bit array in the Bloom filter corresponding to each data set; the length indicates the number of binary bits included in the bit array; Determining the position identifier of each binary bit in the bit array according to the length of the bit array; the values of all binary bits in the bit array are initial values; Determining the binary bits corresponding to the position identifiers matching the multiple hash values in the bit array as target binary bits; Modifying the initial values corresponding to the multiple target binary bits to preset values to obtain the Bloom filter corresponding to the corresponding data set.
5. The method according to claim 4, characterized in that, the determining the length of the bit array in the Bloom filter corresponding to each data set includes: Determining a first required value of the bit array corresponding to each data set according to the data volume of the indication information corresponding to each data set; Determining a second required value of the bit array corresponding to each data set according to the importance degree of each data set; Determine the length of the bit array corresponding to each of the data sets according to the first demand value and the second demand value.
6. The method according to claim 4, wherein, the verifying the data query request by the target Bloom filter to obtain a verification result includes: processing the target indication information in the data query request by using the set of hash functions corresponding to the target Bloom filter to obtain a plurality of hash values to be verified; determining the binary bits corresponding to the position identifiers in the bit array of the target Bloom filter that match the plurality of hash values to be verified as the binary bits to be verified; judging whether the values corresponding to the plurality of binary bits to be verified are the preset values; if so, the verification result indicates that the data query request is valid; if not, the verification result indicates that the data query request is invalid.
7. The method according to claim 1, wherein, the data query request further includes a query condition, and the querying the data to be obtained according to the data query request includes: performing screening in the cache by using the query condition to obtain a screening result; a plurality of data are stored in the cache; the screening result is that data exists or does not exist; when the screening result is that data exists, the data screened out in the cache is used as the data to be obtained; when the screening result is that data does not exist, the query condition is used to perform screening in the database, and the data screened out in the database is used as the data to be obtained.
8. The method according to claim 1, wherein, the method is oriented to a blockchain, and the plurality of Bloom filters are obtained by the following method: dividing the data stored in the blockchain database into a transaction data set, a block data set, and an event data set; respectively performing calculation processing on the first indication information of the transaction data set, the second indication information of the block data set, and the third indication information of the event data set to obtain a first calculation result, a second calculation result, and a third calculation result; and respectively configuring the Bloom filters corresponding to the transaction data set, the block data set, and the event data set based on the first calculation result, the second calculation result, and the third calculation result.
9. A data query device, wherein, the device includes: a receiving module, configured to receive a data query request; the data query request includes target indication information for querying data to be obtained; a target Bloom filter determination module, configured to determine a target Bloom filter for verifying the data query request from a plurality of Bloom filters according to the target indication information in the data query request; the plurality of Bloom filters respectively correspond to different data sets; the Bloom filter is used to represent the existence state of the data in the corresponding data set; a verification module, configured to verify the data query request by the target Bloom filter to obtain a verification result; the verification result indicates that the data query request is valid or invalid; A query module, configured to query the data to be acquired according to the data query request if the verification result indicates that the data query request is valid; A rejection query module, configured to reject the data query request if the verification result indicates that the data query request is invalid.
10. A computer device, characterized in that the computer device includes a processor and a memory; the memory is used for storing a computer program; the processor is configured to execute the data query method according to any one of claims 1 to 8 based on the computer program.
11. A computer-readable storage medium, characterized in that the computer-readable storage medium is used for storing a computer program, and the computer program is used for executing the data query method according to any one of claims 1 to 8.