Method and device for associating data based on cuckoo filter

Abstract

The embodiments of the present application provide a method and device for associating data based on a cuckoo filter, which relates to the field of big data. The method is applied to a terminal, which is equipped with a cuckoo filter. The cuckoo filter processes the acquired identifier of the data to be associated to obtain a fingerprint to be associated. The cuckoo filter matches the fingerprint to be associated with the current fingerprint set, wherein the current fingerprint set is associated with the target data set. Based on the matching results, the association relationship between the data to be associated and the target data set is determined. The method provided by the embodiments of the present application can avoid the situation where the memory resource demand is too high, and can also improve the hit rate of real-time data query in the external database.

Description

Technical Field

[0001] The present application relates to the field of big data technology, and more specifically, to a method, apparatus, device, and storage medium for associating data based on a cuckoo filter. Background Art

[0002] In traditional real-time data processing, when real-time data needs to be associated with external data (for example, the external data is a data table stored in a relational database), there are two solutions: in-memory association and external middleware association.

[0003] However, in actual applications, in-memory association solutions create huge demands on memory when facing large external data tables, resulting in a large amount of temporary use and waste of computing resources; when external middleware association solutions are applied in real-time marketing scenarios, they are prone to generate a large number of invalid network accesses and data queries.

[0004] Therefore, existing solutions cannot efficiently utilize memory and are prone to generate a large number of invalid network accesses and data queries. Summary of the Invention

[0005] The purpose of this application is to solve at least one of the above-mentioned technical deficiencies, especially the technical deficiencies that the existing solutions cannot efficiently utilize memory and are prone to generate a large number of invalid network accesses and data queries.

[0006] According to one aspect of the present application, a method for associating data based on a cuckoo filter is provided. The method is applied to a terminal configured with a cuckoo filter, and the method includes:

[0007] Processing the acquired to-be-associated identifier of the to-be-associated data through a cuckoo filter to obtain a to-be-associated fingerprint;

[0008] Matching the fingerprint to be associated with the current fingerprint set through a cuckoo filter, wherein the current fingerprint set is associated with the target data set;

[0009] According to the matching results, the association relationship between the data to be associated and the target data set is determined.

[0010] According to another aspect of the present application, a device based on cuckoo filter association data is provided, the device comprising a cuckoo filter and a determination module, wherein:

[0011] The cuckoo filter is used to process the to-be-associated identifier of the acquired to-be-associated data to obtain the to-be-associated fingerprint;

[0012] The cuckoo filter is also used to match the fingerprint to be associated with the current fingerprint set, where the current fingerprint set is associated with the target data set;

[0013] The determination module is used to determine the association relationship between the data to be associated and the target data set based on the matching results.

[0014] According to another aspect of the present application, an electronic device is provided, the electronic device comprising:

[0015] processor; and

[0016] The memory is configured to store machine-readable instructions, which, when executed by the processor, enable the processor to perform operations corresponding to the method for associating data based on a cuckoo filter according to one aspect of the present application.

[0017] According to another aspect of the present application, a computer-readable storage medium is provided, which is used to store computer instructions. When the computer instructions are executed on a computer, the computer can perform operations corresponding to the method based on cuckoo filter association data of the aforementioned aspect of the present application.

[0018] The beneficial effects of the technical solution provided by this application are:

[0019] The present application provides a method for associating data based on a cuckoo filter. The method is applied to a terminal, which is equipped with a cuckoo filter. The cuckoo filter processes the obtained to-be-associated identifier of the to-be-associated data to obtain the to-be-associated fingerprint; the cuckoo filter matches the to-be-associated fingerprint with the current fingerprint set, wherein the current fingerprint set is associated with the target data set; and according to the matching result, the association relationship between the to-be-associated data and the target data set is determined. The method is based on the fast query capability provided by the cuckoo filter and the advantage of the small memory resources required by the managed fingerprint set. On the one hand, the cached fingerprint set is matched with the to-be-associated data, wherein the cached fingerprint set does not require high memory resources, thereby avoiding the excessive memory resource requirements that may result from directly caching the target data set, thereby causing waste of memory resources and even affecting the performance of the terminal; on the other hand, by performing the matching operation through the cuckoo filter, it is possible to quickly determine whether the data has an association relationship with the target data set, thereby improving the matching rate between the data and the target data set, and quickly filtering out irrelevant data. From the perspective of real-time data access to external databases, the solution provided by the embodiments of the present application can prevent a large number of invalid network accesses and data queries, avoid situations where excessive memory resource requirements occur, and improve the hit rate of real-time data queries in external databases. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for describing the embodiments of the present application.

[0021] Figure 1 A flowchart of a method for associating data based on a cuckoo filter provided in an embodiment of the present application;

[0022] Figure 2 A schematic diagram of the structure of a system based on cuckoo filter association data provided in an embodiment of the present application;

[0023] Figure 3 A schematic structural diagram of a device based on cuckoo filter association data provided in an embodiment of the present application. DETAILED DESCRIPTION

[0024] The following describes in detail embodiments of the present application, examples of which are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present application, and are not to be construed as limiting the present application.

[0025] First, several terms involved in this application are introduced and explained:

[0026] The process of obtaining fingerprint information from a Cuckoo filter: First, an element is the basic unit of information obtained by the Cuckoo filter. For example, a string can be used as an element. A hash operation is performed on the element to obtain a hash value. A fixed number of digits of this hash value is intercepted as the fingerprint information of the element. Typically, the memory space required for fingerprint information is only one byte. Therefore, when generating corresponding fingerprint information for a large number of elements, the memory space required is not large and can generally be satisfied.

[0027] Cuckoo filter: A highly efficient filter that can manage fingerprint sets composed of large amounts of fingerprint information. When performing a specific filtering operation, the element to be matched is first algorithmically processed to obtain the corresponding fingerprint information to be matched. Then, a matching operation is performed on this fingerprint information based on a preset data structure and algorithm to determine whether the stored fingerprint information contains the fingerprint information to be matched, and further, whether the element to be matched exists in the large number of elements corresponding to the stored fingerprint information. Furthermore, since fingerprint information only contains partial information about an element, during actual matching, it is possible that a match is successful, but the stored element corresponding to the fingerprint information is not actually equal to the element to be matched. Such a matching result is also called a false positive. Experiments have found that false positive results occur in approximately 2% of data.

[0028] HASH table: A hash table is a data structure that is directly accessed based on a key value. A key value (for example, a unique identifier for data stored in a database) can be mapped to a location in a table to access records, speeding up lookups.

[0029] Memory association solution: When the real-time processing program starts, some external data is preloaded into the real-time processing program's memory, thereby establishing an association relationship between the real-time processing program and the external data. When the amount of external data is small, the association efficiency is relatively high. However, when the amount of external data is very large, it consumes a large amount of memory resources, resulting in reduced association efficiency. This is especially true when using a distributed framework for real-time processing (each process needs to associate with this external data), which places an even greater demand on memory resources. Therefore, this solution can easily lead to the temporary use and waste of a large amount of computing resources.

[0030] External middleware association solution: Set up a Nosql (also known as a non-relational database) between the terminal and external data. Use this Nosql to cache external data, and multiple real-time processing processes in the terminal access external data by accessing this Nosql. Nosql stores external data, achieves the effect of sharing memory data, and reduces the occupation of memory resources. However, in some association scenarios, for example: in real-time marketing scenarios, it is necessary to associate relevant information of some target user groups. The real-time push messages may involve the real-time behavior of tens of millions of users, while the target marketing users are only tens of thousands. Since the data that needs to be associated is in Nosql and not in the memory of the real-time processing process, all data needs to be pushed to Nosql for association to confirm whether it is the target user. This will generate a large amount of invalid network access and data query.

[0031] The present application provides a method, apparatus, device and storage medium based on cuckoo filter association data, which aims to solve the above technical problems in the prior art.

[0032] It should be noted that the method of the embodiment of the present application is applied to a terminal, and various real-time processing processes can be run on the terminal, and the real-time processing process corresponds to an instance of the application on the terminal. When the application is a shopping application, the real-time data processed by the real-time processing process can be commodity data; when the application is a news application, the real-time data processed by the real-time processing process can be news data; when the application is a traffic application, the real-time data processed by the real-time processing process can be vehicle data. As long as there is a need to query data for various applications on the application market, they are applicable to the method provided in the embodiment of the present application. Therefore, the embodiment of the present application does not impose any restrictions on the type of application and the type of real-time data.

[0033] The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments. To make the purpose, technical solutions and advantages of this application clearer, the embodiments of this application will be described below with reference to the accompanying drawings.

[0034] See also Figure 1 The embodiment of the present application provides a possible flow diagram of a method for associating data based on a cuckoo filter. The method is applied to a terminal configured with a cuckoo filter, and the method includes:

[0035] S110, processing the to-be-associated identifier of the acquired to-be-associated data through a cuckoo filter to obtain a to-be-associated fingerprint;

[0036] When any real-time processing process in a terminal is in operation, it receives a message queue containing multiple access requests carrying real-time data. The access requests are used to indicate the determination of the association relationship between the real-time data and the target dataset. The access requests also carry the process identifier of the real-time processing process to be processed. After determining the corresponding real-time processing process based on the process identifier, the real-time processing process sequentially retrieves the access requests from the message queue, obtains the multiple pieces of real-time data carried in the access requests, determines the multiple pieces of real-time data as multiple pieces of data to be associated, and determines the unique identifier of each piece of real-time data as the to-be-associated identifier.

[0037] Furthermore, after determining the data to be associated and the identifier to be associated, the identifier to be associated is processed by the cuckoo filter to obtain the fingerprint to be associated. The process of obtaining the fingerprint to be associated according to the identifier to be associated by the cuckoo filter in the embodiment of the present application can refer to the existing technology and will not be repeated here for the sake of simplicity.

[0038] S120, matching the fingerprint to be associated with the current fingerprint set through a cuckoo filter, wherein the current fingerprint set is associated with the target data set;

[0039] Among them, the fingerprint set is managed by the cuckoo filter.

[0040] S130: Determine the association relationship between the data to be associated and the target data set based on the matching result.

[0041] Specifically, if the matching result indicates a match failure, the real-time data corresponding to the data to be associated is determined to be invalid, indicating that invalid data has been screened out from the multiple real-time data sets. Furthermore, by pre-screening invalid data during real-time processing, there is no need to access an external database to query the real-time data, improving the efficiency of data association operations. This technical approach can effectively address the large number of invalid network accesses and data queries that can occur in real-time marketing scenarios.

[0042] The present application provides a method for associating data based on a cuckoo filter, based on the fast query capability provided by the cuckoo filter and the advantage of the small memory resources required by the managed fingerprint set. On the one hand, the cached fingerprint set is matched with the data to be associated, wherein the cached fingerprint set does not require high memory resources, thereby avoiding the excessive memory resource requirements caused by directly caching the target data set, thereby wasting memory resources and even affecting the performance of the terminal; on the other hand, by performing matching operations through the cuckoo filter, it is possible to quickly determine whether the data has an association relationship with the target data set, thereby improving the matching rate between the data and the target data set, and quickly filtering out irrelevant data. From the perspective of real-time data accessing an external database, the solution provided by the embodiment of the present application can prevent a large number of invalid network accesses and data queries, avoid the situation of excessive memory resource requirements, and improve the hit rate of real-time data queries in external databases.

[0043] In a possible implementation, after the terminal starts the real-time processing process, it also needs to start and initialize the cuckoo filter. Only after the initialization operation can the real-time data be obtained and the corresponding matching operation be performed. The process of initializing the cuckoo filter includes:

[0044] Load the state change queue in the external database, where the state change queue includes the identifier of each data in the target dataset;

[0045] Build the current fingerprint set based on the state change queue through the cuckoo filter.

[0046] The embodiment of the present application may adopt a technical means of setting up an external database between the terminal and the target data set. The external database may also include a state change queue corresponding to the target data set, which is used to record the data in the target data set in real time.

[0047] The target data set may be part of the data stored in a target database, which may be a relational database. In addition, the external database may be a non-relational database (ie, a NoSQL database) that supports key-value storage.

[0048] Exemplarily, the target database may be an Oracle database, DB2, MySQL, etc., and the external database may be Redis.

[0049] In addition, when the real-time processing process is started, the external database and the target database can establish a connection. After the connection is established, the external database can synchronize the target data in the target database. The synchronization process can specifically include the process of synchronizing data tables and synchronizing state change queues. In the process of synchronizing data tables, a HASH operation is performed on the acquired target data set to obtain the data table corresponding to the target data set. Therefore, the data table is essentially a HASH table. The content stored in the data table and the target data set is essentially the same, except that the feature of the data table is that special processing is performed on the identifier of each data item to facilitate fast query of the data table. Compared with ordinary data tables, querying data through data tables will be faster.

[0050] In addition, each data item in the target dataset may include a keyword or identifier to uniquely identify the data item, and each data item in the target dataset may also include other attribute information. Therefore, the data table and state change queue should at least include the identifier of each data item in the target dataset. The data table and state change queue may also include other attributes of each data item in the target dataset. This application does not restrict whether or not to include other attributes.

[0051] Illustratively, the structure of the target data set may be as shown in Table 1 below, where the product ID is the identifier of the target data.

[0052]

[0053] The data table obtained from the target data set in Table 1 can be shown in Table 2:

[0054]

[0055] Table 2 Data sheet

[0056] In a possible implementation, the method further includes:

[0057] Send a subscription request to the external database to subscribe to the status change queue.

[0058] The real-time processing process may send a subscription request when the state change queue is initially loaded, may send a subscription request when the cuckoo filter is initialized, or may send a subscription request at other times.

[0059] When the target dataset stored in the target database changes, for example, when new data is added or removed from the target dataset, the identifier of the new data in the target dataset is generated or removed accordingly. Since the state change queue stores the identifier of each data item in the target dataset, after the target dataset is updated, the target database will send an update message to the external database. This update message is used to notify the target database to update the hash table and state change queue.

[0060] Furthermore, after receiving the update message sent by the target database, the external database obtains information such as the new target dataset and performs an update operation on the data table and the state change queue. After the update operation is completed, the data table and the state change queue both include the identifier of each data in the current target dataset.

[0061] In addition, if the external database sends a subscription message to the terminal, the method further includes:

[0062] When receiving a subscription message sent by an external database, the updated state change queue is reloaded, and the current fingerprint set is updated according to the updated state change queue through the cuckoo filter. The subscription message is sent by the external database after updating the state change queue when determining that the target data set is updated.

[0063] Since the cuckoo filter has the problem of false positives, the data to be associated that is successfully matched by the cuckoo filter may not exist in the target dataset.

[0064] The present application embodiment also provides a possible implementation method, in which a verification operation is required when a match is successful, so as to screen out such false positive data. Specifically:

[0065] When the match is successful, a verification request carrying the identifier to be associated is sent to the external database to instruct the external database to verify the identifier to be associated according to the data table;

[0066] When the verification result fed back by the external database is successful, it is determined that the data to be associated has an association relationship with the target data set in the target database.

[0067] The verification request may also carry the data to be associated, that is, in addition to the identifier to be associated, it may also include other attribute data of the data to be associated.

[0068] Specifically, after receiving the verification request, the external database obtains the identifier to be associated, and then performs a verification operation on the identifier to be associated according to the data table. When it is determined that the identifier to be associated exists in the data table, it can be indicated that the data to be associated exists in the data set; when it is determined that the identifier to be associated does not exist in the data table, it can be indicated that the data to be associated does not exist in the data set. Since the external database performs the verification operation based on the complete information of the identifier to be associated, it can be determined that the result of the verification operation performed by the external database is reliable. The re-verification operation of the external database can effectively solve the problem of adding positives when matching data in the cuckoo filter, and the external database actually performs a re-verification operation on the identifier to be associated through a HASH table. As we all know, the query operation on the HASH table is very efficient. Therefore, performing the verification operation through the external database can further improve the filtering efficiency of real-time data.

[0069] See also Figure 2 The present embodiment also provides a possible structural diagram of a system based on cuckoo filter-associated data. This system 20 includes three independent modules: a terminal 210, an external database 220, and a target database 230. The external database 220 is connected to the terminal 210 and the target database 230, respectively. The terminal 210 may include a real-time processing process, which actually instantiates and manages the cuckoo filter configured on the terminal 210. The target database 230 includes a target dataset, and the external database 220 includes a data table and a state change queue generated based on the target dataset. The system's workflow may include S2001-S2008:

[0070] S2001: The terminal 210 starts a real-time processing process.

[0071] S2002: The real-time processing process constructs a cuckoo filter instance and initializes the cuckoo filter.

[0072] Specifically, after the real-time processing process constructs the cuckoo filter instance, it sends a request to the external database 220 to obtain the state change queue.

[0073] S2003: The external database 220 feeds back a status change queue.

[0074] Before external database 220 returns a state change queue, it must send a request to target database 230 to obtain the target dataset. After receiving the target dataset from target database 230, it performs a hash operation on the target dataset to obtain a data table. Alternatively, the target database performs a hash operation on the target dataset to obtain a data table and returns the data table. Alternatively, target database 230 can also return a state change queue, or external database 220 can create a state change queue based on the target dataset.

[0075] The external database 220 feeds back the state change queue to the terminal 210 .

[0076] S2004: The cuckoo filter constructs a fingerprint set based on the state change queue.

[0077] Specifically, terminal 210 receives a state change queue and assigns it to the real-time processing process for processing. The real-time processing process assigns the assigned state change queue to the Cuckoo filter, which then constructs a fingerprint set based on the state change queue. The specific construction process can be found in the prior art and will not be detailed here for simplicity.

[0078] In addition, the real-time processing process of terminal 210 also subscribes to the state change queue by sending a subscription message to the external database 220. When the state change queue is updated (for example, data is added or subtracted), the update information is fed back to the real-time processing process of terminal 210. The real-time processing process of terminal 210 reloads the state change queue and updates the fingerprint set.

[0079] S2005: The real-time processing process receives the message queue assigned by the terminal 210 and processes the messages in sequence.

[0080] Specifically, the terminal 210 assigns a message queue including multiple access requests carrying real-time data to the real-time processing process. The real-time processing process processes the access requests in turn, obtains multiple real-time data carried in each access request, and assigns each real-time data to the cuckoo filter for processing.

[0081] S2006: The cuckoo filter performs a matching operation.

[0082] Specifically, after receiving a piece of real-time data, the Cuckoo filter processes the identifier of the real-time data (for example, the identifier can be a keyword or a unique identifier of the data) to obtain fingerprint information of the real-time data. The Cuckoo filter performs a matching operation based on the fingerprint set and the fingerprint information to obtain a matching result.

[0083] When the matching result is a match failure, subsequent operations are performed. This application does not impose any restrictions on the subsequent operations.

[0084] When the matching result is a successful match, the real-time data needs to be sent to the external database 220 for further verification.

[0085] S2007: The external database 220 performs a verification operation.

[0086] Specifically, after receiving the real-time data, the external database 220 obtains the identifier of the real-time data and queries the data table according to the identifier. If it is determined that the data table has the identifier of the data, it is determined that the data table contains the real-time data; if not, it is determined that the real-time data is false positive data.

[0087] The external database 220 feeds back the query results to the real-time processing process of the terminal 210 .

[0088] S2008: The real-time processing process receives the query result.

[0089] Specifically, after receiving the query result, the real-time processing process may perform subsequent operations, which are not subject to any restrictions in this application.

[0090] In addition, for the query of multiple real-time data, S2006-S2007 are repeatedly executed until the query of all real-time data is completed.

[0091] See also Figure 3 , the embodiment of the present application provides a possible structural diagram of a device based on cuckoo filter association data. Among them, the device 30 may include: a cuckoo filter 301, a determination module 302, wherein

[0092] The cuckoo filter 301 is used to process the to-be-associated identifier of the acquired to-be-associated data to obtain a to-be-associated fingerprint;

[0093] The cuckoo filter 301 is further used to match the fingerprint to be associated with the current fingerprint set, wherein the current fingerprint set is associated with the target data set;

[0094] The determination module 302 is used to determine the association relationship between the data to be associated and the target data set according to the matching result.

[0095] In a possible implementation, the apparatus further includes an initialization module 303, wherein the initialization module 303 is specifically configured to:

[0096] Load the state change queue in the external database, where the state change queue includes the identifier of each data in the target dataset;

[0097] Build the current fingerprint set based on the state change queue through the cuckoo filter.

[0098] In a possible implementation, the apparatus further includes a subscription module 304, wherein the subscription module 304 is specifically configured to:

[0099] Send a subscription request to the external database to subscribe to the status change queue.

[0100] In a possible implementation, the subscription module 303 is further configured to:

[0101] Receive subscription messages sent by external databases;

[0102] Reload the updated state change queue and update the current fingerprint set according to the updated state change queue through the cuckoo filter, wherein the subscription message is sent by the external database after the state change queue is updated when the target data set is determined to be updated.

[0103] In a possible implementation, the external database further includes a data table generated based on the target data set. The determining module 302 is specifically configured to:

[0104] When the match is successful, a verification request carrying the identifier to be associated is sent to the external database to instruct the external database to verify the identifier to be associated according to the data table;

[0105] When the verification result fed back by the external database is successful, it is determined that the data to be associated has an association relationship with the target data set in the target database.

[0106] The device based on cuckoo filter associated data provided in this embodiment can execute the method based on cuckoo filter associated data shown in the above embodiment of this application. The implementation principles are similar and will not be repeated here.

[0107] An embodiment of the present application provides an electronic device, the electronic device comprising: a processor; and

[0108] The memory is configured to store machine-readable instructions, which, when executed by the processor, enable the processor to perform operations corresponding to any one of the methods for associating data based on a cuckoo filter in the embodiments of the present application.

[0109] An embodiment of the present application provides a computer-readable storage medium for storing computer instructions. When the computer instructions are executed on a computer, the computer can perform operations corresponding to any one of the methods based on cuckoo filter association data in the embodiments of the present application.

[0110] Those skilled in the art will appreciate that, unless expressly stated otherwise, the singular forms "one", "said" and "the" used in the embodiments of the present application may also include plural forms. It should be further understood that the term "comprising" used in the specification of the present application refers to the presence of the features, integers, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or groups thereof. It should be understood that when we refer to an element as being "connected" or "coupled" to another element, it can be directly connected or coupled to the other element, or there may be intermediate elements. In addition, "connected" or "coupled" as used herein may include wireless connections or wireless couplings. The term "and / or" used herein includes all or any unit and all combinations of one or more associated listed items.

[0111] It should be understood that although the steps in the flowcharts of the accompanying drawings are shown in sequence as indicated by the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some of the steps in the flowcharts of the accompanying drawings may include multiple sub-steps or multiple stages, and these sub-steps or stages are not necessarily executed at the same time, but can be executed at different times, and their execution order is not necessarily sequential, but can be executed in turn or alternately with other steps or at least a portion of the sub-steps or stages of other steps.

[0112] The above descriptions are only partial embodiments of the present invention. It should be pointed out that ordinary technicians in this technical field can make several improvements and modifications without departing from the principles of the present invention. These improvements and modifications should also be regarded as within the scope of protection of the present invention.

Claims

1. A method for associating data based on a cuckoo filter, characterized in that: Applied to a terminal, the terminal being configured with a cuckoo filter, the method comprising: Processing the acquired to-be-associated identifier of the to-be-associated data through the cuckoo filter to obtain a to-be-associated fingerprint; Matching the fingerprint to be associated with a current fingerprint set through the cuckoo filter, wherein the current fingerprint set is associated with a target data set; wherein the current fingerprint set is cached in a memory, and the target data set is not cached in the memory; Determining the association relationship between the data to be associated and the target data set based on the matching result; Before obtaining the data to be associated, the method further includes: Loading a state change queue in an external database, wherein the state change queue includes an identifier of each data item in the target data set; Constructing the current fingerprint set according to the state change queue by the cuckoo filter; The external database further includes a data table generated according to the target data set, and determining the association relationship between the data to be associated and the target data set according to the matching result includes: When the match is successful, sending a verification request carrying the identifier to be associated to the external database to instruct the external database to verify the identifier to be associated according to the data table; When the verification result fed back by the external database is successful, determining that the data to be associated has an association relationship with the target data set in the target database; When the matching fails, there is no need to query the data to be associated by accessing the external database.

2. The method according to claim 1, characterized in that Also includes: A subscription request is sent to the external database to request subscription to the state change queue.

3. The method according to claim 2, characterized in that Also includes: When a subscription message sent by the external database is received, the updated state change queue is reloaded, and the current fingerprint set is updated according to the updated state change queue through the cuckoo filter, wherein the subscription message is sent by the external database after updating the state change queue when determining that the target data set is updated.

4. A device for associating data based on a cuckoo filter, characterized in that: The device includes a cuckoo filter, a determination module, wherein The cuckoo filter is used to process the to-be-associated identifier of the acquired to-be-associated data to obtain a to-be-associated fingerprint; The cuckoo filter is further configured to match the fingerprint to be associated with a current fingerprint set, wherein the current fingerprint set is associated with a target data set; wherein the current fingerprint set is cached in a memory, and the target data set is not cached in the memory; The determining module is configured to determine the association relationship between the data to be associated and the target data set according to the matching result; The device further includes an initialization module, which is specifically configured to: Loading a state change queue in an external database, wherein the state change queue includes an identifier of each data item in the target data set; Constructing the current fingerprint set according to the state change queue by the cuckoo filter; The external database further includes a data table generated according to the target data set. The determination module is specifically configured to: When the match is successful, sending a verification request carrying the identifier to be associated to the external database to instruct the external database to verify the identifier to be associated according to the data table; When the verification result fed back by the external database is successful, determining that the data to be associated has an association relationship with the target data set in the target database; When the matching fails, there is no need to query the data to be associated by accessing the external database.

5. The device according to claim 4, characterized in that The device further includes a subscription module, which is specifically configured to: A subscription request is sent to the external database to request subscription to the state change queue.

6. An electronic device, characterized in that: The electronic device comprises: processor; and A memory configured to store machine-readable instructions, wherein when the instructions are executed by the processor, the processor performs operations corresponding to the method for associating data based on a cuckoo filter according to any one of claims 1 to 3.

7. A computer-readable storage medium, characterized in that The computer-readable storage medium is used to store computer instructions. When the computer instructions are executed on a computer, the computer can perform operations corresponding to the method based on cuckoo filter association data according to any one of claims 1 to 3.

Images