Information retrieval methods, devices, and computer-readable storage media

Abstract

This application provides an information query method, apparatus, and computer-readable storage medium, relating to the field of network data processing technology. The method includes: querying a target memory within internal memory based on query information; if the corresponding target information is found in the target memory, then a first query result is returned indicating that the query information is not in an information table in external memory; if the corresponding target information is not found in the target memory, then a query is performed in an information table based on the query information; if the corresponding target information is found in the information table, then a second query result is returned indicating that the query information is in the information table; wherein, the information table stores multiple pieces of first information, and the target memory stores multiple pieces of second information that are mutually exclusive with the first information. This application stores different information in the information table and the target memory respectively, and performs the query sequentially, effectively improving the efficiency of querying large-capacity information tables.

Description

Technical Field

[0001] This application relates to the field of network data processing technology, and more specifically, to an information query method, apparatus, and computer-readable storage medium. Background Technology

[0002] In computer networks, large-capacity information tables occupy significant storage space. Since the entire information table cannot be stored in the CPU / chip's internal storage, such as the on-chip memory of a network processor, it must be stored in external, slower memory, such as DRAM. In network access control devices, such as network firewalls and network traffic monitoring devices, the information tables store flow-related information that requires control and sampling of specific traffic. However, the vast majority of search results are negative, meaning they cannot match the contents of the information table.

[0003] The speed at which the CPU / chip accesses external memory is slower than its access to on-chip memory. To improve information processing speed, a common method is to add a filtering operation before table lookup. This reduces the number of information tables accessed by network devices and application systems, thus increasing information processing capabilities. For example, Bloom filters or Cuckoo filters can be used to filter out information not found in the table. However, since the space occupied by the filter's data structure is directly proportional to the size of the information table, when the space occupied by the filter's data structure exceeds the available space in on-chip memory, or when the space occupied by each piece of information is reduced, the aforementioned information filtering may become unusable or its performance may deteriorate. This results in low efficiency for querying large information tables and fails to meet current network access control requirements. Summary of the Invention

[0004] In view of this, the purpose of this application is to provide an information query method, apparatus and computer-readable storage medium to improve the problem of low efficiency when querying information tables in the prior art.

[0005] To address the aforementioned problems, firstly, embodiments of this application provide an information query method, the method comprising:

[0006] The query is performed in the target memory within the internal memory based on the query information;

[0007] If the corresponding target information is found in the target memory, the first query result is returned, indicating that the queried information is not in the information table in the external memory.

[0008] If the corresponding target information is not found in the target memory, then a query is performed in the information table based on the query information;

[0009] If the corresponding target information is found in the information table, the second query result of the query information in the information table will be returned.

[0010] The information table stores multiple pieces of first information, and the target memory stores multiple pieces of second information that are mutually exclusive with the first information.

[0011] In the above implementation process, mutually exclusive information is stored in the target memory in the internal memory and the information table in the external memory. Since the query efficiency in the internal memory is higher than that in the external memory, the sequential query method of first querying the target memory in the internal memory and then querying the information table in the external memory improves the validity and accuracy of various query results. It can also effectively improve the efficiency of querying whether the query information is in the information table, thereby optimizing the performance of network security devices and application systems when performing information queries and meeting various network access control requirements.

[0012] Optionally, the first information is stored in the information table according to control requirements;

[0013] The second information is stored in the target memory according to the information table.

[0014] In the above implementation process, to prevent data conflicts during queries in the information table and the target memory, different information can be stored in the information table and the target memory. The first information stored in the information table is determined by the network access control requirements, while the information stored in the target memory is determined by the information table; that is, the target memory stores data that does not exist in the information table. By storing mutually exclusive data in the information table and the target memory respectively, duplicate queries can be reduced, thereby improving the accuracy of the first and second query results.

[0015] Optionally, the information table includes a first storage table and a second storage table;

[0016] The first information is stored in the information table in the following manner:

[0017] Based on the control requirements, multiple original pieces of the first information are determined;

[0018] Based on the hash function and the fingerprint information of each piece of first information, determine the first position of each piece of first information in the first storage table and the second position in the second storage table;

[0019] Based on the first position or the second position, the corresponding first information is stored.

[0020] In the above implementation process, to reduce conflicts during data storage and retrieval in the information table, two identical storage tables can be set up. Based on network access control requirements, multiple original first pieces of information with specific content are identified. Then, based on a hash function and the fingerprint information of each first piece of information, calculations are performed to determine two storage locations for each piece of information in the two storage tables. A suitable location is then selected between the two storage locations to store the original first piece of information, effectively improving the efficiency and effectiveness of storing the first piece of information in the information table. Furthermore, since the original first pieces of information contain specific information content and occupy a large amount of storage space, the information table can be stored in external storage with lower query efficiency, serving as the target for a second query.

[0021] Optionally, the target memory includes a third storage table and a fourth storage table;

[0022] The second information is stored in the target memory in the following manner:

[0023] Identify multiple unstored pieces of information that are mutually exclusive with the first piece of information;

[0024] Extract the original fingerprint information that was not stored as the second information;

[0025] The third position of each piece of the second information in the third storage table and the fourth position in the fourth storage table are determined based on the hash function.

[0026] The corresponding second information is stored based on the third or fourth position.

[0027] In the above implementation process, to reduce conflicts during data storage and retrieval in the target memory, two identical storage tables can be set up in the target memory. By identifying non-repeating and mutually exclusive unstored information that is not duplicated with the first information, compressed fingerprint information of multiple unstored information is extracted as the second information. Then, based on a hash function and each piece of second information, calculations are performed to determine the two storage locations of each piece of second information in the two storage tables. A suitable location is then selected from the two storage locations to store the compressed second information, effectively improving the efficiency and effectiveness of storing the second information in the target memory. Furthermore, since the second information is compressed information containing specific information content, it occupies less storage space, reducing the storage space requirements of the target memory. Therefore, the target memory can be stored in the internal memory with higher query efficiency as the target for the first query, improving query efficiency.

[0028] Optionally, the step of querying the target memory in the internal memory based on the query information includes:

[0029] Based on the query information, determine the corresponding query fingerprint information;

[0030] Based on the hash function and the query fingerprint information, the corresponding first query position is determined in the target memory;

[0031] If the second information stored in the first query location is the same as the query fingerprint information, then the second information is used as the queried target information.

[0032] In the above implementation process, when querying the target memory based on the query information, the corresponding query fingerprint information can be determined first. This fingerprint information is then calculated using a hash function to determine the corresponding first query position in the target memory. The second information stored in the first query position is then compared with the query fingerprint information. The second information that matches the query fingerprint information is considered the found target information. If the second information differs from the query fingerprint information, it indicates that the target information was not found in the target memory. This allows for the determination of the theoretical storage location of the query information, enabling compressed data comparison and effectively improving the efficiency and accuracy of queries in the target memory.

[0033] Optionally, the step of querying the information table based on the query information includes:

[0034] Based on the query information, determine the corresponding query fingerprint information;

[0035] Based on the hash function and the query fingerprint information, the corresponding second query position is determined in the information table;

[0036] If the first information stored in the second query location is the same as the query information, then the first information is used as the target information retrieved.

[0037] In the above implementation process, if the corresponding target information is not found in the target memory, a secondary query can be performed on the information table based on the query information. During the secondary query, the information can first be used to confirm...

[0038] The corresponding query fingerprint information is determined, and the query fingerprint information is calculated according to a hash function. The corresponding second query position is determined in the information table, and the first information stored in the second query position is compared with the original query information. The first information that matches the query information is the target information found.

[0039] Information. If the first piece of information differs from the query information, it indicates that the target information was not found in the information table. This allows for the determination of the theoretical storage location of the query information, enabling a comparison of the original data and effectively improving the efficiency and accuracy of queries in the information table.

[0040] Optionally, the method further includes:

[0041] If the corresponding target information is not found in the information table, then the verification fingerprint information of each piece of the first information stored in the second query location is obtained;

[0042] 5. If the query fingerprint information is different from all the verification fingerprint information, then the corresponding storage location of the query fingerprint information in the target memory is determined according to the hash function;

[0043] The query fingerprint information is stored in the storage location of the target memory to update the query information in the target memory.

[0044] In the above implementation process, if the corresponding target information is not found in the information table, it indicates that the 0 query information is neither stored in the target memory nor in the information table. To expand the data range during the query and improve query efficiency, the verification fingerprint information of each piece of first information stored in the second storage location can be compared with the query fingerprint information to verify whether there are false negative conflicts. If the query fingerprint information is different from all verification fingerprint information, i.e., not...

[0045] In the event of a conflict, the corresponding storage location of the query fingerprint information in the target memory is calculated according to the hash function, and the query fingerprint information is then stored in the corresponding storage location in the target memory.

[0046] This allows for updating of unsearched information in the target memory. It effectively expands the data range in the target memory by updating the data based on the actual query results. This enables the direct determination of query results in the target memory when querying the same information again, further improving query efficiency.

[0047] Optionally, the method further includes:

[0048] Based on the update requirements of the information table, determine the update information;

[0049] The update information is used to query the target memory;

[0050] If the second information corresponding to the updated information is not found in the target memory, the updated information is added to the information table for updating.

[0051] If the second information corresponding to the updated information is found in the target memory, the second information is deleted, and the updated information is added to the information table for updating.

[0052] In the above implementation process, since the data in the information table may undergo updates and other changes, when updating the information table, a query can first be performed in the target memory based on the updated information to determine if there is corresponding second information in the target memory. If not, it indicates that the updated information is not stored in the target memory, and the updated information can be directly added to the information table for updating. If it exists, it indicates that the updated information is stored in the target memory, and the second information stored in the target memory needs to be deleted before the updated information is added to the information table for updating. This real-time updating of the information table ensures that there is no duplicate data between the information table and the target memory, thus guaranteeing the validity and accuracy of the query results.

[0053] Secondly, embodiments of this application also provide an information query device, the device comprising: a first query module, a first feedback module, a second query module, and a second feedback module;

[0054] The first query module is used to perform a query in the target memory in the internal memory based on the query information;

[0055] The first feedback module is used to provide feedback on the first query result of the information table in which the query information is not in the external memory when the first query module finds the corresponding target information in the target memory.

[0056] The second query module is used to query the information table according to the query information when the first query module does not find the corresponding target information in the target memory;

[0057] The second feedback module is used to provide feedback on the second query result of the query information in the information table when the second query module finds the corresponding target information in the information table;

[0058] The information table stores multiple pieces of first information, and the target memory stores multiple pieces of second information that are mutually exclusive with the first information.

[0059] In the above implementation process, mutually exclusive different information is stored in the target memory in the internal memory and the information table in the external memory. The first query module first queries the target memory in the internal memory, and the first feedback module returns the corresponding first query result based on the query results of the first query module. The second query module performs a second query in the information table in the external memory, and the second feedback module returns the corresponding second query result based on the query results of the second query module.

[0060] Thirdly, embodiments of this application also provide a computer-readable storage medium storing computer program instructions. When the computer program instructions are read and executed by a processor, the steps in any of the above-described information query methods are performed.

[0061] In summary, the embodiments of this application provide an information query method, apparatus, and computer-readable storage medium, which store different information in an information table in an external memory and a target memory in an internal memory. By first querying the target memory, which has a faster query speed, and then querying the information table if no corresponding data is found, the efficiency of querying a large information table is effectively improved. Attached Figure Description

[0062] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings used in the embodiments of this application will be briefly introduced below. It should be understood that the following drawings only show some embodiments of this application and should not be regarded as a limitation of the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0063] Figure 1 A block diagram illustrating an electronic device provided in an embodiment of this application;

[0064] Figure 2 A flowchart illustrating the first information query method provided in this application embodiment;

[0065] Figure 3 A flowchart illustrating the second information query method provided in this application embodiment;

[0066] Figure 4 A flowchart illustrating the third information query method provided in this application embodiment;

[0067] Figure 5 A detailed flowchart of step S200 provided for an embodiment of this application;

[0068] Figure 6A detailed flowchart of step S400 provided for an embodiment of this application;

[0069] Figure 7 A flowchart illustrating the fourth information query method provided in this application embodiment;

[0070] Figure 8 A flowchart illustrating the fifth information query method provided in this application embodiment;

[0071] Figure 9 This is a schematic diagram of the structure of an information query device provided in an embodiment of this application.

[0072] Icons: 100 - Electronic device; 111 - Memory; 112 - Memory controller; 113 - Processor; 114 - Peripheral interface; 115 - Input / output unit; 116 - Display unit; 800 - Information query device; 810 - First query module; 820 - First feedback module; 830 - Second query module; 840 - Second feedback module. Detailed Implementation

[0073] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them. All other embodiments obtained by those skilled in the art based on the embodiments of this application without creative effort are within the scope of protection of the embodiments of this application.

[0074] In computer networks, network devices and application systems need to access large amounts of information tables. For example, Internet routers need to look up routing tables containing millions of routes to determine the forwarding path of data packets; large network address translation (NAT) devices need to implement hundreds of millions of connection tables to track the status of active connections and perform network address translation based on the status of the connections; and network security devices such as firewalls need to look up large access control lists to control network access.

[0075] Therefore, network devices and application systems require high-speed access to large-capacity information tables for network access control. However, these large information tables occupy significant storage space, and the entire table cannot be stored in the CPU / chip's internal storage, such as the on-chip memory of a network processor. Instead, it needs to be stored in external, slower memory, such as DRAM. Since CPU / chip access to external memory is slower than access to on-chip memory, and in network access control devices such as firewalls and network traffic monitoring devices, the information tables store flow-related information that requires control and sampling of specific traffic flows. Most lookup results are negative and cannot match the contents of the information tables. Therefore, to improve information processing speed, a common method is to add a filtering operation before table lookup. This reduces the number of information tables accessed by network devices and application systems, thereby increasing information processing capabilities. For example, Bloom filters (a probability-based data structure used to determine if an element is in a set) or Cuckoo filters (an improved Cuckoo Hash algorithm used to determine if an element is in a set) can filter out information not found in the table. While Bloom or Cuckoo filters can filter out information not in the table, the space occupied by each piece of information in the table increases with the size of the table. Therefore, the application of these filters is limited by the on-chip memory space of the CPU or chip. In other words, the storage space requirements of Bloom or Cuckoo filters restrict their application on network devices with limited on-chip storage or large external storage. For instance, in network security devices, flow tables need to share limited on-chip storage space with routing tables and ACLs. The on-chip memory space allocated to flow tables is limited, while the number of flow table entries to be monitored is enormous, making it impossible to apply Bloom or Cuckoo filters.

[0076] Therefore, the applicant has found that when the space occupied by the filter data structure exceeds the available space of the on-chip memory, or when the space occupied by each piece of information is reduced, the above-mentioned information filtering will become unavailable or the filtering performance will be poor, resulting in low efficiency in querying large-capacity information tables and failing to meet the current network access control requirements.

[0077] To address the aforementioned issues, this application provides an information query method applied to a server. The server can be an electronic device with logical computing capabilities, such as a personal computer (PC), tablet computer, smartphone, or personal digital assistant (PDA). It can store different information in a target memory within the internal memory and an information table in the external memory, thereby improving the efficiency of querying large-capacity information tables by first querying the target memory and then querying the information table.

[0078] Optionally, please refer to Figure 1 , Figure 1 This is a block diagram illustrating an electronic device according to an embodiment of this application. The electronic device 100 may include a memory 111, a memory controller 112, a processor 113, a peripheral interface 114, an input / output unit 115, and a display unit 116. Those skilled in the art will understand that... Figure 1 The structure shown is for illustrative purposes only and does not limit the structure of the electronic device 100. For example, the electronic device 100 may also include components that are more... Figure 1 The more or fewer components shown, or having the same Figure 1 The different configurations shown.

[0079] The aforementioned memory 111, memory controller 112, processor 113, peripheral interface 114, input / output unit 115, and display unit 116 are electrically connected directly or indirectly to each other to achieve data transmission or interaction. For example, these components can be electrically connected to each other through one or more communication buses or signal lines. The aforementioned processor 113 is used to execute executable modules stored in the memory.

[0080] The memory 111 can be, but is not limited to, Random Access Memory (RAM), Read Only Memory (ROM), Programmable Read-Only Memory (PROM), Erasable Programmable Read-Only Memory (EPROM), Electrically Erasable Programmable Read-Only Memory (EEPROM), etc. The memory 111 stores programs. After receiving execution instructions, the processor 113 executes the programs. The methods executed by the electronic device 100 as defined in any embodiment of this application can be applied to the processor 113, or implemented by the processor 113.

[0081] The aforementioned processor 113 may be an integrated circuit chip with signal processing capabilities. The processor 113 can be a general-purpose processor, including a Central Processing Unit (CPU), a Network Processor (NP), etc.; it can also be a digital signal processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field-Programmable Gate Array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. It can implement or

[0082] The user executes the methods, steps, and logic block diagrams disclosed in the embodiments of this application. The general-purpose processor can be a microprocessor or any conventional processor, etc.

[0083] The peripheral interface 114 described above couples various input / output devices to the processor 113 and the memory 111. In some embodiments, the peripheral interface 114, the processor 113, and the memory controller 112 can be implemented on a single chip. In other instances, they can be implemented on separate chips.

[0084] The input / output unit 115 described above is used to provide user input data. Input / output unit 115

[0085] Yes, but not limited to, mice and keyboards.

[0086] The aforementioned display unit 116 provides an interactive interface (e.g., a user interface) between the electronic device 100 and the user, or is used to display image data for the user's reference. In this embodiment, the display unit...

[0087] The unit can be a liquid crystal display (LCD) or a touch display. If it is a touch display, it can be a capacitive touchscreen or a resistive touchscreen that supports single-point and multi-point touch operations. Supporting single-point and multi-point touch operations means that the touch display can sense touch operations generated simultaneously from one or more locations on the touch display and pass the sensed touch operations to the processor for calculation and processing. In the embodiments of this application, the display unit 116 can display information stored in the target memory and the information table, as well as corresponding first query results and second query results, etc.

[0088] The electronic device in this embodiment can be used to execute various steps in the information query methods provided in the embodiments of this application. The implementation process of the information query methods is described in detail below through several embodiments.

[0089] Please see Figure 2 , Figure 2 This is a flowchart illustrating the first information query method provided in an embodiment of this application. The method may include steps S200-S500.

[0090] Step 5, S200: Perform a query in the target memory within the internal memory based on the query information.

[0091] The internal memory can include a target memory. The internal memory can be the CPU's internal memory or on-chip memory, such as SRAM. The target memory can be various types of cache (high-speed cache memory). During a query, the corresponding query information can be obtained by parsing the received data packet, and an initial query can be performed in the target memory based on the query information.

[0092] Step S300: If the corresponding target information is found in the target memory, the first query result of the information table in the external memory is returned.

[0093] The information table can be stored in external storage, such as DRAM. Since data stored in external storage occupies more space than data stored in internal storage, querying data in internal storage is more efficient than querying data in external storage. Therefore, by first querying internal storage and then querying external storage, the number of queries to the information table during the information retrieval process can be reduced, thereby improving the efficiency of information table queries on the network firewall device. The information table stores multiple pieces of first information, and the target storage stores multiple pieces of second information that are mutually exclusive with the first information; that is, there is no duplication of data between the information table and the target storage. Therefore, if the target storage finds the target information corresponding to the queried information, it indicates that the queried information is not in the information table, and the corresponding first query result can be returned.

[0094] Step S400: If the corresponding target information is not found in the target memory, then a query is performed in the information table based on the query information.

[0095] If the target information corresponding to the query information is not found in the target memory, it indicates that the query information is not in the target memory, and a second query needs to be performed in the information table to determine whether the query information is in the information table.

[0096] Step S500: If the corresponding target information is found in the information table, the second query result of the query information in the information table is returned.

[0097] If the corresponding target information is found in the information table, it indicates that the queried information is in the information table, and the corresponding second query result can be returned.

[0098] It should be noted that the first piece of information is stored in an information table according to control requirements; the second piece of information is stored in the target memory according to the information table. To prevent data conflicts during queries in the information table and the target memory, different information can be stored in the information table and the target memory. The first piece of information stored in the information table is determined by the network access control requirements, while the information stored in the target memory is determined by the information table; that is, the target memory stores data that does not exist in the information table. By storing mutually exclusive data in the information table and the target memory respectively, duplicate queries can be reduced, thereby improving the accuracy of the first and second query results.

[0099] exist Figure 2In the illustrated embodiment, the query is performed sequentially by first querying the target memory in the internal memory and then querying the information table in the external memory. This improves the effectiveness and accuracy of various query results and also effectively improves the efficiency of querying whether the query information is in the information table. This optimizes the performance of network security devices and application systems when querying information and meets various network access control requirements.

[0100] Optionally, to reduce conflicts during data storage and retrieval in the information table, two storage tables can be set up within the information table; that is, the information table can include a first storage table and a second storage table. The first and second storage tables can have the same structure; for example, both storage tables can have the same number of rows, and each row can have four storage units, each capable of storing one piece of first information. Please refer to [link / reference]. Figure 3 , Figure 3 This is a flowchart illustrating a second information query method provided in an embodiment of this application. The method may further include steps S610-S630.

[0101] Step S610: Determine multiple original first pieces of information based on control requirements.

[0102] Optionally, since the information table is stored in an external storage device with a large storage space, the data stored therein contains specific information content, and multiple original first pieces of information can be determined according to control requirements. For example, when the information table is an identity information table that needs to be intercepted on the network, the corresponding first pieces of information include various specific information such as the address and account to be intercepted.

[0103] Step S620: Based on the hash function and the fingerprint information of each first piece of information, determine the first position of each first piece of information in the first storage table and the second position in the second storage table.

[0104] The hash function may include a uniform hash function selected according to the size of the first storage table and the second storage table. It may first calculate the fingerprint information of each piece of first information, and use the fingerprint information as the query key when querying. Based on the hash function and the fingerprint information, the first position of the first information in the first storage table and the second position of the first information in the second storage table are calculated respectively.

[0105] Optionally, the first position can be denoted as h1(x), h1(x) = hash(x), and the second position can be denoted as h2(x), h2(x) = h1(x) xor hash(fp(x)), where fp(x) is the fingerprint information of the first information, fp(x) = hash2(x), and hash2 is another hash function different from the above hash function. For example, hash2 can be any available hash function, etc.

[0106] Step S630: Store the corresponding first information based on the first position or the second position.

[0107] Because it has two storage locations, and the first and second locations can be directly XORed to calculate the third location, it effectively reduces location conflicts when storing information and eliminates the need to calculate alternative locations for storing information elements. A blank storage unit can be selected from either the first or second location for storage. For example, if the storage unit corresponding to the first location already stores data, while the storage unit corresponding to the second location does not, the specific first information can be stored in the second location.

[0108] exist Figure 3 In the illustrated embodiment, the original first information can be calculated and stored in the information table according to its corresponding position, which effectively improves the efficiency and effectiveness of storing the first information in the information table.

[0109] Optionally, to reduce conflicts during data storage and retrieval in the target memory, two storage tables can be set up in the target memory; that is, the target memory may include a third storage table and a fourth storage table. The third and fourth storage tables can have the same structure; for example, both storage tables have the same number of rows, and each row has four storage units, each storage unit capable of storing one piece of second information. Please refer to [link to relevant documentation]. Figure 4 , Figure 4 This is a flowchart illustrating the third information query method provided in the embodiments of this application. The method may further include steps S640-S670.

[0110] Step S640: Identify multiple unstored pieces of information that are mutually exclusive with the first piece of information.

[0111] Among these, multiple related data can be mutually exclusive with the first information, that is, multiple original information that are not stored in the information table can be regarded as unstored information.

[0112] Step S650: Extract the original fingerprint information that has not been stored as the second information.

[0113] Since the target memory is stored in a relatively small internal memory, the data stored therein is fingerprint information that has been compressed to reflect the specific information content. The fingerprint information that is not stored can be extracted as the corresponding second information, which can be denoted as fp'(x). The calculation method for fp'(x) can be the same as that for fp(x), or it can be calculated using a corresponding hash function, such as hash2, i.e., fp'(x) = fp(x) = hash2(x).

[0114] Step S660: Determine the third position of each piece of second information in the third storage table and the fourth position in the fourth storage table based on the hash function.

[0115] The hash function may include a uniform hash function selected based on the size of the first storage table and the second storage table, using the second information as the query key when querying, and calculating the third position of the second information in the third storage table and the fourth position of the second information in the fourth storage table based on the hash function and the second information.

[0116] Optionally, the third position can be denoted as h3(x), h3(x) = hash(x), and the fourth position can be denoted as h4(x), h4(x) = h3(x) xor hash(fp'(x)).

[0117] Optionally, the third and fourth positions can be further divided into upper and lower parts, h31(x) and h32(x); h41(x) and h42(x). h31(x) serves as the position of the storage row of information x in the third storage table, and h42(x) serves as the position of the storage row of information x in the fourth storage table. The storage unit in the third storage table stores the second information fp'(x) = fp(x) and h12(x), and the storage unit in the fourth storage table stores the information fp'(x) = fp(x) and h22(x). By dividing the calculation results of h3(x) and h4(x) into upper and lower parts, using the upper part of the result as the storage location, and storing the lower part of the result together with fp(x) in the storage unit, the target memory can be applied to scenarios where high-speed memory space is limited, while maintaining the function of directly calculating another alternative location using XOR.

[0118] Step S670: Store the corresponding second information based on the third or fourth position.

[0119] Because it has two storage locations, and the third and fourth locations can be directly XORed to calculate the other location, it effectively reduces location conflicts when storing information and eliminates the need to calculate alternative locations for storing fingerprint information. A blank storage unit can be selected from either the third or fourth location for storage. For example, if the storage unit corresponding to the third location already stores data, but the storage unit corresponding to the fourth location does not, the corresponding compressed second information can be stored in the fourth location.

[0120] Optionally, since the second information is the corresponding fingerprint information, and the fingerprint information is a hash value obtained after compressing the original information, the storage space required when storing the second information can be effectively reduced, thereby reducing the space occupied by the target memory.

[0121] It should be noted that h1(x), h2(x), and fp(x) in the information table and h3(x), h4(x), and fp'(x) in the target memory can be calculated using the same method. That is, the target memory and the information table share a set of hash functions hash and hash2 with the same properties, which reduces the difficulty of finding suitable hash functions for the target memory and the information table respectively.

[0122] exist Figure 4 In the illustrated embodiment, the compressed second information can be located and stored in the target memory accordingly, effectively improving the efficiency and effectiveness of storing the second information in the target memory.

[0123] Optionally, please refer to Figure 5 , Figure 5 This is a detailed flowchart of step S200 provided in an embodiment of the present application. Step S200 may include steps S210-S230.

[0124] Step S210: Determine the corresponding query fingerprint information based on the query information.

[0125] When performing a query in the target memory based on the query information, the query fingerprint information corresponding to the query information can be determined first based on calculation methods such as hash functions.

[0126] Step S220: Based on the hash function and query fingerprint information, determine the corresponding first query position in the target memory.

[0127] Specifically, a hash function can be used to calculate the query fingerprint information, determining the corresponding first query position in the target memory. It should be noted that since the target memory can include two storage tables, the first query position can also include the positions of two rows from those two storage tables.

[0128] Step S230: If the second information stored in the first query location is the same as the query fingerprint information, then the second information is used as the target information found.

[0129] Specifically, multiple pieces of second information stored in the two storage rows of the first query position can be compared with the query fingerprint information to determine whether the second information is the same as the query fingerprint information. If the second information is the same as the query fingerprint information, it indicates that the query information is stored in the target memory. The second information that is the same as the query fingerprint information can be used as the corresponding key value, that is, the queried target information. If the second information is different from the query fingerprint information, it indicates that the query information is not stored in the target memory.

[0130] exist Figure 5 In the illustrated embodiment, the storage location of the query information theory can be determined, thereby enabling compressed data comparison and effectively improving the efficiency and accuracy of query results when querying in the target memory.

[0131] Optionally, please refer to Figure 6 , Figure 6 This is a detailed flowchart of step S400 provided in an embodiment of the present application. Step S400 may include steps S410-S430.

[0132] Step S410: Determine the corresponding query fingerprint information based on the query information.

[0133] If the corresponding target information is not found in the target storage, a secondary query can be performed on the information table based on the query information. During the secondary query, the query fingerprint information corresponding to the query information can be determined first based on calculation methods such as hash functions.

[0134] Step S420: Based on the hash function and query fingerprint information, determine the corresponding second query position in the information table.

[0135] Specifically, a hash function can be used to calculate the query fingerprint information, and the corresponding second query position can be determined in the information table. It should be noted that since the information table can include two storage tables, the second query position can also include the positions of two rows from those two storage tables.

[0136] Step S430: If the first information stored in the second query location is the same as the query information, then the first information is used as the target information found.

[0137] Specifically, multiple pieces of first information stored in the two storage rows of the second query position can be compared with the query information to determine whether the first information and the query information are the same. If the first information and the query information are the same, it indicates that the query information is stored in the information table, and the first information that is the same as the query information can be used as the corresponding key value, that is, the target information that is retrieved. If the first information and the query information are different, it indicates that the query information is not stored in the information table.

[0138] It should be noted that when querying the information table, corresponding filter tables can be set within the information table to quickly identify query results that are not present in the information table. Although the query filter table is also stored in external storage, its simple algorithm avoids frequent access to the information table stored in external storage, thus improving the query performance of large-capacity information tables.

[0139] exist Figure 6In the illustrated embodiment, the storage location of the query information theory can be determined, thereby enabling the comparison of the original data and effectively improving the efficiency and accuracy of the query results when querying the information table.

[0140] Optionally, please refer to Figure 7 , Figure 7 This is a flowchart illustrating the fourth information query method provided in this application embodiment. The method may further include steps S710-S730.

[0141] Step S710: If the corresponding target information is not found in the information table, then obtain the verification fingerprint information of each piece of first information stored in the second query position.

[0142] If the corresponding target information is not found in the information table, it indicates that the queried information is neither stored in the target storage nor in the information table. To broaden the data range during the query and improve query efficiency, the case where the queried information is not in the information table can be verified first. During verification, the verification fingerprint information of multiple pieces of first information stored in the two storage rows of the second query position can be obtained first.

[0143] Step S720: If the queried fingerprint information is different from all the verified fingerprint information, then the corresponding storage location of the queried fingerprint information in the target memory is determined according to the hash function.

[0144] This can be achieved by comparing each verified fingerprint with the queried fingerprint to verify whether there are any false negatives in the queried information.

[0145] Step S730: Store the query fingerprint information in the storage location of the target memory so as to update the query information in the target memory.

[0146] If the queried fingerprint information is different from all the verified fingerprint information, that is, if no first information in the characterization information table matches the queried information and there is no conflict, then the corresponding storage location of the queried fingerprint information in the target memory can be calculated according to the hash function, and the queried fingerprint information can be stored in the corresponding storage location in the target memory so as to update the relevant information that was not found in the target memory.

[0147] Optionally, the storage location can be the first query location determined when querying the query information in the target memory.

[0148] By adding information not found in the information table to the target memory, when subsequent data packets with the same information are received, the first query result not in the information table can be obtained simply by querying the target memory. Furthermore, since the target memory is stored in high-speed external storage, its query speed is fast, effectively avoiding the time-consuming operation of querying the information table each time, thus improving the efficiency of querying the information table.

[0149] exist Figure 7 In the illustrated embodiment, the data in the target memory can be updated based on the actual query results, effectively expanding the data range in the target memory. This allows the query results to be directly determined in the target memory when querying the same query information in the future, further improving the efficiency of the query.

[0150] Optionally, please refer to Figure 8 , Figure 8 This is a flowchart illustrating the fifth information query method provided in this application embodiment. The method may further include steps S740-S770.

[0151] Step S740: Determine the update information based on the update requirements of the information table.

[0152] Since the data in the information table may also undergo various changes such as updates, when updating the information table, the corresponding update information can be determined first according to the update requirements. The update requirements may be to intercept some information, such as to intercept network access to address xx, etc. Then the corresponding update information is the corresponding network address and identity information.

[0153] Step S750: Query the target memory based on the updated information.

[0154] Among these, the updated information can be queried in the target memory. The query method can be the same as the query method for querying information in the target memory, that is, to determine whether the updated information is stored in the target memory, which will not be elaborated further.

[0155] Step S760: If the second information corresponding to the update information is not found in the target memory, the update information is added to the information table for updating.

[0156] If the second information corresponding to the update information is not found in the target memory, which means that the update information is not stored in the target memory, the update information can be directly added to the information table for updating.

[0157] Step S770: If the second information corresponding to the update information is found in the target memory, the second information is deleted and the update information is added to the information table for updating.

[0158] If the target memory finds the second information corresponding to the updated information, indicating that the target memory stores the updated information, in order to ensure the mutual exclusion between the information table and the data stored in the target memory, the second information stored in the target memory needs to be deleted before the updated information is added to the information table for updating.

[0159] exist Figure 8 In the illustrated embodiment, while updating the information table in real time, it is ensured that there is no duplicate data between the information table and the target memory, so that the information in the information table cannot be found in the target memory, thus ensuring the validity and accuracy of the query results.

[0160] Please see Figure 9 , Figure 9 This is a schematic diagram of the structure of an information query device provided in an embodiment of this application. The information query device 800 may include: a first query module 810, a first feedback module 820, a second query module 830, and a second feedback module 840.

[0161] The first query module 810 is used to perform a query in the target memory in the internal memory based on the query information;

[0162] The first feedback module 820 is used to provide feedback on the first query result of the information table in the external memory when the first query module 810 finds the corresponding target information in the target memory.

[0163] The second query module 830 is used to query the information table based on the query information when the first query module 810 does not find the corresponding target information in the target memory.

[0164] The second feedback module 840 is used to provide feedback on the second query result in the information table when the second query module 830 finds the corresponding target information in the information table.

[0165] The information table stores multiple pieces of first information, and the target memory stores multiple pieces of second information that are mutually exclusive with the first information.

[0166] In an optional implementation, the first information is stored in an information table according to control requirements; the second information is stored in a target memory according to the information table.

[0167] In an optional implementation, the information table includes a first storage table and a second storage table; the information query device 800 may further include a first storage module for determining multiple original first information according to control requirements; determining a first position of each first information in the first storage table and a second position in the second storage table based on a hash function and fingerprint information of each first information; and storing the corresponding first information based on the first position or the second position.

[0168] In an optional implementation, the target memory includes a third storage table and a fourth storage table; the information query device 800 may further include a second storage module for determining multiple unstored information mutually exclusive with the first information; extracting fingerprint information of the original unstored information as second information; determining the third position of each piece of second information in the third storage table and the fourth position in the fourth storage table based on a hash function; and storing the corresponding second information based on the third or fourth position.

[0169] In an optional implementation, the first query module 810 is specifically used to: determine the corresponding query fingerprint information based on the query information; determine the corresponding first query position in the target memory based on the hash function and the query fingerprint information; if the second information stored in the first query position is the same as the query fingerprint information, then the second information is used as the queried target information.

[0170] In an optional implementation, the second query module 830 is specifically used to: determine the corresponding query fingerprint information based on the query information; determine the corresponding second query position in the information table based on the hash function and the query fingerprint information; if the first information stored in the second query position is the same as the query information, then the first information is used as the target information queried.

[0171] In an optional embodiment, the information query device 800 may further include a first update module, which is used to obtain the verification fingerprint information of each piece of first information stored in the second query location when the second query module 830 does not find the corresponding target information in the information table; if the query fingerprint information is different from all the verification fingerprint information, the corresponding storage location of the query fingerprint information in the target memory is determined according to the hash function; and the query fingerprint information is stored in the storage location of the target memory to update the query information in the target memory.

[0172] In an optional embodiment, the information query device 800 may further include a second update module, which is used to determine update information according to the update requirements of the information table; query the target memory according to the update information; if the second information corresponding to the update information is not found in the target memory, the update information is added to the information table for updating; if the second information corresponding to the update information is found in the target memory, the second information is deleted and the update information is added to the information table for updating.

[0173] Since the principle of the information query device 800 in this embodiment is similar to that of the aforementioned information query method, the implementation of the information query device 800 in this embodiment can refer to the description in the above-mentioned information query method, and the repeated parts will not be described again.

[0174] This application also provides a computer-readable storage medium storing computer program instructions. When the computer program instructions are read and executed by a processor, the steps of any of the information query methods provided in this embodiment are performed.

[0175] In the several embodiments provided in this application, it should be understood that the disclosed device can also be implemented in other ways. The device embodiments described above are merely illustrative; for example, the block diagrams in the accompanying drawings illustrate the possible architecture, functions, and operations of the device according to various embodiments of this application. In this regard, each block in the block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions marked in the blocks may occur in a different order than those marked in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagram, and combinations of block diagrams, can be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions.

[0176] In addition, the functional modules in the various embodiments of this application can be integrated together to form an independent part, or each module can exist independently, or two or more modules can be integrated to form an independent part.

[0177] If the aforementioned functions are implemented as software functional modules and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0178] The above description is merely an embodiment of this application and is not intended to limit the scope of protection of this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of protection of this application. It should be noted that similar reference numerals and letters in the following figures indicate similar items; therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures.

[0179] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any changes or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application.

[0180] It should be noted that, in this document, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising..." does not exclude the presence of additional identical elements in the process, method, article, or apparatus that includes said element.

Claims

1. An information retrieval method, characterized in that, The method includes: The query is performed in the target memory within the internal memory based on the query information; If the corresponding target information is found in the target memory, the first query result is returned, indicating that the queried information is not in the information table in the external memory. If the corresponding target information is not found in the target memory, then a query is performed in the information table based on the query information; If the corresponding target information is found in the information table, the second query result of the query information in the information table will be returned. The information table stores multiple pieces of first information, and the target memory stores multiple pieces of second information that are mutually exclusive with the first information. The step of querying the information table based on the query information includes: determining corresponding query fingerprint information based on the query information; determining a corresponding second query position in the information table based on a hash function and the query fingerprint information; if the first information stored in the second query position is the same as the query information, then the first information is used as the queried target information. The method further includes: if the corresponding target information is not found in the information table, obtaining the verification fingerprint information of each piece of the first information stored in the second query location; if the query fingerprint information is different from all the verification fingerprint information, determining the corresponding storage location of the query fingerprint information in the target memory according to the hash function; and storing the query fingerprint information in the storage location of the target memory to update the query information in the target memory.

2. The method according to claim 1, characterized in that, in, The first information is stored in the information table according to control requirements; The second information is stored in the target memory according to the information table.

3. The method according to claim 2, characterized in that, in, The information table includes a first storage table and a second storage table; The first information is stored in the information table in the following manner: Based on the control requirements, multiple original pieces of the first information are determined; Based on the hash function and the fingerprint information of each piece of first information, determine the first position of each piece of first information in the first storage table and the second position in the second storage table; Based on the first position or the second position, the corresponding first information is stored.

4. The method according to claim 2, characterized in that, in, The target memory includes a third storage table and a fourth storage table; The second information is stored in the target memory in the following manner: Identify multiple unstored pieces of information that are mutually exclusive with the first piece of information; Extract the original fingerprint information that was not stored as the second information; The third position of each piece of the second information in the third storage table and the fourth position in the fourth storage table are determined based on the hash function. The corresponding second information is stored based on the third or fourth position.

5. The method according to any one of claims 1-4, characterized in that, The step of querying the target memory in the internal memory based on the query information includes: Based on the query information, determine the corresponding query fingerprint information; Based on the hash function and the query fingerprint information, the corresponding first query position is determined in the target memory; If the second information stored in the first query location is the same as the query fingerprint information, then the second information is used as the queried target information.

6. The method according to claim 1, characterized in that, The method further includes: Based on the update requirements of the information table, determine the update information; The update information is used to query the target memory; If the second information corresponding to the updated information is not found in the target memory, the updated information is added to the information table for updating. If the second information corresponding to the updated information is found in the target memory, the second information is deleted, and the updated information is added to the information table for updating.

7. An information query device, characterized in that, The device includes: a first query module, a first feedback module, a second query module, a second feedback module, and a first update module; The first query module is used to perform a query in the target memory in the internal memory based on the query information; The first feedback module is used to provide feedback on the first query result of the information table in which the query information is not in the external memory when the first query module finds the corresponding target information in the target memory. The second query module is used to query the information table according to the query information when the first query module does not find the corresponding target information in the target memory; The second feedback module is used to provide feedback on the second query result of the query information in the information table when the second query module finds the corresponding target information in the information table; The information table stores multiple pieces of first information, and the target memory stores multiple pieces of second information that are mutually exclusive with the first information. The second query module is specifically used for: determining corresponding query fingerprint information based on the query information; determining a corresponding second query position in the information table based on the hash function and the query fingerprint information; if the first information stored in the second query position is the same as the query information, then the first information is used as the queried target information; The first update module is used to obtain the verification fingerprint information of each piece of the first information stored in the second query position when the second query module does not find the corresponding target information in the information table; if the query fingerprint information is different from all the verification fingerprint information, the corresponding storage position of the query fingerprint information in the target memory is determined according to the hash function; and the query fingerprint information is stored in the storage position of the target memory to update the query information in the target memory.

8. A computer-readable storage medium, characterized in that, The readable storage medium stores computer program instructions, which, when executed by a processor, perform the steps of the method according to any one of claims 1-6.

Images