A method for quickly matching electric energy meter data based on a bloom filter, a storage medium and an equipment

Abstract

The application discloses a kind of based on Bloom filter's electric energy meter data fast matching method, storage medium and equipment, it is related to data query technical field, it solves the technical problem that electric energy meter data matching needs frequently accessing external memory and the time of matching search is too long.Method includes: receiving and analyzing data request obtains the identification information of target data record;According to the identification information, query the Bloom filter in memory, if Bloom filter judges that identification information can exist, then find the matching node matched with identification information in skip list in memory, if find matching node, then according to the address mark in the node information of matching node, read target data record from external memory, and return target data record to data request end;If Bloom filter judges that identification information does not exist, or, if no matching node is found in skip list, then return no data to data request end.

Description

Technical Field

[0001] This invention relates to the field of data query technology, and in particular to a method, storage medium and device for fast matching of electricity meter data based on a Bloom filter. Background Technology

[0002] In power systems, resource-constrained devices with limited memory space, such as electricity meters, data concentrators (data collectors), and edge gateways with local caches of electricity meter data, typically store electricity meter data to handle frequent matching queries of meter data in business scenarios such as generating user electricity bills, grid monitoring and maintenance, and user inquiries about electricity consumption details. The main control chip (MCU, microcontroller) of these resource-constrained devices has a small amount of cache and RAM to store data being calculated; electricity meter data is usually stored in external non-volatile memory. Common methods for accessing and querying electricity meter data, such as linear search and binary search, require multiple accesses to external memory and matching each stored data record. Even if the data record is not stored in external memory, it is still necessary to access external memory and match each stored data record upon receiving a data request.

[0003] In the process of realizing this invention, the inventors discovered at least the following problems in the prior art: The matching of electricity meter data requires frequent access to external storage. Since the data reading speed of external storage is slow, accessing external storage takes up most of the time, which has become the speed bottleneck for matching electricity meter data from resource-constrained devices, resulting in excessively long electricity meter data lookup and matching time. Summary of the Invention

[0004] The purpose of this invention is to provide a method, storage medium, and device for rapid matching of electricity meter data based on a Bloom filter, thereby solving the technical problems of frequent access to external memory and excessively long matching search time in existing technologies. The various technical effects of the preferred solutions among the many technical solutions provided by this invention are detailed below.

[0005] To achieve the above objectives, the present invention provides the following technical solution: This invention provides a fast data matching method for electricity meters based on a Bloom filter. The method includes: receiving a data request sent by a data requesting end; parsing the data request to obtain the identification information of a target data record; querying a Bloom filter in memory based on the identification information, wherein the Bloom filter is used to store the hash mapping bit value of the identification information to determine whether the identification information may exist; if the Bloom filter determines that the identification information may exist, searching for a matching node matching the identification information in a skip list in memory; if the matching node is found, reading the target data record from external memory according to the address marker in the node information of the matching node, and returning the target data record to the data requesting end; wherein the address marker in the node information of each node in the skip list corresponds to the storage physical address of a data record stored in the external memory; if the Bloom filter determines that the identification information does not exist, or if the matching node is not found in the skip list, returning no data to the data requesting end.

[0006] Preferably, the identification information is the sequence number of the target data record; the Bloom filter includes a sequence number bit array; querying the Bloom filter in memory based on the identification information includes: calculating the hash value of the identification information; taking the hash value modulo the length of the sequence number bit array to obtain the hash mapping bit of the identification information in the sequence number bit array; if the hash mapping bit of the identification information in the sequence number bit array is 1, the Bloom filter determines that the identification information may exist; if the hash mapping bit of the identification information in the sequence number bit array is 0, the Bloom filter determines that the identification information does not exist.

[0007] Preferably, the identification information is the time information of the target data record, and the Bloom filter includes a first array for recording timestamps, a second array for recording hours, and a third array for recording minutes; querying the Bloom filter in memory based on the identification information includes: converting the time information into a timestamp, extracting the hours and minutes from the timestamps; calculating the hash values ​​of the timestamp, the hours, and the minutes respectively, and determining the hash mapping bit of the timestamp in the first array, the hash mapping bit of the hours in the second array, and the hash mapping bit of the minutes in the third array based on the hash values ​​of the timestamp, the hours, and the minutes respectively; if the hash mapping bit values ​​of the timestamp in the first array, the hours in the second array, and the minutes in the third array are all 1, then the Bloom filter determines that the identification information may exist; otherwise, the Bloom filter determines that the identification information does not exist.

[0008] Preferably, the skip list includes multiple nodes, each node corresponding to a data record stored in the external memory; the node information of the node includes a hierarchical linked list array, the identification information of the corresponding data record, and an address marker; wherein, the hierarchical linked list array is used to store the forward pointer and / or successor pointer of the node in each level of the skip list.

[0009] Preferably, when a new data record is received, the method further includes: extracting new identification information from the new data record; calculating the hash value of the new identification information, determining the hash mapping bit of the new identification information in the Bloom filter based on the hash value of the new identification information, and setting the hash mapping bit value to 1; storing the new data record in the external memory to obtain the storage physical address of the new data record, and obtaining a new address tag based on the storage physical address of the new data record; inserting a new node into the bottom-level linked list of the skip list based on the new identification information, and creating a hierarchical linked list array of the new node, wherein the node information of the new node includes the new identification information, the new address tag, and the hierarchical linked list array of the new node.

[0010] Preferably, the step of inserting a new node into the bottom-level linked list of the skip list according to the new identifier information and creating a hierarchical linked list array of the new node includes performing the following steps layer by layer, starting from the top-level linked list of the skip list: Step S341, starting from the starting query node of the current linked list, search to the right. If a node with identifier information greater than the new identifier information can be found, then the node preceding the first node with identifier information greater than the new identifier information in the current linked list is taken as the starting query node of the next linked list, and proceed to step S342. If no node can be found... If a node is found whose identifier information is greater than the new identifier information, then the last node of the current linked list is used as the starting query node of the next linked list, and the process proceeds to step S342; in step S342, if the current linked list is not the bottom linked list, then the process proceeds to the next linked list for query and returns to step S341; if the current linked list is the bottom linked list, then a new node is inserted after the determined starting query node of the next linked list, and a level height is randomly set for the new node within the level range of the skip list, and a level linked list array of the new node is constructed according to the determined starting query node of each linked list.

[0011] Preferably, the step of searching for a matching node in the skip list in memory that matches the identification information includes: searching for the matching node layer by layer starting from the top-level linked list of the skip list. In each layer of the linked list search: starting from the starting search node of the current layer of the linked list and searching to the right, comparing the identification information in the node information of each node with the identification information. If a node is found whose identification information in the node information is equal to the identification information, then the node is taken as the matching node. Otherwise, the node whose identification information in the last node information of the current layer of the linked list is less than the identification information is taken as the starting search node of the next layer of the linked list, and the search continues until the terminating node is found. The terminating node is the first node in the bottom layer of the linked list whose identification information in the node information is greater than the identification information.

[0012] Preferably, the address marker in the node information of the node is the address offset of the physical storage address of the corresponding data record in the external memory compared to the base address; the step of reading the target data record from the external memory according to the address marker in the node information of the matching node includes: superimposing the address marker in the node information of the matching node on the base address to obtain the physical storage address of the target data record in the external memory, and reading the target data record from the external memory according to the physical storage address.

[0013] The present invention also provides a computer-readable storage medium storing a computer program, which, when executed, implements a fast matching method for electricity meter data based on a Bloom filter provided by the present invention.

[0014] The present invention also provides a device for rapid matching of electricity meter data, comprising: one or more processors; a memory for storing one or more computer programs, wherein the one or more processors are configured to execute the one or more computer programs stored in the memory, so that the one or more processors execute a method for rapid matching of electricity meter data based on a Bloom filter provided by the present invention.

[0015] Implementing one of the above-described technical solutions of the present invention has the following advantages or beneficial effects: Upon receiving a data request, the system first queries the Bloom filter in memory based on the identifier information of the target data record. Only if the Bloom filter determines that the identifier information of the target data record may exist, it then searches the skip list in memory for a matching node that matches the identifier information of the target data record. Only after a matching node is found is the target record retrieved from external memory using the address marker in the node information and sent back to the data requesting end. This invention eliminates the need to access external memory when the Bloom filter determines that the identifier information of the target data record does not exist, or when a matching node cannot be found in the skip list in memory, and during the matching node search process. External memory is accessed only once after a matching node is found, significantly reducing the frequency of external memory access, extending the lifespan of external memory, and improving the speed of electricity meter data matching. Furthermore, both the Bloom filter and the skip list are stored in the memory of the execution entity, resulting in fast execution speed and further improving the speed of electricity meter data matching. This enables rapid electricity meter data retrieval. The Bloom filter and skip list occupy relatively little memory space and do not affect the normal operation of the execution entity. Attached Figure Description

[0016] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort. In the drawings: Figure 1 This is a flowchart illustrating the rapid data matching method for energy meters based on a Bloom filter according to Embodiment 1 of the present invention. Figure 2 This is a schematic diagram of the Bloom filter structure in Embodiment 1 of the present invention; Figure 3 This is a schematic diagram of the skip list data structure in Embodiment 1 of the present invention; Figure 4 This is a structural block diagram of the Bloom filter in Embodiment 2 of the present invention; Figure 5 This is a schematic diagram of the process for inserting new data records in Embodiment 3 of the present invention; Figure 6 This is a flowchart of inserting a new node into a skip list in Embodiment 3 of the present invention. Detailed Implementation

[0017] To make the objectives, technical solutions, and advantages of the present invention clearer, various exemplary embodiments described below will be referenced to the accompanying drawings, which form part of the exemplary embodiments, illustrating various exemplary embodiments that may be used to implement the present invention. Unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this disclosure. It should be understood that they are merely examples of processes, methods, and apparatuses consistent with some aspects of the present invention disclosed as detailed in the appended claims, and other embodiments may be used, or structural and functional modifications may be made to the embodiments listed herein without departing from the scope and spirit of the present invention.

[0018] In the description of this invention, it should be understood that the terms "center," "longitudinal," "lateral," etc., indicate the orientation or positional relationship based on the accompanying drawings, and are only for the convenience of describing the invention and simplifying the description, and do not indicate or imply that the referred element must have a specific orientation, or be constructed and operated in a specific orientation. The terms "first," "second," etc., are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. The term "multiple" means two or more. The terms "connected" and "linked" should be interpreted broadly, for example, they can be fixed connections, detachable connections, integral connections, mechanical connections, electrical connections, communication connections, direct connections, indirect connections through an intermediate medium, and can be the internal connection of two elements or the interaction relationship between two elements. The term "and / or" includes any and all combinations of one or more of the related listed items. Those skilled in the art can understand the specific meaning of the above terms in this invention according to the specific circumstances.

[0019] To illustrate the technical solution described in this invention, specific embodiments are described below, showing only the parts related to the embodiments of this invention.

[0020] Example 1: like Figure 1 As shown, this invention provides a method for fast matching of electricity meter data based on a Bloom filter. Please see [link / reference]. Figure 1 As shown, the method includes: Step S1: Receive a data request sent by the data requesting end, and parse the data request to obtain the identification information of the target data record; Step S2: Query the Bloom filter in memory based on the identification information. The Bloom filter stores the hash mapping bit value of the identification information to determine whether the identification information may exist. If the Bloom filter determines that the identification information may exist, search for a matching node in the skip list in memory that matches the identification information. If the matching node is found, read the target data record from the external memory according to the address marker in the node information of the matching node, and return the target data record to the data requesting end. The address marker in the node information of each node in the skip list corresponds to the physical storage address of a data record stored in the external memory. If the Bloom filter determines that the identification information does not exist, or if the matching node is not found in the skip list, it returns no data to the data requesting end.

[0021] The fast matching method for electricity meter data based on Bloom filters provided in this embodiment is not limited to resource-constrained devices with limited memory space in power systems, such as electricity meters, data concentrators (data collectors), and edge gateways with local caching of electricity meter data. The execution entity of the fast matching method for electricity meter data based on Bloom filters can be the main control chip (MCU) of the resource-constrained device.

[0022] In this embodiment, electricity meter data refers to various data records generated and stored by the electricity meter during operation. These records mainly include frozen data (electricity readings recorded at fixed time intervals for electricity billing and load analysis), event records (such as the occurrence time and status of abnormal events like power outages, voltage drops, phase loss, and opening of covers), and real-time measurement data (such as instantaneous values ​​of current voltage, current, active / reactive power, and power factor). Electricity meter data is typically stored in the external storage of devices such as electricity meters, concentrators, and edge gateways. The identification information for data records can be the record's sequence number (such as the serial number of a frozen data block) or time information (such as a frozen timestamp), essentially serving as the data record's ID, depending on the business query dimension.

[0023] In this embodiment, a Bloom filter is a probabilistic data structure that can accurately determine whether an element definitely does not exist and probabilistically determine whether it may exist with a very small memory space (usually a bit array). Figure 2The diagram illustrates the structure of the bit array in a Bloom filter. The bit array consists of N bits, where N is a positive integer. The hash mapping bit value represents the value (0 or 1) of the corresponding bit obtained by taking the modulo of the length of the bit array in the Bloom filter after calculating the identifier information using a preset hash function. The specific process of using the Bloom filter to store the hash mapping bit value of the identifier information to determine whether the identifier information may exist is as follows: The identifier information is converted into an integer using a preset hash function (such as FNV1a); the integer converted by the preset hash function is modulo the length of the bit set in the Bloom filter to obtain the hash mapping bit (generally one bit of the bit array) corresponding to the identifier information in the bit set; the value of the hash mapping bit corresponding to the identifier information in the bit set (i.e., the hash mapping bit value) is set to 1 (the bit array is 0 after initialization). Then, when the hash mapping bit value corresponding to the identifier information in the bit set is found to be 1, it indicates that the identifier information may exist; when the hash mapping bit value corresponding to the identifier information in the bit set is found to be 0, it indicates that the identifier information definitely does not exist.

[0024] In this embodiment, a Bloom filter is deployed in the memory (RAM) of a resource-constrained device to pre-filter out requests for non-existent data records before accessing slow external memory.

[0025] In this embodiment, the skip list is a multi-level ordered linked list structure. By building multiple levels of indexes on top of the ordered underlying linked list, it achieves near-binary search efficiency, with an average search time complexity of O(logN1), where N1 represents the total number of data records in the skip list, or the number of nodes. In this embodiment, the order of the underlying linked list is set from left to right, with the marker information in ascending order (serial number in ascending order, and time information from front to back).

[0026] In this embodiment, the skip list is also stored in the RAM of the resource-constrained device. The address marker in the node information of each node points to the physical storage location of the corresponding data record in the external memory. There is a one-to-one mapping relationship between the address marker and the physical storage address of the corresponding data record in the external memory, so as to avoid directly storing complex physical storage addresses in the memory and occupying more memory space. The external memory refers to a non-volatile storage device with an access speed much lower than that of the main memory. Its random read latency is usually thousands of times slower than that of the main memory. Therefore, this invention loads and stores the Bloom filter and the skip list in the main memory, so that the external memory is accessed only once after it is determined that the target data record is stored in the external memory (i.e., a matching node is obtained), which greatly reduces the number of accesses and improves the matching speed of the electricity meter data.

[0027] It should be noted that due to the volatility of RAM, Bloom filter and skip table data will be lost when resource-constrained devices are powered off. Therefore, in practical applications, the bit array of the Bloom filter and the node information of the skip table are pre-stored in external memory. Each time the device powers on, the loading module reads this data from external memory into RAM. During device operation, the Bloom filter and skip table are updated synchronously whenever a new data record from the electricity meter is added.

[0028] In this embodiment, a Bloom filter provides the first level of filtering, filtering out data requests corresponding to identification information not recorded by the Bloom filter. Since the Bloom filter may produce false positives, this invention incorporates a skip list for the second level of filtering. When the Bloom filter determines a possible match, a fast lookup is performed in the skip list. Only after a matching node is found is the external memory accessed once based on the address marker. Furthermore, nodes in the skip list are mapped to their physical storage addresses in the external memory via address markers, reducing memory usage and facilitating migration. Therefore, this embodiment significantly reduces the number of accesses to the external memory, improving the matching speed of electricity meter data.

[0029] As an optional implementation, the identification information is the sequence number of the target data record; the Bloom filter includes a sequence number bit array; querying the Bloom filter in memory based on the identification information includes: calculating the hash value of the identification information; taking the hash value modulo the length of the sequence number bit array to obtain the hash mapping bit of the identification information in the sequence number bit array; if the hash mapping bit of the identification information in the sequence number bit array is 1, the Bloom filter determines that the identification information may exist; if the hash mapping bit of the identification information in the sequence number bit array is 0, the Bloom filter determines that the identification information does not exist.

[0030] This implementation selects the serial number as the identification information of the data record, making full use of the natural uniqueness and ordered integer nature of the serial number. The Bloom filter only sets one hash function and one serial number array to record the identification information. Compared with the standard Bloom filter that uses multiple hash functions, the single hash design can reduce the amount of computation and reduce the overhead of resource-constrained equipment such as electricity meters, and achieve efficient first-level filtering. For the matching request of the data record corresponding to the non-existent identification information, it directly returns no data without querying the external memory.

[0031] It should be noted that the identification information can also be the time information recorded in the target data. Bloom filtering includes a time bit array. The process of recording and querying identification information using the time bit array can be referred to the above-mentioned sequence number bit array, and will not be repeated here. Moreover, the time information includes complex information such as year, month, day, hour, minute, and second. In order to reduce the false positive rate of the time bit array, the number of bits in the time bit array is relatively large, and the memory overhead may be greater than that of the sequence number bit array.

[0032] As an optional implementation, the skip list includes multiple nodes, each node corresponding to a data record stored in the external memory; the node information of the node includes a hierarchical linked list array, the identification information of the corresponding data record, and an address marker; wherein, the hierarchical linked list array is used to store the forward pointer and / or successor pointer of the node in each level of the skip list.

[0033] In this implementation, the total number of levels in the skip list can be set by the user based on the number of nodes, such as 8 or 16. The bottom-level linked list includes all nodes. The length of the level linked list array `list[]` is the total number of levels in the skip list. The i-th element `list[i]` stores the predecessor or successor pointer of the node in the i-th level linked list. The predecessor pointer points to the node preceding the node in the i-th level linked list, and the successor pointer points to the node following the node in the i-th level linked list. Due to the characteristics of skip lists, a node may only appear in some levels. For levels where the node does not appear, the two pointers stored in the corresponding element `list[i]` are initialized to point to the node itself to avoid dangling pointers and improve security.

[0034] In this embodiment, a corresponding node is established in memory for each data record. All node information of the skip list is stored in memory, avoiding repeated access to external storage to obtain the identification information in the node information used for comparison during querying. All comparison operations are completed in memory, and the address marker in the node information points to external storage, forming a two-level storage architecture of memory index and external storage, which optimizes the bottleneck problem of slow external storage access in resource-constrained devices.

[0035] As an optional implementation, the step of searching for a matching node that matches the identification information in a skip list in memory includes: Starting from the top-level linked list of the skip list, the matching node is searched level by level. In the search of each level of the linked list: Starting from the initial search node of the current linked list, the search proceeds to the right, comparing the identifier information in the node information of each node with the identifier information. If a node is found whose identifier information matches the identifier information, that node is taken as the matching node. Otherwise, the node whose identifier information in the last node information of the current linked list is less than the identifier information (this node represents the rightmost node among all nodes less than the identifier information in the current linked list) is taken as the starting search node of the next level linked list, and the search continues in the next level linked list until the terminating node is found. The terminating node is the first node in the underlying linked list whose identifier information is greater than the identifier information.

[0036] In this embodiment, the skip list, from top to bottom, includes the top-level linked list, ..., the i-th level linked list, ..., the bottom-level linked list. For example... Figure 3 As shown, the top-level linked list is the sparsest, containing only a very small number of all nodes, while the bottom-level linked list includes all nodes.

[0037] In this embodiment, the starting search node of the top-level linked list can be set as its starting node. Therefore, except for the top-level linked list, the starting search nodes of all layers are not fixed, but are the last node with less than the target identifier information located by the search of the previous layer. This avoids traversing from the beginning of each layer and skipping the search, which speeds up the search. Moreover, after entering the next layer with a denser node distribution, it can continue to finely approximate the matching node. When the search reaches the terminating node of the bottom-level linked list and no matching node with the same identifier information is found, it indicates that the search for the matching node has failed.

[0038] It should be noted that all comparison operations in the skip list are performed in memory and do not involve access to external memory. Therefore, even if multiple nodes need to be compared, the time overhead is much less than a random access to external memory.

[0039] Example 2: This embodiment also provides a fast matching method for electricity meter data based on a Bloom filter. The difference from Embodiment 1 lies in the structure of the Bloom filter. In electricity meter data matching queries, time-based matching is the most common business scenario (such as querying frozen electricity consumption at a specific time on a specific day, or event records for a specific minute). In traditional methods, regardless of whether data records exist at that time point, slow external memory must be accessed for binary search or linear scan, resulting in significant performance waste. This embodiment, by constructing a multi-granularity Bloom filter, can quickly identify most non-existent time queries, and the use of a multi-granularity bit array reduces the memory overhead of the Bloom filter.

[0040] Specifically, the identification information is set as the time information of the target data record, and the Bloom filter includes a first array of timestamps, a second array of hours, and a third array of minutes; The step of querying the Bloom filter in memory based on the identification information includes: The time information is converted into a timestamp, and the hours and minutes are extracted from the timestamp; Calculate the hash values ​​of the timestamp, the hour, and the minute respectively, and determine the hash mapping bit of the timestamp in the first digit array, the hash mapping bit of the hour in the second digit array, and the hash mapping bit of the minute in the third digit array based on the hash values ​​of the timestamp, the hour, and the minute respectively. If the hash mapping bit value of the timestamp in the first digit array, the hash mapping bit value of the hour in the second digit array, and the hash mapping bit value of the minute in the third digit array are all 1, then the Bloom filter determines that the identification information may exist; otherwise, the Bloom filter determines that the identification information does not exist (i.e., if at least one of the three hash mapping bits is 0, the identification information is considered not to exist).

[0041] In this embodiment, the time information of the data record can be the time of occurrence of frozen data from the electricity meter or an event record, typically given in the format "2026-05-07 14:30:00". Converting this time information into a timestamp involves converting a human-readable time format into a computer-processable integer, usually using a Unix timestamp, and then extracting the coarser-grained hour and minute components from the timestamp. Figure 4 As shown, the Bloom filter uses three relatively short first, second, and third bit arrays to record the timestamp, hour, and minute, respectively, effectively matching the hierarchical structure of time itself. The second bit array corresponding to the hour has 24 possible values, which can be a 32-bit array. The third bit array corresponding to the minute has 60 possible values, which can be a 64-bit array. By using the second and third bit arrays to intercept some illegal matching queries that do not contain data records, the false positive rate of the first bit array corresponding to the timestamp can be relatively high. This allows the first bit array to be shorter, such as 4096 bits. Compared to non-Bloom filters that only use a single bit array for the timestamp, which would require tens of thousands of bits to ensure a low false positive rate, this significantly reduces the overall memory footprint of the Bloom filter. In addition, the three bit arrays can use the same hash function (such as FNV1a) but are independent of each other, avoiding the cross-effect of hash collisions.

[0042] This embodiment utilizes the multi-granularity characteristics of time to quickly intercept queries for data records corresponding to non-existent time information. When an electricity meter generates a data record at time T0, the Bloom filter obtains the hash mapping bits of its hour, minute, and timestamp in the corresponding bit arrays and sets all three hash mapping bit values ​​to 1. During a matching query, if any one, two, or all three hash mapping bit values ​​are 0, it proves that the data record corresponding to time T0 absolutely does not exist. Only when all three hash mapping bit values ​​are 1 is it considered possible that the record exists. The false positive probability of this design is the product of the false positive probabilities of the three bit arrays, which greatly reduces the overall false positive probability of the Bloom filter.

[0043] Example 3: This embodiment also provides a method for rapid matching of electricity meter data based on a Bloom filter. The difference between this method and Embodiments 1 and 2 is as follows: (See below) Figure 5 As shown, when a new data record is received, the method further includes: Step S31: Extract new identification information from the new data record; Step S32: Calculate the hash value of the new identifier information, determine the hash mapping bit of the new identifier information in the Bloom filter based on the hash value of the new identifier information, and set the hash mapping bit value to 1; Step S33: Store the new data record in the external memory to obtain the storage physical address of the new data record, and obtain a new address tag according to the storage physical address of the new data record; specifically, the new address tag can be obtained according to the mapping relationship between the address tag and the storage physical address; Step S34: Insert a new node into the bottom linked list of the skip list according to the new identifier information, and create a hierarchical linked list array of the new node. The node information of the new node includes the new identifier information, the new address marker, and the hierarchical linked list array of the new node.

[0044] In this embodiment, new data records refer to frozen data, event records, or real-time sampled data newly generated during the operation of the electricity meter that need to be persistently stored for subsequent matching and querying. Similarly, new identification information can be a serial number or time information. The hash mapping bits in the Bloom filter refer to the single serial number bit array index obtained after hash function calculation in Embodiment 1, or the three indices of the first, second, and third bit arrays in Embodiment 3.

[0045] As an optional implementation, in step S34, please see... Figure 6 As shown, inserting a new node into the bottom-level linked list of the skip list according to the new identifier information and creating a hierarchical linked list array of the new node includes performing the following steps layer by layer, starting from the top-level linked list of the skip list: Step S341: Starting from the initial query node of the current linked list, search to the right. If a node with an identifier greater than the new identifier can be found, then the node preceding the first node with the identifier greater than the new identifier in the current linked list is taken as the initial query node of the next linked list, and proceed to step S342. If a node with an identifier greater than the new identifier cannot be found, then the last node of the current linked list is taken as the initial query node of the next linked list, and proceed to step S342. Step S342: If the current linked list is not the bottom linked list, proceed to the next level linked list for query and return to execute step S341; if the current linked list is the bottom linked list, insert a new node after the determined starting query node of the next level linked list (which serves as the predecessor node of the new node), randomly set the level height of the new node within the level range of the skip list, and construct a level linked list array of the new node according to the determined starting query node of each level linked list (the determined starting query node of each level linked list is the predecessor node of the new node in that level linked list, and the forward pointer points to it; the node after the predecessor node in that level linked list is the successor node of the new node, that is, the node whose identifier information in the first node information encountered is greater than the new identifier information is the successor node of the new node, and the backward pointer points to it).

[0046] In this embodiment, for the top-level linked list, the starting query node is usually its head node. The new node insertion position is found by skipping from the top to the bottom level, shortening the location time. For non-top-level linked lists, the starting query node is determined during the query of the previous level linked list, avoiding repeated traversal and reducing memory overhead. The skip list, in conjunction with the Bloom filter, ensures data real-time performance and consistency, enabling efficient maintenance of the skip list index and Bloom filter on resource-constrained devices.

[0047] Example 4: This embodiment also provides a fast matching method for electricity meter data based on a Bloom filter. The difference from Embodiments 1, 2, and 3 is that the address marker in the node information of the node is the address offset of the physical address of the data record corresponding to the node in the external memory compared to the base address. In step S2, reading the target data record from the external memory according to the address marker in the node information of the matching node includes: superimposing the address marker in the node information of the matching node on the base address to obtain the physical address of the target data record in the external memory, where the physical address = base address + address marker, and reading the target data record from the external memory according to the physical address.

[0048] In this embodiment, the address tag = physical storage address - base address. The address tag records the offset of the physical storage address of the data record corresponding to the node in the external memory relative to the base address. This ensures that if the base address in the external memory changes (such as during data migration or memory area remapping after firmware upgrade), only the base address needs to be modified, without updating the address tag of each node.

[0049] Example 5: Those skilled in the art will understand that all or part of the features / steps of the above-described method embodiments can be implemented by methods, data processing systems, or computer programs. These features can be implemented without hardware, entirely in software, or in a combination of hardware and software. The aforementioned computer program can be stored in one or more computer-readable storage media. When the computer program is executed (e.g., by a processor), it performs the steps of the above-described embodiment of a method for fast matching of electricity meter data based on a Bloom filter.

[0050] The aforementioned storage media capable of storing program code include: static disks, solid-state drives, random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), optical storage devices, magnetic storage devices, flash memory, magnetic disks or optical disks, and / or combinations of the above devices, that is, they can be implemented by any type of volatile or non-volatile storage devices or combinations thereof.

[0051] Example 6: The present invention also provides an embodiment of a fast matching device for electricity meter data, comprising one or more processors and a memory; wherein the memory is used to store one or more computer programs, and the one or more processors are used to execute the one or more computer programs stored in the memory, so that the processors execute the features / steps of the above-described embodiment of a fast matching method for electricity meter data based on a Bloom filter.

[0052] The embodiment is merely a specific example and does not indicate that this is the only way to implement the present invention.

[0053] The above description is merely a preferred embodiment of the present invention. Those skilled in the art will understand that various changes or equivalent substitutions can be made to these features and embodiments without departing from the spirit and scope of the present invention. Furthermore, under the teachings of the present invention, these features and embodiments can be modified to adapt to specific situations and materials without departing from the spirit and scope of the present invention. Therefore, the present invention is not limited to the specific embodiments disclosed herein, and all embodiments falling within the scope of the claims of this application are within the protection scope of the present invention.

Claims

1. A method for fast matching of electricity meter data based on a Bloom filter, characterized in that, The method includes: Receive a data request sent by the data requesting end, and parse the data request to obtain the identification information of the target data record; The system queries a Bloom filter in memory based on the identification information. The Bloom filter stores the hash mapping bit value of the identification information to determine whether the identification information may exist. If the Bloom filter determines that the identification information may exist, it searches for a matching node in a skip list in memory that matches the identification information. If a matching node is found, the system reads the target data record from external memory based on the address marker in the node information of the matching node and returns the target data record to the data requesting end. The address marker in the node information of each node in the skip list corresponds to the physical storage address of a data record stored in the external memory. If the Bloom filter determines that the identification information does not exist, or if the matching node is not found in the skip list, it returns no data to the data requesting end.

2. The method for fast matching of electricity meter data based on a Bloom filter according to claim 1, characterized in that, The identification information is the sequence number of the target data record; The Bloom filter includes an array of index bits; The step of querying the Bloom filter in memory based on the identification information includes: Calculate the hash value of the identification information; The hash value is modulo the length of the sequence number array to obtain the hash mapping bit of the identification information in the sequence number array; If the hash mapping bit value of the identification information in the sequence number array is 1, the Bloom filter determines that the identification information may exist; if the hash mapping bit value of the identification information in the sequence number array is 0, the Bloom filter determines that the identification information does not exist.

3. The method for fast matching of electricity meter data based on a Bloom filter according to claim 1, characterized in that, The identification information is the time information of the target data record, and the Bloom filter includes a first array of timestamps, a second array of hours, and a third array of minutes. The step of querying the Bloom filter in memory based on the identification information includes: The time information is converted into a timestamp, and the hours and minutes are extracted from the timestamp; Calculate the hash values ​​of the timestamp, the hour, and the minute respectively, and determine the hash mapping bit of the timestamp in the first digit array, the hash mapping bit of the hour in the second digit array, and the hash mapping bit of the minute in the third digit array based on the hash values ​​of the timestamp, the hour, and the minute respectively. If the hash mapping bit value of the timestamp in the first digit array, the hash mapping bit value of the hour in the second digit array, and the hash mapping bit value of the minute in the third digit array are all 1, then the Bloom filter determines that the identification information may exist; otherwise, the Bloom filter determines that the identification information does not exist.

4. A method for rapid matching of electricity meter data based on a Bloom filter according to any one of claims 1-3, characterized in that, The skip list includes multiple nodes, each node corresponding to a data record stored in the external memory; the node information of the node includes a hierarchical linked list array, the identification information of the corresponding data record, and an address marker; wherein, the hierarchical linked list array is used to store the forward pointer and / or successor pointer of the node in each level of the skip list.

5. The method for rapid matching of electricity meter data based on a Bloom filter according to claim 4, characterized in that, When a new data record is received, the method further includes: Extract new identification information from new data records; Calculate the hash value of the new identifier information, determine the hash mapping bit of the new identifier information in the Bloom filter based on the hash value of the new identifier information, and set the hash mapping bit value to 1; The new data record is stored in the external memory to obtain the physical address of the new data record, and a new address tag is obtained based on the physical address of the new data record. A new node is inserted into the bottom-level linked list of the skip list according to the new identification information, and a hierarchical linked list array of the new node is created. The node information of the new node includes the new identification information, the new address marker, and the hierarchical linked list array of the new node.

6. The method for fast matching of electricity meter data based on a Bloom filter according to claim 5, characterized in that, The step of inserting a new node into the bottom-level linked list of the skip list according to the new identifier information and creating a hierarchical linked list array of the new node includes performing the following steps layer by layer, starting from the top-level linked list of the skip list: Step S341: Starting from the initial query node of the current linked list, search to the right. If a node with an identifier greater than the new identifier can be found, then the node preceding the first node with the identifier greater than the new identifier in the current linked list is taken as the initial query node of the next linked list, and proceed to step S342. If a node with an identifier greater than the new identifier cannot be found, then the last node of the current linked list is taken as the initial query node of the next linked list, and proceed to step S342. Step S342: If the current linked list is not the bottom linked list, proceed to the next level linked list for query and return to execute step S341; if the current linked list is the bottom linked list, insert a new node after the starting query node of the determined next level linked list, randomly set the level height of the new node within the level range of the skip list, and construct the level linked list array of the new node according to the starting query node determined by each level linked list.

7. The method for fast matching of electricity meter data based on a Bloom filter according to claim 4, characterized in that, The step of searching for a matching node in the skip list in memory that matches the identification information includes: Starting from the top-level linked list of the skip list, the matching node is searched level by level. In the search of each level of the linked list: Starting from the initial search node of the current linked list, the search proceeds to the right, comparing the identifier information in the node information of each node with the identifier information. If a node is found whose identifier information matches the identifier information, that node is designated as the matching node. Otherwise, the node whose identifier information in the last node information of the current linked list is less than the identifier information is designated as the starting search node of the next level linked list, and the search continues in the next level linked list until the terminating node is found. The terminating node is the first node in the underlying linked list whose identifier information is greater than the identifier information.

8. A method for rapid matching of electricity meter data based on a Bloom filter according to any one of claims 5-7, characterized in that, The address marker in the node information of the node is the address offset of the physical address of the corresponding data record of the node in the external memory compared with the base address; The step of reading the target data record from the external memory according to the address marker in the node information of the matching node includes: superimposing the address marker in the node information of the matching node on the base address to obtain the storage physical address of the target data record in the external memory, and reading the target data record from the external memory according to the storage physical address.

9. A computer-readable storage medium, characterized in that, The storage medium stores a computer program, which, when executed, implements a method for rapid matching of electricity meter data based on a Bloom filter, as described in any one of claims 1-8.

10. A device for rapid matching of electricity meter data, characterized in that, include: One or more processors; A memory for storing one or more computer programs, and one or more processors for executing the one or more computer programs stored in the memory to cause the one or more processors to perform a method for fast matching of electricity meter data based on a Bloom filter as described in any one of claims 1-8.

Images