MAC (Media Access Control) address storage method and device and MAC address verification method and device
By using 48-bit unsigned binary bucket storage based on IEEE OUI vendor code and dynamic switching of container types, the problems of high memory consumption and low query efficiency of MAC address verification under massive device access are solved, and efficient and accurate device legitimacy verification is achieved.
Patent Information
- Application Number
- CN202511611323.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-11-05
- Publication Date
- 2026-02-27
AI Technical Summary
Existing technologies suffer from high memory consumption, poor scalability, and low query efficiency in MAC address verification scenarios with a large number of connected devices. This is especially true when the number of devices changes rapidly, resulting in insufficient system resources and slow response.
We adopt a 48-bit unsigned binary bucket storage method based on the IEEE OUI vendor code, and use the Roaring Bitmap data structure to dynamically switch the container type to array, bitmap or run step type to optimize memory usage and improve query efficiency.
By replacing traditional database traversal with bitwise operations, memory usage is reduced, system performance and scalability are improved, and MAC address verification is ensured to be efficient and accurate.
Smart Images
Figure CN121579377A_ABST
Abstract
Description
Technical Field
[0001] This disclosure belongs to the field of Internet of Things (IoT) device access management technology, and particularly relates to a MAC address storage method, verification method, and apparatus. Background Technology
[0002] With the rapid development of Internet of Things (IoT) technology, more and more devices are connecting to IoT platforms via networks, making device access authentication and management one of the important functions of IoT platforms. During the device access process, the authentication and verification of the device's unique identifier (such as its MAC address) is a key step in ensuring the device's legitimacy and preventing duplicate access.
[0003] In existing technologies, MAC address verification of devices typically employs the following method: when a device connects, it queries the cache or database to see if the MAC address has been entered.
[0004] Specifically, existing MAC address verification mechanisms have the following problems: Problem 1: High memory usage: Existing technical solutions typically allocate one Redis cache resource for each MAC address. When the number of devices reaches tens of millions, memory usage becomes very large, leading to insufficient system resources and affecting overall performance.
[0005] Problem 2: Poor scalability: Existing mechanisms are difficult to adapt to dynamic changes in the scale of equipment, especially when the number of equipment increases or decreases rapidly, the system’s resource allocation and performance optimization face significant challenges.
[0006] Question 3: Low query efficiency: For massive numbers of devices (e.g., tens of millions), directly querying the database or Redis cache to determine the existence of a MAC address will significantly reduce query efficiency. This is especially true when devices are deployed in a concentrated manner, as the verification pressure increases dramatically, potentially leading to slower system response or even system crashes.
[0007] Therefore, how to reduce memory usage and improve system performance while efficiently and accurately verifying MAC addresses in scenarios with a massive number of connected devices has become an urgent technical problem to be solved. Summary of the Invention
[0008] To address the aforementioned issues, this disclosure provides a MAC address storage method, verification method, and apparatus.
[0009] This invention provides a MAC address storage method, comprising: The MAC address is preprocessed to obtain a 48-bit unsigned binary number, where the high 24 bits of the MAC address are the OUI vendor code and the low 24 bits are the device identifier. Use the high 24 bits of the 48-bit unsigned binary number as the bucket index to find the contents of the bucket. Store the MAC address in the found bucket content container according to the update method corresponding to the container type of the found bucket content container; Among them, the container types include: array type, bitmap type, and run-length type, and the corresponding update methods are updating an ordered array, updating a bitmap, and updating run-length encoding respectively.
[0010] Further, the method further includes: monitoring the status of each container, and when adding or deleting elements, determining whether the number of elements in the container is greater than 524288; if so, when the current type of the container is the array type, converting its type to the bitmap type; if not, when the current type of the container is the bitmap type, converting it to the array type.
[0011] Further, the method further includes: starting a timing task, and when the timing duration arrives, determining whether a container of the array type satisfies the condition for switching to the run-length type, and if so, switching to the run-length type; and determining whether a container of the bitmap type satisfies the condition for switching to the run-length type, and if so, switching to the run-length type; Among them, the condition for switching the array type to the run-length type is run_count < element_count / 4; the condition for switching the bitmap type to the run-length type is run_count < 262144; element_count is the number of elements contained in the container when the container type is not the run-length type.
[0012] The present invention also provides a MAC address verification method, including: Receiving a MAC address and converting it into a 48-bit binary number; Extracting the upper 24 bits as the bucket index; According to the bucket index, searching for a bucket content container in the MAC address database, and the MAC address database is created by using the above MAC address storage method; According to the container type of the found bucket content container, using the lower 24 bits to perform corresponding data queries. Among them, when the container type is the array type, the corresponding data query method is binary search; when the container type is the bitmap type, the corresponding data query method is bit operation; when the container type is the run-length type, the corresponding data query method is interval scanning; If the lower 24 bits are queried, it is determined that the MAC address passes the verification; otherwise, it is determined that the verification fails.
[0013] The present invention also provides a MAC address storage device, including: A preprocessing unit for preprocessing the MAC address to obtain a 48-bit unsigned binary number, where the high 24 bits of the MAC address are the OUI manufacturer code and the low 24 bits are the device identifier; A lookup unit for using the high 24 bits of the 48-bit unsigned binary number as a bucket index to look up the bucket content container; A storage unit for storing the MAC address in the found bucket content container according to the update method corresponding to the container type of the found bucket content container; Among them, the container types include: array type, bitmap type, and run-length type, and the corresponding update methods are updating an ordered array, updating a bitmap, and updating run-length encoding, respectively.
[0014] Furthermore, the device further includes: A conversion unit for monitoring the status of each container, and when adding or deleting an element, determining whether the number of elements in the container is greater than 524288; if so, when the current type of the container is the array type, converting its type to the bitmap type; if not, when the current type of the container is the bitmap type, converting it to the array type.
[0015] Furthermore, the conversion unit is further used to start a timing task, and when the timing duration arrives, determine whether the container of the array type meets the condition for switching to the run-length type, and if so, switch to the run-length type; and determine whether the container of the bitmap type meets the condition for switching to the run-length type, and if so, switch to the run-length type; Among them, the condition for the array type to switch to the run-length type is run_count < element_count / 4; the condition for the bitmap type to switch to the run-length type is run_count < 262144; element_count is the number of elements contained in the container when the container type is a non-run-length type.
[0016] The present invention also provides a MAC address verification device, including: A receiving unit for receiving a MAC address and converting it into a 48-bit binary number; An extraction unit for extracting the high 24 bits as a bucket index; A lookup unit for looking up a bucket content container from a MAC address database according to the bucket index, and the MAC address database is created by using the above MAC address storage device; The verification unit is used to perform corresponding data queries using the lower 24 bits based on the container type of the found bucket's contents. Specifically, when the container type is an array, the corresponding data query method is binary search; when the container type is a bitmap, the corresponding data query method is bitwise operation; and when the container type is a run-length type, the corresponding data query method is range scanning. If the query is successful, the MAC address is determined to have passed the verification; otherwise, the verification is determined to have failed.
[0017] The present invention also provides an electronic device, including a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other through the communication bus; Memory, used to store computer programs; The processor implements the above method when executing programs stored in memory.
[0018] The present invention also provides a computer storage medium storing a computer program, which, when executed by a processor, implements the above-described method.
[0019] Compared with the prior art, this disclosure has the following advantages: 1) This disclosure abandons the I / O bottleneck of traditional database row-based traversal query and innovates the MAC address verification method by using a storage architecture based on compressed bitmap, transforming device legitimacy verification into bit operations.
[0020] 2) This disclosure is the first to deeply integrate the IEEE-defined OUI with the Roaring Bitmap data structure, directly using the high 24 bits of the MAC OUI as the bucket key to avoid the risk of collision in hash downsizing operations; and it switches the container type in real time based on the data density in the bucket to optimize memory usage.
[0021] Other features and advantages of this disclosure will be set forth in the description which follows, and will be apparent in part from the description, or may be learned by practicing the disclosure. The objects and other advantages of this disclosure may be realized and obtained by means of the structures pointed out in the description, claims and drawings. Attached Figure Description
[0022] To more clearly illustrate the technical solutions in the embodiments of this disclosure or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this disclosure. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0023] Figure 1 A schematic diagram of a MAC storage process according to an embodiment of the present disclosure is shown; Figure 2 A schematic diagram of a container switching process according to an embodiment of the present disclosure is shown; Figure 3 A schematic diagram of a MAC verification process according to an embodiment of the present disclosure is shown; Figure 4 A block diagram of a MAC storage device according to an embodiment of the present disclosure is shown; Figure 5 A block diagram of a MAC verification device according to an embodiment of the present disclosure is shown; Figure 6 A block diagram of an electronic device according to an embodiment of the present disclosure is shown. Detailed Implementation
[0024] To make the objectives, technical solutions, and advantages of the embodiments of this disclosure clearer, the technical solutions of the embodiments of this disclosure will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this disclosure, and not all embodiments. Based on the embodiments of this disclosure, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this disclosure.
[0025] Figure 1 A schematic flowchart of a MAC address storage method according to an embodiment of the present disclosure is shown, as follows: Figure 1 As shown, the MAC address storage method of this disclosure includes the following steps: Step 101: Enter the MAC address; Step 102: Convert the MAC address to a 48-bit binary number; Steps 101 and 102 above constitute the preprocessing of the MAC address, converting six sets of 12-bit hexadecimal MAC addresses into 48-bit unsigned binary numbers. The specific processing includes: Step 1) Input standardization: Supports MAC format with delimiters (e.g., 00:1B:44:11:22:33 or 00-1B-44-11-22-33).
[0026] Step 2), Character filtering: Remove all separators ":" or "-" to generate a 12-digit hexadecimal string (e.g., 001B44112233).
[0027] Step 3) Number base conversion: Parse the hexadecimal string into a 48-bit unsigned binary number (e.g., 0000 0000 0001 1011 01000100 0001 0001 0010 0010 0011 0011).
[0028] Step 103: Extract the high 24 bits of the OUI and bucket them; The OUI (Organizationally Unique Identifier) is a globally unique network device identifier managed uniformly by the IEEE Standards Association and is a core component of the MAC address. Based on the structural characteristics of the MAC address: the high 24 bits are the OUI vendor code + the low 24 bits are the device identifier. In this embodiment, a custom 48-bit RoaringBitmap is defined.
[0029] In this embodiment of the disclosure, the high 24 bits of the 48-bit MAC address are used as the OUI vendor code. The OUI vendor code is used as a bucket index to store the device identifiers of the same vendor represented by the same OUI vendor code in a bucket container.
[0030] 1) Bucket partitioning principle: The high 24 bits of the MAC are the OUI vendor code, corresponding to 16,777,216 buckets. In reality, the system only connects to devices from some vendors, so the number of OUI buckets will not be too large.
[0031] 2) Index generation: Extract the high 24 bits of the 48-bit binary number as the bucket index (e.g., 0000 0000 0001 1011 0100 01000001 0001 0010 0010 0011 0011>>24).
[0032] Step 103: Use the high 24 bits of the 48-bit unsigned binary number as the bucket index to find the contents of the bucket; Step 104: Store the MAC address in the found container according to the update method corresponding to the container type of the found container; The container types include array type, bitmap type, and run step type, and the corresponding update methods are updating ordered array, updating bitmap, and updating run code, respectively.
[0033] During container initialization, all containers within a bucket can be set to array type. As data increases, the container type can be switched when the conditions for switching the container type are met.
[0034] Query the containers within the bucket based on the bucket index, and update them according to their different types:
[0035] Specific container switching methods, such as Figure 2 As shown, it includes: Step 201: Monitor the status of each container. When adding or deleting elements, execute step 202; when starting a scheduled task, execute step 205. Step 202: Determine if the number of elements in the container is greater than 524288; if yes, proceed to step 203; if no, proceed to step 204. Assuming each bucket stores n data items, an ArrayContainer (array container) occupies 4 bytes per integer bit, so n data items occupy 4n bytes. A BitmapContainer (bitmap container) uses a fixed 24-bit bitmap, occupying 2^24 bits = 2^21 bytes = 2^11 bytes = 2 MB. When 4n bytes <= 2 MB, i.e., n <= 2^19 = 524288, the ArrayContainer is used; otherwise, the BitmapContainer is used.
[0036] Switching the container from array type to bitmap type: Condition: 4 × element_count Bytes > 2^24 bits (i.e., element_count > 524288) Principle: ArrayContainer occupies 4 × element_count bytes, BitmapContainer occupies a fixed 2MB. When adding devices, switching saves more space when the number of devices > 524288.
[0037] Switching between bitmap and array types: Condition: 4 × element_count Byte < 2^24 bits (i.e., element_count < 524288) Principle: ArrayContainer occupies 4 × element_count bytes, BitmapContainer occupies a fixed 2MB. When deleting a device, switching saves more space when the number of device identifiers is less than 524288.
[0038] Step 203: If the container's current type is an array, convert its type to a bitmap type; Step 204: If the container's current type is a bitmap, convert it to an array type.
[0039] Step 205: When the timeout period is reached, determine whether the switching condition is met. If yes, proceed to step 206; otherwise, jump to step 205. Specifically, this step 205 is as follows: Determine whether a container of array type meets the condition for switching to the run-length type. If so, execute step 206; and determine whether a container of bitmap type meets the condition for switching to the run-length type. If so, execute step 206; Among them, the condition for switching the array type to the run-length type is run_count < element_count / 4; the condition for switching the bitmap type to the run-length type is run_count < 262144; run_count is the number of consecutive intervals required to store the MAC address when the container type is switched to the run-length type, and element_count is the number of elements contained when the container type is of non-run-length type.
[0040] Step 206: Switch to the run-length type.
[0041] To prevent performance problems caused by frequent switching of container types, a scheduled task is used when switching the array type to the run-length type or the bitmap type to the run-length type, and it is checked every 1 hour whether the container switching condition is met.
[0042] Switching the array type to the run-length type: Condition: 8 × run_count < 4 × element_count (i.e., run_count < element_count / 4) Principle: The space occupied by ArrayContainer = 4 × element_count bytes, and the space occupied by RunContainer = 8 × run_count bytes. When run_count < element_count / 4, the switch saves more space.
[0043] Switching the bitmap type to the run-length type: Condition: 8 × run_count < 2 × 1024 × 1024 (i.e., run_count < 262,144) Principle: The BitmapContainer bitmap container fixedly occupies 2MB, and the RunContainer run-length container needs to meet 8 × run_count < 2 × 1024 × 1024 → run_count < 262144. When the number of consecutive intervals is less than 262,144, the switch saves more space.
[0044] [[ID=2,6]] Figure 3 A schematic flowchart of a MAC address verification method according to an embodiment of the present disclosure is shown, including:[[ID=,]] Step 301: Receive the input MAC address; Step 302: Convert the MAC address to a 48-bit binary number; Step 303: Extract the high 24 bits of the OUI as the bucket index; Step 304: Determine the container type based on the high 24 bits. If it is an array type, proceed to step 305; if it is a bitmap type, proceed to step 306; if it is a run step type, proceed to step 307. Specifically, based on the bucket index, the contents of the bucket are looked up from the MAC address database, which is created using the MAC address storage method described above; Based on the container type of the bucket's contents found.
[0045] Step 305: Perform a binary search using the lower 24 bits.
[0046] Step 306: Perform bitwise operations using the lower 24 bits.
[0047] Step 307: Perform a range scan using the lower 24 bits.
[0048] In steps 305 to 307 above, if the MAC address is found, it is determined that the verification has passed; otherwise, it is determined that the verification has failed.
[0049] Based on the above method, this disclosure also provides a MAC address storage device corresponding to the above MAC address storage method. Figure 4 A block diagram of a MAC address storage device according to an embodiment of the present disclosure is shown, such as Figure 4 As shown, the MAC address storage device includes: a preprocessing unit, a lookup unit, and a storage unit; wherein: The preprocessing unit is used to preprocess the MAC address to obtain a 48-bit unsigned binary number, in which the high 24 bits of the MAC address are the OUI vendor code and the low 24 bits are the device identifier. The lookup unit is used to use the high 24 bits of a 48-bit unsigned binary number as a bucket index to look up the contents of the bucket. The storage unit is used to store the MAC address in the found bucket container according to the update method corresponding to the container type of the found bucket container; The container types include array type, bitmap type, and run step type, and the corresponding update methods are updating ordered array, updating bitmap, and updating run code, respectively.
[0050] Furthermore, the device also includes: A conversion unit is used to monitor the status of each container. When adding or deleting elements, it determines whether the number of elements in the container is greater than 524288. If so, when the current type of the container is an array type, it converts its type to a bitmap type. If not, when the current type of the container is a bitmap type, it converts it to an array type.
[0051] Furthermore, the conversion unit is also used to start a timing task. When the timing duration arrives, it determines whether a container of array type meets the condition for switching to a step length type. If so, it switches to the step length type; and determines whether a container of bitmap type meets the condition for switching to a step length type. If so, it switches to the step length type. Among them, the condition for an array type to switch to a step length type is run_count < element_count / 4; the condition for a bitmap type to switch to a step length type is run_count < 262144; run_count is the number of consecutive intervals required to store the MAC address when the container type switches to the step length type, and element_count is the number of elements contained in the container when the container type is not the step length type.
[0052] The present disclosure also provides a MAC address verification device corresponding to the above MAC address verification method, as Figure 5 shown. The MAC address verification device includes: a receiving unit, an extraction unit, a lookup unit, and a verification unit, where: The receiving unit is used to receive a MAC address and convert it into a 48-bit binary number. The extraction unit is used to extract the upper 24 bits as a bucket index. The lookup unit is used to look up the bucket content container according to the bucket index. The verification unit is used to perform corresponding data queries using the lower 24 bits according to the container type of the found bucket content container. Among them, when the container type is an array type, the corresponding data query method is binary search; when the container type is a bitmap type, the corresponding data query method is bit operation; when the container type is a step length type, the corresponding data query method is interval scanning. If found, it determines that the MAC address passes the verification; otherwise, it determines that the verification fails.
[0053] Based on the same inventive concept of the above disclosed content, correspondingly, the present disclosure also provides an electronic device, as Figure 6As shown. An electronic device according to an embodiment of this disclosure includes at least one processor and at least one memory electrically connected to the processor, wherein the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the method described above.
[0054] It should be noted that the electrical connection between the above-mentioned units does not necessarily mean the connection between lines. The indirect connection method can be applied to the embodiments of this disclosure as long as it achieves the purpose of this disclosure.
[0055] Based on the same inventive concept, this disclosure also provides a computer storage medium storing a computer program, which, when executed by a processor, implements the above-described method.
[0056] Although the present disclosure has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present disclosure.
Claims
1. A method for storing MAC addresses, characterized in that, include: The MAC address is preprocessed to obtain a 48-bit unsigned binary number, where the high 24 bits of the MAC address are the OUI vendor code and the low 24 bits are the device identifier. Use the high 24 bits of the 48-bit unsigned binary number as the bucket index to find the contents of the bucket. According to the update method corresponding to the container type of the found bucket contents, store the MAC address in the found bucket contents; The container types include array type, bitmap type, and run step type, and the corresponding update methods are updating ordered array, updating bitmap, and updating run code, respectively.
2. The method according to claim 1, characterized in that, The method further includes: monitoring the state of each container, and when adding or deleting elements, determining whether the number of elements in the container is greater than 524288; if so, converting the container's current type to a bitmap type if the container's current type is an array type; if not, converting the container's current type to an array type if the container's current type is a bitmap type.
3. The method according to claim 1, characterized in that, The method further includes: starting a timed task; when the timed duration is reached, determining whether a container of type array meets the condition for switching to a step size type; if so, switching to a step size type; and determining whether a container of type bitmap meets the condition for switching to a step size type; if so, switching to a step size type. The condition for switching from an array type to a run-time step type is `run_count < element_count / 4`; the condition for switching from a bitmap type to a run-time step type is `run_count`. <262144; element_count is the number of elements contained in a container of type non-run-step type.
4. A MAC address verification method, characterized in that, include: Receive the MAC address and convert it into a 48-bit binary number; Extract the high 24 bits as the bucket index; Based on the bucket index, the contents of the bucket are searched from the MAC address database, which is created using the MAC address storage method according to any one of claims 1-3; Based on the container type of the found bucket's contents, the lower 24 bits are used to perform the corresponding data query. When the container type is an array, the corresponding data query method is binary search; when the container type is a bitmap, the corresponding data query method is bitwise operation; and when the container type is a run-length type, the corresponding data query method is range scan. If the lower 24 bits are found, the MAC address is confirmed to have passed verification; otherwise, it is confirmed to have failed verification.
5. A MAC address storage device, characterized in that, include: The preprocessing unit is used to preprocess the MAC address to obtain a 48-bit unsigned binary number, in which the high 24 bits of the MAC address are the OUI vendor code and the low 24 bits are the device identifier. The lookup unit is used to use the high 24 bits of a 48-bit unsigned binary number as a bucket index to look up the contents of the bucket. The storage unit is used to store the MAC address in the found bucket container according to the update method corresponding to the container type of the found bucket container; The container types include array type, bitmap type, and run step type, and the corresponding update methods are updating ordered array, updating bitmap, and updating run code, respectively.
6. The apparatus according to claim 5, characterized in that, The device further includes: The conversion unit is used to monitor the status of each container. When adding or deleting elements, it determines whether the number of elements in the container is greater than 524288. If so, it converts the container's current type to a bitmap type if the current type is an array type. If not, it converts the container's current type to an array type if the current type is a bitmap type.
7. The apparatus according to claim 6, characterized in that, The conversion unit is used to start a timed task. When the timed duration is reached, it determines whether a container of type array meets the conditions for switching to type run step. If so, it switches to type run step. It also determines whether a container of type bitmap meets the conditions for switching to type run step. If so, it switches to type run step. The condition for switching from an array type to a run-time step type is `run_count < element_count / 4`; the condition for switching from a bitmap type to a run-time step type is `run_count`. <262144; element_count is the number of elements contained in a container of type non-run-step type.
8. A MAC address verification device, characterized in that, include: The receiving unit is used to receive MAC addresses and convert them into 48-bit binary numbers. The extraction unit is used to extract the high 24 bits as the bucket index; A lookup unit is configured to look up a bucket contents from a MAC address database based on a bucket index, the MAC address database being created using the apparatus of any one of claims 5-7; The verification unit is used to perform corresponding data queries using the lower 24 bits based on the container type of the found bucket's contents. Specifically, when the container type is an array, the corresponding data query method is binary search; when the container type is a bitmap, the corresponding data query method is bitwise operation; and when the container type is a run-length type, the corresponding data query method is range scan. If the query is successful, the MAC address is determined to have passed the verification; otherwise, the verification is determined to have failed.
9. An electronic device, characterized in that, It includes a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other through the communication bus; Memory, used to store computer programs; A processor, when executing a program stored in memory, implements the method of any one of claims 1-4.
10. A computer storage medium, characterized in that, The computer storage medium stores a computer program, which, when executed by a processor, implements the method described in any one of claims 1-4.