Distributable hash filter for obtaining non-probabilistic set inclusion

Abstract

A distributable hash filter for achieving non-probabilistic set inclusion is provided. In some embodiments, a method includes recursively executing a procedure comprising: updating a bit array using a set of allowed object identifiers and a hash function; further updating the bit array using a set of disallowed object identifiers and the hash function; upon a collision, repeating the procedure with a new allowed set and a new hash function that includes object identifiers from the original allowed set that collide with the disallowed set until a round is reached where no collisions occur; generating a data structure comprising the bit array created during each recursive round; and compressing the data structure.

Description

Background Art

[0001] The membership of an identifier in a set may be performed from time to time. As an example, the identifiers of devices may be a set, and a given user may only have access to some of the devices. Therefore, in order to determine whether a user can access a certain device, the membership of the devices that the user seeks to access may be tested. However, techniques for testing for membership in a set may require a relatively large amount of resources to check the identifier of the device to which access is sought against a larger set of device identifiers. Other techniques may be able to determine probabilistically whether a device identifier is likely to be in a set, but may not be able to provide a definitive answer non-probabilistically. BRIEF DESCRIPTION OF THE DRAWINGS

[0002] Certain embodiments discussed herein will be described with reference to the drawings listed below. However, the drawings illustrate only certain aspects or implementations of the embodiments described herein and are not intended to limit the scope of the claims.

[0003] Figure 1 illustrates a block diagram of an example system for implementing generation of a distributable hash filter for non-probabilistically determining set inclusivity in accordance with one or more embodiments of the present disclosure;

[0004] Figure 2 illustrates an overview of an example method for generating a compressed data structure of a sparse bit array for testing an object identifier for membership in a set of allowed object identifiers, according to one or more embodiments disclosed herein;

[0005] Figure 3 illustrates an overview of an example method for using a compressed data structure of a sparse bit array for testing an object identifier for membership in a set of allowed object identifiers, in accordance with one or more embodiments disclosed herein;

[0006] Figure 4 illustrates an example scenario for testing allowed set membership using a compressed data structure of a sparse bit array according to one or more embodiments of the present disclosure;

[0007] Figure 5 A block diagram of a computing device according to one or more embodiments of the present disclosure is illustrated. DETAILED DESCRIPTION

[0008] There may be scenarios where a particular user in a device ecosystem is only permitted to access a subset of devices, obtain information about a subset of devices, and so on. As an example, a network may include any number of network devices. In such a network, there may be any number of users (e.g., network administrators). Each user may be responsible for only a portion of the network and a subset of the network devices within that portion (e.g., the network devices of a subset of stores in a large chain of stores).

[0009] To restrict a particular user's access to only a subset of devices, a list of devices the user is allowed to access (e.g., an allowed device set) can be obtained. When a user requests information about one or more devices, a determination can be made as to whether the user is permitted to obtain such information. To make this determination, the allowed set can be reviewed to determine whether the allowed set includes the one or more devices for which information is being requested. However, when the set of devices becomes large (e.g., hundreds, thousands, millions, etc. of devices), simply checking the allowed set to determine whether the request can be fulfilled can be expensive in terms of time and / or resources, as the set of unique identifiers for devices can be large, not easily compressible, and therefore not easily distributed. Other techniques can be used to determine whether one or more devices are in the allowed set, such as Bloom filters, XOR filters, Cuckoo filters, etc. However, these data structures are inherently probabilistic and therefore cannot always provide a definitive answer. In addition, techniques such as those discussed above may rely on providing large data structures for access each time a user requests information about one or more devices. Such large data structures can be difficult to efficiently share or access in real time when a user makes a request related to a device.

[0010] To potentially address some or all of the aforementioned issues regarding determining whether a particular user can access or obtain information about one or more devices, certain embodiments of the present disclosure provide techniques for creating, distributing, and using a compact, distributable, and search-optimized way to encode large sets of objects (e.g., devices) and distribute them to various services (e.g., microservices) in a device ecosystem. Services can use a data structure to test the allowed set membership of an object's identifier. In one or more embodiments, the structure is independent of the type of object it encodes or the identifier (e.g., a string or universally unique identifier (UUID)) used in its creation. Unlike existing filters for membership testing (e.g., Bloom, XOR, or Cuckoo filters described above), the data structure disclosed herein can be used to deterministically (i.e., non-probabilistically) determine the membership or non-membership of an object (e.g., a device) and can be compressed to a size similar to or smaller than known probabilistic filters, which can, for example, allow the data structure to fit into a Hypertext Transfer Protocol (HTTP) header used when a user makes a request related to an object (e.g., a device).

[0011] In one or more embodiments, to create a data structure, an allowed set is obtained for a particular user. As discussed above, the allowed set can be a set of object identifiers (e.g., device identifiers) that identify objects (e.g., devices) that the user is authorized to access and / or request relevant information. As an example, the allowed set can be a set of UUIDs of devices in a network that the particular user is allowed to access and / or obtain relevant information. In one or more embodiments, a total set of objects is also obtained (e.g., a total set of identifiers of all devices in the network). In one or more embodiments, the allowed set is removed from the total set to obtain a disallowed set (e.g., of device identifiers).

[0012] In one or more embodiments, once the allowed and disallowed sets are obtained, a bit array can be created. The bit array can initially be filled with all zeros. The size of the bit array can be predetermined based on how quickly the final data structure is expected to be created. In one or more embodiments, a larger array can result in fewer recursive rounds of the technique described below, but more memory resources are used to store the larger array when performing the technique. Conversely, a smaller bit array can use fewer memory resources, but can also use more recursive rounds of the technique.

[0013] In one or more embodiments, each identifier in the allowed set is used as an input to a first hash function to obtain an integer output. The integer output is used as an index value identifying a position in the bit array. The hash function can be selected based at least in part on being able to output an index value that identifies a position in the bit array or otherwise corresponds to an index value in the bit array. Each position corresponding to an index value obtained using an identifier from the allowed set and the first hash function is set from zero to one. If two or more identifiers in the allowed set result in the hash function outputting the same index value, the value at the index position remains one.

[0014] In one or more embodiments, once all identifiers in the allowed set have been used as input to the hash function and the corresponding index positions have been set to one, a disallowed set is obtained. Each identifier in the disallowed set is used as input to a first hash function. In one or more embodiments, when the output from the hash function corresponding to an identifier in the disallowed set is a bit array index value that is already zero, that bit remains zero. In one or more embodiments, when the output from the hash function corresponding to an identifier in the disallowed set is an index value that identifies a position in the bit array that is already set to one, each identifier in the allowed set that also corresponds to that index position is added to the new allowed set, and that index position is reset to zero.

[0015] Thus, in one or more embodiments, after evaluating all identifiers in the disallowed set, the new allowed set includes all identifiers from the previous allowed set that conflict in the bit array with at least one identifier from the disallowed set, and the bit array includes ones at all positions where the allowed set does not conflict with the disallowed set and zeros at all positions where a conflict occurs or where no allowed set identifier corresponds to it. Thus, in one or more embodiments, the first bit array is a sparse array, with many zeros and some number of ones representing a subset of the identifiers in the allowed set.

[0016] In one or more embodiments, another round is performed as described above. Specifically, another bit array comprising all zeros is obtained, the identifiers from the new allowed set (which comprises a subset of the identifiers from the initial allowed set) obtained in the previous round are input into a new hash function, and the bits corresponding to the output index positions from the new hash function are set to ones. In one or more embodiments, the complete set of disallowed identifiers is again used as input to the new hash function, collisions in the bit array again cause the corresponding index positions in the bit array to be reset to zero, and one or more allowed set identifiers corresponding to the collision positions are added to a third allowed set. In one or more embodiments, after this second round, a third allowed set and a second sparse bit array are obtained.

[0017] In one or more embodiments, additional rounds of the above process are recursively performed until a round is reached where there are no conflicts between the full disallowed set and the new smaller allowed set.

[0018] In one or more embodiments, the set of sparse bit arrays generated using the above process can be used to create a data structure comprising the sparse bit arrays. The data structure can then be compressed using lossless compression techniques to produce a space-efficient compressed data structure. The compressed data structure can then be distributed to one or more entities that can determine whether a given object identifier is in the initial allowed set.

[0019] As an example, a data structure can be created for a particular user in the network, and the identifiers in the allowed set can be UUIDs of devices that the user has permission to access or obtain information about. In one or more embodiments, because the compressed data structure is much smaller than, for example, a list of identifiers in an allowed set, when a user requests (e.g., to one or more services in a microservices architecture) access to a device or information related thereto, the data structure can be included to determine whether the user has the appropriate permissions for the request to be serviced.

[0020] The service can use the data structure received as part of the request to determine whether the user has the appropriate permissions without having to access any other service to make a determination as to whether the request can be serviced. As an example, a user can send a request for the average page load time for a group of devices, which are identified by their UUIDs in the request. In one or more embodiments, the entity receiving the request also receives the data structure and can therefore use the UUIDs of the devices in the request to test whether each device is in the allowed set for the user.

[0021] In one or more embodiments, to use the data structure, an entity (e.g., a microservice) can decompress the data structure to obtain the set of sparse bit arrays. The entity can then obtain the UUID of the device and use it as input to each hash function corresponding to the bit array. The output of the hash function can be used as an index value to find the location in the corresponding bit array. In one or more embodiments, if there is a one at that location, the UUID is in the allowed set for the user. In one or more embodiments, if all bit arrays are checked in the manner described above and no bit array results in finding a one, the UUID is in the disallowed set (or at least not in the allowed set), which means that the user is not allowed to access information related to the device corresponding to the UUID.

[0022] Certain embodiments of the present disclosure may improve an entity's ability to test whether an identifier (e.g., of a device) is in an allowed set by providing a compact data structure compressed via lossless compression that can be distributed and used when a request is made to allow the entity to decompress the data structure and test the bit array of the decompressed data structure for membership in the allowed set using the identifier included in the request and a hash function associated with the bit array of the decompressed data structure.

[0023] Figure 1 1 illustrates a block diagram of an example system for implementing generation of a distributable hash filter for non-probabilistically determining set inclusion according to one or more embodiments of the present disclosure. Figure 1 As shown, the system includes a computing device 100. The computing device 100 may include a processor 102, a memory 104, a storage device 106, a bit array generator 108, and a distributable data structure generator 110. Each of these components is described below.

[0024] In one or more embodiments, as used herein, computing device 100 can be any single computing device, a group of computing devices, a portion of one or more computing devices, or any other physical, virtual, and / or logical grouping of computing resources. In one or more embodiments, computing device 100 is any device, a portion of a device, or any collection of devices capable of electronically processing instructions, and can include, but is not limited to, any of the following: one or more processors (e.g., components comprising circuitry), memory (e.g., random access memory (RAM)), input and output device(s), non-volatile storage hardware (e.g., solid state drive (SSD), hard disk drive (HDD) (not shown)), one or more physical interfaces (e.g., network ports, storage ports), any number of other hardware components (not shown), and / or any combination thereof.

[0025] Examples of computing devices include, but are not limited to, servers (e.g., blade servers in blade server chassis, rack servers in racks, etc.), desktop computers, mobile devices (e.g., laptop computers, smart phones, personal digital assistants, tablet computers, automotive computing systems, and / or any other mobile computing devices), storage devices (e.g., disk drive arrays, Fibre Channel storage devices, Internet Small Computer System Interface (iSCSI) storage devices, tape storage devices, flash storage arrays, network attached storage devices, etc.), network devices (e.g., switches, routers, multilayer switches, etc.), virtual machines, virtualized computing environments, logical containers (e.g., for one or more applications), Internet of Things (IoT) devices, node arrays of computing resources, supercomputing devices, data centers or any portion thereof, and / or any other type of computing device having the foregoing requirements. In one or more embodiments, any or all of the foregoing examples may be combined to create a system of such devices, or may be divided into separate logical devices, which may be collectively referred to as computing devices. Other types of computing devices (e.g., Figure 5 and described below) without departing from the scope of the embodiments described herein.

[0026] In one or more embodiments, the storage device (e.g., storage device 106) and / or memory (e.g., memory 104) of a computing device or computing device system can be and / or include one or more data repositories for storing any number of data structures (which store any amount of data (e.g., information)). In one or more embodiments, a data repository is any type of storage unit and / or device (e.g., a file system, a database, a collection of tables, RAM, and / or any other storage mechanism or medium) for storing data. Further, a data repository can include multiple different storage units and / or devices. The multiple different storage units and / or devices may or may not be of the same type or located in the same physical location.

[0027] In one or more embodiments, any storage and / or memory of a computing device or computing device system may be considered, in whole or in part, to be a non-transitory computer-readable medium that stores software and / or firmware.

[0028] Such software and / or firmware may include instructions that, when executed by one or more processors (not shown) and / or other hardware (e.g., circuitry) of a computing device and / or computing device system, cause the one or more processors and / or other hardware components to perform operations in accordance with one or more embodiments described herein.

[0029] The software instructions may be in the form of computer-readable program code for executing the methods, processes, etc. of the embodiments described herein, and may, by way of example, be stored in whole or in part, temporarily or permanently, on a non-transitory computer-readable medium (such as a compact disc (CD), a digital versatile disc (DVD), a storage device, a disk, a tape storage device, a flash memory device, a physical memory, or any other non-transitory computer-readable medium).

[0030] although Figure 1 One computing device 100 is shown, but the system may include any number of computing devices, which may be collectively referred to as computing devices as used herein. All or any portion of the computing devices may be the same type or different types of computing devices.

[0031] In one or more embodiments, computing device 100 includes a processor 102. In one or more embodiments, processor 102 is any component that includes circuitry and / or other hardware for executing instructions (e.g., of a computer program). By way of example, such circuitry may be an integrated circuit implemented at least in part using transistors that implement components such as an arithmetic logic unit, a control unit, registers, a data buffer, a control buffer, and the like. In one or more embodiments, processor 102 may include any number of additional components (e.g., a cache memory). In one or more embodiments, processor 102 is configured to retrieve and / or decode instructions and then execute them. Execution of instructions may include operations on data, which may include reading and / or writing data. In one or more embodiments, instructions and / or data used by the processor are at least temporarily stored in memory (e.g., processor memory 104) of computing device 100. Processor 102 may perform various operations for executing software (e.g., an operating system, applications, etc.). Processor 102 may write data from the computing device 100's memory to a storage device and / or read data from a storage device via the computing device 100's memory. Processor 102 may, for example, be a CPU and may have any number of processing cores. although Figure 1 Computing device 100 is shown with a single processor 102 , but computing device 100 may include any number of processors without departing from the scope of the embodiments disclosed herein.

[0032] In one or more embodiments, the computing device 100 includes a memory 104. The processor memory 104 can be operably connected to the processor 102. In one or more embodiments, the memory 104 is any one or more memory devices. In one or more embodiments, the memory device is any type of computer memory. In one or more embodiments, the memory device includes a volatile memory device. As an example, the memory device can include RAM. In one or more embodiments, the data stored in the memory device is located at any number of memory addresses, and thus the processor 102 can access the data using the memory addresses. The processor 102 can write data to and / or read data from the processor memory 104. Although Figure 1 A single processor-memory 104 is shown, but the computing device 100 may include any number of processor-memories without departing from the scope of the embodiments disclosed herein.

[0033] In one or more embodiments, computing device 100 includes storage 106. In one or more embodiments, storage 106 is any type of computer storage (discussed above) configured to store any one or more types of data structures comprising any type of information. By way of example, storage 106 can be configured to store bit arrays, collections of objects, compressed data structures, and the like.

[0034] In one or more embodiments, computing device 100 includes a bit array generator 108. In one or more embodiments, bit array generator 108 is any hardware, software, firmware, or any combination thereof configured to generate a bit array. The bit array generator can be configured to generate a bit array of any size. The size of the bit array can be predetermined based on how quickly the final data structure is expected to be created. In one or more embodiments, larger arrays can result in fewer recursive rounds of the techniques described herein, but use more memory resources to store the larger array when executing the techniques. Conversely, smaller bit arrays can use fewer memory resources but also use more recursive rounds of the techniques. In one or more embodiments, bit array generator 108 can be configured to generate a bit array that is initially filled with zeros. In one or more embodiments, each position in the bit array generated by bit array generator 108 can be referred to as an index position within the bit array. As an example, a bit array can be generated that includes all zeros at ten positions corresponding to index positions zero through nine. In one or more embodiments, bit array generator 108 is operably connected to processor 102, storage device 106, and memory 104. The bit arrays generated by the bit array generator may be generated at least in part using processor 102 and may be stored in storage 106 and / or memory 104 for use in performing the techniques disclosed herein.

[0035] In one or more embodiments, computing device 100 includes a distributable data structure generator 110. In one or more embodiments, distributable data structure generator 110 is any hardware, software, firmware, or any combination thereof configured to generate a distributable data structure that can be used to test for inclusion of an object in a collection. In one or more embodiments, distributable data structure generator 110 is operatively connected to processor 102, memory 104, and storage 106. Distributable data structure generator 110 can be executed using processor 102 and can obtain data from and store data in storage 106 and / or memory 104.

[0036] In one or more embodiments, the distributable data structure generator 110 is configured to obtain a set of allowed objects (e.g., devices identified by a device identifier) and a total set of objects, the allowed set being a subset of the total set. In one or more embodiments, the distributable data structure generator 110 is configured to use the allowed set and the total set to obtain a set of disallowed objects. As an example, the disallowed set may be all objects in the total set except for objects in the allowed set. For example, the total set may be the set of UUIDs of all devices in a device ecosystem, the allowed set may be the set of UUIDs of devices that a particular user is allowed to access or about which the user is allowed to obtain information, and the disallowed set may be the set of other devices in the device ecosystem for which the particular user does not have such permission.

[0037] In one or more embodiments, the distributable data structure generator 110 is configured to perform any number of hash functions. In one or more embodiments, the distributable data structure generator 110 performs a hash function using an object identifier as input to obtain an output. In one or more embodiments, the output of the hash function is a value corresponding to an index position of the bit array generated by the bit array generator 108. In one or more embodiments, the hash function used by the distributable data structure generator 110 can be selected based on the provided output within a range of index positions of the bit array. As an example, if the bit array has one hundred index positions, a hash function that generates an output integer from zero to ninety-nine can be selected.

[0038] In one or more embodiments, the distributable data structure generator 110 is configured to update the bit array by setting each index position generated as an output of a hash function to one when an object identifier from the allowed set is used as input. If more than one object identifier from the allowed set results in a hash function output corresponding to the same index position, the distributable data structure generator 110 may maintain the value at that index position set to one. In one or more embodiments, once all object identifiers in the allowed set have been used as input to the hash function and the corresponding bits in the bit array are set to one, the result is a sparse bit array having ones at index positions generated using the hash function and the object identifiers of the allowed set.

[0039] In one or more embodiments, the distributable data structure generator 110 is configured to update the sparse bit array using the same hash function and object identifiers from the disallowed set. In one or more embodiments, when an object identifier from the disallowed set used as input to the hash function results in an index position in the output sparse bit array that is already zero, that index position remains zero. In one or more embodiments, when an object identifier from the disallowed set used as input to the hash function results in an index position in the output sparse bit array that was previously set to one (based on one or more allowed set object identifiers), the bit at that index position is reset to zero, and each identifier from the allowed set corresponding to that index position is added to a new allowed set. In one or more embodiments, the result of the above technique includes a new allowed set that is a subset of the initial allowed set and includes an object that conflicts with at least one object from the disallowed set. The result also includes a sparse bit array that includes zeros at all index positions that do not correspond to identifiers from the allowed set or at all index positions at which a conflict occurs between the disallowed set and the allowed set, and ones at all index positions that correspond to one or more identifiers from the allowed set but do not conflict with the disallowed set.

[0040] In one or more embodiments, the above-described technique performed by the distributable data structure generator 110 can be viewed as the first of a set of recursive rounds for generating a data structure comprising a set of sparse bit arrays, each of which is generated after a round of execution of the techniques disclosed herein. For the next round, the bit array generator 108 generates a new bit array comprising all zeros. In one or more embodiments, the distributable data structure generator 110 is configured to update the bit array using the new allowed set and the new hash function, with the bit array being one at the index position corresponding to the output of the new hash function, using as input an identifier from the new allowed set generated during the previous round. In one or more embodiments, the distributable data structure generator 110 is configured to again use the entire disallowed set and the new hash function to maintain the index position in the updated bit array at zero, or to reset the index position to zero when the disallowed set identifier and the new hash function result in a conflict with an index position that was previously set to one based on one or more identifiers from the allowed set. In one or more embodiments, each allowed set identifier corresponding to an index position is added to another new allowed set, and a new updated sparse bit array is obtained.

[0041] In one or more embodiments, any number of rounds may be performed until a round is reached in which no collisions occur with identifiers from the disallowed set, at which point the distributable data structure generator 110 may stop performing additional rounds of the technique. Each new round may use a different hash function than the previous round. The end result may be a set of sparse bit arrays. In one or more embodiments, the distributable data structure generator 110 is configured to use the set of sparse bit arrays to generate a data structure and then compress the data structure. In one or more embodiments, the data structure of the sparse bit arrays is compressed using a lossless compression technique. In one or more embodiments, because the data structure primarily includes zeros and some number of ones, the lossless compression produces a highly compressed data structure. Therefore, in one or more embodiments, the compressed data structure may be distributed to one or more entities (e.g., various services within and / or corresponding to a device ecosystem) that can test whether one or more object identifiers are in the allowed set. As an example, a compressed data structure may correspond to a set of allowed device identifiers for which a particular user in a device ecosystem has authority, and the compressed data structure may be small enough to be included in a header (e.g., an HTTP header) of a request made by the user to access information related to one or more devices in the device ecosystem. In such an example, a service receiving the request may decompress the data structure and use the device identifiers included in the request and the hash function used to generate the data structure to determine whether one or more device identifiers in the request are in the allowed set corresponding to the user. Figure 2 The creation of data structures is discussed further in the description of Figure 3 The use of compressed data structures to test set membership is discussed further in the description of .

[0042] Although Figure 1 A particular configuration of components is shown, but other configurations may be used without departing from the scope of the embodiments described herein. Figure 1 Certain components are shown as part of the same device, but any component may be grouped into a collection of one or more components that can exist and execute as part of any number of independent and operatively connected devices. As another example, a single component may be configured to execute the functions performed by Figure 1 Therefore, the embodiments disclosed herein should not be limited to all or any part of the functions performed by the components shown. Figure 1 Component configuration shown.

[0043] Figure 2An overview of an example method for generating a compressed data structure of a sparse bit array for testing membership of an object identifier in a set of allowed object identifiers according to one or more embodiments disclosed herein is illustrated. The method may be performed, for example, by a computing device (e.g., Figure 1 computing device 100, Figure 5 computing device 500) and / or any one or more components thereof (e.g., Figure 1 The process of executing the invention may be performed by the processor 102, memory 104, storage device 106, bit array generator 108 and / or distributable data structure generator 110 as shown.

[0044] Although Figure 2 The various steps in the flowchart shown are presented and described in sequence, but some or all of the steps may be performed in a different order, some or all of the steps may be combined or omitted, and / or some or all of the steps may be combined or omitted. Figure 2 and / or Figure 3 The other steps are executed in parallel.

[0045] In step 200, the method includes obtaining an allowed set of object identifiers and a total set of object identifiers. The allowed set of object identifiers may include any number of object identifiers and may correspond to a set of objects for which an entity has any type of permissions. For example, the entity may be a user in a device ecosystem, such as a network administrator, and the allowed set may include a set of UUIDs (Uniform User Interface IDs) that correspond to devices in the device ecosystem that the user is permitted to access and / or about which the user may obtain information. In one or more embodiments, the total set of object identifiers includes the object identifiers in the allowed set as well as all other object identifiers in the object set. For example, the total set may include the UUIDs of all devices in the device ecosystem. The allowed set and the total set may be obtained using any technique for obtaining information. For example, a device provider in the device ecosystem may maintain a total set of device identifiers in the device ecosystem, and an entity that controls, owns, or uses devices in the device ecosystem may provide a list of users, each with a corresponding allowed set of device identifiers. In one or more embodiments, an additional abstraction layer may be provided rather than the allowed set corresponding to a user, and the allowed set may be derived using the additional abstraction layer. For example, the device ecosystem may be a set of network devices deployed across multiple locations of a restaurant chain, and each user may be associated with a subset of the locations. Thus, an association between a user and one or more stores can be used to determine a set of devices in those stores whose identifiers are the user's allowed set.

[0046] In step 202, the method includes obtaining a disallowed object identifier set using the allowed set and the total set obtained in step 200. As an example, the object identifiers of the allowed set may be removed from the total set, and the remaining object identifiers may be the disallowed set.

[0047] In step 204, the method includes generating a bit array that initially includes a zero at each position in the bit array. This can be done by, for example Figure 1 The bit array generator 108 shown in and described above generates a bit array. The bit array can be of any size. The size of the bit array can be configurable. In one or more embodiments, the size of the bit array can be determined based on how quickly the final data structure is desired to be created. In one or more embodiments, a larger array can result in Figure 2 The method shown uses fewer recursive rounds, but uses more memory resources to store the larger array when executing the method. Conversely, a smaller bit array can use fewer memory resources, but can also use more recursive rounds of the method.

[0048] In step 206, the method includes updating the bit array using the object identifiers of the allowed set and a hash function. In one or more embodiments, each object identifier in the allowed set is used as an input to the hash function. In one or more embodiments, the output of the hash function is a value (e.g., an integer value) corresponding to the position in the bit array generated in step 204. In one or more embodiments, the bit array may be selected based on its ability to output a value corresponding to a position in the bit array. Figure 2 A set of hash functions used in the methods shown. The output of the hash function can be referred to as an index, and the corresponding position in the bit array can be referred to as an index position in the bit array. In one or more embodiments, each index position output from the hash function based on the input of an object identifier from the allowed set is set to one, while the other index positions remain at zero. In one or more embodiments, if more than one object identifier from the allowed set results in an index position output from the hash function, the value at the index position remains at one. After using each object identifier in the allowed set as input to the hash function and setting or maintaining the corresponding index position at one, the result is an updated bit array that includes a one at each index position corresponding to one or more object identifiers in the allowed set and zeros at the other positions in the bit array.

[0049] In step 208, the method further includes updating the bit array using the object identifiers in the disallowed set and a hash function to obtain an updated sparse bit array and a new allowed set. In one or more embodiments, each object identifier in the disallowed set is used as input to the hash function. In one or more embodiments, the output of the hash function is an index position in the bit array. In one or more embodiments, when the value at the index position output by the hash function based on the input of a given object identifier from the disallowed set is zero, the value remains zero. In one or more embodiments, when the value at the index position output by the hash function based on the input of a given object identifier from the disallowed set is one, due to a conflict (e.g., a hash conflict) between the object identifier from the disallowed set and one or more object identifiers from the allowed set, the bit is changed back to zero. In one or more embodiments, when such a conflict occurs, each of the one or more object identifiers from the allowed set corresponding to the index position where the conflict occurred is added to the new allowed set. Therefore, in one or more embodiments, the new allowed set object identifiers are a subset of the object identifiers of the initial allowed set, or a subset of the object identifiers of the previous allowed set after each round of steps 204 through 208.

[0050] In step 210, the method includes making a determination as to whether any object identifiers collided during the current round of analysis. In one or more embodiments, in step 208, as discussed above, a collision occurs whenever an object identifier from the complete disallowed set is used as input to the hash function for the current round of analysis, resulting in an index position that was previously set to one based on one or more object identifiers from the allowed set and the hash function. If one or more such collisions occur, the method returns to step 204. In one or more embodiments, if no such collisions occur, the method proceeds to step 212.

[0051] In one or more embodiments, steps 204 through 208 are repeated for as many rounds as necessary to reach a determination in step 210 that no collisions have occurred in the current round. In one or more embodiments, for each such round, the new allowed set generated in the last round as described above is used as the allowed set for step 206, and the complete disallowed set is used for step 208. In one or more embodiments, each round uses a different hash function than the hash function used in the previous round. The size of the bit array generated in step 204 can be the same for each round, or it can be different for each round.

[0052] In step 212, the method includes generating a data structure of one or more sparse bit arrays. In one or more embodiments, the number of sparse bit arrays in the data structure is the number of rounds of steps 204 to 208 performed before determining in step 210 that no conflict occurred. The data structure of the sparse bit arrays can be any type of data structure. As a non-limiting example, the data structure can be a simple series connection of the sparse bit arrays. The data structure can include additional information or be associated with the additional information in other ways. As an example, information related to how the data array is separated into the sparse bit arrays generated during each round of steps 204 to 208 can be included. As another example, the data structure can include information indicating what hash function corresponds to each sparse bit array. Other information can be included in the data structure without departing from the scope of the embodiments disclosed herein.

[0053] At step 214, the method includes compressing the data structure generated in step 212 using lossless compression to obtain a compressed data structure. In one or more embodiments, lossless compression is a form of data compression that uses entropy coding based on statistical properties of the compressed data so that the original data can be recovered via a decoding process. As an example, because the data structure includes a set of sparse bit arrays having many zeros and a certain number of ones, the data can be highly compressed based at least in part on knowing the number and positions of the ones in the sparse bit arrays.

[0054] In one or more embodiments, once the data structure is compressed in step 214, the result is a highly compressed data structure that can be distributed to any entity that can test whether an object identifier is in an allowed set, with the answer being deterministic rather than probabilistic.

[0055] In one or more embodiments, after obtaining the compressed data structure, the method may proceed to Figure 3 , where a compressed data structure is used to deterministically test allowed set membership.

[0056] Figure 3 An overview of an example method for using a compressed data structure of a sparse bit array for testing an object identifier for membership in a set of allowed object identifiers according to one or more embodiments disclosed herein is illustrated. The method may be performed, for example, by a computing device (e.g., Figure 1 computing device 100, Figure 5 computing device 500) to execute.

[0057] Although Figure 3 The various steps in the flowchart shown are presented and described in sequence, but some or all of the steps may be performed in a different order, some or all of the steps may be combined or omitted, and / or some or all of the steps may be combined or omitted. Figure 2 and / or Figure 3 The other steps are executed in parallel.

[0058] In step 300, the method includes distributing Figure 2 The compressed data structure obtained in step 214 can be distributed using any technique for providing data from one entity to another. As an example, the compressed data structure can be provided as a token or other form suitable for inclusion in a header when making a service request to an entity. As an example, the compressed data structure can be generated to correspond to a particular user with access rights to a subset of devices in a device ecosystem, where the identifiers of such devices are the allowed set corresponding to the user. In one or more embodiments, the token including the compressed data structure can be included in an HTTP header used each time the user makes a request related to a device in the device ecosystem, so that the service to which the request is sent can receive the compressed data structure without having to obtain information from any other source regarding which devices the user has permissions to. As another example, any number of microservices can be implemented within and / or corresponding to the device ecosystem, and the compressed data structure can be provided to all or any portion of such services that can be configured to receive requests from a user, where a allowed set membership test can be used to determine whether the user has appropriate permissions to access the devices included in the request.

[0059] In step 302, the method includes receiving a request including at least one object identifier. For example, a request may be made to a microservice for information about devices in a device ecosystem. The request may be received from a user of the device ecosystem. The request may include an identifier, such as a UUID, of the device for which information is being requested. The request may further include a compressed data structure associated with the user.

[0060] In step 304, the method includes decompressing the data structure to obtain the data structure. In one or more embodiments, the compressed data structure can be decompressed using any suitable technique for decompressing compressed data. As an example, when the data structure was compressed using entropy encoding, entropy decoding can be used to decompress the data structure.

[0061] In step 306, the method includes testing allowed set membership using at least one object identifier, the decompressed data structure, and one or more hash functions. In one or more embodiments, the decompressed data structure includes a set of one or more sparse bit arrays. In one or more embodiments, to test allowed set membership, each object identifier included in the request can be used as an input to the hash function(s) corresponding to the sparse bit array(s). As an example, for a first sparse bit array of the set of sparse bit arrays in the decompressed data structure, the object identifier can be used as an input to the hash function corresponding to the first sparse bit array to determine whether the sparse bit array includes a one at a result index position. In one or more embodiments, each object identifier included in the request is tested using the hash function and the corresponding bit array to determine whether any bit array includes a one at a corresponding index position of the bit array.

[0062] In step 308, the method includes making a determination as to whether one or more object identifiers are in an allowed set. In one or more embodiments, if the output of any hash function used in step 306 is an index position that includes a one in the corresponding sparse bit array, then the object identifier is in the allowed set and the method proceeds to step 312. In one or more embodiments, if the index position output by the hash function for each sparse bit array in the decompressed data structure includes a zero, then the object identifier is in the disallowed set (or at least not in the allowed set) and the method proceeds to step 310.

[0063] In step 310, the method includes denying the request. The request may be denied in whole or in part. By way of example, if no object identifiers in the request are included in the allowed set, the request may be denied in whole, whereas if only a portion of the object identifiers in the request are not included in the allowed set, the request may be denied only for those object identifiers. Denying the request may include providing a denial notification to the entity making the request and / or any other relevant entities. Denying the request may include logging the attempted request and its denial. Denying the request may include not responding to the request.

[0064] In step 312, the method includes servicing the request. In one or more embodiments, the request can be serviced for any object identifier included in the request that is determined to be in the allowed set. For example, if a user requests a microservice to view information about three devices identified by the device UUIDs in the request, the service can service the request by providing the requested information to the user.

[0065] It should be understood by those of ordinary skill in the art having the benefit of the present disclosure that although the foregoing description includes examples of devices as objects and device identifiers (e.g., UUIDs) as object identifiers, the techniques disclosed herein may use any type of object and any type of corresponding object identifier without departing from the scope of the embodiments disclosed herein. In addition, although the foregoing description uses a bit array initially generated with all zeros, with positions in the bit array set to ones in some manner, any pair of values that are distinguishable from each other may be used without departing from the scope of the embodiments disclosed herein. As an example, the bit array may be initially generated as all ones and set to zeros at positions corresponding to object identifiers in the allowed set.

[0066] Figure 4 Illustrated is an example scenario of using a compressed data structure of a sparse bit array to test allowed set membership according to one or more embodiments of the present disclosure. Figure 4 The examples shown and discussed below are highly simplified examples for explanation purposes only and are not intended to limit the scope of the embodiments described herein. Persons of ordinary skill in the art having the benefit of this disclosure should appreciate that such scenarios may include more network administrators or other users, the device ecosystem may include more devices (e.g., hundreds, thousands, millions, etc.), and / or the compressed data structure of the sparse bit array may be distributed in a manner different from that described below. Additionally, while this example illustrates certain aspects of the embodiments described herein, not all possible aspects of such embodiments may be illustrated in this particular example.

[0067] Consider a scenario where a user is a network administrator 404 who is part of a device ecosystem 406. Device ecosystem 406 is a group of devices from a vendor of retail goods. The vendor of retail goods has three stores: store A 408, store B 414, and store C 420. Store A includes two network devices: device A-1 410 and device A-2 412. Store B includes two network devices: device B-1 416 and device B-2 418. Store C includes two network devices: device C-1 422 and device C-2 424. Device ecosystem 406 also includes a permission set maintainer 422, which is a computing device that keeps track of which devices in device ecosystem 406 various users, such as network administrator 404, are allowed to access and which devices users are allowed to obtain information about.

[0068] In this scenario, network administrator 404 has been granted permission to receive information about all devices in store A 408 and information about device B1 416 in store B 414. Permission set maintainer 422 provides computing device 400 with an allow list in the form of the UUID of each device in each of the three stores that network administrator 404 is allowed to receive information about. Computing device 400 (which may be similar to Figure 1 The computing device 100 shown uses a set of UUIDs associated with the network administrator 404 to perform the above Figure 2 The method discussed in the description of to produce a compressed data structure that can be used to test whether a network administrator has permission to access a given device (e.g., whether the device is in an allowed set corresponding to a user).

[0069] Specifically, the computing device 400 uses the allowed set received from the allowed set maintainer 422 and the total set of UUIDs of the devices in the device ecosystem 406 to obtain the disallowed set corresponding to the network administrator 404. The computing device 400 generates a bit array of all zeros and then uses the UUIDs in the allowed set to set the values at the index positions output by the hash function when the UUID of the allowed set is used as input to one. The computing device then resets to zero any index positions in the bit array where the UUIDs of the devices in the disallowed set result in a hash collision and adds the allowed devices corresponding to the same positions to the new allowed set. As described above in Figure 2 As discussed in the description of , this process can go through several rounds until a round without hash collisions occurs. The resulting set of bit arrays is used to create a data structure, and the computing device compresses the data structure using a lossless compression technique to obtain a compressed data structure.

[0070] The compressed data structure is provided to the device of network administrator 404 in the form of a token to be included in an HTTP header of a request made by network administrator 404 to receive information about devices in device ecosystem 406. Such a request may be made by network administrator 404 to service 402 configured to monitor the performance of devices in device ecosystem 406.

[0071] Continuing with this scenario, a network administrator may desire to view page load times for devices. Network administrator 404 may send a request to service 402. The request may include a token that includes a compressed data structure. The request may also identify that the network administrator wants to view information about devices in store A 408 and store B 414. Service 402 may be configured to convert the stores included in the request into the UUIDs of the devices in those stores (e.g., device A-1 410, device A-2 412, device B-1 416, and device B-2 418).

[0072] Service 402 can decompress the compressed data structure received as part of the request. The decompressed data structure can then be used to check whether the network administrator is allowed to access the requested information for the four devices. To this end, each of the device's UUIDs can be used as input to a hash function corresponding to each sparse bit array in the decompressed data structure. Service 402 determines that the UUIDs of device A-1 410, device A-2 412, and device B-1 416, when used as input to one of the hash functions, produce an index position containing one in the sparse bit array. Service 402 also determines that the UUID of device B-2 418, when used as input to each hash function corresponding to each sparse bit array, does not result in a value of one at any index position in the sparse bit array. Therefore, the service determines that the request can be partially satisfied with the requested information for devices A-1 410, device A-2 412, and device B-1 416, but cannot be satisfied with the requested information for device B-2 418. Thus, service 402 provides the requested page load time data for three devices 410 , 412 , and 416 to network administrator 408 .

[0073] Figure 5 1 illustrates a block diagram of a computing device according to one or more embodiments of the present disclosure. As discussed above, the embodiments described herein may be implemented using a computing device. For example, Figure 1 All or any part of the components shown in can be implemented at least in part using one or more computing devices. Computing device 500 may include one or more computer processors 502, non-permanent storage 504 (e.g., volatile memory, such as random access memory (RAM), cache memory, etc.), permanent storage 506 (e.g., hard disk, optical drive such as compact disc (CD) drive or digital versatile disc (DVD) drive, flash memory, etc.), communication interface 512 (e.g., Bluetooth interface, infrared interface, network interface, optical interface, etc.), input device 510, output device 508 and many other elements (not shown) and functions. Each of these components is described below.

[0074] In one or more embodiments, the computer processor(s) 502 may be integrated circuits for processing instructions. For example, the computer processor(s) may be one or more cores or micro-cores of a processor. The processor 502 may be a general-purpose processor configured to execute program code included in software executed on the computing device 500. The processor 502 may be a special-purpose processor in which certain instructions are incorporated into the processor design. Although in Figure 5Only one processor 502 is shown, but computing device 500 may include any number of processors without departing from the scope of the embodiments disclosed herein.

[0075] The computing device 500 may also include one or more input devices 510, such as a touch screen, keyboard, mouse, microphone, touchpad, electronic pen, motion sensor, or any other type of input device. The input device 510 may allow a user to interact with the computing device 500. In one or more embodiments, the computing device 500 may include one or more output devices 508, such as a screen (e.g., a liquid crystal display (LCD), a plasma display, a touch screen, a cathode ray tube (CRT) monitor, a projector, or other display device), a printer, an external storage device, or any other output device. One or more of the output devices may be the same as or different from the input device(s). The input and output device(s) may be connected locally or remotely to the computer processor(s) 502, the non-persistent storage device 504, and the permanent storage device 506. Many different types of computing devices exist, and the input and output device(s) described above may take other forms. In some instances, a multimodal system may allow a user to provide multiple types of input / output to communicate with the computing device 500.

[0076] Further, the communication interface 512 can facilitate connecting the computing device 500 to a network (not shown) (e.g., a local area network (LAN), a wide area network (WAN) such as the Internet, a mobile network, or any other type of network) and / or another device (e.g., another computing device). The communication interface 512 can perform or facilitate receiving and / or transmitting wired or wireless communications using wired and / or wireless transceivers, including utilizing an audio jack / plug, a microphone jack / plug, a universal serial bus (USB) port / plug, Ports / plugs, Ethernet ports / plugs, fiber optic ports / plugs, proprietary wired ports / plugs, Wireless signal transmission, BLE wireless signal transmission, Wireless signal transmission, RFID wireless signal transmission, near field communication (NFC) wireless signal transmission, dedicated short range communication (DSRC) wireless signal transmission, 802.11 WiFi wireless signal transmission, WLAN signal transmission, visible light communication (VLC), Worldwide Interoperability for Microwave Access (WiMAX), IR communication wireless signal transmission, public switched telephone network (PSTN) signal transmission, integrated services digital network (ISDN) signal transmission, 3G / 4G / 5G / LTE cellular data network wireless signal transmission, ad hoc network signal transmission, radio wave signal transmission, microwave signal transmission, infrared signal transmission, visible light signal transmission, ultraviolet light signal transmission, wireless signal transmission along the electromagnetic spectrum, or some combination thereof. The communication interface 512 may also include one or more global navigation satellite system (GNSS) receivers or transceivers for determining the location of the computing device 500 based on one or more signals received from one or more satellites associated with one or more GNSS systems. GNSS systems include, but are not limited to, the United States' GPS, Russia's Global Navigation Satellite System (GLONASS), China's BeiDou Navigation Satellite System (BDS), and Europe's Galileo GNSS. There is no restriction to operate on any particular hardware arrangement, and therefore as hardware or firmware arrangements evolve, the basic features herein may be readily substituted for improved hardware or firmware arrangements.

[0077] The term "computer-readable medium" includes, but is not limited to, portable or non-portable storage devices, optical storage devices, and various other media capable of storing, containing, or carrying (multiple) instructions and / or data. Computer-readable media may include non-transient media in which data may be stored and does not include carrier waves and / or transient electronic signals transmitted wirelessly or via a wired connection. Examples of non-transient media may include, but are not limited to, disks or tapes, optical storage media such as CDs or DVDs, flash memories, memories, or memory devices. Code and / or machine-executable instructions may be stored on a computer-readable medium, and the code and / or machine-executable instructions may represent any combination of a process, function, subroutine, program, routine, subroutine, module, software package, class, or instruction, data structure, or program statement. A code segment may be coupled to another code segment or hardware circuit by transmitting and / or receiving information, data, independent variables, parameters, or memory contents. Information, independent variables, parameters, data, etc. may be transmitted, forwarded, or transmitted by any suitable means (including memory sharing, message passing, token passing, network transmission, etc.).

[0078] All or any portion of the components of computing device 500 may be implemented in circuits. For example, the components may include and / or may be implemented using electronic circuits or other electronic hardware, which may include one or more programmable electronic circuits (e.g., microprocessors, GPUs, DSPs, CPUs, and / or other suitable electronic circuits), and / or may include and / or be implemented using computer software, firmware, or any combination thereof to perform the various operations described herein. In some aspects, computer-readable storage devices, media, and memories may include wired or wireless signals including bitstreams, etc. However, when referred to, non-transitory computer-readable storage media explicitly excludes media such as energy, carrier signals, electromagnetic waves, and the signals themselves.

[0079] In the description above, many details are set forth as examples of the embodiments described herein. Those skilled in the art (who also have the benefit of this detailed description) will understand that one or more of the embodiments described herein can be practiced without these specific details, and that many changes or modifications can be made without departing from the scope of the embodiments described herein. Certain details known to those of ordinary skill in the art may be omitted to avoid obscuring the description.

[0080] Specific details are provided in the description above to provide a thorough understanding of the aspects and examples provided herein. However, it will be understood by those skilled in the art that these aspects can be put into practice without these specific details. For clarity of explanation, in some instances, the present technology can be presented as including functional blocks, which can include devices, device components, steps or routines in the method embodied in software, or a combination of hardware and software. In addition to the components shown in the figures and / or described herein, additional components can also be used. For example, circuits, systems, networks, processes and other components can be shown as components in block diagram form to avoid blurring these aspects in unnecessary details. In other instances, known circuits, processes, algorithms, structures and techniques can be shown without unnecessary details to avoid blurring these aspects.

[0081] The various aspects described above may be described as processes or methods depicted as flowcharts, flow diagrams, data flow diagrams, structure diagrams, or block diagrams. Although a flowchart may describe operations as a sequential process, many operations may be performed in parallel or simultaneously. In addition, the order of the operations may be rearranged. When the operations of a process are completed, the process terminates, but may have additional steps not included in the diagram. A process may correspond to a method, function, process, subroutine, subprogram, etc. When a process corresponds to a function, the termination of the process may correspond to the function returning to the calling function or main function.

[0082] The process and method according to the above-mentioned example can be implemented using computer-executable instructions stored in a computer-readable medium or otherwise obtained from a computer-readable medium. Such instructions may include, for example, instructions and data that enable a general-purpose computer, a special-purpose computer or a processing device to perform a specific function or function group or otherwise configure a general-purpose computer, a special-purpose computer or a processing device to perform a specific function or function group. The partial computer resources used can be accessed through a network. Computer-executable instructions can be, for example, binary files, intermediate format instructions (such as assembly language, firmware, source code, etc.). The example of a computer-readable medium that can be used to store instructions, information used, and / or information created during the method according to the example includes a disk or optical disk, a flash memory, a USB device equipped with a non-volatile memory, a network storage device, etc.

[0083] In the above description of the drawings, in the various embodiments described herein, any component described with respect to a drawing may be equivalent to one or more identically or similarly named and / or numbered components described with respect to any other drawing. For the sake of brevity, the description of these components may not be repeated for each drawing. Therefore, each embodiment of the components of each drawing is incorporated by reference and is assumed to be optionally present in each other drawing having one or more identically or similarly named and / or numbered components. In addition, according to the various embodiments described herein, any description of the components of the drawings should be interpreted as an optional embodiment, which can be implemented as a supplement to, in combination with, or as an alternative to the embodiment described with respect to the corresponding one or more identically or similarly named and / or numbered components in any other drawing.

[0084] Throughout the application, ordinal numbers (e.g., first, second, third, etc.) may be used as adjectives for elements (i.e., any noun in the application). The use of ordinal numbers does not imply or create any particular order of elements, nor does it limit any element to being only a single element, unless explicitly disclosed, such as by using the terms "before," "after," "single," and other such terms. Rather, ordinal numbers are used to distinguish elements. For example, a first element is different from a second element, and a first element may contain more than one element and be after (or before) a second element in the ordering of elements.

[0085] As used herein, the phrase "operably connected" or "operably coupled" means that there is a direct or indirect connection between elements / components / devices that allows the elements to interact with each other in some manner. For example, the phrase "operably connected" can refer to any direct connection (e.g., a direct wired connection between two devices or components) or indirect connection (e.g., a wired and / or wireless connection between any number of devices or components that connect operably coupled devices). Thus, any path that information can take can be considered an operable connection.

[0086] Although the embodiments discussed herein have been described with respect to a limited number of embodiments, those skilled in the art having benefit of the detailed description will appreciate that other embodiments can be devised which do not depart from the scope of the embodiments disclosed herein. Accordingly, the scope of the embodiments described herein should be limited only by the appended claims.

Claims

1. A device comprising: one or more processors; as well as One or more non-transitory computer-readable media storing instructions that, when executed by the one or more processors, cause the one or more processors to: obtaining an allowed set comprising a first plurality of identifiers corresponding to a plurality of objects; Obtaining a disallowed set including a second plurality of identifiers based on the allowed set and the total set of identifiers; generating an initial bit array comprising a first value at each of a plurality of positions in the initial bit array; obtaining a first set of index values corresponding to a subset of locations of the plurality of locations using the first plurality of identifiers and a first hash function of the plurality of hash functions; setting each position in the subset of positions in the initial bit array to a second value to obtain an updated initial bit array; performing a first collision check using the first hash function, the disallowed set, and the updated initial bit array to obtain a first bit array and a new allowed set including a subset of identifiers of the allowed set; Obtaining a new set of index values using the new allowed set and a new hash function from the plurality of hash functions; setting a new subset of positions of a new initial bit array to the second value using the new set of index values; performing a second collision check using the disallowed set, the new hash function, and the new initial bit array to obtain a second bit array; When the result of the second conflict check is no conflict, generating a data structure including the first bit array; as well as compressing the data structure to obtain a compressed data structure, wherein the compressed data structure is distributed for testing membership in the allowed set in response to a request from a user including an identifier of an object, and The test is used to determine whether the user has permission to access the object included in the request.

2. The device according to claim 1, wherein Each identifier of the first plurality of identifiers and the second plurality of identifiers is an individual device identifier corresponding to a plurality of devices of a device ecosystem.

3. The apparatus of claim 1, wherein: The compressed data structure is compressed using a lossless compression technique.

4. The apparatus of claim 1, wherein: The instructions, when executed by the one or more processors, further cause the one or more processors to: A plurality of rounds of sparse bit array generation are performed after performing the first collision check and before performing the second collision check.

5. The apparatus of claim 1, wherein: The compressed data structure is distributed to the user for inclusion in the request.

6. The apparatus of claim 1, wherein: The instructions, when executed by the one or more processors, further cause the one or more processors to: The plurality of hash functions are selected based at least in part on each of the plurality of hash functions outputting a value corresponding to an index position of the first bit array.

7. A computer-implemented method comprising: obtaining, by a processing device, an allowed set comprising a first plurality of identifiers corresponding to a plurality of objects; Obtaining, by the processing device, a disallowed set including a second plurality of identifiers based on the allowed set and the total set of identifiers; generating, by the processing device, an initial bit array, the initial bit array comprising a first value at each of a plurality of positions in the initial bit array; obtaining a first set of index values corresponding to a subset of locations of the plurality of locations using the first plurality of identifiers and a first hash function of the plurality of hash functions; setting each position in the subset of positions in the initial bit array to a second value to obtain an updated initial bit array; performing a first collision check using the first hash function, the disallowed set, and the updated initial bit array to obtain a first bit array and a new allowed set including a subset of identifiers of the allowed set; Obtaining a new set of index values using the new allowed set and a new hash function from the plurality of hash functions; setting a new subset of positions of a new initial bit array to the second value using the new set of index values; performing a second collision check using the disallowed set, the new hash function, and the new initial bit array to obtain a second bit array; When the result of the second conflict check is no conflict, generating a data structure including the first bit array; as well as compressing the data structure to obtain a compressed data structure, wherein the compressed data structure is distributed for testing membership in the allowed set in response to a request from a user including an identifier of an object, and The test is used to determine whether the user has permission to access the object included in the request.

8. The computer-implemented method of claim 7, wherein: Each identifier of the first plurality of identifiers and the second plurality of identifiers is an individual device identifier corresponding to a plurality of devices of a device ecosystem.

9. The computer-implemented method of claim 7, wherein: The compressed data structure is compressed using a lossless compression technique.

10. The computer-implemented method of claim 7, further comprising: A plurality of rounds of sparse bit array generation are performed after performing the first collision check and before performing the second collision check.

11. The computer-implemented method of claim 7, wherein: The compressed data structure is distributed to the user for inclusion in the request.

12. The computer-implemented method of claim 7, further comprising: The plurality of hash functions are selected based at least in part on each of the plurality of hash functions outputting a value corresponding to an index position of the first bit array.

13. A non-transitory computer-readable medium storing a programming program for execution by one or more processors, the programming program comprising instructions for performing the following operations: obtaining an allowed set comprising a first plurality of identifiers corresponding to a plurality of objects; Obtaining a disallowed set including a second plurality of identifiers based on the allowed set and the total set of identifiers; generating an initial bit array comprising a first value at each of a plurality of positions in the initial bit array; obtaining a first set of index values corresponding to a subset of locations of the plurality of locations using the first plurality of identifiers and a first hash function of the plurality of hash functions; setting each position in the subset of positions in the initial bit array to a second value to obtain an updated initial bit array; performing a first collision check using the first hash function, the disallowed set, and the updated initial bit array to obtain a first bit array and a new allowed set including a subset of identifiers of the allowed set; Obtaining a new set of index values using the new allowed set and a new hash function from the plurality of hash functions; setting a new subset of positions of a new initial bit array to the second value using the new set of index values; performing a second collision check using the disallowed set, the new hash function, and the new initial bit array to obtain a second bit array; When the result of the second conflict check is no conflict, generating a data structure including the first bit array; as well as compressing the data structure to obtain a compressed data structure, wherein the compressed data structure is distributed for testing membership in the allowed set in response to a request from a user including an identifier of an object, and The test is used to determine whether the user has permission to access the object included in the request.

14. The non-transitory computer readable medium of claim 13, wherein: Each identifier of the first plurality of identifiers and the second plurality of identifiers is an individual device identifier corresponding to a plurality of devices of a device ecosystem.

15. The non-transitory computer readable medium of claim 13, wherein: The compressed data structure is compressed using a lossless compression technique.

16. The non-transitory computer readable medium of claim 13, wherein: The program includes further instructions for: A plurality of rounds of sparse bit array generation are performed after performing the first collision check and before performing the second collision check.

17. The non-transitory computer readable medium of claim 13, wherein: The compressed data structure is distributed to the user for inclusion in the request.

Images