Methods and systems for efficient computerized generation and storage of bases for insider risk management

Abstract

A method of insider risk management includes receiving and analyzing data records to generate an analytics outcome. The method includes determining a subset of the data records that affect the analytics outcome. The method includes generating, by hash functions, and storing, in memory, a data record basis marker for a respective data record of the data records and / or an analytics process basis marker for the analytics process. The data record basis marker includes information associated with the analytics process. The analytics process basis marker includes information associated with the subset of the data records. The method includes receiving a user request to recall a basis information relating to the analytics outcome and in response, locating, via a query, the data record basis marker and / or the analytics process basis marker, and providing information from the data record basis marker and / or the analytics process basis marker as the basis information.

Description

Methods and Systems for Efficient Computerized Generation and Storage of Bases for Insider Risk ManagementTECHNICAL FIELD

[0001] The disclosed embodiments relate generally to generating and storing bases for analytic results, and in particular, to using probabilistic data structures to generate and store the bases for analytic results in an efficient manner (e.g., with respect to computer memory).BACKGROUND

[0002] Traditional insider risk management systems are “black box” systems. These black box systems do not provide information to the user regarding which input data and / or which analytics processes are used to generate the analytics output. Without such information, it is often unknown which input data or analytics process substantiates or influences the analytics output.

[0003] Some other traditional risk management systems offer rudimentary information regarding input data and / or analytics processes. These systems duplicate a substantial portion, or all, of the input data. The input data can include a large variety of data types and can be retrieved or ingested from a large number of data sources. Thus, storage of the duplicated copies of this data can be difficult, at least in part because of the extensive volumes of data storage for the duplicative data.SUMMARY

[0004] Various implementations of systems, methods, and devices within the scope of the appended claims each have several aspects, no single one of which is solely responsible for the attributes described herein. Without limiting the scope of the appended claims, after considering this disclosure, and particularly after considering the section entitled “Detailed Description” one will understand how the aspects of various implementations are used to provide bases for analytic results. In one aspect, a data record basis marker and / or an analytics process basis marker is generated, by hash functions, and stored, in memory, to represent information associated with a subset of data records that affected an analyticsoutcome of an analytics process. In some embodiments, the data record basis marker and / or the analytics process basis marker is generated from a Bloom filter or a Cuckoo filter. In some embodiments, the user queries the data record basis marker to identify whether an analytics process was used to analyze a data record. In some embodiments, the user queries the analytics process basis marker to identify whether a data record was used in the analytics process.BRIEF DESCRIPTION OF THE DRAWINGS

[0005] So that the present disclosure can be understood in greater detail, a more particular description may be had by reference to the features of various implementations, some of which are illustrated in the appended drawings. The appended drawings, however, merely illustrate the more pertinent features of the present disclosure and are therefore not to be considered limiting, for the description may admit to other effective features.

[0006] FIG. l is a diagram illustrating an implementation of an insider risk management system, in accordance with some embodiments.

[0007] FIG. 2A is a diagram illustrating an implementation of a Bloom filter or a Cuckoo filter to generate an analytics process basis marker, in accordance with some embodiments.

[0008] FIG. 2B is a diagram illustrating an implementation of a Bloom filter or a Cuckoo filter to generate a data record basis marker, in accordance with some embodiments.

[0009] FIGS. 3A-3B are diagrams illustrating implementations of querying whether a data record is present in an analytics process basis marker, in accordance with some embodiments.

[0010] FIG. 3C is a diagram illustrating an implementation of querying whether a analytics process is present in a data record basis marker, in accordance with some embodiments.

[0011] FIG. 4A-4B are diagrams illustrating an implementation of cross validation between an analytics process basis marker and a data record basis marker, in accordance with some embodiments.

[0012] In accordance with common practice the various features illustrated in the drawings may not be drawn to scale. Accordingly, the dimensions of the various features may be arbitrarily expanded or reduced for clarity. In addition, some of the drawings may not depict all of the components of a given system, method or device. Finally, like reference numerals may be used to denote like features throughout the specification and figures.DETAILED DESCRIPTION

[0013] Storing duplicated data requires a prohibitive and inefficient amount of computer memory. The duplicate data requires additional physical storage requirements. The duplicated data will also require additional computing power to process the additional data (e.g., the duplicated data). In the embodiments disclosed below, to provide a basis for an analytics outcome, representations of data records or analytics processes are generated and stored in memory. The representations of the data records or the analytics process may be generated by a hash function. The basis for an analytics outcome may be stored with a Bloom filter or a Cuckoo filter.

[0014] In some embodiments, the method includes receiving a plurality of data records. The data records are analyzed by an analytics process to generate an analytics outcome. The method includes determining a subset of the plurality of data records that meet presumptive criteria for affecting the analytics outcome for the analytics process. The method includes generating, and storing in memory, a data record basis marker. The data record basis marker is for a respective data record of the subset of the plurality of data records that meet the presumptive criteria for affecting the analytics outcome of the analytics process. The data record basis marker is generated by one or more hash functions with the data record basis marker. The method includes receiving a user request to recall a basis information relating to the analytics outcome. The method includes, in response to the user request to recall the basis information related to the analytics outcome, locating, via a query, the data record basis marker. The method includes providing information from the data record basis marker.

[0015] Consider an example in which employees are being monitored for risk of theft of trade secrets. It is important to know which employee(s) created the risk. The risk may be based on employee behavior that is similar with behavior associated with individuals who have stolen trade secrets. For example, such behavior may include an employee who beginsdownloading a large number of files. The risk may also be based on outside factors. For example, such outside factors may include a wage garnishment.

[0016] In some embodiments, the method includes generating, and storing in memory, an analytics process basis marker. The analytics process basis marker includes information associated with the subset of the plurality of data records that meet the presumptive criteria for affecting the analytics outcome of the analytics process. The analytics process basis marker is generated by one or more hash functions associated with the analytics basis marker. The method includes, in response to the user request to recall the basis information related to the analytics outcome, locating, via a query, the analytics process basis marker. The method includes providing information from the analytics process basis marker.

[0017] In some embodiments, the method includes generating both a data record basis marker for a respective data record and an analytics basis marker for the analytics process.

[0018] In some embodiments, the method includes receiving an updated data record of the data record of the plurality of data records. The information associated with the updated data record is different from information associated with the data record. The method includes analyzing, by the analytics process, the updated data record to update the analytics outcome. The method includes determining that the updated data record meets the presumptive criteria for affecting the analytics outcome for the analytics process. The method includes updating, and storing in the memory, the data record basis marker for the data record. The data record basis marker includes information associated with the analytics process based on the updated data record.

[0019] In some embodiments, the method includes, in response to the analytics process being modified, analyzing, by the modified analytics process, the plurality of data records to generate a modified analytics outcome. The method includes determining a modified subset of the plurality of data records that meet the presumptive criteria for affecting a modified analytics outcome for the modified analytics process. The method includes updating, and storing in the memory, the analytics process basis marker for the analytics process. The analytics process basis marker includes information associated with the modified subset of the plurality of data records that meet the presumptive criteria for affecting the modified analytics outcome of the modified analytics process.

[0020] In some embodiments, the method includes analyzing, by a second analytics process, the plurality of data records to generate a second analytics outcome. The method includes determining a second subset of the plurality of data records that meet presumptive criteria for affecting the second analytics outcome for the second analytics process. The method includes generating, and storing in the memory, a second analytics process basis marker for the second analytics process. The second analytics process basis marker includes information associated with the second subset of the plurality of data records that meet the presumptive criteria for affecting the second analytics outcome of the second analytics process. The second analytics process basis marker is generated by one or more hash function associated with the second analytics process basis marker.

[0021] In some embodiments, the second analytics process is based on the first analytics process. The method includes updating the analytics outcome based on the second analytics outcome and updating the analytics basis marker based on the second analytics basis marker.

[0022] In some embodiments, a size of the data record basis marker is based on a hash collision rate associated with the data record basis marker being below a specified hash collision threshold associated with the data record basis marker. The size of the data record is a number of bits stored in the memory.

[0023] In some embodiments, a size of the analytics process basis marker is based on a hash collision rate associated with the analytics basis marker being below a specified hash collision threshold associated with the analytics basis marker. The size of the analytics process basis marker is a number of bits stored in the memory.

[0024] In some embodiments, the specified hash collision threshold associated with the data record basis marker is based on a number of analytics subprocesses of the analytics process.

[0025] In some embodiments, the specified hash collision threshold associated with the analytics process basis marker is based on a number of the plurality of data records.

[0026] In some embodiments, the method includes providing information associated with the analytics process which, when analyzing the respective data record of the subset ofthe plurality of data records associated with the data record basis marker, affected the analytics outcome.

[0027] In some embodiments, the method includes providing information associated with the subset of the plurality of data records which met the presumptive criteria for affecting the analytics outcome for the analytics process.

[0028] In some embodiments, the method includes storing the data record basis marker and the analytics basis marker as an aggregate basis marker associated with the analytics outcome.

[0029] In some embodiments, the method includes, in response to the user request to recall the basis information related to the analytics outcome, locating, via a query, the aggregate basis marker. The method includes providing information from the aggregate basis marker as the basis information relating to the analytics outcome. The aggregate basis marker includes information from the data record basis marker and the analytics basis marker.

[0030] In some embodiments, the aggregate basis marker provides information associated with the analytics processes which, when analyzing the respective data record of the subset of the plurality of data records associated with the data record basis marker, affected the analytics outcome.

[0031] In some embodiments, the aggregate basis marker provides information associated with the subset of the plurality of data records which met the presumptive criteria for affecting the analytics outcome for the analytics process.

[0032] In some embodiments, the data record basis marker and / or the analytics process basis marker is generated from a Bloom filter or a Cuckoo filter. In some embodiments, the set of hash functions includes a Bloom filter or a Cuckoo filter.

[0033] Numerous details are described herein in order to provide a thorough understanding of the example implementations illustrated in the accompanying drawings. However, some embodiments may be practiced without many of the specific details, and the scope of the claims is only limited by those features and aspects specifically recited in the claims. Furthermore, well-known methods, components, and circuits have not been described in exhaustive detail so as not to unnecessarily obscure more pertinent aspects of the implementations described herein.

[0034] FIG. l is a diagram illustrating an implementation of an insider risk management system 100, in accordance with some embodiments. While some example features are illustrated, various other features have not been illustrated for the sake of brevity and so as not to obscure more pertinent aspects of the example implementations disclosed herein. To that end, as a non-limiting example, insider risk management system 100 includes data source 110. Data source 110 includes one or more data record 120. Data record 120 is evaluated by analytics process 130. Analytics process 130 generates analytics outcome 140. Examples of analytics processes include, but are not limited to, statistical analysis, link analysis, machine learning, artificial intelligence, rules-based analysis, model driven analysis, or combinatorial analytics. A subset of the plurality of data records is determined to meet presumptive criteria for affecting the analytics outcome for the analytics process. For example, the presumptive criteria may be a statistical analysis of how statistically impactful the data record was. In this example, if a data record is insufficiently impactful, the presumptive criteria is not met. Analytics process 130 generates data record basis marker 150 through process A (described in FIG. 2A) and analytics process basis marker 160 through process B (described in FIG. 2B). Generation of data record basis marker 150 and analytics process basis marker 160 can happen concurrently with generation of analytics outcome 140. One advantage of concurrent generation is the basis can be queried quickly without having to re-run the analytics process. Generation of data record basis marker 150 and analytics process basis marker 160 can happen at a later point in time. One advantage of asynchronous generation of the basis markers is reducing the initial run-time required to run the analytics process. Analytics outcome 140, data record basis marker 150, and analytics process basis marker 160 are used to prepare basis 170, which is outputted as report 180.

[0035] Data source 110 may range from a single data source to a large variety of data types and a large number of data sources. Data source 110 includes data from digital network analytics (e.g., network usage, network behavior), enterprise data, open-source intelligence, human resource solutions (performance plans, disciplinary, resignations, terminations), physical security (e.g., physical surveillance, facility access), e-comms (email, social media, instant messaging), endpoint data loss prevention (Endpoint DLP), user activity monitoring, security information and event management (SIEM) (e.g., employee access to sensitive materials or critical assets), and / or user entity and behavior analytics (EUBA). Open-source intelligence includes public record and credit history. SIEM systems may providecybersecurity telemetry in the form of log and event data. EUBA systems may provide alerts based on deviations from established user and network behavior baselines.

[0036] In some embodiments, a user requests to recall basis information relating to the analytics outcome. Preparing basis 170 includes compiling data record basis marker 150 and analytics process basis marker 160. For example, preparing basis 170 for data records 1- 10 includes compiling data record basis markers for data records 1-10 and / or analytics process basis markers for the analytics processes that used data records 1-10. In another example, preparing basis 170 for analytics process 1 includes compiling analytics process basis markers for analytics process 1 and / or data record basis markers for data records used in analytics process 1. The user may select which data record(s) 150 and / or analytics process(es) 130 to recall basis information for.

[0037] In some embodiments, in response to the user request to recall the basis information, the data record basis marker and / or the analytics process basis marker is located via a query. The query may be a hash representing the data record or the analytics process basis marker. The hash may be generated by one or more algorithms. Example algorithms include Bloom filters and Cuckoo filters. A hash representing the data record is considered to be included in the analytics process basis marker when all the marked bits in the hash representing the data record are also marked in the analytics process basis marker. A hash representing the analytics process is considered to be included in the data record basis marker when all the marked bits in the hash representing the analytics process are also marked in the data record basis marker. The information is then provided to the user. The information may be provided as report 180.

[0038] In some embodiments, an updated data record is received. The updated data record is an update relative to the data record. The updated data record contains information that is different from the information associated with the data record. The updated data record is analyzed by the analytics process to update the analytics outcome. If the updated data record meets the presumptive criteria for affecting the analytics outcome, update and store the analytics process basis marker to include a hash representing the updated data record. For example, if the data record was originally included in the analytics process basis marker, and the updated data record still meets the presumptive criteria, update the analytics process basis marker to include the updated data record. However, in another example, if the updated datarecord does not meet the presumptive criteria, do not update the analytics process basis marker. In some embodiments, the hash of the data record associated with the updated data record is removed from the analytics process basis marker if the updated data record does not meet the presumptive criteria.

[0039] In some embodiments, a modified analytics process is received. The modified analytics process contains a modification to the analytics process that changes the analytics process. The data records are analyzed by the modified analytics process to output a modified analytics outcome. A modified subset of the data records is determined to still meet the presumptive criteria for affecting the modified analytics outcome associated with the modified analytics process. The analytics basis marker for the analytics process is updated with the information associated with the modified subset of the plurality of data records that meet the presumptive criteria for affecting the modified analytics outcome of the modified analytics process.

[0040] In some embodiments, the system analyzes multiple analytics processes concurrently (e.g., a process for detecting theft of trade secrets and a process for detecting inappropriate behavior). To that end, in some embodiments, a second analytic process analyzes the aforementioned plurality of data records to generate a second analytics outcome. The second analytics process may be distinct from the analytics process. The second analytics process may be based on the first analytics process. For example, the first analytics process may be a pre-processing step. Alternatively, the analytics process and the second analytics process may be different rule-based heuristics targeted at surfacing different insights but rely on the same fundamental parameters. A second subset of the plurality of data records may be determined to meet presumptive criteria for affecting the second analytics outcome for the second analytics process. A second analytics basis marker for the second analytics process may be generated and stored in memory.

[0041] If a particular data record still meets the presumptive criteria for affecting the analytics outcome, update and store the analytics process basis marker to include a hash representing the updated data record. For example, if the data record was originally included in the analytics process basis marker, and the updated data record still meets the presumptive criteria, update the analytics process basis marker to include the updated data record. However, in another example, if the updated data record does not meet the presumptivecriteria, do not update the analytics process basis marker. In some embodiments, the hash of the data record associated with the updated data record is removed from the analytics process basis marker if the updated data record does not meet the presumptive criteria.

[0042] In some embodiments, the data record basis marker and the analytics process basis marker are stored together as an aggregate basis marker associated with the analytics outcome. The aggregate basis marker includes information from the data record basis marker and the analytics process basis marker.

[0043] In some embodiments, in response to a user request to recall the basis information related to the analytics outcome, the aggregate basis marker is located via a query.

[0044] FIG. 2A is a diagram illustrating an implementation of a Bloom filter or a Cuckoo filter to generate an analytics process basis marker, in accordance with some embodiments. The hash and hash functions described below are scaled down for explanation purposes. It should be appreciated that the hash may be larger (e.g., instead of 4 or 8 bits, the hash may have tens, hundreds, thousand of bits). The hash may be any size that can be stored and processed. Furthermore, it should be appreciated that the hash function may process data of any size to generate a hash of any size.

[0045] A Bloom filter is a probabilistic data structure used to determine if an element “x” is a member of a set “S” with high probability. For example, the Bloom filter may be used to determine if a data record (e.g., “x”) is a member of analytics process 1 basis marker (e.g., “S”). In another example, the Bloom filter may be used to determine if an analytics process 1 (e.g., “x”) is a member data record 1 basis marker (e.g., “S”). A Bloom filter is constructed using an N-bit array that is initially cleared, and has hash functions where 0 < Hash (x,k) < N-l. For each element “x” in set “S,” k hash functions are computed, and the k corresponding bits in the N-bit array are set. In some embodiments, a Bloom filter is initially cleared by resetting the N-bit array to all zeros, and the k corresponding bits in the N-bit array are set to ones. In some embodiments, a Bloom filter is initially cleared by resetting the N-bit array to all ones, and the k corresponding bits in the N-bit array are set to zeros. Any labeling or mapping of memory cell states to data values can be used, as long as it is used consistently.

[0046] As an example, to test an element “w” for membership in the set “S,” the k hash functions are generated for element “w” and the k bit positions are tested. In someembodiments, if the k bit positions are set, then the element “w” is most likely a member of set “S,” with a possibility of this membership being a “false positive.” A false positive is when the Bloom filter returns a result that an element is a member of the set “S,” when in actuality it is not. Bloom filters return fewer false positives when the number of elements in the set “S” is an order of magnitude smaller than the number of bits in the bit array.

[0047] As illustrated in FIG. 2A, data record 1 (210a) is analyzed in analytics processes 1 and 2, data record (210b) is analyzed in analytics processes 1 and 2, and data record 3 (210c) is analyzed in analytics processes 1 and 3. As described in FIG. 1, a data record is analyzed in an analytics process when the data record is analyzed using statistical analysis, link analysis, machine learning, artificial intelligence, rules-based analysis, model driven analysis, or combinatorial analytics. For example, data record 1 may be analyzed by analytics processes 1 and 2, wherein analytics process 1 and 2 are different. In this example, analytics process 1 may be a statistical analysis, and analytics process 2 may be a machine learning model.

[0048] Each data record 210 is hashed using one or more hash functions 220. Hash function 220 may include non-cryptographic hash functions, such as murmur, FNV series of hash functions, CityHash or Jenkins hashes. Hash function 220 may include cryptographic hash functions, such as SHA-256 or MD5.

[0049] In some embodiments, the hashed data records are outputted as hash function outputs 230. For example, hash function output 1 (230a) for data record 1 (210a) is 01010000, hash function output 2 (230b) for data record 2 (210b) is 01000010, and hash function output 3 (230c) for data record 3 (210c) is 10000001. The hash function outputs are representations of data records 210.

[0050] In some embodiments, analytics process basis marker 240 includes the hashed data record(s) that were used in the analytics process associated with analytics process basis marker 240. For example, each data record that was analyzed by analytics process 1 is added to analytics process 1 basis marker (240a). Analytics process 1 basis marker (240a) starts empty (e.g., 00000000). Then, the hash function output for data record 1 (230a) (e.g., 01010000) is added to analytics process 1 basis marker (240a) bitwise. For example, positions 1-8 in analytics process 1 basis marker (240a) are all not marked (e.g., “0”) and positions 2 and 4 in the hash function output of data record 1 (230a) are marked (e.g., “1”),and the remaining positions are not marked. Thus, adding hash function output of data record 1 (230a) to analytics process 1 basis marker (240a) marks positions 2 and 4 in analytics process 1 basis marker (240a). This results in the analytics process 1 basis marker (240a) being 01010000. Then, the hash function output for data record 2 (230b) (e.g., 01000010) is added to analytics process 1 basis marker (240a). In this instance, analytics process 1 basis marker (240a) already has positions 2 and 4 marked, because of hash function output of data record 1 (230a). Hash function output of data record 2 (230b) has positions 2 and 7 marked. Adding hash function output of data record 2 (230b) to analytics process 1 basis marker (240a) additionally marks position 7. Position 2 remains unchanged because it is already marked from hash function output of data record 1 (230a). This results in the analytics process 1 basis marker (240a) being 01010010. The aforementioned steps are repeated for data record 3, which results in analytics process 1 basis marker (240a) being 11010011.

[0051] The aforementioned steps are repeated for analytics process 2 basis marker (240b) with data record 1 (210a) and data record 2 (210b). The aforementioned steps are repeated for analytics process 3 basis marker (240c) with data record 3 (210c).

[0052] The hash function output may be stored in an array and the analytics process basis marker may be stored in a different array with the same length. A size of the analytics process basis marker is based on a hash collision rate associated with the analytics process basis marker being below a specified hash collision threshold associated with the analytics process basis marker.

[0053] FIG. 2B is a diagram illustrating an implementation of a Bloom filter or a Cuckoo filter to generate a data record basis marker, in accordance with some embodiments.

[0054] As illustrated in FIG. 2B, analytics process 1 (250a) is used to analyze data records 1-3, analytics process 2 (250b) is used to analyze data records 1-2, and analytics process 3 (250c) is used to analyze data record 3. Each analytic process 250 is hashed using one or more hash functions 260. Hash function 260 may include non-cryptographic hash functions, such as murmur, FNV series of hash functions, CityHash or Jenkins hashes. Hash function 260 may include cryptographic hash functions, such as SHA-256 or MD5.

[0055] In some embodiments, the hashed analytics processes are outputted as hash function outputs 270. For example, hash function output 1 (270a) for analytics process 1 (250a) is 1001, hash function output 2 (270b) for analytics process 2 (250b) is 1100, and hashfunction output 3 (270c) for analytics process 3 (250c) is 1010. The hash function outputs are representations of analytics processes 250.

[0056] In some embodiments, data record basis marker 280 includes the hashed analytics process(es) that were used to analyze the data record associated with data record basis marker 280. For example, each analytics process that analyzed data record 1 is added to data record basis marker 280a. Data record 1 basis marker 280a starts empty (e.g., 0000). Then, the hash function output for analytics process 1 (270a) (e.g., 1001) is added to data record 1 basis marker 280a bitwise. For example, positions 1-4 in data record 1 basis marker 280a are all not marked (e.g., “0”) and positions 1 and 4 in the hash function output of analytics process 1 (270a) are marked (e.g., “1”), and the remaining positions are not marked. Thus, adding hash function output of analytics process 1 (270a) to data record 1 basis marker 280a marks positions 1 and 4 in data record 1 basis marker 280a. This results in the data record 1 basis marker 280a being 1001. Then, the hash function output for analytics process 2 (270b) (e.g., 1100). In this instance, data record 1 basis marker 280a already has position 1 marked, because of hash function output of analytics process 1 (270a). Hash function output of analytics process 2 (270b) has positions 1 and 2 marked. Adding hash function output of analytics process 2 (270b) to data record 1 basis marker 280a additional markers position 2. Position 1 remains unchanged because it is already marked from hash function output of analytics process 1 (270a). This results in data record 1 basis marker 280a being 1101. The aforementioned steps are repeated for analytics 3, which results in data record basis marker 280a being 1111.

[0057] FIGS. 3A-3B are diagrams illustrating implementations of querying whether a data record is present in an analytics process basis marker, in accordance with some embodiments.

[0058] In some embodiments, analytics process 1 basis marker 240a is queried to determine whether the analytics process 1 basis marker 240a includes hashes representing data records 1-6 (230a-230c and 320a-320c). A hash representing a data record 230, 320 is considered included in an analytics process basis marker when all the marked bits in the hash representing the data record 230, 320, are also marked in the analytics process 1 basis marker 240a. For example, the hash representing data record 1 (230a) has positions 2 and 4 marked. Analytics process 1 basis marker 240a has positions 1, 2, 4, 7, and 8 marked. Data record 1was analyzed by analytics process 1 because analytics process 1 basis marker 240a has all positions of hash representing data record 1 (230a) (e.g., positions 2 and 4) marked.

[0059] In another example, the hash representing data record 2 (230b) has positions 2 and 7 marked. Analytics process 1 basis marker 240a has positions 1, 2, 4, 7, and 8 marked. Data record 2 was analyzed by analytics process 1 because analytics process 1 basis marker 240a has all positions of hash representing data record 2 (230b) (e.g., positions 2 and 7) marked.

[0060] In another example, the hash representing data record 3 (230c) has positions 1 and 8 marked. Analytics process 1 basis marker 240a has positions 1, 2, 4, 7, and 8 marked. Data record 3 was analyzed by analytics process 1 because analytics process 1 basis marker 240a has all positions of hash representing data record 3 (230c) (e.g., positions 1 and 8) marked.

[0061] In another example, the hash representing data record 4 (320a) has positions 3 and 8 marked. Analytics process 1 basis marker 240a has positions 1, 2, 4, 7, and 8 marked. Data record 4 was not analyzed by analytics process 1 because analytics process 1 basis marker 240a does not have all positions of hash representing data record 4 (230c) (e.g., positions 3 and 8) marked. Analytics process 1 basis marker 240a does not have position 3 marked.

[0062] In another example, the hash representing data record 5 (320b) has positions 3 and 5 marked. Analytics process 1 basis marker 240a has positions 1, 2, 4, 7, and 8 marked. Data record 5 was not analyzed by analytics process 1 because analytics process 1 basis marker 240a does not have all positions of hash representing data record 5 (230e) (e.g., positions 3 and 5) marked. Analytics process 1 basis marker 240a does not have positions 3 or 5 marked.

[0063] In another example, the hash representing data record 6 (320c) has positions 1 and 4 marked. Analytics process 1 basis marker 240a has positions 1, 2, 4, 7, and 8 marked. With reference to FIG. 2 A, only data records 1-3 (210a-210c) were analyzed by analytics process 1, and thus only hash function outputs for data records 1-3 (230a-230c) should be included in analytics process 1 basis marker. However, data record 3 appears to be analyzed by analytics process 1 because analytics process 1 basis marker 240a has all positions of hash representing data record 6 (320c) (e.g., positions 1 and 4) marked. In this example, datarecord 6 being identified as being analyzed by analytics process l is a false positive. This false positive may also be referred to as a hash collision. The rate of false positives can be decreased by increase the array size (e.g., increasing the number of bits) of the analytics process 1 basis marker.

[0064] FIG. 3B shows a similar process as FIG. 3 A, with analytics process 3 basis marker instead of analytics process 1 basis marker being queried against the hashes representing data record 1-6.

[0065] FIG. 3C is a diagram illustrating an implementation of querying whether an analytics process is present in a data record basis marker, in accordance with some embodiments.

[0066] In some embodiments, data record 2 basis marker 280b is queried to determine whether the data record 2 basis marker 280b includes hashes representing analytics processes 1-4 (230a-230c and 320a-320c). A hash representing an analytics process 330 is considered included in an analytics process basis marker when all the marked bits in the hash representing the analytics process 330 are also marked in the data record 2 basis marker 280b. For example, the hash representing analytics process 1 (330a) has positions 1 and 4 marked. Data record 2 basis marker 280b has positions 1, 2, and 4 marked. Data record 2 was analyzed by data record 2 basis marker 280b because data record 2 basis marker 280b has all positions of hash representing analytics process 1 (330a) (e.g., positions 1 and 4) marked.

[0067] In another example, the hash representing analytics process 2 (330b) has positions 1 and 2 marked. Data record 2 basis marker 280b has positions 1, 2, and 4 marked. Data record 2 was analyzed by analytics process 2 because data record 2 basis marker 280b has all positions of hash representing analytics process 2 (330b) (e.g., positions 1 and 2) marked.

[0068] In another example, the hash representing analytics process 3 (330c) has positions 1 and 3 marked. Data record 2 basis marker 280b has positions 1, 2, and 4 marked. Data record 2 was not analyzed by analytics process 3 because data record 2 basis marker 280b does not have all positions of hash representing analytics process 3 (330b) (e.g., positions 1 and 3) marked. Data record 2 basis marker 280b does not have position 3 marked.

[0069] In another example, the hash representing analytics process 4 (330d) has positions 2 and 4 marked. Data record 2 basis marker 280b has positions 1, 2, and 4 marked. With reference to FIG. 2B, only analytics processes 1 and 2 were used to analyze data record 2, and thus only hash function outputs for analytics processes 1-2 (330a-330b) should be included in data record 2 basis marker. However, analytics process 4 appears to have analyzed data record 2 because Data record 2 basis marker 280b has all positions of hash representing analytics process 4 (330d) (e.g., positions 2 and 4) marked. In this example, analytics process 4 being identified as having analyzed data record 4 is a false positive. This false positive may also be referred to as a hash collision. The rate of false positives can be decreased by increase the array size (e.g., increasing the number of bits) of the analytics process 1 basis marker.

[0070] FIG. 4A-4B are diagrams illustrating an implementation of cross validation between an analytics process basis marker and a data record basis marker, in accordance with some embodiments.

[0071] In some embodiments, a specified analytics process basis marker and a specified data record basis marker can be compared to each other when it is known that the analytics process associated with the specified analytics process basis marker analyzed the data record associated with the specified data record basis marker.

[0072] For example, in FIG. 4A, analytics process 1 analyzed data record 2. Data record 2 is queried against analytics process 1 basis marker via process C (see discussion for FIG. 3 A). This query indicates that data record 2 was analyzed by analytics process 1. Analytics process 1 is queried against data record 2 basis marker via process E (see discussion for FIG. 3C). This query indicates that analytics process 1 analyzed data record 2. Thus, the analytics process basis marker is validated against the data record basis marker. The positives are unlikely to be false positives.

[0073] In comparison, in FIG. 4B, analytics process 3 did not analyze data record 2. Data record 2 is queried against analytics process 3 basis marker via process D (see discussion for FIG. 3B). This query indicates that data record 2 was not analyzed by analytics process 1. Analytics process 3 is queried against data record 2 basis marker via process E (see discussion for FIG. 3C). This query indicates that analytics process 3 analyzed data record 2. Per the discussion associated with FIG. 3C, this is a false positive result. By validating theanalytics process basis marker with the data record basis marker, some false positives can be eliminated.

[0074] One advantage of validating the analytics process marker with the data record basis marker is the hash size to represent the data record and the analytics process can be further reduced without increasing the false positive rate that would normally be associated with a smaller hash size without the validation process. The smaller hash size further reduces the storage requirement and the associated computing power.

[0075] It will be understood that, although the terms “first,” “second,” etc. may be used herein to describe various elements, these elements should not be limited by these terms. These terms are only used to distinguish one element from another. For example, a first contact could be termed a second contact, and, similarly, a second contact could be termed a first contact, which changing the meaning of the description, so long as all occurrences of the “first contact” are renamed consistently and all occurrences of the second contact are renamed consistently. The first contact and the second contact are both contacts, but they are not the same contact.

[0076] The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the claims. As used in the description of the embodiments and the appended claims, the singular forms “a”, “an” and “the” are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will also be understood that the term "and / or" as used herein refers to and encompasses any and all possible combinations of one or more of the associated listed items. It will be further understood that the terms "comprises" and / or "comprising," when used in this specification, specify the presence of stated features, integers, steps, operations, elements, and / or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof.

[0077] As used herein, the term “if’ may be construed to mean “when” or “upon” or “in response to determining” or “in accordance with a determination” or “in response to detecting,” that a stated condition precedent is true, depending on the context. Similarly, the phrase “if it is determined [that a stated condition precedent is true]” or “if [a stated condition precedent is true]” or “when [a stated condition precedent is true]” may be construed to mean “upon determining” or “in response to determining” or “in accordance with a determination”or “upon detecting” or “in response to detecting” that the stated condition precedent is true, depending on the context.

[0078] The foregoing description, for purpose of explanation, has been described with reference to specific implementations. However, the illustrative discussions above are not intended to be exhaustive or to limit the claims to the precise forms disclosed. Many modifications and variations are possible in view of the above teachings. The implementations were chosen and described in order to best explain principles of operation and practical applications, to thereby enable others skilled in the art.

Claims

What is claimed is:

1. A method, comprising: at a computer system having one or more processors and memory storing instructions for execution by the one or more processors: receiving a plurality of data records; analyzing, by an analytics process, the plurality of data records to generate an analytics outcome; and determining a subset of the plurality of data records that meet presumptive criteria for affecting the analytics outcome for the analytics process; generating, and storing in the memory: a data record basis marker for a respective data record of the subset of the plurality of data records that meet the presumptive criteria for affecting the outcome of the analytics process, wherein the data record basis marker includes information associated with the analytics process, and wherein the data record basis marker is generated by a set of hash functions associated with the data record basis marker; and / or generating an analytics process basis marker for the analytics process, wherein the analytics process basis marker includes information associated with the subset of the plurality of data records that meet the presumptive criteria for affecting the outcome of the analytics process, and wherein the analytics process basis marker is generated by a set of hash functions associated with the analytics process basis marker; and receiving a user request to recall basis information relating to the analytics outcome; and in response to the user request to recall the basis information related to the analytics outcome: locating, via a query, the data record basis marker and / or the analytics process basis marker, and providing information from the data record basis marker and / or the analytics process basis marker as the basis information relating to the analytics outcome.

2. The method of claim 1, including: receiving an updated data record of the respective data record of the plurality of data records, wherein information associated with the updated data record is different from information associated with the respective data record;analyzing, by the analytics process, the updated data record to update the analytics outcome; determining that the updated data record meets the presumptive criteria for affecting the analytics outcome for the analytics process; and updating, and storing in the memory, the data record basis marker for the data record, wherein the data record basis marker includes information associated with the analytics process based on the updated data record.

3. The method of claim 1, including: in response to the analytics process being modified: analyzing, by the modified analytics process, the plurality of data records to generate a modified analytics outcome; determining a modified subset of the plurality of data records that meet the presumptive criteria for affecting a modified analytics outcome for the modified analytics process; updating, and storing in the memory, the analytics process basis marker for the analytics process, wherein the analytics process basis marker includes information associated with the modified subset of the plurality of data records that meet the presumptive criteria for affecting the modified analytics outcome of the modified analytics process.

4. The method of claim 1, including: analyzing, by a second analytics process, the plurality of data records to generate a second analytics outcome; determining a second subset of the plurality of data records that meet presumptive criteria for affecting the second analytics outcome for the second analytics process; generating, and storing in the memory: a second analytics process basis marker for the second analytics process, wherein the second analytics process basis marker includes information associated with the second subset of the plurality of data records that meet the presumptive criteria for affecting the second analytics outcome of the second analytics process, and wherein the second analytics process basis marker is generated by one or more hash functions associated with the second analytics process basis marker.

5. The method of claim 4, wherein the second analytics process is based on the first analytics process, including: updating the analytics outcome based on the second analytics outcome; and updating the analytics process basis marker based on the second analytics process basis marker.

6. The method of claim 1, wherein: a size of the data record basis marker is based on a hash collision rate associated with the data record basis marker being below a specified hash collision threshold associated with the data record basis marker, wherein the size of the data record is a number of bits stored in the memory; and a size of the analytics process basis marker is based on a hash collision rate associated with the analytics process basis marker being below a specified hash collision threshold associated with the analytics process basis marker, wherein the size of the analytics record is a number of bits stored in the memory associated with the analytics process basis marker.

7. The method of claim 6, wherein: the specified hash collision threshold associated with the data record basis marker is based on a number of the plurality of data records; and the specified hash collision threshold associated with the analytics process basis marker is based on a number of analytics subprocesses of the analytics process.

8. The method of claim 1, wherein providing the information from the data record basis marker includes: providing information associated with the analytics process which, when analyzing the respective data record of the subset of the plurality of data records associated with the data record basis marker, affected the analytics outcome.

9. The method of claim 1, wherein providing the information from the analytics process basis marker includes: providing information associated with the subset of the plurality of data records which met the presumptive criteria for affecting the analytics outcome for the analytics process.

10. The method of claim 1, including:storing the data record basis marker and the analytics process basis marker as an aggregate basis marker associated with the analytics outcome.

11. The method of claim 10, including: in response to the user request to recall the basis information related to the analytics outcome: locating, via a query, the aggregate basis marker, and providing information from the aggregate basis marker as the basis information relating to the analytics outcome, wherein the aggregate basis marker includes information from the data record basis marker and the analytics process basis marker.

12. The method of claim 11, wherein: the aggregate basis marker provides information associated with the analytics processes which, when analyzing the respective data record of the subset of the plurality of data records associated with the data record basis marker, affected the analytics outcome; and the aggregate basis marker provides information associated with the subset of the plurality of data records which met the presumptive criteria for affecting the analytics outcome for the analytics process; and13. The method of claim 1, wherein the set of hash functions includes: a Bloom filter or a cuckoo filter.

14. A computer system comprising: one or more processors and memory storing instructions for execution by the one or more processors, including instructions for performing the method of any of claims 1-13.

15. A non-transitory computer-readable storage medium storing instructions for execution by a computer system having one or more processors and memory, including instructions for performing the method of any of claims 1-13.

Images