Value Assignment for Query Responses with Missing Data
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Systems designed to detect events of interest, such as Network Intrusion Detection Systems, face challenges with noisy event collections containing missing fields, which can lead to false positives and false negatives when simply choosing the most probable values for missing fields, ignoring the structure of rules and varying error importance based on application.
Innovation Solution
A processor calculates false positive and false negative rates for each possible value assignment in response to missing values in a dataset, selecting a value assignment based on these rates to generate accurate responses to queries, rather than relying solely on the most probable values.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If the most probable values are chosen for missing fields, then the process is simple and fast, but the accuracy of event detection deteriorates due to false positives and false negatives
Solution Approach 1:
The patent changes the parameter of value assignment from simple probabilistic selection to a multi-criteria optimization approach that considers false positive rates, false negative rates, and rule structure. This transforms the problem from choosing the most probable value to selecting the value assignment that optimizes detection accuracy while accounting for error costs and rule constraints.
Solution Approach 2:
The patent performs preliminary analysis of the rule structure and error costs before making value assignments. By pre-evaluating which missing values are critical for rule satisfaction and calculating the potential false positive and false negative rates for different assignments, the system prepares an optimized assignment strategy in advance rather than making simple probabilistic choices during query processing.
2Reliability
If simple probabilistic value assignment is used, then the computation is fast, but the reliability of query results deteriorates
Solution Approach 1:
The patent segments the value assignment problem into distinct components: identifying which missing values affect rule satisfaction, calculating false positive rates for each assignment option, calculating false negative rates for each assignment option, and evaluating rule structure constraints. This segmentation allows the system to focus computational effort on the most critical aspects rather than uniformly processing all missing values with simple probability.
Solution Approach 2:
The patent introduces feedback mechanisms where the system evaluates the impact of each possible value assignment on false positive and false negative rates, then uses this feedback to select the optimal assignment. This iterative evaluation and selection process improves reliability by considering the consequences of each assignment choice rather than relying solely on prior probabilities.
3Measurement precision
If all possible value assignments are evaluated to minimize errors, then the accuracy improves, but the computational complexity increases significantly
Solution Approach 1:
The patent applies local quality by focusing computational resources on specific missing values that have the greatest impact on rule satisfaction and error rates. Rather than uniformly evaluating all possible assignments for all missing fields, the system identifies and prioritizes assignments that are critical for detecting events of interest, allocating computational effort where it provides the most benefit to accuracy.
Data Source
AI summary
In an example computer-implemented method, a dataset and a query including an expression to be matched to the dataset is received via a processor. A false positive rate (FPR) and a false negative rate (FNR) is calculated via the processor for each possible value assignment of a plurality of possible value assignments in response to detecting a missing value in the dataset. A value assignment is selected, via the processor, from the plurality of possible value assignments based on the FPR and the FNR. A response to the query is generated via the processor based on the selected value assignment.