Non-hypergeometric overlap probability for microarray gene sets
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current methods for determining the probability of overlap between two sets of genes from different microarray chips are either inaccurate due to oversimplification or excessively time-consuming, particularly when the underlying populations are different and overlapping, leading to compromised accuracy and increased likelihood of false negatives or positives in identifying biologically significant gene signatures.
Innovation Solution
A computer-implemented method that calculates the overlap probability by considering the total number of ways to select gene subsets from each microarray, rather than just the shared genes, using a formula that accounts for all genes in each array, providing a more accurate estimation of the probability of an overlap set occurring by chance.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of manufacture
If the hypergeometric distribution is used to determine overlap probability, then the calculation is simple, but the accuracy is compromised when populations are different and overlapping
Solution Approach 1:
The patent changes the fundamental parameters of the probability calculation by transitioning from the hypergeometric distribution (which assumes identical populations) to a new distribution that incorporates separate population sizes (N1 and N2) and their overlap (M). This parameter change allows accurate modeling of different and overlapping populations while maintaining computational feasibility through a closed-form solution.
Solution Approach 2:
The patent inverts the traditional approach by not reducing the problem to common genes only, but rather considering all genes in both populations and calculating the probability based on the full population parameters. This inversion leads to a more accurate model that doesn't require oversimplification of the underlying population structure.
2Measurement precision
If a permutation method is employed to determine overlap probability, then the solution is accurate, but it is very time consuming
Solution Approach 1:
The patent replaces the computational mechanism of permutation methods (which require extensive sampling and iteration) with a direct mathematical formula based on combinatorial probability. This substitution eliminates the need for time-consuming permutations while maintaining accuracy by directly calculating the probability based on population parameters.
Solution Approach 2:
The patent performs preliminary action by deriving a closed-form solution that incorporates all necessary population parameters upfront, eliminating the need for iterative permutation sampling. The formula directly computes the probability using pre-defined population sizes and overlap, avoiding repeated calculations.
3Ease of manufacture
If genes common to both chips are considered only, then the hypergeometric distribution can be utilized, but the accuracy is compromised
Solution Approach 1:
The patent segments the population analysis into distinct components: population 1 with size N1, population 2 with size N2, and their overlap M. By maintaining this segmentation throughout the calculation rather than reducing to common genes only, the method preserves the structural differences between populations while enabling accurate probability calculation.
4Measurement precision
If the total number of genes in each microarray is considered, then the accuracy increases, but the calculation complexity increases
Solution Approach 1:
The patent introduces new parameters (N1, N2, M) that represent the total genes in each microarray and their overlap, replacing the simplified hypergeometric parameters. Despite the increased parameter count, the closed-form solution maintains computational efficiency by avoiding iterative calculations, thus achieving high accuracy without excessive complexity.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
Methods, software, and systems are provided for determining the probability of an overlap set of entities having an overlap size, where the overlap set is independently selected from two sets of non-identical entities. Applications of the invention to microarrays are provided. Probability distributions are provided for determining the probability that the size of an overlap gene set from two different microarrays occurs by chance. Microarray analysis for determining the size of a statistically significant overlap gene set given two different microarrays is described. Overlap set size probability determinations that account for the total number of genes in two different microarrays and not just the common genes are described.