Continuous feature-independent determination of features for deviation analysis

Abstract

The system and method include: determining, for each of a plurality of discrete features, a statistic based on a number of occurrences of each discrete value of the discrete feature in the data; determining a first summary statistic based on the determined statistic; determining a dissimilarity for each discrete feature based on the first summary statistic and the statistic determined for the discrete feature; determining a candidate discrete feature based on the determined dissimilarity; determining, for each of the candidate discrete features, a second summary statistic based on a value of a continuous feature associated with each discrete value of the candidate discrete feature; determining a deviation score for each of the candidate discrete features based on the second summary statistic; and transmitting the candidate discrete feature for display in association with a selected continuous feature based on the determined deviation score.

Description

Technical Field

[0001] The inventive concept of the present application relates to data mining, and in particular, to deviation analysis for mining deviation relationships between selected continuous features and discrete features of a large data set. Background Art

[0002] Today's organizations collect and store vast amounts of data at an ever-increasing rate. Examples of these big data streams include sensor data and financial data. The Internet of Things has dramatically increased the number of deployed sensors, which has led to an exponential increase in the amount of sensor data generated. The financial industry generates vast amounts of data to facilitate forecasting, pattern recognition, and strategic planning.

[0003] Performing calculations or identifying patterns within these large data sets can be time-consuming or even infeasible. Modern data analysis attempts to assist humans in effectively understanding this data. For example, data mining uses machine learning and / or statistical techniques to discover potentially useful patterns within large amounts of data stored in databases, data warehouses, or other information repositories.

[0004] Deviation analysis is a type of data mining that can involve determining the deviations between selected continuous features (e.g., sales) and discrete features (e.g., color, type) of a data set. For example, a deviation analysis can determine whether all discrete values ​​(blue, red, green) of a discrete feature (color) behave in a generally similar manner relative to the continuous feature (sales). If the discrete values ​​do not behave in a generally similar manner, the deviation analysis can further indicate the extent to which any one discrete value deviates from the norm (e.g., blue contributes much more to sales than red or green). Deviation analysis can be performed for each of multiple discrete features relative to the same continuous feature to determine which discrete feature includes the most deviating discrete values.

[0005] Mining deviation relationships between selected continuous features and discrete features of a large dataset can be computationally expensive, particularly where the dataset contains many discrete features, each of which contains many discrete values. An improved system for performing deviation analysis is desired that requires fewer computational resources than conventional systems while producing deviation analysis of suitable quality. Summary of the Invention

[0006] One embodiment of the present disclosure provides a system comprising: a memory storing processor-executable program code; and a processing unit executing the processor-executable program code to cause the system to: receive data comprising a plurality of discrete features, each of the plurality of discrete features being associated with a plurality of discrete values; determine, for each of the plurality of discrete features, a statistic based on a number of occurrences of each discrete value of the discrete feature in the data; determine a first summary statistic based on the statistic determined for each of the plurality of discrete features; determine, for each discrete feature, a dissimilarity based on the first summary statistic and the statistic determined for the discrete feature; determine a candidate discrete feature for the plurality of discrete features based on the determined dissimilarity, the candidate discrete feature comprising less than all of the plurality of discrete features; determine a first selected continuous feature of the received data; determine, for each of the candidate discrete features, a second summary statistic based on a value of the first selected continuous feature associated with each discrete value of the candidate discrete feature; determine a deviation score for each of the candidate discrete features based on the second summary statistic; and transmit the candidate discrete feature for display in association with the first selected continuous feature based on the determined deviation score.

[0007] Another embodiment of the present disclosure provides a method, comprising: receiving data comprising a plurality of discrete features, each of the plurality of discrete features being associated with a plurality of discrete values; determining, for each of the plurality of discrete features, a statistic based on a number of occurrences of each discrete value of the discrete feature in the data; determining a first summary statistic based on the statistic determined for each of the plurality of discrete features; determining, for each discrete feature, a dissimilarity based on the first summary statistic and the statistic determined for the discrete feature; determining a candidate discrete feature of the plurality of discrete features based on the determined dissimilarity, the candidate discrete feature including less than all of the plurality of discrete features; determining a first selected continuous feature of the received data; determining, for each of the candidate discrete features, a second summary statistic based on a value of the first selected continuous feature associated with each discrete value of the candidate discrete feature; determining a deviation score for each of the candidate discrete features based on the second summary statistic; and transmitting the candidate discrete feature for display in association with the first selected continuous feature based on the determined deviation score.

[0008] Yet another embodiment of the present disclosure provides a non-transitory medium storing processor-executable program code, the processor-executable program code executable by a processing unit of a computing system to cause the computing system to: receive data comprising a plurality of discrete features, each of the plurality of discrete features being associated with a plurality of discrete values; determine, for each of the plurality of discrete features, a statistic based on a number of occurrences of each discrete value of the discrete feature in the data; determine a first summary statistic based on the statistic determined for each of the plurality of discrete features; determine, for each discrete feature, a dissimilarity based on the first summary statistic and the statistic determined for the discrete feature; determine, based on the determined dissimilarity, candidate discrete features for the plurality of discrete features, the candidate discrete features including less than all of the plurality of discrete features; determine a first selected continuous feature of the received data; determine, for each of the candidate discrete features, a second summary statistic based on a value of the first selected continuous feature associated with each discrete value of the candidate discrete feature; determine a deviation score for each of the candidate discrete features based on the second summary statistic; and transmit the candidate discrete feature for display in association with the first selected continuous feature based on the determined deviation score. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] Figure 1A is a block diagram of an architecture for identifying candidate discrete features for deviation analysis, according to some embodiments.

[0010] Figure 1B is a block diagram of an architecture for performing deviation analysis on multiple discrete features relative to one or more selected continuous features, according to some embodiments.

[0011] Figure 2 A flow chart is included of a process for performing deviation analysis on a plurality of discrete features relative to one or more selected continuous features according to some embodiments.

[0012] Figure 3 Includes a tabular representation of selected discrete feature data according to some embodiments.

[0013] Figure 4 Graphs illustrating statistics determined for selected discrete features according to some embodiments.

[0014] Figure 5 Illustrated are statistics determined for selected discrete features for input to a deviation analysis, in accordance with some embodiments.

[0015] Figure 6 is an external representation of a user interface presenting the proportional contribution of each of a plurality of discrete values ​​to an output value according to some embodiments.

[0016] Figure 7Illustrated is a system providing data analysis according to some embodiments.

[0017] Figure 8 is a block diagram of a hardware system for determining a proportional contribution of each of a plurality of discrete values ​​to an output value, according to some embodiments. DETAILED DESCRIPTION

[0018] The following description is provided to enable anyone skilled in the art to make and use the described embodiments and sets forth the best mode contemplated for carrying out some embodiments. However, various modifications will be apparent to those skilled in the art.

[0019] As used herein, a feature refers to an attribute of a dataset. In the case of tabular data, each column can be considered to represent a corresponding feature of the data, while each row is an instance of the value of each feature of the data. Continuous features are represented using numerical data with an infinite number of possible values ​​within a selected range. Discrete features are represented by data with a finite number of possible values, referred to below as discrete values. Temperature is an example of a continuous feature, while day of the week and gender are examples of discrete features.

[0020] Some embodiments provide for the automatic and efficient selection of discrete features of a dataset to which deviation analysis can then be applied. The selected discrete features are referred to herein as "candidate" features. Thus, embodiments can avoid performing deviation analysis on unselected discrete features and the associated resource consumption, while providing results similar to those that would be produced if deviation analysis were applied to all discrete features of the dataset.

[0021] According to some embodiments, the selection of candidate discrete features is independent of any continuous features within the dataset. Thus, the candidate discrete features for a given dataset (or subset) can be determined before a user selects any continuous features for deviation analysis. Once a user has selected one or more continuous features for deviation analysis, deviation analysis can be performed only with respect to the candidate discrete features, as opposed to the more time-consuming task of performing deviation analysis for each selected continuous feature given all discrete features of the dataset.

[0022] The determination of candidate features as described herein attempts to identify those discrete features within a data set for which a deviation analysis is more likely to produce an abnormal deviation score relative to any continuous feature of the data set, and therefore exhibits a stronger deviation relationship with the given continuous feature relative to other discrete features within the data set. This identification can allow other discrete features to be ignored during the deviation analysis, thereby improving its efficiency without adversely affecting the results. Although the embodiments described below subject the identified candidate features to subsequent deviation analysis, the embodiments are not limited thereto.

[0023] Figure 1A is a block diagram of an architecture 100a for identifying candidate discrete features for deviation analysis according to some embodiments. The illustrated components can be implemented using any suitable combination of computing hardware and / or software that is or becomes known. In some embodiments, two or more components are implemented by a single computing device. Figure 1A Two or more components of an application can be co-located. One or more components can be implemented as a cloud service (e.g., software as a service, platform as a service). Figure 1A (or Figure 1B ) can elastically allocate computing resources based on demand, need, price and / or any other metric.

[0024] The data 110 may include values ​​of a database table. More specifically, the data 110 may include rows of a database table, where each row includes the value of a corresponding database column or feature. The data 110 may be composed of at least one continuous feature and one or more discrete features.

[0025] Discrete feature selection component 120 identifies one or more discrete features of data 110. The one or more discrete features may be selected by a user (not shown). In some embodiments, the user does not select any discrete features, in which case discrete feature selection component 120 selects all discrete features of data 110. Figure 1A In this example, column 130 includes values ​​of data 110 from discrete features selected by discrete feature selection component 120 .

[0026] In some embodiments, the discrete feature selection component 120 selects one or more continuous features of the data 110 while selecting the columns 130. As described below, these selected continuous features are not involved in the processing of the architecture 100a, but they may be used to Figure 1B The processing of architecture 100b is in progress.

[0027] The candidate feature identification component 140 determines a dissimilarity score for each selected discrete feature. The dissimilarity score for the discrete feature is determined based on statistics determined for the discrete feature. As will be described below, the statistics for the discrete feature can be determined based on the number of occurrences of each discrete value of the discrete feature.

[0028] The dissimilarity scores determined for discrete features represent the likelihood that the candidate feature analysis component 170 will determine a significant deviation score for the discrete feature. According to some embodiments, the candidate feature identification component 140 outputs the discrete features associated with the N highest dissimilarity scores as candidate features 145 .

[0029] Candidate features 145 may be stored in a storage device 150, which may include volatile (e.g., random access) memory and / or persistent (e.g., disk-based) memory. Thus, in some embodiments, a user's selection or use of data 110 may trigger a Figure 1A Operates to generate and store candidate features 145. Thus, if a user later requests a top contributor analysis in view of one or more continuous features of the data 110, Figure 1B The process illustrated in can be efficiently performed based on known stored candidate features 145 .

[0030] Architecture 100b includes a continuous feature selection component 160 for selecting one or more continuous features of data 110. In this example, column 165 includes values ​​of data 110 from the continuous features selected by continuous feature selection component 160. Column 165 is provided to candidate feature analysis component 170 along with candidate features 145 from storage device 150. As described above, in some embodiments, discrete feature selection component 120 of architecture 100a can be operative to select one or more continuous features, in which case the selected one or more continuous features and the determined candidate discrete features can be provided directly to candidate feature analysis component 170 after the candidate discrete features are determined.

[0031] The candidate feature analysis component 170 applies a deviation analysis algorithm to the received columns 165 and the columns of the data 110 associated with the received candidate discrete features 145 to generate a deviation score for each candidate discrete feature 145. For each selected continuous feature, the candidate feature analysis component 170 determines the deviation score for each candidate discrete feature 145 based on statistics that are, in turn, determined based on the value of the continuous feature corresponding to each discrete value of the candidate discrete feature 145.

[0032] Each deviation score indicates the deviation relationship of the candidate discrete feature 145 from the selected continuous feature. The candidate discrete features 145 are then sorted (i.e., ranked) and output as ranked features 180 based on their corresponding deviation scores. Each selected continuous feature can be associated with a different deviation score for a given discrete feature, so the ranking of the candidate discrete features 145 can be different for each selected continuous feature.

[0033] Embodiments can generate an ordered list of discrete features and corresponding deviation scores faster and / or using fewer resources than a system that determines deviation scores for all selected discrete features. In particular, preliminary determination of candidate discrete features from a set of selected discrete features allows for avoiding the determination of deviation scores for discrete features that may not be associated with significant deviations from the subject continuous feature. Furthermore, by decoupling the determination of candidate discrete features from the analysis of deviations from one or more continuous features, candidate discrete features can be determined prior to a user's instruction to analyze a particular one or more continuous features.

[0034] Figure 2 is a flow chart of process 200 for performing deviation analysis on multiple discrete features relative to one or more continuous features, according to some embodiments. Process 200 and other processes described herein can be performed using any suitable combination of hardware and software. The program code embodying these processes can be stored by any non-transitory tangible medium (including fixed disk, volatile or non-volatile random access memory, DVD, flash drive, or tape) and can be executed by any one or more processing units (including but not limited to processors, processor cores, and processor threads). Embodiments are not limited to the examples described below.

[0035] Process 200 may be initialized by a user selecting a particular data set, such as a table of transaction data (sales) or a subset thereof (sales, EMEA, 2020). The user may select such data for analysis, for example, via a data analysis application.

[0036] At S210, a data set is received in a structured form, such as a table format. The structured format facilitates the definition of one or more continuous features and one or more discrete features within the data. For example, each column of a table includes all values ​​of a single specific feature of the table, and each of these values ​​is associated with exactly one specific row of the table.

[0037] At S220, a plurality of discrete features are selected. In some embodiments, if the user does not select a discrete feature at S220, it is assumed that all discrete features of the data set are selected at S220. According to some embodiments, one or more continuous features are also selected at S220. The selected continuous features are continuous features, and the deviation of each selected discrete feature relative to those continuous features is determined and ranked.

[0038] At S230, statistics are determined for each discrete feature. Statistics for a given discrete feature are determined based on the number of occurrences of each discrete value of the discrete feature within the data set. Examples of statistics determined at S230 include, but are not limited to, minimum, maximum, mean, variance, and magnitude (i.e., the number of different discrete values ​​of the discrete feature).

[0039] Figure 3 Columns 310 of four discrete features selected at S220 according to some embodiments are illustrated. Columns 310 may include columns of a data set received at S210 and including one or more other columns of discrete or continuous features. Each of columns 310 includes discrete values ​​for its corresponding discrete feature.

[0040] As described above, at S230, for each discrete feature, statistics are determined based on the number of occurrences of the discrete values ​​of each discrete feature within the data set. Table 410 illustrates statistics determined for the discrete features DiscrFeatA, DiscrFeatB, DiscrFeatC, and DiscrFeatD based on the number of occurrences of each of the discrete values ​​corresponding to these discrete features.

[0041] Row 412 of table 410 shows the statistics determined for DiscrFeatA at S230. Figure 3 418. As shown in FIG, the discrete value A3 of discrete feature DiscrFeatA occurs 2 times, while discrete values ​​A1 and A2 occur 3 times and 4 times respectively. Therefore, row 412 shows that the minimum number of occurrences is 2, the maximum number of occurrences is 4, the average number of occurrences (that is, across all discrete values) is 3, and the variance is 1. The discrete values ​​B1, B2, B3 and B4 of discrete feature DiscrFeatB occur 2, 1, 3 and 3 times respectively. Therefore, row 414 indicates that the minimum number of occurrences is 1, the maximum number of occurrences is 3, the average number of occurrences is 2.25, and the variance is .917. The statistics of discrete features DiscrFeatC and DiscrFeatD are similarly determined and are shown in rows 416 and 418.

[0042] Based on the statistics determined for each discrete value at S230, a dissimilarity score is determined for each discrete feature at S240. According to some embodiments, the dissimilarity score for the discrete feature is determined based on an average of the statistics determined for the discrete features and a dissimilarity of the statistics determined for each discrete feature to the average. While the average of the discrete feature-specific statistics is used to determine dissimilarity in this example, embodiments may employ another statistical summary of the discrete feature-specific statistics and determine dissimilarity based on the statistical summary.

[0043] Figure 4 Row 420 of table 410 illustrates the average value of each statistic determined for the discrete feature at step S230. Step S240 may include determining a score representing the distance from the statistic of each row of table 410 to the average statistic of row 420. The score may represent a maximum distance, an average distance, or any other value determined based on the determined distance. The distance may be calculated as cosine dissimilarity, but the embodiment is not limited thereto.

[0044] The cosine similarity algorithm measures the similarity between two vectors (e.g. lists of values). This similarity is defined as the cosine angle between the two vectors and indicates the extent to which the two vectors point in the same direction.

[0045] In general, cosine similarity in and

[0046] Thus, S240 may include determining a cosine similarity between the average statistic of all discrete features (e.g., the value of row 420) and the statistic associated with each discrete feature (e.g., the value of each of rows 412, 414, 416, 418). Thus, a cosine similarity is determined for each discrete feature. The determined cosine similarity can also be considered a dissimilarity score, because a lower cosine similarity corresponds to a greater dissimilarity.

[0047] At S250, a set of candidate discrete features is determined based on the determined dissimilarity scores. In this regard, a dissimilarity score is determined at S240 for each discrete feature selected at S220. In some embodiments, at S250, the selected discrete features associated with the N largest dissimilarity scores (i.e., the N smallest cosine similarities) are determined as candidate discrete features. N can be any desired number and can be selected to limit the amount of processing resources required to perform the deviation analysis described below on the candidate discrete features.

[0048] As described above, S210 to S250 can be determined immediately after the data set is identified, without requiring the user to select any continuous features. Therefore, S210 to S250 can provide a set of candidate discrete features for performing deviation analysis relative to any continuous features of the data set. Assuming no continuous features are selected, one or more continuous features are selected at S260.

[0049] For each selected continuous feature, a second statistic is determined for each candidate discrete feature at S270 in preparation for the deviation analysis at S280. The second statistic determined for the discrete feature is based on a value of the selected continuous feature associated with each discrete value of the discrete feature. In some embodiments of S270, the continuous values ​​associated with each discrete value of the discrete feature are summed, and the second statistic for the discrete feature is determined based on the sum.

[0050] Figure 5 Column 510 includes the values ​​of the selected continuous feature ContFeat1 of the data set, which includes Figure 3 310. Table 520 shows the sum of the values ​​of the continuous feature ContFeat1 associated with each of the discrete values ​​B1, B2, B3, B4 of the discrete feature DiscrFeatB. According to this example, the second statistics of the discrete feature DiscrFeatB include the mean of the sum (11) and the value of α (e.g., the minimum value of the sum in the case where the mean is negative or the maximum value of the sum in the case where the mean is positive).

[0051] In this way, second statistics are determined for each discrete feature relative to each selected continuous feature. For example, if only the continuous feature ContFeat1 is selected at S260, then S270 includes determining four (mean, α) pairs, wherein each pair corresponds to one of the discrete features DiscrFeatA, DiscrFeatB, DiscrFeatC, and DiscrFeatD. If the continuous feature ContFeat1 and another continuous feature (e.g., ContFeat3) are selected at S260, then S270 includes determining eight (mean, α) pairs, wherein each pair corresponds to one of the discrete features DiscrFeatA, DiscrFeatB, DiscrFeatC, and DiscrFeatD and one of the continuous features ContFeat1 and ContFeat3. Any type of second statistics of discrete features can be determined at S270 based on the value associated with each discrete value of the selected continuous feature with respect to the discrete feature. In general, the second statistics determined are those statistics that are required as input to the specific deviation analysis algorithm to be adopted at S280.

[0052] For each selected continuous feature, a deviation score is determined for each candidate discrete feature at S280. The deviation score is determined based on the second statistical data determined at S270. According to a non-exhaustive embodiment, the deviation score of the discrete feature is determined as follows:

[0053]

[0054] Among them, as described above:

[0055]

[0056] According to the above example, given the continuous feature ContFeat1, the deviation score determined for the discrete feature DiscrFeatB is (14−11) / 11 == 0.273.

[0057] At S290, for each selected continuous feature, candidate discrete features are presented according to their corresponding deviation scores. For example, for the continuous feature ContFeat1, the candidate discrete features may be ranked from the highest deviation score determined relative to the continuous feature ContFeat1 to the lowest deviation score determined relative to the continuous feature ContFeat1, wherein a larger deviation score indicates a larger deviation behavior of the discrete feature relative to the discrete values ​​of the continuous feature ContFeat1.

[0058] Figure 6 Illustrated is a user interface 600 of a data analytics application according to some embodiments. A user may execute a web browser to access the data analytics application via the hypertext transfer protocol and receive the user interface 600 in return.

[0059] The user interface 600 includes a drop-down field 610 for selecting a table that the user has access to. The selection of a table may cause the execution of the process 200 to determine candidate discrete features of the table. In this example, all discrete features of the table are selected at S220.

[0060] At the same time, the drop-down menu 620 is populated with a list of selectable continuous features of the selected table. After the user selects one or more continuous features, S260 to S290 can be executed to determine the deviation score of each candidate discrete feature for each selected continuous feature. Area 630 of the user interface 600 shows these candidate discrete features presented in order according to the deviation score relative to the selected gross profit margin continuous feature. The embodiment is not limited to the user interface 600. The embodiment can utilize any interface metaphor for selecting one or more continuous features of a table (and optionally one or more discrete features of a table) and for presenting candidate discrete features based on the corresponding determined deviation scores.

[0061] Figure 7 The diagram illustrates a system 700 for providing data analysis, including data mining, to applications in accordance with some embodiments. Application server 710 may include a local or cloud-implemented server that provides an execution platform and services to applications, such as application 712. Application 712 may include program code executable by a processing unit to provide functionality to a user, such as user 720, based on coded logic and data 716 stored in a data store 714. Data 716 may include tabular data stored in a columnar or row-based format, object data, or any other type of data that is or becomes known. Data store 714 may include any suitable storage system, such as a database system, which may be partially or completely remote from application server 710 and may be distributed as is known in the art.

[0062] According to some embodiments, user 720 may interact with application 712 (e.g., via a web browser executing a front-end UI application associated with application 712) to request deviation analysis of discrete features given one or more continuous features of data table 716. Application 712 may access analysis platform 730 to service the request. Analysis platform 730 may also be implemented by a local or cloud-based server.

[0063] The analysis platform 730 includes program code for a feature deviation analysis framework 732 that can be executed to determine candidate discrete features and a deviation score for each candidate discrete feature relative to one or more continuous features as described herein. The analysis platform 730 can provide the candidate discrete features and the deviation scores to the application 712 for subsequent presentation to the user 720. The analysis platform 730 can provide additional functionality to the application, such as, but not limited to, machine learning model training and inference.

[0064] Figure 8 800 is a block diagram of a hardware system for determining candidate discrete features and corresponding deviation scores relative to one or more continuous features, according to some embodiments. Hardware system 800 may include a general-purpose computing device and may execute program code to perform any of the functions described herein. In some embodiments, hardware system 800 may be implemented by a distributed cloud-based server and may include an embodiment of analysis platform 730. According to some embodiments, hardware system 800 may include other components not shown.

[0065] The hardware system 800 includes a processing unit(s) 810 operatively coupled to an I / O device(s) 820, a data storage device(s) 830, one or more input devices 840, one or more output devices 850, and a memory 860. The I / O device(s) 820 can facilitate data exchange with external devices (such as an external network, cloud, or data storage device(s). The input device(s) 840 can include, for example, a keyboard, keypad, mouse or other pointing device, microphone, knobs or switches, infrared (IR) port, docking station, and / or touch screen. The input device(s) 840 can be used, for example, to enter information into the hardware system 800. The output device(s) 850 can include, for example, a display (e.g., a screen), speakers, and / or a printer.

[0066] Data storage device 830 may include any suitable persistent storage device, including a combination of magnetic storage devices (e.g., magnetic tape, hard drives, and flash memory), optical storage devices, read-only memory (ROM) devices, and RAM devices, and memory 860 may include a RAM device.

[0067] The data storage device 830 stores program code for the feature deviation analysis framework, which can be executed by the processing unit(s) 810 to cause the hardware system 800 to implement any component and perform any one or more of the processes described herein. Embodiments are not limited to executing these processes by a single computing device. The data storage device 830 may also store data and other program code necessary to provide additional functionality and / or the operation of the hardware system 800, such as device drivers, operating system files, etc.

[0068] Foregoing figure represents the logical framework for describing process according to some embodiments, and actual implementation can include more or different components arranged in other ways.Other topologies can be used in conjunction with other embodiments.In addition, each component or device described herein can be implemented by any number of devices that communicate via any number of other public and / or private networks.Two or more of these computing devices can be located away from each other and can communicate with each other via any known (multiple) network modes and / or dedicated connections.Each component or device can include any number of hardware and / or software elements that are suitable for providing functions described herein and any other functions.For example, any computing device used in some embodiments of the embodiment can include a processing unit to execute program code so that the computing device operates as described herein.

[0069] The embodiments described herein are for illustration purposes only. Those skilled in the art will recognize that other embodiments can be practiced by modifying and altering the embodiments described above.

Claims

1. A system for deviation analysis, comprising: a memory for storing processor-executable program codes; as well as a processing unit that executes the processor-executable program code to enable the system to: receiving data comprising a plurality of discrete features, each of the plurality of discrete features being associated with a plurality of discrete values; determining, for each of the plurality of discrete features, statistics based on a number of occurrences of each discrete value of the discrete feature in the data; determining a first summary statistic based on the statistic determined for each of the plurality of discrete features; determining, for each discrete feature, a dissimilarity based on the first summary statistic and the statistic determined for the discrete feature; determining candidate discrete features of the plurality of discrete features based on the determined dissimilarity, the candidate discrete features including less than all of the plurality of discrete features; determining a first selected continuous characteristic of the received data; determining, for each of the candidate discrete features, a second summary statistic based on the value of the first selected continuous feature associated with each discrete value of the candidate discrete feature; determining a deviation score for each of the candidate discrete features based on the second summary statistic; as well as The candidate discrete feature is transmitted for display in association with the first selected continuous feature based on the determined deviation score.

2. The system according to claim 1, wherein: Determining the second summary statistic for the first candidate discrete feature includes determining, for each discrete value of the first candidate discrete feature, a sum of values ​​of the first selected continuous feature associated with the discrete value and determining the second summary statistic based on the determined sum.

3. The system according to claim 1, wherein: The determination of the dissimilarity comprises determination of a cosine similarity.

4. The system of claim 1 , wherein the processing unit executes the processor-executable program code to cause the system to: determining a second selected continuous characteristic of the received data; determining, for each of the candidate discrete features, a third summary statistic based on the value of the second selected continuous feature associated with each discrete value of the candidate discrete feature; determining a second deviation score for each of the candidate discrete features based on the third summary statistic; as well as The candidate discrete feature is transmitted for display in association with the second selected continuous feature based on the determined second deviation score.

5. The system according to claim 4, wherein: determining the second summary statistic for the first candidate discrete feature comprises determining, for each discrete value of the first candidate discrete feature, a sum of the values ​​of the first selected continuous feature associated with the discrete value and determining the second summary statistic based on the determined sum, and Wherein, determining the third summary statistic for the first candidate discrete feature comprises determining, for each discrete value of the first candidate discrete feature, a second sum of the values ​​of the second selected continuous feature associated with the discrete value and determining the third summary statistic based on the second sum.

6. The system according to claim 5, wherein: The determination of the dissimilarity comprises determination of a cosine similarity.

7. A method for deviation analysis, comprising: receiving data comprising a plurality of discrete features, each of the plurality of discrete features being associated with a plurality of discrete values; determining, for each of the plurality of discrete features, statistics based on a number of occurrences of each discrete value of the discrete feature in the data; determining a first summary statistic based on the statistic determined for each of the plurality of discrete features; determining, for each discrete feature, a dissimilarity based on the first summary statistic and the statistic determined for the discrete feature; determining candidate discrete features of the plurality of discrete features based on the determined dissimilarity, the candidate discrete features including less than all of the plurality of discrete features; determining a first selected continuous characteristic of the received data; determining, for each of the candidate discrete features, a second summary statistic based on the value of the first selected continuous feature associated with each discrete value of the candidate discrete feature; determining a deviation score for each of the candidate discrete features based on the second summary statistic; as well as The candidate discrete feature is transmitted for display in association with the first selected continuous feature based on the determined deviation score.

8. The method according to claim 7, wherein: Determining the second summary statistic for the first candidate discrete feature includes determining, for each discrete value of the first candidate discrete feature, a sum of values ​​of the first selected continuous feature associated with the discrete value and determining the second summary statistic based on the determined sum.

9. The method according to claim 7, wherein: Determining the dissimilarity includes determining a cosine similarity.

10. The method according to claim 7, further comprising: determining a second selected continuous characteristic of the received data; determining, for each of the candidate discrete features, a third summary statistic based on the value of the second selected continuous feature associated with each discrete value of the candidate discrete feature; determining a second deviation score for each of the candidate discrete features based on the third summary statistic; as well as The candidate discrete feature is transmitted for display in association with the second selected continuous feature based on the determined second deviation score.

11. The method according to claim 10, wherein: determining the second summary statistic for the first candidate discrete feature comprises determining, for each discrete value of the first candidate discrete feature, a sum of the values ​​of the first selected continuous feature associated with the discrete value and determining the second summary statistic based on the determined sum, and Wherein, determining the third summary statistic for the first candidate discrete feature comprises determining, for each discrete value of the first candidate discrete feature, a second sum of the values ​​of the second selected continuous feature associated with the discrete value and determining the third summary statistic based on the second sum.

12. The method according to claim 11, wherein Determining the dissimilarity includes determining a cosine similarity.

13. A non-transitory medium storing processor-executable program code executable by a processing unit of a computing system to cause the computing system to: receiving data comprising a plurality of discrete features, each of the plurality of discrete features being associated with a plurality of discrete values; determining, for each of the plurality of discrete features, statistics based on a number of occurrences of each discrete value of the discrete feature in the data; determining a first summary statistic based on the statistic determined for each of the plurality of discrete features; determining, for each discrete feature, a dissimilarity based on the first summary statistic and the statistic determined for the discrete feature; determining candidate discrete features of the plurality of discrete features based on the determined dissimilarity, the candidate discrete features including less than all of the plurality of discrete features; determining a first selected continuous characteristic of the received data; determining, for each of the candidate discrete features, a second summary statistic based on the value of the first selected continuous feature associated with each discrete value of the candidate discrete feature; determining a deviation score for each of the candidate discrete features based on the second summary statistic; as well as The candidate discrete feature is transmitted for display in association with the first selected continuous feature based on the determined deviation score.

14. The medium according to claim 13, wherein Determining the second summary statistic for the first candidate discrete feature includes determining, for each discrete value of the first candidate discrete feature, a sum of values ​​of the first selected continuous feature associated with the discrete value and determining the second summary statistic based on the determined sum.

15. The medium according to claim 13, wherein The determination of the dissimilarity comprises determination of a cosine similarity.

16. The medium of claim 13, the processor-executable program code executable by the processing unit of the computing system causing the computing system to: determining a second selected continuous characteristic of the received data; determining, for each of the candidate discrete features, a third summary statistic based on the value of the second selected continuous feature associated with each discrete value of the candidate discrete feature; determining a second deviation score for each of the candidate discrete features based on the third summary statistic; as well as The candidate discrete feature is transmitted for display in association with the second selected continuous feature based on the determined second deviation score.

17. The medium according to claim 16, wherein determining the second summary statistic for the first candidate discrete feature comprises determining, for each discrete value of the first candidate discrete feature, a sum of the values ​​of the first selected continuous feature associated with the discrete value and determining the second summary statistic based on the determined sum, and Wherein, determining the third summary statistic for the first candidate discrete feature comprises determining, for each discrete value of the first candidate discrete feature, a second sum of the values ​​of the second selected continuous feature associated with the discrete value and determining the third summary statistic based on the second sum.

18. The medium according to claim 17, wherein The determination of the dissimilarity comprises determining a cosine similarity.

Images