Feature Engineering in Neural Network Optimization

Abstract

A transitive closure data structure is constructed for pairs of features represented in a vector space corresponding to the input dataset. The data structure contains a set of entries corresponding to the set of all possible paths between the first feature in the pair and the second feature in the pair in a graph of the vector space. The data structure is reduced by removing a subset of the set of entries so that only a single entry corresponding to a single path remains in the transitive closure data structure. Feature intersections are formed from clusters of features remaining in the reduced ontology graph resulting from reducing the transitive closure data structure. Layers are configured in the neural network to represent feature intersections, which enable the neural network to generate predictions within a defined accuracy range associated with the dataset.

Description

[Technical field]

[0001] The present invention relates generally to methods, systems, and computer program products for configuring neural networks to generate predictions based on large data sets. More particularly, the present invention relates to methods, systems, and computer program products for feature engineering in neural network optimization. [Background technology]

[0002] A dataset, as used herein, is a collection, body, or corpus of data in which data items correspond to one or more features from a feature set. A feature is an entity that is represented in the data. For example, in a dataset that is an excerpt from a book of English literature, distinct words may be considered features. As another example, in a dataset that includes traffic flow data, days of the week and times of the day may be considered features.

[0003] Features are not only entities that are directly or explicitly represented in the data, but also entities that can be inferred from the data. For example, assume that a dataset contains product ownership data, and one data item in the dataset indicates that company X owns product Y. In the dataset, X and Y are directly represented features (as the general classes "company" and "product"), but the fact that X also owns version 10 of product Y can also be inferred from X owning Y and Y having version 10. Thus, "Y version 10" is also a feature that can be inferred from the dataset.

[0004] A Feature Cross (FC) is a combination or concatenation of two or more features. For example, again using a dataset with traffic data as an example, "Monday" and "2pm" may be features in the dataset, but "Monday+2pm" (i.e., 2pm on Monday) is a feature cross formed from combining the features "Monday" and "2pm". That is, one data item may correspond to "Monday" but not necessarily to "2pm". Another data item may correspond to "2pm" but may correspond to "Thursday" but not to "Monday". Only data items that correspond to both "Monday" and "2pm" will correspond to the feature cross "Monday+2pm".

[0005] Given a dataset as input, a neural network (NN) can be configured to make predictions based on the dataset. For example, given a dataset that is an excerpt from a book of English literature, a neural network can be configured to predict whether certain words occur together or at some specified distance from each other. In another example, in a dataset that includes traffic flow data, a neural network can be configured to predict traffic volume at a certain time on a future date.

[0006] An ontology, as referred to herein, is a graph representation of a dataset where features form the vertices and relationships between features, either explicitly present in the dataset or inferred from it, form the edges. An ontology according to an example embodiment may have vertices that represent features, feature intersections, or a combination thereof.

[0007] An artificial neural network (ANN), simply called a neural network, is a computing system made of several simple, highly interconnected processing elements (nodes). The processing elements process information by their dynamic states in response to external inputs. ANNs are processing devices (algorithms and / or hardware) that are loosely analogous to the neuronal structure of the mammalian cerebral cortex, albeit on a much smaller scale. Large ANNs may have hundreds or thousands of processor units, whereas mammalian brains have billions of neurons that increase in size corresponding to the magnitude of their overall interactions and emergent behaviors.

[0008] A neural network may be configured to represent a set of features, feature intersections, or some combination thereof. The illustrative embodiments recognize that configuring a neural network such that predictions are consistent with a dataset and expected results (or actual results, if available) is a challenging problem for a variety of reasons. For example, the size of a neural network is related to the number of features or feature intersections over which the neural network is expected to operate.

[0009] Consider again the example of a traffic dataset. There are only 31 features in total, 7 days of the week and 24 hours of the day. However, not all times of the day are equally important to the prediction. For example, one may be more interested in traffic predictions on Friday at 8pm than on Monday at 8pm. However, now, with only two types of features being crossed, a total of 7x24=168 feature crosses are possible. The exemplary embodiment recognizes that a neural network representing 31 features is significantly less complex than a neural network constructed for 168 feature crosses.

[0010] Further, the exemplary embodiments recognize that a data set may be represented as a matrix where the features form columns and each data item is a row. A row gets an entry in a cell under the column if the data item in that row is relevant to the feature in that column. As can be seen, even a matrix with 31 columns for 7 days and 24 hours may be sparsely populated in that not every hour of every day has traffic data that may be available / useful / meaningful. The matrix of feature intersections will be even sparser compared to the matrix of features to which the feature intersections correspond. For example, for a simply day+hour type feature intersection, the modified matrix would now have 168 columns instead of 31, and the rows would now correspond to a smaller percentage of cells in the matrix. A neural network configured for these 168 example feature intersections would have to keep a very sparse matrix in memory to perform the calculations.

[0011] In practice, matrices are often thousands of columns by thousands of rows, and neural networks have multiple layers with each layer containing thousands of nodes. By training a neural network on a dataset, the weights of these nodes are calculated and adjusted to bring the output of the neural network within a desired threshold range of accuracy. Adjusting neural network weights is a computationally expensive process. Maintaining a large but sparse matrix requires unnecessarily large amounts of memory for training and operation of the neural network. The illustrative embodiments recognize that feature crossover is desirable to increase the relevance of the neural network output, but feature crossover also further sparses the matrix, increasing the complexity and resource requirements of the neural network. Summary of the Invention

[0012] Exemplary embodiments provide methods, systems, and computer program products. Viewed from a first aspect, the invention provides a method of constructing, with a processor and memory, a transitive closure data structure for a pair of features represented in a vector space corresponding to an input dataset, where the transitive closure data structure includes a set of entries corresponding to a set of paths in a graph of the vector space, the set of paths including all possible paths in the graph between a first feature in the pair and a second feature in the pair. An embodiment reduces the transitive closure data structure by removing a subset of the set of entries from the transitive closure data structure such that only a single entry corresponding to a single path remains in the transitive closure data structure. An embodiment forms feature intersections from clusters of features remaining in a reduced ontology graph resulting from reducing the transitive closure data structure. An embodiment configures layers in a neural network to represent the feature intersections, and the configured layers in the neural network cause the neural network to generate predictions that are within a defined accuracy range associated with the dataset.

[0013] Preferably, the present invention provides a method further comprising: using a processor and a memory to calculate, for a layer, an upper bound of a layer embedding corresponding to the feature crossings; and calculating, for a layer, a lower bound of a layer embedding corresponding to the feature crossings; wherein a layer size of a layer for the layer embedding is based on the upper bound and the lower bound.

[0014] Preferably, the present invention provides a method where the bed size is the calculated average of the upper and lower limits.

[0015] Preferably, the invention provides a method further comprising: converting the dataset into a sparse matrix; constructing an ontology graph corresponding to the sparse matrix, the ontology graph comprising a set of vertices corresponding to the set of features in the dataset; and performing forward materialization on the graph, the forward materialization adding an inference vertex to the set of vertices, the set of vertices including the inference vertex being used in constructing a transitive closure data structure.

[0016] Preferably, the present invention provides a method whereby the single remaining path meets the efficiency requirements.

[0017] Preferably, the invention provides a method, wherein the efficiency requirement comprises a minimum distance between a first feature and a second feature.

[0018] Preferably, the invention provides a method, wherein the efficiency requirement comprises a maximum utilization ratio between the first feature and the second feature.

[0019] Preferably, the invention provides a method wherein the reducing comprises a transitive reduction algorithm.

[0020] Preferably, the present invention provides a method in which a subset of vertices is removed from the ontology graph to form a reduced ontology graph by reducing a transitive closure data structure.

[0021] Preferably, the present invention provides a method further comprising identifying a plurality of clusters in the reduced ontology graph, wherein the plurality of clusters comprises the cluster described above, the cluster comprising at least two features such that a combination of the two features is usable to distinguish at least a first portion of the dataset from a second portion of the dataset.

[0022] Viewed from a second aspect, the present invention provides a computer usable program product including a computer readable storage device and program instructions stored on the storage device, the stored program instructions including: program instructions for, using a processor and memory, constructing a transitive closure data structure for pairs of features represented in a vector space corresponding to an input dataset, the transitive closure data structure comprising a set of entries corresponding to a set of paths in a graph of the vector space, the set of paths comprising all possible paths in the graph between a first feature in the pair and a second feature in the pair; program instructions for reducing the transitive closure data structure by removing a subset of the set of entries from the transitive closure data structure such that only a single entry corresponding to a single path remains in the transitive closure data structure; program instructions for forming feature intersections from clusters of features remaining in the reduced ontology graph resulting from reducing the transitive closure data structure; and program instructions for configuring layers in a neural network to represent the feature intersections, the configured layers in the neural network causing the neural network to generate predictions that are within a defined accuracy range associated with the dataset.

[0023] Preferably, the present invention provides a computer usable program product further comprising program instructions for computing, using a processor and memory, an upper bound for a layer embedding corresponding to feature crossings for a layer, and a lower bound for a layer embedding corresponding to feature crossings for a layer, wherein a layer size for the layer for the layer embedding is based on the upper bound and the lower bound.

[0024] Preferably, the present invention provides a computer usable program product, wherein the layer size is the calculated average of the upper and lower limits.

[0025] Preferably, the invention provides a computer usable program product further comprising program instructions for converting the dataset to a sparse matrix, and constructing an ontology graph corresponding to the sparse matrix, the ontology graph comprising a set of vertices corresponding to the set of features in the dataset, and program instructions for performing forward materialization on the graph, the forward materialization adding an inference vertex to the set of vertices, the set of vertices including the inference vertex being used in constructing a transitive closure data structure.

[0026] Preferably, the present invention provides a computer usable program product in which the single remaining pass satisfies the efficiency requirement.

[0027] Preferably, the present invention provides a computer usable program product, wherein the efficiency requirement comprises a minimum distance between a first feature and a second feature.

[0028] Preferably, the present invention provides a computer usable program product, wherein the efficiency requirement comprises a maximum utilization ratio between the first feature and the second feature.

[0029] Preferably, the present invention provides a computer usable program product, wherein the stored program instructions are stored in a computer readable storage device within a data processing system and the stored program instructions are transported over a network from a remote data processing system. The computer usable program product of claim 11, wherein the stored program instructions are stored in a computer readable storage device within a server data processing system and the stored program instructions are downloaded over the network to the remote data processing system for use in a computer readable storage device associated with the remote data processing system, further comprising program instructions for metering usage of the computer usable code associated with the request, and program instructions for generating an invoice based on the metered usage.

[0030] Viewed from a third aspect, a computer system includes a processor, a computer readable memory, and a computer readable storage device, program instructions stored on the storage device for execution by the processor via the memory, the stored program instructions including program instructions for constructing, using the processor and the memory, a transitive closure data structure for a pair of features represented in a vector space corresponding to an input dataset, the transitive closure data structure including a set of entries corresponding to a set of paths in a graph of the vector space, the set of paths including all possible paths in the graph between a first feature in the pair and a second feature in the pair. the program instructions for reducing the transitive closure data structure by removing a subset of the set of entries from the transitive closure data structure such that only a single entry corresponding to a single path remains in the transitive closure data structure; forming feature intersections from clusters of features remaining in the reduced ontology graph resulting from reducing the transitive closure data structure; and configuring layers in a neural network to represent the feature intersections, wherein the configured layers in the neural network cause the neural network to generate predictions that are within a defined accuracy range associated with the dataset.

[0031] Certain novel features believed characteristic of the present invention are set forth in the appended claims, however the invention itself, as well as a preferred mode of use, further objects and advantages thereof, will best be understood by reference to the following detailed description of illustrative embodiments when read in conjunction with the accompanying drawings. [Brief description of the drawings]

[0032] [Figure 1] 1 depicts a block diagram of a network of data processing systems in which illustrative embodiments may be implemented; [Diagram 2]1 illustrates a block diagram of a data processing system in which illustrative embodiments may be implemented; [Diagram 3] 1 depicts a block diagram of an example problem that may be solved in an illustrative embodiment; [Figure 4] 1 shows a block diagram of an example embedding according to an exemplary embodiment; [Diagram 5] FIG. 1 shows a block diagram of an example application for feature engineering in neural network optimization, in accordance with an illustrative embodiment. [Figure 6] 1 illustrates an embodiment of inference in accordance with an illustrative embodiment; [Figure 7] 1 illustrates an output graph resulting from the graph, according to an embodiment. [Figure 8A] 1 shows the placement of an ontology in a vector space. [Figure 8B] 1 illustrates a substantiation of reasoning about candidate feature intersections. [Figure 9] 1 shows a flowchart of an example process for feature engineering in neural network optimization, in accordance with an illustrative embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0033] Neural network training and optimization is a well-recognized and challenging technical field. The current state of the art in this field has certain shortcomings and limitations. The operation and / or configuration of the illustrative embodiments provide additional or new capabilities to improve upon existing techniques in this field, particularly in the areas of feature selection, feature intersection, and embedding.

[0034] A sparse matrix is ​​a matrix in which a percentage of cells below a threshold are non-null, non-void, non-zero, or some other representation of an indication that the cells are occupied or to be used. A dense matrix is ​​a matrix in which a percentage of cells above a threshold are non-null, non-void, non-zero, or some other representation of an indication that the cells are occupied or to be used. Exemplary embodiments recognize that it is desirable to have a dense matrix such that, within a defined accuracy tolerance, the dense matrix still represents the data in the sparse dataset. One way to achieve a dense matrix for a dataset is by selectively crossing certain features, omitting certain features or feature crossings, or both.

[0035] Exemplary embodiments recognize that selecting appropriate features to cross to form feature crossovers that result in a dense matrix is ​​a difficult problem to solve in a practical matrix. Currently available techniques rely on the personal knowledge and expertise of a person who specializes in a subject area to select features to cross from a dataset related to that subject. A judicious selection of features to cross can result in a dense matrix from a sparse matrix. For example, a comprehensive feature crossover approach would result in 168 feature crossovers in an example traffic dataset, but not all 168 feature crossovers are actually useful for practical purposes. A person skilled in traffic data analysis may selectively determine that Friday 8pm tends to be of interest and should form a feature crossover, but Monday 2pm and Wednesday 3pm seem to be roughly similar in the traffic data, and therefore Monday 2pm and Wednesday 3pm should be crossed together as one feature crossover. The person may determine that there is negligible or no data for Tuesday 1am, and therefore Tuesday 1am should not even be represented in the feature crossover matrix.

[0036] On the other hand, crossing too many features or incorrectly related features can lead to data loss, inaccurate predictions, or both. For example, if one crosses features for all weekdays Monday-Friday 12AM-5AM, all weekdays Monday-Friday 6AM-11AM, all weekdays Monday-Friday 12PM-5PM, and all weekdays Monday-Friday 6PM-11PM, a very small and dense matrix with only four feature crossings will indeed result. However, such a matrix cannot distinguish between features for Monday 7PM and Friday 7PM, which may be important for analysis and prediction.

[0037] Thus, currently, manual work along with specific skills and knowledge is required to convert a sparse matrix into a useful dense matrix. Exemplary embodiments recognize that automated methods for selecting intersecting features to build a dense matrix corresponding to a sparse data set without unacceptable loss of data granularity or unacceptable loss of precision would be useful. Exemplary embodiments further recognize that such automated methods would be useful for iteratively improving the dense matrix using machine learning methods via feedback from a generative implementation of a neural network trained on the dense matrix.

[0038] According to an exemplary embodiment, embedding is the process of configuring a layer of a neural network with nodes that correspond to feature crossings in a dense matrix such that the output of the neural network is within a defined accuracy tolerance when compared to the output of another neural network in which the layer is configured according to the features or feature crossings of the sparse matrix from which the dense matrix is ​​formed. The layer size is the number of nodes in the layer, the number of which are trained to correspond to feature crossings from the dense matrix. In a schematic embodiment, the number of nodes in the layer has an n:m correspondence with the number of feature crossings in the dense matrix (i.e., n nodes per m feature crossings). In one embodiment, the number of nodes in the layer has a 1:1 correspondence with the number of feature crossings in the dense matrix (i.e., one node per feature crossing). In another embodiment, the number of nodes in the layer has a 1:n correspondence with the number of feature crossings in the dense matrix (i.e., one node per n feature crossings). In another embodiment, the number of nodes in the layer has an n:1 correspondence with the number of feature crossings in the dense matrix (i.e., n nodes per feature crossing).

[0039] Exemplary embodiments recognize that larger layer sizes require greater amounts of computational and data storage resources for both training and operation compared to smaller layer sizes, while at the same time providing greater accuracy in the output of the layer compared to the output of smaller sized layers.

[0040] Exemplary embodiments recognize that feature selection for crossover and layer size are also interdependent, since layer size depends on a set of hyperparameters, including but not limited to feature crossover. For example, if a given layer configuration does not produce the desired output or accuracy, the hyperparameters of the neural network must be adjusted. Features are hyperparameters, and if feature crossover is changed, the change may change one or more hyperparameters of the neural network. Changing the hyperparameters requires that the neural network be retrained with the new hyperparameters. Training a neural network with thousands of nodes in multiple layers with matrices that are thousands of columns wide and deep is computationally prohibitively expensive. Thus, the less the hyperparameters of a neural network are perturbed, the lower the cost of training and deploying the neural network. Thus, again, automated and smart selection of features for crossover becomes useful.

[0041] Furthermore, given that a particular dense matrix with a certain density is achieved by forming certain feature intersections, a particular embedding will produce an output of a certain precision. Different embeddings are possible from the same dense matrix by choosing different ways of representing sets or subsets of feature intersections from the matrix into the embedding, each of which produces a different output of different precision. Thus, the illustrative embodiments recognize that there are ranges of embeddings for each subset of feature intersections for which the embedding produces an output of acceptable precision.

[0042] The illustrative embodiments recognize that currently there is no systematic and objective method for selecting an embedding. Typically, those skilled in the art of neural network training experiment with various embeddings, at great computational cost, to arrive at an acceptable embedding. The illustrative embodiments recognize that a systematic and objective computation of upper and lower bounds on the embedding for a given dense matrix would be useful.

[0043] The illustrative embodiments recognize that currently available tools or solutions do not address or provide sufficient solutions for these needs / problems. The illustrative embodiments used to describe the present invention generally address and solve the above-mentioned problems and other related problems through feature engineering in neural network optimization.

[0044] The embodiments may be implemented as a combination of certain hardware components and software applications. Implementations of the embodiments, or one or more components thereof, may be configured as modifications to existing neural network training and prediction configurations, with companion software applications running on any combination of (i) the existing neural network training and prediction configuration itself, (ii) a data processing system that communicates with the existing neural network training and prediction configuration over a short-range wireless or local area network (LAN), and (iii) a data processing system that communicates with the existing neural network training and prediction configuration over a wide area network (WAN).

[0045] The embodiments form an ontology from a given dataset. In other words, the embodiments label the vertices with labels (features) and the edges with relationships found in the dataset. The embodiments further build the ontology by inputting additional vertices and edges that correspond to the labels and relationships that are derived or inferred. This act of building an ontology through inference is called forward materialization.

[0046] The embodiment forms a vector space corresponding to the graph of the ontology. The vector space includes a numerical representation of the data. The vector space formed by the embodiment includes a numerical representation of the features input to the graph. The vector space generated by the embodiment is useful for determining the similarity between concepts present in the vector space. For example, the embodiment applies a similarity function to determine the similarity between such concepts underlying the features. A concept is an abstraction or generalized representation of a feature. For example, Monday is a feature and Tuesday is a feature. Even though the data set itself may not indicate anywhere that Monday and Tuesday are days of the week, they are similar because they are both manifestations of the concept of days of the week. The concepts are not limited to this trivial example, and seemingly unrelated features may be linked through esoteric concepts in the similarity analysis of the embodiment. Cosine similarity is one non-limiting example of a similarity function that may be used in the embodiment for the purpose of similarity determination in the vector space.

[0047] An embodiment constructs a transitive closure data structure corresponding to a vector space. A transitive closure data structure is a data structure that contains information about all possible paths from getting from point A to point B in a vector space. The transitive closure data structure contains one or more paths between features, inferences, and concepts. An embodiment analyzes the transitive closure data structure to determine and select a most desirable path from multiple paths between two entities represented in the transitive closure data structure. In one embodiment, a path is desirable if it is the shortest or most efficient of all paths in the transitive closure data structure. In other embodiments, a path may also be desirable if it is the most used path in the transitive closure data structure. In general, any selection criteria may be used to select a path from multiple paths in the transitive closure data structure.

[0048] An embodiment removes non-selected paths from the transitive closure data structure. By removing non-selected paths from the transitive closure data structure, one or more vertices, edges, or combinations thereof are also removed from the ontology graph. Removing a vertex or edge from the graph indicates that the contribution of the vertex or edge in the graph falls below a desired level. In other words, the removed vertex or edge is deemed not to contribute to the graph in a sufficiently meaningful way, and its removal is unlikely to affect an undesirable (more than a threshold) decrease in accuracy of predictions using the post-removal dataset.

[0049] Thus, embodiments perform graph reduction to form a reduced graph such that the reduced graph still fully represents the original dataset, but includes features, inferences, and concepts interconnected through the most desirable paths in the graph. Such graph reduction is called transitive reduction. The transitive reduction of embodiments may be controlled by the extent to which inferences are drawn, parameters of a similarity function for finding similarities and concepts, criteria for selecting desirable paths, removal or non-removal of certain undesirable vertices or edges, or some combination thereof.

[0050] The remaining vertices in the reduced graph are called discriminative nodes because they have the ability to discriminate between data items in the matrix and are not redundant, perfunctory, or redundant. An embodiment identifies one or more clusters in the remaining vertices in the reduced graph. The clusters identify features that can be crossed to form useful feature crossings that do not cause unacceptable levels of data loss, loss of precision, or both. Each identified feature crossing has an upper and lower bound of cells in the dense matrix. The upper bound is the calculation of all positive instance records for each feature that contributes to the cluster regardless of the transitive reduction cost formula. For example, in the traffic data example, if we do not have any data for 2:00 AM on Monday, those vertices and edges will drop out during reduction without any loss of information. Thus, at the upper bound, the graph is the original graph minus the features that do not contribute to the graph (do not have a path through them). In the traffic data example, this may again reduce the graph from 168 possible feature crossings to, for example, 142, or some other smaller number. In a more selective embodiment, Monday at 2 am may have less than a minimum number of data items, or less than a statistical distribution cutoff number, and may also be dropped. In that case, the graph will be smaller, but still without significant loss of data or precision. Outlier data items and corresponding features may be filtered out in this manner.

[0051] The lower bound is the function output of all discriminatory nodes and their distinct values. Now, since all undesirable paths have been removed, the removal may have removed some feature crossings that were meaningful but not the most meaningful. In other words, by reduction, the graph contains only nodes that are highly discriminatory in the vector space. It always removes less discriminatory features, feature crossings, and corresponding data items. In the example traffic dataset, the lower bound may therefore be much lower, for example 56 or some smaller number, but with an associated loss of data or precision or both.

[0052] Thus, embodiments generate upper and lower bounds for the embedding. Embodiments perform dimensionality reduction in the embedding using the upper and lower bounds of the identified feature intersections. For example, one embodiment takes a statistical average of the upper and lower bounds to produce a number that forms the layer size for the embedding.

[0053] The method of feature engineering in neural network optimization described herein is not available in methods currently available in the art of neural network construction and operation for prediction. The method of the embodiments described herein, when implemented to run on a device or data processing system, comprises a significant advance in the functionality of that device or data processing system in systematically and objectively optimizing the data sets used to train the neural network, and in generating neural networks whose layer sizes are optimized to generate predictions of desired accuracy with significantly reduced computational resources.

[0054] Exemplary embodiments are described with respect to certain types of data items, data sets, features, graphs, feature intersections, algorithms, formulas, functions, reductions, neural networks, layers, nodes, relationships, clusters, data structures, devices, data processing systems, environments, components, and applications, by way of example only. Any particular manifestation of these and other similar artifacts is not intended to limit the invention. Any suitable manifestation of these and other similar artifacts may be selected within the scope of the exemplary embodiments.

[0055] Additionally, the exemplary embodiments may be implemented with respect to any type of data, data source, or access to a data source on a data network. Any type of data storage device may provide data to the embodiments of the present invention locally at a data processing system or over a data network within the scope of the present invention. Where an embodiment is described with a mobile device, any type of data storage device suitable for use with a mobile device may provide data to such embodiments locally at the mobile device or over a data network within the scope of the exemplary embodiments.

[0056] Exemplary embodiments are described using specific code, designs, architectures, protocols, layouts, schematics, and tools as examples only, and are not limited to exemplary embodiments. Furthermore, exemplary embodiments are described in some instances using specific software, tools, and data processing environments as examples only for clarity of explanation. Exemplary embodiments may be used in conjunction with other equivalent or similar purpose structures, systems, applications, or architectures. For example, other equivalent mobile devices, structures, systems, applications, or architectures therefor may be used in conjunction with such embodiments of the present invention within the scope of the present invention. Exemplary embodiments may be implemented in hardware, software, or a combination thereof.

[0057] The examples in this disclosure are used only for clarity of explanation and are not limited to the exemplary embodiments. Additional data, operations, actions, tasks, activities, and operations are contemplated by this disclosure and are considered within the scope of the exemplary embodiments.

[0058] Any advantages listed herein are merely examples and are not intended to be limiting to example embodiments. Additional or different advantages may be realized by specific example embodiments. Furthermore, a particular example embodiment may have some, all, or none of the advantages listed above.

[0059] Referring now to the drawings, and in particular to Figures 1 and 2, these are exemplary diagrams of data processing environments in which illustrative embodiments may be implemented. Figures 1 and 2 are merely examples and are not intended to assert or imply any limitation with respect to the environments in which different embodiments may be implemented. Particular implementations may make many modifications to the environments depicted based on the following description.

[0060] 1 illustrates a block diagram of a network of data processing systems in which exemplary embodiments may be implemented. Data processing environment 100 is a network of computers in which exemplary embodiments may be implemented. Data processing environment 100 includes network 102. Network 102 is a medium used to provide communications links between various devices and computers connected together in data processing environment 100. Network 102 may include connections such as wired, wireless communication links, or fiber optic cables.

[0061] Client or server are merely example roles of certain data processing systems connected to network 102 and are not intended to exclude other configurations or roles for these data processing systems. Server 104 and server 106 are coupled to network 102 along with storage unit 108. Software applications may execute on any computer in data processing environment 100. Clients 110, 112, and 114 are also coupled to network 102. A data processing system, such as server 104 or 106 or client 110, 112, or 114, may contain data and may have software applications or software tools executing thereon.

[0062] Merely by way of example, and without implying any limitation to such architecture, FIG. 1 illustrates certain components that may be used in an example implementation of an embodiment. For example, servers 104 and 106 and clients 110, 112, 114 are illustrated as servers and clients by way of example only, and do not imply any limitation to a client-server architecture. As another example, an embodiment may be distributed across multiple data processing systems and data networks as illustrated, and another embodiment may be implemented on a single data processing system within the scope of an exemplary embodiment. Data processing systems 104, 106, 110, 112, and 114 also represent example nodes in clusters, partitions, and other configurations suitable for implementing an embodiment.

[0063] Device 132 is an example of a device described herein. For example, device 132 may take the form of a smartphone, a tablet computer, a laptop computer, a stationary or portable version of client 110, a wearable computing device, or any other suitable device. Any software application described as executing in another data processing system of FIG. 1 may be configured to execute in device 132 in a similar manner. Any data or information stored or generated in another data processing system of FIG. 1 may be configured to be stored or generated in device 132 in a similar manner.

[0064] Application 105 implements the embodiments described herein. Application 105 implements the remotely enabled features of the embodiments described herein. Application 111 implements the locally enabled features of the embodiments described herein or the natively enabled features of the embodiments described herein. Application 134 implements the natively enabled features of the embodiments described herein. To distribute certain features of the embodiments, applications 105 and 111 may be used in combination, applications 105 and 134 may be used in other combinations, and applications 105, 111, and 134 may be used in other combinations. Application 105 implements the embodiments described herein. Input data 109 is an example data set that may be used in the operation of application 105 in the manner described herein. Application 105 uses input data 109 to train a neural network, which may then be operated to make predictions consistent with input data 109 in the manner described herein.

[0065] Servers 104 and 106, storage unit 108, and clients 110, 112, and 114, and device 132 may be coupled to network 102 using wired connections, wireless communication protocols, or other suitable data connectivity. Clients 110, 112, and 114 may be, for example, personal computers or network computers.

[0066] In the illustrated embodiment, server 104 may provide data such as boot files, operating system images, and applications to clients 110, 112, and 114. Clients 110, 112, and 114 may be clients to server 104 in this embodiment. Clients 110, 112, 114, or some combination thereof, may include their own data, boot files, operating system images, and applications. Data processing environment 100 may include additional servers, clients, and other devices not shown.

[0067] In the depicted example, data processing environment 100 may be the Internet. Network 102 may represent a collection of networks and gateways that communicate with each other using Transmission Control Protocol / Internet Protocol (TCP / IP) and other protocols. At the heart of the Internet is a backbone of data communication links between major nodes or host computers, including thousands of commercial, government, educational, and other computer systems that route data and messages. Of course, data processing environment 100 may also be implemented as a number of different types of networks, such as, for example, an intranet, a local area network (LAN), or a wide area network (WAN). FIG. 1 is intended as an example and not as an architectural limitation for different illustrative embodiments.

[0068] Among other uses, the data processing environment 100 may be used to implement a client-server environment in which the exemplary embodiments may be implemented. A client-server environment allows software applications and data to be distributed across a network such that applications function by using interactivity between client and server data processing systems. The data processing environment 100 may also employ a service-oriented architecture in which interoperable software components distributed across a network may be packaged together as a coherent business application. The data processing environment 100 may also take the form of a cloud and employ a cloud computing model of service delivery to enable convenient, on-demand network access to a shared pool of configurable computing resources (e.g., networks, network bandwidth, servers, processing, memory, storage, applications, virtual machines, and services) that may be rapidly provisioned and released with minimal administrative effort or interaction with a service provider.

[0069] Referring now to Figure 2, a block diagram of a data processing system in which the illustrative embodiments may be implemented is shown. Data processing system 200 is an example of a computer, such as servers 104 and 106 or clients 110, 112, and 114 in Figure 1, or another type of device in which computer usable program code or instructions implementing processes may be located for the illustrative embodiments.

[0070] Data processing system 200 also represents a data processing system or configuration therein, such as data processing system 132 in FIG. 1, in which computer usable program code or instructions implementing the processes of the exemplary embodiments may be located. Data processing system 200 is described as a computer only as an example and without limitation. Implementations in the form of other devices, such as device 132 in FIG. 1, may modify data processing system 200, such as by adding a touch interface, or even remove certain illustrated components from data processing system 200 without departing from the general description of the operation and functionality of data processing system 200 described herein.

[0071] In the illustrated embodiment, data processing system 200 employs a hub architecture including a north bridge and memory controller hub (NB / MCH) 202 and a south bridge and input / output (I / O) controller hub (SB / ICH) 204. A processing unit 206, a main memory 208, and a graphics processor 210 are coupled to north bridge and memory controller hub (NB / MCH) 202. Processing unit 206 may include one or more processors and may be implemented using one or more heterogeneous processor systems. Processing unit 206 may be a multi-core processor. Graphics processor 210 may be coupled to NB / MCH 202 through an accelerated graphics port (AGP) in some embodiments.

[0072] In the illustrated embodiment, a local area network (LAN) adapter 212 is coupled to the south bridge and I / O controller hub (SB / ICH) 204. An audio adapter 216, a keyboard and mouse adapter 220, a modem 222, a read-only memory (ROM) 224, a universal serial bus (USB) and other ports 232, and PCI / PCIe devices 234 are coupled to the south bridge and I / O controller hub 204 through a bus 238. A hard disk drive (HDD) or solid state drive (SSD) 226 and a CD-ROM 230 are coupled to the south bridge and I / O controller hub 204 through a bus 240. The PCI / PCIe devices 234 may include, for example, an Ethernet adapter, an add-in card, and a PC card for a notebook computer. PCI does not use a card bus controller, and PCIe does not use a card bus controller. ROM 224 may be, for example, a flash binary input / output system (BIOS). Hard disk drive 226 and CD-ROM 230 may use, for example, an Integrated Drive Electronics (IDE), Serial Advanced Technology Attachment (SATA) interface, or variations such as external SATA (eSATA) and micro SATA (mSATA). Super I / O (SIO) device 236 may be coupled to south bridge and I / O controller hub (SB / ICH) 204 through bus 238.

[0073] Memory such as main memory 208, ROM 224, or flash memory (not shown) are some examples of computer usable storage devices. Hard disk drives or solid state drives 226, CD-ROM 230, and other similar usable devices are some examples of computer usable storage devices that include computer usable storage media.

[0074] An operating system runs on the processing unit 206. The operating system coordinates and provides control of various components within the data processing system 200 in FIG. 2. The operating system may be a commercially available operating system for any type of computing platform, including, but not limited to, server systems, personal computers, and mobile devices. An object-oriented or other type of programming system may operate in conjunction with the operating system and provide calls to the operating system from programs or applications executing on the data processing system 200.

[0075] Instructions for an operating system, an object oriented programming system, and an application or program, such as application 105 of FIG. 1, may be located on a storage device, such as in the form of code 226A on hard disk drive 226, and loaded into at least one of the one or more memories, such as main memory 208, for execution by processing unit 206. The processes of the illustrative embodiments may be executed by processing unit 206 using computer-implemented instructions, which may be located in a memory, such as main memory 208, read-only memory 224, or in one or more peripheral devices, for example.

[0076] Additionally, in one case, code 226A may be downloaded from remote system 201B over network 201A, and similar code 201C is stored on storage device 201D. In another case, code 226A may be downloaded to remote system 201B over network 201A, and the downloaded code 201C is stored on storage device 201D.

[0077] The hardware in Figures 1-2 may vary depending on the implementation. Other internal hardware or peripheral devices, such as flash memory, equivalent non-volatile memory, or optical disk drives, may be used in addition to or in place of the hardware depicted in Figures 1-2. Additionally, the processes of the illustrative embodiments may be applied to multiprocessor data processing systems.

[0078] In some illustrative examples, data processing system 200 may be a personal digital assistant (PDA), which is generally configured with flash memory to provide non-volatile memory for storing operating system files and / or user-generated data. The bus system may include one or more buses, such as a system bus, an I / O bus, and a PCI bus. Of course, the bus system may be implemented with any type of communications fabric or architecture that provides for data transport between different components or devices attached to the fabric or architecture.

[0079] The communications unit may include one or more devices used to transmit and receive data, such as a modem or network adapter. The memory may be, for example, main memory 208 or a cache, such as a cache found in north bridge and memory controller hub 202. The processing unit may include one or more processors or CPUs.

[0080] 1-2 and above-described embodiments are not meant to imply architectural limitations. For example, data processing system 200 may also be a tablet computer, a laptop computer, or a telephone device in addition to taking the form of a mobile or wearable device.

[0081] When a computer or data processing system is described as a virtual machine, device, or component, the virtual machine, device, or component operates in the manner of data processing system 200 using a virtualized representation of some or all of the components shown in data processing system 200. For example, in a virtual machine, device, or component, processing unit 206 is represented as a virtualized instance of all or some of the hardware processing units 206 available in the host data processing system, main memory 208 is represented as a virtualized instance of all or some portions of main memory 208 that may be available in the host data processing system, and disk 226 is represented as a virtualized instance of all or some portions of disk 226 that may be available in the host data processing system. The host data processing system in such a case is represented by data processing system 200.

[0082] 3, this figure shows a block diagram of an example problem that may be solved in an exemplary embodiment. Matrix 302 is an example of input data 109 of FIG.

[0083] Assume that matrix 302 represents an example traffic dataset described herein. As shown, matrix 302 includes individual features for 7 days of the week and 24 hours of the day. As can be seen, matrix 302 is very sparsely populated. Matrices 304 and 306 illustrate the problem of manual feature intersection. When intersecting all hours of the day with all days of the week, some significant intersections are obtained, but the result is a larger matrix, now 168 columns wide, that is even sparser than matrix 302. Conversely, when intersecting too many features or irrelevant features, as in matrix 306, a smaller matrix may be achieved, but its cells will not be as representative of matrix 302 as desired.

[0084] Referring to FIG. 4, this figure shows a block diagram of an example embedding according to an exemplary embodiment. The configuration 400 shows a neural network 402 including several layers, such as layer 402A, layer 402B, ... layer 402X. Feature intersections 404, including feature intersections FC1, FC2, ... FCn, are embedded in layer 402B as an example. Any number of layer embeddings can be formed with feature intersections in this manner. For example, feature intersections 406 are similarly embedded in layer 402X in the manner described herein.

[0085] 5, a block diagram of an example application for feature engineering in neural network optimization is shown, in accordance with an illustrative embodiment. Application 502 may be implemented as application 105 of FIG.

[0086] Application 502 receives input data 504, which may be in the form of input data 109 of FIG. 1 or matrix 302 of FIG. 3. Component 506 forms an ontology using forward materialization in a manner described herein. Component 508 builds a vector space corresponding to the ontology and performs a similarity search in a manner described herein. Component 510 transitively reduces the graph using a transitive closure data structure formed from the ontology graph.

[0087] Component 512 identifies clusters of discriminative nodes (features) that remain in the reduced graph. Clusters of discriminative nodes extracted from or isolated within the reduced graph form a set of candidate feature intersections. Candidate feature intersections are discriminative feature intersections that can represent at least a portion of the original data set with a desired degree of accuracy.

[0088] Component 514 calculates upper and lower bounds for candidate feature intersections as described herein. Given an embedding range defined by the upper and lower bounds, component 516 determines the dimensionality, i.e., layer size, of the embedding for a particular feature intersection. In one embodiment, component 516 reduces or modifies or varies the dimensionality of the embedding to set the layer size.

[0089] In one embodiment, application 502 outputs a set of candidate feature intersections 518. In another embodiment, application 502 outputs the set of candidate feature intersections 518 along with upper and lower bounds 520 for each candidate feature intersection. In another embodiment, application 502 outputs the set of candidate feature intersections 518 along with (i) upper and lower bounds 520, (ii) a calculated layer size 522, or (iii) both (i) and (ii) for each candidate feature intersection.

[0090] The outputs 518, 520, 522, or some combination thereof, can be used to configure a neural network. For example, in one embodiment, the application 502 causes the neural network to embed a subset of the candidate feature intersections 518 according to a layer embedding size 522. According to one embodiment, the application 502 further configures the neural network to form a model implementation 524 in a prediction engine, such as prediction engine 107. In one embodiment, the configuration, implementation, or both are automatic without human intervention. In another embodiment, the configuration, implementation, or some combination thereof is performed after human review / tuning / selection of one or more outputs.

[0091] Once the neural network is configured and implemented, in one embodiment, the application 502 receives information including the set of implemented feature intersections as feedback 526. The set of implemented feature intersections may fully or partially overlap with a subset of the candidate feature intersections 518. For example, a user may configure some feature intersections from the candidate feature intersections 518 and may also configure some feature intersections that are not in the set of candidate feature intersections 518 from the user's own experience.

[0092] The model implementation 524 generates model outputs 528, e.g., predictions. In one embodiment, the application 502 receives the outputs 528 directly from the model implementation 524 as feedback. In another embodiment, the application 502 receives the outputs 528 after a user applies adjustments 530 to the outputs 528. In either case, the application 502 uses the feedback 526 and the adjusted or unadjusted outputs 528 to retrain the neural network using suitable known machine learning techniques.

[0093] A detailed description of the implementation of a particular embodiment for solving a particular example problem is provided herein. This description is not intended to be limiting to all embodiments described herein. Those skilled in the art may be able to adapt details from this embodiment to other embodiments, and such adaptations are considered within the scope of the example embodiments.

[0094] Given a semantic graph (e.g., an ontology), we want to query whether eg A is related to eg D through eg p and enumerate the resolution roots. The set of rules for computing the transitive closure for a given relation can be expressed in an API-independent manner. Statement is [urn x-hp eg / A, urn x-hp eg / p,urn x-hp eg / D] Rule rule1 concluded(eg A eg p eg D)<-Fact(eg A eg p eg B)

[0095] As a reminder, transitive closure is the method by which, given a directed graph, for every pair of vertices (u,v) in the given graph, the system determines whether vertex v is reachable from another vertex u. Reachable means that there exists a path from vertex u to vertex v. The reachability matrix is ​​called the transitive closure of the graph.

[0096] This reachability matrix is ​​a way to compute the cost of inference. First-order transitive inference shows the instantiation of a → c with the explicit notion of a → b → c as having a cost of a single node to traverse. If the traversed node is within the same subclass, the cost remains unchanged. If the traversed node occurs within the context of a grandparent subclass, the cost is increased by a function of the cosine similarity between the classes. def transitiveClosure(V,graph) reach = [i[] for i in graph] for k in range(V) for i in range(V) for j in range(V) reach[i][j]=reach[i][j] or (reach[i][k] and reach[k][j]) return reach graph = [[1,1,0,1], [0,1,1,0], [0,0,1,1], [0,0,0,1]] transitiveClosure(graph)

[0097] 6 illustrates the instantiation of inference according to an exemplary embodiment. Inference is illustrated in this figure by a transitive closure graph 604 (right side) for a deep semantic graph 602 (left side).

[0098] In the example, graph 602 illustrates a subset of the semantic graph. Fine-grained nodes are entities in the graph that are connected by rdfs:subClassOf or equivalent relationships to non-grained nodes (parents). Graph 604 on the right shows inference instantiations. Inference instantiations are a form of transitive closure. Dark edges represent transitive reductions of the semantic graph, meaning that a path is made directly from node A (for example) to node F. The ability to compute the semantic cost of each transitive reduction instantiation in graph 604 is a feature enabled by the example embodiment.

[0099] The function takes each instantiated entity (λ) across the entire graph multiplied by its transitive closure value (x) multiplied by the cosine similarity between parent nodes (cosθ). A non-limiting example cosine similarity algorithm is shown below for reference:

[0100]

number

[0101] The output of each function lies approximately within the range of a normal distribution. The z-score is calculated by taking the point value of each entity and subtracting the average of the result divided by the standard deviation of all the values. The decision of how to handle the z-score is somewhat policy-based, but in general it removes statistically significant outliers. Skewness of the distribution will drive policy in this regard.

[0102] Figure 7 shows an output graph resulting from graph 604 of Figure 6, according to an embodiment. Graph 702 represents the output of the formula on virtual subgraph 604 of Figure 6. Entities B, G (indicated by line patterns) are linked to a single intersection feature. Without the benefit of the formula given above, the existing linking strategy may include entity E to the detriment of the trained model. The method similarly links entities C, E, F (indicated by fine patterns) to a single intersection feature. Entity D is excluded because the cost calculation is too high.

[0103] Moving from the hypothetical example to a real-world graph, assume that the Tensorboard visualization shows the placement of an ontology in a vector space. FIG. 8A shows the placement of an ontology in a vector space. FIG. 8B shows the instantiation of the inference about candidate feature intersections. Exemplary candidate feature intersections are shown by line nodes in graph 852. The line nodes indicate clusters that are being formed and represent candidate intersection features for the statistical model. The embedding lower bound is calculated by summing the instance data of all entities that contribute to each cluster (intersection features) as described herein.

[0104] 9, a flowchart of an example process for feature engineering in neural network optimization is shown, in accordance with an exemplary embodiment. The process 900 may be implemented in the application 502 of FIG.

[0105] The application receives an input dataset (block 902). The application builds an ontology graph from the dataset (block 904). The application performs forward materialization with reasoning on the graph (block 906).

[0106] The application builds a vector space of entities (features) in the forward materialized graph (block 908). The application identifies one or more concepts in the vector space (block 910). The application determines the similarity between the concepts (block 912).

[0107] The application builds a transitive closure data structure for pairs of entities that are linked by their conceptual similarity (block 914). Using the transitive closure data structure, the application performs transitive reduction (block 916). The application performs transitive reduction on one or more of the transitive closure data structures.

[0108] The application identifies clusters of surviving nodes / entities / features in the reduced graph resulting from the transitive reduction operation (block 918). The clusters are candidate feature intersections. The application computes upper bounds for the candidate feature intersections (block 920). The application computes lower bounds for the candidate feature intersections (block 922). The application computes an embedding layer size using the upper and lower bounds for the feature intersections (block 924).

[0109] The application configures layers in the neural network according to the calculated embedding layer size (block 926). In this manner, any number of features may be embedded in any number of layers. The application trains the neural network whose layers are thus configured (block 928). The application deploys the trained neural network in a prediction engine (block 930). The application may then terminate process 900.

[0110] In one embodiment, the application further receives feedback from the deployment in the prediction engine (block 932). The application adjusts the feature intersections, the embedding layer size, or both according to the feedback (block 934), as described herein. The application then exits process 900.

[0111] Thus, computer-implemented methods, systems or apparatus, and computer program products are provided in exemplary embodiments for feature engineering in neural network optimization and other related features, functions, or operations. Where an embodiment or portion thereof is described in terms of a type of device, the computer-implemented method, system or apparatus, computer program product, or portions thereof, is adapted or configured for use with the appropriate and equivalent representation of that type of device.

[0112] Where an embodiment is described as being implemented in an application, delivery of the application in a Software as a Service (SaaS) model is considered within the scope of the exemplary embodiments. In a SaaS model, the capabilities of the application implementing the embodiment are provided to a user by executing the application in a cloud infrastructure. The user may access the application using a variety of client devices through a thin-client interface such as a web browser (e.g., web-based email) or other light-client application. The user does not manage or control the underlying cloud infrastructure, including the network, servers, operating systems, or storage of the cloud infrastructure. In some cases, the user may not even manage or control the capabilities of the SaaS application. In some other cases, the SaaS implementation of the application may allow for the possible exception of limited user-specific application configuration settings.

[0113] The invention may be a system, method, or computer program product, or combination thereof, at any possible level of technical detail of integration. The computer program product may include a computer-readable storage medium (or media) having computer-readable program instructions thereon for causing a processor to perform aspects of the invention.

[0114] A computer readable storage medium may be a tangible device capable of holding and storing instructions for use by an instruction execution device. A computer readable storage medium may be, for example, but not limited to, an electronic storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the foregoing. A non-exhaustive list of more specific examples of computer readable storage media includes portable computer diskettes, hard disks, random access memory (RAM), read only memory (ROM), erasable programmable read only memory (EPROM or flash memory), static random access memory (SRAM), portable compact disk read only memory (CD-ROM), digital versatile disk (DVD), memory stick, floppy disk, punch cards or mechanically encoded devices such as ridge structures in grooves that allow instructions to be recorded thereon, and any suitable combination of the foregoing. As used herein, computer readable storage media, including but not limited to computer readable storage devices, should not be construed as being ephemeral signals per se, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through a waveguide or other transmission medium (e.g., light pulses passing through a fiber optic cable), or electrical signals transmitted through electrical wires.

[0115] The computer readable program instructions described herein may be downloaded from a computer readable storage medium to each computing / processing device or to an external computer or storage device via a network, such as the Internet, a local area network, a wide area network, or a wireless network, or a combination thereof. The network may include copper transmission cables, optical transmission fiber, wireless transmission, routers, firewalls, switches, gateway computers, or edge servers, or a combination thereof. A network adapter card or network interface in each computing / processing device receives the computer readable program instructions from the network and transfers the computer readable program instructions for storage in a computer readable storage medium in the respective computing / processing device.

[0116] The computer readable program instructions for carrying out the operations of the present invention may be either source code or object code written in any combination of one or more programming languages, including assembler instructions, instruction set architecture (ISA) instructions, machine instructions, machine dependent instructions, microcode, firmware instructions, state setting data, configuration data for integrated circuits, or object oriented programming languages ​​such as Smalltalk®, C++, and procedural programming languages ​​such as the "C" programming language or similar programming languages. The computer readable program instructions may execute completely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or completely on a remote computer or server. In the latter scenario, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN). Alternatively, the connection may be made to an external computer (e.g., through the Internet using an Internet Service Provider). In some embodiments, electronic circuitry including, for example, a programmable logic circuit, a field programmable gate array (FPGA), or a programmable logic array (PLA), may execute computer readable program instructions by utilizing state information of the computer readable program instructions to individualize the electronic circuitry to perform aspects of the invention.

[0117] Aspects of the present invention are described herein with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer readable program instructions.

[0118] These computer readable program instructions may be provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus to manufacture a machine, such that the instructions executed by the processor of the computer or other programmable data processing apparatus create means for performing the functions / operations specified in one or more blocks of the flowcharts and / or block diagrams. These computer readable program instructions may also be stored in a computer readable storage medium that may instruct a computer, programmable data processing apparatus, or other device, or combination thereof, to function in a particular manner, such that the computer readable storage medium having instructions stored therein comprises an article of manufacture including instructions for performing aspects of the functions / operations specified in one or more blocks of the flowcharts and / or block diagrams.

[0119] Computer readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device that causes the computer, other programmable apparatus, or other device to perform a series of operational steps to create a computer-implemented process, such that the instructions executing on the computer, other programmable apparatus, or other device perform the functions / operations specified in one or more blocks of the flowcharts and / or block diagrams.

[0120] The flowcharts and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present invention. In this regard, each block in the flowcharts or block diagrams may represent a module, segment, or portion of instructions, including one or more executable instructions for implementing a specified logical function. In some alternative implementations, the functions described in the blocks may occur out of the order described in the figures. For example, two blocks shown in succession may in fact be executed substantially simultaneously, or the blocks may be executed in the reverse order depending on the functionality involved. It should also be noted that each block of the block diagrams and / or flowchart illustrations, and combinations of blocks in the block diagrams and / or flowchart illustrations, may be implemented by a dedicated hardware-based system that performs the specified functions or operations, or executes a combination of dedicated hardware and computer instructions.

[0121] The embodiments of the present invention may also be delivered as part of a service agreement with a client company, a non-profit organization, a government agency, an internal organizational structure, and the like. Aspects of these embodiments may include configuring a computer system to perform some or all of the methods described herein, and deploying software, hardware, and web services that perform some or all of the methods described herein. Aspects of these embodiments may also include analyzing client behavior, generating recommendations in response to the analysis, building a system that performs some of the recommendations, integrating the system into existing processes and infrastructure, metering the use of the system, allocating costs to users of the system, and charging for the use of the system. Although the above embodiments of the present invention have each been described by stating individual advantages, the present invention is not limited to that particular combination. To the contrary, such embodiments may also be combined in any manner and number in accordance with the intended deployment of the present invention without losing their advantageous effects.

Claims

1. 1. A processor-implemented method comprising: constructing a transitive closure data structure for pairs of features represented in a vector space corresponding to an input dataset, the transitive closure data structure including a set of entries corresponding to a set of paths in a graph of the vector space, the set of paths including all reachable paths in the graph between a first feature in the pair and a second feature in the pair; reducing the transitive closure data structure by removing a subset of the set of entries from the transitive closure data structure such that only a single entry corresponding to a single path remains in the transitive closure data structure; forming feature intersections from clusters of features remaining in a reduced ontology graph resulting from said reducing said transitive closure data structure; configuring layers in a neural network to represent the feature intersections, the configured layers in the neural network causing the neural network to generate predictions that are within a defined accuracy range associated with the dataset; A method comprising:

2. 1. A processor-implemented method comprising: reducing the transitive closure data structure by removing a subset of the set of entries from the transitive closure data structure such that only a single entry corresponding to a single path remains in the transitive closure data structure, the transitive closure data structure including the set of entries corresponding to a set of paths in a graph of a vector space, the set of paths including all reachable paths in the graph between a first feature and a second feature in a pair of features, the pair of features corresponding to an input dataset; forming feature intersections from clusters of features remaining in a reduced ontology graph resulting from said reducing said transitive closure data structure; configuring layers in a neural network to represent the feature intersections, the configured layers in the neural network causing the neural network to generate predictions that are within a defined accuracy range associated with the dataset; A method comprising:

3. Computing for the layer an upper bound on a layer embedding corresponding to the feature intersection; and calculating for the layer a lower bound of the layer embedding corresponding to the feature intersection, wherein a layer size of the layer for the layer embedding is based on the upper bound and the lower bound.

4. The method of claim 3 , wherein the layer size is the calculated average of the upper and lower limits.

5. converting the dataset into a sparse matrix; constructing an ontology graph corresponding to the sparse matrix, the ontology graph including a set of vertices corresponding to a set of features in the dataset; performing a forward materialization on the graph, the forward materialization adding an inference vertex to the set of vertices, the set of vertices including the inference vertex being used in constructing the transitive closure data structure; The method of any one of claims 1 to 4, further comprising:

6. The method of claim 1 , wherein the single remaining path satisfies an efficiency requirement.

7. The method of claim 6 , wherein the efficiency requirement comprises a minimum distance between the first feature and the second feature.

8. The method of claim 6 , wherein the efficiency requirement comprises a maximum utilization ratio between the first feature and the second feature.

9. The method of claim 1 , wherein the reducing comprises a transitive reduction algorithm.

10. 10. The method of claim 1, wherein by said reducing the transitive closure data structure, a subset of vertices is removed from the ontology graph to form the reduced ontology graph.

11. further comprising identifying a plurality of clusters in the reduced ontology graph that include the cluster; 11. The method of claim 1, wherein the clusters comprise at least two features such that a combination of the two features can be used to distinguish at least a first portion of the dataset and a second portion of the dataset.

12. A computer program product causing a processor to carry out the steps of the method according to any one of claims 1 to 11.

13. 13. The computer program product of claim 12, wherein the computer program product is stored in a computer readable storage device in a data processing system, and wherein the computer program product is transported over a network from a remote data processing system.

14. the computer program being stored in a computer readable storage device in a server data processing system, the computer program being downloaded over a network to a remote data processing system for use in a computer readable storage device associated with the remote data processing system; The processor, Measuring the use of computer usable code associated with the request; and generating a bill based on said metered usage; and The computer program product of claim 12 , further comprising:

15. 1. A computer system comprising a processor, a computer readable memory, and a computer readable storage device, wherein program instructions are stored on the storage device for execution by the processor via the memory, the stored program instructions comprising: program instructions for constructing, with a processor and memory, a transitive closure data structure for a pair of features represented in a vector space corresponding to an input dataset, the transitive closure data structure including a set of entries corresponding to a set of paths in a graph of the vector space, the set of paths including all reachable paths in the graph between a first feature in the pair and a second feature in the pair; program instructions for reducing the transitive closure data structure by removing a subset of the set of entries from the transitive closure data structure such that only a single entry corresponding to a single path remains in the transitive closure data structure; program instructions for forming feature intersections from clusters of features remaining in a reduced ontology graph resulting from said reducing said transitive closure data structure; program instructions for configuring a layer in a neural network to represent the feature intersection, the configured layer in the neural network causing the neural network to generate predictions that are within a defined accuracy range associated with the dataset; 1. A computer system comprising:

16. 1. A computer system comprising a processor, a computer readable memory, and a computer readable storage device, wherein program instructions are stored on the storage device for execution by the processor via the memory, the stored program instructions comprising: program instructions for reducing a transitive closure data structure by removing a subset of a set of entries from the transitive closure data structure such that only a single entry corresponding to a single path remains in the transitive closure data structure, the transitive closure data structure including a set of entries corresponding to a set of paths in a graph of a vector space, the set of paths including all reachable paths in the graph between a first feature and a second feature in a pair of features, the pair of features corresponding to an input data set; program instructions for forming feature intersections from clusters of features remaining in a reduced ontology graph resulting from said reducing said transitive closure data structure; program instructions for configuring a layer in a neural network to represent the feature intersection, the configured layer in the neural network causing the neural network to generate predictions that are within a defined accuracy range associated with the dataset; 1. A computer system comprising:

Images