Augmenting decision trees to facilitate parallel inference processing

Abstract

Processing in a computing environment is facilitated by building an augmented decision tree from a source decision tree, where the building includes inserting one or more virtual decision nodes into the source decision tree to obtain the augmented decision tree. Decision node data and leaf node data of the augmented decision tree are determined and provided to an inference accelerator to facilitate accelerated processing of the augmented decision tree and thereby determine which of a plurality of leaf nodes of the augmented decision tree to select.

Description

TECHNICAL FIELD

[0001] The present invention relates to machine learning, and in particular to parallel inference processing in machine learning. BACKGROUND

[0002] Decision trees are a method of machine learning that is widely used in many disciplines. Inference performance of decision trees can be an important issue in production environments. Traditionally, decision tree inference processing is inherently sequential. Decision nodes are compared in a hierarchical order as prescribed by the decision tree structure. Decision nodes branch to other decision nodes based on the results of the comparisons, where the logic fetches the next node of the tree to be processed. These branches are skip sequential memory accesses, which can slow down the pipeline speed in a central processing unit due to memory access latency. This memory access pattern is specific to the decision tree structure. To improve performance, decision tree inference workloads can be offloaded to a peripheral computing device to reduce resource consumption on, for example, one or more central processing units (CPUs). SUMMARY

[0003] Certain deficiencies of the prior art are overcome and additional advantages are provided through the provision, in one or more aspects, of a computer system that facilitates processing in a computing environment. The computer system includes a memory and a processing circuit in communication with the memory and is configured to perform a method that includes establishing, by the processing circuit, an augmented decision tree from a source decision tree. The establishing includes inserting one or more virtual decision nodes into the source decision tree to obtain the augmented decision tree. Further, the method includes determining, by the processing circuit, decision node data and leaf node data of the augmented decision tree and providing, by the processing circuit, the decision node data and the leaf node data to an inference accelerator to facilitate accelerated processing of the augmented decision tree and thereby determine which of a plurality of leaf nodes of the augmented decision tree to select.

[0004] Advantageously, augmenting the decision tree enables the inference accelerator to more efficiently process the decision tree. In the augmented decision tree or fully populated decision tree, each leaf node has a fixed and known path through the decision nodes. The inference accelerator utilizes this knowledge to reduce the number of parameters that need to be transmitted to the inference accelerator, thus reducing the input / output of the inference accelerator, which reduces the turnaround time of the inference processing and frees up logical space for other work, thereby improving overall performance.

[0005] In one example, the source decision tree is a pruned decision tree, the decision node data includes data of the positioned decision nodes of the augmented decision tree, and the leaf node data includes data of the positioned leaf nodes of the augmented decision tree. Advantageously, the generalization benefits of the pruned source decision tree are preserved, while the inference accelerator is accelerated with the augmented or fully populated decision tree, which improves inference performance.

[0006] In one or more embodiments, the accelerated processing comprises processing that is accelerated by being processed in parallel by the plurality of leaf nodes based on respective predetermined path vectors of each leaf node of the plurality of leaf nodes through the augmented decision tree. Advantageously, the inference accelerator is configured to facilitate fully parallel processing by the plurality of leaf nodes of the decision node result vector based on respective predetermined path vectors through the decision tree to each leaf node of the plurality of leaf nodes. In this way, latency of the decision tree inference processing is reduced, improving processing throughput of the decision tree and / or collection of decision trees.

[0007] In one embodiment, the determining comprises, for a leaf node of the plurality of leaf nodes, descending the leaf node to a maximum depth of the augmented decision tree and adjusting a path vector of the leaf node based thereon to obtain a respective predetermined path vector of the leaf node. Descending each leaf node to the maximum depth of the augmented decision tree eliminates the requirement to transmit to the inference accelerator a path to each leaf node, thereby reducing input / output and configuration processing of the inference accelerator. Descending the leaf node to the maximum depth of the augmented decision tree results in a fully populated augmented decision tree, which means that there is no need to transmit to the accelerator one or more masks derived from the respective predetermined path vector for inference processing in the inference accelerator.

[0008] In one example, the adjusting comprises determining a string value to be inserted into the path vector based on a distance the leaf node descends on the descending and inserting the string value into the path vector to obtain the respective predetermined path vector of the leaf node. In one embodiment, the string value to be inserted is equivalent to a last entry in the fully populated path table of a size of the descended level.

[0009] In one example, the adjusting comprises determining an insertion position of the string value in the path vector by identifying a last decision node in the path vector and, based on the last decision node being true or the last decision node being at an end of the path, the insertion position being a position beyond the last decision node (1016). In another example, based on the last decision node being false and not being at an end of the path, the insertion position being two positions beyond the last decision node.

[0010] In one example, for at least one other leaf node of the plurality of leaf nodes, an "irrelevant" string is inserted into its path vector at the same insertion position, wherein the "irrelevant" string is of a same size as the determined string value.

[0011] Computer-implemented methods and computer program products related to one or more aspects are also described and claimed herein. In addition, services related to one or more aspects are also described and can be claimed herein.

[0012] Other features and advantages will be apparent from the following detailed description, from the drawings, and from the claims. BRIEF DESCRIPTION OF DRAWINGS

[0013] One or more aspects of the application are set forth in particularity and claimed herein below in the claims. The above and other objects, features and advantages of the present application will become apparent from the following detailed description, which is to be read in connection with the accompanying drawings, in which:

[0014] Figure 1 One embodiment of a computing environment process for accelerated inference of a decision tree and a set of decision trees in accordance with one or more aspects of the present application is described;

[0015] Figure 2A One embodiment of accelerated inference of a decision tree in accordance with one or more aspects of the present application is described;

[0016] Figure 2B One embodiment of accelerated inference of a set of decision trees in accordance with one or more aspects of the present application is described;

[0017] Figure 3A One embodiment of a decision tree for which a leaf node path vector is to be determined in accordance with one or more aspects of the present application is described;

[0018] Figure 3B One embodiment of a predetermined leaf node path vector for a decision tree embodiment of Figure 3A in accordance with one or more aspects of the present application is described;

[0019] Figure 4A One embodiment of a leaf node mask derived from a predetermined leaf node path vector in accordance with one or more aspects of the present application is described;

[0020] Figure 4B One embodiment of a data structure for accelerated inference by a decision tree leaf node in accordance with one or more aspects of the present application is described;

[0021] Figure 5A and 5B Different logical embodiments for accelerated inference by a decision tree leaf node in accordance with one or more aspects of the present application are described;

[0022] Figure 6 An example of accelerated inference by a decision tree leaf node in accordance with one or more aspects of the present application is described, illustrating two leaf nodes, one selected and one not selected;

[0023] Figures 7A-7COne embodiment of a workflow illustrating certain aspects of one or more embodiments of the application is described;

[0024] Figure 8A One example of a pruned decision tree to be augmented in accordance with one or more aspects of the application is described;

[0025] Figure 8B One example of a fully populated decision node tree augmented from Figure 8A a pruned decision tree in accordance with one or more aspects of the application is described;

[0026] Figure 9A One example of decision node data and leaf node data obtained for Figure 8A a pruned decision tree that is adjusted for augmentation of the decision tree by insertion of one or more virtual decision nodes into the pruned decision tree in accordance with one or more aspects of the application is described;

[0027] Figure 9B One example of decision node data and leaf node data obtained for Figure 8B an augmented (or fully populated decision node) tree in accordance with one or more aspects of the application is described;

[0028] Figures 10A-10B One embodiment of a workflow illustrating certain aspects of one or more embodiments of the application is described;

[0029] Figure 11 One example of a computing environment that includes and uses one or more aspects of the application is described;

[0030] Figure 12A Another example of a computing environment that includes and uses one or more aspects of the application is described;

[0031] Figure 12B One example of further details of a memory of Figure 12A in accordance with one or more aspects of the application is described;

[0032] Figure 13 Another example of a cloud computing environment in accordance with one or more aspects of the application is described;

[0033] Figure 14 One example of an abstraction model layer in accordance with one or more aspects of the application is described. DETAILED DESCRIPTION

[0034] Like reference numbers in the drawings indicate like or functionally similar elements, and the figures accompany the detailed description, are part of the detailed description, and illustrate various aspects of the present application, and together with the detailed description, serve to explain various aspects of the present application. In this regard, it is noted that the description of known systems, devices, accelerators, processing techniques, etc. has been omitted so as not to obscure the disclosure of the present application. It is noted, however, that the detailed description and specific examples, while indicating aspects of the application, are given by way of illustration only, and are not by way of limitation. Various substitutions, modifications, additions and / or arrangements, within the spirit and scope of the underlying inventive concepts, are possible. Further, it is noted that numerous aspects and features of the present application are disclosed herein, and each disclosed aspect or feature can be combined with any other disclosed aspect or feature, unless otherwise contraindicated.

[0035] It is also noted that the examples herein are described using specific code, designs, architectures, accelerators, protocols, layouts, diagrams or tools, by way of example only and not by way of limitation. Moreover, in certain instances, specific hardware, software, tools or data processing environments are used in the description of the examples merely for

[0036] As understood by those skilled in the art, program code referred to in this application can include hardware and software. For example, program code in certain embodiments of the application can include fixed function hardware, but other embodiments can utilize software-based implementations of the described functionality. Certain embodiments incorporate both of these types of program code. One example of program code, also referred to as one or more programs or program instructions, is described in Figure 11

[0037] ​As previously noted, decision tree inference processing generally has inherent sequentiality, with decision node comparisons generally tested in a hierarchical order as dictated by the decision tree structure. Decision nodes branch to other decision nodes based on the results of the comparisons, with the logic fetching the next node to process from there. These branches are out-of-order memory accesses, which can slow down the pipeline in the CPU due to memory access latency. This memory access pattern is specific to decision tree structures.

[0038] By contrast, decision tree inference processing, as with other machine learning techniques, provides full parallelization of processing time in clock cycles rather than milliseconds, with an inference accelerator such as disclosed herein. In one embodiment, parallelism over the range of segments of a decision tree set is disclosed, with all decision nodes processed in parallel at the same time, and all leaf nodes processed in parallel at the same time, for example as described herein. Parallel results can be aggregated with a parallel adder or adder tree. In addition, library support can be provided to interface with the accelerator, and processing can be performed over multiple iterations if demand exceeds the accelerator supply of any internal resources, as described herein. In addition, in one embodiment, a decision tree can be augmented as necessary to conform to a fully populated form, depending on the specific implementation. Decision trees can be converted to the format required by the inference accelerator, and decision trees or sets of decision trees can be divided into different processing segments as necessary. For example, as previously noted, all decision nodes can be processed in parallel by the accelerator, followed by parallel processing of all leaf nodes.

[0039] In one embodiment, a decision tree set can include multiple independent decision trees, each tree being a weak learner. Decision tree results can be weighted, for example, amplifying signals or values from accurate trees or attenuating information or values from inaccurate trees. For example, a random forest is a machine learning model based on a set of decision trees.

[0040] Figure 1One embodiment of a computing environment process 100 according to one or more aspects of the present disclosure is described. By way of example, the computing environment process 100 illustrates one embodiment of an overall process that facilitates accelerated decision tree inference as disclosed herein, which in one embodiment can be embedded or resident in an inference tree inference accelerator 140. As shown, training data 101 is used to train the inference accelerator to generate, in one embodiment, a trained machine learning model 111. Decision tree or set of decision trees training 110 can use an existing software process such as SKLearn (i.e., a machine learning library for the Python™ programming language) to develop the decision tree and save in a software determined format. The trained machine learning model 111 is then compiled 120, which in one embodiment is performed by a specialized compiler that converts or formats the decision tree or set of decision trees into a form usable by the inference accelerator 140. For example, in one implementation, the compilation process converts the SKLearn decision tree (or set of decision trees) into a format required by the inference accelerator. Further, the compilation process can include, for example, expanding a pruned tree (as described herein) into a fully populated tree of a specified depth. In one or more embodiments, a list of decision nodes and leaf nodes is generated for transfer to the inference accelerator. In one embodiment, the machine learning model 121 prepared for processing by the inference accelerator 140 can be saved into an inference library 130 for invocation by a user program 131 to drive the inference accelerator 140, e.g., to send an input data vector and the trained decision tree to the inference accelerator. In one embodiment, the library interface responds to the caller's request, performs data conversion to provide to the inference accelerator 140, sends the decision nodes and input data vector to the inference accelerator, and waits for the output result.

[0041] In one or more embodiments, the inference accelerator can be installed outside of the central processing unit (CPU), and, in one or more embodiments, it can be a multi-core inference engine configured to read the decision nodes, fetch data from the caller's memory, and perform processing on the decision tree (or set of decision trees), including processing the decision nodes in parallel or concurrently, in one embodiment, and processing the leaf nodes in parallel, respectively, for example, with bit-level parallelism. Further, in one embodiment, a summer or tree of summers can be provided to accumulate the output values in parallel, the output of the inference accelerator being returned to the caller (e.g., a user program). In another embodiment, the encoding of the leaf node selection process representing the indices of the selected leaf nodes can be returned to the calling process for further processing. This would eliminate the need for a summer in the inference accelerator and help reduce the amount of information that needs to be transferred to the inference accelerator. Thus, the embodiments shown and described herein with respect to containing one or more summers or tree of summers are merely one example of an inference accelerator with the enhancements described for decision tree leaf node processing. Figure 2A and 2B A timeline of the concurrent process implemented by the inference accelerator as disclosed herein (illustrated by way of example only as having one or more output summers or tree of summers) in one or more embodiments is described.

[0042] Referring to Figure 2A , a timeline of the concurrent steps of one embodiment of the inference accelerator is shown, where an input data vector 200 is provided to the decision nodes 201 of a decision tree for processing, for example, in parallel, to produce a decision node result vector 202, which is provided as input to the leaf node processing, which, as described herein, uses (in one embodiment) a plurality of masks (e.g., M1 i , M2 i ) for parallel processing of the leaf nodes. The parallel processing of the leaf nodes using the masks produces data provided to, for example, a selection register 204, which is fed to an encoder 205 to obtain the indices of the selected leaf node values 206, the applicable leaf node values being the output results fed (in one embodiment) to a summer 207. Alternatively, as previously described, the indices of the selected leaf node values 206 can be the output results.

[0043] As a further example, Figure 2B A timeline of the concurrent steps shown in Figure 2A is described for a set of decision trees, where multiple decision trees are processed in parallel, the summer tree 207 being expanded to sum the leaf node value outputs (in one embodiment).

[0044] In one or more implementations, a decision tree comprises a hierarchical structure of decision nodes, where leaf nodes are located at the end of each path traversing the tree. Each decision node can compare a variable with a constant value, generating a Boolean value that is either true or false. Each decision node includes two edges representing paths to the next node in the tree, which can be another decision node or a leaf node. Reaching a leaf node based on the test results of the decision node is called selecting that leaf node. Each leaf node has one or more associated values, which, once selected, are output as the result of the decision tree.

[0045] This paper discloses an inference accelerator that uses the path traversing each of the multiple leaf nodes of a decision tree to infer whether to select that leaf node. In one embodiment, all decision nodes of the decision tree are first compared to obtain a decision node result vector by the inference accelerator. In one example, the decision nodes can be processed in parallel, although this is not required.

[0046] In one or more embodiments, the inference accelerator uses two masks derived from the path to each leaf node. These masks are used to determine whether the leaf node is a selected leaf node in the set of leaf nodes of the decision tree. For any given input data vector, inference processing is used to select only one leaf node for the decision tree. In this way, the inference accelerator can process the entire set of leaf nodes of the decision tree in parallel. Advantageously, this approach reduces the latency of decision tree inference and provides fast turnaround of the decision tree or set of decision trees, thereby improving processing throughput.

[0047] like Figure 3A As shown in the example of a decision tree, the decision tree includes two types of nodes: decision nodes 300 and leaf nodes 301. Each decision node 300 performs a comparison (true or false) to determine which branch to take. Decision nodes 300 are arranged in a hierarchical structure and are selected based on the decisions of higher-level nodes (except the root decision node). Leaf nodes 301 are located at the ends of the paths traversing the tree. For the input data vector during the inference process, the path followed traversing the tree is different, and the test results of the decision nodes determine the path. Once a leaf node is reached, the selected leaf node will indicate which value to process from the decision tree.

[0048] The depth of leaf node 301 is the distance from the root decision node to the leaf node. Figure 3A An example decision tree with depth = 3 is shown, where decision nodes are numbered in depth-first order, as are leaf nodes (in one example). Figure 3A In this context, leaf nodes are denoted by the prefix "L". It is assumed that the branch associated with the comparison result "true" is on the left, while the branch associated with "false" is on the right.

[0049] Decision nodes 300 are ordered in a depth-first order, which is a common technique for ordering the decision nodes of a decision tree. Leaf nodes are also ordered in a depth-first manner, although it can also be seen that they are ordered from left to right.

[0050] All leaf nodes 301 of a fully populated decision tree are the same distance from the root decision node. The inference accelerator disclosed herein does not require that the decision tree be in a fully populated form; for illustrative purposes only, Figure 3A is shown as fully populated.

[0051] The leaf node distance is the number of edges between the root decision node and a leaf node, referred to as the depth. For a given depth of a fully populated tree, the number of leaf nodes is two raised to the power of the depth, and the number of decision nodes minus one. For example, a fully populated tree of depth 3 has 2 3 or 8 leaf nodes and 7 decision nodes.

[0052] As previously mentioned, a decision tree has been trained and is used, such as by the disclosed inference accelerator. In one embodiment, the trained decision tree is subjected to a compilation process that extracts and / or formats information needed by the inference accelerator.

[0053] In one or more implementations, a path through the decision nodes is identified for each leaf node of the decision tree, and two masks are derived for each leaf node from the respective path. Figure 3B The path through Figure 3A the example set of decision nodes to each leaf node 301 is depicted. Each leaf node indicates whether a decision node is “true” (1) or “false” (0). If a decision node is not part of a leaf node path, it is represented by a “.”, as Figure 3B shown.

[0054] The two masks (referred to herein as masks M1, M2) can be derived from the path vector, such as from the vector shown in Figure 3B As explained further below, the two masks are used by the accelerator to provide parallelization opportunities for analyzing leaf nodes. One of the masks (M1) is used to identify which decision nodes in the path vector took the “false” branch, and is represented by a “1” in the relevant position, and “0” for all other positions. The other mask (M2) identifies which decision nodes in the path were used (whether “true” or “false”), with the decision node position shown as a “1”, and “0” for all other positions. Figure 4A An example of one mask M1 and the other mask M2 obtained or derived from the respective path vector for each leaf node is shown with Figure 3B the example.

[0055] In processing a decision tree, each decision node is provided a Boolean indicator that indicates whether the comparison result was true or false. The collection of decision node indicators for a decision tree is the decision node result vector (RV) in Figure 2A and 2B The decision node result vector length (in bits) is equal to the number of decision nodes in the tree, one bit for each decision node, and follows the order of the decision nodes.

[0056] In one solution for decision tree processing, it is possible to reach each leaf node with the complete combination of decision node results and store it in an internal table. However, the number of combinations explodes because there are 2 7 or 128 combinations. For a tree of depth 4, there are 2 15 or 32K combinations; for a tree of depth 5, there are 2 31 or 2G combinations. In contrast, using the predetermined path vector technique described herein, for a decision tree of depth 3, there are 8 leaf nodes, each with 3 decision nodes in its path. For a tree of depth 4, there are 16 leaf nodes, each with 4 decision nodes in its path; for a tree of depth 5, there are 32 leaf nodes, each with 5 decision nodes in its path.

[0057] In one or more embodiments of the parallel inference processing disclosed herein, each leaf node processing is provided with a copy of the decision node processing set's decision node result vector with its own decision node result vector. In one implementation, one mask Ml is applied to the decision node result vector by a bitwise exclusive OR (XOR) and another mask M2 is applied to the updated result vector by a bitwise logical AND (AND). The result of this operation is then compared to another mask M2 with a bitwise exclusive NOR (XNOR). A reduction AND can then be applied to all bits of the result vector to obtain a single result. This single result (in one embodiment) is a single indicator that indicates whether or not the leaf node was selected. If all bits in the leaf node inference result vector are binary ones, the combination is "true" and the leaf node is selected; otherwise, the result is "false" and the leaf node is not selected. For any given input data vector, only one leaf node of the decision tree is selected.

[0058] As shown in Figure 4B , this process can be represented by a truth data structure that shows the application of two masks Ml i and M2 i to the result vector bits RV iThe result of this is that, by definition, two of the conditions are impossible, denoted by "--", because if a node is in the path of a "false" branch in mask Ml, it must be a taken node (i.e., must be "1" in mask M2). Furthermore, if both mask Ml and mask M2 are "0", the path of the decision node to the leaf node is not significant, and the conditions do not prevent selection of the leaf node.

[0059] As noted above, each leaf node inference process is independent of the other leaf nodes, so the leaf nodes can be easily processed in parallel on the decision node result vector. More specifically, each bit of the result vector input to a leaf node, and the position matching bit in both masks, is independent of all other bits processed for that leaf node.

[0060] For a given data input vector, only one leaf node will be activated, and the set of activated bits is a one-hot vector (all bits are "0", except for the activated leaf node, which is "1"). The set of activated bits can be passed to an encoder that converts the result to a numerical index equivalent to the position of the leaf node in the tree, as shown in Figure 2A and 2B .

[0061] Figure 5A One embodiment of leaf node logic 500 of inference accelerator 140 is described, which is used to perform accelerated inference by or for a leaf node. In one embodiment, each leaf node process is independent of all other leaf nodes, and the leaf node processing is performed in parallel within inference accelerator 140. In one embodiment, the accelerator includes an inference process for each leaf node that determines whether it is the selected leaf node. The process includes taking the decision node result vector (RV DN ) as a register, and applying two masks to determine whether the leaf node is the selected leaf node. As noted previously, given an input vector, only one leaf node of the decision tree is selected. During processing, a mask (Ml LNi ) is used to identify which decision nodes in the path of the leaf node take the "false" branch, where a mask Ml is applied to the decision node result vector using "exclusive or" (XOR) logic 501 (in one embodiment). Another mask (M2 LNiThe decision node in the leaf node path is identified by a 502 AND logic. Specifically, another mask M2 for the leaf node is applied to the updated result vector (generated by the XOR logic 501) using AND logic 502 to generate a further updated result vector. This further updated result vector is compared with another mask M2 for the leaf node using exclusive NOR logic 503. A reduced AND 504 is then applied to the result array to indicate whether a leaf node has been selected. If the leaf node result vector matches the corresponding mask M2 value, the leaf node LN is selected. Y .

[0062] Figure 5B An inference accelerator 140 with leaf node logic 500' according to one or more aspects of the present invention is described. In this optional sequence, a corresponding mask M1 is applied to the decision node result vector using XOR logic 501, and another mask M2 is applied to the updated result vector using AND logic 502, as follows. Figure 5A As shown in the leaf node logic embodiment. Figure 5B In the embodiment, the further updated result vector is then compared with the mask M2 using XOR logic 505 to determine whether a leaf node has been selected. Then... Figure 5B The example uses reduction logic, such as a NOR 506, to obtain an output LN indicating whether the leaf node was selected based on the path traversed from the decision node. Y .

[0063] Figure 6 The description of leaf node inference processing, such as Figure 3A and 3B The diagram shows an example of the sample result vector of a fully filled depth-3 decision tree. For illustrative purposes only, assume that the decision nodes arbitrarily output a result vector of 1010101. The diagram shows the path vectors and masks M1 and M2 for the 4th and 7th leaf nodes. Figure 6 In the embodiments described above, the above-described and the above-explained methods are used. Figure 5A The leaf node logic is applied. In this example, leaf node L4 is selected, while leaf node L7 is not. For the result vector 1010101, the one-hot result for L1 to L8 is 00010000, where only L4 is selected. This is called the selection vector. The inference accelerator then passes this selection vector to the encoder, which produces an index of 3 that points to L4 in zero-based encoding. By using this index, the inference accelerator can combine... Figures 2A-2B Locate the leaf node value in the stored leaf node value table shown.

[0064] Figures 7A-10BIt is described that the inference accelerator processing is further enhanced by converting the source decision tree to be processed to a fully populated format, in accordance with one or more aspects described herein.

[0065] In general, a process is disclosed that improves the overall performance of the decision tree inference processing by the inference accelerator by changing the structure of the source decision tree. As described above, in one or more implementations, the inference accelerator is an adjunct to a general purpose processor, such as a central processing unit (CPU). Advantageously, in one or more embodiments, the structure of the source decision tree is changed to conform to the expectations of the inference accelerator. This advantageously reduces the input / output across accelerator boundaries, further speeding up the overall inference process. By limiting the processing to a fully populated or augmented decision tree, the masks (M1, M2) described above do not need to be transmitted to the inference accelerator. Note that an augmented decision tree or a fully populated tree refers here to a decision tree where, for a given depth, all decision nodes of the decision tree are occupied. More specifically, in one or more embodiments, an augmented or fully populated decision tree refers to a pruned decision tree that has been augmented to be complete. For example, bypassed decision nodes are inserted as virtual decision nodes and are "invalid" in the resulting decision tree definition object. Missing leaf nodes are inserted as "non-existent" in the leaf node portion of the decision tree definition. Once the decision tree conforms to the fully populated form, the leaf node paths will take the default values for the given depth of the decision tree. Thus, the masks used by the inference accelerator are fixed and can be embedded into the accelerator. Advantageously, with the fixed and known tree structure, the inference process does not need to include the tree structure in the inference transmission to the peripheral inference accelerator. Moreover, since there is no need to respond to a variable structure, the turnaround time is reduced, improving the overall decision tree inference processing. Furthermore, additional processing steps that would otherwise be needed are avoided, freeing up logical space for other work.

[0066] A trained decision tree is typically not fully populated to the maximum depth of the construction. During training, the splits determined in the data and embedded in the decision nodes can result in many different forms of decision tree structures. Moreover, many decision tree training algorithms use pruning, i.e., the removal of decision nodes from the trained tree, to reduce overfitting and improve generalization. This can be beneficial to the outcome of the inference process of the decision tree. This process results in most of the decision trees to be processed (i.e., the source decision tree) will not conform to the expectations of the corresponding, fully populated decision tree structure (e.g., the structure of depth = 3 as shown in Figure 3A

[0067] ​Advantageously, disclosed herein is a computer system, a computer-implemented method, and a computer program product having program code that, when executed, performs a method that includes establishing, by processing circuitry, an augmented decision tree from a source decision tree, where the establishing includes inserting one or more virtual decision nodes into the source decision tree to obtain the augmented decision tree. The decision node data and leaf node data of the augmented decision tree are determined and provided to an inference accelerator to facilitate accelerated processing of the augmented decision tree, and from which it is determined which of a plurality of leaf nodes of the augmented decision tree to select, as described herein. Advantageously, the augmentation of the source decision tree causes the tree to take the form of a fully-populated tree of a selected depth. This enables the inference accelerator to more efficiently process the pruned or otherwise shaped source decision tree. In this way, the augmentation of the decision tree is part of the overall process of accelerating the processing of the decision tree inference, and can be (in one embodiment) embedded, for example, in or in association with a compilation process (such as the compilation process 120 of Figure 1 As described above, the source decision tree or decision tree ensemble is a trained decision tree that has been processed or formatted by the compiler described above for use with an inference accelerator (e.g., a multi-core inference engine).

[0068] According to one or more aspects, program code is provided for augmenting a decision tree (i.e., a source decision tree) so that it matches the corresponding, fully-populated decision tree format expected by an inference accelerator. As part of the augmentation process, one or more virtual decision nodes are inserted into the source decision tree structure to represent missing decision nodes. The virtual decision nodes only produce spurious comparison results. In this way, leaf nodes will be moved to appropriate locations as a result of the insertion of the decision nodes. For example, the restructuring of the decision tree advantageously enables, for example, the reuse of leaf node path knowledge in the associated inference accelerator, thereby advantageously reducing the amount of communication between the CPU and the inference accelerator.

[0069] Once the decision tree is augmented to be a fully-populated decision tree of a given depth, there is one decision node for each decision position of the tree. When leaf nodes are moved to accommodate the restructuring, empty leaf node positions can arise. The now-missing leaf node positions are unreachable and are therefore ignored by the parallel inference processing described herein. In a fully-populated or augmented decision tree, each leaf node has a fixed and known path through the decision nodes. The inference accelerator takes advantage of this knowledge to reduce the number of parameters that need to be transmitted to the inference accelerator, thereby improving overall performance by reducing the input / output of the peripheral inference accelerator and freeing up logical space for other work. In this way, the generalization benefits of a pruned decision tree are preserved, while also obtaining the performance improvements of using the fixed structure of a fully-populated augmented decision tree, as described herein.

[0070] For a given trained decision tree or source decision tree and a target depth of the tree, the program code will (in one embodiment) first determine if any virtual decision nodes need to be inserted into the tree structure. For a fully populated tree of a given depth, it is known how many decision nodes and leaf nodes there are. The number of leaf nodes for a fully populated or augmented tree of any given tree depth is 2 raised to the number of tree depth, and the number of decision nodes is one less than the number of leaf nodes. Thus, for a tree of depth 3, there are 8 leaf nodes and 7 decision nodes. To reach a leaf node, control needs to pass through the same number of decision nodes as the depth.

[0071] Figure 7A One embodiment of the program code processing for augmenting a source decision tree and obtaining an augmented (or fully populated) decision tree as described herein to adjust leaf node tree depth positions is described. As shown in FIG. 7, Figure 7A The process 700 includes obtaining a next leaf node from the source decision tree (702), obtaining a leaf node path through the decision nodes to the leaf node (704). The value associated with the leaf node is obtained (706), and then the process determines if there are other leaf nodes (708). If so, the process returns to obtain a next leaf node from the source decision tree. As shown in FIG. 7, Figure 7A The process determines a fully populated decision node structure for the specified tree depth as a template (710), and for each leaf node (712), determines if the leaf node is at the maximum tree depth (714). If no, the leaf node position is lowered to the maximum depth position (716) to facilitate obtaining the data for the augmented decision tree’s adjusted position decision nodes and adjusted position leaf nodes to be stored in, for example, Figure 1 the inference library / database 130 for forwarding to the inference accelerator 140 upon invocation. Once all leaf nodes have been processed (718), the processing of the source decision tree is complete (720). Figure 8A and 8B One example of this process is described.

[0072] As described above, Figure 3A a fully populated decision tree of depth 3 is shown, Figure 3B a path vector is shown for this fully populated tree of depth 3. The path vector specifies the decision nodes, and thus the path to each leaf node. Each decision node position of the leaf’s path is marked with a 1 or 0, indicating a true or false decision, and the decision nodes not in the path are indicated with a dot, indicating they are irrelevant “don’t cares.”

[0073] Figure 8A An example of a pruned source decision tree of maximum depth 3 is described. The decision nodes of this tree are to be augmented to conform to the augmented decision tree as described in FIG. 7, Figure 3AThe fully populated decision tree structure is shown with a depth of 3. If any leaf node does not have the required number of decision nodes in its path, the leaf node needs to descend in the tree structure. In this example, leaf node LI needs to descend 2 levels, leaf node L2 needs to descend 1 level, and leaf nodes L3 and L4 are at the lowest level and do not need to descend. Figure 8A The current respective paths to each leaf node are also shown in this example.

[0074] Figure 8B One embodiment of an augmented decision tree obtained from the pruned source decision tree of Figure 8A is depicted. In this embodiment, virtual decision nodes have been inserted and leaf nodes have been moved to reflect the altered structure. In this example, leaf node LI is relocated as leaf node L4, leaf node L2 is relocated as leaf node L6, and leaf nodes L3 and L4 are relocated as leaf nodes L7 and L8. In this way, the location of the data to be transmitted to the inference accelerator is changed to be consistent with the format of the path vectors of Figure 3B .

[0075] Figure 9A and 9B A specific example is described to illustrate the comparisons associated with the decision nodes, the values associated with the leaf nodes, and their relocated positions in the list based on the augmented decision tree. In Figure 9A , the comparisons for the individual decision nodes 1-3 are illustrated, as are the corresponding leaf node values. In Figure 9B , using the standard decision node and leaf node notation, decision nodes 1-3 have become decision nodes 1, 5, and 7 in the expanded tree, with the corresponding comparisons remaining the same; leaf nodes 1-4 have become leaf nodes 4 and 6-8, with the values remaining as they were initially shown in Figure 9A . In addition, Figure 8B the pruned path data of Figure 9B is repeated for the relocated leaf nodes of the augmented decision tree.

[0076] Figure 7B and 7C One embodiment of a process implemented by program code for descending one or more leaf nodes to the maximum depth of a decision tree structure is described. Referring first to Figure 7B , the process obtains the insertion position for each leaf node that needs to descend to the maximum depth (730). Figure 7COne embodiment of this is described in FIG. 7. As shown, the process locates the last decision node in the leaf node path vector (740), determines whether the last decision node is true or the last decision node is at the end of the path (742). If "yes," the insertion position is one position beyond the last decision node (744). Otherwise, the insertion position is two positions beyond the last decision node (746).

[0077] As shown in FIG. 7, the string value to be inserted into the leaf node path vector is determined based on the distance to be descended (732). The insertion string is equivalent to the last entry in a fully populated path table of that size. For example, for a descent of 2 levels, the insertion string in a table of maximum depth = 2 is "0.0." For a descent of 3 levels, the insertion string would be the last entry in a path table of maximum depth = 3, e.g., "0...0.0." Figure 7B

[0078] The string value is inserted into the path vector for that leaf node at the insertion position (734), and for other leaf nodes in the tree, an "irrelevant" string of the same size is inserted at the same position (736). For example, for a descent of 2 levels, the "irrelevant" string is "...". Once the string values have been inserted to adjust or normalize the leaf node path vectors, the process returns to the flow of FIG. 6 to determine that all leaf nodes have been processed. Figure 7A

[0079] Further details regarding one embodiment involving one or more aspects of the present invention that facilitate processing in a computing environment are described with reference to Figures 10A-10B

[0080] Further details regarding one embodiment involving one or more aspects of the present invention that facilitate processing in a computing environment are described with reference to Figure 10A In one embodiment, the processing circuitry establishes an augmented decision tree from a source decision tree, where the establishing includes inserting one or more virtual decision nodes into the source decision tree to obtain the augmented decision tree (1000). The processing circuitry determines decision node data and leaf node data for the augmented decision tree (1002), and provides the decision node data and the leaf node data to an inference accelerator to facilitate accelerated processing of the augmented decision tree, and thereby determine which of a plurality of leaf nodes in the augmented decision tree to select (1004). Advantageously, augmenting the decision tree enables the inference accelerator to more efficiently process the decision tree. In the augmented decision tree or fully populated decision tree, each leaf node has a fixed and known path through the decision nodes. The inference accelerator utilizes this knowledge to reduce the number of parameters that need to be transmitted to the inference accelerator, thus reducing the input / output of the inference accelerator, which reduces the turnaround time of the inference processing and frees up logical space for other work, thereby improving overall performance.

[0081] ​​​In one embodiment, the source decision tree is a pruned decision tree, and the decision node data includes data of the repositioned decision nodes of the expanded decision tree, while the leaf node data includes data of the repositioned leaf nodes of the expanded decision tree (1006). Advantageously, the generalization benefits of the pruned source decision tree are preserved, and the inference accelerator improves inference performance by using the expanded or fully filled decision tree for accelerated processing.

[0082] In one example, the accelerated processing includes processing (1008) that is accelerated by processing the decision node result vector in parallel by the plurality of leaf nodes based on the corresponding predetermined path vector of each of the plurality of leaf nodes through the extended decision tree. Advantageously, the inference accelerator is configured to facilitate fully parallel processing of the decision node result vector by the plurality of leaf nodes based on the corresponding predetermined path vector of each of the plurality of leaf nodes through the decision tree. In this way, the latency of decision tree inference processing is reduced, and the processing throughput of the decision tree and / or the set of decision trees is improved.

[0083] In one embodiment, the determination includes, for each of the plurality of leaf nodes, descending the leaf node to the maximum depth of the expanded decision tree and adjusting the path vector of the leaf node accordingly to obtain a corresponding predetermined path vector for the leaf node (1010). Descending each leaf node to the maximum depth of the expanded decision tree eliminates the requirement to transmit the path to each leaf node to the inference accelerator, thereby reducing the input / output and configuration processing of the inference accelerator. Descending the leaf nodes to the maximum depth of the expanded decision tree results in a fully filled expanded decision tree, meaning that it is not necessary to transmit one or more masks derived from the corresponding predetermined path vectors for inference processing in the inference accelerator to the accelerator.

[0084] like Figure 10B As shown, in one example, the adjustment includes determining a string value to be inserted into the path vector based on the distance the leaf node descends as described in the descent, and inserting the string value into the path vector to obtain a corresponding predetermined path vector for the leaf node (1012). In one embodiment, the string value to be inserted corresponds to the last entry of the descending layer size in a fully filled path table (1014).

[0085] In one example, the adjustment involves determining the insertion position of the string value by identifying the last decision node in the path vector, and, based on the last decision node being true or located at the end of the path, the insertion position is one position other than the last decision node (1016). In another example, based on the last decision node being false and not located at the end of the path, the insertion position is two positions other than the last decision node (1018).

[0086] In one example, for at least one other leaf node of the plurality of leaf nodes, a "not relevant" string is inserted in the same insertion position in its path vector, where the "not relevant" string has a size that is the same as the size of the determined string value (1020).

[0087] Those skilled in the art will note from the description provided herein that a decision tree inference accelerator is provided that obtains a decision node result vector based on processing an input data vector through a decision node of an augmented decision tree in one embodiment. In one embodiment, the inference accelerator accelerates leaf node processing of the decision node result vector and facilitates processing of the decision node result vector by a plurality of leaf nodes in parallel based on a respective predetermined path vector through the augmented decision tree for each leaf node of the plurality of leaf nodes, thereby determining which leaf node of the plurality of leaf nodes to select. In this way, the inference accelerator enhances processing within a computing environment, for example, by processing the decision node result vector by the plurality of leaf nodes in parallel based on the respective predetermined path vector through the augmented decision tree to each leaf node. As a result, latency of decision tree inference processing is reduced, and processing throughput of a decision tree and / or a collection of decision trees is improved.

[0088] Other variants and embodiments are possible as well.

[0089] The inference accelerator(s) of one or more aspects of the present disclosure can be employed in a number of computing environments. With reference to Figure 11 , an example of a computing environment is described. For example, the computing environment is based on the z / Architecture® hardware architecture provided by International Business Machines Corporation of Armonk, New York, USA. The z / Architecture hardware architecture is just one example architecture, however. The computing environment can also be based on other architectures, including but not limited to the Intel x86 architecture, other architectures of International Business Machines Corporation, and / or architectures of other companies.

[0090] As shown in Figure 11 , the computing environment 1100 includes a computer system 1102, for example, in the form of a general purpose computing device as shown. The computer system 1102 can include, but is not limited to, one or more processors or processing units 1104 (e.g., central processing units (CPUs)), memory 1106 (e.g., system memory, main memory, main storage, central storage, or storage), and one or more input / output (I / O) interfaces 1108 coupled to each other by one or more buses and / or other connections. For example, the processor 1104, along with one or more inference accelerators 1121 and the memory 1106, are coupled to the I / O interface 1108 by one or more buses 1110, and the processor 1104 and the accelerator 1121 are coupled to each other by one or more buses 1111.

[0091] For instance, bus 1111 is a memory or cache coherency bus, and bus 1110 is representative of one or more of a variety of bus structures, including a memory bus or memory controller, a peripheral bus, an accelerated graphics port, and a processor or local bus using a variety of bus structures. Such architectures include, for example, without limitation, the Industry Standard Architecture (ISA), the Micro Channel Architecture (MCA), the Enhanced ISA (EISA), the Video Electronics Standards Association (VESA) local bus, and the Peripheral Component Interconnect (PCI).

[0092] For instance, one or more special purpose processors (e.g., auxiliary processors) can be separate from, but coupled to, one or more general purpose processors, and / or can be embedded therein. Many variations are possible.

[0093] For instance, memory 1106 can include a cache 1112, such as a shared cache, which can be coupled to a local cache 1114 of processor 1104 and / or an auxiliary processor 1121 by way of, for example, one or more buses 1111. Moreover, memory 1106 can include one or more programs or application programs 1116, at least one operating system 1118, one or more computer readable program instructions 1120, and one or more inference libraries 1122. Computer readable program instructions 1120 and inference libraries 1122 can be configured to perform or facilitate the functions of embodiments of aspects of the present application.

[0094] For instance, in one embodiment, memory 1106 (e.g., at least one hardware system region of memory 1106) is coupled to one or more inference accelerators 1121 by way of one or more buses 1111.

[0095] Computer system 1102 can communicate with one or more external devices 1130 such as a user terminal, a magnetic disc drive, a pointing device, a display, one or more data storage devices 1134, etc. through I / O interface 1108. Data storage device(s) 1134 can store one or more programs 1136, one or more computer readable program instructions 1138, and / or data, etc. Computer readable program instructions can be configured to perform the functions of embodiments of aspects of the present application.

[0096] Computer system 1102 can also communicate with a network interface 1132 through, for example, I / O interface 1108, which enables communication with one or more networks such as a Local Area Network (LAN), a General Area Network (WAN), and / or a public network such as the Internet, and which provides communication over the one or more networks between computer system 1102 and other computing devices or systems.

[0097] The computer system 1102 can include and / or be coupled to removable / non-removable, volatile / non-volatile computer system storage media. For example, it can include and / or be coupled to a non-removable, non-volatile magnetic media (typically called a "hard drive"), a magnetic disk drive for reading and writing removable, non-volatile magnetic disk (e.g., a "floppy disk"), and / or an optical disk drive for reading and / or writing removable, non-volatile optical disk (such as a CD-ROM, DVD-ROM or other optical disk media). It should be understood that other hardware and / or software components could be used in conjunction with the computer system 1102. Examples include, but are not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data archival storage systems, etc.

[0098] The computer system 1102 can be operational with numerous other general purpose or special purpose computing system environments or configurations. Examples of well- known computing systems, environments, and / or configurations that can be suitable for use with computer system 1102 include, but are not limited to, personal computer (PC) systems, server computer systems, thin clients, thick clients, handheld or laptop devices, multiprocessor systems, microprocessor-based systems, set top boxes, programmable consumer electronics, network PCs, minicomputer systems, mainframe computer systems, and distributed cloud computing environments that include any of the above systems or devices, and the like.

[0099] With reference to Figure 12A , another embodiment of a computing environment that includes and uses one or more aspects of the present application is described. In this example, the computing environment 1210 includes, for example, a native central processing unit (CPU) 1212, memory 1214, and one or more input / output devices and / or interfaces 1216, which are coupled via, for example, one or more buses 1218 and / or other connections. For example, the computing environment 1210 can include an IBM® eServer™ System p® computer system, with a IBM® POWER5™ processor, available from International Business Machines Corporation in Armonk, New York, a HP Superdome computer system, with Intel Itanium II processors, available from Hewlett-Packard Co., Palo Alto, California, and / or other machines based on architecture offered by IBM, Hewlett-Packard, Intel, Oracle, or others. The processors; HP Superdome equipped with Intel Itanium II processors available from Hewlett-Packard Co., Palo Alto, California; and / or other machines based on architecture offered by IBM, Hewlett-Packard, Intel, Oracle, or others. PowerPC is a trademark or registered trademark of International Business Machines Corporation in at least one jurisdiction. Intel and Itanium are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States and other countries.

[0100] The native central processing unit 1212 includes one or more native registers 1220, such as one or more general registers and / or one or more special registers used during processing within the environment. These registers include information representative of the state of the environment at any particular moment.

[0101] In addition, the native central processing unit 1212 executes instructions and code stored in the memory 1214. In one particular example, the central processing unit executes emulator code 1222 stored within the memory 1214. This code enables a computing environment configured in one architecture to emulate another architecture. For example, the emulator code 1222 allows a machine based on an architecture other than the z / Architecture hardware architecture, such as a PowerPC processor, an HP Superdome server, etc., to emulate the z / Architecture hardware architecture and execute software and instructions developed based on the z / Architecture hardware architecture.

[0102] Reference is made to Figure 12B for more details related to the emulator code 1222. Guest instructions 1230 stored in the memory 1214 include software instructions (e.g., instructions related to machine instructions) developed for execution in an architecture other than the native CPU 1212. For example, the guest instructions 1230 can be designed to execute on a processor based on the z / architecture hardware architecture, but are actually emulated on the native CPU 1212, which can be, for example, an Intel Itanium II processor. In one example, the emulator code 1222 includes an instruction fetch routine 1232 for fetching one or more guest instructions 1230 from the memory 1214 and optionally providing local buffering for the fetched instructions. It also includes an instruction translation routine 1234 for determining the type of guest instruction that has been fetched and translating the guest instruction into one or more corresponding native instructions 1236. The translation includes, for example, identifying the function that the guest instruction is to perform and selecting a native instruction to perform that function.

[0103] In addition, the emulator code 1222 includes an emulation control routine 1240 for causing the native instructions to be executed. The emulation control routine 1240 can cause the native CPU 1212 to execute a native instruction routine that emulates one or more previously fetched guest instructions and, at the end of the execution, return control to the instruction fetch routine to emulate fetching of the next guest instruction or set of guest instructions. Execution of the native instructions 1236 can include loading data from the memory 1214 into the registers; storing data from the registers back to the memory; or performing some type of arithmetic or logical operation determined by the translation routine.

[0104] For example, each routine can be implemented in software while stored in memory and executed by the native central processing unit 1212. In other examples, one or more routines or operations are implemented in firmware, hardware, software, or combinations thereof. Registers of the emulated processor can be emulated using locations in the registers 1220 or memory 1214 of the native CPU. In embodiments, the guest instructions 1230, native instructions 1236, and emulator code 1222 can reside in the same memory or can be allocated among different memory devices.

[0105] Further, in one embodiment, the computing environment 1210 includes one or more inference accelerators 1215 coupled to the memory 1214. The one or more accelerators are defined in one architecture and configured to emulate another architecture. For example, the accelerator takes guest commands of the emulated architecture, converts the guest commands to native commands of one architecture, and executes the native commands.

[0106] The computing environments described above are merely examples of computing environments that can be used. Other environments can be used, including but not limited to non-partitioned environments, partitioned environments, cloud environments, and / or emulated environments; embodiments are not limited to any one environment. Although various examples of computing environments are described herein, one or more aspects of the present disclosure can be used in a variety of types of environments. The computing environments provided herein are merely examples.

[0107] Each of the computing environments can be configured to include one or more aspects of the present disclosure. For example, each can be configured for inference acceleration facilities in accordance with one or more aspects of the present disclosure.

[0108] Although various embodiments are described herein, many variations and alternatives are possible and contemplated. It should be noted that each of the aspects or features described herein can be combined with any other aspect or feature unless explicitly stated otherwise.

[0109] One or more aspects can relate to cloud computing.

[0110] It should be understood that while the present disclosure includes detailed descriptions of cloud computing, implementation of the teachings given herein are not limited to cloud computing environments. Rather, embodiments of the present disclosure are capable of being implemented in conjunction with any other type of computing environment now known or later developed.

[0111] Cloud computing is a model of service delivery for enabling convenient, on-demand network access to a shared pool of configurable computing resources (e.g. networks, network bandwidth, servers, processing, memory, storage, applications, virtual machines, and services) that can be rapidly provisioned and released with minimal management effort or interaction with a provider of the service. This cloud model can be composed of at least five characteristics, at least three service models, and at least four deployment models.

[0112] The characteristics are as follows:

[0113] Broad network access: capabilities are available over a network and accessed through standard mechanisms that promote the use of heterogeneous thin or thick client platforms (e.g., mobile phones, laptops, and PDAs).

[0114] Resource pooling: the provider's computing resources are pooled to serve multiple consumers using a multi-tenant model, with different physical and virtual resources dynamically assigned and reassigned according to consumer demand. There is a sense of location independence in that the consumer generally has no control or knowledge over the exact location of the provided resources but can be able to specify location at a higher level of abstraction (e.g., country, state, or datacenter).

[0115] Rapid elasticity: capabilities can be rapidly and elastically provisioned, in some cases automatically, to quickly scale out and rapidly release to quickly scale in. To the consumer, the capabilities available for provisioning commonly appear to be unlimited and can be purchased in any quantity at any time.

[0116] Measured service: cloud systems automatically control and optimize resource use by leveraging utilization of resources in an economical and efficient manner, including their ability to deliver specific services at varying levels of quality of service. Cloud systems often automatically provision and often release resources based on demand.

[0117] Measured service: cloud systems automatically control and optimize resource use by leveraging utilization of resources in an economical and efficient manner, including their ability to deliver specific services at varying levels of quality of service. Cloud systems often automatically provision and often release resources based on demand.

[0118] The service models are as follows:

[0119] Software as a Service (SaaS): the capability provided to the consumer is to use the provider's applications running on a cloud infrastructure. The applications are accessible from various client devices through a thin client interface such as a web browser (e.g., web-based e-mail). The consumer does not manage or control the underlying cloud infrastructure including network, servers, operating systems, storage, or even individual application capabilities, with the possible exception of limited user-specific application configuration settings.

[0120] Platform as a Service (PaaS): the capability provided to the consumer is to deploy onto the cloud infrastructure consumer-created or acquired applications created using programming languages and tools supported by the provider. The consumer does not manage or control the underlying cloud infrastructure including networks, servers, operating systems, or storage, but has control over the deployed applications and possibly application hosting environment configurations.

[0121] Infrastructure as a Service (laaS): the capability provided to the consumer is to provision processing, storage, networks, and other fundamental computing resources where the consumer is able to deploy and run arbitrary software, which can include operating systems and applications. The consumer does not manage or control the underlying cloud infrastructure but has control over operating systems, storage, deployed applications, and possibly limited control of select networking components (e.g., host firewalls).

[0122] Deployment models are as follows:

[0123] Private cloud: the cloud infrastructure is operated solely for an organization. It can be managed by the organization or a third party and can exist on-premises or off-premises.

[0124] Community cloud: the cloud infrastructure is shared by several organizations and supports mission-oriented business

[0125] Public cloud: the cloud infrastructure is made available to the general public or a large industry group and is owned by an organization selling cloud services.

[0126] Hybrid cloud: the cloud infrastructure is a composition of two or more clouds (private, community, or public) that remain unique entities but are bound together, creating the hybrid cloud.

[0127] A cloud computing environment is service-oriented, focused on stateless, low-coupled, modular, and semantic interoperability. At the core of cloud computing is an infrastructure comprising a network of interconnected nodes.

[0128] Referring now to the drawings Figure 13, an illustrative cloud computing environment 50 is described. As shown, cloud computing environment 50 includes one or more cloud computing nodes 10 with which local computing devices used by cloud consumers, such as, for example, personal digital assistant (PDA) or cellular telephone 54A, desktop computer 54B, laptop computer 54C, and / or automobile computer system 54N can communicate. Nodes 10 can communicate with one another. They can be grouped (not shown) physically or virtually, in one or more networks, such as Private, Community, Public, or Hybrid clouds as described hereinabove, or a combination thereof. This allows cloud computing environment 50 to offer infrastructure, platforms and / or software as services with Figure 13 The types of computing devices 54A-N shown in

[0129] Referring now to Figure 14 , a set of functional abstraction layers provided by cloud computing environment 50 (best shown in FIG. 1C) is shown. It should be understood that Figure 13 The components, layers, and functions shown in Figure 14 are merely intended to be illustrative; the embodiments of the present application are not limited thereto. As illustrated, the following layers and corresponding functions are provided:

[0130] Hardware and software layer 60 includes hardware and software components. Examples of hardware components include: mainframes 61; RISC (Reduced Instruction Set Computer) architecture based servers 62; servers 63; blade servers 64; storage devices 65; and networks and networking components 66. In some embodiments, software components include network application server software 67 and database software 68.

[0131] Virtualization layer 70 provides an abstraction layer from which the following examples of virtual entities can be provided: virtual servers 71; virtual storage 72; virtual networks 73, including virtual private networks; virtual applications and operating systems 74; and virtual clients 75.

[0132] In one example, management layer 80 can provide the functions described below. Resource provisioning 81 provides dynamic procurement of computing resources and other resources that are utilized to perform tasks within the cloud computing environment. Metering and Pricing 82 provide cost tracking as resources are utilized within the cloud computing environment, and billing or invoicing for consumption of these resources. In one example, these resources can include application software licenses. Security provides identity verification for cloud consumers and tasks, as well as protection for data and other resources. User portal 83 provides access to the cloud computing environment for consumers and system administrators. Service level management 84 provides cloud computing resource allocation and management such that required service levels are met. Service Level Agreement (SLA) planning and fulfillment 85 provide pre-arrangement for, and procurement of, cloud computing resources for future requirement in accordance with SLA.

[0133] Workloads layer 90 provides examples of functionality for which the cloud computing environment can be utilized. Examples of workloads and functions which can be provided from this layer include: mapping and navigation 91; software development and lifecycle management 92; virtual classroom education delivery 93; data analysis processing 94; transaction processing 95; and, inference accelerator and related processing 96.

[0134] The present application can be a system, a method, and / or a computer program product at any possible technical detail level of integration. The computer program product can include a computer readable storage medium (or media) having computer readable program instructions thereon for causing a processor to carry out aspects of the present application.

[0135] The computer readable storage medium can be a tangible device that can retain and store instructions for use by an instruction execution device. The computer readable storage medium can be, for example, but is not limited to, an electronic storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the foregoing. A non-exhaustive list of more specific examples of the computer readable storage medium includes the following: a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), a static random access memory (SRAM), a portable compact disc read-only memory (CD-ROM), a digital versatile disk (DVD), a memory stick, a floppy disk, a mechanically encoded device such as punch-cards or raised structures in a groove having instructions recorded thereon, and any suitable combination of the foregoing. A computer readable storage medium, as used herein, is not to be construed as being transitory signals per se, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through a waveguide or other transmission media (e.g., light pulses passing through a fiber-optic cable), or electrical signals transmitted through a wire.

[0136] Computer readable program instructions described herein can be downloaded to respective computing / processing devices from a computer readable storage medium or to an external computer or external storage device via a network, for example, the Internet, a local area network, a wide area network and / or a wireless network. The network can comprise copper transmission cables, optical transmission fibers, wireless transmission, routers, firewalls, switches, gateway computers and / or edge servers. A network adapter card or network interface in each computing / processing device receives computer readable program instructions from the network and forwards the computer readable program instructions to a computer readable storage medium within the respective computing / processing device for execution by the computing device.

[0137] Computer readable program instructions for carrying out operations of the present application can be assembly instructions, instruction-set-architecture (ISA) instructions, machine instructions, machine dependent instructions, microcode, firmware instructions, state-setting data, or source code or object code written in any combination of one or more programming languages, including an object oriented programming language such as Java, Smalltalk, C++ or the like and conventional procedural programming languages such as the "C" programming language or similar programming languages. The computer readable program instructions can execute entirely on the user's computing device, partly on the user's computing device, as a stand-alone software package, partly on the user's computing device and partly on a remote computing device or entirely on the remote computing device or server. In the latter scenario, the remote computing device can be connected to the user's computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection can be made to an external computing device, for example, through the Internet using an Internet Service Provider. In some embodiments, electronic circuitry including, for example, programmable logic circuitry, field-programmable gate arrays (FPGA), or programmable logic arrays (PLA) can execute the computer readable program instructions by utilizing state information of the computer readable program instructions to personalize the electronic circuitry, in order to perform aspects of the present application.

[0138] The computer readable program instructions can also be loaded onto a computing / processing device, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computing / processing device, other programmable apparatus or other device to produce a computer implemented process such that the instructions which execute on the computing / processing device, other programmable apparatus, or other device implement the functions / acts specified in the flowchart and / or block diagram block or blocks.

[0139] These computer readable program instructions can be provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions / acts specified in the flowchart and / or block diagram block or blocks. These computer readable program instructions can also be stored in a computer readable storage medium that can include a non-transitory computer readable storage medium that can be a computer- readable storage medium having no data, programs, program modules, e.g., instructions for operation, or digital content stored thereon or therein for a short time or not at all. The computer readable storage medium of claim 1, including a general purpose computer "readable" medium, e.g., a computer hard drive, RAM, ROM, EEPROM, CD-ROM or DVD-ROM, floppy disk, etc. The computer readable program instructions can also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable apparatus or other device to produce a computer implemented process, such that the instructions which execute on the computer, other programmable apparatus, or other device implement the functions / acts specified in the flowchart and / or block diagram block or blocks.

[0140] These computer readable program instructions can be provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions / acts specified in the flowchart and / or block diagram block or blocks. These computer readable program instructions can also be stored in a computer readable storage medium that can include a non-transitory computer readable storage medium that can be a computer- readable storage medium having no data, programs, program modules, e.g., instructions for operation, or digital content stored thereon or therein for a short time or not at all. The computer readable storage medium of claim 1, including a general purpose computer "readable" medium, e.g., a computer hard drive, RAM, ROM, EEPROM, CD-ROM or DVD-ROM, floppy disk, etc. The computer readable program instructions can also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable apparatus or other device to produce a computer implemented process, such that the instructions which execute on the computer, other programmable apparatus, or other device implement the functions / acts specified in the flowchart and / or block diagram block or blocks.

[0141] The flow and block diagrams in the drawings show architectural, functional, and operational representations of possible implementations of systems, methods, and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flow and block diagrams can represent a module, segment, or portion of instructions, which includes one or more executable instructions for implementing the specified logical function(s). In some alternative implementations, the functions noted in the blocks can occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks can sometimes be executed in the reverse order, depending on the functionality involved. It will also be noted that each block of the block diagrams and / or flowchart illustrations, and combinations of blocks in the block diagrams and / or flowchart illustrations, can be implemented by special purpose hardware-based systems that perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.

[0142] In addition to the above, one or more aspects can be provided, offered, deployed, managed, serviced, etc. by a service provider who offers to deploy, manage, support, etc. one or more aspects. For example, one or more aspects can be offered by a service provider who provides one or more of computer infrastructure as a service, computer infrastructure offering on a subscription or fee basis, processes executed on the computer infrastructure as a service, etc. In one example, the service provider can deploy a customer's computer infrastructure, e.g., offering to be used for deploying, managing, supporting, etc. one or more aspects. The customer can in response to the service provider's offer, subscribe to or otherwise acquire the service as provided by the service provider. The service provider resources can then be used by the customer to host one or more aspects.

[0143] In one aspect, an application for performing one or more embodiments can be deployed. As one example, the deployment of the application comprises providing computer infrastructure operable when implemented to perform one or more embodiments. In one aspect, an application for performing one or more embodiments can be deployed. As one example, the deployment of the application comprises providing computer infrastructure operable when implemented to perform one or more embodiments.

[0144] As a further aspect, a computing infrastructure can be deployed comprising integrating computer readable code on an on-premises computing system, wherein the code in combination with the on-premises computing system is capable of carrying out one or more embodiments.

[0145] As a further aspect, a process for integrating computing infrastructure can be provided that comprises integrating computer readable code on a computer system. The computer system comprises a computer readable medium, wherein the computer readable medium comprises one or more embodiments. The code in combination with the computer system is capable of carrying out one or more embodiments.

[0146] While various embodiments have been described above, it should be appreciated that these have been presented by way of example only. For example, other architectures can be used to include and use one or more embodiments. Further, different instructions, commands, or operations can be used. Also, different types of indications or labels can be specified. Numerous variations and modifications can be possible.

[0147] Various aspects are described herein. Moreover, numerous variants are possible, without departing from the spirit of aspects of the disclosure. It should be noted that each of the aspects described herein and variants thereof can be combined with any other aspect or feature, unless otherwise inconsistent.

[0148] Further, other types of computing environments can also benefit and be used. As an example, a data processing system suitable for storing and / or executing program code includes at least two processors coupled directly or indirectly to memory elements through a system bus. The memory elements include, for example, local memory employed during actual execution of the program code, bulk storage, and cache memory, which provide temporary storage of at least some program code in order to reduce the number of times programming code must be retrieved from bulk storage during execution.

[0149] Input / output or I / O devices (including but not limited to keyboards, displays, pointing devices, DASD, tape, CD, DVD, thumb drive, and other storage media, etc.) can be coupled to a system either directly or through intervening I / O controllers. Network adapters can also be coupled to the system to enable the data processing system to become coupled to other data processing systems, or remote printers or storage devices through intervening private or public networks. Modems, cable modems, and Ethernet cards are just a few of the available types of network adapters.

[0150] The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting. As used herein, the singular forms "a," "an," and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will be further understood that the terms "comprises" and / or "comprising," when used in this specification, specify the presence of stated features, integers, steps, operations, elements, and / or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof.

[0151] All devices or steps plus functional recitations in the claims below are intended to include any structure, material, or acts for performing the functions in combination with other claimed elements as specifically claimed. The description set forth herein is presented for the purpose of illustrating and describing the described description, but is not intended to be exhaustive or limiting to the precise forms described. Many modifications and variations are possible in light of this disclosure without departing from the scope and spirit of the disclosure. Embodiments were chosen and described in order to best explain the various aspects and practical applications, and to enable others skilled in the art to understand the different embodiments with various modifications being suited to the particular use contemplated.

Claims

1. A computer-implemented method of facilitating processing within a computing environment, the computer-implemented method comprising: establishing, by processing circuitry, an augmented decision tree from a source decision tree, the establishing comprising inserting one or more dummy decision nodes into the source decision tree to obtain the augmented decision tree, wherein the augmented decision tree refers to a decision tree in which, for a given depth, all decision nodes are occupied; determining, by the processing circuitry, decision node data and leaf node data for the augmented decision tree, wherein the determining comprises, for a leaf node of the plurality of leaf nodes, descending the leaf node to a maximum depth of the augmented decision tree and adjusting a path vector of the leaf node based thereon to obtain a respective predetermined path vector for the leaf node, wherein the adjusting comprises determining a string value to insert into the path vector based on a distance the leaf node descends according to the descending and inserting the string value into the path vector to obtain the respective predetermined path vector for the leaf node; and providing, by the processing circuitry, the decision node data and the leaf node data to an inference accelerator to facilitate accelerated processing of the augmented decision tree and thereby determine which leaf node of the plurality of leaf nodes of the augmented decision tree to select, wherein the accelerated processing comprises processing of decision node outcome vectors accelerated by the plurality of leaf nodes processing the decision node outcome vectors in parallel based on the respective predetermined path vector of each leaf node of the plurality of leaf nodes through the augmented decision tree, the method further comprising, for at least one other leaf node of the plurality of leaf nodes, inserting an "irrelevant” string in the same insertion position in its path vector, wherein the "irrelevant” string is of the same size as the determined string value.

2. The computer-implemented method of claim 1, wherein, the source decision tree is a pruned decision tree, the decision node data comprises data of the positioned adjusted decision nodes of the augmented decision tree, and the leaf node data comprises data of the positioned adjusted leaf nodes of the augmented decision tree.

3. The computer-implemented method of claim 1, wherein, the string value to insert corresponds to a last entry of the descended layer size in a fully populated path table.

4. The computer-implemented method of claim 1, wherein, the adjusting comprises determining the insertion position of the string value in such a way that the last decision node in the path vector is identified and, based on the last decision node being true or the last decision node being at the end of the path, the insertion position is one position beyond the last decision node.

5. The computer-implemented method of claim 4, wherein, based on the last decision node being false and not being at the end of the path, the insertion position is two positions beyond the last decision node.

6. A computer system for facilitating processing within a computing environment, the computer system comprising: a memory; and processing circuitry in communication with the memory, wherein the computer system is configured to perform the method of any one of claims 1 to 5.

7. A computer program product for facilitating processing within a computing environment, the computer program product comprising program instructions embodied on the computer program product, the program instructions readable by processing circuitry to cause the processing circuitry to perform the method of any one of claims 1 to 5.

Images