Architecture and Method for Classification of Decision Tree Ensembles

Abstract

Architecture and Method for Classification of Decision Tree Ensembles The architecture for classifying a decision tree ensemble includes a ranking calculation block (1) and a tree processing block (2). The ranking calculation block (1) includes one calculation unit (10) per feature fi to be classified, wherein each calculation unit (10) includes memory (11) and logic (12). The memory (11) is configured to store threshold values ​​of the trees of the decision tree ensemble, wherein the memory is arranged to store threshold values ​​for one group per feature to be classified. The threshold values ​​of the groups are sorted according to the threshold index. The logic (12) is arranged to perform node comparison of the threshold values ​​of the corresponding threshold groups and output a rank as a result of the comparison, wherein the rank represents an encoded address for accessing the threshold values. The tree processing block (2) includes one tree processing unit (20) per tree of the decision tree ensemble, and each processing unit is configured to determine the class of the feature fi to be classified as a function of the rank.

Description

Technology Field

[0001] The present disclosure relates to an architecture and method for classifying decision tree ensembles. Background Technology

[0002] Random forests, RF, or decision tree ensembles, or DTE for short, are used for various classification tasks in both hardware and software. DTEs are intuitively evaluated by traversing each tree sequentially and then aggregating the outputs of each tree through averaging or majority voting. Another approach is to compute all node comparisons in parallel and convert the trees into Boolean functions that compute tree outputs for aggregation or compute the aggregated outputs directly.

[0003] DTEs can be trained by machine learning algorithms such as random forest, AdaBoost, or gradient boosted trees. An architecture for classification of decision tree ensembles can be configured as a hardware architecture for inference of DTEs in FPGA or ASIC technology. Such an architecture can be integrated into various applications in the field of edge machine learning, such as but not limited to smart sensors, wireless sensor networks, structural health monitoring, predictive maintenance, and healthcare.

[0004] An architecture for decision tree ensemble classification, or DTE hardware accelerator, typically consists of one tree processing unit per decision tree in the ensemble and an aggregation unit that calculates ensemble votes as illustrated in FIG. 13. Such a tree processing unit can also be used as a hardware accelerator for a single decision tree itself.

[0005] Fig. 14a illustrates an example of a single classification tree. To predict the class for a new set of feature inputs, each node of the tree is one of the features f i node threshold t j It is compared to. Starting from the root node, the result of the comparison at the next node defines a path through the tree, which terminates at a leaf node. The tree corresponds to class C that is reached at the leaf. x Votes on it. Then, all tree votes in the ensemble are aggregated by majority vote to obtain the final vote of the tree ensemble. Features compared by nodes f index, threshold value t , class labels at leaf nodes C , and the structure of the tree are parameters defined by training.

[0006] Two types of architectures are widely used to implement decision trees. Basically, a serial architecture stores all parameters of the decision tree in memory and evaluates them from the root to the leaf. A parallel architecture has a single comparator for each node and evaluates the entire tree at once. More specifically, the serial architecture traverses the decision tree of the trained model from the root to the leaf nodes, as depicted in Fig. 14b. At each node, it fetches the threshold and feature inputs of the given node from memory or an input multiplexer. It then compares the two and selects the left or right child address as the next node to visit. This is done until a leaf node is reached and the corresponding class vote is output. The parallel architecture computes comparisons of all nodes in the tree at once, as illustrated in Fig. 14c. Each comparator represents a single tree node. The threshold is stored in a register containing the comparator, which is connected to the feature input depending on the node it represents. Tree voting is calculated from the comparator output by a Boolean function representing the tree structure. The Boolean function is specific to the trained model and is derived from the tree as follows.

[0007] The classification tree from DTE is a rooted, oriented, acyclic graph using the following notation. It can be expressed mathematically as:

[0008] V The peak of a set , where L is a set of leaf nodes and N is the threshold It is a set of internal nodes containing, which is one of the features, feature f i Corresponding to,

[0009] D Each node Node comparison for A set of boolean variables defined by And,

[0010] Edges are taken when processing an E-tree To define the result of Labeled by A set of edges having And,

[0011] C A set of classes assigned to leaf nodes by.

[0012] Each class For, a set of paths is defined as follows, and

[0013]

[0014] Regarding, as, silver , and having Root node am.

[0015] The parallel architecture of Fig. 14c implements each node as a fixed comparator. The tree structure is derived from the comparator output of the class c A Boolean function that calculates tree votes for It can be formulated as. The Boolean function can be expressed as follows:

[0016]

[0017]

[0018] For example, the tree depicted in Fig. 14c(2) causes the following:

[0019]

[0020] and

[0021]

[0022] However, both serial and parallel architectures leave room for improvement. The problem to be solved

[0023] The object of the present disclosure is to provide an architecture for classifying decision tree ensembles and a method for classifying decision tree ensembles, which improves the decision tree ensemble classification architecture in terms of, for example, speed, energy consumption and / or area.

[0024] These objectives are achieved by the subject matter of the independent claim. Further developments and embodiments are described in the dependent claims. means of solving the problem

[0025] It will be understood that any feature described in connection with any one embodiment may be used alone or in combination with other features described herein, and may also be used in combination with one or more features of any other embodiment among the embodiments, or in any combination of any other embodiment among the embodiments unless otherwise described. Furthermore, equivalents and modifications not described below may also be used without departing from the scope of the architecture for classifying a decision tree ensemble and the method for classifying a decision tree ensemble, as defined in the appended claims.

[0026] The following concerns an improved concept in the field of decision tree ensembles. The proposed concept proposes an architecture and method for solving inference in classification and regression trees and decision tree ensembles, abbreviated as DTE. The proposed concept can be implemented based on a hardware architecture, for example, in FPGA or ASIC technology. However, the method can also be implemented in software.

[0027] The proposed concept proposes grouping and pre-sorting of thresholds of all comparisons of a given decision tree ensemble to quickly compute all nodes using logic such as binary search or in-memory search. Furthermore, the architecture can be supplemented with a lookup table holding pre-computed entries to further increase classification speed.

[0028] In at least one embodiment, the architecture for classifying a decision tree ensemble includes a rank calculation block and a tree processing block. The rank calculation block includes one computation unit per feature from a set of features to be classified. Each computation unit includes memory and logic. The memory is configured to store threshold values ​​of the trees in the decision tree ensemble. The memory is configured to store threshold values ​​for one group per feature. Furthermore, the threshold values ​​of the groups are pre-sorted according to a threshold index to allow for efficient search algorithms such as binary search. The threshold index allows defining search criteria such as "<" or ">".

[0029] The logic is arranged to perform a comparison of threshold values ​​of a corresponding threshold group and output the rank of the evaluated feature value. The rank represents an encoded address representing the threshold value. A tree processing block is configured to determine the class of a set of features to be classified as a function of the rank. For example, in some embodiments, the tree processing block includes one tree processing unit per tree in the decision tree ensemble. Each processing unit is configured to determine the class of a set of features to be classified as a function of the rank. In other embodiments, there may be only one tree processing unit, which includes multiple look-up tables and combiners as discussed in more detail below.

[0030] Preliminary results show that the new architecture can match the hardware cost and energy efficiency of both serial and parallel architectures, each excelling in only one of two figures of merit. Rankings can be used for additional processing by tree processing blocks as they retain the same information as the calculated comparison of all nodes.

[0031] In at least one embodiment, the features from a set of features to be classified are feature indices f having i It is defined as, where I represents the number of features to be classified. The threshold represents a node of the tree in the decision tree ensemble.

[0032] The logic is a set of features f to be classified i cast It is arranged to perform a search that receives and splits the feature into values ​​less than and greater than a threshold, and returns a rank for each feature. The rank is output, for example, to a corresponding tree processing unit. Finally, the tree processing unit is configured to determine the class by evaluating a Boolean function representing the structure of each decision tree in the decision tree ensemble as a function of the received rank. Alternatively, the rank is output to a corresponding lookup table.

[0033] In at least one embodiment, the tree processing block includes an aggregation unit arranged to receive a determined class from a tree processing unit and to form an ensemble vote as a function of the determined class.

[0034] In at least one embodiment of the memory, the memory comprises a group of memory cells for each feature from a set of features to be classified. The memory cells within the group are addressable according to a threshold index.

[0035] In at least one embodiment, the logic includes a comparator further comprising a first input operable to receive a feature and a second input operable to receive a threshold value from memory.

[0036] In at least one embodiment, the tree processing unit includes one or more comparators arranged to perform address decoding of a received rank to evaluate a Boolean function based on the address-decoded rank. The comparators are operable to perform a comparison based on a rank representing a node comparison of a threshold of a corresponding threshold group. In a sense, the rank is decoded to perform an address comparison.

[0037] In at least one embodiment, each calculation unit from the rank calculation block is connected to a dedicated lookup table. The lookup table includes an input side operable to receive a dedicated rank from the calculation unit connected to the lookup table. Furthermore, the lookup table is operable to output an intermediate value such as a function of the input rank.

[0038] In at least one embodiment, each tree processing unit may be connected to a dedicated lookup table. The lookup table includes an output side arranged to provide an intermediate value to the tree processing unit connected to the lookup table. The tree processing unit is operable to determine the class of a set of features to be classified as a function of the received intermediate value.

[0039] In another embodiment, there may no longer be a dedicated lookup table for each tree processing unit. Rather, the lookup table may be considered as part of a tree processing block and supplemented by a combiner. Content corresponding to the tree structure is stored for all trees combined into one lookup table per feature. Lookup table entries correspond to possible configurations and, together with the combiner, form a tree processing block. However, several trees may also be combined internally into a "sub-ensemble."

[0040] In at least one embodiment, the architecture further includes a combiner having an arithmetic unit. The combiner is operable to receive intermediate values, combine intermediate values ​​and / or weight them, and output a class of features to be classified. The combiner may be implemented as a common unit for all trees or sub-ensembles.

[0041] In at least one embodiment, the lookup table is implemented based on an associative memory comprising one or more comparators operable to perform "less than" or "greater than" comparisons.

[0042] In at least one embodiment, a method for classifying a decision tree ensemble comprises the following steps. First, threshold values ​​of the decision trees of the decision tree ensemble are stored as one group per feature to be classified. The threshold values ​​of the groups are sorted according to a threshold index. Node comparisons of the threshold values ​​of corresponding threshold groups are performed. Ranks are output as a result of the node comparisons, wherein the rank represents an encoded address for evaluating the threshold values. Finally, a class of a set of features to be classified is determined as a function of the ranks.

[0043] In at least one embodiment, the features from a set of features to be classified are feature indices f having i It is indicated as, where I represents the number of features to be classified. The threshold represents a node of the tree in the decision tree ensemble. The method further includes the step of receiving features to be classified by logic, and using logic to perform a search that returns a rank for each feature that divides the threshold into values ​​less than and greater than the feature value. The ranks are output to the corresponding tree processing unit. The class of the features to be classified is determined by evaluating a Boolean function representing each tree in the decision tree ensemble as a function of the received rank.

[0044] In at least one embodiment, the rank is determined from a binary search using logic. Alternatively, the rank is determined from an in-memory search.

[0045] In at least one embodiment, the evaluation of the decision tree involves a lookup table filled with pre-calculated values ​​stored as lookup table entries, wherein the class is determined according to the lookup table entries.

[0046] In at least one embodiment, lookup table entries are determined by training software. The training software determines the assignment of lookup table entries such that both the classification architecture and the mathematical description of the DTE are equivalent.

[0047] The following description of the drawings of exemplary embodiments may further illustrate and describe aspects of the improved concept. Each component and part having the same structure and the same effect is represented by an equivalent reference. Insofar as components and parts correspond to each other in terms of their function in different drawings, the description does not need to be repeated for each of the following drawings.

[0048] Additional embodiments, features, and advantages will become readily apparent from the following detailed description, the accompanying drawings, and the claims. Brief explanation of the drawing

[0049] In the drawing: FIG. 1 illustrates an exemplary top-level diagram of an architecture for classification of a decision tree ensemble, and FIG. 2 illustrates an exemplary embodiment of a computing unit, and FIG. 3 illustrates an exemplary decision tree ensemble to illustrate grouping and pre-sorting methods, and FIG. 4 illustrates an exemplary embodiment of a computing unit, and Figure 5 illustrates the size FOM of exemplary architectures for several models trained on different datasets using different training algorithms, and Figure 6 illustrates an energy comparison of exemplary architectures for several models trained on different datasets using different training algorithms, and FIG. 7 illustrates another exemplary embodiment of a computing unit, and FIG. 8 illustrates generating a classification result from a ranking using an LUT, and FIG. 9 illustrates an exemplary embodiment of combined in-memory rank calculation and LUT lookup, and FIG. 10 illustrates the calculation of a classification result from a LUT entry, and FIG. 11 illustrates further details of an exemplary embodiment of a combiner having I = 7 features, and FIG. 12 illustrates further details of an exemplary embodiment of a combiner, and FIG. 13 illustrates the architecture of a prior art for decision tree ensemble classification, and Figure 14 illustrates an example of a classification concept. Specific details for implementing the invention

[0050] FIG. 1 illustrates an exemplary top-level diagram of an architecture for classifying a decision tree ensemble. The architecture includes a ranking calculation block (1) and a tree processing block (2). The ranking calculation block (1) includes a plurality of calculation units (10). In fact, a feature f to be classified i There is one dedicated computation unit per (index i or feature index represents an integer of the I feature to be classified). Each computation unit further includes memory (11) and logic (12).

[0051] Memory (11) contains multiple memory cells and is configured to store multiple threshold values ​​according to the decision tree from the decision tree ensemble. In other words, the memory stores the threshold values ​​of the tree, which is a characteristic of the decision tree ensemble. The threshold values ​​are grouped into feature groups, as discussed in more detail below. Basically, the memory contains one threshold group per feature to be classified. The threshold values ​​of the group are sorted according to the threshold index. Logic (12) allows access to the threshold values ​​of the corresponding threshold group. According to the process further discussed below, the logic outputs the rank of the accessed threshold value. The rank represents the encoded address for accessing the threshold value, which in turn represents the threshold value.

[0052] The tree processing block (2) includes one tree processing unit (20) per tree in the decision tree ensemble. The tree processing units are arranged to perform classification or tree voting. In fact, each processing unit is configured to evaluate a Boolean function representing a tree in the decision tree ensemble as a function of rank.

[0053] Boolean functions can be realized in soft logic on an FPGA or hardwired logic on an ASIC, or using a scheme further discussed below. Ranks are received from a rank calculation block and a corresponding calculation unit. The tree processing block further includes an aggregation unit (21) (or tree combiner). This unit receives votes (or classifications) from the tree processing unit and forms the ensemble votes. In other words, the aggregation unit performs a majority vote.

[0054] FIG. 2 illustrates an exemplary embodiment of a computational unit in more detail. Memory (11) comprises a plurality of memory cells, each having a unique address. The drawing indicates that the memory holds threshold values ​​that are sorted according to a threshold index. The threshold values ​​stored in the memory belong to a dedicated group associated with a corresponding feature (f1 in this example). The feature can be input through the input side (13) of a comparator (14) (indicated by "<" in the drawing). Another input (15) is arranged to access the threshold value from the memory and receive it. In this way, the comparator can perform a comparison of the feature (f1 in this example) associated with the dedicated group according to the tree definition. Furthermore, logic (12) is arranged to access the comparator through another input side (16). As a result of this control, the comparator compares the input feature with the threshold value. The memory outputs a corresponding encoded address, which, in turn, represents a threshold / feature comparison (or tree node). Ranks can be used for further processing by tree processing blocks as they hold the same information as the calculated comparison of all nodes.

[0055] In conclusion, similar to a serial architecture, the computation unit includes a comparator and memory. However, the memory includes a single threshold group instead of a single complete decision tree. The comparator (14) is connected only to corresponding features, thus eliminating the complexity of the input multiplexer typically required in other architectures. Furthermore, the memory includes only the threshold and does not include tree structure information such as feature indices or child addresses, which significantly reduces the memory size. For example, the search logic performs a binary search to quickly compare input features to the threshold.

[0056] FIG. 3 illustrates an exemplary decision tree ensemble illustrating grouping and pre-sorting methods. The proposed concept is based on pre-sorting threshold values, for example, to increase the speed of comparison. The figure illustrates an example of how threshold values ​​can be grouped and sorted. The figure shows features f 1 , f 2 , and f 3 In addition, two exemplary classes C 1 and C 2 Describe three exemplary decision trees containing. The tree nodes represent the threshold with threshold index j. and the corresponding input feature f having feature index i i It is considered a comparison between them. . The thresholds of all decision trees in the ensemble can be grouped by the corresponding feature index of each node. Exemplary resulting groups of thresholds by feature are shown in Table 1. Threshold value Eun, for example As shown, it can be sorted according to sorting criteria. The logic (12) can be arranged to use these sorting criteria to perform a fast search operation. An example of the result of the sorted group is shown in Table 2.

[0057]

[0058] Table 1: Group thresholds by feature

[0059]

[0060] Table 2: Groups sorted by feature

[0061] The resulting threshold groups are individually sorted by threshold values, where the sorting criteria, e.g. This enables fast search methods such as binary search. Instead of traversing the tree sequentially or computing all nodes, the DTE can be evaluated by performing a binary search on each feature within a corresponding threshold group. Binary search yields addresses (i.e., ranks) for each feature that divide the threshold into values ​​below and above the feature. These addresses, which are the ranks of the features, hold the same information as the computed comparisons of all nodes. Therefore, tree voting can be computed by a Boolean function, as is done in a parallel architecture, for example, while requiring fewer comparisons.

[0062] The complete architecture includes one computation unit per feature, one tree voting function per tree, and, optionally, an aggregation unit to form an ensemble as illustrated in FIG. 1. Tree voting has similarities to the parallel architecture. However, while the input is an encoded address, the parallel architecture encodes each comparison as a 1-bit. Therefore, the tree voting function of the parallel architecture is extended into a different Boolean function. Using the nomenclature introduced above in the introduction of the disclosure, the tree structure is class c or rank from the comparator output R A Boolean function that calculates votes for It can be formulated as. The Boolean function can be expressed as follows:

[0063]

[0064]

[0065]

[0066] Here, The node It indicates the calculated rank of the features used in and is a node in a pre-ordered dedicated threshold group It is the address of the threshold value.

[0067] FIG. 4 illustrates an exemplary embodiment of a computation unit. A rank calculation block votes for a corresponding class by providing a number of corresponding ranks to a tree processing block (2), i.e., input to the computation unit. One implementation of the computation unit is similar to a parallel architecture as illustrated in FIG. 14. The computation unit includes one comparator (22) for each tree node. However, the comparator of the proposed concept decodes rank addresses and does not perform actual node comparisons. In practice, addresses represent comparisons and can instead be used directly for the calculation of a Boolean function. Address decoding can be minimized, for example, to a tree voting function by a synthesis tool when mapped to FPGA or ASIC technology.

[0068] Some of the advantages of the proposed concept include the following:

[0069] Memory needs to contain only the threshold values ​​of the corresponding features. The node's feature index and child addresses do not need to be stored, which significantly reduces memory usage.

[0070] Duplicates of grouped thresholds can be removed, for example, by binary search, as they do not bring additional information regarding memory access. This allows for even further reductions in memory size and memory access energy.

[0071] Compared to serial architectures, feature inputs do not need to be fetched from memory or input multiplexers. This reduces circuit complexity and energy consumption.

[0072] Decision trees are generally unbalanced, which can lead to varying execution times for different decisions. This makes parallelization between trees more difficult and requires synchronization. The proposed concept suggests a balanced search through thresholds, for example, using binary search, thereby reducing circuit complexity and execution time. Although execution time may vary from tree to tree, it can be adjusted by introducing parallelism into search comparisons, allowing it to remain constant from decision to decision.

[0073] Binary search performs significantly fewer comparisons than parallel architectures that compare all of them. In some cases, binary search computes even fewer comparisons than serial architectures, particularly when long branches occur in the tree. This reduces execution time, energy, or circuit size, depending on the amount of parallelization.

[0074] The advantages discussed above can also be seen in the experimental results. To compare the number of comparisons for serial and parallel architectures, two models were trained on different datasets and statistically analyzed using the corresponding test dataset. For the parallel architecture, the number of comparisons is simply the number of nodes in all trees. For the serial architecture, tree iterations were performed on the actual test data to statistically evaluate the number of comparisons per inference. Similarly, binary search was performed on the test data to obtain the number of comparisons for the proposed architecture.

[0075] Table 1 shows the statistical or exact number of comparisons for two models trained on different datasets. Since the execution time of serial architectures is irregular due to imbalanced trees, the table includes (minimum, median, maximum) tuples.

[0076]

[0077] Table 3: Exemplary architecture and number of comparisons for two different datasets.

[0078] To compare the hardware domain and energy consumption of the architectures, three of them were implemented in SystemVerilog and synthesized on commercial FPGAs. The hardware domain figure of merit (FOM) was based on flip-flop (FF) and look-up table (LUT) resources after synthesis. It is defined as. Energy consumption is evaluated based on switching activities simulated with actual test data classification using global estimation tools from FPGA vendors.

[0079] Figures 5 and 6 illustrate the size FOM and energy comparisons of all three architectures for several models trained on different datasets. The results clearly show that the new architecture performs well in both figures of merit and even outperforms the serial architecture in size FOM for most datasets. On the other hand, the serial architecture exhibits the worst energy efficiency, and the parallel architecture exhibits the worst performance in size FOM. This proves that the architecture is the best candidate for making it widely usable and for balancing the trade-off between size and energy by introducing parallelization.

[0080] FIG. 7 illustrates another exemplary embodiment of a computation unit. As discussed above, the rank calculation block (1) provides a rank that is input to the tree processing block (2), i.e., the computation unit (10), to evaluate or vote on a class. By using the rank, the computational effort required to perform the evaluation of the decision tree by the feature values ​​can be reduced compared to working directly on the feature values. In practice, the computation step can be performed in advance and made available in a lookup table (17) (or LUT). Since a single LUT (whose input consists of all features) can be very large, one LUT (17) per feature is used. As illustrated in the figure, the LUT receives the rank from the rank calculation block and outputs intermediate results tmp_results. The intermediate results can be processed by relatively simple structured arithmetic (or "combiner" 18) to process the final results of the classification.

[0081] This results in decisive advantages compared to more naive implementations, including, for example, high speeds along with significantly lower energy requirements. The classification consists of several parallel and independent memory accesses and a single, energy-efficient, and parallel arithmetic. The use of LUTs allows for high flexibility, as the contents of the LUT can be freely selected even after fabrication in ASIC technology. Furthermore, the hardware structure can be highly uniform, and the complete reconfiguration of the combiner is possible. This allows for the mapping of any DTE model.

[0082] FIG. 8 illustrates a combiner of an exemplary embodiment for calculating classification results from an LUT. The figure may be used to demonstrate LUT principles in a simplified or high-level manner. A more detailed exemplary implementation is further described below.

[0083] The combiner (18) includes a plurality of lookup tables. In fact, in the drawing, rank[i] Displayed as , There is a single dedicated LUT that receives the corresponding ranks. In other words, for each feature i, there is a single dedicated LUT[i] There is. The LUT originates from the training procedure and contains trained entries for each value of the corresponding rank representing the structure and nodes of each decision tree. The LUT is an intermediate result tmp_result[i] Outputs. The intermediate result may be further processed to combine multiple features using voting of a sub-ensemble consisting of multiple trees and / or tree voting. Voting may be further processed and, for example, weighted in the multiplier by each weight denoted by w[m], where and M is the total number of trees and / or sub-ensembles. An adder can combine intermediate results (weighted or unweighted) into a combined intermediate result. These results are input to a comparator to determine the threshold value c_thr A comparison with is performed to present the classification result. The latter part, starting from weighting, may already be part of the aggregation unit.

[0084] In contrast to the exemplary embodiment without additional LUTs, the contents of the LUT need to be additionally determined. The contents are determined according to predetermined criteria. For example, the contents (or LUT entries) will represent an equivalent description of the hardware architecture. In this way, LUT entries can describe hardware typically implemented as an ASIC, or variations in a set of features. LUT entries are highly configurable and can be discovered by training based on a training model using solver software, such as the open-source software Z3 Theorem Prover. For example, on the one hand, a description of the hardware structure and on the other hand, a description of the DTE are input into the solver. The solver then [instructs] the LUTs LUT[i] The assignment of the entry is found in, where the two descriptions are equivalent for all possible input assignments, namely rank or feature value.

[0085] The use of solvers, such as SMT / SAT solvers, supports the fact that hardware implementations adhere to and match specified models. The use of solvers for calculating LUT entries has already been described in technically different fields of FPGA logic synthesis. However, due to its high complexity, it is not actually used. The proposed concept involves the use of solver software in the fields of Artificial Intelligence (AI) and Machine Learning (ML).

[0086] By separating the LUT content from explicit coding, the DTE model can be significantly compressed. Compared to the explicit assignment of leaf nodes in every tree for each single output bit per LUT, a compression rate of up to 400% has been achieved so far. Furthermore, the proposed architecture does not use the actual values ​​of features directly, but uses their ranks as addresses for the LUT. In the tree of the DTE for each node, starting from the root, feature values ​​are compared against trained thresholds. For each feature, there are several thresholds that define intervals. Ultimately, the values ​​of important information lie within such intervals. If the thresholds are sorted in ascending or descending order according to sorting criteria, the index of the interval within which the feature value is located corresponds to its rank, as implemented in the rank calculation block. The number of intervals is clearly lower than all possible values ​​that a characteristic can assume. Therefore, fewer bits are also required for encoding. The bit width of a feature or its rank corresponds to the bit width of the LUT address input. For example, if a 16-bit feature can be represented by a 4-bit rank, the LUT must consist of only 16 entries instead of 16,384 entries. This reduction in features relative to rank is made possible by using pre-sorting.

[0087] The described concept of classifier hardware (a combiner for calculating classification results from LUT intermediates) can further improve the proposed concept. Training to pre-calculate the contents of the LUT, such as using a solver like an SMT / SAT solver, provides improved results in terms of both circuit size and memory. The use of ranks instead of actual feature values ​​enables a minimal reduction in memory size. Furthermore, the combiner logic can be small and energy-efficient despite its complete reconfigurability. In this way, there are no limitations on the grouping of LUT outputs and, consequently, the generation of contents. The summation of variable-bit-width groups and simple comparison with thresholds can be particularly emphasized.

[0088] FIG. 9 illustrates an exemplary embodiment of in-memory rank calculation. Although the aforementioned near-memory calculation of rank (i.e., based on binary search operations) is already very efficient and fast, there may be applications that exhibit possible bottlenecks in the performance of the classifier hardware. An estimate of log2 of the number of thresholds per clock cycle of features (or corresponding bit widths of ranks) may be required. All other stages of classification typically have a throughput of one classification per clock cycle through pipelining. An additional concept based on comparator logic allows for the calculation of feature ranks in memory itself.

[0089] The drawing illustrates a computation unit (10) for a feature to be classified. The unit includes memory (11) for a given feature and a grouped pre-sorted threshold. Furthermore, the unit includes logic (12) that performs a comparison with the threshold. Based on the comparison, the logic outputs a rank in 1-hot encoding, rather than binary, which is then input into the corresponding lookup table. Instead of binary coding, there may be one line per possible state value. Only one line corresponding to the current rank is active ("hot"). Depending on the entry of the lookup table LUT entry, an intermediate result tmp_result is output by the computation unit.

[0090] The structure is similar to that of the Content Addressable Memory (CAM) complementary lookup table approach. However, in contrast to a regular CAM, the computational unit performs comparisons for "<" (or ">") instead of "=". This means that match lines of all threshold memory cells having values ​​less than (or greater than) the applied feature value are asserted. This implements the thermometer-encoding of ranks. The limits for the next larger (or smaller) threshold values ​​are determined from the thermometer-encoding and can be converted to 1-hot encoding by inverters and gates per memory row. The result can be used directly, for example, as a word line for the LUT.

[0091] Most of the peripherals (e.g., external search logic, address decoders for LUT memory) are no longer required in these exemplary embodiments. Since the memory is also used for read-only purposes, the memory cells can be optimized for leakage current (e.g., by using Schmitt trigger structures or ULP diodes) and read performance. Write operations are required only to initialize the contents. This also enables the use of NVRAM or ROM to increase power and area efficiency. In addition to the performance gains of rank calculation per clock, additional power and area can be saved. Instead of logarithmic memory accesses for binary search, only a single access is required. For this, an address decoder or similar is also not required, but the energy consumption of the match line is added. Since this is likely located in an area similar to that of the peripherals, it ultimately results in similar energy consumption per access.

[0092] FIG. 10 illustrates another exemplary embodiment of a combiner for calculating classification results from an LUT. The simplified structure of the combiner shown in FIG. 8 is described in more detail for one possible implementation. As can be seen here, although the complexity is somewhat higher, the structure is still very regular and fully reconfigurable.

[0093] rank[i] One dedicated to LUT[i] is described. Individual bits of the intermediate LUT result are having tmp_result[i][x] Named as X is the total bit width of the LUT. These individual bits tmp_result[i][x] is configurable bit width p[q] of Q Non-nested binary values c[i][q]으 They are joined adjacently, Is Q It is the index of individual values. These binary values c[i][q]은 All features, that is f having i Summed up over It causes. Then, the sum s[q]은 Corresponding sum-threshold value st[q] It is compared with, that is, v[q] = s[q] < st[q] is. Comparison result v[q] is an adjacent, non-nesting group of configurable size by AND-reduction across M Group voting It is combined. The results of these groups can correspond to the voting of individual trees and / or sub-ensembles of the entire DTE. The group vote g[m] is an adjustable weight. w[m] Scaled by It can be summed as. Any class to which the record ultimately belongs, As such, threshold value c_thr It is determined by. The latter part starting from weighting may already be part of the aggregation unit.

[0094] FIG. 11 illustrates further details of an exemplary embodiment of a combiner. The implementation of individual parts comprises several features, namely It is explained below using an example, and (LUT) is explained. Step 1 (binary number c[i][q] index of The selection / combination and addition of individual bits with x are performed by a multiplexer and an adder. Here, the LUT output is first transformed and added column-wise; that is, the 7 LSBs of all LUTs (index i) are added to the 3-bit value. The result is added to the 3-bit value of the next column (after effectively multiplying the latter by 2, i.e., shifting it left by 1 bit). Therefore, the value c[i][1] Corresponding to the bit width of p[1] fever s[1] , that is, 7 added p[1] - Causes the sum of bit values. The following sum s[2] Contributing to Index For the next column having, the previous value and tmp_result[i][p[1]] of Instead of 3 bits from the sum, the constant "0" is selected by the multiplexer, and thus the adder chain is interrupted. Word width of the LUT X For each column or each bit x, a configuration bit containing the settings of the corresponding multiplexer cfg_mux[x] exists. Therefore, all c[i][q] The bit width of can be freely configured. The threshold value 2^(p[q]-2) Comparison by limiting to s[q] < st[q](= 2^(p[q]-2)) It is also simple. s[q] 2 from Only the most significant bit (MSB) is checked for the value "00", and the least significant bit (LSB) is irrelevant. If the bit is not set, then v [q] = 1. To achieve complete reconfigurability of the classifier, each column can be implemented identically, for example, including a 3-bit multiplexer having construct bits and 2-bit comparator logic as well as a 3-bit adder, but only the result where the next multiplexer (having index x+1) is set to break the adder-chain is relevant and corresponds to the value depicted in FIG. 10. The remaining result s[x], v[x] is ignored.

[0095] FIG. 12 illustrates further details of an exemplary embodiment of a combiner. The figure illustrates the AND-OR chain of the combiner. As one next step, comparator logic v[q] The result of value g[m] It is ANDed down to. This is performed by the AND-OR chain described across all comparator-results. Construct bits cfg_mux[x+1]는 Irrelevant values ​​to consider only related values v[x]을 Can be masked . In addition, the configuration bits of the additional set cfg_and[x] blocks the AND-OR chain at the boundary between groups and corresponds to the value depicted in Fig. 10, where Phos, related value g[x] to be used to determine It is possible.

[0096] Votes (or classifications) from the combiner unit can be evaluated by a separate aggregation unit that performs majority voting. Optionally, a 1-bit value g[m] is weight w[m] It can be multiplied by. This operation is g[m]=0 Muting the weights for This can be performed by a set of AND gates or a multiplexer, but may not be necessary for the applications discussed herein. A column having a value not to be considered x It consists of weights "0". The result of weighting is the sum c_sum It is summed into an adder tree. Adjustable threshold value c_thr을 The comparator that has Finally, output the class corresponding to the feature on the classifier's input.

[0097] So far, the aforementioned AND / OR array, XOR circuit, and arithmetic used to calculate the total result from the LUT values ​​have been examined. Although the proposed implementation already works well, it cannot be ruled out that a more compact and more efficient structure exists to solve this task and further increase the compression ratio.

[0098] While this description of the improved concept contains many details, it should not be interpreted as a limitation on the scope of the concept or what has been claimed or may be claimed, but rather as a description of features specific to specific embodiments of the present invention. Specific features described in this disclosure in conjunction with separate embodiments may also be implemented in combination in a single embodiment. On the other hand, features described in relation to a single embodiment may also be implemented individually in several embodiments or in any suitable sub-combination. Furthermore, while features may be described as operating in a specific combination and even originally claimed as such, in some cases, one or more features may be omitted from the claimed combination, and the claimed combination may be directed toward a sub-combination or a variation of the sub-combination.

[0099] Therefore, even if the operations in the drawings are presented in a specific order, this should not be understood to mean that these operations must be performed in the order depicted or in a sequential order, or that all depicted operations must be performed to obtain the desired result. Under certain circumstances, multitasking and parallel processing may be advantageous.

[0100] Many implementations have been described. Nevertheless, various modifications may be made without departing from the spirit and scope of the invention. Accordingly, other implementations are within the scope of the claims. Explanation of the symbols

[0101] 1st priority calculation block 2 Tree Processing Blocks 10 calculation units 11 threshold memory 12 Logic 13 Input Comparator 14 Comparator 15 Comparator Input 16 Comparator Output 17 Lookup Table 18 combiners 20 tree processing units 21 tally units

Claims

Claim 1 A classification device for classifying a decision tree ensemble comprising a ranking calculation block (1) and a tree processing block (2), wherein the ranking calculation block (1) comprises one calculation unit (10) per feature of a set of features to be classified, and each calculation unit (10) comprises memory (11) and logic (12), wherein the memory (11) is configured to store a threshold value of the decision tree in the decision tree ensemble, wherein the memory is arranged to store the threshold value in one group per feature, and the threshold values ​​of the group are sorted according to a threshold index, wherein the logic (12) is arranged to perform a node comparison of the threshold value of the corresponding threshold value group and output a rank as a result of the comparison, wherein the rank represents an encoded address representing the threshold value, wherein the tree processing block (2) for the tree of the decision tree ensemble is configured to determine the class of the feature to be classified as a function of the rank, and the feature is a feature index f having i A classification device, wherein I represents the number of features to be classified, and the threshold represents a node of the tree of the decision tree ensemble, where: - the logic (12) is arranged to receive features to be classified and to perform a search that returns the rank for each feature that divides the threshold into values ​​smaller or larger than the feature value, - the rank is output to each corresponding tree processing unit (20), and - the tree processing unit (20) is configured to determine the class by evaluating each Boolean function representing the decision tree of the decision tree ensemble as a function of the received rank. Claim 2 delete Claim 3 A classification device according to claim 1, wherein the tree processing block (2) includes an aggregation unit (21) arranged to receive the determined class from the tree processing unit (20) and form an ensemble vote as a function of the determined class. Claim 4 In claim 1, the memory (11) comprises: - a group of memory cells for each feature from the set of features to be classified, and - the memory cells within the group are addressable according to the threshold index, a classification device. Claim 5 In claim 1, - the logic (12) comprises a comparator (14) comprising a first input (13) operable to receive a feature and a second input (15) operable to receive a threshold value from the memory (11), and - the comparator (14) is operable to perform a node comparison of the threshold value of a corresponding threshold value group, a classification device. Claim 6 In paragraph 5, the tree processing unit (20) comprises one or more comparators (22) arranged to perform address decoding of the received ranks in order to evaluate the Boolean function based on the address decoded ranks, in a classification device. Claim 7 A classification device according to claim 1, wherein each of the calculation units (10) is connected to a dedicated lookup table (17), the lookup table (17) includes an input side operable to receive a dedicated rank from the calculation unit (10) connected to the lookup table, and the lookup table (17) is operable to output an intermediate value as a function of the rank received at the input side. Claim 8 In claim 7, - each of the ranking calculation blocks (1) is respectively connected to a dedicated lookup table (17), - the lookup table (17) includes an output side operable to provide the intermediate value of the tree processing unit (20) connected to the lookup table, and - the tree processing unit (20) is operable to determine the class of the feature to be classified as a function of the received intermediate value, a classification device. Claim 9 In claim 7, the classification device further comprises a combiner (18) having an arithmetic unit, which is operable to receive the intermediate value and to combine and / or weight the intermediate value and output the class of the feature to be classified. Claim 10 A classification device according to claim 7, wherein the processing block (2) comprises one or more lookup tables, and the lookup tables are implemented based on an associative memory comprising one or more comparators operable to perform "less than" or "greater than" comparisons. Claim 11 A method for classifying a decision tree ensemble performed by a classification device, comprising: - storing threshold values ​​representing the decision trees of the decision tree ensemble as one group per feature to be classified; - sorting the threshold values ​​of the group according to a threshold index; - performing a node comparison of the threshold values ​​of the corresponding threshold value groups; - outputting a rank as a result of the node comparison, wherein the rank represents an encoded address representing the threshold value; - determining the class of the feature to be classified as a function of the rank, wherein the feature to be classified is a feature index f having i A method defined as, where I represents the number of features to be classified, and the threshold represents a node of the tree of the decision tree ensemble, and the method further comprises: - receiving features to be classified by logic (12), and using the logic to perform a search that returns the rank for each feature that divides the threshold into values ​​smaller or larger than the feature value, the threshold being smaller or larger than the feature value, the step of outputting the rank to each corresponding tree processing unit (20), and - determining the class of the features to be classified by evaluating a Boolean function representing each tree of the decision tree ensemble as a function of the received rank. Claim 12 delete Claim 13 In paragraph 11, the above ranking is determined from a binary search using the above logic (12), or the above ranking is determined from in-memory. Claim 14 In paragraph 11, the evaluation of the decision tree comprises a lookup table (17) having a pre-calculated value stored as a lookup table entry, and the class is determined according to the lookup table entry. Claim 15 In paragraph 14, the method wherein the lookup table entry is determined by training software that determines the assignment of the lookup table entry such that both the classification architecture and the mathematical description of the DTE are equivalent.

Images