Accelerating decision tree inference

Abstract

A method, computer program product, or system, or combination thereof, is provided that performs the operations of: setting up a memory buffer containing contiguous memory blocks; obtaining a decision tree including nodes including branch nodes and leaf nodes, each of the branch nodes including at least two child nodes ordered according to a likelihood of accessing the child node after each of the branch nodes; mapping the nodes to respective blocks of memory blocks, each of the memory blocks storing attributes of a corresponding one of the nodes, each of the branch nodes and child nodes of each branch node being mapped to successive blocks, and the same one ordered child node of the branch node being mapped to successive blocks; and executing the nodes by processing the attributes of the nodes as they are accessed from memory according to the order of the memory blocks in the memory buffer.

Description

Technical Field

[0001] The present invention generally relates to the field of decision trees. Specifically, the present invention aims to accelerate decision tree inference.

Summary of the Invention

[0002] According to an aspect of the present invention, an operation of setting a memory buffer including adjacent memory blocks in a memory, an operation of obtaining a decision tree including nodes including branch nodes and leaf nodes, each of the branch nodes including at least two child nodes ordered according to a likelihood of accessing child nodes after each of the branch nodes, an operation of mapping nodes to each block of the memory blocks, each of the memory blocks storing a corresponding one of the attributes of the nodes, the mapping being performed according to a depth-first search order, each of the branch nodes and any child nodes of each branch node being mapped to successive blocks of the memory blocks, and the same ordered child nodes of the branch nodes being mapped to successive blocks of the memory blocks, an operation of executing a node by processing the attributes of the nodes as accessed from the memory according to the order of the memory blocks in the memory buffer, and an operation of returning an inference result based on the execution result of the node (not necessarily in this order) are provided, in the form of a method, a computer program product, or a system, or a combination thereof.

[0003] These and other objects, features, and advantages of the present invention will become apparent from the following detailed description of embodiments of the present invention read in conjunction with the accompanying drawings. Each figure is provided to clarify in an easy-to-understand manner for those skilled in the art to understand the present invention together with the detailed description.

Brief Description of the Drawings

[0004] [Figure 1]This figure shows the first decision tree, which includes branch nodes and leaf nodes, according to an embodiment of the present disclosure. [Figure 2] This figure shows a rearranged version of the decision tree in Figure 1, where, according to embodiments of the present disclosure, some nodes are swapped according to the likelihood of access, taking into account the mapping of node attributes to blocks of memory buffers. [Figure 3] Figure 2 shows the selection of connected nodes in a decision tree according to an embodiment of the present disclosure, along with the feature identifiers and thresholds of the nodes used to execute such nodes. [Figure 4A] This figure shows a block of memory buffers to which node attributes are mapped according to an embodiment of the present disclosure. Each branch node is mapped to an 8-byte memory block, and each block is broken down into two 4-byte subblocks. According to an embodiment of the present disclosure, each leaf node is mapped to a 4-byte block. [Figure 4B] This figure shows a block of memory buffers to which node attributes are mapped according to an embodiment of the present disclosure. Each branch node is mapped to an 8-byte memory block, and each block is broken down into two 4-byte subblocks. According to an embodiment of the present disclosure, each leaf node is mapped to an 8-byte block. [Figure 5] This flowchart shows the high-level preliminary steps performed to obtain a decision tree containing appropriately ordered nodes, according to embodiments of the present disclosure. [Figure 6] This flowchart shows high-level steps of a method for accelerating inference by executing reordered nodes of a decision tree according to embodiments of the present disclosure. [Figure 7] This figure schematically represents a general-purpose computerized system suitable for carrying out one or more steps of the methods relating to embodiments of the present invention. [Modes for carrying out the invention]

[0005] The attached drawings show simplified representations of devices or parts of devices involved in the embodiments. Similar or functionally similar elements in each figure are assigned the same reference numeral unless otherwise indicated.

[0006] Herein, computerized methods and computer program products embodying the present invention are described as non-limiting examples.

[0007] A part of this disclosure provides a method, system, computer program product, or combination thereof for accelerating decision tree inference. In particular, embodiments of the method, system, computer program product, or combination thereof provide swapping some nodes of a decision tree according to the likelihood of accessing those nodes. Nodes are executed by processing the attributes of nodes accessed from memory according to the order of memory blocks in a memory buffer, and the attributes of the decision tree nodes are mapped to memory blocks according to a depth-first search order.

[0008] Decision tree learning is a predictive modeling technique used in machine learning. Decision tree learning relies on one or more decision trees to form a predictive model. Decision trees are widely used machine learning algorithms due to their simplicity and interpretability. Various types of decision trees are known, including classification trees and regression trees. A binary decision tree is essentially a structure containing a coupled decision process. Starting from the root, features are evaluated, and one of the two branches of the root node is selected. This procedure is repeated until a leaf node is reached, and the value of this leaf node is used to construct the final result.

[0009] Random forests and gradient boosting are important machine learning methods based on binary decision trees. In such methods, multiple decision trees are "explored" in parallel until leaf nodes are reached. The results obtained from the leaf nodes are then averaged (regression) or used in a majority vote (classification). Such computations can be time-consuming (and resource-intensive), and therefore, there is sometimes a need to accelerate tree-based inference, especially for random forests, gradient boosting, or both.

[0010] Accelerating tree-based inference, such as random forest methods and gradient boosting, can be achieved by (i) speeding up individual binary decision tree processing, or (ii) speeding up parallel processing of multiple binary decision trees, or both. While this invention focuses on the first method, embodiments also address the second method.

[0011] Herein, with reference to Figures 2-6, a first aspect of the present invention that provides a method for accelerating decision tree inference is described in detail. Embodiments of the method may basically include setting up a memory buffer in S20, as shown in Figure 6, mapping appropriately placed nodes of the decision tree to blocks of the memory buffer in S30 and S40, executing the nodes in S50, and returning the results in S60. Note that this method and its variations may be collectively referred to as the “Method” in this disclosure. All references Sn refer to steps of the method in the flowcharts of Figures 5 and 6, while reference numbers relate to physical parts or components of the computerized unit shown in Figure 7, and concepts used in embodiments of the present invention (such as decision trees and decision tree nodes).

[0012] In detail, the memory buffer is set up in the memory of the computerized unit 101 or computerized system used to carry out the method (in step S20 of Figure 6). This memory buffer includes adjacent memory blocks to which no data has been added at this point.

[0013] Next, decision tree 20 is accessed (step S10 in Figure 6). It is assumed that this tree is properly rearranged. That is, given the initial tree 10 (Figure 1), at least some of the nodes 110 are reordered according to the likelihood of accessing the nodes in order to obtain decision tree 20 as shown in Figure 2. Nodes 120 include branch nodes and leaf nodes. In the examples in Figures 1 and 2, branch nodes are indicated by references SN0 (corresponding to the root node) to SN14, while leaf nodes are indicated by references LN0 to LN15.

[0014] After the tree has been rearranged (Figure 2), each branch node contains at least two child nodes, ordered according to the likelihood of subsequent access to that branch node. In other words, within each layer, child nodes of the same parent are ordered according to the likelihood of access such that the first child node of a particular branch node is statistically more likely to be accessed than the second child node of the same branch node, and so on. In terms of the graphical representation of tree 20, this essentially means that all child nodes are ordered along the same direction, such as from the same side of the decision tree 20 (for example, from the left side in the illustrative tree in Figure 2).

[0015] Multiple methods can be used to determine the likelihood of access. The ordering of nodes 120 can, in particular, be based on the conditional probability of a node's execution, e.g., the probability of a parent node accessing a particular child after it has been executed. If a clear path (or order) of nodes that is "hotter" with respect to the decision path can be identified than a path through surrounding nodes, child nodes can be swapped so that the most likely node is always on the left (or always on the right). For example, the likelihood of access can be determined using training data or any other representative data (e.g., validation data, test data, etc.), as will be described in detail later.

[0016] Next, the reordered nodes 120 are mapped to each block of the memory blocks of buffers 31, 32 that were previously set up (in step S30). As a result of the mapping operation in step S30, each memory block stores the attributes of the corresponding node 120, which may be either a branch node or a leaf node. The attributes of a node may include operands required to perform the node. The attributes of a node may include, for example, a feature identifier (also called a feature selector) or a threshold used for comparison, or both, and more generally, all arguments required to evaluate the rule captured by the node of the decision tree. Each branch node of the decision tree is labeled with a feature identifier and associated with a threshold for performing an operation, thereby comparing the feature value corresponding to the feature identifier with the threshold, for example, as is known in itself. This is shown in Figure 3, which shows the selected nodes 120 of the tree 20 along with the values ​​of each feature identifier ("feature ID") and thresholds.

[0017] Interestingly, in some embodiments, in S30, mapping may be performed according to depth-first search (DFS), as opposed to the breadth-first order typically used in decision trees. As a result, each branch node and all of its child nodes are mapped to a continuous block of memory blocks. On the other hand, since at least some of the nodes 120 are reordered, the DFS mapping also results in ordered child nodes of the same parent node (branch node) being mapped to a continuous block.

[0018] Importantly, "successive" does not necessarily mean "consecutive," that is, directly following the previous one. Rather, "successive" means coming after something else. For example, 1, 5, 9, 18 are consecutive numbers, but not consecutive numbers (like 1, 2, 3, 4). However, the first child node of each branch node (i.e., the node that is more likely to be accessed after that branch node during operation) can be favorably mapped to a memory block that is contiguous with respect to the block corresponding to that branch node. Note that each memory block to which a node's attribute is mapped may, for reasons that will become clear later, actually consist of two contiguous subblocks.

[0019] Subsequently, in S50, node 120 is executed by processing the attributes of node 120 as they are accessed from memory. However, such attributes are accessed according to the order of the memory blocks in memory buffers 31, 32. For example, in some embodiments, the access order is imposed by the order set in memory buffers 31, 32 at the time after the mapping operation in S30. Note that memory buffers 31, 32 may typically be implemented in software as virtual data buffers, but in principle, they may also be implemented in hardware at specific memory locations of that hardware. Memory buffers can be advantageously used to map the data they contain to adjacent memory cache lines, as in some embodiments described later.

[0020] Finally, the inference result is returned in step S60. This result is based on the execution result of node 120. Multiple decision trees may be executed similarly, as in a set model (e.g., a random forest or gradient boosting model). Thus, for example, for classification or regression purposes, the final overall result may be constructed based on the inference results obtained from each decision tree.

[0021] According to this embodiment of the method, the block structure of the memory buffers 31, 32 used to execute the nodes reflects the arrangement of the decision tree 20 obtained after reordering at least a portion of the nodes 120. That is, for each branch node of the decision tree 20, the first child node is always on the same processing side. The memory blocks of the memory buffers are located at contiguous offsets, which do not need to be constant but may be. Using constant offsets for branch nodes and leaf nodes facilitates relying on implicit ordering of blocks, so that explicit pointers to child blocks (i.e., blocks corresponding to child nodes) do not need to be stored in the memory blocks.

[0022] The proposed method can bring multiple advantages. First, for example, due to the optimized order of memory blocks, the proposed method can enable better utilization of the spatial locality of data, whether the data is within a memory block or within a cache line to which such a block is mapped. As an example, at the time of node execution, a single cache line containing the attributes of the nodes on the most likely execution path can be read. The single cache line can, in some cases, contain all the necessary attributes required to execute all the nodes on that path. This can then result in higher cache performance and utilization of memory bandwidth in the cache and main memory. That is, fewer memory accesses may be required.

[0023] Various embodiments of the present invention bring a significant improvement in inference latency and throughput in addition to a significant reduction in storage requirements. In particular, the embodiments can shorten the inference time to (about) 1 / 1.5 to 1 / 70, and this shortening rate is determined by the test data, reference method, and platform being considered. In practice, however, the inference time can usually be shortened to 1 / 2 to 1 / 3.

[0024] Furthermore, the present disclosure can advantageously support vector processing of decision trees, as will be described below with reference to some embodiments. As a result, multiple nodes of the same decision tree can be processed in parallel, resulting in higher processing efficiency in the case of a specific memory access granularity (e.g., cache line). The present invention can be further applicable to multiple inference platforms, including, for example, a central processing unit (CPU) and a field programmable gate array (FPGA).

[0025] All of this will now be described in detail with reference to a particular embodiment of the present invention. Firstly, as previously described, memory buffers 31, 32 can be favorably mapped to cache lines of cache memory used to execute the node, in accordance with the order of blocks set in the memory buffers (step S40 in Figure 6). That is, the memory blocks of memory buffers 31, 32 are mapped in order to the memory blocks of the cache lines, in accordance with the same order of memory blocks set in buffers 31, 32, as shown in Figure 4B. Thus, in S50, node 120 is executed by processing the attributes of node 120 as obtained from the memory blocks of the cache lines.

[0026] In particular, note that when node 120 is executed by a CPU, embodiments of the present disclosure can be implemented using any cache level. In implementations using an FPGA, cache lines can be fetched from main memory and supplied to the FPGA's internal memory, which can be considered cache memory for the FPGA.

[0027] Advantageously, in some embodiments, the memory blocks corresponding to branch nodes do not need to contain pointers to the child nodes of these branch nodes. Rather, embodiments of the present disclosure can rely on the implicit ordering of memory blocks. To simplify this, in some embodiments, it may be imposed that all memory blocks corresponding to branch nodes have the same size S, while all memory blocks corresponding to leaf nodes may have the same size L. Note that L does not need to be equal to S. In practice, L can usually be less than or equal to S. Knowing the offsets between memory blocks makes it easier to determine from where in memory to access the attributes of node 120, according to the implicit ordering of memory blocks resulting from the selected mapping. Thus, it is not necessary to store pointers in the memory blocks.

[0028] The following describes how relevant memory blocks may be retrieved, with reference to some preferred embodiments. Firstly, some parent-child node pairs may be mapped in a manner favorable to contiguous blocks (not merely continuous). That is, in S30, any two memory blocks consisting of a branch block and a first child block may be mapped such that they become contiguous memory blocks in memory buffers 31, 32. The branch block corresponds to a branch node in tree 20, while the first child block corresponds to the first child node of that branch node (in their ordered state). This allows for more efficient memory access and processing, as the attributes of the most likely child node can be read from the memory block immediately following the block corresponding to the parent of that child node.

[0029] In some embodiments, as shown in Figures 4A and 4B, for example, all memory blocks corresponding to branch nodes may have the same size of S bytes (e.g., S=8), while all memory blocks corresponding to leaf nodes may have the same size of L bytes, for example, S=4 in Figure 4A or S=8 in Figure 4B. Therefore, in memory buffers 31 and 32, the memory blocks following the memory blocks corresponding to branch nodes are offset by S bytes, while the memory blocks following the memory blocks corresponding to leaf blocks are offset by L bytes. Thus, to access the attributes of the first child node after processing the attributes of the parent node, it is sufficient to read data shifted by 8 bytes in buffers 31 and 32 or the corresponding cache lines. In the unlikely situation where a second child node is executed instead of the first child node, the attributes of this node can be accessed by reading data shifted by a larger number of bytes. Therefore, instead of relying on pointers, the implicit ordering of blocks can be utilized in this way. As will be explained in more detail later, precise rules can be devised to gain further access to the nodes.

[0030] As can be understood, the fact that nodes 120 in tree 20 are (at least partially) reordered can be captured simply by appropriate flags stored with the node attributes, as illustrated here with reference to Figures 4A and 4B. That is, in some embodiments, the mapping operation performed in step S30 may include setting a flag for each branch block indicating the operation to be performed when this branch node is executed. This flag is a value set according to the ordering of the child nodes of that branch node, for example, according to how the child nodes of this branch node are ordered in tree 20 according to the likelihood of access to the child nodes. In detail, due to the lateral ordering of the child nodes of each branch node at each tree level, all the most likely nodes are on the same side laterally of tree 20 (e.g., the left side in Figure 2). Thus, parent nodes swapped out of the default arrangement that the parent nodes had in the original tree 10 (Figure 1) can favorably include flags that determine the mathematical operations performed as part of executing such parent nodes. Swapping nodes in this way essentially means simply changing the flags associated with the parent nodes of those nodes. The block corresponding to the child node most likely to be accessed after a branch node can be directly adjacent to the block corresponding to that branch node, and the block corresponding to that branch node does not need to contain a pointer to the child node. The algorithm knows the value of the shift to consider from the flag value.

[0031] In some embodiments, the decision tree 20 can typically be a binary decision tree. In that case, the flag indicates one of two complementary mathematical comparison operations, for example, based on an inequality. As an example, one operation may consist of comparing whether the feature value x is strictly less than a threshold value t (e.g., x < t), in which case the complementary operation consists of comparing whether x is greater than or equal to t (e.g., x ≧ t). Conversely, one of the operations can be ">" and in that case the complementary operation will be "≦". In other words, the two complementary mathematical comparison operations can consist of testing whether x < t or x ≧ t is true, or whether x > t or x ≦ t is true. More generally, in cases other than binary decision trees where one or more of the branch nodes of the tree may include more than two child nodes, a set of complementary operations can be devised similarly.

[0032] In Figure 2, the parent node of the swapped child node is marked with an asterisk, which in Figures 4A and 4B corresponds to the block that stores a flag value equal to 1. In the examples in Figures 4A and 4B, the memory block is constructed on subblocks of 4 bytes each. The length of each branch block is S = 8 bytes. The first two subblocks are left empty, according to an optional design choice aimed at ensuring compatibility with other computerized methods. The root node (SN0) is then mapped to the following two subblocks. The first 4-byte subblock is used to encode both the feature selector (FT) and the flag value, while the second 4-byte subblock is used to encode the comparison threshold (TH). For example, the first subblock (4 bytes) may contain a 31-bit feature selector (FT) and a 1-bit flag value, where, for example, bit value 0 corresponds to "<" and bit value 1 corresponds to "≧". Therefore, the flag can be efficiently encoded and occupy less memory space than a pointer, which can be a significant advantage in methods involving hundreds or thousands of trees, each containing thousands of nodes. The second subblock (4 bytes) stores a floating-point threshold (TH), which is a single-precision value. Each result value returned by a leaf node can be a single-precision value mapped to a single 4-byte subblock, as in Figure 4A, or a double-precision value stored in two subblocks (totaling 8 bytes, as in Figure 4B). In variations, the result values ​​can correspond to integers, for example, for classification purposes, and can be mapped to a smaller number of bytes.

[0033] As previously mentioned, precise rules can be devised for retrieving relevant blocks. In some embodiments, this implicit ordering is retrieved according to the following rules: Firstly, the offset of the first child block (corresponding to the first child node of this particular branch node) is equal to the offset of the branch block corresponding to this branch node plus S bytes, as previously described. Secondly, the offset of the second child block (corresponding to the second child node of this branch node) is equal to the offset of the corresponding branch block plus 2(R-1)×(S+L) bytes, where R is the number of tree levels in the decision tree 20 below the tree level in which the branch node under consideration resides.

[0034] For example, in the case of the root node SN0 of tree 20 shown in Figure 2, R=4. Again, all child nodes of the same branch node are ordered from left to right at each level of decision tree 20 according to the likelihood of accessing the node. Therefore, accessing the attribute of the first (e.g., left) node of the root node requires reading a value shifted by S=8 bytes relative to the block corresponding to the root node. Conversely, accessing the attribute of the second (e.g., right) node of the root node requires reading a value shifted by 2(4-1)×(8+8)=128 bytes relative to the block corresponding to the root node (assuming L=8). Then, accessing the attribute of the first (e.g., left) node of any subsequent branch node requires reading a value shifted by S=8 bytes relative to the block corresponding to that branch node. Accessing the attributes of the second (for example, to the right) node of a second-level branch node (below and to the right of the root node's level, so that R=3) would require reading a value shifted by 2(3-1)×(8+8)=64 bytes, and so on. As can be understood, in practice, for branch nodes below the root node, it is sufficient to divide the shift calculated for the branch node above by 2, so it is not necessary to explicitly calculate equation 2(R-1)×(S+L).

[0035] As previously explained, in some cases, vector processing may be used to execute multiple instances of the decision tree 20 in parallel based on different sets of input data. The vector processing capabilities of state-of-the-art CPUs can be advantageously utilized to accelerate the parallel processing of multiple trees. For example, due to the simple processing steps required by the nodes of the decision tree, the processing of multiple trees can be executed in parallel using vector instructions.

[0036] Furthermore, in some embodiments, the method may be performed on two or more decision trees that form part of a set model. In this case, the final inference result is based on the result of step S50 obtained for each decision tree included. For example, inference may be performed on some or all of the test data across all binary decision trees that may form part of a random forest model, a gradient boosting model, etc. Nevertheless, each binary decision tree is processed independently, starting from each root node.

[0037] The following describes how nodes may be reordered, with reference to Figure 5. Essentially, the objective is to reorder the child nodes from the same side laterally in the decision tree 20 according to the likelihood of subsequent node access by the parent node, before mapping node 120 to each block of the memory buffer (in step S30). Furthermore, as previously mentioned, in some embodiments, the rearrangement of the decision tree can be captured by storing appropriate flag values ​​with respect to the parent node, such flag values ​​determining the operations performed when the parent node is executed.

[0038] To that end, during the preparation phase (Figure 5), the initial tree is set up (step S1), and in S2, this decision tree is trained on training data as usual to obtain a trained decision tree 10 as shown in Figure 1. That is, the cognitive model on which the decision tree is based learns its own parameters by processing training examples as usual. Next, some appropriate input data is fed to the model, and in step S3, the (previously trained) decision tree 10 is executed, taking into account the likelihood of accessing child nodes, for example, using a heatmap (where the model parameters are fixed). For example, the number of accesses may be monitored. Then, for each node of the binary tree (starting from the root node), the number of accesses of two child nodes may be compared. The child node with the largest number of accesses becomes the "first" node, and so on. This process is repeated until all child nodes have been compared. Finally, the rearranged tree 20 is obtained as shown in Figure 2.

[0039] The input data used to learn the likelihood of access may simply be the training data itself, some validation data, or both. Interestingly, the decision tree 20 may, in some cases, be further reordered at a later stage, for example, after performing inference on a considerable amount of test data. That is, access counts may be monitored from time to time, with consideration to updating the mapping of nodes to memory blocks (and therefore cache lines if necessary).

[0040] The preferred scenario is as follows: First, in step S1 (Figure 5), the initial tree is set up. This tree is trained on training data later in S2 to obtain a trained decision tree 10. Next, in S3, tree 10 is executed on representative input data (e.g., training data, validation data, some test data, etc.) to learn the likelihood of access. In S4, based on the learned likelihood of access, some nodes are finally swapped (e.g., flags are stored relative to the corresponding parent node) to obtain a decision tree 20. This tree 20 is then used in S5 and S10 to perform steps S20-S60 shown in Figure 6. That is, in step S20, the memory buffer is set up, taking into account that node attributes will be mapped to memory blocks in the memory buffer in step 30. Then, in step S40, taking into account (or while) executing the nodes of tree 20 in S50 to obtain the inference results in S60, such blocks are mapped to memory cache lines.

[0041] Next, according to another aspect, embodiments of the present invention may be embodied as a computer program product for accelerating machine learning inference. This computer program product comprises a computer-readable storage medium on which program instructions are embodied, and these program instructions are executable by processing means (processors) 105 of one or more computerized units 101 (see Figure 7), causing such processing means to perform the steps previously described with reference to the present method. In particular, such instructions can cause the computerized units to utilize vector processing, as previously described.

[0042] Computerized systems and devices can be appropriately designed to carry out embodiments of the invention described herein. In this regard, it can be understood that the methods described herein are primarily non-interactive and automated. In exemplary embodiments, the methods described herein may be carried out in an interactive system, a partially interactive system, or a non-interactive system. The methods described herein may be implemented in software, hardware, or a combination thereof. In exemplary embodiments, the methods proposed herein are implemented in software as executable programs, the latter being executed by appropriate digital processing devices. More generally, embodiments of the invention may be implemented in locations where a virtual machine or general-purpose digital computer, or both, such as a personal computer or workstation, are used.

[0043] For example, Figure 7 schematically represents a computerized unit 101 (e.g., a general-purpose computer or a special-purpose computer), which may, in some cases, exchange information with other similar units to enable the steps to be carried out according to this method.

[0044] In exemplary embodiments, with respect to the hardware architecture, as shown in Figure 7, each unit 101 includes at least one processor 105 and memory 111 coupled to a memory controller 115. In some cases, multiple processors (e.g., CPUs, GPUs, or both) may be included in each unit 101. For that purpose, each CPU / GPU may be assigned a memory controller, as is known in itself. In a variation, the controller of unit 101 may be coupled to an FPGA, as described above, and for example, some of the CPU / GPUs shown in Figure 7 may be replaced by FPGAs.

[0045] One or more input / output (I / O) devices 145, 150, 155 (or peripherals) are communicatively coupled via a local input / output controller 135. The input / output controller 135 may be coupled to or include one or more buses and system buses 140, as known in the prior art. The input / output controller 135 may include additional elements, such as controllers, buffers (caches), drivers, repeaters, and receivers, which are omitted for simplicity, to enable communication. Furthermore, the local interface may include address, control, or data connections, or a combination thereof, to enable proper communication between the aforementioned components.

[0046] Processor 105 is a hardware device for executing software instructions. Processor 105 can be any custom-made or commercially available processor. Generally, processor 105 may include any type of semiconductor-based microprocessor (in the form of a microchip or chipset), or generally any device for executing software instructions.

[0047] The memory 111 typically includes volatile memory elements (e.g., random access memory) and may further include non-volatile memory elements. Furthermore, the memory 111 may incorporate electronic, magnetic, optical, or other types of storage media, or a combination thereof.

[0048] The software in memory 111 may include one or more separate programs, each of which includes executable instructions for implementing logical functions. In the example of Figure 7, the instructions loaded into memory 111 may include instructions resulting from the execution of the computerized method described herein, according to an exemplary embodiment. Memory 111 may further load a suitable operating system (OS). The OS essentially controls the execution of other computer programs or instructions and provides scheduling, input / output control, file and data management, memory management, and communication control and related services.

[0049] In some cases, a conventional keyboard and mouse may be coupled to the input / output controller 135. Other I / O devices 140-155 may be included. The computerized unit 101 may further include a display controller 125 coupled to a display 130. Optionally, in some embodiments, the computerized unit 101 may be coupled to a storage device (e.g., internal, external, etc.), such as storage 121. Any computerized unit 101 typically includes a network interface or transceiver 160 to connect to a network and then enable data transfer with other external components (e.g., other units 101).

[0050] The network transmits and receives data between a specific computerized unit 101 and another computerized unit 101 or other computing device, or both. The network may, in some cases, be implemented wirelessly using wireless protocols and technologies such as WiFi or WinMax. The network may, in particular, be a fixed wireless network, a wireless local area network (LAN), a wireless wide area network (WAN), a personal area network (PAN), a virtual private network (VPN), the Internet, or other suitable network system, and may include equipment for transmitting and receiving signals. However, it is preferable that the network allows for very high-speed messaging between units.

[0051] The network can also be an IP-based network for communication between any specific unit 101 and any external unit via a broadband connection. In an exemplary embodiment, the network can be a managed IP network controlled by a service provider. In addition, the network can be a packet-switched network such as a LAN, WAN, Internet network, or Internet of Things network.

[0052] Therefore, the present invention may be a method, a system, or a computer program product, or a combination thereof. The computer program product may include one or more computer-readable storage media containing computer-readable program instructions for causing a processor to perform an aspect of the present invention.

[0053] A computer-readable storage medium can be a tangible device capable of holding and storing instructions for use by an instruction-executing device. A computer-readable storage medium may, but is not limited to, electronic storage devices, magnetic storage devices, optical storage devices, electromagnetic storage devices, semiconductor storage devices, or any suitable combination thereof. A non-exhaustive list of further specific examples of computer-readable storage media includes portable computer diskettes, hard disks, random-access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), static random-access memory (SRAM), portable compact disk read-only memory (CD-ROM), digital versatile disks (DVDs), memory sticks, floppy disks, mechanically encoded devices such as punched cards or grooved raised structures on which instructions are recorded, and any suitable combination thereof. When used herein, computer-readable storage media should not be interpreted as transient signals themselves, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through waveguides or other transmitting media (e.g., light pulses passing through optical fiber cables), or electrical signals transmitted through wires.

[0054] The computer-readable program instructions described herein may be downloaded from a computer-readable storage medium to each computing device / processing device, or to an external computer or external storage device via a network (e.g., the Internet, a local area network, a wide area network, or a wireless network, or a combination thereof). This network may include copper transmission cables, optical transmission fibers, wireless transmitters, routers, firewalls, switches, gateway computers, or edge servers, or a combination thereof. A network adapter card or network interface within each computing device / processing device receives computer-readable program instructions from the network and transfers those computer-readable program instructions for storage on a computer-readable storage medium within each computing device / processing device.

[0055] The computer-readable program instructions for performing the operation of the present invention may be assembler instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, state setting data, configuration data for integrated circuits, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Smalltalk®, C++, and procedural programming languages ​​such as the C programming language or similar programming languages. The computer-readable program instructions may be executed as a whole on the user's computer, partially as a standalone software package on the user's computer, partially on the user's computer and on a remote computer, respectively, or as a whole on a remote computer or on a server. In the latter scenario, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or wide area network (WAN), or the connection may be to an external computer (for example, via the Internet using an Internet service provider). In some embodiments, to carry out aspects of the present invention, an electronic circuit including, for example, a programmable logic circuit, a field-programmable gate array (FPGA), or a programmable logic array (PLA) may be customized by executing computer-readable program instructions using state information of computer-readable program instructions.

[0056] Aspects of the present invention will be described herein with reference to flowcharts or block diagrams, or both, of methods, systems, and computer program products, according to embodiments of the present invention. It will be understood that each block in a flowchart or block diagram, or both, and combinations of blocks contained in a flowchart or block diagram, or both, can be implemented by computer-readable program instructions.

[0057] These computer-readable program instructions may be provided to a general-purpose computer, a dedicated computer, or a processor of another programmable data processing device to create a machine, so that instructions executed via the processor of a computer or other programmable data processing device can create means to perform functions / operations specified in one or more blocks of a flowchart or block diagram or both. These computer-readable program instructions may be stored on a computer-readable storage medium containing instructions that include instructions to perform modes of functions / operations specified in one or more blocks of a flowchart or diagram or both, and can instruct a computer, a programmable data processing device, or other device, or a combination thereof, to function in a particular manner.

[0058] Computer-readable program instructions may be read into a computer, another programmable data processing device, or other device so that instructions executed on a computer, another programmable device, or other device perform functions / operations specified in one or more blocks of a flowchart or block diagram or both, thereby causing a series of operable steps to be executed on a computer, another programmable device, or other device that generates a computer implementation process.

[0059] The flowcharts and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present invention. In this regard, each block in the flowchart or block diagram may represent a module, segment, or portion of instructions comprising one or more executable instructions for implementing a defined logical function. In some alternative implementations, the functions shown in the blocks may occur in an order different from the order shown in the figures. For example, two consecutively shown blocks may actually be executed substantially simultaneously, or possibly in reverse order, depending on the functions they contain. Note also that each block in the block diagram or flowchart diagram or both, and any combination of blocks contained in the block diagram or flowchart diagram or both, may be implemented by a dedicated hardware-based system that performs a defined function or operation, or executes a combination of dedicated hardware and computer instructions.

[0060] While the present invention has been described with reference to a limited number of embodiments, modifications, and accompanying drawings, it will be understood by those skilled in the art that various modifications may be made and equivalents may be substituted without departing from the scope of the invention. Specifically, features (such as devices or methods) mentioned in certain embodiments, modifications, or shown in the drawings may be combined with or replaced with other features in other embodiments, modifications, or drawings without departing from the scope of the invention. Accordingly, various combinations of features described with respect to any of the above embodiments or modifications may be contemplated without departing from the scope of the appended claims. In addition, many minor modifications may be made without departing from the scope of the invention to adapt certain situations or materials to the content of the invention. Thus, it is intended that the invention is not limited to the specific embodiments disclosed, but rather encompasses all embodiments included in the appended claims. In addition, many modifications other than those explicitly mentioned above may be contemplated.

Claims

1. Setting up a memory buffer in memory that includes adjacent memory blocks, This involves obtaining a decision tree that includes nodes containing branching nodes and leaf nodes, Each of the branch nodes includes at least two child nodes ordered according to the likelihood of accessing a child node after each of the branch nodes, Mapping the nodes to each block of the memory block, wherein each of the memory blocks stores one corresponding attribute of the nodes, the mapping is performed in a depth-first search order, each of the branch nodes and each branch node's child nodes are mapped to a continuous block of the memory block, and the ordered at least two child nodes of each branch node are mapped to a continuous block of the memory block. Executing the node by processing the attributes of the node in such a way that they are accessed from memory according to the order of the memory blocks in the memory buffer, wherein access from each of the branch nodes to each of the at least two child nodes is performed by calculation based on the location of the corresponding memory block of the branch node. The inference result is returned based on the execution result of the aforementioned node. A method of execution by computer information processing, including [the specified method].

2. The memory blocks corresponding to the branch nodes have the same size, and the memory blocks corresponding to the leaf nodes have the same size, The memory block corresponding to one of the branch nodes does not contain a pointer to one of the child nodes of the branch node, During the execution of the node, the attributes of the node are accessed from memory according to the implicit ordering of the memory blocks in the memory buffer. The method according to claim 1.

3. The method according to claim 1, further comprising mapping the memory buffer to a cache line of cache memory, wherein the memory blocks of the memory buffer are mapped in an orderly manner to the memory blocks of the cache line according to the order of the memory blocks, and the node processes the attributes of the node as they are obtained from the memory blocks of the cache line.

4. The mapping is performed such that a memory block containing a branch block and a first child block is mapped to a contiguous memory block in the memory buffer. The aforementioned branch block corresponds to one of the multiple branch nodes, The first child block corresponds to one of the first ordered child nodes of the branch block, The method according to claim 1.

5. All of the memory blocks corresponding to the branch nodes have the same size of S bytes. The method according to claim 4, wherein all of the memory blocks corresponding to the leaf nodes have the same size of L bytes, and L is less than or equal to S.

6. The computer implementation method according to claim 1, wherein mapping the node to each block of the memory block includes setting a flag for each branch block indicating an operation to be performed when the corresponding branch node is executed, the flag being set in a manner in which the child nodes of the corresponding branch node are ordered according to the likelihood of access to the node.

7. The method according to claim 6, wherein the decision tree is a binary decision tree, and the flag indicates one of two complementary mathematical comparison operations.

8. The method according to claim 7, wherein the two complementary mathematical comparison operations are based on inequalities.

9. The method according to claim 7, wherein the two complementary mathematical comparison operations include testing whether x is less than t or whether x is greater than or equal to t, and testing whether x is greater than t or whether x is less than or equal to t, where x corresponds to the feature value being tested and t corresponds to a threshold.

10. All of the memory blocks corresponding to the branch nodes have the same size of S bytes. All of the memory blocks corresponding to the leaf nodes have the same size of L bytes, and L is less than or equal to S. The memory block corresponding to the branch node does not contain a pointer to any of the child nodes of the branch node. During the execution of the node, the attributes of the node are accessed from memory according to the implicit ordering of the memory blocks in the memory buffer. For each branch node among the multiple branch nodes, the implicit ordering of the memory blocks is as follows: The rule is that the offset of the first child block corresponding to the first child node of each branch node is equal to the value obtained by adding S bytes to the offset of the branch block corresponding to each branch node, and The rule is that the offset of the second child block corresponding to the second child node of each branch node is equal to the offset of the branch block corresponding to each branch node plus 2^(R-1) × (S+L) bytes, where R represents the number of tree levels in the decision tree below the tree level containing any of the branch nodes. The method according to claim 7, which causes access to the attributes of the child nodes of each branch node accordingly.

11. The computer implementation method according to claim 6, wherein in mapping the nodes to each of the blocks of the memory block, the size S of all branch blocks is equal to 8 bytes, of which 4 bytes are used to encode the comparison threshold and 4 bytes are used to encode both the feature selector and the flag.

12. The method according to claim 1, wherein multiple instances of the decision tree are executed in parallel based on different sets of input data using vector processing.

13. The method according to claim 1, wherein two or more decision trees form part of a set model such that the inference result returned is based on the result of executing each of the two or more decision trees' nodes.

14. The method according to claim 1, further comprising ordering each of the child nodes of the branch node from the same side laterally of the decision tree according to the likelihood of access for each child node after each of the branch nodes, before mapping the node to each of the blocks of the memory block.

15. The method, before ordering the child nodes, The decision tree is trained based on the training data, The decision tree is executed as it has been trained to identify the likelihood of access based on the input data. The method according to claim 14, further comprising:

16. The method according to claim 15, wherein the decision tree is executed as it has been trained to identify the likelihood of the access based on the input data, and the input data includes at least a portion of the training data and validation data.

17. A computer program for accelerating machine learning inference, Setting up a memory buffer that contains adjacent memory blocks within the memory of a computerized system, Obtaining a decision tree that includes nodes containing branch nodes and leaf nodes, wherein each of the branch nodes includes at least two child nodes ordered according to the likelihood of accessing a child node after each of the branch nodes, Mapping the nodes to each block of the memory block, wherein each of the memory blocks stores a corresponding attribute of the nodes, the mapping is performed in a depth-first search order, each of the branch nodes and each branch node's child nodes are mapped to a continuation block of the memory block, the ordered at least two child nodes of each of the branch nodes are mapped to a continuation block of the memory block, and each of the memory blocks is mapped such that access from each of the branch nodes to each of the at least two child nodes is performed by calculation based on the location of the corresponding memory block of the branch node. A computer program that performs a certain action.

18. Executing the node by processing the attributes of the node in such a way that they are accessed from memory according to the order of the memory blocks in the memory buffer, The inference result is returned based on the execution result of the aforementioned node. The computer program according to claim 17, which further causes the computer program to perform the following.

19. A computer-readable storage medium recording the computer program according to claim 17 or 18.

20. Processor set and A computer system comprising a computer-readable storage medium, The processor set is structured, arranged, connected, and programmed to execute program instructions stored in the computer-readable storage medium. The stored program instructions, A program instruction programmed to set up a memory buffer that includes adjacent memory blocks, A program instruction programmed to obtain a decision tree including nodes containing branch nodes and leaf nodes, wherein each of the branch nodes includes at least two child nodes ordered according to the likelihood of accessing child nodes after each of the branch nodes, A program instruction programmed to map the nodes to each block of the memory block, wherein each of the memory blocks stores one corresponding attribute of the nodes, the mapping is performed in a depth-first search order, each of the branch nodes and each branch node's child nodes are mapped to a continuation block of the memory block, and the ordered at least two child nodes of each branch node are mapped to a continuation block of the memory block; A program instruction programmed to execute a node by processing the attributes of the node in such a way that they are accessed from memory according to the order of the memory blocks in the memory buffer, wherein access from each of the branch nodes to each of the at least two child nodes is performed by calculation based on the location of the corresponding memory block of the branch node; A program instruction programmed to return an inference result based on the execution result of the aforementioned node. A computer system that includes [a certain feature].

Images