Processing core operation suppression based on contribution estimate

Abstract

Processing cores having the ability to suppress operations based on estimates of the contributions of those operations to improve overall performance of the core are disclosed. Related methods that can be implemented by such processing cores are also disclosed. One such method includes generating a reference value for a composite computation. A full execution of the composite computation generates an accurate output and requires execution of a set of component computations. The method also includes generating a component computation approximation. The method also includes evaluating the component computation approximation with the reference value. The method also includes performing a partial execution of the composite computation using the component computation approximation to produce an estimated output. The method also includes suppressing the component computations while performing the partial execution based on the evaluation of the component computation approximation with the reference value.

Description

[0001] This application is a divisional application of patent application No. 201880040941.4, filed on May 11, 2018, entitled "Processing Core Operation Suppression Based on Contribution Estimation".

[0002] Cross-reference to related applications

[0003] This application claims the benefits of U.S. Provisional Patent Application No. 62 / 505,775, filed May 12, 2017, and U.S. Patent Application No. 15 / 975,930, filed May 10, 2018. Background Technology

[0004] If you fold a sheet of paper in half just fifty times, the resulting stack will be as thick as the distance from the Earth to the Sun. While each fold is a simple operation that doubles the thickness of the stack, the overall task is anything but simple. Similarly, many calculations performed by modern computing systems are complex calculations composed of numerous simple components. Performing each component operation may be insignificant, but the number of components can be astronomical, making complex calculations far from insignificant. In fact, basic calculations that can be easily handled from the beginning can be combined to produce complex calculations that are quite challenging for a given application.

[0005] The field of machine learning algorithms, particularly artificial neural networks (ANNs), is hampered by the computational complexity of implementing traditional algorithms used to instantiate ANNs. Assuming that executing a given ANN to recognize words from a sound file requires 10 billion computations, even if each of these component computations could be performed in a microsecond, executing the combined task would still take over 150 hours. Operating speech recognition technology at this speed is essentially the same as not performing speech recognition technology at all. The reason machine intelligence applications are so resource-intensive is that the data structures to be manipulated are typically very large, and the number of discrete primitive computations that must be performed on each of these data structures is equally enormous. Traditional ANNs introduce input vectors, perform operations using a set of input vectors and weight vectors, and then produce an output vector. Each weight vector in the set of weight vectors is often called a layer of the network, and the output of each layer is used as the input to the next layer. In traditional networks, the layers are fully connected, requiring every element of the input vector to participate in the operation along with every element of the weight vector. Therefore, the number of operations involved increases exponentially with the size of each layer.

[0006] The recent surge in interest in machine learning algorithms is primarily attributable to improvements in the hardware and software used to implement the complex operations employed in performing ANNs, rather than the development of new algorithms. These hardware and software improvements take various forms. For example, graphics processing units (GPUs), traditionally used to process vectors for rendering polygons in computer graphics, have been efficiently reused to manipulate data elements used in machine intelligence processing. As another example, certain classes of hardware have been designed from scratch to implement machine intelligence algorithms using dedicated processing elements such as systolic arrays. Further progress focuses on mimicking the behavior of neurons in traditional ANNs directly in hardware using collections of transistors and storage elements. Undoubtedly, the field of machine intelligence has benefited greatly from these improvements. However, despite the significant interest invested in these approaches, machine intelligence systems still represent one of the most computationally and energy-intensive computing applications of our time and present an area ripe for further development. Summary of the Invention

[0007] A processing core with the ability to suppress operations based on contribution estimates to improve overall core performance is disclosed. Related methods that can be implemented by such a processing core are also disclosed. One such method includes generating reference values ​​for composite computations. A complete execution of the composite computation produces an accurate output and requires the execution of a set of component computations. The method also includes generating component computation approximations for the component computations in the set of component computations. The method further includes evaluating the component computation approximations using the reference values. The method also includes performing a partial execution of the composite computation to produce an estimated output. The partial execution of the composite computation uses the component computation approximations. The method further includes suppressing component computations while performing the partial execution based on the evaluation of the component computation approximations using the reference values.

[0008] Another method that can be implemented according to the processing core of this disclosure includes storing reference values ​​for composite computation in memory. A complete execution of the composite computation produces an accurate output and requires the execution of a set of component computations. The method also includes generating component computation approximations for the component computations in the set of component computations. The method further includes evaluating the component computation approximations using the reference values. The method also includes performing a partial execution of the composite computation to produce an estimated output. The partial execution of the composite computation uses the component computation approximations. The method also includes suppressing component computations while performing the partial execution, based on the evaluation of the component computation approximations using the reference values.

[0009] Another method that can be implemented according to the processing core of this disclosure includes storing at least one reference value for composite computation. A complete execution of the composite computation includes a set of component computations. The method also includes generating a set of component computation approximations for the set of component computations. The method further includes performing a partial execution of the composite computation to produce an estimated output. The partial execution of the composite computation uses the component computation approximations. The method also includes selectively suppressing and performing component computations in the set of component computations while performing the partial execution, based on at least one reference value and the set of component computation approximations. Attached Figure Description

[0010] Figure 1 A directed graph instantiation of an artificial neural network that generates inferences based on two different inputs, according to relevant techniques, is shown.

[0011] Figure 2 Two data flow diagrams are shown for generating accurate outputs from a complete execution of the computation and estimated outputs from a partial execution of the computation, according to the methods disclosed herein.

[0012] Figure 3 The generation of a suppression command for sum-of-product calculations according to the method disclosed herein is shown.

[0013] Figure 4 A first data flow diagram for generating an estimated output from a partial execution is shown, in which component contribution approximations are used in the arithmetic operations used to generate the estimated output, and a second data flow diagram is shown, in which previously generated component calculation approximations are used as reference values, both according to the method disclosed herein.

[0014] Figure 5 The execution of filtered convolution computation according to the method disclosed herein is illustrated, wherein the method disclosed herein utilizes data from... Figure 4 Two methods for creating two data flow diagrams.

[0015] Figure 6 The generation of a suppression command for calculating the product of continuously updated reference values, according to the method described herein, is illustrated.

[0016] Figure 7 The generation of contribution estimates for the components including floating-point variables according to the method disclosed herein is shown.

[0017] Figure 8 The generation of reference values ​​for selecting components from a composite calculation for suppression, according to the method disclosed herein, is illustrated.

[0018] Figure 9It is a block diagram of the hardware component on the processing core, which suppresses component computation from composite computation by completely discarding component computation according to the method disclosed herein.

[0019] Figure 10 This is a block diagram of the hardware implementation of the suppression scheme, in which the computation of the suppressed component is suppressed, while a replacement value is provided to replace the output of the suppressed computation.

[0020] Figure 11 It is a block diagram of hardware components on the processing core that generate reference values ​​for controlling the partial execution of composite computation according to the method disclosed herein.

[0021] Figure 12 It is a block diagram of the hardware components on the processing core, which use component calculation approximations to generate reference values ​​for control part composite calculations, and perform comparisons with the reference values ​​according to the methods disclosed herein. Detailed Implementation

[0022] Some complex computations can be approximated by performing a small subset of their component computations. From a computational resource perspective, this is problematic because less critical component computations tend to consume the same amount of time and power but do not produce nearly the same amount of value as the critical subset. To conserve resources, less critical component computations can be pruned from the complex computation to allow partial execution. Ideally, partial execution will produce results approximately equivalent to those produced by the full execution of the complex computation. If pruning of the complex computation is implemented reasonably, partial execution will have sufficient fidelity to the full execution while consuming fewer computational resources.

[0023] The execution of ANNs is an example of the type of problem described earlier. The execution of an ANN intended to generate inferences from input typically involves a large number of component computations that do not significantly contribute to the values ​​represented by the generated inferences. This is because ANNs are often over-parameterized for any given inference. They are trained to use many different latent inputs, but can only process one input at a time. For example, an ANN might be able to identify multiple subjects in an input image, but only a small fraction of ANNs can respond to any one of those subjects in a meaningful way. Figure 1 In the example, a single ANN 100 is given two distinct inputs. Input 101 is an input tensor encoding an image of a cat. After all operations are performed, ANN 100 produces an inference 110 from input 101 in the form of the text string "CAT". Input 102 is an input tensor encoding an image of a dog. After all operations are performed, ANN 100 produces an inference 120 from input 102 in the form of the text string "DOG".

[0024] Different parts of an ANN may contribute strongly to the output when the object is a dog, but contribute nothing when the object is a cat. As shown in the figure, the different paths through ANN 100 are emphasized for each execution, indicating the most active parts of the ANN during each inference. Each node in the directed graph of the ANN shown involves a large number of computations, including the sum-of-products of matrix multiplications in fully connected layers of the ANN and convolutions between filters and previous layers in convolutional layers of the ANN. In the case of inference 110, all the computations contained in the lower part of the directed graph are essentially wasted, as they do not contribute anything meaningful to the generated values. However, the lower part of the directed graph is crucial for generating inference 120. The computation of the output of an ANN layer or a sub-segment of an ANN layer can be viewed as component computations of a composite computation consisting of the overall execution of the ANN.

[0025] although Figure 1 The need for graph-level optimization at the ANN level is illustrated, but the same principles apply to the entire system. Individual product sums, convolutions of filters or filter banks with the input data set, and multi-product computations such as matrix multiplication are examples of composite computations that can be optimized because, according to the methods disclosed herein, these composite computations are essentially composed of component computations that can be selectively suppressed or performed entirely based on estimates of the contribution of component computations to higher-level computations. In the case of ANNs, component computations can be pruned without unduly impacting the results of higher-level computations because many component computations within a single product sum operation can be neglected during ANN execution, even if the output of a single product sum operation is critical to the ANN's execution. In other words, individual multiplications in a product sum operation are component computations that can be pruned while maintaining fidelity to the output of the composite product sum operation.

[0026] As described in more detail below, several methods can be used to "suppress" (as used herein) operations. These methods include discarding the entire computation being performed without replacing it with anything, performing a computationally minor operation in place of the computation, or retrieving a substitute value from memory for use in the computation. In some methods, the substitute value can also be a value used to estimate the contribution of the component computation. These methods are particularly advantageous because they consume a small amount of processing resources, thus providing both a value for determining whether the computation should be pruned and a rough estimate of what the computation would produce if performed at high fidelity, which can be used as a substitute for the suppressed computation.

[0027] like Figure 1As shown, from a preliminary perspective, it is difficult to prune composite computations because it is not immediately clear which component computations are low-valued and which are important. Pruning the computations at the bottom of the graph will produce a correct inference in response to input 101, but may not produce a correct inference in response to input 102. However, the computational resources consumed by the composite computation can be reduced by obtaining an estimate of the actual or expected contribution of each component computation to the output of the composite computation, and using this estimate to prune component computations during subsequent or simultaneous partial execution of the composite computation. Here, the term "partial execution" is used in contrast to full execution, which includes the execution of the entire set of component computations that define the composite computation (i.e., a subset of component computations that is not suppressed).

[0028] In the methods disclosed herein, the overhead associated with obtaining the estimates discussed in the preceding paragraph is generally less than the computational resources conserved by partially performing the composite computation using the obtained information. Such methods include those in which the composite computation comprises numerous component parts, and the relative contribution of each component part to the overall output of the composite computation varies considerably. Typically, the composite computation performed during the execution of an ANN for the purpose of deriving inferences from it is an example of this problem.

[0029] The execution of composite computations containing sets of component computations can be simplified by obtaining estimates of the contributions of component computations to the full execution of the composite computation and suppressing those component computations during partial execution of the composite computation. Partial execution will produce estimated outputs that may not be equivalent to the exact outputs produced by the full execution of the composite computation. However, if component computations are suppressed according to certain methods disclosed herein, the reduction in computational resources consumed by partial execution will outweigh the reduction in accuracy. When applied to the domain of composite computations used to generate the outputs of sparse ANNs, the savings in computational resources can be substantial, while the reduction in accuracy becomes negligible.

[0030] The outlined methods may include a system capable of determining which component computations can be suppressed while still maintaining sufficient fidelity to the complete execution. In some methods, it involves determining which component computations (if any) in a composite computation can be suppressed, which includes the generation of component computation approximations. Component computation approximations provide an approximation of the output of a given computation, or an estimate of its impact on the overall composite computation as part of it. Component computation approximations can be generated by performing computations that are less precise than the component computations themselves. For example, approximations can be generated using hardware components with lower precision than those used to perform the component computations. An 8-bit multiplier can be used to generate approximations for 32-bit multiplications. As another example, approximations can be generated using a set of logic gates that determine whether any operand of the component computation is zero, one, negative one, or another value that significantly affects the computation output. In the case of multiplication computations, detecting that one of the operands is zero provides a component computation approximation because it is known that the product of any number and zero is zero.

[0031] In some methods, reference values ​​are also used to determine which computations can be suppressed. Reference values ​​can be evaluated together with approximations to determine an estimate of the contribution to component computations. This contribution estimate can be used to suppress individual component computations while preserving the fidelity of partial execution to the full execution. For example, if an approximation is smaller than a reference value, making the impact of the computation likely small, then the associated component computations will be suppressed and not performed. As another example, if an approximation equals the reference value of zero, then the associated component computations can be suppressed and not performed.

[0032] Figure 2 A diagram is provided illustrating a class of methods for reducing the computational complexity of composite computations. Figure 2 Two data flow diagrams are included, illustrating the generation of the estimated output of a composite computation. In data flow diagram 200, the input tensors [X] and [Y] undergo a full execution 201 of the computation. In the illustrated case, the full execution 201 involves the dot product of the two tensors to produce the output tensor [Z]. For example, the two tensors could be one-dimensional vectors, and the full execution might involve both product and composite computations. Tensors [X] and [Y] could be very large, making the computation of the full execution 201 computationally intensive. In this case, data flow diagram 210 represents a more efficient process because it generates an estimated output [Z'] that is approximately equivalent to the output [Z], while consuming fewer computational resources in terms of power, time, and hardware. Alternatively, tensors [X] and [Y] could be multidimensional, and the computation could be any tensor computation, including convolution, dot product, or cross product.

[0033] Data flow diagram 210 illustrates a set of steps of a computer-implemented method. Each step of the diagram can be performed by a combination of a processing core and non-transitory computer-readable storage. Execution of data flow diagram 210 consumes a first amount of computational resources. Execution of data flow diagram 200 consumes a second amount of computational resources. The first amount is less than the second amount. Partial execution 213 can be performed using circuitry on the processing core, such as multipliers, registers, adders, accumulators, and other logic, and can be used to generate output data from input data in response to received control input. Partial execution 213 may also include a register file that provides operands to a set of logic or mathematical units. In some methods, partial execution 213 may also include suppression logic that selectively provides operands from the register file to a set of logic or mathematical units.

[0034] The data flow diagram includes step 211, in which component computation approximations are generated for the component computations. Branches in steps 211, 212, 214, and 215 can be implemented in parallel or serially with partial execution 213. Branches in the steps can also be implemented using suppression logic. The component computation approximation is an estimate of the output of the component computation, generated using a process with less computational effort than the actual component computation performed. The component computation approximation can provide an approximation of the relative or absolute weight of one of the component computations included in a given composite computation. For example, the approximation can provide information about the magnitude of the output of the component computation. Execution of step 211 can also include generating a set of component computation approximations. In practice, step 211 can include generating approximations for each component computation in the overall composite computation, a predetermined subset of component computations, or a randomly sampled subset of component computations. Step 211 can utilize one or both of the input tensors [X] and [Y]. In some methods, step 211 will involve a low-resolution execution of the component computations. For example, if the component computation is a 16-bit multiplication of two numbers, step 211 could include 8-bit multiplication of those numbers. In other methods, step 211 would include evaluating the operands used for the component computation to determine if any of the operands significantly affects the outcome of the operation. For example, step 211 could determine that one of the operands used for the multiplication operation is zero, and the multiplication operation could be replaced by passing zero as the output of the multiplication.

[0035] Data flow diagram 210 also includes an optional step 212 for generating a reference value. Step 212 is optional because the reference value can alternatively be pre-programmed or provided from a separate system and stored in memory for later use, as in step 214. This value can be stored in RAM or programmed into the processing core in ROM memory. Step 212 can utilize one or both of the input tensors [X] and [Y]. The reference value can be used to more accurately determine the priority that should be given to any given component computation. For example, if tensors [X] and [Y] contain values ​​that vary greatly in magnitude, the reference value can be set to be equal to the largest order of magnitude of the values ​​in the input tensors, and component computation approximations that are not at or above that order of magnitude will be suppressed. Reference values ​​for evaluating component computation approximations can also be derived from other component computation approximations in the same composite computation. This reference value can be a reference value for the entire composite computation or can be generated separately for a subset of component computations. Other methods for generating reference values ​​are discussed below.

[0036] Data flow diagram 210 includes step 215, which evaluates approximations of component computations using reference values. This step can be as simple as an analysis of whether the component computation approximation is equal to, less than, or greater than the reference value, or a more complex analysis. For example, this step could determine whether any of the operands used in the multiplication is zero, or determine that one of the approximations is significantly lower than the average approximation generated for the set of component computations. The purpose of the comparison is to filter component computations based on their importance level to the overall composite computation. Thus, the comparison can output binary values ​​that classify components into computations that should be fully executed and computations that should be suppressed. Furthermore, the output of the comparison can determine the output value or substitution operation that should be executed instead of the suppressed computation. The evaluation in step 215 can also classify component computations into multiple importance levels, such as high, medium, and low. Once classified into different levels, partial execution 213 can be adjusted based on the information obtained in step 215, thereby suppressing or allowing component computations in different ways.

[0037] Partial execution 213 uses tensors [X] and [Y] as input and generates an estimated output [Z']. The estimated output [Z'] is not equivalent to the output [Z]. However, the difference between the two values ​​is negligible, and generating the estimated output [Z'] requires less computation. Furthermore, unlike full execution 201, partial execution 213 also uses the data generated in step 215 for conditioning. Partial execution 213 may include a step 216 to suppress component computations based on component computation approximations and reference values. Step 216 may also include selectively suppressing and performing component computations in the component computation set based on the output of step 215.

[0038] Suppressed computations can be implemented in various ways. For example, suppressed computations can involve completely removing the computation from a composite computation, providing a value from memory to replace the computation, implementing a lower-resolution version of the computation, or otherwise replacing component computations with simplified operations. For instance, in a product-sum computation where a specific operation is suppressed, a subset of product operations may not be performed, and therefore their outputs will not be included in the summation. As another example, a specific multiplication operation in a product-sum computation can be implemented using a lower-precision multiplier or lower-precision operands. In another example, a pre-programmed average can be retrieved from memory to be used as the output of the suppressed computation. In another example, zero, the value of an operand, or the opposite of an operand can replace the output of a component computation. In yet another example, an approximation of a component computation can replace the output of a component computation.

[0039] A reference value for composite computation can be calculated from operands, component approximations, or both directly. However, the comparison between the reference value and the component approximation can also include a programmable tolerance, which adjusts the reference value to filter and suppress the degree of component approximation. The programmable tolerance can be a fixed value or a fixed percentage. It can be set to the square of the output of the direct operation. For example, when the component approximation is half, a quarter, or an eighth of the reference value of the direct operation, the tolerance can be used to generate a suppression signal for the component computation. This approach is suitable for hardware implementation if the output of the direct operation and the component approximation are encoded in binary. In a specific example, the direct operation will determine the maximum value of the component approximation for a given composite computation, and the reference value will be set to half of that value via a programmable tolerance.

[0040] Figure 3 A diagram 300 illustrating the execution of step 215 according to the preceding paragraph is provided. In this case, the composite computation is the dot product of two tensors [X] and [Y]. Listing 301 shows three associated component computation approximations used for this composite computation, which are calculated using rounded values ​​of the operands in tensor [X]. These values ​​are generated by component computation approximation according to this disclosure because integer multiplication is less computationally intensive than multiplication involving decimals. In this example, the direct computation output 302 is the average of the component computation approximations. Note that this is a case where the reference value is general to the set of component computations and is determined only after the set of approximations has been computed.

[0041] Comparator 303 operates on the component calculation approximation and reference value to generate information for suppressing component calculations during partial execution. This information is provided in vector 304. This information can be stored for later use or applied in real-time to the suppression operation. As shown, the first operation is scheduled to be suppressed because the approximation "5" is significantly smaller than the average "14.3", while the second operation is scheduled to be performed because the associated approximation "28" is greater than the average "14.3". However, a third approximation is somewhat close to the reference value, and therefore its entry in the vector is marked with a question mark. This illustration aims to show that component calculations can be marked as suppressed or executed depending on the programmable tolerance. For example, if the tolerance is set to half of the direct calculation output, the associated calculation will be performed during partial execution because 10 is greater than half of 14.3.

[0042] Approximations of component computations can be generated in several ways. As mentioned earlier, approximations are typically generated through lower-fidelity computations. The relative term "lower" is used because the exact nature of a lower-fidelity computation will depend on the component computation itself. For example, if the component computation is a multiplication between two 16-bit integers, the component computation approximation could involve the multiplication of two 8-bit integers. As another example, an approximation could involve reducing the number of bits used to represent the operands or output of the component computation. As yet another example, the data structure used to represent the operands could be simplified to a lower-resolution version (e.g., from 8-bit floating-point to 4-bit fixed-point). The data structure format of operands can be converted between all formats while bringing them into the data RAM on the processor core via direct memory access. Approximations can also simplify one or more operands used for computation while keeping other operands in their original format.

[0043] Figure 4 Includes two data flow diagrams, 400 and 410, which illustrate the generation of the estimated output for composite computation. These data flow diagrams are for reference. Figure 2 Specific examples of the described method categories. In data flow diagram 400, the input tensors [X] and [Y] undergo partial execution 213. Partial execution can be used to estimate the dot product, filtered convolution, or other computations with the input data, the output of which is the output tensor [Z]. Tensors [X] and [Y] can be simple one-dimensional vectors, or alternatively, multi-dimensional vectors, and the computation can be any tensor computation, including convolution, dot product, or cross product.

[0044] Data flow diagram 400 includes many of the same steps as data flow diagram 210, with similar operations identified using similar reference numerals. However, data flow diagram 400 is a clear example of how the component calculation approximation generated in step 211 can be used not only in evaluation step 215 but also as part of partial execution 213. Specifically, if operation engine 401 receives a suppression command from evaluation step 215, the component calculation approximation generated in step 211 for the suppression operation can be used to generate an estimated output Z' instead of performing component calculations. In the case of product summation, this would involve summing the value generated in step 211 with the cumulative total of the product summation, instead of multiplying the set of operands used in step 211 and summing the resulting value with the cumulative total of the product summation in a higher fidelity operation.

[0045] Data flow diagram 410 includes many of the same steps as in data flow diagram 210, with similar operations identified using similar reference numerals. However, data flow diagram 410 provides a clear example of how the component calculation approximation generated in step 211 can be stored as reference 411 for subsequent execution of evaluation step 215. As in the previous diagram, step 211 may include generating calculated values ​​for component calculations. Component calculations 211 can then be used in evaluation step 215, along with the stored reference values, to generate commands that suppress or execute corresponding calculations in partial execution 213. Alternatively, the component calculations generated in step 211 can also be stored in another iteration of step 411 for use as a reference for evaluating subsequent component calculations.

[0046] Figure 5A data flow diagram combining the above methods is provided, where composite computation is a multi-product computation between the filter and the input data set, and component computation is a product computation between individual units of the filter data and individual entries of the input data. The multi-product computation can then be component computation of the filter convolution computation during the execution of the convolutional layers of the neural network 500. Filter convolution computation can be performed on tiles of the input data. As shown, the input data tile 502 is an n-dimensional tensor represented by repeated two-dimensional planes in a stack. One two-dimensional plane in the stack is represented by layer 501. In the case of image data, a single layer can be the red, green, and blue channels of a two-dimensional image. The convolution operation can involve an n-dimensional filter 515 “slid” around the input data tile 502. The n-dimensional filter 515 can be decomposed into another stack of two-dimensional planes, with filter segments 504 representing a two-dimensional filter plane. Sliding a filter around the input data can involve multiplying each value in the filter by the corresponding value in the input data, storing the product in an output matrix with coordinates corresponding to the coordinates of the input data used in the operation, and moving the filter's "position" relative to the input data. This operation can be repeated once the filter has been placed in its new position. The filter's step size or "stride" can vary depending on the application. Any overlap with a previous position is because the step size is smaller than the filter size in the direction of movement. Each two-dimensional filter segment can be slid around the corresponding two-dimensional plane of the input data block in this way, and the combined output data at each two-dimensional position can be summed to produce the output data. The resulting output produces an output tensor. As shown in the figure, output data block 503 is a simple two-dimensional matrix.

[0047] In the language of this instruction manual, Figure 5 The composite computation is the convolution between filter 515, to which filter segment 504 belongs, and input data block 502. Considering this is a much more complex computation than a single product, many component computations are included. However, one such component computation, which will be discussed below, is a multi-product computation between the input data set 505 from input data block plane 501 and a filter segment 504 of filter 515. Component computations comprise subsets of component computations that take the form of product computations implemented using filter entries from filter segment 504 and entries from input data set 505.

[0048] Figure 5The process begins with a low-fidelity multi-product operation 506 involving values ​​from filter segment 504 and input data set 505. The low-fidelity multi-product operation generates numerous products based on a one-to-one correspondence between filter data entries and input data entries. As shown, the multi-product operation generates a set 507 of component computation approximations that form a matrix of these values. The component computation approximations 507 can be used to determine whether a corresponding higher-fidelity computation should be performed, or to replace the output of a higher-fidelity computation if it is suppressed. Higher-fidelity computation can be a product operation where each product contains a higher-resolution product, such as 32-bit operations instead of 8-bit operations. In this figure, the stored component computation approximations 507 or the output 509 of the high-fidelity component computation 508 are applied to generate output data 510 in output data block 503.

[0049] In step 516, a decision is made regarding whether to suppress or perform high-fidelity calculations. This step can be implemented according to step 215 and includes evaluating the component calculation approximation 507 using reference values. In the illustrated case, generating the reference value includes invoking information related to the second component calculation approximation previously obtained from another filter segment in filter 515. Thus, generating the reference value can also include generating those values ​​to be invoked first. The second filter segment will be the filter segment that previously slid around a separate two-dimensional plane around input block 502. Since the first filter segment in a set of filter segments will not precede another filter segment from which reference values ​​can be generated, the first filter segment in the set can use a set of pre-programmed values ​​instead of pre-stored reference values, or can simply be executed fully at high fidelity while storing the information needed for later execution of step 516 for another filter. The reference value can be generated by generating the second component calculation approximation using filter entries from the second filter segment and corresponding entries from input data block 502. Reference values ​​are advantageously generated from different filter segments operating at the same two-dimensional location on different two-dimensional planes of the input data block 502, because determining the suppression or enable operation based on this comparison ensures that only the filter segment and the two-dimensional input data plane that create the strongest response for the output data will be executed with high fidelity, while others will not. This is beneficial and easy to use in CNNs because certain filter segments and two-dimensional input data planes tend to evaluate the input data for specific characteristics that may be mutually exclusive between input planes. In this way, the input data can have a strong and important response to the filter represented by the response of the input data plane, and a small and negligible response to the filter on other input data planes.

[0050] Figure 5The process also includes sampling the component computation approximations. According to the specific method disclosed herein, a set of component computation approximations can be sampled, and the execution of a component computation set corresponding to a higher fidelity set can be determined based on the evaluation of a subset of the sampled component computation approximations. In the illustrated case, sampling involves sampling exponents 511 from the component computation approximation set 507. In this method, entries in the component computation approximation set are floating-point numbers represented using exponent values ​​in memory. Therefore, step 511 may include selecting the largest exponent from a subset of entries in the matrix, such as the largest exponent in each block of four entries in the component computation approximation set 507. Different degrees of sampling are possible and involve trade-offs between the overhead associated with running comparisons and the accuracy of suppression. The sampling procedure then uses these sampled exponents 512 to create a data structure for evaluating the component computation approximation set 507. Other kinds of sampling are possible depending on the data types included. In one approach, the entire value can be sampled and stored for subsequent comparisons.

[0051] For the comparison step used to make the decision to suppress or enforce the operation to function properly with the sampled version of the component calculation approximation, the stored component calculation approximation, which will be used as a reference value, must also be sampled. In the case shown, this is somewhat trivial because step 516 continuously compares the stored index with a new set of indexes and controls the suppression of the operation based on this determination. As a result, it inherently identifies which indexes in the new set 512 are greater than those in the stored set 513, and replaces the smaller indexes with the larger ones in process 514.

[0052] The sampled index set 512 is compared with the stored sampled index set 513. The stored sampled index set is the maximum value sampled from different filter segments and input data from different input data planes located at the same two-dimensional position, calculated using previous conditions. During comparison step 516, whenever it is determined that an index in set 512 is greater than the corresponding index in set 513, the component calculation approximation from which the larger index is sampled will not be suppressed and will be performed with high fidelity 508. In the case shown, this would mean that all four product calculations in the set of product calculations that sample the largest index will be performed with high fidelity. Alternatively, if it is determined during the comparison step that an index in set 512 is less than or equal to the corresponding index in step 513, the component calculation will be suppressed, and the component calculation approximation from which those values ​​are derived will be output from memory and applied to the composite calculation. In the case shown, this would involve adding a subset of the values ​​of the sampled indexes from the component calculation approximation set 507 to the output data 510. Therefore, instead of unsuppressed higher-fidelity calculations, the same set of component approximations can be applied entirely or partially to the composite calculation or discarded. As illustrated, this would include values ​​from sets 507 and 509 being used to produce output data 510.

[0053] Reference can be modified Figure 5 The described concepts are adapted to accommodate various complex computations. For example, a similar approach would apply if multiple filters were applied to a single block of input data to produce an output block representing the sum of the responses of that input data block to the multiple filters. In this variation, component computations containing the responses of the input data block to a single filter can be selectively suppressed or performed entirely, based on a comparison of how the same portion of the input data responds to another filter in the set of multiple filters. In the case shown, the stored maximum exponent 513, or other sampled values ​​representing the previous response, would be the response of the same input data to a filter different from the one used to produce the sampled maximum exponent 512. Other variations include changing the sampling degree from one-quarter of an entry to other proportions, such as one-fifth or one-twentieth, and varying the level of suppression of component computations based on the order of magnitude of the difference between the stored response and the newly measured response. For example, if the sampled value in the newly measured response is ten times larger than the stored value, the entire set of component computations can be performed with high fidelity, while if the sampled value is only slightly larger than the stored value, only a closely related subset of the component computations can be performed with high fidelity.

[0054] Figure 6 600 provides another method for implementing step 215. This method... Figure 3The method in this example combines the concept of continuously updating a reference value equal to the accumulated sum of the product calculations during the product calculation. In 600, the two values ​​from tensors [X] and [Y] are represented by floating-point numbers. Step 601, which generates component calculation approximations, provides an example of using rounded values ​​to generate the product of two sets of values ​​from the input tensor. 603-605 show the comparison of the component calculation approximation from step 601 with the current reference value. The generated reference value is calculated according to the equation under reference numeral 606. Figure 3 The method described above does not require generating approximations for all component calculations in order for a reference value to be available. Instead, component calculations can be generated in real time using the overall partial execution of the composite calculation. As seen in 603, the first reference value is set to zero by default, and the first component calculation approximation "5" is greater than this value, which results in an "Execute" command indicating that a higher-fidelity calculation should be performed. Therefore, the first non-zero reference value generated under reference numeral 606 is 5.1 × 1, which is the current cumulative total calculated after the first operation. This process is repeated at 604, resulting in the generation of the "Execute" command and another high-fidelity operation, followed by a "Suppress" command, in 605. This method can be enhanced by adjusting the reference value with a programmable tolerance, making it more likely that calculations towards the start of the composite calculation will not be performed. For example, the programmable tolerance might require the component calculation approximation to exceed 100% of the reference value when the component calculation is first performed, and to exceed 50% of the reference value when the final component calculation is performed. Throughout the execution of the composite calculation, the tolerance can be programmed to scale as needed in any way.

[0055] If component computation is a multiplication operation between the first floating-point input and the second floating-point input, then component computation approximation can include... Figure 7 The data flow diagram is shown in Figure 700. In this example, the two values ​​[Y] and [Y] from tensor [X] are represented by floating-point numbers 701 and 702. According to the format shown, floating-point numbers are stored in memory as a sign, exponent, and mantissa. The mantissa can be normalized to exist in the range [1.0, 2.0), such that 2 is excluded but 1 is included. The exponent can be viewed as a rough representation of the number's value, while the mantissa is used for fine-tuning. Discarding the mantissa to produce an estimate of the associated operand will result in a worst-case factor-of-two error range. Typically, assuming the operands in the system are uniformly distributed, the average error will be 1.5 times the error range. Furthermore, multiplication in the exponent field is equivalent to the direct addition of the exponents. Therefore, an approximation can be calculated using the components of the multiplication of values ​​701 and 702 provided by equation 703. In equation 703, M is the substitution value, which is an approximation of the mantissa of the data elements from tensors [X] and [Y].

[0056] As shown in the figure, according to Figure 7The method for generating approximate component calculations involves: adding the exponent of a first floating-point input to the exponent of a second floating-point input to generate an exponent sum, and multiplying the exponent sum by a mantissa estimate to generate an exponent-mantissa product. The mantissa used in this multiplication operation can be a fixed estimate or adjusted based on various factors. For example, the mantissa stand-in can also be adjusted based on the inputs used for approximating the component calculations. In a particular class of methods, the mantissa stand-in is generated along with a reference value used for the composite calculation during the analysis of the operands of the composite calculation.

[0057] As an added benefit to the method described in the previous paragraphs, the implementation of Equation 703 is inexpensive in terms of hardware and computational resource consumption. Figure 7 As seen, for the exponent, this implementation only requires a low-precision adder 704. The mantissa M can then be simply appended to the exponent. The adder's precision is low because the exponent of a floating-point value is inherently represented by fewer bits than the floating-point value itself. For example, a 32-bit floating-point value can have an 8-bit exponent, a 16-bit floating-point value can have a 5-bit exponent, and custom floating-point formats have fewer than 5 bits. Therefore, the adder's precision is "low" relative to the mathematical unit that operates on the entire fixed-point variable. The exponent and the sum are generated by the low-precision adder 704. The mantissa can be appended to the exponent from memory 705. The memory can be a register or ROM. The sign term can also be computed by a single XOR operation requiring a single logic gate 706. In the case shown, the processing unit 707 is the basic processing block that implements equation 703. This block multiplies the inputs from the low-precision adder 704 and memory 705 and sets the sign of the output based on the binary input from the single logic gate 706. The operands from either branch can be combined to represent an approximate output of the component computation, which can then be stored in a single register or other memory location. The resulting hardware can utilize a fraction of the computational resources associated with the full execution of the multiplication of two floating-point numbers to compute a reasonably accurate approximation of the component computation.

[0058] If component computation involves fixed-point numbers, relevant procedures can be used to generate component computation approximations. For example, the hardware on the processing core can detect the number of leading zeros in the operands used for component computation. If these values ​​are binary encoded, this will provide the highest power of two present in the numerical values ​​of the two operands. The number obtained by summing the leading zeros of a known fixed-point number type can then be viewed as... Figure 7The "exponent" in the previous example. In this approach, the summation of numbers obtained by summing leading zeros of two or more operands provides an approximation of the multiplication of two fixed-point numbers. The "exponent" is obtained using the known precision of the operands and the number of leading zeros. One possible approach is to set the approximation as equal to: (total precision_operand_X – leading_zero_operand_X) + (total_precision_operand_Y – leading_zero_operand_Y). The complexity and resource consumption of counting leading zeros and performing these additions and subtractions are significantly reduced compared to the hardware required to multiply two fixed-point numbers of any considerable bit size.

[0059] Reference values ​​can be generated and utilized in various ways. For a given composite computation, the reference value can be a pre-programmed value stored in memory independently of the operands. This approach leverages knowledge related to the composite computation expected to be performed using the processing core, or the general effect of component computations with specific types of operands. For example, the reference value can be zero, one, or negative one. Alternatively, the reference value can be generated based on an evaluation of the operands of the composite computation, an evaluation of component approximations, or an evaluation of the output of a partial execution of other component computations in a given composite computation. Moreover, the reference value can be generated after analyzing the operands of the component computations, after generating component computation approximations, or in conjunction with the partial execution of the composite computation. As an example, the reference value can be set as the fraction of the largest component computation approximation operated on for a given composite computation, or the fraction of the largest output of a partial execution of a suppressed component computation in the composite computation. In the case of product-sum operations, the reference value can be the largest component approximation of the component product computation. In this approach, component product computations with corresponding component approximations having fractions smaller than this reference value are suppressed during partial execution. For a given composite computation, the reference value can be continuously updated as additional approximations or partial execution outputs are generated.

[0060] The method of generating the reference value will affect how the reference value is used. For example, if the reference value is chosen to be the largest component approximation in the set of component approximations, the comparison used to determine which operations are suppressed can be set to determine whether the associated approximation is greater than the reference value by a fraction. In another example, if the reference value is chosen to be the average of the component approximations, the comparison used to determine which operations are suppressed can be set to determine whether the associated approximation is greater than or less than the reference value. In some cases, the comparisons will utilize programmable tolerances for these comparisons.

[0061] Figure 8A data flow diagram 800 includes a specific method for generating reference values. In this method, the reference values ​​are continuously updated during the execution of the implementation portion. In data flow diagram 800, the partial execution of the product sum operation of the elements of two vectors X and Y is implemented by a multiplication-accumulation unit (MAC) 804 and auxiliary circuitry for suppressing component calculations and generating reference values. Each set of values ​​provided to the MAC unit 804 is utilized in the component calculations of the composite calculation. The MAC unit 804 multiplies its received inputs and adds them to the current accumulated value. The current accumulated value is provided at the output labeled with the MAC output.

[0062] The values ​​from vectors X and Y are also provided to component calculation approximation unit 801, which determines an estimate of only the portion of the multiplication performed by MAC unit 804. This estimate is used as the component calculation approximation. Component calculation approximation unit 801 may be a low-order multiplier, circuitry from data flow diagram 700, or some other hardware for generating an approximation of the multiplication of the values ​​provided to MAC unit 804. For example, calculation approximation unit 801 may include conversion circuitry for changing the precision of the numbers from vectors X and Y when they are retrieved from RAM memory.

[0063] Then, the approximate value generated by unit 801 is used in the comparison step implemented by comparison unit 803. This comparison unit can be similar to the comparison unit utilized in step 215 of flowchart 200. However, Figure 8 The comparison is performed using a reference value derived from the MAC output. In practice, the current accumulated value is used as the reference value. The feedback path may include a timing element 802 to synchronize the system with the values ​​provided from the input vectors. The comparison step generates a suppression control signal that instructs the MAC unit 804 to selectively suppress or perform all component calculations using the inputs from vectors X and Y.

[0064] exist Figure 8 In this example, the reference value is continuously updated during the execution of the component. This allows the programmable tolerance utilized by comparison unit 803 to scale during the execution of the composite calculation. For example, the programmable tolerance can be doubled after each component calculation is executed or suppressed. The reference value can be initialized to a pre-programmed value, zero, a first approximation, or some other value. Figure 8 The advantage of the method in this paper is that, when generating information related to the output of the composite computation, it can provide a direct estimate of how much each component will contribute to the composite computation.

[0065] The hardware used to suppress computation can take different forms and will depend on the type of suppression employed. As a basic example, a suppression scheme that retrieves a value from memory to replace the output of the suppressed operation might require space to store multiple replacement values ​​for various suppressed operations in RAM, while a suppression scheme that uses approximations calculated from components as replacement values ​​might simply provide that value from a set of registers. See below for reference. Figure 9-10 Specific examples of hardware that can implement the various methods for the suppression operations disclosed herein are discussed.

[0066] Figure 9 Block diagram 900 includes a hardware implementation of the suppression scheme, in which the computation of the suppressed component is simply discarded from the composite computation. Figure 9 This includes a register file 901, suppression logic 902, a multiplier 903, and an accumulator 904. There are 3N pairs of operands stored in the register file 901, 3N pairs of operand connections to the suppression logic 902, and N pairs of operand connections to the multiplier 903. In this example, the composite computation is a product sum of the operands in the register file. In a method where no suppression operation is performed and all product sums are executed, the suppression logic 902 can be removed. However, the diagram also needs to be expanded to include three times the number of multipliers. The suppression logic is able to compute component approximations, prune computations that are not expected to contribute meaningfully to the output (e.g., those with small magnitudes), and prevent multiplicands associated with less meaningful computations from being passed to the multiplier 903 via the suppression logic 902, thus achieving an effective three-fold increase in multiplication throughput while consuming far fewer than three times the computational resources in terms of both energy and silicon area.

[0067] exist Figure 9In the example, if the suppression logic 902 can choose to suppress two-thirds of the component computations, the number of operand pairs and multipliers will be correct, and all multiplicands required for partial execution will be sent to the multipliers in a single clock cycle. In practice, in some approaches, the behavior of the suppression logic can be forced to reduce a certain number of component computations to ensure that the ratio of suppressed operations to allowed operations matches the ratio set by the available arithmetic or logic elements for the set of data elements in the register file. In the case shown, the suppression logic will force the suppression of the least significant two-thirds of the component computations. However, the suppression logic 902 can also operate over multiple clock cycles to ensure that all component computations that rise to a certain level of relevance are executed. For example, if the suppression logic 902 finds that it can cautiously suppress only one-third of the component computations, the processing core can still execute all component computations in two clock cycles. If the suppression logic 902 finds that it can suppress fewer component computations while maintaining the required fidelity for full execution, then all three clock cycles will be required. However, even in this case, power can be saved because multipliers that are not used on each clock cycle can be gated (e.g., by not clocking their input pipeline stages).

[0068] The gating in suppression logic 902 may include a multiplexer whose data input lines are communicatively coupled to register file operand lines, and whose data output lines are communicatively coupled to multiplier operand lines. The control inputs of the multiplexer may be connected to circuitry that determines which computations should be suppressed. In some approaches, operand pairs will have a finite number of multipliers to which they can be routed. As a result, suppression logic 902 cannot arbitrarily suppress any mode of component computation, but still performs the portion of the computation at the optimal number of clock cycles mentioned above (i.e., 1 clock cycle when 2 / 3 of the computations can be suppressed). In these approaches, suppression logic 902 may consider constraints on subsets of component computations that can be effectively suppressed from each other. For example, the relative importance of two component computations can be compared by analyzing their approximations relative to each other and relative to a reference value, and one of the two component computations may be suppressed even if it would not otherwise be suppressed. This comparison, one using the reference value and the other using the component computation, can be implemented serially or in parallel. If the comparisons are performed sequentially, a second comparison (where approximations are compared on a relative basis) can be suppressed if there are no conflicts between the computed subsets.

[0069] Figure 10This is block diagram 1000 for the hardware implementation of the suppression scheme, where the suppressed component computations are suppressed while replacement values ​​are provided to replace the output of the suppressed computations. In this particular approach, the replacement values ​​are component computation approximations used to select which component computations should be performed and which should be suppressed. This is a nuanced point of significant importance. Figure 10 This includes register file 1001, suppression logic 1002, multiplier 1003, approximation processor 1004, and accumulator 1005. There are 3N pairs of operands stored in register file 1001, 3N pairs of operand connections to suppression logic 1002, and N pairs of operand connections to multiplier 1003. In this example, the composite computation is the product and sum of operands in the register file. These aspects of the hardware implementation are related to... Figure 9 The implementation is similar to that in [the text]. Figure 10 and Figure 9 The difference lies in that block diagram 1000 contains a set of approximation processors 1004. Furthermore, the suppression logic 1002 differs from suppression logic 902 in that, in addition to the routing operand, it also uses the routing operand or an approximation from the suppressed computation for higher-priority computations.

[0070] The higher logic overhead placed on the suppression logic 1002, and the additional hardware and computational resources consumed by the circuitry used to generate the replacement values ​​for the suppressed computations, will still be less than the hardware and computational resources required to perform all component computations. As previously mentioned, the replacement values ​​for the suppressed component computations can be generated by a lower-precision multiplier or other hardware that generates the replacement values ​​for the component computations in a manner that consumes fewer computational resources. These methods also require additional lines to the output from the suppression logic. Figure 10 In the example, the equivalent method would require 2N pairs of multiplicands instead of 2N lines of approximations for the suppression logic 1002 to the approximation processor 1004.

[0071] In the illustrated scenario, the approximation processor 1004 receives 2N component computation approximations from the suppression logic 1002. In some methods, the component computation approximations are generated using 2N pairs of operands fed from register file 1001 to the suppression logic 1002, and are associated with a set of 2N component computations selected by the suppression logic 1002 to be suppressed. The approximation processor can perform basic processing on the component computation approximations, such as changing their sign or annotating them with exponents in the case of floating-point numbers. For example, the approximation processor can be used as... Figure 7 The processing unit 707 in the middle. This method is advantageous because, as by using with Figure 7 The associated circuitry demonstrates that the circuitry required to generate the substitution value is cheaper than the circuitry used to implement a multiplier. Therefore, such as Figure 10The method can achieve a throughput of 3N with only slightly more computing resources than a machine with a throughput of N.

[0072] As Figure 9 The suppression logic 902 and Figure 10 The actual value of three, the factor between the number of inputs and outputs in 1002, is only used as an example, and the concept extends to factors other than 3. Additionally, although... Figure 9 and Figure 10 Limited to a set of two operands provided to a multiplier, this concept can be applied to any arithmetic or logic unit and the methods in which these units each employ any number of operands. For example, operand lines could carry three operands for a set of XOR (exclusive OR) gates instead of a multiplier, while still maintaining a 3:1 ratio between operand lines entering and exiting suppression logic. As another example, for ANN applications, the ratio of operand lines entering and exiting suppression logic can be 4–7, and for applications with very sparse datasets providing operands for composite computations, this ratio is much greater than 7.

[0073] The hardware used to generate reference values ​​and control signals for the suppression logic can take different forms and will depend on the specific form of suppression employed. For example, if the component computation is multiplication and the suppression scheme generates an approximation by filtering out zero-valued operands, the hardware used to generate the component computation approximation could be a logic OR gate whose inputs are configured to receive operands for the composite computation. In this example, if any operand to the multiplication is zero, the logic OR gate will output zero, which also provides an accurate approximation of the output of the complete execution of the multiplication. As another example, if the component computation is two-operand multiplication and the suppression scheme generates an approximation by filtering out operands with values ​​of 1 or -1, the hardware could be a comparator used to evaluate the operands and values ​​stored in ROM or RAM that are equal to 1 or -1. If the operand is identified as 1, the suppression logic would be a logic gate that replaces the output of the component computation with a substitute operand. If the operand is identified as -1, the suppression logic would be a logic gate that replaces the output of the component computation with a substitute operand whose sign bit is modified. Direct, independent operand evaluations performed for the purpose of determining approximate values ​​for component calculations can also utilize programmable tolerances so that operands determined to be nearly equal to 0, 1, or -1 are rounded to that value for the purpose of controlling suppression logic.

[0074] Figure 11This is a block diagram 1100 of hardware elements on a processing core for generating reference values ​​for controlling the partial execution of composite computations according to the method disclosed herein. Block diagram 1100 again includes a register file 1101. Register file 1101 includes a set of lines that transmit N pairs of multiplicands to a set of component computation approximation generators 1102. In this case, component computation is multiplication, and the approximation generators provide approximate values ​​of the multiplication of their inputs without performing a full execution of the multiplication operation on these operands. The output of the approximation generators 1102 is transmitted to logic circuitry 1103. Logic circuitry 1103 can perform various operations on its inputs. For example, logic circuitry 1103 can select the maximum value from its inputs, calculate the sum of its inputs, calculate the average of its inputs, or perform some other operation to generate a useful reference value for controlling the partial execution. Logic circuitry 1103 can then transmit its output to multiplier 1104. By multiplying the output of logic circuit 1103 by a programmable tolerance, the multiplier can apply the aforementioned programmable tolerance to the output of logic circuit 1103 to produce a value equal to a fraction of the output of logic circuit 1103. For example, the programmable tolerance can be a negative power of 2, so the fraction is one-half, one-quarter, one-eighth, or the reciprocal of another power of 2. The output of multiplier 1104 can be used as a reference value for the execution of the control section.

[0075] Figure 12 This is a block diagram 1200 of the hardware components on the processing core. These hardware components utilize component-calculated approximations to generate reference values ​​for composite calculations in the control section, and perform comparisons with these reference values ​​according to the method disclosed herein. The block diagram of the hardware components shown can be used instead of... Figure 9 The suppression logic 902 is used in block diagram 1200. In block diagram 1200, the suppression sub-block 1201 can receive control signals in the form of both a reference value from multiplier 1104 and a component approximation from computational approximation generator 1102. As shown by mixing line 1202, the sub-block then receives the original operands from register file 901 and, based on those control inputs, prevents a first subset of the operand set from being passed through the suppression logic block, while sending a second subset of the operand set through the suppression logic block to multiplier 903. Various variations of this method are available, including replacing the suppression logic 902 with similar circuitry and enabling the suppression sub-block 1201 to directly pass the computational approximation generated by computational approximation generator 1102 to accumulator 904.

[0076] Although this specification has been described in detail with respect to specific embodiments of the invention, it should be understood that modifications, variations, and equivalents to these embodiments will readily occur to those skilled in the art upon understanding the foregoing. Any method steps discussed above can be implemented by a processor that operates using a computer-readable non-transitory medium storing instructions for these method steps. The computer-readable medium may be memory within a personal user device or network-accessible memory. While the examples in this disclosure generally pertain to product-sum operations and convolutions, the same approach can be used to simplify the execution of any compound operations or compound logical operations. These and other modifications and variations can be made to the invention by those skilled in the art without departing from the scope of the invention, the scope of which is more specifically set forth in the appended claims.

Claims

1. A computer-implemented method, comprising: Evaluate the contribution estimates used for component computation, wherein the component computation is part of a composite computation performed by multiple computational entities; as well as Based on the evaluation steps and during the execution of the portion of the composite calculation, the component calculation is suppressed. The suppression is performed by a first computational entity among the plurality of computational entities, and the suppression includes discarding the component computations from the composite computation by prohibiting operand propagation to a second computational entity downstream of the first computational entity, and the second computational entity performs the plurality of component computations of the composite computation.

2. The computer-implemented method according to claim 1, wherein the composite computation portion generates a first result of the composite computation and requires a first amount of computing resources; The complete execution of the composite computation generates a second result of the composite computation and requires a second amount of computing resources; The first result is not equal to the second result; and The first quantity is less than the second quantity.

3. The computer-implemented method according to claim 1, wherein the evaluation step further comprises: Generate component calculation approximations for the component calculations; as well as The approximate value of the component calculation is evaluated using the reference value used for the composite calculation.

4. The computer-implemented method of claim 3, wherein the reference value is continuously updated during the execution of a portion of the composite calculation.

5. The computer-implemented method according to claim 1, wherein: The composite computation is a multi-product computation between the input data set and a first filter segment from the first filter; and The component calculation is a product calculation performed using the following: (i) filter entries from the first filter segment; and (ii) entries from the input data set.

6. The computer-implemented method according to claim 5, further comprising: Generate a reference value for the composite calculation, wherein generating the reference value includes selecting the largest exponent of the second component to calculate the approximation, wherein the second component calculates the approximation using the input data set; as well as The reference value is used to evaluate the component calculation approximation, wherein evaluating the component calculation approximation includes comparing the maximum exponent of the component calculation approximation with the maximum exponent of the second component calculation approximation; The suppression step includes suppressing the component calculation when the maximum exponent of the component calculation approximation is less than the maximum exponent of the second component calculation approximation.

7. The computer-implemented method according to claim 1, wherein: The component calculation is a product calculation; and The evaluation step includes determining whether the operand is zero, one, or negative one.

8. The computer-implemented method of claim 1, wherein the evaluation step includes evaluating the exponent of the operand.

9. The computer-implemented method of claim 1, wherein the evaluation step includes evaluating the number of leading zeros of the operand.

10. A processing core, comprising: A register file that provides a set of operands for composite computations, wherein the composite computations include component computations; A set of mathematical units; as well as The suppression logic block is programmed as follows: Use a comparator to evaluate either the operands used for the component computation or the contribution estimate used for the component computation; as well as Based on the evaluation, and during the partial execution of the composite calculation, the component calculation is suppressed. The suppression logic block is programmed to suppress component computation by preventing the operands from propagating to mathematical units located downstream of and separate from the suppression logic block within the group of mathematical units.

11. The processing core according to claim 10, wherein The composite computation portion generates a first result of the composite computation and requires a first amount of computing resources; The complete execution of the composite computation generates a second result of the composite computation and requires a second amount of computing resources; The first result is not equal to the second result; and The first quantity is less than the second quantity.

12. The processing core of claim 10, wherein the evaluation includes: Generate component calculation approximations for the component calculations; as well as The comparator is used to evaluate the approximate value of the component calculation using a reference value for the composite calculation.

13. The processing core of claim 12, wherein the reference value is continuously updated in memory during the partial execution of the composite computation.

14. The processing core according to claim 10, wherein: The composite computation is a multi-product computation between the input data set and a first filter segment from the first filter; and The component calculation is a product calculation performed using the following: (i) filter entries from the first filter segment; and (ii) entries from the input data set.

15. The processing core according to claim 14, wherein the suppression logic block is further programmed to: Generate a reference value for the composite calculation, wherein generating the reference value includes selecting the largest exponent of the second component to calculate the approximation, wherein the second component calculates the approximation using the input data set; as well as The component calculation approximation is evaluated using the reference value, wherein evaluating the component calculation approximation includes comparing the maximum exponent of the component calculation approximation with the maximum exponent of the second component calculation approximation; The suppression step includes suppressing the component calculation when the maximum exponent of the component calculation approximation is less than the maximum exponent of the second component calculation approximation.

16. The processing core according to claim 10, wherein: The component calculation is a product calculation; and The evaluation step includes using the comparator to determine whether the operand is zero, one, or negative one.

17. The processing core of claim 10, wherein the evaluation step includes evaluating the exponent of the operand using the comparator.

18. The processing core of claim 10, wherein the evaluation step includes evaluating the number of leading zeros of the operand using the comparator.

19. A computer-implemented method, comprising: Evaluate one of the set of operands used for component computation or the contribution estimate used for said component computation, wherein said component computation is part of a composite computation performed by multiple computational entities; Based on this assessment, it was determined that the suppression of the component computation resulted in a reduction in the computational resources consumed by the partial execution of the composite computation that outweighed the reduction in the accuracy of the output from the partial execution of the composite computation. as well as Based on the determination to suppress the calculation of the component, The suppression is performed by a first computational entity among the plurality of computational entities, and the suppression includes discarding the component computation from the composite computation by prohibiting at least one operand in the operand set from propagating to a second computational entity downstream of the first computational entity, and the second computational entity performs the plurality of component computations of the composite computation.

20. The computer-implemented method of claim 19, wherein the suppression step comprises: Generate component calculation approximations for the component calculations.

Images