Processing core with data correlation and adaptive rounding

Abstract

This paper discloses a processing core and association method with data-associative adaptive rounding. A disclosed processing core includes: an arithmetic logic unit cluster configured to generate values ​​of cells in the oriented graph data using input oriented graph data; a comparator coupled to a threshold register and a data register; a core controller configured to load a threshold into the threshold register when the values ​​of cells in the oriented graph data are loaded into the data register; and rounding circuitry. This rounding circuitry is configured to receive the values ​​of cells in the oriented graph data from the arithmetic logic unit cluster and conditionally round the values ​​of the cells in the oriented graph data based on the comparator output from the comparator.

Description

[0001] This application is a divisional application of the invention patent application filed on September 29, 2019, with application number "201910936061.1" and invention title "Processing Core with Data Association Adaptive Rounding".

[0002] Cross-references to related applications

[0003] This application claims the benefit of U.S. Provisional Patent Application No. 62 / 738,286, filed September 28, 2018, the entire contents of which are incorporated herein by reference for all purposes. Technical Field

[0004] This disclosure relates to a processing core with adaptive rounding based on data association. Background Technology

[0005] The recent surge in the performance of machine intelligence systems is not due to the development of revolutionary new algorithms. In fact, the core algorithms used in machine intelligence applications today are the body of work of more than half a century. Instead, it has become the hardware and software improvements that have fueled the recent surge in the efficient implementation of machine intelligence algorithms. Algorithms that were once too computationally intensive for even the most complex computers to implement in a useful way can now run on dedicated hardware on a single user's smartphone. These hardware and software improvements take many forms. For example, graphics processing units (GPUs), traditionally used to process vectors for rendering polygons in computer graphics, have been efficiently repurposed to manipulate data elements used in machine intelligence processing. As another example, certain classes of hardware have been designed from scratch to implement machine intelligence algorithms using dedicated processing elements such as systolic arrays. Further progress has focused on mimicking the behavior of neurons in traditional artificial neural networks (ANNs) directly in hardware using a combination of transistors and storage elements. Undoubtedly, the field of machine intelligence has benefited greatly from these improvements. However, despite the great interest in these approaches, machine intelligence systems still represent one of the most computationally and energy-intensive computing applications in modern times, and are poised for further development.

[0006] The reason machine intelligence applications are so resource-constrained is that the data structures to be operated on are typically very large, and the number of discrete primitive computations that must be performed on each data structure is equally large. Traditional ANNs take an input vector, perform computations using a set of input vectors and weight vectors, and then produce an output vector. Each weight vector in the set of weight vectors is usually called a layer of the network, and the output of each layer serves as the input to the next layer. In traditional networks, the layers are fully connected, which requires that every element of the input vector participates in the computation along with every element of the weight vector. Therefore, the number of computations involved increases with the size of each layer in a power-law manner. Furthermore, this aspect of machine intelligence algorithms makes them difficult to parallelize, because the computation of each layer depends on the output of the previous layer.

[0007] Modern ANNs exacerbate the problems mentioned earlier. Modern ANN schemes are often referred to as "deep learning" schemes in industry and literature. This usually refers to the large number of layers involved, or the complexity of the relationships between the outputs of one layer and the inputs of other layers. For example, in modern deep learning ANNs, the outputs of downstream layers can be fed back to the previous layer, adding recursive elements to the overall computation. Both the increase in layers and the additional complexity associated with the recursive relationships between layers increase the computational resources required to implement modern ANNs.

[0008] Figure 1 The diagram illustrates a directed graph 100 used for computation in a modern machine intelligence system. The input to the directed graph 100 is an input tensor X. The output of the directed graph 100 is an output tensor Y. The input can be an encoding of a scene, such as an image of a cat 101. In this example, the operation of the directed graph 100 involves a graph that provides an encoding of a text guess about what the encoded image contains. The graph output can be called the inference generated by the directed graph because the machine intelligence system is effectively inferring what the scene depicts based on the encoding of the scene. Thus, if the directed graph... Figure 10 A well-trained machine intelligence system, represented by Figure 100, with input tensor X, will produce an output tensor Y, which, as shown in the figure, encodes the word "cat".

[0009] The edges of the oriented graph 100 represent the computations required to run the graph. In this example, the graph is divided into two parts—a convolutional part 102 and a fully connected part 103. The convolutional part can be called a convolutional neural network (CNN). The vertices in the oriented graph of CNN 102 form a set of layers, including layers 106, 107, and 108. Each layer includes a set of tensors, such as tensors 109, 110, and 111. The vertices of the fully connected part 103 in the oriented graph also form a set of layers including layers 112 and 113. Each edge in the oriented graph 100 represents a computation involving the origin vertex of the edge. In CNN 102, the computation is a convolution between the origin vertex and a filter. Each edge in CNN 102 is connected to a different filter F. 11 F n1 F 12 F n2 Equivalent relationships. As shown in the figure, filter F 12 Tensor 109 undergoes a full convolution to generate an element of tensor 111. Filter F 12 Slide the filter around tensor 109 until a convolution operation has been performed between the filter and the origin vertex. In other schemes, filter F... 12 A portion of tensor 109 is multiplied to generate one element of tensor 111, and a full convolution is used to generate multiple elements of tensor 111. In the fully connected portion 103, the computation is the product of the weight set and values ​​from the previous layer. In the fully connected portion 103, each edge is associated with a unique weight value that will be used in the computation. For example, edge 114 represents weight w. n The product of input value 115. The value of element 116 is the sum of the set of identical operations involving all elements of layer 112 and the set of weight values ​​uniquely corresponding to the origin vertex of each edge that leads to element 116.

[0010] The operation of Oriented Graph 100 involves numerous computations. In the illustration, points are used in the vertical direction to indicate the high degree of repetition involved in the Oriented Graph. Furthermore, Oriented Graph 100 represents a relatively simple ANN, as modern ANNs can include more layers and have more complex interrelationships between them. Although Oriented Graph 100 is not shown, the output of one layer can be looped back as the input to the previous layer to form what is commonly known as a Recurrent Neural Network (RNN). The high flexibility of machine intelligence systems due to their numerous elements, coupled with the increasing complexity of the number of layers and their interrelationships, makes it unlikely that the complexity of machine intelligence systems will decrease in the future. Therefore, the computational complexity of machine intelligence systems is more likely to increase than decrease in the future. Summary of the Invention

[0011] This paper discloses a processing core and associative rounding method with data association. Some of the processing cores disclosed herein can be advantageously applied to the operation of oriented graphs. Some of the processing cores disclosed herein can be advantageously applied to the operation of artificial neural networks (ANNs), in which inferences are generated in response to inputs to the ANN. In certain embodiments, all other things being equal, the use of data association rounding improves computational efficiency and reduces the power consumption associated with generating inferences from an ANN using the processing core. The use of data association rounding allows the generated inferences to maintain the same fidelity as inferences generated without rounding, while reducing the power consumption and computational resource consumption of the processing core.

[0012] Some of the core processing methods disclosed in this article are specifically modifiable to suit sparse orientations that require precise operation. Figure 1 Use it. The example of a sparse oriented graph is used to instantiate an ANN, where most computations involve zero or near-zero data values ​​that have no significant impact on the inferences generated from the ANN. However, rounding all near-zero values ​​in such an ANN would not result in accurate operation, as some near-zero data values ​​could have a decisive impact on the inferences generated by the ANN. Therefore, it is generally insufficient to apply the blanket rule, in which all near-zero data values ​​are forced to zero to alleviate the computational complexity of generating inferences from the ANN, as this scheme would result in generating approximations that do not maintain sufficient fidelity to the appropriate inference. Instead, rounding can be applied with a pattern associated with the underlying data of the oriented graph, thereby minimizing computational and power resource consumption while maintaining the accuracy of the generated inferences.

[0013] Figure 2The diagram illustrates a single orientation graph 200 that generates separate inferences 210 and 220 from two distinct corresponding inputs 201 and 202. As shown, orientation graph 200 executes as needed in both cases, generating an inference "cat" in response to a cat image and an inference "dog" in response to a dog image. The orientation graph is highlighted differently in both cases. This highlighting reflects the fact that, based on the provided inputs, a single orientation graph will have high activation levels at different locations. The highlighting path of the orientation graph illustrates the most important values ​​computed during the graph's execution. As shown, the output of the second layer includes tensor 230, which has significant dispositive activations in both runs, and tensor 231, which has less significant activations in both runs. Therefore, the processing core configured to allow the execution of orientation graph 200 must be careful to maintain the accuracy of the computation of the generated tensor 230, but may be able to relax the accuracy of the computation of the generated tensor 231 while still generating correct inferences in both cases. In this way, different levels of rounding can be associated with the oriented graph data as tensor 230, compared to the oriented graph data as tensor 231.

[0014] Some processing cores disclosed herein apply data association rounding by associating thresholds with directional graph data and rounding computations involving that directional graph data using the associated thresholds. Processing cores can associate thresholds with directional graph data in various ways, which will be described with specific examples in the detailed disclosure below. As used herein, all data used to instantiate, run, and modify directional graphs can be referred to as “directional graph data.” In the specific case of an ANN, directional graph data may include specific activations, weights, biases, or any intermediate results required to derive inferences from the network or to train the network. As used herein, association with directional graph data refers to a data structure in the directional graph that is the opposite of the actual value held by that data structure during any given execution of the directional graph. In the specific processing cores disclosed herein, thresholds can be “associated” with directional graph data by using data stored in the processing core and / or in a cache memory that the processing core can randomly access. The processing core can store a set of thresholds, each of which is associated with a different unit of the directional graph data. These thresholds can be programmatically set via an external controller. The processing core can also store a set of values ​​used to calculate thresholds within its operations, wherein each value in the set is associated with a different unit of the orientation map data. These values ​​can be programmatically set via an external controller.

[0015] In the specific case where the processing core runs a directional graph to generate inferences from an ANN, the processing core can be configured such that the weights and / or filters of the ANN are stored in registers on the processing core and / or in a cache memory accessible via random access memory. Consistent with the same example, the processing core can be configured to store data necessary to support associations and apply thresholds when needed, in registers on the processing core and / or in the same cache memory. In certain embodiments, thresholds and associations can be programmatically provided to internal memory locations or cache memory via an external controller.

[0016] In a particular embodiment of the invention, a processing core is provided. The processing core includes a processing cluster configured to generate values ​​for cells of the orientation graph data using input orientation graph data. The processing core also includes a core controller configured to load a threshold into a threshold register when the values ​​of cells in the orientation graph data are loaded into a data register. The processing core further includes a comparator coupled to the threshold register and the data register, and configured to generate a comparator output based on: (i) a threshold in the threshold register; and (ii) the values ​​of cells in the orientation graph data in the data register. The processing core also includes rounding circuitry configured to: (i) receive the values ​​of cells in the orientation graph data from the processing cluster; and (ii) conditionally round the values ​​of cells in the orientation graph data based on the comparator output from the comparator.

[0017] In a specific embodiment of the present invention, a method is provided. The method includes: associating a threshold with cells of directional graph data in a processing core. The directional graph data is derived from a directional graph. The method further includes running the directional graph using the processing core. The method further includes generating values ​​for the cells of the directional graph data while running the directional graph and using a processing cluster in the processing core. The method further includes comparing the generated values ​​of the cells of the directional graph data with a threshold. The method further includes conditionally rounding the values ​​of the cells of the directional graph data based on the comparison and using rounding circuitry in the processing core.

[0018] In a specific embodiment of the present invention, a processing core is provided. This processing core includes: a component for associating a threshold and units of directional graph data according to an association; a component for generating values ​​for units of directional graph data using the input directional graph data; a component for generating a comparison between the threshold and the values ​​of units of directional graph data using the association; and a component for conditionally rounding the values ​​of units of directional graph data based on the comparison. Attached Figure Description

[0019] Figure 1 The diagram illustrates the orientation of an artificial neural network based on relevant technologies.

[0020] Figure 2 The diagram illustrates the orientation of an artificial neural network that generates two different inferences from two different inputs based on relevant techniques.

[0021] Figure 3 This is a block diagram of the processing core according to a specific embodiment of the present invention.

[0022] Figure 4 This is a flowchart of a method for running a directional graph using a processing core according to a specific embodiment of the present invention.

[0023] Figure 5 It is used to achieve this by using the correlation between directional graph data and thresholds. Figure 4 Functional block diagram of a specific method disclosed in the document.

[0024] Figure 6 It is used to achieve this by using the correlation between the input orientation map data and the threshold. Figure 4 Functional block diagram of a specific method disclosed in the document.

[0025] Figure 7 It is used to establish a correlation between the input orienteering graph data and the values ​​used to calculate the threshold using the same orienteering graph data. Figure 4 Functional block diagram of a specific method disclosed in the document.

[0026] Figure 8 It is used to achieve this by using the correlation between the input orientation map data and the threshold. Figure 4 Functional block diagram of a specific method disclosed in the document.

[0027] Figure 9 This is a block diagram of a processing core connected to an external controller and a cache memory according to a specific embodiment of the present invention.

[0028] Figure 10 This is a block diagram of a processing core with an integrated compression and decompression system in the main data path, according to a specific embodiment of the present invention.

[0029] Figure 11 This is a block diagram of a processing core with an integrated metadata generator in the main data path, according to a specific embodiment of the present invention. Detailed Implementation

[0030] This section provides specific methods and systems associated with a data-associative adaptive rounding processing core according to the above-described invention. The methods and systems disclosed in this section are non-limiting embodiments of the invention, provided for illustrative purposes only, and should not be construed as limiting the full scope of the invention.

[0031] Figure 3The processing core 300 is used to describe the operation of a specific embodiment of the invention. The processing core includes a core controller 301, a control connection 302, and at least two memory connections 303 and 304. The memory may be an external cache memory, such that the memory connections are cache memory connections. The controller may be an external controller, such that the control connection is an external controller connection. The core controller 301 may receive instructions via the control connection 302. These instructions may include input data and kernel identifiers. For example, during the operation of an ANN in response to a set of input data “input_1” and having a first layer of weights “weight_layer_1”, the control connection may receive instructions of the form: {conv, input_1, weights_layer_1}. The input data provided to the orientation graph may be referred to herein as input orientation graph data, where the set of input data “input_1” is a set of input orientation graph data including multiple entries of orientation graph data with specific values. The kernel used for the convolution operation “conv” can then be accessed from external memory by the core controller and used to form the instruction thread 305 of the processing core. Here, "conv" refers to the convolution between input_1 and weights_layer_1, and serves as the kernel identifier for the kernel that will perform the convolution operation. The instruction can contain the addresses of input_1 and weights_layer_1 in memory, or it can contain the values ​​themselves. The instruction can therefore involve a convolution between the ANN's input data and the data from the network's first layer. Thread 305 can then be used to manage the main data path 306 of the processing kernel to run this convolution. In this example, thread 305 will cause the value output_layer_1 to be generated by the main data path 306 and stored in cache memory.

[0032] The main data path 306 of the processing core may involve retrieving data from memory connection 303, loading it into source memory 307 of processing cluster 308 for processing, and storing the output in destination memory. Source memory 307 and destination memory 309 may be registers associated with the processing cluster. The memory from which input data is retrieved and output data is stored may be the same memory and may be an external cache memory suitable for random access. Processing cluster 308 may include the main computational unit of the processing core. Examples of such computational units include an arithmetic logic unit (ALU) and other logic blocks. As used herein, the term “ALU” refers to a circuit block that operates on fixed-point data elements and a floating-point unit (FPU) that operates on floating-point data elements. Consistent with the examples in the previous paragraphs, the values ​​of input_1 and weights_layer_1 can be retrieved from memory and the component values ​​of these vectors can be provided to processing cluster 308 according to instruction thread 305. The connection between the circuit blocks of the core controller 301 and the main data path 306 may involve providing configuration data to the configuration registers of each block, providing instructions for computation, and routing metadata to and from the core controller 301, as described below. The output of the convolution of input_1 and weight_layer_1 (which we may call output_layer_1) can then be stored in memory.

[0033] Processing core 300 includes a rounding block 310 located downstream of processing cluster 308 on the main data path 306. Rounding block 310 can be configured to round specific values ​​from destination memory 309. Rounding can be performed according to a data-associative rounding scheme. These values ​​can be rounded to zero. These values ​​can also be rounded to the nearest integer, one, negative one, or any other value or set of values ​​that simplifies further downstream computation while maintaining acceptable fidelity of the data path. In the context of a directed graph instantiating an ANN, maintaining "acceptable fidelity" involves ensuring that the same inferences are generated by the ANN as when data-associative rounding is disabled. In a particular embodiment of the invention, a subset of values ​​generated during the execution of one layer of the directed graph will be rounded to zero to simplify further computations involved in the execution of subsequent layers of the directed graph.

[0034] In a particular embodiment of the invention, a rounding block, such as rounding block 310, may include rounding circuitry. The rounding circuitry may include a comparator and a rounded value based on a comparison generated by the comparator. The comparison may be a Boolean value, and if the Boolean value is true, the value may be rounded to zero. The rounding block may apply a threshold and a value to the comparator to generate a comparison. The rounding block may round the value provided to the comparator based on this comparison. In a particular embodiment of the invention, the specific threshold utilized by the rounding block may be associated with the data being evaluated. Consistent with the example above of the processing core running a thread for the instruction {conv, input_1, weights_layer_1}, the threshold may be associated with the input_1 data structure or the weights_layer_1 data structure and may be applied to the rounding block to determine whether the value in output_layer_1 should be rounded before storing output_layer_1 in memory. The association managed by the processing core may relate to the data structure itself, rather than the value in the data structure. In other words, whenever input data is provided to a given orientation graph instantiated by the processing core, the same association threshold can be applied to the rounding block, which is the opposite of whenever a specific value is provided to the processing core.

[0035] In specific embodiments of the invention, the association between directional graph data and thresholds can take various forms. The association can be between different units of the directional graph data. For example, a single threshold can be associated with a data structure storing a single data value, a data structure containing a set of related data values ​​within a vector or other tensor, a single plane of a multidimensional data structure, an entire tensor, a data tile used to store one or more elements of the directional graph data, an entire layer of the directional graph, or the entire graph. The association can also be between different types of directional graph data. For example, a single threshold can be associated with input data to the graph, an edge or node of the directional graph, or the output of the directional graph. In the example of a directional graph used to instantiate an ANN, the association can be used for cumulative values, weight values, filter values, bias values, output values, input values, or any other data structure used or generated during inference from the network generation process. It will be clear from the following examples that the association between a threshold and a data structure can be an association with the operands of the computation while rounding the output of the computation using the threshold, or it can be a direct association between the outputs of the computation. In other words, associations can prompt the processing core application to round the output of any computation using the data structure, or the output of any computation that produces the value of that data structure during inference, given a threshold.

[0036] In specific embodiments of the invention, thresholds can be provided to the processing core in various ways. For example, thresholds can be provided to the processing core programmatically via an external controller. Once set, this threshold can be referred to as a programmatically selected threshold. The external controller and the processing core can be configured to allow specifying an association between a data structure and a threshold as a programmatically selected association at the level of human-readable source code to be compiled and implemented in the processing core. Similarly, they can be configured to allow specifying a value for the threshold, or an equation used to derive these values, as a programmatically selected threshold at the level of human-readable source code to be compiled and implemented in the processing core. The external controller can be a RISC controller capable of receiving PC commands specifying the association and threshold. The external controller can also provide the association in the form of an identifier of the data to which the threshold should be associated. Alternatively, the external controller can store the threshold in memory at a location allocated by the processing core to associate it with a given cell of orientation graph data, such that the association is an inherent feature of the processor, and that the association is set by the external controller when a data value is loaded to that particular address. These thresholds can be stored in the memory of the core controller on the processing core board, such as core controller 301, or in an external cache memory available to the processing core. The external cache memory can be random access memory. The thresholds can also be stored in a metadata header along with their associated data structures. Thus, the processing core can be configured to obtain the thresholds as part of the regular process of obtaining values ​​from the data structures, with some additional modifications to account for accessing the thresholds from the structure's header. For example, the data structure associated with input_data_1 can include a header, and the threshold to be applied to any calculations using input_data_1 can be stored in that header. In this way, whenever input_data_1 is recalled from memory, the required threshold will also be retrieved, as it resides in the data header used to identify and recall the required value from memory.

[0037] Thresholds can take various forms. Thresholds can be fixed numbers, programmatically configured via an external controller. For example, the external controller can set a threshold to "100," and any value below "100" associated with that threshold can be rounded. Thresholds can also be adaptively derived based on data values ​​in a directional graph data structure. In these cases, the equations used to derive the threshold can be programmatically configured via the external controller. For example, the external controller can set the value to "60," and the processing core will internally derive the threshold based on a series of equations, with the threshold set to 60% of the maximum value of the set of data values. Alternatively, or in combination, all or part of these equations can be configured in hardware on the rounding module. For example, a set of values ​​provided from the destination memory of the core computation block can be evaluated via a series of logic gates to produce an average of that set of values, which can be used as a threshold or for calculating the threshold. The rounding block can include different series of logic gates that can be programmatically applied to different data structures based on associations.

[0038] In certain embodiments of the invention, thresholds can be adaptively derived to round values ​​that are relatively insignificant relative to other values ​​in the same cell of the directional graph data. For example, a threshold can be derived as a programmatically defined fraction of the maximum value in a set of data values ​​in a given data structure (e.g., 1 / 4 of the maximum value in a vector of cumulative values ​​generated during ANN operation). When these schemes are applied to the runtime of an ANN, they may exhibit certain advantages because larger values ​​in some data structures will dominate the role of smaller values ​​in generating inferences. The equations used to derive thresholds may be more complex and may involve finding the maximum, mode, mean, median, or minimum value in a given data structure. These equations may also involve determining the standard deviation of the data values, analyzing histograms of these values, and performing complex statistical analyses on the data values. The equations used to derive thresholds and perform these intermittent analyses can be programmatically defined, associated with a kernel accessible to the processing core, and / or implemented directly in hardware within the rounding module of the processing core.

[0039] In certain embodiments of the invention, the threshold can be adaptively derived as a rounded value that is relatively insignificant relative to other values ​​in different units of the orientation graph data. The relationships between these different units of the orientation graph data can also be defined programmatically, such as in the equations from which the threshold is derived. For example, the threshold for the value in the output of the second layer of a CNN can be set to half of the average filter value used to calculate the output of the second layer of the CNN.

[0040] In specific embodiments of the invention, a threshold can be provided by referring to a specific aspect of the data value relative to its absolute logical value. The schemes according to these embodiments exhibit certain advantages in terms of efficiency in comparing the threshold and the data value. For example, in a scheme where the data value is stored as a floating-point number, the threshold can be provided or derived by referring to the exponent value of the floating-point number. Thus, the threshold can be a set value such as "8", and the comparison will involve a basic comparison operation between the integer value "8" and the value stored in the exponent bits of the floating-point number. The advantage of this scheme is that the system does not need to analyze the entire data value, and can practically ignore it entirely. For example, the mantissa of the floating-point number does not need to be analyzed. Meanwhile, the exponent provides a good source of information about the relative impact that different data values ​​may have on the operation of the orientation graph, making it possible in some applications to maintain the fidelity of the orientation graph's operation while minimizing the computational resource consumption of the operation through exponent rounding.

[0041] The association between data and thresholds can be applied in various ways during computation. For example, the core controller can store the threshold in the instruction thread and apply it to the configuration registers of the rounding block when the associated data value is transferred from the destination memory of the processing cluster. In this sense, the association is applied through the timing of the main data path orchestrated by the instruction thread. In these scenarios, the threshold can be stored as a data value in the instruction thread, and the threshold can be provided in its unmodified form from the thread controller to the comparator in the rounding block when the data value to which the threshold is to be applied is passed to the alternative input of the comparator. As another example, the threshold can be stored in memory for the entire data structure or a subset thereof. The data structure may include a portion of oriented graph data. Then, one or more thresholds can be pre-programmed by the core controller block and remain stable until the data movement is complete before the data is removed from the FPU cluster output. The core controller can program a threshold for all mathematical results performed, or it can update the threshold for each data structure or a subset thereof.

[0042] In certain embodiments of the invention, rounding can be performed at various points along the main data path. For example... Figure 3 As shown, after the value is obtained from the computing cluster, it is rounded immediately in rounding block 310. Alternatively, a similar scheme can be applied to round the value before it is provided to the main computing cluster (rather than as shown). Figure 3 (As shown) the value is rounded. However, as described below, some benefits are due to a scheme in which the value is rounded before being stored, as opposed to storing it in its unrounded form and then rounding it when it is called for later use.

[0043] Appropriate thresholds and associations can be selected to improve the efficiency of the processing core while maintaining the accuracy of the orientation graph at an "acceptable level." Appropriate thresholds and associations can be determined after the processing core is configured to run the orientation graph by feeding it a set of input data labeled with appropriate inferences that should be generated by the orientation graph. Training data can be used to modify the thresholds using a supervised machine learning scheme. Known supervised learning techniques can be used to select or solve for thresholds and associations, for evaluation using any kind of iterative approach. For example, training data can be used to train a regression classifier to determine the set of thresholds for the orientation graph. Unlike some applications where large amounts of labeled input data are difficult to obtain, the processing core can generate its own training data by disabling rounding operations in the rounding block. This state can be achieved using commands provided by an external controller. A series of input data can then be fed to the core, deriving the "correct" output from running the orientation graph. The inputs and derived outputs can then be combined to form a training dataset. The resulting training data can then be used to set the thresholds and associations. Alternatively, the processing core can generate its training data in place because when the processing core is asked to run new input data, the state of the processing core switches continuously between rounding activation and deactivation (i.e., labeled training data can be harvested while the processing core is operating and the rounding feature is off).

[0044] Figure 4 A flowchart 400 provides a set of methods for running a directional graph on a processing core. This flowchart includes a step 401 that associates a threshold with a cell of directional graph data. This step is illustrated by a data link between the identifier of the ANN's weight layer "layer_1_weight" and a fixed value of 0.8. As mentioned above, the value 0.8 can be the threshold itself or a value used later to derive the threshold. This step can be performed using components for association, including an external controller, a cache, and a core controller on the processing core.

[0045] The flowchart continues with step 402 of the orientation graph execution. Step 402 may involve running a single instruction related to the overall execution of the orientation graph. As shown, this instruction may be used for a single convolution operation between the orientation graph input and the weight layer in the orientation graph. The flowchart continues to step 403, where values ​​are generated using the cells of the orientation graph data. Step 403 can essentially be a sub-step of step 402, performed as part of the instruction execution. In this case, the two cells of the orientation graph data are "input" and "layer_1_weight", and the operation involves convolving the values ​​of the orientation graph data to produce the value of "layer_1_output". Step 403 can be performed by a component used to generate values ​​under the control of the core controller using operands obtained from cache memory, such as the main compute cluster of the processing core.

[0046] The flowchart continues to step 404, where a comparison value is generated. This step may involve passing the output of step 403 and the threshold to the input of a comparator for comparison of the comparator's output. In the processing core, this may involve passing the orientation map data to data register 410 and passing the threshold to threshold register 411. Data may be temporarily stored in these registers until the time is reached to run the comparison using comparator 412 when those registers are accessed. This timing may be performed by the controller of the processing core (e.g., core controller 301) to ensure that the comparator output 413 provided at any given time is related to the data currently stored by the processing core's rounding circuitry (e.g., the rounding circuitry in rounding block 310) for potential rounding operations. As shown, the threshold is associated with the operand used in step 403, and the associated threshold is passed to the comparator for comparison with the output of the operation in step 403. However, as previously stated, the threshold may already be associated with the output data structure for which step 403 generates a value. Regardless, while providing the associated threshold to the comparator, the values ​​of the units in the orientation map data generated in step 403 are also provided to the comparator. Step 404 can be performed by a set of components for generating the comparison (e.g., the comparator) and control elements (including a core controller for the processing core), which coordinate the delivery of the desired values ​​to it. In a particular scenario, the units of the orientation map data provided to the comparator are part of the operands or output of step 403. For example, the units of the orientation map data could be exponents of the fixed-point data values ​​used or generated in step 403.

[0047] The flowchart includes an optional step 405, in which a threshold is calculated. The threshold can be calculated based on the data values ​​used or generated in step 403 and pre-stored values. Variations of this scheme have been described above. Additionally, the value 0.8 shown may not be a fixed value, but may be an equation or algorithm independent of the execution of step 403, as in step 405. As previously mentioned, the threshold can be derived from values ​​in data structures not involved in the current or recent actions of the processing unit. Thus, step 405 can be fed independently into step 404, as opposed to a link from step 403 to step 404. The threshold can be generated using a set of entries in a data structure. Step 405 can be performed by components used to generate the threshold, such as a series of logic gates coupled to the destination memory and / or external controller and cache memory of the processing cluster of the aforementioned processing core.

[0048] The flowchart continues to step 406, where conditional rounding is performed based on the comparison pair value generated in step 404. Rounding can be performed by a rounding circuit block configured to provide a rounded value instead of the original value. The rounding circuitry can be located in the main data pipeline of the processing core, such as... Figure 3 As shown. The components used for rounding may include a rounding block, such as rounding block 310, and retrieve values ​​from hardware memory, modify received values, or ground or power specific circuit nodes to perform rounding operations, wherein the inputs to the block are the values ​​from step 403 as inputs and the comparisons from step 404 as control signals.

[0049] Figure 5-8 It is provided in Figure 4 A functional block diagram of an example of a specific method enabled and disclosed herein is provided. Block diagram 500 illustrates an association 501 formed between a cell of input orientation map data “Z” and a threshold “TH”. The associations and values ​​of Z and TH are all available prior to the flow shown in block diagram 500. Association 501 can take any form disclosed herein and can be created using any scheme disclosed herein. The controller of the processing core can utilize association 501 to ensure that the value of TH is available in the threshold register 502, while the value of Z is available in the data register 503. The two values ​​can then be compared using comparator 504 in rounding block 505. The comparator output 510 of comparator 504 can then be used to control the operation of rounding circuit 506 in rounding block 505. As a result, the calculated product involving values ​​X and Y, generated using calculation unit 507, will be conditionally rounded based on the pre-stored association 501 and the value TH. Thus, by using association 501, the threshold is applied whenever the value of the data structure used for association is calculated. See again Figure 3Rounding block 505 can replace rounding block 310, calculation unit 507 can be used as part of processing cluster 308, and input data registers 508 and 509 can be part of source memory 307.

[0050] Figure 6 It is used to achieve the correlation between the input orientation graph data and the threshold. Figure 4 A functional block diagram of a specific method disclosed herein is provided. Block diagram 600 illustrates an association 601 formed between a cell of the input orientation map data “X” and a threshold “TH”. The association and value of X and TH can all be obtained prior to the flow shown in block diagram 600. Association 601 can take any form disclosed herein and can be created using any scheme disclosed herein. The controller of the processing core can utilize association 601 to ensure that the value of TH is available in the threshold register 502, while the value of Z is available in the data register 503. Therefore, association 601 differs from association 501 in that the threshold is applied whenever the input orientation map data X is used as input to the calculation and for rounding the output of that calculation. Although there is a difference in this respect, Figure 5 and 6 The similarity lies in the fact that the threshold is not calculated independently (i.e., it is not utilized in block diagrams 500 and 600). Figure 4 Optional step 405 in calculating the threshold. Conversely. Figure 7 and 8 The diagram illustrates the scheme that requires step 405 to calculate the threshold.

[0051] Figure 7 It is used to achieve the correlation between the input orientation map data and the value used to calculate the threshold. Figure 4 A functional block diagram of a specific method disclosed herein. Block diagram 700 illustrates the association 701 formed between the cells and values ​​"V" of the input orientation map data "X". Block diagram 700 includes... Figure 4 The execution of step 405 in the diagram is not performed because the threshold is not a pre-stored value. Instead, as shown, the value for data structure V is loaded into register 703 and used by calculation block 702 along with the value for data structure Z to calculate the threshold for data structure TH. The form of calculation block 702 will depend on the specific way the threshold is calculated. For example, the calculation may involve a histogram of the values ​​in data structure Z or a more basic calculation, such as splitting the maximum value in data structure Z in half, and the complexity of calculation block 702 will be set to accommodate this calculation. The calculated value by calculation block 702 is then stored in threshold register 502. Then, a similar process is used... Figure 5 and 6The scheme uses the value of TH to conditionally round the value of data structure Z. The associations and values ​​of X and V can all be obtained before the flow shown in block diagram 700. Association 701 can take any form disclosed herein and can be created using any scheme disclosed herein. The controller of the processing core can utilize association 701 to ensure that the calculated value of TH is available in threshold register 502, while the value of Z is available in data register 503. Therefore, association 701 is used to ensure that a custom rounding value based on the value of output data Z is used for rounding the output data, wherein the custom rounding is set by a value associated with input data X, which is used to calculate the value of data structure Z. In other embodiments of the invention, association 701 may instead be already present between data structure Z and data structure V, and the flowchart will otherwise be similar to... Figure 7 The same as shown.

[0052] Figure 8 It is used to achieve the correlation between the input orientation graph data and the threshold. Figure 4 A functional block diagram of a specific method disclosed herein. Block diagram 800 utilizes, for example... Figure 7 That same association 701 is formed between the cell "X" and the value "V" in the input orientation map data. Furthermore, it can be used with... Figure 7 Modify in the same way Figure 8 This allows the association to potentially already exist between data structure Z and data structure V, while the flowchart is otherwise related to... Figure 8 The same as shown. Block diagram 800 contains... Figure 4 The execution of step 405 in the calculation is not performed because the threshold is not a pre-stored value. Instead, as shown, the value for data structure V is loaded into register 703 and, together with the value for data structure X, is used by calculation block 801 to calculate the threshold for data structure TH. This is consistent with... Figure 7 The difference in the scheme is that the input data is used to calculate the threshold, which is the opposite of the output data. The form of calculation block 801 will depend on the specific way the threshold is calculated, as in the example of calculation block 702. The calculated value is then stored in the threshold register 502. The value of TH is then used to use a method similar to... Figure 5 and 6 The scheme conditionally rounds the value of data structure Z. The associations and values ​​of X and V can all be obtained before the flow shown in block diagram 700. The controller of the processing core can use association 701 to ensure that the calculated value of TH is available in threshold register 502, while the value of Z is available in data register 503. Therefore, association 701 is used to ensure that the output data is rounded using a custom rounding value based on the value of the cell in the directional graph data X.

[0053] Figure 9A block diagram of a processing core 300 is provided, which is connected to external memory 900 and can receive control input from an external controller 901. In this configuration, the processing core can receive command input by accessing a known location in memory 900 where the external controller 901 loads commands. External memory 900 can store orientation graph data executed by the processing core, along with the kernel used to perform that execution, the associations and thresholds disclosed herein, and other data required for the operation of the processing core. In the illustrated case, external memory 900 is an L1 cache, and external controller 901 is a Reduced Instruction Set Computing (RISC) controller. However, other types of memory and controllers can serve the same purpose as the illustrated components. An external controller, such as external controller 901, can execute any method steps disclosed herein that require an external controller. A memory, such as memory 900, can relate to any method steps disclosed herein that require external memory. A higher-level PC controller can have a connection to external controller 901 and an optional orientation connection to core controller 301. The memory 900 can be shared among multiple processing cores, which cooperate to run the orientation graph in parallel.

[0054] Figure 9 The flowchart step 902 is also shown, which can be... Figure 4 The preceding step of step 401. Flowchart step 902 involves iterative machine learning techniques for training the processing core 300 to optimize the thresholds and associations required for the operation of the orientation graph. Training can be performed by combining the aforementioned PC controller with controller 901 and memory 900. Training may involve providing test inputs to controller 901 and final programming the thresholds and associations to be stored in memory 900 and other storage locations on configuration registers and processing core 300.

[0055] Figure 10The processing core 1000 is shown, which includes decompression 1002 and compression 1001 blocks in the main data path. The decompression and compression blocks can limit the amount of processing and storage bandwidth resources consumed by running the directed graph. The decompression and compression blocks can include compression and decompression circuitry for compressing binary data, or compression can be performed on the scale of the directed graph data values. In the case of sparse directed graphs, reducing the size of data structures that need to be moved through the system during compression and decompression can provide significant gains and relatively low overhead costs. Specifically, compression block 1001 can utilize run-length style encoding, which changes long strings of the same value into a sign representing the value and a sign representing the run length (i.e., "00000000" can be compressed into "80" representing eight consecutive zeros). Such an operation can be called run-length compression. The run-length selected value, with run-length encoding as the tracking value, can advantageously be set to the value to which rounding circuitry rounds these values. For example, if the compression system uses "0" run length encoding, the rounding circuit will be configured to round values ​​close to zero to 0, and will also facilitate rounding values ​​of oriented graph data structures that seem to have no decisive impact on the operation of the oriented graph to zero. Compression block 1001 can also be configured to encode run lengths for multiple values, wherein the multiple values ​​are selected based on which values ​​the rounding block rounds to.

[0056] The flowchart also includes flowchart step 1003, which can be... Figure 4 Run after step 406. The benefits of running these steps in this order have already been described above, as rounding increases the run length and enhances compression efficiency. Step 1003 can be performed via, for example... Figure 10 Compression is performed using components such as compression block 1001. Compression block 1001 may include a non-sparse value counter, a reset circuit that resets the counter when a sparse value is detected, and a register for storing the output of the counter when the counter is reset. The circuit elements can be configured to count the maximum number of consecutive non-sparse values ​​to measure an increment (deltas) of a given size. For example, the register can be configured to store, the counter can be configured to count, and the reset circuit can be configured to automatically reset when an X-bit value (worth) of a sparse value is detected. Specifically, the increment may be less than X by a power of two sparse values.

[0057] Figure 11The illustration shows a processing core 1100 that includes a metadata generation block 1101 in the main data path 306. The metadata generation block 1101 can evaluate the output of the main computing cluster and generate metadata reflecting the values ​​in that output. The metadata generation block 1101 can be metadata generator circuitry that uses sequential logic and stored values ​​for comparison to evaluate the output of a rounding block. The metadata generator circuitry may include configuration registers for receiving commands or other configuration information from the processing core controller. The metadata can then be associated with the header of the data structure where the core controller stores the output and / or the data structure where the output is stored. In particular, the metadata generation block 1101 can generate data used by the rounding block or other circuitry to derive appropriate thresholds for cells used in oriented graph data. For example, as previously described, thresholds can be generated based on a histogram or standard deviation of values ​​in the data structure, and this information can be generated by the metadata generation block 1101. Alternatively, or in combination, the metadata generation block can generate data indicating how sparse the output data from the computing cluster is. For example, if all values ​​in a data structure are zero, metadata generation block 1101 can generate a zero tag associated with that data structure. This tag can then be used at a later time to suppress computations using the data structure where the fact that all values ​​are zero is decisive. Thus, connecting metadata generation block 1101 to the output of the rounding block in the main data path yields significant benefits, as it generates more zero tags and further increases the efficiency of running the directed graph using the processing core.

[0058] The flowchart also includes flowchart step 1102, which can be... Figure 4 Run after step 406. The benefits of running these steps in this order have already been described above, as rounding can increase the number of zero tags or other metadata based on sparsity or repetition, which is then used to increase the efficiency of computations using associative data structures. Step 1102 can be run using components for generating zero tags, such as... Figure 11 Metadata generator block 1101 can include a comparator, logic gates, and an output register. The comparator can evaluate a set of values ​​from a rounding block for a fixed value (e.g., zero). If the evaluated set of values ​​from the rounding block is zero, the logic gates can generate a zero flag. The output register can then potentially store the zero flag along with several other zero flags, making it possible to eventually read the zero flag and associate it with the data evaluated by the comparator.

[0059] Although this specification has been described in detail with reference to specific embodiments of the invention, it should be understood that modifications, variations, and equivalents to these embodiments will readily occur to those skilled in the art upon understanding the foregoing. While the examples in this disclosure generally pertain to the processing core of executing convolution instructions, the same approach can be used to simplify the execution of any complex computations or complex logical operations, including multiplication and calculation. These and other modifications and variations can be made to the invention by those skilled in the art without departing from the scope of the invention, the scope of which is more specifically set forth in the appended claims.

Claims

1. A processing core, comprising: The processing cluster is configured to use input orientation graph data to generate values ​​for cells of the orientation graph data, wherein the values ​​for cells of the orientation graph data are generated during the operation of one layer of the orientation graph based on the values ​​of the input orientation graph data and the convolution operation between the values ​​of the input orientation graph data and the weight layers in the orientation graph; The core controller is configured to load a threshold into a threshold register when the value of a cell in the oriented map data is loaded into the data register; A comparator, coupled to the threshold register and the data register, is configured to generate a comparator output based on: (i) the threshold in the threshold register; and (ii) the value of a cell in the orientation map data in the data register. and The rounding circuit is configured to (i) receive the values ​​of the cells in the orientation graph data from the processing cluster; and (ii) conditionally round the values ​​of the cells in the orientation graph data based on the comparator output from the comparator.

2. The processing core according to claim 1 further includes: A series of logic gates configured to compute a threshold using at least one of the following: (i) the value of a cell in the orientation map data; The values ​​of the input orientation map data described in (ii).

3. The processing core according to claim 2, wherein: The cells of this orientation graph data are data structures used for the cumulative values ​​of the orientation graph.

4. The processing core according to claim 1 further includes: A computation block that calculates a threshold based on the cells of the second input orientation map data and at least one of the following: (i) the value of the cell in the orientation map data; The values ​​of the input orientation map data described in (ii).

5. The processing core according to claim 4, wherein: This calculation block uses at least one of the following to calculate the threshold: histogram calculation, average calculation, and standard deviation calculation.

6. The processing core according to claim 1 further includes: The correlation between the threshold and the cells in the orientation map data; and Specifically, when the value of a cell in the directional graph data is loaded into the data register, the core controller uses this association to load the threshold into the threshold register.

7. The processing core according to claim 1 further includes: The correlation between the threshold and the cells of the input orientation map data; and Specifically, when the value of a cell in the directional graph data is loaded into the data register, the core controller uses this association to load the threshold into the threshold register.

8. The processing core according to claim 1, further comprising: The correlation between the threshold and the cells in the orientation map data; The second association between the second threshold and the cells of the second orientation map data; The threshold and the second threshold are in the threshold set on the processing core; and The core controller uses associations to select thresholds from the threshold set.

9. The processing core according to claim 1, further comprising: A cache memory, which is a unit for storing orientation map data; and The threshold is located in the header of the cell in the directional graph data.

10. The processing core according to claim 1, further comprising: Cache memory; and The core controller is configured to move the directional map data to a cache memory after the cells have been rounded by the rounding circuit.

11. The processing core according to claim 1, further comprising: Cache memory; A compression circuit is configured to compress the cells of the orientation map data after the cells of the orientation map data have been rounded by the rounding circuit; and The core controller is configured to move the directional map data to a cache memory after the cells have been rounded by the rounding circuit and compressed by the compression circuit.

12. A method for processing a kernel, comprising: In the processing core, a threshold is associated with a cell in the orientation map data, which comes from the orientation map; Use the processing core to run the orientation graph; While running the orientation graph and using the processing cluster in the processing core, the values ​​of the cells of the orientation graph data are generated, wherein the values ​​of the cells of the orientation graph data are generated during the running of one layer of the orientation graph based on the values ​​of the input orientation graph data and the convolution operation between the weight layers in the orientation graph; Comparison of cell values ​​and thresholds for generating directional map data; and Based on this comparison, and using the rounding circuit in the processing core, the values ​​of the cells in the orientation map data are conditionally rounded.

13. The method of claim 12, further comprising: The threshold is calculated using a series of logic gates on the processing core. The values ​​of the cells used to generate the orientation map data are taken from the cells of the input orientation map data; and The threshold is calculated using at least one of the following: (i) the value of a cell in the orientation map data; and (ii) the value of a cell in the input orientation map data.

14. The method of claim 13, wherein: The cells of this orientation graph data are data structures used for the cumulative values ​​of the orientation graph.

15. The method of claim 12, further comprising: The threshold is calculated using a computational block on the processing core based on a cell of the second input oriented graph data and at least one of the following: (i) the value of a cell in the oriented graph data; The values ​​of the input orientation map data described in (ii).

16. The method of claim 15, wherein: The calculation uses at least one of histogram calculation, average calculation, and standard deviation calculation.

17. The method according to claim 12, wherein: This association forms a link between the threshold and the cells in the orientation map data; This comparison is performed using the threshold value loaded into the threshold register and the cell values ​​of the directional map data loaded into the data register; and Before generating this comparison, the following steps are performed: using the association to load the threshold into the threshold register and to load the values ​​of the cells in the orientation map data into the data register.

18. The method of claim 12, further comprising: In the processing core, a second threshold is associated with a cell of second orientation map data, which comes from an orientation map, wherein the threshold and the second threshold are in a threshold set on the processing core, and wherein the association forms an association between the threshold and the cell of the orientation map data; and Thresholds for generating comparisons are selected from a set of thresholds by using the correlation between thresholds and cells in the orientation map data.

19. The method of claim 12, further comprising: The cells of the orientation map data are stored in the cache memory of the processing core; and The threshold is located in the header of the cell in the directional graph data.

20. The method of claim 12, further comprising: After the values ​​of the cells in the orientation graph data have been conditionally rounded, the cells in the orientation graph data are moved to the cache memory of the processing core.

21. The method of claim 12, further comprising: After the values ​​of the cells in the orienteering plot data have already been rounded in the conditional rounding of the cell values ​​in the orienteering plot data, the cells in the orienteering plot data are compressed. and After compressing the cells of the orientation graph data, the cells of the orientation graph data are moved to the cache memory of the processing core.

22. A processing core, comprising: A component for generating values ​​of cells in a directional graph using input directional graph data, wherein the values ​​of cells in the directional graph data are generated during the operation of one layer of the directional graph based on the values ​​of the input directional graph data and a weight layer in the directional graph; A component used to compare the values ​​of the cells used to generate threshold and orientation map data; and A component used to conditionally round the values ​​of cells in the orientation map data based on this comparison.

23. The processing core according to claim 22, further comprising: A component used to associate units of threshold and orientation map data based on association; and The component used to generate the comparison uses this association.

24. The processing core according to claim 22, further comprising: The component used to calculate the threshold using at least one of the following: (i) the value of a cell in the orientation map data; The values ​​of the input orientation map data described in (ii).

25. The processing core according to claim 22, further comprising: A component for calculating a threshold using a cell of directional map data as a second input and at least one of the following: (i) the value of the cell of the directional map data; The values ​​of the input orientation map data described in (ii).

26. The processing core according to claim 22, further comprising: Cache memory; Core controller; and The core controller is configured to move the directional graph data to a cache memory after the unit has been rounded by the rounding component.

27. The processing core according to claim 22, further comprising: Cache memory; Core controller; A component used to compress the cells of the orientation map data after the cells of the orientation map data have been rounded by the component used for rounding; and The core controller is configured to move the directional graph data to a cache memory after the units used for rounding have been rounded and the units used for compression have been compressed.

Images