kernel quantization

Abstract

Apparatus, systems, and techniques are disclosed for kernel quantization, specifically for optimizing memory usage when performing matrix operations. In at least one embodiment, matrices are optimized to limit memory and storage requirements while minimizing precision loss of the sum of the matrix members.

Description

TECHNICAL FIELD

[0001] The present application relates generally to optimizing processing resources and minimizing information loss in the conversion of large data sets between data representation schemes. For example, at least one embodiment relates to a processor or computer system for converting a matrix of values stored as real number representations to fixed point representations with minimal loss of the sum of the values within the matrix. BACKGROUND

[0002] Matrix conversion from real to fixed point representations is a common operation that occurs in a variety of fields and applications. These filter kernels can be of arbitrary size and can or can not have symmetry. In many cases, the sum of the kernel coefficients is a particularly important property, especially in the case where the sum is an integer. This is an added constraint in addition to obtaining the required filter parameters for the required usage.

[0003] Reference signs

[0004] Figure 1 A representation of a real number matrix is shown;

[0005] Figure 2 A representation of a real number matrix with horizontal and vertical symmetry is shown;

[0006] Figure 3 A representation of a real number matrix with horizontal symmetry, vertical symmetry, and diagonal symmetry is shown;

[0007] Figure 4 A flow diagram of one or more embodiments is shown.

[0008] Figure 5 A flow diagram of one or more embodiments is shown.

[0009] Figure 6 A data center system according to at least one embodiment is shown; and

[0010] Figure 7 A computer system according to at least one embodiment is shown. DETAILED DESCRIPTION

[0011] Within a set of kernel coefficients, the values can be represented in a number of different formats, each with separate storage constraints. For example, for a given large number, the choice of one storage format (e.g., a real number format) has a range that can represent the number in memory. However, in many applications, a particular number format can be required for performing other operations, or can result in more efficient use of computer resources. For example, a number that is initially stored as a real number can be converted to a fixed point representation in order to perform matrix operations with fewer computer resources. However, when converting between number formats, the loss of precision can be significant. Furthermore, after performing additional operations on the converted values, the error values of the results can include additional, undesirable errors. For example, converting a value from a floating point to a fixed point representation introduces an error.

[0012] A set of kernel coefficients for a transformation between linear subspaces often includes very large quantities. The kernel coefficients are often represented by a matrix that can have an arbitrary dimension, depending on the application. Because the coefficients are very large numbers with high precision, and the matrix can have a large dimension, a large amount of storage space can be required. Furthermore, the mathematical operations performed on the coefficients typically use more processing resources than the same numerical operations on the same values stored with a lower precision. Storing the values so that each coefficient is stored using fewer bits can optimize the resource requirements, but reduces the precision.

[0013] In some embodiments, a matrix is received. The matrix can include a plurality of values, each value can be represented by a mantissa and an exponent, such as a floating point value. In some embodiments, a matrix can be received, and one or more operations performed on the matrix. For example, a matrix can be received with a request to perform matrix addition and / or subtraction. Because these operations can require a large amount of resources to perform on floating point values, one or more components can determine that the resources required to perform the operation exceed a threshold based on the size and dimension of the matrix. The matrix can be converted to a format that is more conducive to addition, subtraction, and / or multiplication, such as a fixed point representation. After conversion, the matrix can be checked again to determine if the resources required are acceptable. If so, the operation can be performed and the result returned. Because the sum of the values of the matrix is often an important value for certain applications, the conversion is performed to preserve the sum with minimal error.

[0014] This embodiment assumes that a matrix of values each represented as a floating point value is converted to a fixed point value. The conversion is performed so that the sum of the values is preserved with minimal error terms. For each initial value, the real number is represented as a floating point value with a mantissa and an exponent. The mantissa can be stored as a signed two's complement integer with a number of bits. The exponent value also has a number of bits. The resulting fixed point values are each represented by an integer value and a fractional value, each with a fixed number of bits.

[0015] As an example, numbers can be stored using a standard single-precision floating point representation. This representation allows values between 1.2 X 10 -38 and 3.4 X 10 38 to be represented, and can be represented by 32 bits, which are mantissa bits. In this representation, numbers can be stored with 32 bits of precision, and with an exponent that can be stored with 8 bits. Thus, with a sign bit, numbers can be stored in a total of 32 bits. For fixed point representations, the same amount of storage results in a much lower precision, and thus a much smaller range of values. However, arithmetic operations performed on fixed point representations are more efficient than the same operations performed on floating point representations.

[0016] For this embodiment, the following equations are used for the metes and bounds of a matrix of values (e.g., a filter kernel of coefficients). For ease of illustration, a 5 X 5 two-dimensional array is described. However, the same embodiment applies to matrices of any size or dimension.

[0017] Each kernel coefficient can be stored as a mantissa, m, and an exponent, e. For a given representation with B m mantissa bits, the integer mantissa bounds for the i-th kernel coefficient, m i are as follows:

[0018] M min ≤ m i ≤ M max , where

[0019] M min = -2^(B m - 1) and

[0020] M max ≡ (2^(B m - 1))-1

[0021] Further, the exponent, e i , for the i-th kernel coefficient is bounded to the range:

[0022] (E g - E i ) ≤ e i ≤ E g

[0023] E gmin ≤ E g < E gmax

[0024] where E g is a global integer scaling factor shared by the entire kernel, and E i is the maximum integer scaling adjustment of any coefficient. Thus, for the i-th kernel coefficient, C i can be defined as:

[0025]

[0026] This equation can be interpreted as an integer mantissa with an exponent of 2. It is these C values that are used to preserve the matrix with the minimum error in the sum of the converted values.

[0027] Referring to Figure 1 , a representation of a kernel coefficient matrix 100 of size 5X5 is shown. Each cell 105 includes a floating point value labeled K0 through K 24 , each of which will be converted to a fixed point value. For each value, a C value can be calculated (e.g., C0 from K0, C1 from K1).

[0028] Figure 1 The matrix contains a unique value for each cell. However, in some embodiments, the matrix can have horizontal symmetry, vertical symmetry, and / or diagonal symmetry, particularly when the matrix has kernel coefficients. For example, referring to Figure 2 , a matrix 200 with vertical and horizontal symmetry is shown. As shown, region 205 includes only cell values that are unique to the matrix. The remaining cells have one of these values such that the left side of the matrix is a mirror image of the right side of the matrix, a characteristic of vertical symmetry. In addition, the upper half of the matrix is a mirror image of the lower half of the matrix, a characteristic of horizontal symmetry. Referring to Figure 3 , a matrix with diagonal symmetry is shown. In matrix 300, both diagonals in both directions mirror around the center of the matrix, labeled K 12 , and shown as shaded cells. The values in region 305 are the only unique values in the matrix.

[0029] For any matrix, the values of r i can be calculated, defined as:

[0030]

[0031] For a matrix without symmetry and without repeated values, such as matrix 100, since 2 0 = 1, all r values will be 0. For a symmetric matrix, the r values will reflect the number of occurrences of each value. For example, for matrix 200, because K0 occurs four times in the matrix, r0 = 2. Similarly, the r values for matrix 200 are:

[0032] r0 = 2

[0033] r1 = 2

[0034] r2 = 1

[0035] r5 = 2

[0036] r6= 2

[0037] r7= 1

[0038] r 10 = 1

[0039] r 11 = 1

[0040] r 12 = 0

[0041] Similarly, matrix 300 has r values:

[0042] r0= 2

[0043] r1= 3

[0044] r2= 2

[0045] r6= 2

[0046] r7= 2

[0047] r 12 = 0

[0048] The sum S of the kernel coefficients can be computed as follows:

[0049]

[0050] The factor accounts for the number of times each coefficient appears in the kernel. Since the individual coefficients are quantized, the sum is also quantized. Instead of the previous definition of C i and the exponentiation operation,

[0051]

[0052] Since m i is an integer, the ith coefficient can only affect the sum in increments of This quantity is the granularity g i of the values in each cell of the matrix. Considering all g i values, the sum of the matrix g is the minimum of all g i values. Thus, the sum of the matrix will always be a multiple of g and it is not possible to have intermediate values.

[0053] Referring to Figure 4, a flowchart of the steps of an embodiment of the method described herein is provided. In some embodiments, one or more steps of the illustrated flowchart can be omitted. For example, in some embodiments, step 405 can be omitted where the coefficients have already been packed. In some embodiments, one or more additional steps can be included. For example, in some embodiments, step 435 can be followed by one or more processes or components that utilize the transformed coefficients. As illustrated, because steps 405 and 440 are outside of the core method 401, no knowledge of the matrix geometry, such as its symmetry, is required.

[0054] At step 405, the coefficients are packed for further processing. In this step, the matrix is examined to determine the r values of the matrix and the symmetry of the matrix. Any r values of 0 are removed from the list of results. Additionally, the C value for each value is determined based on the equation defined previously.

[0055] At step 410, the coefficients of the matrix are pre-processed. The e value and g value of each value in the matrix is determined. This step also includes determining the maximum e value of the matrix by examining the e value of each coefficient and selecting the highest value. This e value E g Related to the values in the matrix, the value has the most amount of information that can be lost in the transformation process.

[0056] At step 415, the e value of each matrix cell is clipped to fall within a valid range. This range is based on the value E g However, for each e value that is clipped, an additional error is introduced into the sum of the matrix S. This error is referred to as the residue error R. This value is the difference between the original value of each cell of the matrix and the quantized coefficient C i of each cell of the matrix. The sum of the R values of each value of the matrix results in the matrix residue R:

[0057]

[0058]

[0059]

[0060] By minimizing R, less error is introduced in the determination of the sum of the matrix. For each value, the mantissa, granularity, and R value are determined by the following equations:

[0061]

[0062]

[0063]

[0064] After step 415, for each value in the matrix, the e value (exponent value), g value (granularity value), and R value (residual value) are known. At this point, all R values are positive. However, the mantissa value is preliminary at this point. For any subsequent incremented mantissa, the R value for that value in the matrix will be negative. This is the primary mechanism for minimizing R.

[0065] At step 420, the values in the matrix with corresponding r values are sorted. This step is optional. However, a list of values sorted by decreasing g value can improve the efficiency of subsequent steps. Any known sorting method can be used to sort the list of values.

[0066] At step 425, the coefficients are grouped into blocks. Each block is processed, and then new blocks are assembled and processed until all coefficients have been processed. Processing coefficients from coefficients with higher g values to coefficients with lower g values can yield better results. Thus, sorting the list at step 420 improves the results of this step.

[0067] The blocks can be determined based on two criteria. First, a maximum block size can be specified. Once the block includes the specified number of values, the next block can be processed. Second, the blocks can be limited to a constant granularity value. Once a different granularity value is identified, the current block can be terminated, and a new block can be started. In some embodiments, one or both of these criteria can be specified. As described herein, the block size will be the number of values in the matrix, for a matrix of Figures 1 to 3 The number of values is 25 for a matrix of

[0068] At step 430, the mantissa of each value is adjusted to minimize the R value of the block. The minimized R value can be determined as follows:

[0069] If R > ag i ,

[0070] m i = m i + 1

[0071] R = R - g i

[0072] The value α is a unitless constant that controls the convergence of R for a given block. For example, in some embodiments, α can be set to 1 / 2, thereby constraining R to a value between -1 / 2 and 1 / 2 for the granularity of a given coefficient. Thus, for each coefficient in a block, if the current R value for that block is greater than half the g value for a given coefficient, R is decremented by the g value (a constant for that block) and the mantissa of that coefficient is incremented. The new R value is then compared to the next coefficient in the block. This continues until R no longer exceeds the convergence range.

[0073] At step 435, excess residue is processed. This involves distributing any excess residue remaining from processing the last block. This may occur if the last block contains coefficients with the smallest g value, but the coefficients of the last block are insufficient to constrain the R value of that block to within the convergence range, as described with respect to step 430. In some embodiments, this may not occur, such as when the sum of the input kernels is an integer.

[0074] For each coefficient of the final block, the mantissa is checked against the maximum and minimum number of bits allowed to represent the mantissa, as described previously. The delta value is calculated for that block as follows:

[0075]

[0076] Value α f is a unitless constant that controls the R convergence of the final block. For each coefficient in the last block, the mantissa is adjusted by Δ. If mi + Δ is not within the allowed mantissa range, the mantissa is adjusted to the maximum allowed mantissa or the minimum allowed mantissa. This clips the mantissa of the final block to fall within the valid range.

[0077] At step 440, the now-transformed coefficients are unpacked and the matrix is ​​repopulated. This step is the inverse of step 405 and includes filling the matrix with coefficients that are repeated in multiple cells, for example due to the symmetry of the matrix, and filling all other cells with the corresponding transformed coefficients. The resulting matrix is ​​a fixed-point coefficient matrix that is transformed to minimize the sum of the coefficients.

[0078] refer to Figure 5 , provides a flow chart that illustrates one or more embodiments described herein. In some embodiments, one or more steps may be omitted. In some embodiments, one or more additional steps may be included.

[0079] At step 505, the system receives a real number matrix. The matrix can be of any size and dimension. In some embodiments, the received matrix can have one or more symmetry properties, such as diagonal symmetry, vertical symmetry, or horizontal symmetry. The values stored in the matrix are real numbers, and can be stored, for example, in floating point representation. Thus, each value can be stored as a mantissa and exponent portion.

[0080] At step 510, a mathematical operation to be performed on the matrix is identified. In some embodiments, the mathematical operation can be provided to the system with the matrix. In some embodiments, one or more other components can provide the mathematical operation. The mathematical operation can be, for example, a matrix addition, subtraction, or one or more matrix transformations. In some embodiments, the mathematical operation can include an operation performed using only one matrix. For example, the operation can include manipulation of one or more values stored in the matrix. In some embodiments, the operation can include multiple matrices and / or vectors. For example, the operation can include multiplication of two matrices, multiplication of a matrix and a vector, and / or other operations that can include multiple matrices, each of which can be transformed prior to performing the operation.

[0081] At step 515, the system calculates computer resource requirements for performing the mathematical operation on the matrix. The computer resource requirements can be, for example, an estimate of the number of computer cycles that can be required to perform the operation, storage in memory that can be required to perform the operation, or one or more other computer resources that can be consumed during performance of the operation.

[0082] At step 520, the computer resource requirements can be compared to a threshold to determine whether transformation of the matrix will improve operation of the computing system. The threshold can be a fixed value, such as a maximum amount of computer power allocated to the operation, a time constraint for the computation, or a memory limit for performing the operation. In some embodiments, the system proceeds to the next step only if the computer resource requirements exceed the threshold. For example, the calculated computer resource requirements can exceed a threshold amount of time as a time constraint for performing the operation. In some embodiments, the transformation of the matrix can only be performed if the threshold is exceeded. In some embodiments, if the threshold is not exceeded, the mathematical operation can be performed on the original matrix.

[0083] At step 525, the matrix is transformed into a matrix of fixed point values. The transformation of the matrix can be accomplished using one or more techniques described herein. For example, the matrix transformation can be performed using the techniques shown. Figure 4 Thus, the transformation can be accomplished so that the transformed matrix retains the sum of the matrix with minimal error.

[0084] At step 530, a mathematical operation is applied to the converted matrix. For example, if the mathematical operation includes matrix addition, the system can perform matrix addition, which is less computationally complex between fixed point and floating point numbers. In some embodiments, the mathematical operation can include one or more other matrices that can have previously been converted from floating point to fixed point. At step 535, the result of applying the mathematical operation is provided to one or more systems for further processing.

[0085] Figure 6 An example data center 600 is shown in which at least one embodiment can be used. In at least one embodiment, data center 600 includes a data center infrastructure layer 610, a framework layer 620, a software layer 630, and an application layer 640.

[0086] In at least one embodiment, as shown in Figure 6 In at least one embodiment, data center infrastructure layer 610 can include a resource orchestrator 612, grouped computing resources 614, and node computing resources (“node C.R.s”) 616(1 )-616(N), where “N” represents any whole, positive integer. In at least one embodiment, node C.R.s 616(1 )-1016(N) can include, but are not limited to, any number of central processing units (“CPUs” or “processors”) including accelerators, field programmable gate arrays (FPGAs), graphics processors, and the like, memory devices (e.g., dynamic random access memory), storage devices (e.g., solid state or disk drives), network input / output (“NW I / O”) devices, network switches, virtual machines (“VMs”), power modules, cooling modules, and the like. In at least one embodiment, one or more of node C.R.s 616(1 )-1016(N) can be a server having one or more of the above computing resources.

[0087] In at least one embodiment, grouped computing resources 614 can include individual groups of node C.R.s housed within one or more racks (not shown) or housed within individual groups of many racks in various geographic locations (also not shown) in a data center. Individual groups of node C.R.s within grouped computing resources 614 can include grouped computing, networking, memory, or storage resources that can be configured or allocated to support one or more workloads. In at least one embodiment, several node C.R.s including CPUs or processors can be grouped within one or more racks to provide computing resources to support one or more workloads. In at least one embodiment, one or more racks can also include any number of power modules, cooling modules, and network switches in any combination.

[0088] In at least one embodiment, resource coordinator 612 may configure or otherwise control one or more node CRs 616(1)-616(N) and / or grouped computing resources 614. In at least one embodiment, resource coordinator 612 may comprise a software design infrastructure ("SDI") management entity for data center 600. In at least one embodiment, resource coordinator 612 may comprise hardware, software, or some combination thereof.

[0089] In at least one embodiment, Figure 6 As shown, framework layer 620 includes a job scheduler 622, a configuration manager 624, a resource manager 626, and a distributed file system 628. In at least one embodiment, framework layer 620 may include a framework for supporting software 632 of software layer 630 and / or one or more applications 642 of application layer 640. In at least one embodiment, software 632 or one or more applications 642 may include web-based service software or applications, such as those provided by Amazon Web Services, Google Cloud, and Microsoft Azure. In at least one embodiment, framework layer 620 may include, but is not limited to, a free and open source software web application framework, such as Apache Spark™ (hereinafter referred to as "Spark"), which may utilize distributed file system 628 for large-scale data processing (e.g., "big data"). In at least one embodiment, job scheduler 622 may include a Spark driver to facilitate scheduling of workloads supported by various layers of data center 600. In at least one embodiment, the configuration manager 624 can be capable of configuring different layers, such as the software layer 630 and the framework layer 620, including Spark and a distributed file system 628, to support large-scale data processing. In at least one embodiment, the resource manager 626 can manage the cluster or group computing resources mapped to or allocated to support the distributed file system 628 and the job scheduler 622. In at least one embodiment, the cluster or group computing resources can include the group computing resources 614 on the data center infrastructure layer 610. In at least one embodiment, the resource manager 626 can coordinate with the resource coordinator 612 to manage these mapped or allocated computing resources.

[0090] In at least one embodiment, software 632 included in software layer 630 can include software used by node C.R.s 616(1)-616(N), grouped computing resources 614, and / or at least a portion of distributed file system 628 of framework layer 620. One or more types of software can include, but are not limited to, Internet web page search software, email virus scanning software, database software, and streaming video content software.

[0091] In at least one embodiment, one or more applications 642 included in application layer 640 can include one or more types of applications used by at least a portion of node C.R.s 616(1)-616(N), grouped computing resources 614, and / or distributed file system 628 of framework layer 620. One or more types of applications can include, but are not limited to, any number of genomics applications, cognitive computing and machine learning applications (including training or inferencing software, machine learning framework software (e.g., PyTorch, TensorFlow, Caffe, etc.), or other machine learning applications used in conjunction with one or more embodiments.

[0092] In at least one embodiment, any of configuration manager 624, resource manager 626, and resource orchestrator 612 can implement any number and type of self-modifying actions based on any amount and type of data obtained in any technically feasible fashion. In at least one embodiment, self-modifying actions can relieve data center operators of data center 600 from making possibly poor configuration decisions, and can avoid underutilized and / or poorly performing portions of a data center.

[0093] In at least one embodiment, data centers can use CPUs, application specific integrated circuits (ASICs), GPUs, FPGAs, or other hardware to perform training and / or inferencing using resources described above. Moreover, one or more software and / or hardware resources described above can be configured to allow users to train or perform services that infer information, such as image recognition, speech recognition, or other artificial intelligence services.

[0094] Figure 7is a block diagram illustrating an exemplary computer system, which can be a system with interconnected devices and components, a system on a chip (SOC), or some combination thereof 700, formed with a processor that can include execution units to execute an instruction, according to at least one embodiment. In at least one embodiment, computer system 700 can include, without limitation, components such as processor 702 to employ execution units including logic to to execute instructions including algorithms for performing processes data, according to at least one embodiment. In at least one embodiment, computer system 700 can include processors such as Core TM or Nervana TM microprocessors, although other systems (including PCs, workstations, set-top boxes, etc. with other microprocessors) can also be used. In at least one embodiment, computer system 700 can execute a version of the WINDOWS operating system available from Microsoft Corporation of Redmond, Washington, although other operating systems (UNIX and Linux, for example), embedded software, and / or graphical user interfaces, can also be used.

[0095] Embodiments can be used in other devices such as handheld devices and embedded applications. Some examples of handheld devices include cellular phones, Internet Protocol devices, digital cameras, personal digital assistants (“PDAs”), and handheld PCs. In at least one embodiment, embedded applications can include a microcontroller, a digital signal processor (“DSP”), a system on a chip, network computers (“NetPCs”), set-top boxes, network hubs, wide area network (“WAN”) switches, or any other system that can perform one or more instructions according to at least one embodiment.

[0096] In at least one embodiment, computer system 700 can include, without limitation, a processor 702 that can include, without limitation, one or more execution units 708 to perform, e.g., machine learning model training and / or inferencing, in accordance with techniques described herein. In at least one embodiment, computer system 700 is a single processor desktop or server system, but in another embodiment, computer system 700 can be a multiprocessor system. In at least one embodiment, processor 702 can include, without limitation, a complex instruction set computer (“CISC”) microprocessor, a reduced instruction set computing (“RISC”) microprocessor, a very long instruction word (“VLIW”) microprocessor, a processor implementing a combo of instruction sets, or any other processor device, such as a digital signal processor. In at least one embodiment, processor 702 can be coupled to a processor bus 710 that can transmit data signals between processor 702 and other components in computer system 700.

[0097] In at least one embodiment, processor 702 can include, without limitation, level 1 (“L1”) internal cache memory (“cache”) 704. In at least one embodiment, processor 702 can have a single internal cache or multiple levels of internal cache. In at least one embodiment, cache memory can reside in the processor 702’s external. Other embodiments can also include a combination of internal and external cache memory depending on the specific implementation and requirements. In at least one embodiment, register file 706 can store different types of data within various registers including, without limitation, integer registers, floating point registers, status registers, and instruction pointer registers.

[0098] In at least one embodiment, execution unit(s) 708, including, without limitation, logic to perform integer and floating point operations, also reside in processor 702. In at least one embodiment, processor 702 can also include a microcode (“ucode”) read only memory (“ROM”) that stores microcode for certain macro instructions. In at least one embodiment, execution unit(s) 708 can include logic to handle a packed instruction set 709. In at least one embodiment, by including packed instruction set 709 in instruction set of a general- purpose processor 702, along with associated circuitry to fetch and execute the instructions, operations used by many multimedia applications can be performed using packed data in a general-purpose processor 702. In one or more embodiments, by using full width of processor’s data bus for operations on packed data, many multimedia applications can be accelerated and executed more efficiently, which can not require transferring smaller units of data across processor’s data bus to perform one or more operations on data elements one at a time.

[0099] In at least one embodiment, execution unit 708 may also be used in a microcontroller, an embedded processor, a graphics device, a DSP, and other types of logic circuits. In at least one embodiment, computer system 700 may include, but is not limited to, memory 720. In at least one embodiment, memory 720 may be implemented as a dynamic random access memory ("DRAM") device, a static random access memory ("SRAM") device, a flash memory device, or other storage device. In at least one embodiment, memory 720 may store one or more instructions 719 and / or data 721 represented by data signals that may be executed by processor 702.

[0100] In at least one embodiment, a system logic chip can be coupled to the processor bus 710 and the memory 720. In at least one embodiment, the system logic chip can include, but is not limited to, a memory controller hub ("MCH") 716, and the processor 702 can communicate with the MCH 716 via the processor bus 710. In at least one embodiment, the MCH 716 can provide a high-bandwidth memory path 718 to the memory 720 for instruction and data storage, as well as for storage of graphics commands, data, and textures. In at least one embodiment, the MCH 716 can direct data signals between the processor 702, the memory 720, and other components in the computer system 700, and bridge data signals between the processor bus 710, the memory 720, and the system I / O 722. In one embodiment, the system logic chip can provide a graphics port for coupling to a graphics controller. In at least one embodiment, the MCH 716 may be coupled to the memory 720 via a high-bandwidth memory path 718, and the graphics / video card 712 may be coupled to the MCH 716 via an Accelerated Graphics Port ("AGP") interconnect, a PCI Express bus, NVLink, AXI, or other proprietary bus 711 having similar characteristics.

[0101] In at least one embodiment, computer system 700 can use system I / O 722, which is a proprietary hub interface bus, to couple MCH 716 to I / O controller hub (“ICH”) 730. In at least one embodiment, ICH 730 can provide direct connections to some I / O devices and via a local I / O bus to other devices. In at least one embodiment, local I / O bus can include, without limitation, a high-speed I / O bus for connecting peripherals to memory 720, chipset, and processor 702. Examples can include, without limitation, audio controller 729, firmware hub (“Flash BIOS”) 728, wireless transceiver 726, data storage 724, legacy I / O controller 723 containing user input and keyboard interfaces 725, serial expansion port 727 (e.g., Universal Serial Bus (USB)), and network controller 734. Data storage 724 can include a hard disk drive, a floppy disk drive, a CD-ROM device, a flash memory device, or other mass storage device.

[0102] In at least one embodiment, Figure 7 Systems are shown that include interconnected hardware devices or “chips,” while in other embodiments, Figure 7 An exemplary system on a chip (“SoC”) can be shown. In at least one embodiment, components of computer system 700 can be fabricated Figure 7 The illustrated connections are to proprietary interconnects, standardized interconnects (e.g., PCIe), or some combination thereof. In at least one embodiment, one or more components of computer system 700 are interconnected using a compute express link (CXL) interconnect.

[0103] Other variations are within the spirit of the present disclosure. Thus, while the disclosed technology is susceptible to various modifications and alternative constructions, certain illustrated embodiments thereof are shown in the drawings and have been described above in detail. It should be understood, however, that there is no intention to limit the disclosure to the specific form or forms disclosed, but on the contrary, the intention is to cover all modifications, alternative constructions, and equivalents falling within the spirit and scope of the disclosure, as defined in the appended claims.

[0104] Unless otherwise indicated, the use of the terms "a" and "an" and "the" and similar referents in the context of describing the disclosed embodiments (especially in the context of the following claims) are to be construed to cover both the singular and the plural, unless otherwise indicated or clearly contradicted by context. The terms "comprising," "having," "including," and "containing" are to be construed as open-ended terms (i.e., meaning "including, but not limited to,") unless otherwise noted. The term "connected" is to be construed as partly or fully contained (i.e., connected to the extent that the parts or items can be contiguous to each other and / or in working contact

[0105] Unless otherwise indicated or contradicted by context, conjunctive language such as phrases of the form "at least one of A, B, and C," or "at least one of A, B, and C" is understood in the context to mean that the item, term, etc. can be A or B or C, or any non-empty subset of the set of A and B and C. For example, in a context having three members, the conjunctive phrases "at least one of A, B, and C" and "at least one of A, B, and C" refer to any of the following sets: {A}, {B}, {C}, {A, B}, {A, C}, {B, C}, {A, B, C}. Thus, such conjunctive language is generally not intended to imply that certain embodiments require the existence of at least one of A, at least one of B, and at least one of C. In addition, unless otherwise noted or contradicted by context, the term "plurality" denotes a plural state (e.g., "a plurality of items" denotes multiple items). A plurality is at least two items, but can be more when expressly or by context indicated. Further, unless otherwise noted or clearly dictated by context, the phrase "based on" means "based at least in part on" rather than "based solely on."

[0106] The operations of a process described herein can be performed in any suitable order unless otherwise indicated herein or otherwise clearly contradicted by context. In at least one embodiment, a process, such as those described herein (or variations and / or combinations thereof) is performed under the control of one or more computer systems configured with executable instructions to perform the operations of the process, and configured to implement the code (e.g., executable instructions, one or more computer programs or one or more applications) through hardware or combinations thereof on one or more processors to collectively perform the operations of the process. In at least one embodiment, the code, for example, in the form of computer programs, is stored on a computer-readable storage medium that includes a plurality of instructions executable by one or more processors. In at least one embodiment, a computer-readable storage medium is a non-transitory computer-readable storage medium that excludes transitory signals (e.g., a propagating transient electric or electromagnetic transmission) but includes non-transitory data storage circuitry (e.g., buffers, cache, and queues) of transceivers that are part of non-transitory computer-readable storage media. In at least one embodiment, code (e.g., executable code or source code) is stored on a set of one or more non-transitory computer-readable storage media (or other memory that stores executable instructions) having stored thereon executable instructions that, when executed by one or more processors of a computer system (i.e., as a result of being executed), cause the computer system to perform operations described herein. In at least one embodiment, a set of non-transitory computer-readable storage media includes multiple non-transitory computer-readable storage media and one or more individual non-transitory storage media of the multiple non-transitory computer-readable storage media lack all of the code, while the multiple non-transitory computer-readable storage media collectively store all of the code. In at least one embodiment, executable instructions are executed by different processors, e.g., non-transitory computer-readable storage media stores instructions and a main central processing unit (“CPU”) executes some instructions, while a graphics processing unit (“GPU”) executes other instructions. In at least one embodiment, different components of a computer system have separate processors and different processors execute different subsets of instructions.

[0107] Thus, in at least one embodiment, a computer system is configured to implement one or more services that individually or collectively perform operations of processes described herein, and such a computer system is configured with applicable hardware and / or software capable of implementing the operations. Moreover, a computer system implementing at least one embodiment of the present disclosure is a single device, and in another embodiment, is a distributed computer system that includes multiple devices operating in different manners such that the distributed computer system performs operations described herein and such that a single device does not perform all of the operations.

[0108] The use of any and all examples, or exemplary language (e.g., "such as") provided herein, is intended merely to better illuminate embodiments of the disclosure and does not pose a limitation on the scope of the disclosure unless otherwise claimed. No language in the specification should be construed as indicating any non-claimed element as essential to the practice of the disclosure.

[0109] All references, including publications, patent applications, and patents, cited herein are hereby incorporated by reference to the same extent as if each reference were individually and specifically indicated to be incorporated by reference and were set forth in its entirety herein.

[0110] In the description and in the claims, the terms "coupled" and "connected," along with derivatives thereof, can be used. It should be understood that these terms are not intended as synonyms for each other. Rather, in particular embodiments, "connected" or "coupled" can be used to indicate that two or more elements are in direct or indirect physical or electrical contact with each other. "Coupled" can also mean that two or more elements are not in direct contact with each other, but yet still co-operate or interact with each other.

[0111] Unless specifically stated otherwise, it can be appreciated that throughout the specification, terms such as "processing," "computing," "calculating," "determining," or the like, refer to the action and / or processes of a computer or computing system, or similar electronic computing device, that manipulates and / or transforms data represented as physical quantities (e.g., electronic) within the computing system's registers and / or memories into other data similarly represented as physical quantities within the computing system's memories, registers or other such information storage, transmission or display devices.

[0112] In a similar manner, the term "processor" can refer to any device or portion of a device that processes electronic data from registers and / or memory to transform that electronic data into other electronic data that can be stored in registers and / or memory. As a non-limiting example, a "processor" can be a CPU or GPU. A "computing platform" can include one or more processors. As used herein, a "software" process can include, for example, software and / or hardware entities such as tasks, threads, and intelligent agents that perform work over time. Also, each process can refer to multiple processes to execute instructions in series or in parallel, continuously or intermittently. Because a system can embody one or more methods and a method can be considered a system, the terms "system" and "method" are used interchangeably herein.

[0113] In this document, obtaining, acquiring, receiving, or inputting analog or digital data into a subsystem, computer system, or computer-implemented machine can be referenced. Obtaining, acquiring, receiving, or inputting analog and digital data can be accomplished in a variety of ways, such as by receiving data as a parameter of a function call or a call to an application programming interface, for example. In some embodiments, the process of obtaining, acquiring, receiving, or inputting analog or digital data can be accomplished by transmitting data via a serial or parallel interface. In another embodiment, the process of obtaining, acquiring, receiving, or inputting analog or digital data can be accomplished by transmitting data from a providing entity to an acquiring entity via a computer network. Providing, outputting, transmitting, sending, or rendering analog or digital data can also be referenced. In various examples, the process of providing, outputting, transmitting, sending, or rendering analog or digital data can be accomplished by transmitting data as an input or output parameter of a function call, a parameter of an application programming interface, or an interprocess communication mechanism.

[0114] Although the above discussion discusses example implementations of the described technology, other architectures can be used to implement the described functionality and are intended to be within the scope of this disclosure. Moreover, although specific allocations of responsibilities are defined above for discussion purposes, various functions and responsibilities can be allocated and divided in different ways depending on the circumstances.

[0115] Moreover, although the subject matter has been described in language specific to structural features and / or methodological acts, it is to be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or acts described. Rather, the specific features and acts are disclosed as example forms of implementing the claims.

Claims

1. A method for optimizing matrix conversion, comprising: Receives a matrix of real numbers, where each real number is represented by a mantissa and an exponent; identifying a mathematical operation to be performed on the matrix; determining computing resource requirements based on the matrix and the mathematical operation; converting the matrix into a transformed matrix when a required computational resource requirement exceeds a threshold, wherein the transformed matrix minimizes an error between a sum of the matrices and a sum of the transformed matrices, wherein the conversion is not performed when the required computational resource requirement does not exceed the threshold, wherein the real numbers are grouped into blocks for processing during the conversion; generating a result based on the mathematical operation and the transformed matrix; as well as The results are provided. The method of claim 1 , wherein the matrix is ​​a filter kernel. 3 . The method of claim 1 , wherein the matrix has at least one of horizontal symmetry, vertical symmetry, or diagonal symmetry, and wherein the matrix is ​​transformed based at least in part on the symmetry of the matrix.

4. The method of claim 1 , wherein converting the matrix into a transformed matrix comprises: One or more coefficients are pruned to reduce a number of storage bits for the one or more coefficients, and wherein the pruned storage bits are used to determine the error.

5. The method of claim 4, wherein clipping the one or more coefficients comprises: Reduces the number of storage bits for the mantissa of one or more coefficients.

6. The method according to claim 1, further comprising: determining a second computing resource requirement based on the converted matrix and the mathematical operation; as well as It is determined that the required second computing resource requirement does not exceed the threshold. The method of claim 1 , wherein the converted matrix comprises entries represented as fixed-point numbers.

8. The method of claim 1, wherein determining that the required computing resource requirement exceeds a threshold is based on at least one of a size of the matrix, a maximum storage limit of the matrix, and a maximum computing time to perform the operation on the matrix.

9. The method according to claim 1, further comprising: Receives a second matrix of real numbers; determining to perform the mathematical operation on the matrix and the second matrix; as well as converting the second matrix into a second transformed matrix, wherein the second transformed matrix minimizes an error between the sum of the second matrices and the sum of the second transformed matrices; The generating of the result is also based on the second transformed matrix.

10. A system comprising: one or more processors, the one or more processors including a math processor; a math processing memory operable to perform one or more mathematical operations; as well as a memory containing instructions that, when executed by the one or more processors, cause the system to: Receives a matrix of real numbers, where each real number is represented by a mantissa and an exponent; identifying a mathematical operation to be performed on the matrix by the math processor using the math processing memory; determining computing resource requirements based at least on the matrix and the mathematical operation; converting the matrix into a transformed matrix when a required computational resource requirement exceeds a threshold, wherein the transformed matrix minimizes an error between a sum of the matrices and a sum of the transformed matrices, wherein the conversion is not performed when the required computational resource requirement does not exceed the threshold, wherein the real numbers are grouped into blocks for processing during the conversion; providing the transformed matrix to the math processing memory so that a math processor generates a result based on the math operation and the transformed matrix; as well as The results are provided.

11. The system of claim 10, wherein the memory further comprises instructions to: determining a second computing resource requirement based on the converted matrix and the mathematical operation; and It is determined that the required second computing resource requirement does not exceed the threshold.

12. The system of claim 10, wherein the memory further comprises instructions to: One or more coefficients are pruned to reduce a number of storage bits for the one or more coefficients, and wherein the pruned storage bits are used to determine the error.

13. The system of claim 12, wherein clipping the one or more coefficients comprises: Reduces the number of storage bits for the mantissa of one or more coefficients.

14. The system of claim 12, wherein the transformed matrix includes entries represented as fixed-point numbers.

15. The system of claim 12, wherein the memory further comprises instructions to: The result is used as a filter kernel to perform one or more image processing applications.

16. A non-transitory machine-readable medium having stored thereon a set of instructions that, if executed by one or more processors, cause the one or more processors to at least: Accepts a square matrix of real numbers, where each real number is represented by a mantissa and an exponent; identifying a mathematical operation to be performed on the matrix; determining computing resource requirements based on the matrix and the mathematical operation; converting the matrix into a transformed matrix when a required computational resource requirement exceeds a threshold, wherein the transformed matrix minimizes an error between a sum of the matrices and a sum of the transformed matrices, wherein the conversion is not performed when the required computational resource requirement does not exceed the threshold, wherein the real numbers are grouped into blocks for processing during the conversion; generating a result based on the mathematical operation and the transformed matrix; as well as The results are provided.

17. The non-transitory machine-readable medium of claim 16, wherein the set of instructions further comprises instructions to: determining a second computing resource requirement based on the transformed matrix and the mathematical operation; and It is determined that the required second computing resource requirement does not exceed the threshold.

18. The non-transitory machine-readable medium of claim 16, wherein the set of instructions further comprises instructions to: One or more coefficients are pruned to reduce a number of storage bits for the one or more coefficients, and wherein the pruned storage bits are used to determine the error.

19. The non-transitory machine-readable medium of claim 16, wherein clipping the one or more coefficients comprises: Reduces the number of storage bits for the mantissa of one or more coefficients.

20. The non-transitory machine-readable medium of claim 16, wherein the converted matrix includes entries represented as fixed-point numbers.

Images