Compression of numerical data and sparse matrices in deep learning

Abstract

To provide a device and a method for enhancing performance of a deep learning application.SOLUTION: A method includes: receiving a data packet including data samples of plural cycles; passing, for each cycle of the data samples, the data samples of the cycles to a compression dictionary; identifying tags corresponding to the respective data samples from the compression dictionary; and storing the tags as compressed data, for compressing the data samples into compressed cycle data. Values of the data samples identified by a first tag or a second tag are excluded from the compressed cycle data, at the same time the data samples identified by the first tag are compressed using the first tag, and the data samples identified by the second tag are compressed using the second tag.SELECTED DRAWING: Figure 6A

Description

Technical Field

[0001] This disclosure generally relates to machine learning, and more particularly to the compression of numerical data and sparse matrices in deep learning.

Background Art

[0002] Unlike natural intelligence shown by humans or animals, artificial intelligence (AI) is intelligence shown by machines. As machines become more capable, machine learning becomes possible. Machine learning is a method by which an empowered machine learns to recognize the surrounding situation and change its own operation without human intervention. Its goal is to enable the machine to learn by itself using the provided data and make accurate predictions.

[0003] Deep learning is part of machine learning. It is the next evolution of machine learning. Deep learning algorithms are inspired by how information is processed by the human brain. Compared with machine learning, deep learning can handle large-scale data sets, is more accurate, utilizes more calculations, and makes better predictions.

[0004] Deep learning applications need to process and transmit / receive a large amount of numerical data. In order to save routing bandwidth and storage capacity, the data is compressed.

[0005] Efficient implementation of compression / decompression can improve the performance of deep learning applications.

Brief Description of Drawings

[0006] To allow for a detailed understanding of the above-described features of this embodiment, a more detailed description of the embodiments briefly summarized above can be provided by referring to embodiments some of which are shown in the accompanying drawings. However, it should be noted that the accompanying drawings show typical embodiments and should therefore not be considered as limitations on the scope. The drawings are not to scale. Generally, the same reference numerals are used throughout the drawings and accompanying descriptions to refer to identical or similar parts.

[0007] [Figure 1] This is a block diagram of an exemplary computing system, which can be used to perform compression of numerical data and sparse matrices in deep learning according to the implementation of this disclosure.

[0008] [Figure 2] This disclosure provides an exemplary compression engine that performs compression of numerical data and sparse matrices in deep learning.

[0009] [Figure 3A] This disclosure provides an exemplary data sample format based on its implementation.

[0010] [Figure 3B] This disclosure shows an example of a data packet that should be compressed by the implementation of this disclosure.

[0011] [Figure 4] This disclosure provides an exemplary compression dictionary for compressing numerical data and sparse matrices in deep learning, based on its implementation.

[0012] [Figure 5A] This disclosure provides an exemplary data sample of a data packet compressed using the compression of numerical data and sparse matrices in deep learning.

[0013] [Figure 5B]This disclosure provides an example of a compressed data packet using the compression of numerical data and sparse matrices in deep learning, as implemented in this disclosure.

[0014] [Figure 6A] This is a flowchart illustrating an embodiment of a method for compressing numerical data and sparse matrices in deep learning.

[0015] [Figure 6B] This flowchart illustrates another embodiment of a method for compressing numerical data and sparse matrices in deep learning.

[0016] [Figure 7] This is a schematic diagram of an exemplary electronic computing device for enabling the compression of numerical data and sparse matrices in deep learning, according to several embodiments. [Modes for carrying out the invention]

[0017] The implementation of this disclosure describes the compression of numerical data and sparse matrices in deep learning. In computer engineering, computing architecture is a set of rules and methods that describe the function, organization, and implementation of a computer system. Today's computing systems are expected to handle large workloads for execution while delivering near-zero wait responsiveness and excellent performance. Thus, computing architectures are constantly changing (e.g., improving) to cope with demanding workloads and increasing performance expectations.

[0018] Examples of large workloads include neural networks, artificial intelligence (AI), and machine learning. Such workloads are becoming more widespread as they are implemented in a variety of computing devices, such as personal computing devices and business computing devices. Furthermore, with the increasing use of large-scale machine learning and neural network workloads, new silicon is being manufactured specifically targeted at running these large workloads. This new silicon includes dedicated hardware accelerators (e.g., graphics processing units (GPUs), field-programmable gate arrays (FPGAs), and vision processing units (VPUs)) customized to run large neural networks using data parallelism or model parallelism.

[0019] Artificial intelligence (AI), including machine learning (ML), deep learning (DL), and / or other artificial machine-driven logic, enables machines (e.g., computers, logic circuits, etc.) to use a model to process input data and produce an output based on patterns and / or associations previously learned by the model, utilizing a training process. For example, a model can be trained with data to recognize patterns and / or associations and to follow such patterns and / or associations when processing input data, so that other inputs result in outputs that match the recognized patterns and / or associations.

[0020] There are many different types of machine learning models and / or machine learning architectures. In some of the examples disclosed in this application, convolutional neural networks are used. By using convolutional neural networks, it becomes possible to classify objects in areas such as images and natural language processing. Generally, machine learning models / architectures suitable for use in the exemplary approaches disclosed in this application can include convolutional neural networks. However, other types of machine learning models such as recurrent neural networks, feedforward neural networks, etc. can be used additionally or alternatively.

[0021] Generally, the implementation of an ML / AI system includes two phases: a learning / training phase and an inference phase. In the learning / training phase, a training algorithm is used to train a model to operate according to patterns and / or correlations based on, for example, training data. Generally, a model includes internal parameters that guide how input data is converted into output data, and converts input data into output data through, for example, a series of nodes and connections within the model. Further, hyperparameters (such as, for example, the learning rate, the number of layers used in the machine learning model, etc.) are used as part of the training process to control how learning is performed. Hyperparameters are defined as training parameters that are determined before the training process begins.

[0022] Based on the type of the ML / AI model and / or the expected output, different types of training may be performed. For example, supervised training uses the input and the corresponding expected (e.g., labeled) output to select the parameters of the ML / AI model that reduce the model error (e.g., by iterating over combinations of selection parameters). As used in this application, labeling refers to the expected output of the machine learning model (e.g., classification, expected output value, etc.). Alternatively, unsupervised training (e.g., used in deep learning, a subset of machine learning, etc.) involves inferring patterns from the input and selecting parameters for the ML / AI model (e.g., without the benefit of an expected (e.g., labeled) output).

[0023] As described above, ML / AI is a way for an empowered machine to perceive its surroundings and learn to change its behavior without human influence. ML / AI aims for the machine to learn on its own using the provided data and be able to make accurate predictions. DL algorithms focus on how information is processed by the human brain. Compared with ML, DL can handle larger data sets, is more accurate, utilizes more calculations, and makes better predictions.

[0024] DL applications process and transmit / receive large amounts of numerical data. To save routing bandwidth and storage capacity, the data is compressed. To improve the performance of DL applications, an efficient implementation of compression / decompression is used. Hardware compression / decompression technologies used in ML and DL workloads aim to achieve an efficient hardware implementation of such technologies. Concerns about achieving improved performance in ML / DL applications lie in the area of memory bandwidth consumption. Compression addresses this concern and enables silicon solutions for rapidly growing neural networks.

[0025] Compression can involve computationally intensive data retrieval and calculations. Therefore, implementing compression techniques often introduces both data routing latency and timing problems in deep submicron silicon. Deep submicron silicon can refer to the high-density integration of transistors in integrated circuit (IC) designs. In some implementations, deep submicron technology may refer to ICs manufactured using, for example, 0.18 μm, 0.13 μm, or 7 nm process technologies.

[0026] Data routing delays introduced by compression techniques can degrade system performance. Compression of wide data (e.g., 512 bits) is limited by timing closures in deep submicron silicon. Conventional compression approaches do not compress data within a single clock cycle. Instead, each data cycle in conventional approaches takes two cycles for compression. As a result, conventional approaches use two compression engines to maintain data throughput. Each compression engine processes every other packet. The compressed data from both compression engines is then merged back into a single data stream. The data packet size is... Because the duration of compression of a fixed data size varies and depends on the data, the alternating packet distribution between two compression engines does not evenly distribute the compression bandwidth. Over time, the slower compression engine may leave the other engine idle. Therefore, conventional compression techniques using two compression engines do not meet data throughput requirements. Furthermore, managing packet distribution and merging the compression results not only introduces data latency but also results in the introduction of additional circuitry such as FIFOs and large amounts of logic. In addition to area and power costs, the ability to split and merge data also adds data latency and power consumption costs to conventional compression techniques.

[0027] In ML / DL learning applications, the output of activation functions is generally sparse (e.g., containing a large number of zeros), making it suitable for compression, while other data may be nearly random. Thus, the overall compression gain depends on how well the sparse data is compressed. Conventional compression techniques utilize hundreds of bits and are therefore not optimal.

[0028] For example, timing closure in conventional compression techniques in 7nm process technology presents another challenge, showing that it only satisfies about 50% of the speed target. This degradation means that, on average, computing elements spend half their time either waiting for input data to arrive or waiting for generated data to be acquired, making routing a bottleneck in system performance.

[0029] Furthermore, conventional compression techniques utilize compression ratios that are not optimized for typical DL applications. In DL applications, the activation function generates many 0s and 1s in feature map data. Moreover, quantization also introduces more 0s and 1s in the weight values. These are the best-case scenarios, as they can dramatically reduce the size. Conventional compression techniques treat values ​​of 0 specially, while values ​​of 1 receive no special treatment. In this case, if values ​​like 1 are not subjected to special compression treatment, the size reduction achieved is limited to half of the original data size. The compressed data size is at least half of the original data size. Even if the input data 0s are specially treated, the minimum size of the compression ratio is 81% (i.e., 19% of the original data size, which is not optimal).

[0030] Furthermore, conventional compression techniques are packet-based. For example, the first 512 bits of each data packet are encoded with a silicon-hardcoded set of hash values ​​that cannot be customized for each application. In real-world applications, the chances of hitting this set of silicon-hardcoded hash values ​​are minimal. Therefore, when using conventional compression techniques, the first 512 bits of data in each packet are rarely compressed.

[0031] The implementation of this disclosure addresses the above shortcomings by providing a single compression engine matched to the full data throughput with the optimal compression gain during timing requests. In one implementation, the compression engine can perform compression of deep learning numerical data and sparse matrices by receiving data packets containing data samples from multiple cycles. For each cycle of data samples, the compression engine can pass the cycle's data samples to a compression dictionary. The compression engine uses the compression dictionary to identify a tag for each of the data samples in the cycle. In one implementation, the compression dictionary includes at least a first tag for data with a value of 0 and a second tag for data with a value of 1. The compression engine can then compress the data samples into compressed cycle data by storing the tags as compressed data.

[0032] In one implementation, data samples identified by the first or second tag are compressed using the first or second tag, while the values ​​of the data samples identified by the first or second tag are excluded from the compressed cycle data. The compression dictionary further includes tags that show partial matches and tags that show mismatches with values ​​in the compression dictionary. The tag values ​​are stored in the compressed data cycle, followed by the portion of the data sample with a partial match (e.g., the sign and exponent values), and then the entire portion of the data sample that does not match. The compressed data cycles of the data packet are then concatenated together within the data packet, and the data packet is zero-padded at the end of the compressed data cycle.

[0033] The implementation of this disclosure provides compression techniques for ML and DL activation functions such as Rectified Linear Unit (ReLU), sigmoid, and Tanh. The implementation of this disclosure simplifies data management, provides lower latency, and provides higher compression gains. Furthermore, the implementation can be carried out with fewer gates and lower power. Conventional compression methods implemented for DL ​​or AI focus on either weight or activation compression. The implementation of this disclosure provides a solution that applies to all data of ML or DL ​​workloads, including weights and activations. Because the implementation of this disclosure can compress both weight and activation data, it provides a better overall compression ratio.

[0034] Furthermore, conventional compression techniques utilize run-length coding of non-zero values, where non-zero values ​​and their positions are coded. In contrast, the implementation of this disclosure focuses on large chunks of zeros, leveraging spare bits in the packet header field, resulting in a smaller compressed packet size. Moreover, decompressing large chunks of zeros takes less time than decode run-length coding. As a result, the implementation of this disclosure benefits silicon timing in the implementation.

[0035] In conclusion, the implementation of this disclosure provides an innovative compression algorithm with improved compression gain for sparse and non-sparse data packets, along with a feasible implementation optimized for ML and DL applications. The implementation results in fewer gate counts and lower power utilization. Furthermore, the implementation is validated with lower latency and satisfies various timing requirements (e.g., operation above 2.05 GHz on a 7nm process node).

[0036] Figure 1 is a block diagram of an exemplary computing system that can be used to implement the compression of numerical data and sparse matrices in deep learning by an implementation of the present disclosure. The exemplary computing system 100 may be implemented as a component of another system, such as a mobile device, wearable device, laptop computer, tablet, desktop computer, server, etc. In one embodiment, computing system 100 may include, but is not limited to, a server-based game platform; a game console including a game and media console; a mobile gaming console, a handheld game console, or an online game console, or be integrated into these. In some embodiments, computing system 100 is part of a mobile internet-connected device such as a mobile phone, smartphone, tablet computing device, or laptop with small internal storage capacity.

[0037] In some embodiments, the computing system 100 is typically part of an Internet of Things (IoT) device, which is a resource-constrained device. IoT devices may include embedded systems, wireless sensor networks, control systems, automation (including home and building automation), and other devices and equipment (such as lighting fixtures, thermostats, home security systems, cameras, and other consumer electronics) that support one or more common ecosystems and can be controlled using devices associated with that ecosystem, such as smartphones and smart speakers.

[0038] The computing system 100 may also include, combine with, or be integrated into: wearable devices such as smartwatch wearable devices; smart eyewear or clothing enhanced with augmented reality (AR) or virtual reality (VR) capabilities that provide visual, auditory, or tactile output to assist the real-world visual, auditory, or tactile experience, or otherwise provide text, audio, graphics, video, holographic images or videos, or haptic feedback; other augmented reality (AR) devices; or other virtual reality (VR) devices. In some embodiments, the computing system 100 includes or is part of a television or set-top box device. In one embodiment, the computing system 100 may include, combine with, or be integrated into: autonomous vehicles such as buses, tractor-trailers, automobiles, motorcycles or electric bicycles, airplanes or gliders (or any combination thereof). The autonomous vehicle may use the computing system 100 to process the environment sensed around the vehicle.

[0039] As illustrated in the example, in one embodiment, the computing system 100 may include any number and types of hardware and / or software components, such as (but not limited to) graphics processing units ("GPU", general-purpose GPU (GPGPU), or simply "graphics processors") 112, hardware accelerators 114, central processing units ("CPU" or simply "application processors") 115, memory 130, network devices, drivers, etc., as well as input / output (I / O) sources 160 such as touchscreens, touch panels, touchpads, virtual or conventional keyboards, virtual or conventional mice, ports, connectors, etc. The computing system 100 may also include an operating system 110 that acts as an interface between the hardware and / or physical resources of the computing system 100 and the user. In some implementations, the computing system 100 may include one or more combinations of a CPU 115, a GPU 112, and / or a hardware accelerator 114 on a single system-on-chip, or it may not include a GPU 112 or a visual output (e.g., a hardware accelerator 114).

[0040] As used herein, “hardware accelerator,” such as hardware accelerator 114, refers to a hardware device configured to provide efficient processing. In particular, hardware accelerators can be used to provide offloading of some processing task from a central processing unit (CPU) or other general-purpose processor, and hardware accelerators may be intended to provide better processing task processing efficiency than software running on the CPU or other processor. Hardware accelerators may include, but are not limited to, graphics processing units (GPUs), visual processing units (VPUs), neural processing units, artificial intelligence (AI) processors, field-programmable gate arrays (FPGAs), or application-specific integrated circuits (ASICs).

[0041] The GPU 112 (or graphics processor 112), hardware accelerator 114, and / or CPU 115 (or application processor 115) of the exemplary computing system 100 may include a model trainer 125 and a model executor 105. Although the model trainer 125 and model executor 105 are depicted as part of the CPU 115, in some implementations the GPU 112 and / or hardware accelerator 114 may include the model trainer 125 and model executor 105.

[0042] The exemplary model executor 105 accesses input values ​​(for example, via an input interface (not shown)) and processes these input values ​​based on a machine learning model stored in the model parameter memory 135 of memory 130 to generate output values ​​(for example, via an output interface (not shown)). Input data may be received from one or more data sources (for example, via one or more sensors, via a network interface, etc.). However, input data may be received from external devices in any way (for example, via wired and / or wireless communication channels). In some examples, multiple different types of inputs may be received. In some examples, input data and / or output data are received via the inputs and / or outputs of a system that is a component of the computing system 100.

[0043] In the example shown in Figure 1, the exemplary neural network parameters stored in the model parameter memory 135 are trained by the model trainer 125, and as a result, the input training data (e.g., received via a training value interface (not shown)) yields output values ​​based on the training data.

[0044] In the example shown in Figure 1, the model trainer 125 and / or model executor 105 use the compression engine 150 when processing the model during training and / or inference. The exemplary model executor 105, exemplary model trainer 125, and exemplary compression engine 150 are implemented by one or more logic circuits, such as hardware processors. In some examples, one or more of the exemplary model executor 105, exemplary model trainer 125, and exemplary compression engine 150 may be implemented by the same hardware component (e.g., the same logic circuit) or by different hardware components (e.g., different logic circuits, different computing systems, etc.). However, any other type of circuitry, such as one or more analog or digital circuits, logic circuits, programmable processors, application-specific integrated circuits (ASICs), programmable logic devices (PLDs), field-programmable logic devices (FPLDs), digital signal processors (DSPs), etc., may be used additionally or alternatively.

[0045] In the examples disclosed herein, the model executor 105 executes a machine learning model. The exemplary machine learning model may be implemented using a neural network (e.g., a feedforward neural network). However, any other past, present, and / or future machine learning topology and / or architecture, such as a CNN, may be used additionally or alternatively.

[0046] To run the model, the exemplary model executor 105 accesses the input data. The exemplary model executor 105 applies the model (defined by model parameters stored in the model parameter memory 135, e.g., neural network parameters including weights and / or activations) to the input data.

[0047] The exemplary model parameter memory 13 in the example shown in Figure 1 can be implemented by any memory, storage device, and / or storage disk for storing data, such as flash memory, magnetic media, or optical media. Furthermore, the data stored in the exemplary model parameter memory 135 may be in any data format, such as binary data, comma-separated data, tab-separated data, or structured query language (SQL) structures. In the illustrated example, the model parameter memory 135 is shown as a single element, but the model parameter memory 135 and / or any other data storage elements described herein can be implemented by any number and / or type of memory. In the example shown in Figure 1, the exemplary model parameter memory 135 stores model weighting parameters used by the model executor 105 to process inputs to produce one or more outputs as output data.

[0048] In the examples disclosed herein, the output data may be information classifying the received input data (for example, as determined by the model executor 105). However, any other type of output that can be used for any other purpose may be used additionally or alternatively. In the examples disclosed herein, the output data may be output by an input / output (I / O) source 160 that displays an output value. However, in some examples, the output data may be provided as an output value to another system (for example, another circuit, an external system, a program executed by the computing system 100, etc.). In some examples, the output data may be stored in memory.

[0049] The exemplary model trainer 125 shown in Figure 1 compares the expected output (e.g., received as training values ​​in the computing system 100) with the output generated by the exemplary model executor 105 to determine the amount of training error, and updates the model parameters (e.g., model parameter memory 135) based on the amount of error. After training iterations, the amount of error is evaluated by the model trainer 125 to determine whether training should continue. In the examples disclosed herein, an error is identified when the input data does not yield the expected output. That is, the error is expressed as the number of inaccurate outputs under the condition that the input has the expected output. However, any other approach to expressing the error, such as the percentage of input data points that produced the error, may be used additionally or alternatively.

[0050] Model trainer 125 determines whether the training error is less than the training error threshold. If the training error is less than the training error threshold, the model has been trained to produce a sufficiently small amount of error, and no further training is performed. In the example disclosed herein, the training error threshold is 10 errors. However, any other threshold may be used additionally or alternatively. Furthermore, other types of factors may be considered when determining whether model training is complete. For example, the number of training iterations performed and / or the length of time elapsed during the training process may be considered.

[0051] The training data used by the model trainer 125 includes example inputs (corresponding to the input data expected to be received) and expected output data. In the examples disclosed herein, the exemplary training data is provided to the model trainer 125 to enable the model trainer 125 to determine the amount of training error.

[0052] In the examples disclosed herein, an exemplary model trainer 125 and an exemplary model executor 105 use a compression engine 150 to perform compression of deep learning numerical data and sparse matrices. In one implementation, the compression engine 150 performs compression of deep learning numerical data and sparse matrices for the model executor 105 and / or model trainer 125 by receiving data packets containing data samples for multiple cycles. For each cycle of data samples, the compression engine 150 can pass the cycle's data samples to a compression dictionary. The compression engine 150 uses the compression dictionary to identify a tag for each of the data samples in the cycle. In one implementation, the compression dictionary includes at least a first tag for data with a value of 0 and a second tag for data with a value of 1. The compression engine 150 can then compress the data samples into compressed cycle data by storing the tags as compressed data. In one implementation, data samples identified by the first or second tag are compressed using the first or second tag when excluding the values ​​of the data samples identified by the first or second tag from the compressed cycle data.

[0053] Further discussion and detailed explanation of the implementation of the exemplary compression engine 150 by model trainer 125 and / or model executor 105 are provided below in relation to Figure 2-6.

[0054] The exemplary I / O source 160 illustrated in Figure 1 enables communication of the model stored in the model parameter memory 135 with other computing systems. In some implementations, the I / O source 160 may include, but is not limited to, network devices, microprocessors, cameras, robotic eyes, speakers, sensors, display screens, media players, mice, touch-sensitive devices, etc. In this way, a central computing system (e.g., a server computer system) can train the model and distribute it to edge devices for use (e.g., to perform inference operations using the model). In the examples disclosed herein, the I / O source 160 is implemented using an Ethernet network communicator. However, any other past, present, and / or future type of communication technology may be used additionally or alternatively to enable the model to communicate with individual computing systems.

[0055] An exemplary method of implementing computing system 100 is shown in Figure 1, but one or more of the elements, processes, and / or devices shown in Figure 1 may be combined, divided, rearranged, omitted, removed, and / or implemented in any other way. Furthermore, the exemplary model executor 105, exemplary model trainer 125, exemplary compression engine 150, I / O source 160, and / or more generally the exemplary computing system 100 in Figure 1 may be implemented by hardware, software, firmware, and / or any combination of hardware, software, and / or firmware. Therefore, for example, the exemplary model executor 105, the exemplary model trainer 125, the exemplary compression engine 150, the exemplary I / O source 160, and / or more generally the exemplary computing system 100 in Figure 1 can be implemented by one or more analog or digital circuits, logic circuits, programmable processors, programmable controllers, graphics processing units (GPUs), digital signal processors (DSPs), application-specific integrated circuits (ASICs), programmable logic devices (PLDs), and / or field-programmable logic devices (FPLDs).

[0056] Some implementations of this disclosure provide software and / or firmware implementations of at least one of the exemplary model executor 105, exemplary model trainer 125, exemplary compression engine 150, exemplary I / O source 160, and / or more generally, exemplary computing system 100 of Figure 1. Such implementations may include non-transient computer-readable storage devices or storage disks, such as memory, digital versatile disks (DVDs), compact discs (CDs), Blu-ray discs, etc. (including software and / or firmware). Furthermore, exemplary computing system 100 of Figure 1 may include, in addition to or instead of, those shown in Figure 1, one or more elements, processes, and / or devices, and / or any one or more or all of the illustrated elements, processes, and devices. As used in this case, the phrase "in communications" (including variations thereof) encompasses direct communications and / or indirect communications through one or more intermediate components, and further encompasses selective communications that do not utilize direct physical (e.g., wired) communications and / or regular communications, but rather periodic intervals, scheduled intervals, non-periodic intervals, and / or one-off events.

[0057] Figure 2 shows an exemplary compression engine 200 for performing compression of deep learning numerical data and sparse matrices according to an implementation of the present disclosure. In one implementation, the compression engine 200 is the same as the compression engine 150 described with respect to Figure 1. The compression engine 200 is further shown to include a tag encoder 210, a compression dictionary 220, a packet generator 230, and / or a padding component 240. The exemplary compression engine 200 in Figure 2 may include, in addition to or instead of, one or more elements, processes, and / or devices shown in Figure 2, and / or any one or more or all of the elements, processes, and devices shown. In some examples, one or more of the exemplary tag encoder 210, exemplary compression dictionary 220, exemplary packet generator 230, and / or exemplary padding component 240 may be implemented by the same hardware components (e.g., the same logic circuit) or by different hardware components (e.g., different logic circuits, different computing systems, etc.). However, any other type of circuitry may be used additionally or alternatively, such as one or more analog or digital circuits, logic circuits, programmable processors, application-specific integrated circuits (ASICs), programmable logic devices (PLDs), field-programmable logic devices (FPLDs), digital signal processors (DSPs), etc.

[0058] In some implementations, the compression engine 200 can receive input data. For example, the input values ​​may be input weight values ​​202 and / or input activation values ​​204 of an ML model. The compression engine 200 can compress the input weights 202 and / or input activations 204 as part of the training and / or inference process associated with the ML model. In some implementations, the compression engine 200 can decompress (or decompress) the input weights 202 and / or input activations 204 in the reverse process of the compression process described herein. For the sake of clarity, the following description discusses the compression process. However, implementations of this disclosure also include decompression processes that are similar to the compression process described herein.

[0059] In some implementations, input waits 202 and / or input activations 204 may arrive in the form of an uncompressed data packet containing multiple data samples. These data samples may include, for example, input waits or input activations. In some implementations, the data packet may contain multiple cycles of data samples (e.g., 32 cycles of 32 data samples in a 512-bit data packet).

[0060] Figure 3A shows exemplary data sample formats 310, 320, and 330 that can be processed by the compression engine in an implementation of the present disclosure. In a particular implementation, exemplary data sample formats 310, 320, and 330 may represent data samples that can be processed by the compression engine 200 described in relation to Figure 2. A compression engine like the 200 is capable of processing a variety of arbitrary data types, including, but not limited to, floating-point 16, floating-point 32, single-precision 8, single-precision 16, and single-precision 32. Other data types not enumerated herein may be processed by an implementation of the present disclosure.

[0061] Data sample 310 illustrates an exemplary half-precision floating-point 16 (fp16) data type. As shown, the half-precision fp16 data sample 310 includes a 1-bit sign 312, a 5-bit exponent 314, and an 11-bit mantissa 416 (10 explicitly stored). The exponent may be referred to as the literal in this case. The mantissa may also be referred to as the mantissa in this case. With respect to the half-precision fp16 data type, the exponent has a bias of 15, and integers are (-1) sign bit ×2 exponent-15 It is possible to define it as ×1.significantbits2.

[0062] Data sample 320 illustrates an exemplary bf16 (bf16) data type. As shown in the illustration, the bf16 data sample 320 is shown to contain two 16-bit portions. Each 16-bit portion of the bf16 data sample 320 contains a 1-bit sign 321, 325, an 8-bit exponent 322, 326, and a 7-bit mantissa 323, 328. As shown by data samples 310, 320, the bf16 data sample 320 has more exponent bits and fewer mantissa bits than the fp16 data sample 310.

[0063] Data sample 330 illustrates an exemplary floating-point 32 (fp32) data type. As shown, the fp32 data sample 330 includes a 1-bit sign 332, an 8-bit wide exponent 334, and a 23-bit precision mantissa 336.

[0064] Figure 3B shows an exemplary data packet 350 that is compressed by an implementation of the present disclosure. In one implementation, the exemplary data packet 350 may represent a data packet that can be processed by the compression engine 200 described with respect to Figure 2. The exemplary data packet 350 is shown in uncompressed form. As shown, in one example, the data packet 350 includes a header 370 of one cycle 360 ​​and a data sample 365 of 32 cycles 360. Data packets 350 in other formats may be used and processed by the compression engine in an implementation of the present disclosure, and are not limited to the examples shown in Figure 3B.

[0065] In the exemplary data packet 350 shown in Figure 3B, each cycle 360 ​​of the data packet 350 has 32 BF16 data samples, indicated as "data 0" through "data 1023". Each BF16 data sample has 16 bits. In one example, each data sample 365 may be identical to the data sample 320 described with respect to Figure 3A. Thus, the data cycles 360 in the data packet 350 have a total of 32 × 16 = 512 bits.

[0066] Referring again to Figure 2, in one implementation, the tag encoder 210 can parse a data packet, such as the data packet 350 in Figure 3B, to obtain individual data samples. The data samples within the data packet are compressed cycle by cycle. The tag encoder 210 can pass one cycle of data samples to the compression dictionary 220. Each data sample in the data packet is compressed using the compression dictionary 220. In one implementation, the compression dictionary 220 may be a lookup table or a hash table. In one implementation, the compression dictionary 220 has 16 entries. However, it is possible to implement a compression dictionary 220 of other sizes, and this disclosure is not limited to a 16-entry compression dictionary 220. In one implementation, the compression dictionary 220 is subdivided into multiple compression subdictionaries. For example, the compression dictionary 220 may be divided into two compression dictionaries (e.g., two hash tables) each with 8 entries.

[0067] Figure 4 shows an exemplary compression dictionary 400 for compressing numerical data and sparse matrices in deep learning according to an implementation of the present disclosure. In some implementations, the compression dictionary 400 is the same as the compression dictionary 220 described with respect to Figure 2. The compression dictionary 400 can represent one of the compression subdictionaries described above. Thus, in some implementations, two copies (or, in some embodiments, more than two copies) of the compression dictionary 400 may constitute the compression dictionary 220 described with respect to Figure 2.

[0068] In the implementation of this disclosure, the lower half of the data sample of each cycle (e.g., the lower half data sample 380 in Figure 3B) is processed by one compression sub-dictionary 400, while the upper half of the data sample of each cycle (e.g., the upper half data sample 390 in Figure 3B) is processed by the other compression sub-dictionary 400. The lower half data sample 380 and the upper half data sample 390 are compressed independently and in parallel by the compression engine 200. The compression dictionary 400 is used by the tag encoder 210 to examine each 256-bit chunk of data (e.g., the lower half data sample 380 or the upper half data sample 390) to check for any of the following cases: (1) value 0 or 1; (2) partial match of data sample: the matching tactic value in the dictionary, its remaining sign bit and literal value are different; or (3) no match. As shown in Figure 4, the compression dictionary 400 includes tag values ​​410 corresponding to the above descriptions 420.

[0069] For example, a data sample with a word value equal to 1 receives a tag value 410 of 0x5, a data sample with a word value equal to 0 receives a tag value 410 of 0x6, and a data sample that does not match any of the descriptions 420 in the compressed dictionary 400 receives a tag value 410 of 0x7. A data sample that partially matches any of the description sets 420 in the compressed dictionary 400 receives a corresponding tag value 410 between 0x0 and 0x4, depending on the partial match. In some implementations, a partial match means that the mantissa value of the data sample matches one of the descriptions 420 in the compressed dictionary 400.

[0070] The tag encoder 210 uses the compression dictionary 220 to determine a tag value 410 corresponding to each data sample in the cycle and provides the determined tag value 410 to the packet generator 230. The packet generator 230 compresses the data samples in the cycle by assigning a special tag value 410 to all data samples with a value of 0s or 1s in the packet header and not storing any further information about these data samples (for example, the actual value of the data sample is excluded from the compressed cycle data). For data samples that have a non-0 or non-1 data value but whose dummy values ​​match (i.e., partial match), the corresponding tag value 410 stored in the compression dictionary 400 is assigned, and the sign and exponential (literal) values ​​of these data samples are stored. For data samples that do not have a matching value, a tag value 410 indicating that they do not match is assigned, and the entire portion of these data samples is stored. The entire portion of the data sample refers to the memory of both the significant (mantissa) and exponent (literal) values ​​for the non-matching data samples.

[0071] In the implementation of this disclosure, the packet generator 230 compresses data samples from the data packets of each cycle into compressed cycle data. The packet generator 230 can combine information from identified tag values ​​410 for the upper and lower half of the data samples of the cycle (e.g., from each sub-dictionary of the compression dictionary 220) into a single compressed cycle data for the data packets of each cycle.

[0072] Figure 5A shows exemplary compressed cycle data 500 for a data packet based on compression of deep learning numerical data and sparse matrices according to an implementation of the present disclosure. In one implementation, the compressed cycle data may be a compressed version of one cycle 360 ​​of data samples 365 of the data packet 350 described with respect to Figure 3B. In one implementation, the packet generator 230 of the compression engine 200 described with respect to Figure 2 can generate the compressed cycle data 500 using compression of deep learning numerical data and sparse matrices according to an implementation of the present disclosure.

[0073] In the implementation of this disclosure, in order to maintain byte alignment, the packet generator 230 first groups all tags identified for a given cycle of data samples into a tag portion 510 of the compressed cycle data. The tag portion 510 may be followed by a size byte 520 indicating the size of the compressed cycle data 560-585. In some implementations, the size byte 520 is added in the compressed data packet 550 to indicate the end of the current compressed cycle data. In some implementations, the size byte 520 may be used by a decompressor to immediately allocate the next compressed cycle data.

[0074] A size byte 520 may be followed by literal data 530. The literal data 530 may contain stored information about partially matching data samples within a cycle. The stored information about partially matching data samples may contain the exponential values ​​of the partially matching data samples. In this case, the mantissas of these data samples match the compression dictionary 220 as indicated by their associated tag values ​​and are therefore not stored in the compressed cycle data 500 to save on compression.

[0075] Finally, the literal data 530 may be followed by the mismatched data 540. The mismatched data may contain stored information about the mismatched data sample of the cycle. The stored information about the mismatched data sample may contain the exponent and mantissa of the mismatched data sample.

[0076] The packet generator 230 can concatenate the compressed cycle data 500 generated for each cycle of the data packet into the compressed data packet. In one implementation, the compressed data packet is output by the compression engine as an output weight 252 or an output activation 254.

[0077] Figure 5B shows an exemplary compressed data packet 550 based on the compression of deep learning numerical data and sparse matrices according to an implementation of the present disclosure. In one implementation, the compressed data packet 550 may be a compressed version of the data packet 350 described with respect to Figure 3B. In one implementation, the packet generator 230 of the compression engine 200 described with respect to Figure 2 can generate the compressed data packet 550 by utilizing the compression of deep learning numerical data and sparse matrices according to an implementation of the present disclosure.

[0078] The compressed data packet 550 may include a compressed data packet header 555 and a cycle 552 of compressed cycle data 560-585. In one implementation, each compressed cycle data 560-585 corresponds to a cycle of uncompressed data samples of an uncompressed data packet, such as the data packet 350 described with respect to Figure 3B. In one example, compressed data 0 560 may correspond to a compressed data sample for cycle 0 360 of data packet 350. Furthermore, each of the compressed data 560-585 may be in the same format as the compressed cycle data 500 described with respect to Figure 5A. As described above, each compressed cycle data 560-585 is concatenated into the compressed data packet 550. Any compressed data extending beyond the allocated bits of cycle 552 is continued in the next cycle 552 of the compressed data packet 550. For example, compressed data 1 565a,565b is shown to be partially stored in cycle 0 552 (i.e., 565a) and cycle 1 552 (i.e., 565b). Similarly, compressed data 3 575a,575b is shown to be partially stored in cycle 1 552 (i.e., 575a) and cycle 2 552 (i.e., 575b).

[0079] After the last compressed data (for example, compressed data 31-585), the remainder of the compressed data packet is zero-padding up to the next 512-bit data boundary of the compressed data packet 550-590. Zero-padding means filling a determined portion of the data packet with zero values. In some implementations, the padding component 240 of the compression engine 200, as described with respect to Figure 2, can perform zero-padding.

[0080] In the implementation of this disclosure, the compression dictionary 220 may be implemented as a “running dictionary”. The running dictionary version of the compression dictionary 220 uses matching results from preceding cycles of the data packet for partial match values ​​in the compression dictionary 220. For example, in a compression dictionary that provides five possible partial match options, the first five values ​​found in the current 256 bits are stored and used to match the next 256-bit data of the same half (e.g., the lower half or upper half of the data sample) in the next cycle of the data sample, until the last data of the packet is reached. The compression dictionary 220 may be initialized with default values ​​for partial match values, for example, based on data profiling that includes a known range of data values ​​or values ​​that frequently occur in a particular workload. These default partial match values ​​used to initialize the compression dictionary 220 may be stored in registers of the processor, hardware accelerator, and / or other hardware providing the compression engine 200.

[0081] The implementation of this disclosure allows the software to program a default set of compression dictionaries to "on the fly" partial match values. For example, if the user knows the range of the next data sample, it is possible to load default values ​​within that range into the compression dictionary in real time (e.g., while an ML / AI / DL workload is running) to increase the chances of a hit in the data sample cycle.

[0082] Alternatively, the compression dictionary can retain the most frequently used values ​​from the data of the last cycle in a previous packet and continue to apply them to the first data of the next packet. That is, the implementation of this disclosure can utilize the compression dictionary from the last cycle of data samples in the current data packet to the first cycle of data samples in the next packet of data, in the same manner that the compression dictionary is utilized during the cycle of data samples in a data packet. Since most data crossing packet boundaries belongs to a single data stream, they are within a range of similar size, and the probability of a match is similar to the probability of matching data within a data cycle in a packet. This technique can increase the likelihood that the first data sample of the next data packet will match or partially match.

[0083] In the implementation of this disclosure, if the input data samples are outputs from activation functions such as ReLU, sigmoid, and tanh functions, most data sample values ​​are likely to be 0 or 1. For example, packet header bits can be used to represent all 0s and all 1s within 16 × 32 samples. By removing the entire data portion within the packet, a 100% compression gain (2KB + 1 header to header) can be achieved. This is an improvement over conventional approaches to 384-bit compressed data (e.g., 81% compression gain) when all data sample values ​​are all 0.

[0084] In the implementation of this disclosure, if the input data is in the form of a sparse data matrix and most data sample values ​​are 0 or 1, there may be a small number of non-0 or non-1 data samples. This can be handled by extending the header bits to represent half or a quarter of the size of all 0s and 1s. A single-bit header represents an entire data packet of all 0s and all 1s. Using two bits in the header allows each bit to specify all 0s and 1s in a half-size packet. Using three bits allows each bit to specify all 0s and 1s in a quarter-size packet. If all data sample values ​​are all 0s or 1s, the size is compressed with the header bits without storing any information in the data. Otherwise, it is compressed as a partial match. If a quarter-size packet is all 0s or 1s, it is compressed without storing any data. This can cover all sparse matrices.

[0085] Figure 6A is a flowchart illustrating an embodiment of Method 600 for compressing numerical data and sparse matrices in deep learning. Method 600 can be implemented by processing logic which may include hardware (e.g., circuits, dedicated logic, programmable logic, etc.), software (e.g., instructions executed on a processing device), or a combination thereof. More specifically, Method 600 may be implemented in one or more modules as a set of logic instructions stored in a machine- or computer-readable storage medium such as RAM, ROM, PROM, firmware, flash memory, etc., in configurable logic such as a programmable logic array (PLA), field-programmable gate array (FPGA), complex-programmable logic device (CPLD), or in fixed-function logic hardware using circuit technology such as application-specific integrated circuits (ASICs), complementary metal-oxide-semiconductor (CMOS) or transistor-transistor logic (TTL) technology, or any combination thereof.

[0086] The process of Method 600 is shown in a linear sequence for the sake of brevity and clarity in its presentation; however, it is assumed that any number of these steps can be executed in parallel, asynchronously, or in different orders. Furthermore, for the sake of brevity, clarity, and ease of understanding, many of the components and processes described with respect to Figure 1-5 may not be repeated or described below. In one implementation, a compression engine such as the compression engine 150 in Figure 1 or the compression engine 200 in Figure 2 may execute Method 600.

[0087] Method 600 begins in processing block 610, where a data packet is received. In some implementations, the data packet contains data samples for multiple cycles. In block 620, blocks 630, 640, and 650 are executed for each cycle of the data samples. In the block, the data samples for each cycle are passed to the compressed dictionary.

[0088] Next, in block 640, a tag is identified for each data sample using a compression dictionary. In one implementation, the compression dictionary includes at least a first tag for data with a value of 0 and a second tag for data with a value of 1. Finally, in block 650, the cycle data samples are compressed into compressed cycle data. In one implementation, the data samples are compressed by storing the tags as compressed data, and the data samples identified by the first or second tag are compressed using the first or second tag, while excluding the values ​​of the data samples identified by the first or second tag.

[0089] Figure 6B is a flowchart illustrating another embodiment of Method 660 for compressing numerical data and sparse matrices in deep learning. Method 660 can be implemented by processing logic which may include hardware (e.g., circuits, dedicated logic, programmable logic, etc.), software (e.g., instructions executed on a processing device), or a combination thereof. More specifically, Method 660 may be implemented in one or more modules as a set of logic instructions stored in a machine- or computer-readable storage medium such as RAM, ROM, PROM, firmware, flash memory, etc., in configurable logic such as PLA, FPGA, CPLD, etc., or in fixed-function logic hardware using circuit technology such as ASIC, CMOS, or TTL technology, or any combination thereof.

[0090] The process of Method 660 is shown in a linear sequence for the sake of brevity and clarity in its presentation; however, it is assumed that any number of these steps can be executed in parallel, asynchronously, or in different orders. Furthermore, for the sake of brevity, clarity, and ease of understanding, many of the components and processes described with respect to Figures 1-5 may not be repeated or described below. In one implementation, a compression engine such as the compression engine 150 in Figure 1 or the compression engine 200 in Figure 2 may perform Method 660.

[0091] Method 660 begins in processing block 665, in which the processor stores a group of tags identified for each data sample in a data packet cycle in the compressed cycle data. In one implementation, the tags include at least a first tag indicating that the data sample has a value of 0, or a second tag indicating that the data sample has a value of 1. In block 670, the processor may store in the compressed cycle data a size byte indicating the size of the compressed cycle data, following the group of tags.

[0092] In block 675, the processor may store, in the compressed cycle data, a partial value of a data sample having a partially matching tag within a group of tags, following the size in bytes. In one implementation, the partial value includes the sign and exponent values ​​of the data sample identified by the partially matching tag. Subsequently, in block 680, the processor may store, in the compressed data sample, a complete portion of a data sample identified by a mismatching tag within a group of tags, following the partial value. In one implementation, the complete portion includes the exponent and mantissa values ​​of the data sample identified by the mismatching tag.

[0093] In block 685, the processor may concatenate compressed cycle data in a compressed data packet with other compressed cycle data in the data packet. Finally, in block 690, the processor may add zero padding following the compressed data sample to the end of the compressed data packet.

[0094] Figure 7 is a schematic diagram of exemplary electronic computing devices for enabling compression of numerical data and sparse matrices in deep learning according to several embodiments. In some embodiments, the computing device 700 includes one or more processors 710, including one or more processor cores 718 and one or more neural network accelerators 764, the neural network accelerators 764 which implement compression of numerical data and sparse matrices in deep learning as shown in Figure 1-6. In some embodiments, the computing device 700 includes a hardware accelerator 768, the hardware accelerator includes a machine learning model 784. In some embodiments, the computing device accelerates a neural network implementing the machine learning model 784 using compression of numerical data and sparse matrices in deep learning as provided in Figure 1-6.

[0095] The computing device 700 may further include one or more of the following: a cache 762, a graphical processing unit (GPU) 712 (which may be a hardware accelerator in some implementations), a wireless input / output (I / O) interface 720, a wired I / O interface 730, system memory 740 (e.g., memory circuitry), power management circuitry 750, a non-temporary storage device 760, and a network interface 770 for connecting to a network 772. The following description provides a brief general description of the components that make up the exemplary computing device 700. The exemplary, non-limiting computing device 700 may include a desktop computing device, a blade server device, a workstation, or similar devices or systems.

[0096] In the embodiment, the processor core 718 can execute a machine-readable instruction set 714, read data and / or the instruction set 714 from one or more storage devices 760, and write data to one or more storage devices 760. Those skilled in the art will understand that the illustrated embodiment and other embodiments may be implemented in other processor-based device configurations, including portable or handheld electronic devices such as smartphones, portable computers, wearable computers, consumer electronics, personal computers (PCs), network PCs, minicomputers, server blades, and mainframe computers. For example, the machine-readable instruction set 714 may include instructions for implementing compression of numerical data and sparse matrices for deep learning, as provided in Figure 1-6.

[0097] The processor core 718 may include any number of hardwired or configurable circuits, some or all of which may include programmable or configurable combinations of electronic components, semiconductor devices, and / or logic elements located in a PC, server, or other computing system, in part or in whole, that are capable of executing processor-readable instructions.

[0098] The computing device 700 includes a bus or similar communication link 716 that communicatively connects and facilitates the exchange of information and / or data between various system components, including a processor core 718, a cache 762, a graphics processor circuit 712, one or more wireless I / O interfaces 720, one or more wired I / O interfaces 730, one or more storage devices 760, and / or one or more network interfaces 770. Although the computing device 700 may be referred to singly herein, it is not intended to limit embodiments to a single computing device 700, because in some embodiments there may be more than one computing device 700 incorporating, including, or encompassing any number of communicatively coupled, deployed, or remotely networked circuits or devices.

[0099] The processor core 718 may include any number or type of currently available or future-developed devices, or any combination thereof, capable of executing a machine-readable instruction set.

[0100] The processor core 718 may include (or be coupled with) any currently available or future-developed single or multi-core processor or microprocessor, such as one or more system-on-a-chip (SOCs); central processing units (CPUs); digital signal processors (DSPs); graphics processing units (GPUs); application-specific integrated circuits (ASICs), programmable logic units, or field-programmable gate arrays (FPGAs). Unless otherwise stated, the configuration and operation of the various blocks shown in Figure 7 are conventional designs. Such blocks are therefore not described in further detail herein but will be understandable to those skilled in the art. The bus 716 interconnecting at least some components of the computing device 700 may use any currently available or future-developed serial or parallel bus structure or architecture.

[0101] The system memory 740 may include read-only memory (ROM) 742 and random-access memory (RAM) 746. A portion of the ROM 742 may be used to store or otherwise hold a basic input / output system (BIOS) 744. The BIOS 744 provides basic functionality to the computing device 700 by, for example, causing the processor core 718 to load and / or execute one or more machine-readable instruction sets 714. In embodiments, at least a portion of the one or more machine-readable instruction sets 714 causes at least a portion of the processor core 718 to provide, create, generate, transition, and / or function as a dedicated, specific, and particular machine, such as a word processing machine, a digital image acquisition machine, a media playback machine, a gaming system, a communication device, a smartphone, or something similar.

[0102] The computing device 700 may include at least one wireless input / output (I / O) interface 720. At least one wireless I / O interface 720 may be communicatively connected to one or more physical output devices 722 (such as a haptic device, video display, audio output device, or hardcopy output device). At least one wireless I / O interface 720 may be communicatively coupled to one or more physical input devices 724 (such as a pointing device, touchscreen, keyboard, or haptic device). At least one wireless I / O interface 720 may include any wireless I / O interface currently available or to be developed in the future. Exemplary wireless I / O interfaces include, but are not limited to, BLUETOOTH®, Near Field Communication (NFC), and similar technologies.

[0103] The computing device 700 may include one or more wired input / output (I / O) interfaces 730. At least one wired I / O interface 730 may be communicatively coupled to one or more physical output devices 722 (such as haptic devices, video displays, audio output devices, hardcopy output devices, etc.). At least one wired I / O interface 730 may be communicatively coupled to one or more physical input devices 724 (such as pointing devices, touchscreens, keyboards, haptic devices, etc.). The wired I / O interfaces 730 may include any I / O interfaces currently available or to be developed in the future. Exemplary wired I / O interfaces include, but are not limited to, Universal Serial Bus (USB), IEEE 1394 ("FireWire"), etc.

[0104] The computing device 700 may include one or more communicably coupled non-transient data storage devices 760. The data storage devices 760 may include one or more hard disk drives (HDDs) and / or one or more solid-state storage devices (SSDs). The one or more data storage devices 760 may include any storage devices, network storage devices, and / or systems currently or in the future. Non-limiting examples of such data storage devices 760 may include, but are not limited to, any non-transient storage devices currently or in the future, such as one or more magnetic storage devices, one or more optical storage devices, one or more electroresistive storage devices, one or more molecular storage devices, one or more quantum storage devices, or various combinations thereof. In some implementations, one or more data storage devices 760 may include one or more removable storage devices, such as one or more flash drives, flash memory, flash storage units, or similar devices that are coupled to and detachable from the computing device 700.

[0105] One or more data storage devices 760 may include an interface or controller (not shown) that connects each storage device or system to the bus 716 in a communicative manner. One or more data storage devices 760 may store, hold, or otherwise contain machine-readable instruction sets, data structures, program modules, data storage devices, databases, logical structures, and / or other data that is useful to the processor core 718 and / or graphics processor circuit 712, and / or one or more applications run on or by the processor core 718 and / or graphics processor circuit 712. In some examples, one or more data storage devices 760 may be communicatively coupled to the processor core 718, for example, via bus 716, or via one or more wired communication interfaces 730 (e.g., Universal Serial Bus, i.e., USB); one or more wireless communication interfaces 720 (e.g., Bluetooth®, Near Field Communication, i.e., NFC); ​​and / or one or more network interfaces 770 (e.g., IEEE 802.3 or Ethernet, IEEE 802.11, or Wi-Fi®).

[0106] Processor-readable instruction sets 714 and other programs, applications, logical sets, and / or modules may be stored in system memory 740, either entirely or partially. Such instruction sets 714 may be transferred, either entirely or partially, from one or more data storage devices 760. During execution by the processor core 718 and / or graphics processor circuit 712, instruction sets 714 may be loaded, stored, or otherwise held in system memory 740, either entirely or partially.

[0107] The computing device 700 may include a power management circuit 750 that controls one or more operating modes of the energy storage device 752. In embodiments, the energy storage device 752 may include one or more primary (i.e., non-rechargeable) or secondary (i.e., rechargeable) batteries, or similar energy storage devices. In embodiments, the energy storage device 752 may include one or more supercapacitors or ultracapacitors. In embodiments, the power management circuit 750 can change, adjust, or control the flow of energy from the external power source 754 to the energy storage device 752 and / or to the computing device 700. The power source 754 may include, but is not limited to, a photovoltaic system, a commercial power grid, a portable generator, an external energy storage device, or any combination thereof.

[0108] The processor core 718, graphics processor circuit 712, wireless I / O interface 720, wired I / O interface 730, storage device 760, and network interface 770 are conveniently shown to be coupled to each other via bus 716, thereby providing connectivity between the above-described components. In alternative embodiments, the above-described components may be coupled to each other in a manner different from that shown in Figure 7. For example, one or more of the above-described components may be coupled directly to other components, or to each other via one or more intermediate components (not shown). In another example, one or more of the above-described components may be integrated into the processor core 718 and / or graphics processor circuit 712. In some embodiments, all or part of bus 716 may be omitted, and the components may be coupled to each other directly using appropriate wired or wireless connections.

[0109] Flowcharts representing exemplary hardware logic, machine-readable instructions, hardware implementation state machines, and / or any combination thereof for implementing the computing system 100 (Figure 1), the compression engine 200 (Figure 2), method 600 (Figure 6A), and method 660 (Figure 6B) have already been described. The machine-readable instructions may be one or more executable programs or parts of executable programs for execution by a computer processor such as the processor 710 shown in the exemplary computing device 700 described above in relation to Figure 7. The programs may be implemented as software stored on a non-temporary computer-readable storage medium such as a CD-ROM, floppy disk, hard drive, DVD, Blu-ray disk, or memory associated with the processor 710, but alternatively, the entire program or parts thereof may be executed by a device other than the processor 710, and / or implemented in firmware or dedicated hardware. Furthermore, exemplary programs are described with reference to the flowcharts shown in Figure 6A and / or Figure 6B, but many other methods for implementing the exemplary computing system 100 may be used alternatively. For example, the execution order of the blocks may be changed, and / or some of the described blocks may be modified, deleted, or combined. Additionally or alternatively, all or some of the blocks may be implemented by one or more hardware circuits (e.g., discrete and / or integrated analog and / or digital circuits, FPGAs, ASICs, comparators, operational amplifiers (op-amps), logic circuits, etc.) configured to perform the corresponding operations without running software or firmware.

[0110] The machine-readable instructions described herein can be stored in one or more of the following formats: compressed format, encrypted format, fragmented format, compiled format, executable format, packaged format, etc. The machine-readable instructions described herein may be stored as data (e.g., instruction parts, code, code representations, etc.) that can be used to create, manufacture, and / or generate machine-executable instructions. For example, machine-readable instructions may be stored fragmentarily in one or more storage devices and / or computing devices (e.g., servers). Machine-readable instructions can be installed, modified, adapted, updated, combined, supplemented, configured, decrypted, unpackaged, distributed, reassigned, compiled, etc., to make them directly readable, interpretable, and / or executable by computing devices and / or other machines. For example, machine-readable instructions may be stored in multiple parts, each individually compressed, encrypted, and stored in individual computing devices, which, when decrypted, uncompressed, and combined, form a set of executable instructions that implement a program such as those described herein.

[0111] In other examples, machine-readable instructions can be stored in a certain state, in which state they can be read by a computer, but extensions such as libraries (e.g., dynamic-link libraries (DLLs)), software development kits (SDKs), and application programming interfaces (APIs) can be used to execute the instructions on a specific computing device or other device. In yet another example, machine-readable instructions can be set before the machine-readable instructions and / or corresponding programs are executed in whole or in part (e.g., stored settings, data inputs, recorded network addresses, etc.). Thus, disclosed machine-readable instructions and / or corresponding programs are intended to encompass such machine-readable instructions and / or programs, regardless of the specific format or state of the machine-readable instructions and / or programs, whether stored or otherwise suspended or transported.

[0112] The machine-readable instructions described herein can be expressed in any past, present, or future instruction language, scripting language, programming language, etc. For example, machine-readable instructions may be expressed using any of the following languages: C, C++, Java, C#, Perl, Python, JavaScript, HTML (HyperText Markup Language), SQL (Structured Query Language), Swift, etc.

[0113] As described above, the exemplary processes in Figures 5 and / or 6 may be implemented using executable instructions (e.g., computer and / or machine-readable instructions) stored on non-temporary computer and / or machine-readable media, such as hard disk drives, flash memory, read-only memory, compact disks, digital multipurpose disks, caches, random-access memory, and / or any other storage device or storage disk where information is stored over any period of time (e.g., long-term, permanently, short-term, temporary buffering, and / or caching of information). As used herein, the term “non-temporary computer-readable media” includes any type of computer-readable storage device and / or storage disk, but is explicitly defined to exclude propagating signals and transmission media.

[0114] In this case, “contains” and “have” (and all their forms and tenses) are used as open-ended terms. Therefore, whenever a claim uses any form of “contains” or “have” (e.g., contain, have, include, possess, etc.) as a preamble or in any kind of claim statement, it should be understood that additional elements, terms, etc., may exist without falling outside the scope of the corresponding claim or statement. When the phrase “at least” is used as a transitional term, for example in the claim preamble, as it is used in this case, it is open-ended, just as the terms “contains” and “have” are open-ended.

[0115] The term "and / or" refers to any combination or subset of A, B, and C when used in the form, for example, A, B, and / or C, for example, (1) A alone, (2) B alone, (3) C alone, (4) A and B, (5) A and C, (6) B and C, and (7) A and B and C. When used in this context in relation to structures, components, items, objects and / or things, the phrase "at least one of A and B" is intended to refer to an implementation that includes any of (1) at least one A, (2) at least one B, and (3) at least one A and at least one B. Similarly, when used in relation to structures, components, items, objects and / or things, the phrase "at least one of A or B" is intended to refer to an implementation that includes any of (1) at least one A, (2) at least one B, and (3) at least one A and at least one B. When used in the context of describing the execution or performance of a process, instruction, action, activity, and / or step, the phrase "at least one of A and B" is intended to refer to an implementation that includes (1) at least one A, (2) at least one B, and (3) at least one A and at least one B. When used in the context of describing the execution or performance of a process, instruction, action, activity, and / or step, the phrase "at least one of A or B" is intended to refer to an implementation that includes (1) at least one A, (2) at least one B, and (3) at least one A and at least one B.

[0116] In the context of this application, a single reference (e.g., "a", "an", "first", "second", etc.) does not exclude multiple entities. In the context of this application, the term "a" or "a certain" entity refers to one or more such entities. The terms "a" or "a certain", "one or more", and "at least one" are interchangeable in this application. Furthermore, even if listed individually, multiple means, elements, or method operations may be implemented, for example, by a single unit or processor. Moreover, individual features may be included in different examples or claims, but they may be combined in some cases, and being included in different examples or claims does not mean that the combination of features is unfeasible and / or advantageous.

[0117] Descriptors such as “First,” “Second,” and “Third” are used in this case to identify multiple elements or components that may be referred to individually. Unless otherwise specified or understood based on their context of use, these descriptors do not imply any sense of priority, physical order, or arrangement within the list, and their temporal order is merely used as labels to refer to multiple elements or components separately in order to facilitate understanding of the disclosed examples. In some examples, the descriptor “First” may be used to refer to an element in a detailed description, and the same element may be referred to in a claim along with different descriptors such as “Second” or “Third.” In such cases, it should be understood that such descriptors are simply used to facilitate referencing multiple elements or components.

[0118] The following embodiments relate to further embodiments. Embodiment 1 is a device that facilitates the compression of numerical data and sparse matrices in deep learning. The device of Embodiment 1: receives a data packet containing data samples of multiple cycles; and for each cycle of data samples: passes the data sample of that cycle to a compression dictionary; identifies a tag for each data sample from the compression dictionary, wherein the compression dictionary includes at least a first tag for data with a value of 0 and a second tag for data with a value of 1; and compresses the data samples into compressed cycle data by storing the tags as compressed data, wherein the values ​​of the data samples identified by the first tag or the second tag are excluded from the compressed cycle data, and at the same time, the data samples identified by the first tag are compressed using the first tag and the data samples identified by the second tag are compressed using the second tag; Includes a processor that includes a compression engine.

[0119] In Example 2, the subject matter of Example 1 may optionally include the following: the compressed dictionary includes a look-up table or a hash table. In Example 3, any one of the items in Examples 1-2 may optionally include the following: the tag further includes at least a third tag indicating no match in the compressed dictionary, and a number of additional tags indicating a partial match of the data sample to at least one value in the compressed dictionary.

[0120] In Example 4, any one of the items in Examples 1-3 may optionally include the following: a compression engine that compresses data samples further: stores a partial value of a data sample identified by one of several additional tags, following the tags of the compressed cycle data, wherein the partial value includes the sign and exponent values ​​of the data sample having one of the several additional tags; and stores the entire portion of the data sample identified by a third tag, following the partial value.

[0121] In Example 5, any one of the items in Examples 1-4 may optionally include the following: the compressed cycle data for each cycle of the data packet is concatenated in the compressed data packet, and the remainder of the compressed data packet following the last compressed cycle data is zero-padded. In Example 6, any one of the items in Examples 1-5 may optionally include the following: compressing the data sample further includes adding size bytes to the compressed cycle data, where size bytes represent the size of the compressed cycle data. In Example 7, any one of the items from Examples 1-6 may optionally include the following: Size bytes are stored after the tag and before the partial value in the compressed cycle data.

[0122] In Example 8, any one of the items from Examples 1-7 may optionally include the following: Multiple additional tags are updated after each cycle of the data packet is compressed, using data values ​​from data samples from the preceding compressed cycle of the data packet. In Example 9, any one of the items in Examples 1-8 may optionally include the following: for each cycle of data samples: splitting the data sample into upper and lower data samples; passing the upper data samples to the first subdictionary of the compression dictionary and the lower data samples to the second subdictionary of the compression dictionary, wherein the first and second subdictionaries contain the same entries; using the first subdictionary to identify tags for each of the upper data samples; using the second subdictionary to identify tags for each of the lower data samples; and the identification of tags for each of the upper and lower data samples is performed in parallel.

[0123] In Example 10, any one of the items from Examples 1-9 may optionally include the following: the tag contains a 3-bit value, the compressed dictionary stores 16 entries, the first sub-dictionary contains 8 entries from the compressed dictionary, and the second sub-dictionary contains another 8 entries from the compressed dictionary. In Example 11, any one of the subjects from Examples 1-10 may optionally include the following: the compression engine is applied to at least one output of the weight values ​​or activation function of a machine learning or deep learning workload.

[0124] Example 12 is at least one non-temporary machine-readable storage medium for facilitating the compression of numerical data and sparse matrices in deep learning. The non-temporary computer-readable storage medium of Example 12 stores executable computer program instructions, and when an instruction is executed by one or more processors, it causes one or more processors to perform an operation, which includes: receiving a data packet containing multiple cycles of data samples by at least one processor; and for each cycle of data samples: passing the cycle of data samples to a compression dictionary; identifying a tag for each of the data samples from the compression dictionary, wherein the compression dictionary includes at least a first tag for data with a value of 0 and a second tag for data with a value of 1; and compressing the data samples into compressed cycle data by storing the tags as compressed data, wherein the values ​​of the data samples identified by the first tag or the second tag are excluded from the compressed cycle data, and at the same time, the data samples identified by the first tag are compressed using the first tag and the data samples identified by the second tag are compressed using the second tag.

[0125] In Example 13, the subject matter of Example 12 may optionally include the following: the tags further include at least a third tag indicating no match in the compressed dictionary, and several additional tags indicating a partial match of the data sample to the value in the compressed dictionary. In Example 14, any one of the items in Examples 12-13 may optionally include the following: at least one processor compresses a data sample, further including at least one processor: storing a partial value of a data sample identified by one of a plurality of additional tags, following a tag of the compressed cycle data, wherein the partial value includes the sign and exponent values ​​of the data sample having one of the plurality of additional tags; and storing the entire portion of the data sample identified by a third tag, following the storage of the partial value.

[0126] In Example 15, any one of the items in Examples 12-14 may optionally include the following: the compressed cycle data for each cycle of the data packet is concatenated in the compressed data packet, and the remainder of the compressed data packet following the last compressed cycle data is zero-padded. In Example 16, any one of the items in Examples 12-15 may optionally include the following: at least one processor compresses the data sample, and furthermore, at least one processor adds size bytes to the compressed cycle data, where size bytes represent the size of the compressed cycle data.

[0127] Example 17 is a method for compressing numerical data and sparse matrices in deep learning. The method of Example 17 comprises: receiving a data packet containing multiple cycles of data samples by at least one processor; and for each cycle of data samples: passing the cycle of data samples to a compression dictionary; identifying a tag for each data sample from the compression dictionary, wherein the compression dictionary includes at least a first tag for data with a value of 0 and a second tag for data with a value of 1; and compressing the data samples into compressed cycle data by storing the tags as compressed data, wherein the values ​​of the data samples identified by the first tag or the second tag are excluded from the compressed cycle data, and at the same time, the data samples identified by the first tag are compressed using the first tag and the data samples identified by the second tag are compressed using the second tag.

[0128] In Example 18, any one of the subjects from Example 17 may optionally include the following: the tag further includes at least a third tag indicating no match in the compressed dictionary, and several additional tags indicating a partial match of the data sample to the value in the compressed dictionary. In Example 19, any one of the items in Examples 17-18 may optionally include the following: compressing a data sample, further including storing a partial value of the data sample identified by one of several additional tags, following the tags of the compressed cycle data, wherein the partial value includes the sign and exponent values ​​of the data sample having one of the several additional tags; storing the entire portion of the data sample identified by a third tag, following the storage of the partial value; and adding a size byte to the compressed cycle data, wherein the size byte indicates the size of the compressed cycle data.

[0129] In Example 20, any one of the items from Examples 17-19 may optionally include the following: the compressed cycle data for each cycle of the data packet is concatenated in the compressed data packet, and the remainder of the compressed data packet following the last compressed cycle data is zero-padded.

[0130] Example 21 is a system that facilitates the compression of numerical data and sparse matrices in deep learning. The system of Example 21 may optionally include memory and a processor communicatively coupled to memory. The processor of the system of Example 21 may include a compression engine that: receives a data packet containing multiple cycles of data samples; and for each cycle of data samples: passes the cycle of data samples to a compression dictionary; identifies a tag for each of the data samples from the compression dictionary, wherein the compression dictionary includes at least a first tag for data with a value of 0 and a second tag for data with a value of 1; and compresses the data samples into compressed cycle data by storing the tags as compressed data, wherein the values ​​of the data samples identified by the first tag or the second tag are excluded from the compressed cycle data, and at the same time, data samples identified by the first tag are compressed using the first tag and data samples identified by the second tag are compressed using the second tag.

[0131] In Example 22, any one of the items in Example 12 may optionally include the following: the compressed dictionary includes a look-up table or a hash table. In Example 23, any one of the subjects from Examples 21-22 may optionally include the following: the tag further includes at least a third tag indicating no match in the compressed dictionary, and a number of additional tags indicating a partial match of the data sample to at least one value in the compressed dictionary.

[0132] In Example 24, any one of the items in Examples 21-23 may optionally include the following: the compression engine compresses a data sample, further including: storing a partial value of the data sample identified by one of a plurality of additional tags, following the tag of the compressed cycle data, wherein the partial value includes the sign and exponent values ​​of the data sample having one of the plurality of additional tags; and storing the entire portion of the data sample identified by a third tag, following the storage of the partial value.

[0133] In Example 25, any one of the subject items from Examples 21-24 may optionally include the following: the compressed cycle data for each cycle of the data packet is concatenated in the compressed data packet, and the remainder of the compressed data packet following the last compressed cycle data is zero-padded. In Example 26, any one of the items in Examples 21-25 may optionally include the following: compressing the data sample further includes adding size bytes to the compressed cycle data, where size bytes represent the size of the compressed cycle data. In Example 27, any one of the items from Examples 21-26 may optionally include the following: Size bytes are stored after the tag and before the partial value in the compressed cycle data.

[0134] In Example 28, any one of the items in Examples 21-27 may optionally include the following: Multiple additional tags are updated after each cycle of the data packet is compressed, using data values ​​from data samples from the preceding compressed cycle of the data packet. In Example 29, any one of the subjects from Examples 21-28 may optionally include the following: for each cycle of data samples: splitting the data sample into upper and lower data samples; passing the upper data samples to the first subdictionary of the compression dictionary and the lower data samples to the second subdictionary of the compression dictionary, wherein the first and second subdictionaries contain the same entries; using the first subdictionary to identify tags for each of the upper data samples; using the second subdictionary to identify tags for each of the lower data samples; and the identification of tags for each of the upper and lower data samples is performed in parallel.

[0135] In Example 30, any one of the items from Examples 21-29 may optionally include the following: the tag contains a 3-bit value, the compressed dictionary stores 16 entries, the first sub-dictionary contains 8 entries from the compressed dictionary, and the second sub-dictionary contains 8 entries from the compressed dictionary. In Example 31, any one of the subjects from Examples 21-30 may optionally include the following: the compression engine is applied to at least one output of the weight values ​​or activation function of a machine learning or deep learning workload.

[0136] Example 32 is a device that facilitates the compression of numerical data and sparse matrices in deep learning by an implementation of the present disclosure. The device of Example 32 includes means for receiving a data packet containing data samples of multiple cycles; and for each cycle of data samples: passing the data sample of the cycle to a compression dictionary; identifying a tag for each of the data samples from the compression dictionary, wherein the compression dictionary includes at least a first tag for data having a value of 0 and a second tag for data having a value of 1; and compressing the data samples into compressed cycle data by storing the tags as compressed data, wherein the values ​​of the data samples identified by the first tag or the second tag are excluded from the compressed cycle data, and at the same time, the data samples identified by the first tag are compressed using the first tag and the data samples identified by the second tag are compressed using the second tag.

[0137] In Example 33, the subject matter of Example 32 may optionally include the following: the apparatus is further configured to perform the method described in any one of Examples 18-20.

[0138] Example 34 is at least one machine-readable medium containing a plurality of instructions, which, in response to being executed by a computing device, cause the computing device to perform a method according to any one of Examples 17-20. Example 35 is a device that facilitates the compression of numerical data and sparse matrices in deep learning, and is configured to perform the method according to any one of Examples 17 to 20. Example 36 is a device for facilitating the compression of numerical data and sparse matrices in deep learning, and includes means for performing a method according to any one of Examples 17 to 20. The specific details in the examples may be used in any one or more embodiments.

[0139] The foregoing description and drawings should be interpreted illustratively, not restrictively. Those skilled in the art will understand that various modifications and changes may be made to the embodiments described herein without departing from the broad spirit and scope of the features described in the attached claims.

Claims

1. A device for promoting the compression of numerical data and sparse matrices in deep learning, Receiving data packets containing data samples from multiple cycles; and For each cycle of the aforementioned data sample: Passing the data samples from the aforementioned cycle to a compressed dictionary; Identifying a tag for each of the aforementioned data samples from the compressed dictionary, wherein the compressed dictionary includes at least a first tag for data having a value of 0 and a second tag for data having a value of 1; and Compressing the data samples into compressed cycle data by storing the tags as compressed data, wherein the values ​​of the data samples identified by the first tag or the second tag are excluded from the compressed cycle data, and at the same time, the data samples identified by the first tag are compressed using the first tag, and the data samples identified by the second tag are compressed using the second tag; A device including a processor that includes a compression circuit that performs this function.

2. The apparatus according to claim 1, wherein the compressed dictionary includes a look-up table or a hash table.

3. The apparatus according to any one of claims 1-2, wherein the tag further includes at least a third tag indicating no match in the compressed dictionary, and a plurality of additional tags indicating a partial match of the data sample to at least one value in the compressed dictionary.

4. The compression circuit that compresses the aforementioned data samples further includes: To store, following the tags of the compressed cycle data, a partial value of the data sample identified by any of the plurality of additional tags, wherein the partial value includes the sign and exponent values ​​of the data sample having any of the plurality of additional tags; and Following the storage of the partial value, the storage of the entire portion of the data sample identified by the third tag; The apparatus according to claim 3, which performs the following.

5. The apparatus according to any one of claims 1-4, wherein the compressed cycle data of each cycle of the data packet are concatenated in the compressed data packet, and the remainder of the compressed data packet following the last compressed cycle data is zero-padding.

6. The apparatus according to any one of claims 1 to 5, wherein compressing the data sample further includes adding size bytes to the compressed cycle data, the size bytes representing the size of the compressed cycle data.

7. The apparatus according to claim 6, referencing claim 4, wherein the size bytes are stored in the compressed cycle data after the tag and before the partial value.

8. The apparatus according to claim 3, wherein the plurality of additional tags are updated after each cycle of the data packet has been compressed, using the data values ​​of the data samples from the preceding compressed cycle of the data packet.

9. For each cycle of the data sample: Dividing the aforementioned data sample into an upper data sample and a lower data sample; Passing the upper half data sample to the first subdictionary of the compression dictionary, and passing the lower half data sample to the second subdictionary of the compression dictionary, wherein the first subdictionary and the second subdictionary contain the same entry; Using the first sub-dictionary, identify a tag for each of the upper half data samples; Using the second sub-dictionary, identify a tag for each of the lower half data samples; The apparatus according to any one of claims 1 to 8, wherein the identification of tags for each of the upper data samples and each of the lower data samples is performed in parallel.

10. The apparatus according to claim 9, wherein the tag includes a 3-bit value, the compressed dictionary stores 16 entries, the first sub-dictionary includes 8 entries of the compressed dictionary, and the second sub-dictionary includes another 8 entries of the compressed dictionary.

11. The apparatus according to any one of claims 1 to 10, wherein the compression circuit is applied to at least one output of a weight value or activation function of a machine learning workload or a deep learning workload.

12. A method for promoting the compression of numerical data and sparse matrices in deep learning, Receiving a data packet containing data samples from multiple cycles by at least one processor; and For each cycle of the aforementioned data sample: Passing the data samples from the aforementioned cycle to a compressed dictionary; Identifying a tag for each of the aforementioned data samples from the compressed dictionary, wherein the compressed dictionary includes at least a first tag for data having a value of 0 and a second tag for data having a value of 1; and Compressing the data samples into compressed cycle data by storing the tags as compressed data, wherein the values ​​of the data samples identified by the first tag or the second tag are excluded from the compressed cycle data, and at the same time, the data samples identified by the first tag are compressed using the first tag, and the data samples identified by the second tag are compressed using the second tag; A method that includes this.

13. The method according to claim 12, wherein the tag further includes at least a third tag indicating no match in the compressed dictionary, and a plurality of additional tags indicating a partial match of the data sample to a value in the compressed dictionary.

14. Compressing the aforementioned data samples further: To store, following the tags of the compressed cycle data, a partial value of the data sample identified by any of the plurality of additional tags, wherein the partial value includes the sign and exponent values ​​of the data sample having any of the plurality of additional tags; Following the storage of the partial value, the storage of all portions of the data sample identified by the third tag; and Adding size bytes to the compressed cycle data, wherein the size bytes represent the size of the compressed cycle data; The method according to claim 13, including the method described in claim 13.

15. The method according to any one of claims 12-14, wherein the compressed cycle data of each cycle of the data packet are concatenated in the compressed data packet, and the remainder of the compressed data packet following the last compressed cycle data is zero-padded.

16. A system that facilitates the compression of numerical data and sparse matrices in deep learning. memory; and A processor connected to the aforementioned memory in a communicative manner; The compression circuit provided in the processor includes: Receiving data packets containing data samples from multiple cycles; and For each cycle of the aforementioned data sample: Passing the data samples from the aforementioned cycle to a compressed dictionary; Identifying a tag for each of the aforementioned data samples from the compressed dictionary, wherein the compressed dictionary includes at least a first tag for data having a value of 0 and a second tag for data having a value of 1; and Compressing the data samples into compressed cycle data by storing the tags as compressed data, wherein the values ​​of the data samples identified by the first tag or the second tag are excluded from the compressed cycle data, and at the same time, the data samples identified by the first tag are compressed using the first tag, and the data samples identified by the second tag are compressed using the second tag; A system that performs this task.

17. The system according to claim 16, wherein the compressed dictionary includes a look-up table or a hash table.

18. The system according to any one of claims 16-17, wherein the tag further includes at least a third tag indicating no match in the compressed dictionary, and a plurality of additional tags indicating a partial match of the data sample to at least one value in the compressed dictionary.

19. The compression circuit that compresses the aforementioned data samples further includes: To store, following the tags of the compressed cycle data, a partial value of the data sample identified by any of the plurality of additional tags, wherein the partial value includes the sign and exponent values ​​of the data sample having any of the plurality of additional tags; and Following the storage of the partial value, the storage of the entire portion of the data sample identified by the third tag; The system according to claim 18, which performs the following:

20. The system according to any one of claims 16-19, wherein compressing the data sample further comprises adding size bytes to the compressed cycle data, the size bytes representing the size of the compressed cycle data.

21. The system according to claim 18, wherein the plurality of additional tags are updated after each cycle of the data packet is compressed, using the data values ​​of the data samples from the preceding compressed cycle of the data packet.

22. For each cycle of the data sample: Dividing the aforementioned data sample into an upper data sample and a lower data sample; Passing the upper half data sample to the first subdictionary of the compression dictionary, and passing the lower half data sample to the second subdictionary of the compression dictionary, wherein the first subdictionary and the second subdictionary contain the same entry; Using the first sub-dictionary, identify a tag for each of the upper half data samples; Using the second sub-dictionary, identify a tag for each of the lower half data samples; The system according to any one of claims 16-21, wherein the following are performed in parallel: identifying tags for each of the upper data samples and each of the lower data samples.

23. A device that facilitates the compression of numerical data and sparse matrices in deep learning, Means for receiving data packets containing data samples from multiple cycles; and For each cycle of the aforementioned data sample, Passing the data samples from the aforementioned cycle to a compressed dictionary; Identifying a tag for each of the aforementioned data samples from the compressed dictionary, wherein the compressed dictionary includes at least a first tag for data having a value of 0 and a second tag for data having a value of 1; and A means for compressing data samples into compressed cycle data by storing the tags as compressed data, wherein the values ​​of the data samples identified by the first tag or the second tag are excluded from the compressed cycle data, and at the same time, the data samples identified by the first tag are compressed using the first tag, and the data samples identified by the second tag are compressed using the second tag; A device that includes this.

24. The apparatus according to claim 23, further configured to perform the method described in any one of claims 13-15.

25. A computer program that causes a computing device to perform the procedure described in any one of claims 12-15.

Images