Data processing system

Abstract

A data processing system is provided that includes a processing element and a storage device. The storage device stores a code portion that, when executed by the processing element, causes the data processing system to obtain a plurality of variable length codes. Each variable length code has a first portion of variable length and a second portion. The variable length codes are separated into the first portions of the variable length codes and the second portions of the variable length codes. The data processing system uses a set of control rules to form a processed stream in chunks such that for each chunk of the processed stream that contains data from the first portions of the variable length codes, the data from the first portions forms a substream within the chunk of the processed stream having a size determined according to the control rules.

Description

TECHNICAL FIELD

[0001] The present invention relates to a data processing system and more particularly to a data processing system for processing variable length codes. BACKGROUND

[0002] One way of performing lossless data compression known in the art is to convert values into Golomb codes. To convert a numerical value into a Golomb code, a parameter called a divisor is chosen. To create a Golomb code, the numerical value is divided by the divisor to generate two parts. The first part is the quotient, which is the number of times the divisor divides the numerical value. The second part is the remainder, which is the amount left over (if any) after the divisor divides the numerical value.

[0003] Figure 1 Examples of Golomb codes are shown in the table below. In the table, the numerical value is shown in the left column, the divisor is shown in the middle column, and the Golomb code is shown in the right column. Figure 1 In the example shown, values between 0 and 10 are shown as Golomb codes. The first part of the Golomb code (i.e. the quotient) is represented in unary format. In this format, the number is represented by a number of '1's equal to the value of the quotient, followed by a stop bit (i.e. a '0'). For example, the unary part of the number 9 is '1110', which is three '1's followed by a stop bit '0', because 3 divides 9 three times. The second part of the Golomb code is a fixed length binary part. Since the divisor in this example is '3', the remainder can only be 0, 1, or 2. Therefore, this can be represented by a fixed length binary of two bits. The last two bits in each Golomb code represent the remainder in binary form. The remainder can be referred to as the'mantissa' of the Golomb code, because it appears after the stop bit of the unary part of the Golomb code.

[0004] Because the size of the unary part of a Golomb code is variable, Golomb codes are a type of variable length code. Such variable length codes can be slow to decode in a processor, because each code needs to be considered individually in order to decode. SUMMARY

[0005] According to a first aspect, there is provided a method for use by a processing element, the method comprising: obtaining a plurality of variable length codes, wherein each variable length code has a first part of variable length and a second part; separating the variable length codes into the first parts of the variable length codes and the second parts of the variable length codes; and using a set of control rules to form a processed stream in chunks, such that for each chunk of the processed stream containing data from the first parts of the variable length codes, the data from the first parts forms a sub-stream within the chunk of the processed stream having a size determined in accordance with the control rules.

[0006] According to a second aspect, there is provided a method of decoding a processed data stream, the method comprising: obtaining a processed data stream containing data relating to a plurality of variable length codes, wherein each variable length code has a first part of variable length and a second part, and the processed stream is formed in blocks, wherein at least one block of the processed stream comprises a sub-stream of data formed from the first parts of the variable length codes; and using a set of stream control rules to identify and extract the sub-streams within the blocks of the processed data stream.

[0007] According to a third aspect, there is provided a data processing system comprising a processing element and a storage device, the storage device storing a code portion which, when executed by the processing element, causes the data processing system to: obtain a plurality of variable length codes, wherein each variable length code has a first part of variable length and a second part; separate the variable length codes into the first parts of the variable length codes and the second parts of the variable length codes; and using a set of control rules to form a processed stream in blocks, such that for each block of the processed stream containing data from the first parts of the variable length codes, the data from the first parts forms a sub-stream within the block of the processed stream having a size determined according to the control rules. BRIEF DESCRIPTION OF DRAWINGS

[0008] The techniques of the present application will now be further described, by way of example only, and with reference to the embodiments illustrated in the drawings in which:

[0009] Figure 1 is a table showing the Golomb-Rice codes between the values of 0 and 10 using a divisor of 3;

[0010] Figure 2 shows components of a neural processing unit writing activation data to a storage device;

[0011] Figure 3 is a flow diagram showing processing of a Golomb-Rice code stream for storage;

[0012] Figure 4 shows the structure of a data stream during separation of Golomb-Rice codes;

[0013] Figure 5 shows a plurality of block structures used by an encoder and a decoder;

[0014] Figure 6 is a flow diagram showing steps performed by an encoder;

[0015] Figure 7 is a flow diagram showing steps for deciding a block structure when decoding a first block of a stream encoded by an encoder;

[0016] Figure 8 is a flowchart illustrating steps for deciding on a chunk structure when decoding a subsequent chunk of a stream encoded by an encoder;

[0017] Figure 9a is a flowchart illustrating steps for decoding a unary data;

[0018] Figure 9b illustrates bit processing when decoding a series of unary values;

[0019] Figure 10a illustrates a mobile device;

[0020] Figure 10b is a diagram illustrating hardware of a mobile device;

[0021] Figure 11 is a diagram illustrating a system architecture installed on a mobile device;

[0022] Figure 12 is a diagram illustrating components of a neural processing unit;

[0023] Figure 13 is a table illustrating flow control rules for encoding unary data related to weight values; and

[0024] Figure 14 is a table illustrating flow control rules for decoding unary data related to weight values. DETAILED DESCRIPTION

[0025] The following description of embodiments and associated advantages comes more fully into focus after reviewing the illustrative embodiments.

[0026] According to one embodiment, there is provided a method for use by a processing element, the method comprising: obtaining a plurality of variable length codes, wherein each variable length code has a first part of variable length and a second part; separating the variable length codes into the first parts of the variable length codes and the second parts of the variable length codes; and using a set of control rules to form a processed stream in chunks such that for each chunk of the processed stream containing data from the first parts of the variable length codes, the data from the first parts forms a substream within the chunk of the processed stream having a size determined according to the control rules. In this way, the processed stream can have substreams of the first parts of the variable length codes that are predictable according to the control rules. This can enable the processor to more efficiently process the first parts of the variable length codes.

[0027] The first portion of the variable length code can be a unary portion, and the second portion of the variable length code can be a mantissa portion. In some cases, the mantissa portion of the variable length code is a truncated binary portion. The unary portion of each variable length code can represent a quotient of an original value represented by the variable length code. Further, in some cases, the mantissa portion of each variable length code is a fixed length binary code that represents a remainder of the original value represented by the variable length code. In some implementations, the variable length codes are Golomb codes.

[0028] Each chunk of the processed stream can include data from the first portion of the variable length code and data from the second portion of the variable length code, include data from the first portion of the variable length code but not data from the second portion of the variable length code, or include data from the second portion of the variable length code but not data from the first portion of the variable length code.

[0029] The control rules can be configured to sequentially determine the size of the unary sub-streams within each chunk. In some cases, the control rules are configured to determine the size of the unary sub-streams within each chunk based on a measure of the unary portions of the variable length codes that are still to be added to the processed stream. For some implementations, the processed stream is formed of units having a predetermined length, and the measure of the unary portions of the variable length codes that are still to be added to the processed stream is the number of bits of the unary portions of the variable length codes associated with the units that are still to be added to the processed stream. By processing the data in the units, the stream control rules can allow the size of the unary sub-streams to be predicted by a decoder.

[0030] The control rules can be rules that select between a set of predetermined chunk structures. Each chunk structure can define the type of data that should be made at each position within a chunk. The set of chunk structures can include at least one chunk structure for a first chunk in a unit. The at least one chunk structure for the first chunk in the unit can include a header portion that includes information about the data included in the unit.

[0031] In some methods according to the first embodiment, the processed stream is formed of unit runs having a predetermined length, where each unit of the processed stream has a header and a plurality of chunks, and where the header indicates the length of the unit and the length of the unary portions of the variable length codes within the unit. This can allow a decoder to track the amount of unary data that is held in a unit during decoding.

[0032] The first embodiment can also include obtaining a second plurality of variable length codes, and dividing the second plurality of variable length codes into a first portion of the variable length codes and a second portion of the variable length codes; wherein the step of forming the processed stream in chunks uses a set of flow control rules such that for each chunk of the processed stream containing data from a first portion of at least one of the first plurality of variable length codes and the second plurality of variable length codes, the flow control rules determine a number of the first portion of the first plurality of variable length codes and a number of the first portion of the second plurality of variable length codes to include in the chunk.

[0033] In some implementations, forming the processed stream includes maintaining a slack value that records a difference between a number of the first portion of the first plurality of variable length codes that have been included in chunks and a number of the first portion of the second plurality of variable length codes that have been included in chunks, and the flow control rules determine the number of the first portion of the first plurality of variable length codes and the number of the first portion of the second plurality of variable length codes to include in a chunk based on the slack value.

[0034] In some applications, the plurality of variable length codes represent weight values used in a neural network. In other applications, the plurality of variable length codes represent values in an activation stream that is output by a layer of a neural network.

[0035] According to a second embodiment, there is provided a method of decoding a processed data stream, the method including obtaining a processed data stream containing data relating to a plurality of variable length codes, wherein each variable length code has a first portion of variable length and a second portion, and the processed stream is formed in chunks, wherein at least one chunk of the processed stream includes a substream of data formed from the first portions of the variable length codes; and using a set of flow control rules to identify and extract the substream within the chunk of the processed data stream.

[0036] In some implementations, each first portion of a variable length code is encoded as a unary data having a stop bit. The method can also include obtaining a plurality of identified and extracted substreams from a plurality of chunks of the processed data; converting each extracted substream into an intermediate form representing a list of bit positions of stop bits within the substream; combining a plurality of converted substreams to form an expanded list of bit positions; and measuring distances between the stop bits in the expanded list of bit positions to recover values of the first portions of the variable length codes in the first stream.

[0037] In some embodiments, the flow control rules are configured to determine a size of a unary substream within each chunk to be decoded based on an amount of unary portions of variable length codes that remain to be decoded from the processed stream. The processed stream can be formed of units having a predetermined length, and the amount of unary portions of variable length codes that remain to be decoded from the processed stream can be a number of bits of unary portions of variable length codes associated with the units that remain to be decoded from the processed stream.

[0038] In other embodiments, the processed stream can additionally include a first portion of variable length codes belonging to a second plurality of variable length codes. The flow control rules can be configured such that, for each chunk of the processed stream to be decoded that contains data of a first portion of at least one of the plurality of variable length codes and the second plurality of variable length codes, the flow control rules determine a number of the first portion of the plurality of variable length codes and a number of the first portion of the second plurality of variable length codes included in the chunk to be decoded.

[0039] Identifying and extracting substreams within chunks of the processed stream can include maintaining a balance value that records a difference between a number of the first portion of the plurality of variable length codes that have been extracted from the processed stream and a number of the first portion of the second plurality of variable length codes that have been extracted from the processed stream. The flow control rules can be used to determine the number of the first portion of the plurality of variable length codes and the number of the first portion of the second plurality of variable length codes that should be extracted from a chunk based on the balance value.

[0040] According to a third embodiment, there is provided a non-transitory computer- readable storage medium storing code portions that, when executed on a processing element, cause the processing element to perform the following method: obtaining a plurality of variable length codes, wherein each variable length code has a first portion of variable length and a second portion; dividing the variable length codes into the first portions of the variable length codes and the second portions of the variable length codes; and using a set of control rules to form a processed stream in chunks such that, for each chunk of the processed stream that contains data from the first portions, the data from the first portions forms a substream within the chunk of the processed stream having a size determined according to the control rules.

[0041] According to a fourth embodiment, there is provided a non-transitory computer readable storage medium storing code portions which, when executed on a processing element, cause the processing element to perform a method of decoding a processed data stream, the method comprising: obtaining a processed data stream containing data relating to a plurality of variable length codes, wherein each variable length code has a first portion of variable length and a second portion, and the processed stream is formed in chunks, wherein at least one chunk of the processed stream comprises a sub-stream of data formed from the first portions of the variable length codes; and using a set of stream control rules to identify and extract the sub-streams within the chunks of the processed data stream.

[0042] According to a fifth embodiment, there is provided a data processing system comprising a processing element and a storage device, the storage device storing code portions which, when executed by the processing element, cause the data processing system to: obtain a plurality of variable length codes, wherein each variable length code has a first portion of variable length and a second portion; separate the variable length codes into first portions of the variable length codes and second portions of the variable length codes; and using a set of control rules to form a processed stream in chunks, such that for each chunk of the processed stream containing data from the first portions of the variable length codes, the data from the first portions forms a sub-stream within the chunk of the processed stream having a size determined according to the control rules.

[0043] According to a sixth embodiment, there is provided a data processing system comprising a processing element and a storage device, the storage device storing code portions which, when executed by the processing element, cause the data processing system to: obtain a processed data stream containing data relating to a plurality of variable length codes, wherein each variable length code has a first portion of variable length and a second portion, and the processed stream is formed in chunks, wherein at least one chunk of the processed stream comprises a sub-stream of data formed from the first portions of the variable length codes; and using a set of stream control rules to identify and extract the sub-streams within the chunks of the processed data stream.

[0044] Particular embodiments will now be described with reference to the drawings.

[0045] Figure 2 Some, but not all, components 2 of a neural processing unit (NPU) are shown, the NPU being a dedicated chip for performing computations relating to artificial intelligence applications, and in particular computations relating to neural networks. In other words, the NPU allows hardware acceleration of certain computations relating to neural networks. Component 2 is a component for writing activation values to a DRAM (not shown) external to the NPU.

[0046] When performing computations related to neural networks, computations can be performed for each layer of the neural network. These computations generate outputs called activation data, which can be large in quantity and need to be stored before further computations can be performed using the data. Storing and retrieving activation data from memory can be a relatively slow process due to limitations on data transfer from external memory to the processor. Therefore, to improve processor performance, it is desirable to compress data from activation layers using Golomb codes.

[0047] The component 2 is configured to process activation data for storage. Activation data is received and grouped into data blocks. A data block is defined as a group of 8x8 elements, where one element is 8-bit uncompressed activation data. A processing element in the form of an encoder 20 is configured to compress received activation data by converting the activation data into Golomb codes. Further steps, which will be described below, are then performed to make the activation data more easily decodable.

[0048] When decoding variable length codes such as Golomb codes, it is difficult to parse at a high rate. This is because there is a sequence dependency between Golomb codes, so the length of the previous Golomb code needs to be known before the next Golomb code can be identified and decoded. Therefore, when parsing directly using a single parser, a typical hardware implementation for decoding Golomb codes can achieve a rate of 1 or 2 Golomb codes per clock cycle.

[0049] The techniques described in the first particular implementation take a different approach. Figure 3 is a flowchart showing the steps performed by the encoder 20 receiving uncompressed activation data. In step S30, Golomb codes are obtained, in this case by conversion by the encoder 20. Then, in step S31, the encoder 20 separates the Golomb codes into a stream of unary values and a stream of remainder values and stores them in Figure 2 the RAM 21 shown.

[0050] Figure 4 Three data streams are shown. The source data stream 40 is a stream of Golomb codes. The source data stream 40 includes a series of Golomb codes indicated by the values GR1 to GR5. Each Golomb code has a variable length unary part and a fixed length binary part of the type described in the description of the related art. Although a three-bit fixed length binary part is shown in Figure 3 , the length of the binary part is not important and other lengths can be used. The encoder 20 divides the Golomb codes into two parts to generate Figure 4 the other two streams 41 and 42 shown. The first stream 41 is a unary stream, while the second stream 42 is a stream of remainders, each binary value having a fixed length.

[0051] In step S32, Figure 2 The splicing processor 22 shown splices the first and second streams together to form a processed stream. This is done on a unit-by-unit basis, where each unit represents 32 blocks of uncompressed data (2,048 elements) stored in a 2112 byte slot. The slot is larger than the unit in order to allow for some overhead and rounding to an integer number of 64 bytes.

[0052] Each unit is formed by the splicing processor 22 in a number of chunks. The first chunk of a unit always includes a header. One meta data from stream 4 is always spliced by the splicing processor 22 into the chunk of the processed stream in the 32 bit portion.

[0053] The unit is formed by the splicing processor 22 using a set of stream control rules, as will now be described with reference to Figure 5 and Figure 6 Figure 5 Different structures of the chunks that the splicing processor 22 can use to form a unit are shown, and Figure 6 The steps to be performed by the splicing processor 22 when forming a unit are shown.

[0054] As mentioned above, the first chunk of a unit must include a header that provides information about the length of the unit and the length of the meta sub-streams included within the unit. The length of the remainder value within the unit is not included in the header, but can be derived from the length of the unit and the length of the meta sub-streams.

[0055] Figure 5 Different structures of the chunks that the splicing processor 22 can use to form a unit are shown, and Figure 5 The top two chunk structures 51 and 52 shown are the first chunk structures of a unit, and can be selected for use when forming the first chunk of a unit. Both chunk format structures include a 32 bit length header portion. After the first chunk of a unit has been issued by the encoder 20, the next five chunk structures 53 to 57 below the first chunk structures 51 and 52 in Figure 5

[0056] In order to select the appropriate chunk structure to use, the encoder 20 uses a set of stream control rules. Once the chunk structure of the next chunk has been identified, the chunk structure can be filled with the appropriate data and issued. The stream control rules used by the splicing processor 22 are as follows. When the first chunk structure is selected for a unit, if there is more than 32 bits of meta data available to the splicing processor 22 to include in the chunk in the unit, then the​​Figure 5 The first chunk structure 51 is used to form the first chunk of a unit as illustrated. Otherwise, the chunk structure 52 is used to form the first chunk of a unit as the chunk 52 does not require any unary data. The case of 32-bit unary data unavailability can occur if all uncompressed active data of a unit does not have a unary part, i.e. each value is smaller than the divisor used to generate the Golomb-Rice code. In this case, the encoder 20 does not encode the stop bit as unary data.

[0057] For subsequent chunks of a unit issued by the encoder 20 after the first chunk has been issued, the chunk structure 53 is used if more than 128 bits of unary data remain to be included in the unit. As the unary data is included in the chunk in the 32-bit part, the unit will eventually remain with less than 128 bits of unary data to be encoded. The chunk structure 54 is used if the unit remains with 96 bits of unary data to be encoded, the chunk structure 55 is used if the unit remains with 64 bits of unary data to be encoded, and the chunk structure 56 is used if the unit remains with 32 bits of unary data to be encoded. In the case that all unary data of a unit has been encoded, the chunk structure 57 is used to issue the remainder data. It should be noted that the selection of the unary data included in the above chunk structures is independent of the base block and element grouping of the underlying active data, such that unary data from different blocks and / or elements can be included in the same chunk.

[0058] Figure 6 The above method is illustrated in Figure 6. In step S60, the length of the unary sub-stream to be included in a unit is evaluated from the 32 uncompressed data blocks. The length of the remainder data to be included in the unit is also evaluated. Figure 5 The first chunk structure is selected from the chunk structures 51 and 52 as illustrated. As mentioned above, this selection depends on the availability of 32-bit unary data. After the selection of the chunk structure, the header part is created by the splicing processor 22. The splicing processor 22 evaluates the length of the unary sub-stream to be included in the unit based on the 32 uncompressed data blocks and adds information indicating this length to the header part. The length of all data to be included in the unit is also evaluated and added to the header part. The data from the first stream 41 of unary data and the data from the second stream of remainder data are added to the chunk according to the selected chunk structure selected by the splicing processor 22, if necessary.

[0059] In step S61, the next chunk of the processed data stream is formed by selecting the appropriate chunk structure from the chunk structures 53 to 57 according to the stream control rules described above. After the selection of the chunk structure, the chunk is formed by filling the relevant parts of the chunk structure with data from the first stream 41 of unary data and the second stream of remainder data.

[0060] In step S62, the stitching processor 22 determines whether there is more data to form a chunk. If there is more data to form a chunk, the method proceeds to S61 to form the next chunk. If there is no more data to process, the method proceeds to S63, where it ends.

[0061] The above method assumes that a complete 128-bit chunk can be formed from 32 blocks in a cell, and that the unary part can be stitched into 32-bit parts. In practice, these conditions can not be met, in which case the first stream of the unary part and the second stream of the remainder part are padded with stop bits '0' for padding until they reach the desired size. Since the length of the data added to the cell is stored in the header, the length of the data within the cell can be identified, and the start of the padding identified when decoding the processed data stream.

[0062] Next, the method of decoding the stored activation data by a decoder will be described with reference to Figure 7 and Figure 8 In this case, the decoder is part of an NPU, the part allowing the activation data to be read from the DRAM for further computation. The decoder stores a copy of the chunk structure shown in Figure 5 which was used by the encoder 20 to store the processed streams in the RAM. In step S70 of Figure 7 the decoder receives the first chunk of a cell of activation data from the RAM in order to decode it. The decoder reads the header and identifies the length of the unary data in the cell. In step S71, the decoder identifies whether the length of the unary data specified in the header is 32 bits or more. If the length of the unary is 32 bits or more, the first chunk of the cell is formed according to the chunk structure 51, and the unary data and the remainder data can be extracted from the chunk according to the known chunk structure. If the length of the unary data identified in the header is less than 32 bits (zero, since the unary data is stitched into 32-bit parts), the first chunk is formed according to the chunk structure 52, and decoded accordingly. When decoding the data, the decoder maintains a parameter U_left, which is initially set to the value of the length of the unary in the cell when the header of the cell is checked, and is updated each time the unary data is retrieved from the chunk to record the amount of unary data remaining in the cell. Thus, if the chunk structure 51 is used for the first chunk, the parameter U_left will decrease by 32 after the 32-bit unary data is extracted from the first chunk.

[0063] Figure 8A method used by the decoder to determine the chunk structure of each subsequent chunk is shown. In step S80, the subsequent chunk of stored activation data is received. In step S81, the parameter U_left is checked to determine if the amount of unary data remaining to be extracted for the unit is greater than or equal to 128 bits. If the amount of unary data to be extracted is greater than or equal to 128 bits, the decoder determines that chunk structure 53 is being used. In step S82, the decoder extracts data from the chunk according to chunk structure 53 and updates the parameter U_left to account for the amount of unary data extracted.

[0064] If less than 128 bits of unary data remains to be extracted, the method proceeds to step S83. In step S83, the parameter U_left is checked to determine if the amount of unary data remaining to be extracted for the unit is equal to 96 bits. If the amount of unary data to be extracted is equal to 96 bits, the decoder determines that chunk structure 54 is being used. In step S84, the decoder extracts data from the chunk according to chunk structure 54 and updates the parameter U_left to account for the amount of unary data extracted.

[0065] If less than 96 bits of unary data remains to be extracted, the method proceeds to step S85. In step S85, the parameter U_left is checked to determine if the amount of unary data remaining to be extracted for the unit is equal to 64 bits. If the amount of unary data to be extracted is equal to 64 bits, the decoder determines that chunk structure 55 is being used. In step S86, the decoder extracts data from the chunk according to chunk structure 55 and updates the parameter U_left to account for the amount of unary data extracted.

[0066] If less than 64 bits of unary data remains to be extracted, the method proceeds to step S87. In step S87, the parameter U_left is checked to determine if the amount of unary data remaining to be extracted for the unit is equal to 32 bits. If the amount of unary data to be extracted is equal to 32 bits, the decoder determines that chunk structure 56 is being used. In step S86, the decoder extracts data from the chunk according to chunk structure 56 and updates the parameter U_left to account for the amount of unary data extracted. If the amount of unary data to be extracted is not equal to 32 bits (which is equal to zero), the decoder determines that chunk structure 57 is being used. In step S89, the decoder extracts data from the chunk according to chunk structure 57.

[0067] Based on the above described process in conjunction with Figure 7 and Figure 8 the decoder can efficiently recover the first stream of unary data 41 and the second stream of residue data 42 from the processed data stream stored in RAM by the encoder 20. By using Figure 7 and 8The flow control rules shown allow the decoder to determine the data type that will be found at any point in the input stream without incurring bit costs to indicate the data type within the compressed input data stream.

[0068] After extracting the primitive data and remainder data from the processed data stream, the decoder needs to decode the Columbus code to recover the uncompressed active data. The second stream 42 of remainder data is an array of fixed-length binary values ​​and can be directly decoded using techniques known in the art. Therefore, this process will not be discussed further here.

[0069] Now combined Figure 9a The following describes the decoding of the first stream 41 of unary data. In step S90, the 8-bit blocks of unary data are converted into binary form, where the binary form indicates one or more positions of stop bits within the binary block. This is done using a lookup table. In step S91, since the unary code can span 8-bit blocks, four 8-bit blocks are combined into a 32-bit block, and then four 32-bit blocks are combined into a 128-bit block. The 128-bit block is also a list of stop bit positions. To retrieve the unary value, in step S92, a difference is taken between the values ​​at each adjacent stop bit position, the difference giving the value of the unary code.

[0070] This can be achieved by using a lookup table to parse the 8-bit block of one-dimensional data into an intermediate form. Figure 9a The method. This is in Figure 9b The diagram shows the bit positions within each byte shown below, with the top line 94 indicating the bit positions. The first bit in each byte is bit '0', and the last bit is bit '7'. The second line 95 shows the bytes of one-bit metadata. Recall that the stop bit in one-bit metadata is '0'. In the intermediate form shown in the third line 96, each byte is expanded into a list of up to eight 3-bit codes indicating the position of the stop bit within the byte. A radix-4 combination of four 8-bit segments to 32-bit segments is performed, and the combination is then... Figure 9b The explanation is provided in lines 4.97 and 5.98. In line 4.97, a pair of flag bits are added as the most significant bit (MSB) to the code. For the first byte, the value '00' is added to the 3-bit code, for the third byte, the value '10' is added to the 3-bit code, and so on. In line 5, the 5-bit code is concatenated to form a list of stop bit positions in a 32-bit word. Subsequent radix-4 combinations of four 32-bit segments to 128-bit segments use a similar process to generate a list of 7-bit codes indicating the positions of unary stop bits. As in step S92, subtraction of adjacent values ​​yields the length of the unary data element, and thus the value of the unary data element.

[0071] As described above, the first particular implementation combines the unary part and the remainder part of the Golomb-Rice code within a cell. Each cell can include both the unary part and the remainder part. The advantage of mixing the unary part and the remainder part of the activation data within a cell is that the unary part and the remainder part can be spread across the processed data stream retrieved from the DRAM. This allows reducing the size of the parsing buffer at the decoder for storing the unary data before decoding, thus reducing the hardware requirements.

[0072] In the first particular implementation, the compression of activation data is discussed. In the second particular implementation, the techniques are applied to the compression of weight values. Figure 10a A mobile device 10 is shown illustrating the second particular implementation. Although a mobile device 10 is described herein, the described techniques can be applied to any type of computing device retrieving weight values associated with a neural network, including but not limited to a tablet computer, a laptop computer, a personal computer (PC), a server, etc. Figure 10b The hardware of the mobile device 10 is shown. The mobile device 10 includes a processing element in the form of a CPU 100 and a specialized processor in the form of a neural processing unit (NPU) 101. The mobile device 10 additionally includes a storage in the form of a random access memory (RAM) 102. Additional non-volatile storage is also provided, but it is not shown in Figure 10b The mobile device 10 includes a display 103 for displaying information to a user and a communication system 104 to allow the mobile device 10 to connect through various data networks to transmit and receive data using technologies such as Wi-Fi TM and LTE TM .

[0073] Figure 11 A system architecture installed on a mobile device 10 associated with the NPU 101 is shown. The system architecture allows software applications 110 to access the NPU 101 for hardware acceleration of computations related to a neural network. The system architecture is the Android (RTM) software architecture used on mobile phones, tablet computers, etc.

[0074] A software application 110 has been developed to leverage a machine learning library 111 to hardware accelerate certain processes related to neural network processing. Below the library is provided a runtime environment 112 (referred to as Android (RTM) Neural Network Runtime) that receives instructions and data from the application 110. The runtime environment 112 is an intermediate layer that is responsible for communication between the software application 110 and the NPU 101 and scheduling of execution tasks on the most suitable hardware. Below the runtime environment 112, there is provided at least one processor driver and associated specialized processor, in this case the NPU 101. Multiple processors and associated drivers can be provided below the runtime environment 112, such as digital signal processors, neural network processors, and graphics processing units (GPUs). However, to avoid repetitive description, only the NPU 101 and associated processor driver 113 will be described in connection with the second particular embodiment.

[0075] Figure 12 Subcomponents of the NPU 101 are shown. The NPU 101 includes a weight decoder 120 connected to a direct memory access component 121 that handles data transfers to the RAM 102 of the mobile device 10 on an external interface. Decoded values from the weight decoder 120 are sent to a multiplier accumulator unit 122 for subsequent processing by the NPU 101.

[0076] In the second particular embodiment, the processor driver 113 stores the weight values in the RAM 102 according to a chunk structure determined by a set of flow control rules. Subsequently, the direct memory access component 121 retrieves the weight values from the RAM 102 and the weight decoder 120 extracts data from the chunk structure.

[0077] The processor driver 113 obtains a set of uncompressed (raw) weight values for a neural network. The source of the uncompressed weight values is immaterial to the technology discussed herein. However, in one example, the uncompressed weight values can be provided to the Android Neural Network Runtime by the application 110.

[0078] The weight values are received in uncompressed format, such as binary data. A first step performed by the processor driver 113 is zero run length coding. Zero run length coding is beneficial if weight values with a value of 0 frequently occur in the weight stream. For a sequence of weight values that includes n non-zero weight values, the processor driver 113 forms an array of weight values (weight_values) as a sequence of non-zero weight values. The processor driver 113 also identifies an array of zero run length between the non-zero weight values (zruns). The array of zero runs has a length of n+1. In the zruns sequence, zruns[0] is the initial zero run length and zruns[n] is the ending zero run.

[0079] For example, consider the following sequence of weights: 0, 5, 6, 0, 0, 0, 7, 0. The processor driver 113 will code n = 3, because there are 3 non-zero values. The sequence of weight values is: weight_values = {5, 6, 7}, and the sequence of zero runs is zruns = {1, 0, 3, 1}. From these two sequences, the original sequence of weights can be reconstructed. In this way, the processor driver 113 separates the input stream of weight values into a sequence of weight_values and a sequence of zruns.

[0080] weight_values is converted into a Golomb-Rice code using a first divisor, and zruns is converted into a Golomb-Rice code using a second divisor. In the same way as in the first particular embodiment, the Golomb-Rice codes are separated by the processor driver 113 into a unary stream and a remainder stream. Thus, the processor driver 113 generates four different data streams to be included in the chunk for storage in the RAM 102: a unary part of weight_values (wunary), a remainder part of weight_values (wremain), a unary part of zruns (zunary), and a remainder part of zruns (zremain). These different data types are added to the chunk structure using a set of stream control rules, as described below.

[0081] The weight values received by the processor driver 113 are coded with slices, each slice having a slice header at the beginning of each slice. The slice header includes information about the divisor used to create the weight_value Golomb-Rice code and the divisor used to create the zrun Golomb-Rice code. The slice header also includes information about the slice length. Following the slice header is a plurality of chunks, which encode the different data types described above. The chunks alternate between including unary values and remainder values, the remainder values (wremain and zremain) being included in chunks following the chunks containing the corresponding unary values (wunary and zunary). Each chunk encoding a unary value (wunary and zunary) encodes at most 12 weight symbols and 12 zero run symbols, each symbol corresponding to the first part of a Golomb-Rice code. The number varies because the unary values corresponding to the symbols have variable length, and fewer unary values can be encoded if the length of the symbols is shorter than the length of the symbols more often. The length of the unary chunks has a maximum predefined value, which is chosen based on the characteristics of the decoder. The chunks following the unary data and including the remainder values have variable length, because they include remainder values corresponding to the unary symbols included in the previous chunk, as described above.

[0082] For a chunk that includes unary data, the addition of unary values is controlled by tracking a remainder, which is the number of wunary values added to chunks in the slice so far minus the number of zunary values added to chunks in the slice so far. If the remainder is greater than or equal to 8, only zunary values are included in the next unary chunk. If the remainder is less than 0, only wunary values are added to the next unary chunk.

[0083] To form the first chunk that includes unary data, up to 12 symbols of value wunary are added to the chunk, as long as the maximum chunk size is not exceeded. The remainder of the chunk is then padded with zunary values corresponding to the zrun symbols. The second chunk includes the remainder values (wremain and zremain) associated with the unary portion of the symbols added to the first chunk. When forming the third chunk, there are three possibilities. First, if the aforementioned remainder is between 0 and 7, the same process as for the first chunk follows, adding up to 12 symbols of wunary data to the chunk, followed by padding with zunary values corresponding to the zrun symbols. Second, if the remainder is greater than or equal to 8, the chunk is padded only with zunary values. This will allow the zunary values to catch up if more wunary values have been encoded. Third, if the remainder is less than 0, the chunk is padded only with wunary values. This will allow the wunary values to catch up if more zunary values have been encoded. The fourth chunk includes the remainder values corresponding to the symbols included in the third chunk. The logic of the aforementioned flow control rules is shown in Figure 13 The process continues until all of the data of the slice has been encoded.

[0084] At some later time, the direct memory access component 121 retrieves the weight data for use in the weight decoder 120. The weight decoder 120 identifies the values extracted from the processed data stream retrieved from the direct memory access component 121 as follows. The logic for decoding the chunks is shown in Figure 14 As with the encoding described previously, a remainder value is maintained by the weight decoder 120. The remainder value at the weight decoder 120 is the number of wunary values extracted from the chunks in the slice so far minus the number of zunary values extracted from the chunks in the slice so far.

[0085] In the first chunk to be decoded, the weight decoder 120 begins extracting wunary values until all of the data from the chunk has been extracted or 12 symbols of value wunary have been extracted. The subsequent values in the first chunk are extracted as zunary values.

[0086] As discussed in connection with encoding, the group chunk following the unary chunk includes remain values corresponding to the sign of the symbols in the previous group chunk. The number of symbols in the previous group chunk having a value of wunary is known. Accordingly, the same number of wremain values corresponding to the wunary values in the previous group chunk are extracted, and any subsequent values are extracted as zremain values. For the subsequent unary group chunk, the remainder is checked. If the remainder is less than zero, the weight decoder 120 extracts all the unary values as wunary values. If the remainder is 8 or greater, the weight decoder extracts all the values as zunary values. If the remainder is between 0 and 7, the first unary data up to 12 symbols is extracted as wunary values, and any subsequent values are extracted as zunary values.

[0087] In this way, the weight decoder 120 can extract the unary and remainder portions of the zrun and weight sequence data without incurring any bit cost in the data stream to identify the type of data.

[0088] The extracted zrun and weight sequence data are subsequently decoded to recover the underlying weight values. Decoding of the unary and remainder portions of the Golomb-Rice code is discussed in connection with the first embodiment, and is not repeated here.

[0089] The above-described embodiments should be understood as illustrative examples. Other embodiments are contemplated. For example, in the first embodiment, the unary data is added to the group chunk in 32-bit portions. However, the size of the portions is not important, and Figure 5 The illustrated group chunk structure can be adjusted according to the specific requirements of the processor that will parse the group chunk.

[0090] The second embodiment utilizes the Android neural network architecture. However, depending on the circumstances, the techniques described herein can be applied to different software architectures. For example, a different software architecture will be used in the context of a server-based implementation.

Claims

1. A method for use by a processing element, the method comprising: obtaining a first plurality of variable length codes, wherein each variable length code has a variable length unary portion and a fixed length mantissa portion, separating the variable length codes into unary portions of the variable length codes and mantissa portions of the variable length codes; and using a selected chunk structure to form a processed stream in chunks such that for each chunk of the processed stream containing data from the unary portions of the variable length codes, the data from the unary portions forms a substream within the chunk of the processed stream having a size determined based on an amount of the unary portions of the variable length codes that remain to be added to the processed stream.

2. The method of claim 1, the mantissa portions of the variable length codes being truncated binary portions.

3. The method of claim 1, wherein each chunk of the processed stream can include data from the unary portions of the variable length codes and data from the mantissa portions of the variable length codes, include data from the unary portions of the variable length codes but not data from the mantissa portions of the variable length codes, or include data from the mantissa portions of the variable length codes but not data from the unary portions of the variable length codes.

4. The method of claim 1, wherein the processed stream is formed of units having a predetermined length, and wherein the amount of the unary portions of the variable length codes that remain to be added to the processed stream is a number of bits of the unary portions of the variable length codes associated with a unit that remain to be added to the processed stream. obtaining a second plurality of variable length codes, and 5. The method of claim 1, further comprising: separating the second plurality of variable length codes into unary portions of the variable length codes and mantissa portions of the variable length codes; wherein forming the processed stream in chunks includes, for each chunk of the processed stream containing data from unary portions of at least one of the first plurality of variable length codes and the second plurality of variable length codes, including in the chunk a number of unary portions of the first plurality of variable length codes and a number of unary portions of the second plurality of variable length codes. maintaining a balance value that records a difference between the number of unary portions of the first plurality of variable length codes that have been included in a chunk and the number of unary portions of the second plurality of variable length codes that have been included in a chunk, and that is used to determine the number of unary portions of the first plurality of variable length codes and the number of unary portions of the second plurality of variable length codes that are included in a chunk.

6. The method of claim 5, wherein forming the treated stream comprises:

7. A method of decoding a processed data stream, the method comprising: ​ obtaining a processed data stream containing data related to a plurality of variable length codes, wherein each variable length code has a variable length unary portion and a fixed length mantissa portion, and the processed data stream is formed in chunks, wherein at least one chunk of the processed data stream includes a substream of data formed from the unary portions of the variable length codes; and identifying and extracting the substreams within the chunks of the processed data stream based on the amount of the unary portions of the variable length codes that remain to be extracted from the processed data stream using a selected chunk structure.

8. The method of claim 7, wherein each unary portion of the variable length codes is encoded as unary data having a stop bit, the method further comprising: obtaining a plurality of identified and extracted substreams from a plurality of chunks of the processed data stream; converting each extracted substream into an intermediate form representing a list of bit positions of stop bits within the substream; combining a plurality of converted substreams to form an expanded list of bit positions; and measuring distances between the stop bits in the expanded list of bit positions to recover values of the unary portions of the variable length codes in the processed data stream.

9. A data processing system comprising a processing element and a storage device, the storage device storing a code portion which when executed by the processing element causes the data processing system to: obtain a plurality of variable length codes, wherein each variable length code has a variable length unary portion and a fixed length mantissa portion, divide the variable length codes into unary portions of the variable length codes and mantissa portions of the variable length codes; and form a processed stream in chunks using a selected chunk structure such that for each chunk of the processed stream containing data from the unary portions of the variable length codes, the data from the unary portions forms a substream within the chunk of the processed stream having a size determined based on an amount of the unary portions of the variable length codes that remain to be added to the processed stream.

Images