Regular expression filter for Unicode transform format strings

Abstract

The invention is entitled "Regular Expression Filter for Unicode Transformation Format Strings". The invention provides an apparatus comprising a memory system having a memory die, the memory system comprising circuitry configured to receive a plurality of bytes at a first input terminal, the plurality of bytes comprising an input stream of a plurality of first variable length encoded symbols, and the circuitry configured to determine a sequence of symbol byte lengths for each of the plurality of first variable length encoded symbols.

Description

Background Technology

[0001] A host device coupled to a storage system, such as a solid-state drive (SSD), that can execute software applications to search for data stored on the SSD. Regular expressions are commonly used to perform such searches. A regular expression is a sequence of symbols that forms a search pattern in string matching. For example, a regular expression can be used to search for all files, documents, directories, or other similar data objects on an SSD that contain a specific word or phrase.

[0002] Typically, this type of regular expression matching is performed on the host device. Specifically, the host device reads the data to be searched from the SSD, and then performs various functions to determine whether any retrieved data matches the regular expression search pattern. While this technique achieves the desired results, transferring large amounts of data between the SSD and the host device to perform regular expression matching is inefficient and time-consuming.

[0003] One possibility for improving the speed and efficiency of regular expression matching is to implement regular expression filters in hardware on the memory device to reduce the amount of data transferred from the memory device to the host device. Hardware regular expression filters do not perform a complete regular expression matching operation. Instead, they typically search for strings that match the first few symbols of the regular expression search pattern (sometimes called "triggers") to identify relevant strings that can match the entire regular expression search pattern.

[0004] The filtered data, including only the identified relevant strings, can then be transferred to a processor (e.g., on the host device) to perform the remaining regular expression matching in the software. In this respect, hardware regular expression filters reduce the amount of data that must be transferred between the memory device and the host device for regular expression matching.

[0005] Existing hardware regular expression filters are primarily used for network packet filtering and are designed for high bandwidth and high parallelism. Since almost all network traffic is ASCII encoded, existing hardware regular expression filters typically operate on ASCII data, where each symbol is encoded using a single byte. Hardware regular expression filters for such fixed-length encoding schemes are simple to implement. For example, if the input consists of a string of N single-byte symbols, the hardware regular expression filter can simply operate on N parallel comparators, each operating on N individual bytes of the input string.

[0006] In contrast, variable-length encoding schemes, such as the Unicode Transform Format (UTF) encoding scheme, are typically used to encode text data (such as in documents and web pages). The most commonly used variable-length encoding schemes are UTF-8 and UTF-16. UTF-8 uses one to four byte code units to encode symbols, while UTF-16 uses one or two word code units (i.e., two bytes or four bytes) to encode symbols. Therefore, designing hardware regular expression filters for such variable-length encoded data is more complex than designing hardware regular expression filters for ASCII encoded data. In fact, if the input string contains N symbols, each of which can be encoded using one to four bytes, a hardware regular expression filter cannot simply operate on N parallel comparators, each operating on N individual bytes of the input string.

[0007] In addition to the complexity caused by variable-length encoding, UTF has a considerable number of variations in data representation and can be encoded in either big-endian or little-endian order. Furthermore, UTF has four different graph encoding methods: Normalization Form Canonical Decomposition (NFD), Normalization Form Canonical Composition (NFC), Normalization Form Compatibility Decomposition (NFKD), and Normalization Form Compatibility Composition (NFKC). Additionally, there are localized variants of Unicode, such as the Chinese national standard GB 18030. Due to this complexity, designing hardware regular expression filters that can be used with all variants of variable-length encoding schemes (such as Unicode) and correctly interpret data is extremely difficult. Attached Figure Description

[0008] Components with similar numbers refer to common parts in different drawings.

[0009] Figure 1 A block diagram depicting one implementation of a memory system.

[0010] Figure 2 This is a block diagram of one implementation scheme for a memory die.

[0011] Figure 3 A perspective view of one implementation of a three-dimensional memory structure.

[0012] Figure 4 A flowchart illustrating an implementation of a process for determining the associated byte length value for each byte of a UTF-8 encoded symbol stream.

[0013] Figure 5 A diagram illustrating an implementation of a hardware circuit for determining the byte length value associated with a byte of a UTF-8 encoded symbol.

[0014] Figure 6 A diagram illustrating an exemplary byte stream of UTF-8 encoded symbols.

[0015] Figure 7 A diagram illustrating an implementation of a technique for filtering S-byte length values ​​from a stream of byte length values.

[0016] Figure 8A An implementation scheme for a hardware switching circuit configured to perform the exchange of byte-length values.

[0017] Figure 8B An illustration depicting an exemplary hardware jump filter.

[0018] Figure 8C To depict Figure 8B A diagram illustrating a specific implementation of an exemplary hardware jump filter.

[0019] Figure 9 This is a flowchart illustrating an implementation of a process for determining the byte length value of a UTF-16 encoded symbol stream.

[0020] Figure 10 This is a diagram illustrating an implementation of a hardware circuit for determining the byte length value of a UTF-16 encoded symbol.

[0021] Figure 11 An illustration depicting an exemplary word stream of UTF-16 encoded symbols.

[0022] Figure 12A An illustration depicting an exemplary hardware jump filter.

[0023] Figure 12B To depict Figure 12A A diagram illustrating a specific implementation of an exemplary hardware jump filter.

[0024] Figure 13A A diagram illustrating an implementation of a hardware circuit for determining the byte length value of symbols encoded in both UTF-8 and UTF-16.

[0025] Figure 13B for Figure 13AA diagram illustrating an implementation scheme for a UTF-8 / 16 high-bit-length decoder circuit.

[0026] Figure 13C for Figure 13A A diagram illustrating an implementation scheme for a UTF-8 / 16 low-bit-length decoder circuit.

[0027] Figure 14A An illustration of an exemplary word stream depicting UTF-8 encoded symbols.

[0028] Figure 14B An illustration depicting an exemplary word stream of UTF-16 encoded symbols.

[0029] Figure 15A A diagram illustrating an implementation of a hardware circuit for determining the associated byte length value of an N-symbol string of symbols encoded in UTF-8 or UTF-16.

[0030] Figure 15B for Figure 15A A diagram illustrating the specific implementation scheme of the hardware circuit.

[0031] Figure 16 A diagram illustrating an implementation of a hardware circuit for determining the byte length value of a 4N consecutive N symbol string of UTF-8 or UTF-16 encoded symbols.

[0032] Figure 17 A diagram illustrating an implementation of a hardware circuit for determining the byte length value of a 4N consecutive N symbol string of UTF-8 or UTF-16 encoded symbols. Detailed Implementation

[0033] This invention describes a technique for a hardware regular expression filter that can be used with all variants of variable-length encoding schemes (such as Unicode) and correctly interprets data. In embodiments, a hardware regular expression filter is described that does not determine the composition of the actual symbols or encoded symbols in the input stream. Furthermore, the described hardware regular expression filter does not require knowledge of the type of graph encoding (NFD / NFC / NFKD / NFKC) used to encode the symbols.

[0034] As mentioned above, UTF-8 uses one to four byte code units to encode symbols, while UTF-16 uses one or two word code units (i.e., two bytes or four bytes) to encode symbols. A technique for a hardware regular expression filter is described, which determines the number of encoded bytes (referred to herein as the "symbol byte length" or "SBL") for each encoded symbol in an input stream. For example, the input stream could be a data stream read from a memory system such as an SSD or other input data source. The determined symbol byte length is then compared to the symbol byte length of symbols in a trigger to identify any strings in the input stream that match the symbol byte length of the trigger.

[0035] In the implementation, the trigger may include N symbols, where N is an integer. For example, the trigger may include N = 6 symbols: In UTF-8, "E" is encoded using 1 byte, "s" is encoded using 1 byte, "p" is encoded using 1 byte, and "a" is encoded using 1 byte. It is encoded using 2 bytes, and "a" is encoded using 1 byte. Therefore, the N=6 symbol byte length of this exemplary trigger is: [1][1][1][1][2][1], corresponding to the symbol...

[0036] The described hardware regular expression filter implementation identifies strings (referred to herein as "N-symbol strings") comprising N symbols having a symbol byte length matching the symbol byte length of a flip-flop. Specifically, the described hardware regular expression filter implementation determines the symbol byte length of each symbol in each N-symbol string of the input data and compares the determined symbol byte length of each N-symbol string with the N-symbol byte length of the flip-flop to identify N-symbol strings having a symbol byte length matching the flip-flop's symbol byte length. To reduce complexity and make hardware implementation feasible, the described hardware circuit does not identify or compare actual symbols—only the symbol byte lengths are compared.

[0037] Any N-character string of the input data with a matching symbol length of byte length is identified as relevant because such a string can match the entire regular expression search pattern. For example, if the input stream contains an N=6-character string... The length of the symbol bytes of the string is [1][1][1][1][2][1], and therefore the string is identified as relevant because such a string can match the entire regular expression search pattern.

[0038] Any N-character strings in the input data that do not have a matching symbol length of byte length are irrelevant and can be ignored because such strings may never match the trigger, and therefore may never match the entire regular expression search pattern. For example, if the input stream contains N=6 character strings... The length of the symbol byte in the string is [3][1][1][1][2][1], and therefore the string can be ignored because the symbol in the string cannot match the symbol in the trigger.

[0039] Because the described hardware regular expression filter does not recognize or compare these actual symbols, false positives may occur. For example, if the input stream contains a string of N=6 symbols... The length of the symbol byte of the string is [1][1][1][1][2][1], and therefore the string is identified as relevant because the length of the symbol byte of the string matches the length of the symbol byte of the trigger, so such a string can match the entire regular expression search pattern.

[0040] The filtered data, including the identified relevant strings, can then be transmitted to a processor (e.g., on a host device) to perform the remaining regular expression matching in the software. Even if the identified relevant strings may include false positives, the filtered data is still eliminated from the input data stream of considered strings that cannot match the regular expression. In fact, there is no reason to transmit data to the host if there is no N-symbol string that matches the symbol byte length of the trigger, since the data cannot match the regular expression.

[0041] Therefore, the described implementation of the hardware regular expression filter can be used to filter data that cannot be matched with a regular expression. In this respect, the speed and performance of the regular expression software engine can be reduced, and the complexity of the hardware regular expression filter can be reduced, thus making hardware regular expression filters feasible for variable-length encoded data (such as Unicode data).

[0042] Figure 1 This is a block diagram illustrating an embodiment of the memory system 100 for implementing the described technology. In one embodiment, the memory system 100 is an SSD. The memory system 100 may also be a memory card, a USB drive, or other type of storage system. The technology of the present invention is not limited to any one type of memory system. The memory system 100 is connected to a host 102, which may be a computer, server, electronic device (e.g., a smartphone, tablet, or other mobile device), appliance, or another device that uses memory and has data processing capabilities. In some embodiments, the host 102 is separate from but connected to the memory system 100. In other embodiments, the memory system 100 is embedded within the host 102.

[0043] Figure 1 The components of the memory system 100 depicted are electronic circuits. The memory system 100 includes a controller 104 connected to one or more memory dies 106 and local high-speed volatile memory 108 (e.g., DRAM). Each of the one or more memory dies 106 includes a plurality of non-volatile memory cells. More information regarding the structure of each memory die 106 is provided below. The controller 104 uses the local high-speed volatile memory 108 to perform certain functions.

[0044] Controller 104 includes a host interface 110 that connects to and communicates with host 102. In one embodiment, host interface 110 provides a PCIe interface. Other interfaces, such as SCSI, SATA, etc., may also be used. Host interface 110 is also connected to a network on-chip (NOC) 112, which is a communication subsystem located on an integrated circuit. In other embodiments, NOC 112 may be replaced by a bus.

[0045] Connected to and communicating with NOC 112 are processor 114, ECC engine 116, memory interface 118, and DRAM controller 120. DRAM controller 120 is used to operate and communicate with local high-speed volatile memory 108 (e.g., DRAM). In other embodiments, local high-speed volatile memory 108 may be SRAM or another type of volatile memory.

[0046] ECC engine 116 performs error correction. For example, ECC engine 116 performs data encoding and decoding according to implemented ECC technology. In one embodiment, ECC engine 116 is a software-programmable electronic circuit. For example, ECC engine 116 may be a programmable processor. In other embodiments, ECC engine 116 is a custom-designed dedicated hardware circuit without any software. In yet another embodiment, the functionality of ECC engine 116 is implemented by processor 114.

[0047] Processor 114 performs various controller memory operations, such as programming, erasing, reading, and memory management processes. In some embodiments, processor 114 is programmed by firmware. In other embodiments, processor 114 is a custom-designed dedicated hardware circuit without any software. In some embodiments, processor 114 also implements a conversion module, either as a software / firmware process or as dedicated hardware circuitry.

[0048] In one implementation, memory interface 118 communicates with one or more memory dies 106. In another implementation, memory interface 118 provides a mode-switching interface. Other interfaces may also be used. In some exemplary implementations, memory interface 118 (or another part of controller 104) implements a scheduler and buffer for transferring data to and receiving data from one or more memory dies.

[0049] Figure 2 This is a functional block diagram of one embodiment of the memory die 200. Figure 1 Each of the one or more memory dies 106 can be implemented as Figure 2 The memory die 200. Figure 2 The components depicted are electronic circuits. In an embodiment, each memory die 200 includes a memory structure 202, control circuitry 204, and read / write circuitry 206. The memory structure 202 is addressable via word lines through a row decoder 208 and via bit lines through a column decoder 210.

[0050] In one embodiment, the read / write circuitry 206 includes a plurality of sensing blocks 212 (including SB1, SB2, ..., SBp (sensing circuitry)) and allows one or more data pages of a plurality of memory cells to be read or programmed (written) in parallel. In one embodiment, each sensing block 212 includes a sensing amplifier and a set of latches connected to a bit line. The latches store data to be written and / or data already read. In one embodiment, the sensing amplifier of each sensing block 212 includes a bit line driver. In one embodiment, commands and data are transmitted between the controller 104 and the memory die 200 via line 214. In one embodiment, the memory die 200 includes a set of input and / or output (I / O) pins connected to line 214.

[0051] In one embodiment, control circuitry 204 cooperates with read / write circuitry 206 to perform memory operations (e.g., write, read, erase, etc.) on memory structure 202. In another embodiment, control circuitry 204 includes a state machine 216, an on-chip address decoder 218, and a power control module 220. In yet another embodiment, control circuitry 204 also includes a regular expression filter circuit 222.

[0052] In one implementation, state machine 216 provides die-level control of memory operations. In another implementation, state machine 216 may be software-programmable. In yet another implementation, state machine 216 is implemented entirely in hardware (e.g., electronic circuitry) without using software. In some implementations, state machine 216 may be replaced by a microcontroller or microprocessor. In one implementation, control circuitry 204 includes buffers such as registers, ROM fuses, and other storage devices for storing default values ​​(such as base voltage and other parameters).

[0053] The on-chip address decoder 218 provides the address interface between addresses used by the controller 104 to the hardware addresses used by the row decoder 208 and the column decoder 210. The power control module 220 controls the power and voltage supplied to the word lines and bit lines during memory operations. The power control module 220 may include a charge pump for generating voltage.

[0054] As described in more detail below, the regular expression filter circuit 222 includes circuitry for determining the symbol byte length of symbols in an N-symbol string of input data (e.g., a data stream read from memory structure 202), and comparing the determined symbol byte length of each N-symbol string with the N symbol byte lengths of a flip-flop to identify an N-symbol string having a symbol byte length that matches the symbol byte length of the flip-flop.

[0055] For the purposes of this document, control circuitry 204, read / write circuitry 206, row decoder 208, and column decoder 210 include control circuitry for memory structure 202. In other embodiments, other circuitry supporting and operating on memory structure 202 may be referred to as control circuitry. For example, in some embodiments, controller 104 may operate as control circuitry or may be part of control circuitry. Control circuitry may also be implemented as a microprocessor or other types of processors that are hard-wired or programmed to perform the functions described herein.

[0056] In one embodiment, memory structure 202 is a three-dimensional memory array of non-volatile memory cells. In another embodiment, memory structure 202 is a monolithic three-dimensional memory array in which multiple memory layers are formed over a single substrate, such as a wafer. The memory structure can be any type of non-volatile memory formed in one or more physical layers of the memory cell array, having an active region disposed over a silicon (or other type) substrate. In one example, the non-volatile memory cells of memory structure 202 comprise vertical NAND strings having a charge-trapping material (such as the material described). The NAND strings comprise memory cells connected by channels.

[0057] In another embodiment, memory structure 202 includes a two-dimensional memory array of non-volatile memory cells. In this example, the non-volatile memory cells are NAND flash memory cells utilizing floating gates. Other types of memory cells (e.g., NOR flash memory) may also be used.

[0058] The exact type of memory array architecture or memory cell included in memory structure 202 is not limited to the examples above. Many different types of memory array architecture or memory cell technologies can be used to form memory structure 202. For the purposes of the new techniques described herein, no specific non-volatile memory technology is required.

[0059] Other examples of suitable technologies for the memory cells of memory structure 202 include ReRAM memory, magnetoresistive memory (MRAM), phase-change memory (PCM), etc. Examples of suitable technologies for the architecture of memory structure 202 include two-dimensional arrays, three-dimensional arrays, cross-point arrays, stacked two-dimensional arrays, vertical bitline arrays, etc.

[0060] One example of a crosspoint memory includes reversible resistor switching elements arranged in a crosspoint array accessed by X-line and Y-line (e.g., word line and bit line). In another embodiment, the memory cell may include a conductive bridge memory element. A conductive bridge memory element may also be referred to as a programmable metallized cell.

[0061] Based on the physical repositioning of ions within a solid electrolyte, a conductive bridge memory element can be used as a state-changing element. In some cases, a conductive bridge memory element may comprise two solid metal electrodes, one relatively inert (e.g., tungsten) and the other electrochemically active (e.g., silver or copper), with a thin film of solid electrolyte between the two electrodes.

[0062] MRAM uses magnetic storage elements to store data. A magnetic storage element is formed of two ferromagnetic plates, each of which remains magnetized and is separated by a thin insulating layer. One of the plates is a permanent magnet set to a specific polarity; the magnetization of the other plate can be changed to match the magnetization of an external magnetic field to store memory. Memory devices are constructed from a grid of such memory cells. In one embodiment for programming, each memory cell is located between a pair of write lines arranged at right angles to each other, parallel to the cell, one above and one below. When current passes through them, an induced magnetic field is generated.

[0063] Phase-change memories (PCMs) utilize the unique properties of chalcogenide glasses. One implementation uses a GeTe-Sb₂Te₃ superlattice to achieve a non-thermal phase transition by simply altering the coordination state of germanium atoms with a laser pulse (or a light pulse from another source). Therefore, the programming dose is the laser pulse. Programming of the memory cells can be suppressed by preventing the memory cells from receiving light.

[0064] Those skilled in the art will recognize that the techniques described herein are not limited to a single specific memory structure, but encompass many related memory structures within the scope of the techniques described herein and as understood by those skilled in the art.

[0065] Figure 3 This is a perspective view as part of an embodiment of a three-dimensional memory array including memory structure 202. In this embodiment, memory structure 202 includes a plurality of non-volatile memory cells. For example, Figure 3 A portion of a block of memory cells is shown. The depicted structure includes a set of bit lines BL, which sit above an alternating stack of dielectric and conductive layers. For illustrative purposes, one of the dielectric layers is labeled D, and one of the conductive layers (also known as word line layers) is labeled W.

[0066] The number of alternating dielectric and conductive layers can vary depending on specific implementation requirements. One set of embodiments includes between 108 and 300 alternating dielectric and conductive layers. An example embodiment includes 96 data word line layers, 8 select layers, 6 dummy word line layers, and 110 dielectric layers. More or fewer than 108 to 300 layers may also be used. In this embodiment, the alternating dielectric and conductive layers are divided into four regions by local interconnects LI. Figure 3 Two regions and two local interconnects LI are shown.

[0067] The source line layer (SL) lies beneath alternating dielectric and word line layers. Memory vias are formed within this stack of alternating dielectric and conductive layers. For example, a memory via is labeled MH. Note that in... Figure 3 In the diagram, the dielectric layers are depicted as a perspective view, allowing the reader to see the memory holes located within the stack of alternating dielectric and conductive layers.

[0068] In one embodiment, a NAND string is formed by filling memory holes with a material including a charge-trapping material to form a vertical column of memory cells. In another embodiment, each memory cell may store one or more data bits. In yet another embodiment, each memory hole MH is associated with and coupled to a corresponding bit line BL in the bit lines BL. In yet another embodiment, each bit line BL is coupled to one or more memory holes MH.

[0069] As described above, a technique for hardware regular expression filters is presented that can be used with all variants of variable-length encoding schemes (such as Unicode) and correctly interprets data. In the implementation, a hardware regular expression filter is described that is uncertain about the actual symbols or the composition of symbols in the input stream. Instead, as described in more detail below, hardware regular expression filters are described that determine the symbol byte length in a string of variable-length encoded input data and identify strings with symbol byte lengths that match the symbol byte length of the trigger.

[0070] As mentioned above, text data (such as in documents and web pages) is typically encoded using variable-length encoding schemes (such as UTF-8 and UTF-16), and therefore this text data, encoded as UTF-8 or UTF-16, is usually stored in memory systems (such as...). Figure 1 In the memory system 100), UTF-8 uses one to four byte code units to encode symbols. UTF-8 encoded symbols have the format shown in Table 1 below:

[0071] Table 1

[0072]

[0073] The "x" character is replaced by a bit of the code corresponding to a specific symbol.

[0074] As shown in Table 1, the symbol byte length of a UTF-8 encoded symbol can be determined based on the first byte. Specifically, if the most significant bit of the byte is 0, the symbol byte length is 1. For all other encodings, the number of leading 1s in the byte equals the symbol byte length. That is, if bits 7 to 5 of the byte equal 110, the symbol byte length is 2; if bits 7 to 4 of the byte equal 1110, the symbol byte length is 3; and if bits 7 to 3 of the byte equal 11110, the symbol byte length is 4. For multi-byte encoded symbols, all bytes following the first byte have bits 7 to 6 equal to 10. Such bytes will be referred to herein as "skip bytes" with a symbol byte length specified as "S". Other values ​​can be used for the symbol byte length of the skip bytes.

[0075] For example, the symbols @, §, The UTF-8 encoding is shown in Table 2:

[0076] Table 2

[0077]

[0078] For example, as mentioned above, The symbol's sign byte length is four bytes. Bits 7 to 3 of the first byte equal 11110, and bits 7 to 6 of bytes 2 to 4 equal 10. Therefore, bytes 2 to 4 are the jump bytes. The symbol for the '#' symbol is three bytes long. Bits 7 to 4 of the first byte equal 1110, and bits 7 to 6 of bytes 2 and 3 equal 10. Therefore, bytes 2 and 3 are jump bytes. The symbol for the '#' symbol is two bytes long. Bits 7 to 5 of the first byte equal 110, and bits 7 to 6 of the second byte equal 10. Therefore, byte 2 is a jump byte. The symbol for the '#' symbol is one byte long. Bit 7 of the first byte equals 0.

[0079] Referring again to Table 1, each byte of a UTF-8 encoded symbol is either a first byte that includes information that can be used to determine the symbol byte length of the associated encoded symbol, or a jump byte that does not include such symbol byte length information. In an implementation, each byte of a UTF-8 encoded symbol has an associated “byte length value” or “BLV”, which can be identified based on the presence or absence of a predetermined bit pattern in each byte.

[0080] In the implementation, as described above, the associated byte length value of each first byte is 1, 2, 3, or 4 based on the values ​​of bits 7 to 3 of the byte. In the implementation, each jump byte has an associated byte length value S determined based on bits 7 to 6 of that byte. Therefore, if the associated byte length value of a byte of a UTF-8 encoded symbol is 1, 2, 3, or 4, then the symbol byte length of the encoded symbol is equal to the byte length value associated with that byte. If the associated byte length value of a byte of a UTF-8 encoded symbol is S, then that byte is a jump byte and does not include the symbol byte length information of the encoded symbol.

[0081] Figure 4 This is a flowchart illustrating an embodiment of a process 400 for determining the associated Byte Length Value (BLV) for each byte of a UTF-8 encoded symbol stream. In an exemplary embodiment, the aforementioned control circuitry is used to control the memory die 106 ( Figure 1 Process 400 is executed on state machine 216. For example, process 400 can be executed on state machine 216. Figure 2 ) in the direction of. In another exemplary embodiment, in controller 104 ( Figure 1 The execution process is 400.

[0082] At step 402, byte data is received from an input stream of UTF-8 encoded symbols. For example, the input stream could be from memory die 106 ( Figure 1 (or other input data sources) are streams of data.

[0083] At step 404, it is determined whether bit 7 of the received byte is equal to 0. If so, at step 406, the associated byte length value of that byte is set to 1. Then, process 400 proceeds to step 408 to determine if there are any additional bytes to be evaluated. If so, process 400 returns to step 402 to receive the next byte. If it is determined at step 408 that there are no additional bytes to be evaluated, process 400 terminates.

[0084] If it is determined at step 404 that bit 7 of the received byte is not equal to 0, then at step 410, it is determined whether bits 7 through 5 of the received byte are equal to 110. If so, then at step 412, the associated byte length value of that byte is set to 2. Then, process 400 proceeds to step 408 to determine if there are any additional bytes to be evaluated. If so, process 400 returns to step 402 to receive the next byte. If it is determined at step 408 that there are no additional bytes to be evaluated, then process 400 terminates.

[0085] If it is determined at step 410 that bits 7 through 5 of the received byte are not equal to 110, then at step 414, it is determined whether bits 7 through 4 of the received byte are equal to 1110. If so, at step 416, the associated byte length value of that byte is set to 3. Then, process 400 proceeds to step 408 to determine if there are any additional bytes to be evaluated. If so, process 400 returns to step 402 to receive the next byte. If it is determined at step 408 that there are no additional bytes to be evaluated, process 400 terminates.

[0086] If it is determined at step 414 that bits 7 through 4 of the received byte are not equal to 1110, then at step 418, it is determined whether bits 7 through 3 of the received byte are equal to 11110. If so, then at step 420, the associated byte length value of that byte is set to 4. Then, process 400 proceeds to step 408 to determine if there are any additional bytes to be evaluated. If so, process 400 returns to step 402 to receive the next byte. If it is determined at step 408 that there are no additional bytes to be evaluated, then process 400 terminates.

[0087] If it is determined at step 418 that bits 7 through 3 of the received byte are not equal to 11110, then at step 422, it is determined whether bits 7 through 6 of the received byte are equal to 10. If so, the byte is a jump byte, and at step 424, the associated byte length value of the byte is set to S. Then process 400 proceeds to step 408 to determine if there are any additional bytes to be evaluated. If so, process 400 returns to step 402 to receive the next byte. If it is determined at step 408 that there are no additional bytes to be evaluated, process 400 terminates.

[0088] If, at step 422, it is determined that bits 7 through 6 of the received byte are not equal to 10, then the byte corresponds to an incorrect symbol encoding (also known as "unresolved encoding") because the received byte does not conform to the format depicted in Table 1 above. For example, an error in one or more bits may result in unresolved encoding. In the implementation, such unresolved encoding is treated as a jump byte, causing such erroneous symbols to be ignored.

[0089] Therefore, at step 426, the associated byte length value of the byte is set to S. Process 400 then proceeds to step 408 to determine if any additional bytes are present to be evaluated. If so, process 400 returns to step 402 to receive the next byte. If it is determined at step 408 that no additional bytes are present to be evaluated, process 400 terminates. Those skilled in the art will understand that the order of the steps in process 400 can be altered.

[0090] Figure 5 This illustration shows an embodiment of a hardware circuit (UTF-8 length decoder circuit 500) used to determine the byte length value associated with a byte of a UTF-8 encoded symbol. In this embodiment, the UTF-8 length decoder circuit 500 may be located in one or more memory dies 106 ( Figure 1 ), controller 104 ( Figure 1 Implemented on ( ) or a combination thereof. In an implementation, the UTF-8 length decoder circuit 500 may be implemented on the regular expression filter circuit 222 ( ) Figure 2 In other embodiments, the UTF-8 length decoder circuit 500 may be included in the memory system 100. Figure 1 It can be implemented in or coupled to other hardware circuits of the memory system.

[0091] In one embodiment, the UTF-8 length decoder circuit 500 includes comparator circuits 502a to 502e, decoder circuit 504, OR gate 506, and combination circuit 508. In another embodiment, the UTF-8 length decoder circuit 500 receives 8 bits (one byte) of input data at a time from the input stream at input terminal 510, determines the associated byte length value for each received input byte, and outputs an encoded byte length value (EBLV) corresponding to the determined byte length value at output terminal 512.

[0092] Each of the comparator circuits 502a to 502e has an input terminal coupled to input terminal 510 and provides a single bit output value (0 or 1) for each received input byte to the decoder circuit 504. In an embodiment, the comparator circuits 502a to 502e determine the associated byte length value for each received input byte.

[0093] Specifically, the comparator circuit 502a compares bit 7 of the received input byte with the binary value "0" (corresponding to...). Figure 4 (Step 404 in process 400). If bit 7 = 0, the comparator circuit 502a provides an output of 1 (which indicates that the associated byte length of the received input byte is 1). Otherwise, if bit 7 ≠ 0, the comparator circuit 502a provides an output of 0.

[0094] Comparator circuit 502b compares bits 7 through 5 of the received input byte with the 3-bit binary value "110" (corresponding to...). Figure 4 (Step 410 in process 400). If bits 7 to 5 = 110, the comparator circuit 502b provides an output of 1 (indicating that the associated byte length of the received input byte is 2). Otherwise, if bits 7 to 5 ≠ 110, the comparator circuit 502b provides an output of 0.

[0095] Comparator circuit 502c compares bits 7 through 4 of the received input byte with the 4-bit binary value "1110" (corresponding to...). Figure 4 (Step 414 in process 400). If bits 7 to 4 = 1110, the comparator circuit 502c provides an output of 1 (indicating that the associated byte length of the received input byte is 3). Otherwise, if bits 7 to 4 ≠ 1110, the comparator circuit 502c provides an output of 0.

[0096] Comparator circuit 502d compares bits 7 through 3 of the received input byte with the 5-bit binary value "11110" (corresponding to...). Figure 4(Step 418 in process 400). If bits 7 to 3 = 11110, then the comparator circuit 502d provides an output of 1 (which indicates that the associated byte length of the received input byte is 4). Otherwise, if bits 7 to 3 ≠ 11110, then the comparator circuit 502d provides an output of 0.

[0097] Comparator circuit 502e compares bits 7 through 6 of the received input byte with the 2-bit binary value "10" (corresponding to...). Figure 4 (Step 422 in process 400). If bits 7 to 6 = 10, the comparator circuit 502e provides an output of 1 (indicating that the received input byte is a jump byte and the associated byte length value is S). Otherwise, if bits 7 to 6 ≠ 10, the comparator circuit 502e provides an output of 0.

[0098] Decoder circuit 504 receives the 1-bit output of comparator circuits 502a to 502d as a four-bit input, generates a 2-bit decoder output based on the input to decoder circuit 504, and also generates a 1-bit "jump output" based on the input to decoder circuit 504. In an embodiment, the outputs of comparator circuits 502a to 502d constitute bits 3 to 0 of the input to decoder circuit 504, respectively.

[0099] like Figure 5 As depicted, decoder circuit 504 provides a 2-bit decoder output corresponding to the input to decoder circuit 504, which is shown in Table 3 below:

[0100] Table 3

[0101] Decoder input Decoder output byte length value 1000 00 1 0100 01 2 0010 10 3 0001 11 4 0000 00 S

[0102] The last row of Table 3 contains the non-discriminable encoding for the case where all comparators 502a to 502d generate zero output (corresponding to...). Figure 4 (Step 426 in process 400). As described above, such unresolved codes are treated as jump bytes, causing such error symbols to be ignored. Figure 5 As depicted, decoder circuit 504 provides a 1-bit jump output corresponding to the input to decoder circuit 504, which is shown in Table 4 below:

[0103] Table 4

[0104] Decoder input Decoder jump output 1000 0 0100 0 0010 0 0001 0 0000 1

[0105] The jump output of decoder circuit 04 and the output of comparator circuit 502e are coupled to the input of OR gate 506, which provides a 1-bit output to combination circuit 508. Therefore, if the output of comparator circuit 502e or the jump output of decoder circuit 504 is equal to 1 (which indicates the jump byte), the output of OR gate 506 is equal to 1, otherwise it is equal to 0.

[0106] The combinational circuit 508 receives the output of the decoder circuit 504 and the output of the OR gate 506 as inputs, and provides an encoded byte length value EBLV at the output terminal 512 corresponding to the determined byte length value for each received input byte.

[0107] In this implementation, the encoded byte length value EBLV is a 3-bit signal, wherein the 2-bit decoder output of decoder circuit 504 forms bits 1 and 0, and the output of OR gate 506 forms bit 2 of the encoded byte length value EBLV. Table 5 below depicts exemplary encoded byte length values ​​of the UTF-8 length decoder circuit 500:

[0108] Table 5

[0109] Encoded byte length value byte length value 000 1 001 2 010 3 011 4 100 S

[0110] In this implementation, a byte length value of 1 is encoded as 000, a byte length value of 2 is encoded as 001, a byte length value of 3 is encoded as 011, a byte length value of 4 is encoded as 011, and a byte length value of S is encoded as 100. Those skilled in the art will understand that other byte length values ​​can be used for encoding.

[0111] Figure 6 Symbols for depicting UTF-8 encoding A diagram illustrating an exemplary byte stream of §@. As described above and as depicted in Table 2, the symbols... The UTF-8 symbol byte length is 4 bytes. The UTF-8 symbol byte length of the symbol § is 3 bytes, the symbol byte length of the symbol § is 2 bytes, and the UTF-8 symbol byte length of the symbol @ is 1 byte. Therefore, the exemplary UTF-8 byte stream includes a total of 4+3+2+1=10 bytes: B0, B1, ..., B9.

[0112] Figure 6 An exemplary UTF-8 byte stream can be used as a... Figure 5 The input of the UTF-8 length decoder circuit 500 is applied. Figure 6 The diagram depicts the corresponding outputs of comparator circuits 502a to 502e, as well as the associated byte length value BLV and encoded byte length value EBLV for each received input byte.

[0113] Specifically, byte B0 (11110000) is a UTF-8 encoded symbol. The first byte. The outputs of comparator circuits 502a to 502d are 0001, corresponding to the associated byte length value BLV = 4. Decoder circuit 504 provides a decoder output of 11 and a jump output of 0. The output of comparator circuit 502e is 0, and the output of OR gate 506 is 0. Therefore, for byte B0, combinational circuit 508 provides the encoded byte length value EBLV = 011.

[0114] Byte B1 (10011111) is a UTF-8 encoded symbol. The second byte. The outputs of comparator circuits 502a to 502d are 0000, corresponding to the associated byte length value BLV = S. Decoder circuit 504 provides a decoder output of 00 and a jump output of 1. The output of comparator circuit 502e is 1, and the output of OR gate 506 is 1. Therefore, for byte B1, combinational circuit 508 provides the encoded byte length value EBLV = 100.

[0115] Byte B2 (10000000) is a UTF-8 encoded symbol. The third byte. The outputs of comparator circuits 502a to 502d are 0000, corresponding to the associated byte length value BLV = S. Decoder circuit 504 provides a decoder output of 00 and a jump output of 1. The output of comparator circuit 502e is 1, and the output of OR gate 506 is 1. Therefore, for byte B2, combinational circuit 508 provides the encoded byte length value EBLV = 100.

[0116] Byte B3 (10011001) is a UTF-8 encoded symbol. The fourth byte. The outputs of comparator circuits 502a to 502d are 0000, corresponding to the associated byte length value BLV = S. Decoder circuit 504 provides a decoder output of 00 and a jump output of 1. The output of comparator circuit 502e is 1, and the output of OR gate 506 is 1. Therefore, for byte B3, combinational circuit 508 provides the encoded byte length value EBLV = 100.

[0117] Byte B4 (11100001) is a UTF-8 encoded symbol. The first byte. The outputs of comparator circuits 502a to 502d are 0010, which corresponds to the associated byte length value BLV = 3. Decoder circuit 504 provides a decoder output of 10 and a jump output of 0. The output of comparator circuit 502e is 0, and the output of OR gate 506 is 0. Therefore, for byte B4, combination circuit 508 provides the encoded byte length value EBLV = 010.

[0118] Byte B5 (10111010) is a UTF-8 encoded symbol. The second byte. The outputs of comparator circuits 502a to 502d are 0000, corresponding to the associated byte length value BLV = S. Decoder circuit 504 provides a decoder output of 00 and a jump output of 1. The output of comparator circuit 502e is 1, and the output of OR gate 506 is 1. Therefore, for byte B5, combination circuit 508 provides the encoded byte length value EBLV = 100.

[0119] Byte B6 (10001010) is a UTF-8 encoded symbol. The third byte. The outputs of comparator circuits 502a to 502d are 0000, corresponding to the associated byte length value BLV = S. Decoder circuit 504 provides a decoder output of 00 and a jump output of 1. The output of comparator circuit 502e is 1, and the output of OR gate 506 is 1. Therefore, for byte B6, combinational circuit 508 provides the encoded byte length value EBLV = 100.

[0120] Byte B7 (11000010) is the first byte of the UTF-8 encoded symbol §. The outputs of comparator circuits 502a to 502d are 0100, corresponding to the associated byte length value BLV = 2. Decoder circuit 504 provides a decoder output of 01 and a jump output of 0. The output of comparator circuit 502e is 0, and the output of OR gate 506 is 0. Therefore, for byte B7, combinational circuit 508 provides the encoded byte length value EBLV = 001.

[0121] Byte B8 (10100111) is the second byte of the UTF-8 encoded symbol §. The outputs of comparator circuits 502a to 502d are 0000, corresponding to the associated byte length value BLV = S. Decoder circuit 504 provides a decoder output of 00 and a jump output of 1. The output of comparator circuit 502e is 1, and the output of OR gate 506 is 1. Therefore, for byte B8, combinational circuit 508 provides the encoded byte length value EBLV = 100.

[0122] Byte B9 (01000000) is the first (and only) byte of the UTF-8 encoded symbol @. The outputs of comparator circuits 502a to 502d are 1000, corresponding to the associated byte length value BLV = 1. Decoder circuit 504 provides a decoder output of 00 and a jump output of 0. The output of comparator circuit 502e is 0, and the output of OR gate 506 is 0. Therefore, for byte B9, combinational circuit 508 provides the encoded byte length value EBLV = 000.

[0123] Therefore, as Figure 6 As depicted, the associated byte length values ​​for bytes B0, B1, B2, B3, B4, B5, B6, B7, B8, and B9 are 4, S, S, S, 3, S, S, 2, S, and 1, respectively. As mentioned above, the jump byte does not include symbol byte length information. The symbol byte length of the encoded symbol can be obtained by filtering the S-byte length values ​​from the stream of byte length values ​​determined by the UTF-8 length decoder circuit 500. Therefore, in this example, the encoded symbol... The UTF-8 symbol byte lengths for § and @ are 4, 3, 2, and 1, respectively.

[0124] Figure 7 A conceptual diagram depicts an embodiment of a technique for filtering S-byte length values ​​from a stream of byte length values ​​determined by the UTF-8 length decoder circuit 500, as shown in the example above. In this embodiment, the technique is a variant that uses the S-byte length value as a parity sorting indicator for the exchange.

[0125] In the example shown, bytes B0 through B9 are displayed at the top line, where their associated byte length values ​​(determined as described above) are, in order: 4S SS 3S S 2S 1. Note the additional byte B with an associated byte length value of S. 10 Appended to the end of the value determined by the UTF-8 length decoder circuit 500, as described below. A series of comparisons are performed on the parity index pairs of adjacent byte length values. In the implementation, for each index pair, if the left byte length value is S and the right byte length value is not S (i.e., 1, 2, 3, or 4), the byte length values ​​are swapped.

[0126] Figure 7 Square brackets are used to indicate the index pairs for each swap iteration. For example, starting with iteration 0 (odd number), the first index pair includes the first byte length value and the second byte length value (4 and S), the second index pair includes the third byte length value and the fourth byte length value (S and S), the third index pair includes the fifth byte length value and the sixth byte length value (3 and S), the fourth index pair includes the seventh byte length value and the eighth byte length value (S and 2), and the fifth index pair includes the ninth and tenth byte length values ​​(S and 1).

[0127] For each index pair where the left byte length is S and the right byte length is not S, the byte length values ​​of such index pairs are swapped. In this example, the swap occurs on the byte length values ​​of the fourth and fifth index pairs. The byte length values ​​of all other index pairs remain unchanged. Therefore, after swapping iteration 0, the byte length values ​​are: 4S S S3S 2S 1S.

[0128] In swap iteration 1 (even), the first index pair includes the second and third byte length values ​​(S and S), the second index pair includes the fourth and fifth byte length values ​​(S and 3), the third index pair includes the sixth and seventh byte length values ​​(S and 2), the fourth index pair includes the eighth and ninth byte length values ​​(S and 1), and the fifth index pair includes the tenth and eleventh byte length values ​​(S and S). As described above, there is an additional byte B with a corresponding byte length value of S. 10 It is appended to the end of the value determined by the UTF-8 length decoder circuit 500. This allows index pairs to be formed for all byte length values ​​even during iteration.

[0129] Next, for each index pair where the left byte length is S and the right byte length is not S, the byte length values ​​of such index pairs are swapped. In this example, the swap occurs on the byte length values ​​of the second, third, and fourth index pairs. The byte length values ​​of all other index pairs remain unchanged. Therefore, after swap iteration 1, the byte length values ​​are: 4S S 3S 2S 1S S.

[0130] In swap iteration 2 (odd), the first index pair includes the first byte length value and the second byte length value (4 and S), the second index pair includes the third byte length value and the fourth byte length value (S and 3), the third index pair includes the fifth byte length value and the sixth byte length value (S and 2), the fourth index pair includes the seventh byte length value and the eighth byte length value (S and 1), and the fifth index pair includes the ninth byte length value and the tenth byte length value (S and S).

[0131] For each index pair where the left byte length is S and the right byte length is not S, the byte length values ​​of such index pairs are swapped. In this example, the swap occurs on the byte length values ​​of the second, third, and fourth index pairs. The byte length values ​​of all other index pairs remain unchanged. Therefore, after swap iteration 2, the byte length values ​​are: 4S 3S 2S 1S SS.

[0132] In swap iteration 3 (even number), the first index pair includes the second byte length value and the third byte length value (S and 3), the second index pair includes the fourth byte length value and the fifth byte length value (S and 2), the third index pair includes the sixth byte length value and the seventh byte length value (S and 1), the fourth index pair includes the eighth byte length value and the ninth byte length value (S and S), and the fifth index pair includes the tenth byte length value and the eleventh byte length value (S and S).

[0133] For each index pair where the left byte length is S and the right byte length is not S, the byte length values ​​of such index pairs are swapped. In this example, the swap occurs on the byte length values ​​of the first, second, and third index pairs. The byte length values ​​of all other index pairs remain unchanged. Therefore, after swapping iteration 3, the byte length values ​​are: 4 3S 2S 1S S SS.

[0134] In swap iteration 4 (odd number), the first index pair includes the first byte length value and the second byte length value (4 and 3), the second index pair includes the third byte length value and the fourth byte length value (S and 2), the third index pair includes the fifth byte length value and the sixth byte length value (S and 1), the fourth index pair includes the seventh byte length value and the eighth byte length value (S and S), and the fifth index pair includes the ninth byte length value and the tenth byte length value (S and S).

[0135] For each index pair where the left byte length is S and the right byte length is not S, the byte length values ​​of such index pairs are swapped. In this example, the swap occurs on the byte length values ​​of the second and third index pairs. The byte length values ​​of all other index pairs remain unchanged. Therefore, after swapping iteration 4, the byte length values ​​are: 4 3 2S1S SSSS.

[0136] In swap iteration 5 (even number), the first index pair includes the second byte length value and the third byte length value (3 and 2), the second index pair includes the fourth byte length value and the fifth byte length value (S and 1), the third index pair includes the sixth byte length value and the seventh byte length value (S and S), the fourth index pair includes the eighth byte length value and the ninth byte length value (S and S), and the fifth index pair includes the tenth byte length value and the eleventh byte length value (S and S).

[0137] For each index pair where the left byte length is S and the right byte length is not S, the byte length values ​​of such index pairs are swapped. In this example, the swap occurs on the byte length value of the second index pair. The byte length values ​​of all other index pairs remain unchanged. Therefore, after swapping iteration 5, the byte length values ​​are: 4 3 2 1S SSS SS.

[0138] Therefore, in this example, after five swap iterations, the S-byte length value is filtered (shifted right) from the stream of byte length values ​​determined by the UTF-8 length decoder circuit 500, where the lengths of the UTF-8 encoded symbol bytes are determined to be 4, 3, 2, and 1, respectively.

[0139] Figure 8AAn implementation scheme of a hardware switching circuit 800 is described, which is configured to perform the switching of byte-length values, as described above. Figure 7 As described in the example. The switching circuit 800 includes a first MUX circuit 802a, a second MUX circuit 802b, a first (left) input terminal, a second (right) input terminal, a first (left) output terminal, and a second (right) output terminal. The first MUX circuit 802a and the second MUX circuit 802b each have a first (left) input terminal, a second (right) input terminal, a control terminal, and an output terminal.

[0140] The left input terminal of the switching circuit 800 is coupled to the left input terminal of the first MUX circuit 802a and the left input terminal of the second MUX circuit 802b, and the right input terminal of the switching circuit 800 is coupled to the right input terminal of the first MUX circuit 802a and the right input terminal of the second MUX circuit 802b. The output terminal of the first MUX circuit 802a is the left output terminal of the switching circuit 800, and the output terminal of the second MUX circuit 802b is the right output terminal of the switching circuit 800.

[0141] The control terminals of the first MUX circuit 802a and the second MUX circuit 802b are coupled to the control signal: Left in Skip (LIS). If the input signal coupled to the left input terminal of the switching circuit 800 is S, the value of LIS is 1. Otherwise, if the input signal coupled to the right input terminal of the switching circuit 800 is not S, the value of LIS is 0.

[0142] In one implementation, if LIS = 0, the left input terminal of the first MUX circuit 802a is coupled to the output terminal of the first MUX circuit 802a, and the right input terminal of the second MUX circuit 802b is coupled to the output terminal of the second MUX circuit 802b. In another implementation, if LIS = 1, the right input terminal of the first MUX circuit 802a is coupled to the output terminal of the first MUX circuit 802a, and the left input terminal of the second MUX circuit 802b is coupled to the output terminal of the second MUX circuit 802b.

[0143] More specifically, if LIS = 0, then the left input terminal of the switching circuit 800 is coupled to the left output terminal of the switching circuit 800, and the right input terminal of the switching circuit 800 is coupled to the right output terminal of the switching circuit 800. Therefore, if the input signal coupled to the left input terminal of the switching circuit 800 is not S, or if the input signals coupled to both the left and right input terminals of the switching circuit 800 are S, then no switching is performed.

[0144] If LIS = 1, then the left input terminal of the switching circuit 800 is coupled to the right output terminal of the switching circuit 800, and the right input terminal of the switching circuit 800 is coupled to the left output terminal of the switching circuit 800. Therefore, if the input signal coupled to the left input terminal of the switching circuit 800 is S, and the input signal coupled to the right input terminal of the switching circuit 800 is not S, then a switch is performed.

[0145] Figure 8B This illustration depicts an exemplary hardware jump filter 810 for filtering S-byte length values ​​from a stream of byte length values ​​determined by a UTF-8 length decoder circuit 500. Specifically, the jump filter 810 is configured to filter S-byte length values ​​from a stream of byte length values ​​to provide a symbol byte length of N symbols. The jump filter 810 includes a switching circuit with an array of [3(N-1)+1] rows and 2N columns: SC00, SC01, SC02, ..., SC3(N-1) 2N-1 Each switching circuit is Figure 8A An example of an exemplary switching circuit 800.

[0146] As described above, each UTF-8 encoded symbol can include up to 4 bytes, and therefore N symbols can include up to 4N bytes. Therefore, the jump filter 810 includes features for receiving bytes B0, B1, B2, ..., B... 4N-1 4N inputs of byte length values: BLV0, BLV1, BLV2, ..., BLV 4N-1 In the implementation scheme, if less than 4N bytes of length value are received, the value of all remaining bytes of length value is assigned as S.

[0147] In the implementation scheme, the switching circuits SC00, SC01, SC02, ..., SC3(N-1) have [3(N-1)+1] rows. 2N-1 Each of them performs an iterative exchange, as described above. Specifically, the switching circuits SC00, SC01, SC02, ..., SC0 2N-1 Execute switch iteration 0, switching circuits SC10, SC11, SC12, ..., SC1 2N-1 Execute switching iteration 1, switching circuits SC20, SC21, SC22, ..., SC2 2N-1 Perform exchange iteration 2, ..., and exchange circuits SC3(N-1)0, SC3(N-1)1, SC3(N-1)2, ..., SC3(N-1) 2N-1 Perform exchange iteration 3(N-1). Exchange circuits SC3(N-1)0, SC3(N-1)1, SC3(N-1)2, ..., SC3(N-1) 2N-1Output the lengths of the symbol bytes in UTF-8 encoding: SBL0, SBL1, SBL2, ..., SBL1. 4N-1 The lengths of the N first symbol bytes are SBL0, SBL1, ..., SBL. N-1 The length of the symbol bytes that make up the N UTF-8 encoded symbols in the byte stream.

[0148] Figure 8C For depiction use Figure 6 Exemplary jump switching circuit 810a for exemplary byte length values Figure 8B An illustration of an embodiment of the jump filter 810, where N = 4. In the case of N = 4, the jump switching circuit 810a includes switching circuits SC00, SC01, SC02, ..., SC97 in an array having [3(N-1)+1] = 10 rows and 2N = 8 columns, wherein each switching circuit is... Figure 8A An example of an exemplary switching circuit 800. A jumper switching circuit 810a is configured to filter S byte length values ​​from a stream of byte length values ​​to provide a symbol byte length of N = 4 symbols.

[0149] As described above, each UTF-8 encoded symbol can include up to 4 bytes, and therefore 4 symbols can include up to 16 bytes. Therefore, the jump filter 810 includes features for receiving bytes B0, B1, B2, ..., B... 15 The 16 inputs are associated byte length values: BLV0, BLV1, BLV2, ..., BLV 15 In implementations that receive values ​​less than 16 bytes in length, all remaining bytes are assigned the value S. Figure 6 In the example, the symbol §@ Encode using the following 10 bytes: B0, B1, ..., B9, and have the following associated byte length values:

[0150] <![CDATA[BLV0]]> <![CDATA[BLV1]]> <![CDATA[BLV2]]> <![CDATA[BLV3]]> <![CDATA[BLV4]]> <![CDATA[BLV5]]> <![CDATA[BLV6]]> <![CDATA[BLV7]]> <![CDATA[BLV8]]> <![CDATA[BLV9]]> 4 S S S 3 S S 2 S 1

[0151] Therefore, in Figure 8C In the middle, the byte length value BLV 10 To BLV 15 All values ​​are assigned to S.

[0152] Each of the 10-row switching circuits SC00, SC01, SC02, ..., SC97 performs a switching iteration as described above. Specifically, switching circuits SC00, SC01, SC02, ..., SC07 perform switching iteration 0; switching circuits SC10, SC11, SC12, ..., SC17 perform switching iteration 1; switching circuits SC20, SC21, SC22, ..., SC27 perform switching iteration 2, ...; and switching circuits SC90, SC91, SC92, ..., SC97 perform switching iteration 9. Switching circuits SC90, SC91, SC92, ..., SC97 output UTF-8 encoded symbol byte lengths SBL0, SBL1, SBL2, ..., SBL... 15 N = 4. The lengths of the first symbol bytes SBL0 = 4, SBL1 = 3, SBL2 = 2, and SBL3 = 1 constitute the symbol. §@ UTF-8 symbol byte length.

[0153] As mentioned above, UTF-16 uses one or two word code units (i.e., two bytes or four bytes) to encode symbols. UTF-16 encoded symbols have the format shown in Table 6 below:

[0154] Table 6

[0155] Symbol word / byte length First character The second character 1 word / 2 bytes Non-(110110yy_yyyyyyy) 2 words / 4 bytes 110110yy_yyyyyyyy 110111yy_yyyyyyyy

[0156] The "y" character is replaced by the bit of the code corresponding to a specific symbol.

[0157] Because each UTF-16 code unit consists of two 8-bit bytes, the order of the bytes may depend on the “byte order” (byte order) of the computer architecture used to encode the data. Individual code units can be stored in “big-endian” (most significant bit first) or “little-endian” (least significant bit first). To help identify the byte order of code units, UTF-16 allows a byte order mark (BOM) indicating the byte order of UTF-16 code units to precede the actual first encoded value.

[0158] For the remainder of this discussion, the byte order of all UTF-16 code units is assumed to be big-endian. Those skilled in the art will understand that the byte order of the UTF-16 encoded symbols should actually be checked, and byte swapping may be necessary to convert the byte order to big-endian for use with the hardware circuitry described herein.

[0159] In addition, those skilled in the art will understand that the hardware circuitry described herein may alternatively be configured according to the byte order of all UTF-16 code units (assuming little-endian), and may require byte swapping to convert the byte order to little-endian for use with such alternative hardware circuitry.

[0160] As shown in Table 6, the symbol byte length of a UTF-16 encoded symbol can be determined from the first 6 bits (15 to 10) of each word, and therefore each word will have an associated symbol byte length. Specifically, if bits 15 to 10 of a word equal 110110, the symbol byte length associated with that word is 4 bytes. Otherwise, the symbol byte length associated with that word is 2 bytes. For a two-word UTF-16 encoded symbol, bits 15 to 10 of the second word equal 110111. In an implementation, the symbol byte length associated with the second word of a two-word UTF-16 encoded symbol is “S” to indicate that the word includes a jump byte. Other values ​​can be used for the symbol byte length associated with words that include jump bytes.

[0161] For example, the symbols @, §, The UTF-16 encoding is shown in Table 7:

[0162] Table 7

[0163]

[0164] For example, as mentioned above, symbols Encoded using 2 words / 4 bytes. Bits 15 to 10 of the first word equal 110110, and therefore the symbol byte associated with this first word is 4 bytes long. Bits 15 to 10 of the second word equal 110111, and therefore the symbol byte associated with this second word is 'S' bytes long. Therefore, this second word includes a jump byte. Symbol Each of the symbols § and @ is encoded using 1 word / 2 bytes. For each of these symbols, bits 15 to 10 of the first word are not equal to 110110, and therefore the symbol byte length associated with that first word is 2 bytes.

[0165] Referring again to Table 6, each word of a UTF-16 encoded symbol is either a first word that includes information that can be used to determine the symbol byte length of the associated encoded symbol, or a second word that does not include such symbol byte length information, such as a jump byte. In an implementation, each word of a UTF-16 encoded symbol has an associated "byte length value," which can be identified based on the presence or absence of a predetermined bit pattern in each word.

[0166] In the implementation, each word of a UTF-16 encoded symbol has an associated byte length value based on the values ​​of bits 15 to 10 of that word, as described above. In the implementation, each first word has an associated byte length value of 2 or 4, and each second word has an associated byte length value of S. Therefore, if a word of a UTF-16 encoded symbol has an associated byte length value of 2 or 4, then the symbol byte length of the encoded symbol is equal to the byte length value associated with the word. If a word of a UTF-16 encoded symbol has an associated byte length value of S, then the word includes a jump byte and does not include the symbol byte length information of the encoded symbol.

[0167] Figure 9 This is a flowchart illustrating an embodiment of a process 900 for determining the byte length value of a UTF-16 encoded symbol stream. In an exemplary embodiment, the aforementioned control circuitry is used to control the memory die 106 ( Figure 1 Procedure 900 is executed on state machine 216. For example, procedure 900 can be executed on state machine 216. Figure 2 ) in the direction of. In another exemplary embodiment, in controller 104 ( Figure 1 The execution process is 900.

[0168] At step 902, words of data are received from the input stream of UTF-16 encoded symbols. For example, the input stream could be from memory die 106 ( Figure 1 (or other input data sources) are streams of data.

[0169] At step 904, it is determined whether bits 15 to 10 of the received word are equal to 110111. If so, the word includes a jump byte, and at step 906, the associated byte length value of the word is set to S. Process 900 then proceeds to step 908 to determine if any additional words exist to be evaluated. If so, process 900 returns to step 902 to receive the next word. If it is determined at step 908 that no additional words exist to be evaluated, process 900 terminates.

[0170] If it is determined at step 904 that bits 15 to 10 of the received word are not equal to 110111, then at step 910, it is determined whether bits 15 to 10 of the received word are equal to 110110. If so, then at step 912, the associated byte length value of the word is set to 4. Then, process 900 proceeds to step 908 to determine if there are any additional words to be evaluated. If so, process 900 returns to step 902 to receive the next word. If it is determined at step 908 that there are no additional words to be evaluated, then process 900 terminates.

[0171] If it is determined at step 910 that bits 15 to 10 of the received word are not equal to 110110, then at step 914, the associated byte length value of the word is set to 2. Then, process 900 proceeds to step 908 to determine if there are any additional words to be evaluated. If so, process 900 returns to step 902 to receive the next word. If it is determined at step 908 that there are no additional words to be evaluated, process 900 terminates. Those skilled in the art will understand that the order of the steps in process 900 can be changed.

[0172] Figure 10 This illustration shows an embodiment of a hardware circuit (UTF-16 length decoder circuit 1000) used to determine the byte length value of a UTF-16 encoded symbol. In this embodiment, the UTF-16 length decoder circuit 1000 may be located in one or more memory dies 106 ( Figure 1 ), controller 104 ( Figure 1 Implemented on ( ) or a combination thereof. In an implementation, the UTF 16 length decoder circuit 1000 may be implemented on the regular expression filter circuit 222 ( ) Figure 2 In other embodiments, the UTF-16 length decoder circuit 1000 may be included in the memory system 100. Figure 1 It can be implemented in or coupled to other hardware circuits of the memory system.

[0173] The UTF-16 length decoder circuit 1000 receives one word (2 bytes) of input data at a time from the input stream at input terminal 1006, determines the associated byte length value for each received input word, and outputs the encoded byte length value EBLV corresponding to the determined byte length value for each received input word at output terminal 1008.

[0174] The UTF-16 length decoder circuit 1000 includes comparator circuits 1002a and 1002b and a combination circuit 1004. Each of the comparator circuits 1002a and 1002b has an input terminal coupled to input terminal 1006 and provides the combination circuit 1004 with a single bit output value (0 or 1) for each received input word. In an embodiment, the comparator circuits 1002a and 1002b determine the byte length value associated with each received input word.

[0175] Specifically, the comparator circuit 1002a compares bits 15 to 10 of the received input word with the 6-bit binary value "110111" (corresponding to...). Figure 9(Step 904 in process 900). If bits 15 to 10 = 110111, then the comparator circuit 1002a provides an output of 1 (which indicates that the associated byte length value of the received input word is S). Otherwise, if bits 15 to 10 ≠ 110111, then the comparator circuit 1002a provides an output of 0.

[0176] Comparator circuit 1002b compares bits 15 to 10 of the received input word with the 6-bit binary value "110110" (corresponding to...). Figure 9 (Step 910 in process 900). If bits 15 to 10 = 110110, then the comparator circuit 1002b provides an output of 1 (indicating that the associated byte length of the received input word is 4 bytes). Otherwise, if bits 15 to 10 ≠ 110110, then the comparator circuit 1002b provides an output of 0 (indicating that the associated byte length of the received input word is 2 bytes).

[0177] The combinational circuit 1004 receives the output of the comparator circuit 1002a, the fixed value 0, and the output of the comparator circuit 1002b as inputs, and provides an encoded byte length value EBLV at the output terminal 1008 corresponding to the determined byte length value associated with each received input word.

[0178] In this implementation, the encoded byte length value EBLV is a 3-bit signal, where 1 bit of the output of comparator circuit 1002a forms bit 2, a fixed value of 0 forms bit 1, and 1 bit of the output of comparator circuit 1002b forms bit 0 of the encoded byte length value EBLV. Table 8 below depicts exemplary encoded byte length values ​​of the UTF-16 length decoder circuit 1000:

[0179] Table 8

[0180] Encoded byte length value byte length value 000 2 001 4 100 S

[0181] In this implementation, a byte length value of 2 is encoded as 000, a byte length value of 4 is encoded as 001, and a byte length value of S is encoded as 100. Those skilled in the art will understand that other byte length values ​​can be used for encoding.

[0182] Figure 11 Symbols for depicting UTF-16 encoding An illustration of an exemplary word flow of §@. As described above and as depicted in Table 7, the symbols... The UTF-16 symbol byte length is 4 bytes. The UTF-16 symbol byte length is 2 bytes, the UTF-16 symbol byte length of the symbol § is 2 bytes, and the UTF-16 symbol byte length of the symbol @ is 2 bytes. Therefore, the exemplary UTF-16 word stream includes a total of 10 bytes / 5 words: W0, W1, W2, W3, W4.

[0183] Figure 11 An exemplary UTF-16 byte stream can be used as a... Figure 10 The input of the UTF-16 length decoder circuit 1000 is applied. Figure 11 The diagram depicts the corresponding outputs of comparator circuits 1002a and 1002b, as well as the associated byte length value and encoded byte length value EBLV for each received input word.

[0184] Specifically, the character W0 (11011000 00111100) is the encoding symbol. The first word. The outputs of comparator circuits 1002a and 1002b are 01, which corresponds to the associated byte length value BLV = 4. Therefore, for word W0, combinational circuit 1004 provides the encoded byte length value EBLV = 001.

[0185] The character W1 (11011100 00000100) is the encoded symbol. The second word. The outputs of comparator circuits 1002a and 1002b are 10, which corresponds to the associated byte length value BLV = S. Therefore, for word W1, combinational circuit 1004 provides the encoded byte length value EBLV = 100.

[0186] The character W2 (00011110 10001010) is the encoded symbol. The first (and only) word. The outputs of comparator circuits 1002a and 1002b are 00, which corresponds to the associated byte length value BLV = 2. Therefore, for word W2, combinational circuit 1004 provides the encoded byte length value EBLV = 000.

[0187] Word W3 (00000000 00111100) is the first (and only) word of the encoded symbol §. The outputs of comparator circuits 1002a and 1002b are 00, which corresponds to the associated byte length value BLV = 2. Therefore, for word W3, combinational circuit 1004 provides the encoded byte length value EBLV = 000.

[0188] Word W4 (00000000 01000000) is the first (and only) word of the encoded symbol @. The outputs of comparator circuits 1002a and 1002b are 00, which corresponds to the associated byte length value BLV = 2. Therefore, for word W4, combinational circuit 1004 provides the encoded byte length value EBLV = 000.

[0189] Therefore, as Figure 11 As depicted, the associated byte length values ​​for words W0, W1, W2, W3, and W4 are 4, S, 2, 2, and 2, respectively. As mentioned above, the jump byte does not include symbol byte length information. The symbol byte length of the encoded symbol can be obtained by filtering the S-byte length value from the stream of byte length values ​​determined by the UTF-16 length decoder circuit 1000. Therefore, in this example, the encoded symbol... The UTF-16 symbol byte lengths for § and @ are 4, 2, 2, and 2, respectively.

[0190] Figure 12A This illustration depicts an exemplary hardware jump filter 1200 for filtering S-byte length values ​​from a stream of byte length values ​​determined by a UTF-16 length decoder circuit 1000. Specifically, the jump filter 1200 is configured to filter S-byte length values ​​from a stream of byte length values ​​to provide a symbol byte length of N symbols. The jump filter 1200 includes a switching circuit with an array of N rows and N columns: SC00, SC01, SC02, ..., SC(N-1). N-1 Each switching circuit is Figure 8A An example of an exemplary switching circuit 800.

[0191] As described above, each UTF-16 encoded symbol can include up to 2 words, and therefore N symbols can include up to 2N words. Therefore, the jump filter 1200 includes features for receiving words W0, W1, W2, ..., W... 2N-1 The associated byte length values ​​BLV0, BLV1, BLV2, ..., BLV 2N-1 The system receives 2N inputs. In the implementation, if less than 2N bytes of length are received, the remaining bytes are assigned the value S.

[0192] In the implementation scheme, the switching circuits SC00, SC01, SC02, ..., SC(N-1) have N rows. N-1 Each of them performs an iterative exchange, as described above. Specifically, the switching circuits SC00, SC01, SC02, ..., SC0 N-1 Execute switch iteration 0, switching circuits SC10, SC11, SC12, ..., SC1 N-1Execute switching iteration 1, switching circuits SC20, SC21, SC22, ..., SC2 N-1 Perform exchange iteration 2, ..., and exchange circuits SC(N-1)0, SC(N-1)1, SC(N-1)2, ..., SC(N-1) N-1 Perform a switching iteration (N-1). The switching circuits are SC(N-1)0, SC(N-1)1, SC(N-1)2, ..., SC(N-1) N-1 Output the lengths of the UTF-16 symbol bytes SBL0, SBL1, SBL2, ..., SBL respectively. 2N-1 The lengths of the N first symbol bytes are SBL0, SBL1, ..., SBL. N-1 The length of the symbol bytes that make up the N UTF-16 encoded symbols in the byte stream.

[0193] Figure 12B For depiction use Figure 11 Exemplary jump switching circuit 1200a for exemplary byte length values Figure 12A An illustration of an embodiment of the jump filter 1200, wherein N=4, is shown. In the case of N=4, the jump switching circuit 1200a includes switching circuits SC00, SC01, SC02, ..., SC33 arranged in an array of N=4 rows and N=4 columns, wherein each switching circuit is... Figure 8A An example of an exemplary switching circuit 800. The jumper switching circuit 1200a is configured to filter S byte length values ​​from a stream of byte length values ​​to provide a symbol byte length of N = 4 symbols.

[0194] As described above, each UTF-16 encoded symbol can include up to two words, and therefore four symbols can include up to eight words. Therefore, the jump filter 1200 includes eight inputs for receiving the associated byte length values ​​BLV0, BLV1, BLV2, ..., BLV7 of words W0, W1, W2, ..., W7, respectively. In an embodiment, if less than eight byte length values ​​are received, the values ​​of all remaining byte length values ​​are assigned to S. Figure 11 In the example, the symbol §@ is encoded using the following 5 characters: W0, W1, ..., W4, and has the following byte length values:

[0195] <![CDATA[BLV0]]> <![CDATA[BLV1]]> <![CDATA[BLV2]]> <![CDATA[BLV3]]> <![CDATA[BLV4]]> 4 S 2 2 2

[0196] Therefore, in Figure 12B In this context, the byte length values ​​BLV5 to BLV7 are assigned as S.

[0197] Each of the four-row switching circuits SC00, SC01, SC02, ..., SC33 performs a switching iteration as described above. Specifically, switching circuits SC00, SC01, SC02, SC02, and SC03 perform switching iteration 0; switching circuits SC10, SC11, SC12, and SC13 perform switching iteration 1; switching circuits SC20, SC21, SC22, and SC23 perform switching iteration 2; and switching circuits SC30, SC31, SC32, and SC33 perform switching iteration 3. Switching circuits SC30, SC31, SC32, and SC33 output UTF-16 symbol byte lengths SBL0, SBL1, SBL2, ..., SBL7, respectively. N = 4 first symbol byte lengths SBL0 = 4, SBL1 = 2, SBL2 = 2, and SBL3 = 2 constitute the symbol... §@ UTF 16 symbol byte length.

[0198] Figure 13A This illustration shows an embodiment of a hardware circuit (UTF-8 / 16 length decoder circuit 1300) used to determine the byte length value of symbols encoded in both UTF-8 and UTF-16. In this embodiment, the UTF-8 / 16 length decoder circuit 1300 includes a first circuit (UTF-8 / 16 high-byte length decoder circuit 1302a) and a second circuit (UTF-8 / 16 low-byte length decoder circuit 1302b).

[0199] In the implementation scheme, the UTF-8 / 16 length decoder circuit 1300 can be located in one or more memory dies 106 ( Figure 1 ), controller 104 ( Figure 1 Implemented on ( ) or a combination thereof. In an implementation, the UTF-8 / 16 length decoder circuit 1300 may be implemented on the regular expression filter circuit 222 ( Figure 2 In other embodiments, the UTF-8 / 16 length decoder circuit 1300 may be included in the memory system 100. Figure 1 It can be implemented in or coupled to other hardware circuits of the memory system.

[0200] In this implementation, the UTF-8 / 16 length decoder circuit 1300 receives one word (2 bytes) of input data at a time from the input stream at the first input terminal 1304, and receives one bit of binary input signal U16 at the second input terminal 1306. Input signal U16 indicates whether the received input data is UTF-16 encoded (e.g., U16 = 1 if the received input data is UTF-16 encoded, and U16 = 0 if the received input data is UTF-8 encoded). Each input data word includes a first byte (bits 15 to 8) (also referred to herein as the "high-order byte") and a second byte (bits 7 to 0) (also referred to herein as the "low-order byte").

[0201] In the implementation, the UTF-8 / 16 high-byte length decoder circuit 1302a determines the associated first byte length value (BLVH) for each received input word and outputs a first encoded byte length value (EBLVH) corresponding to the determined first byte length value at the first output terminal 1308.

[0202] In the implementation, the UTF-8 / 16 low-byte length decoder circuit 1302b determines the associated second byte length value (BLVL) for each received input word and outputs a second encoded byte length value (EBLVL) corresponding to the determined second byte length value at the second output terminal 1308b.

[0203] In one implementation, if the input data is UTF-8 encoded, the first byte length value BLVH is the byte length associated with the received high-order byte, and the second byte length value BLVL is the byte length associated with the received low-order byte. In another implementation, if the input data is UTF-16 encoded, the first byte length value BLVH is the byte length associated with the received input word, and the second byte length value BLVL is always S.

[0204] Figure 13B This is a diagram illustrating an embodiment of the UTF-8 / 16 high-bit-length decoder circuit 1302a. In this embodiment, the UTF-8 / 16 high-bit-length decoder circuit 1302a can be implemented in one or more memory dies 106 ( Figure 1 ), controller 104 ( Figure 1 Implemented on ( ) or a combination thereof. In the implementation, the UTF-8 / 16 high-bit-length decoder circuit 1302a can be implemented on the regular expression filter circuit 222 ( Figure 2 In other embodiments, the UTF-8 / 16 high-bit-length decoder circuit 1302a may be included in the memory system 100. Figure 1The UTF-8 / 16 high-length decoder circuit 1302a is implemented in or coupled to other hardware circuits of the memory system. The UTF-8 / 16 high-length decoder circuit 1302a includes comparator circuits 1310a to 1310h, MUX circuits 1312a to 1312e, decoder circuit 1314a, OR gate 1316a, and combinational circuit 1318a.

[0205] Each of the comparator circuits 1310a to 1310h has an input terminal coupled to the first input terminal 1304 and provides a single bit output value (0 or 1) for each received input word to a corresponding one of the MUX circuits 1312a to 1312e. Each of the MUX circuits 1312a to 1312e has a first (left) input terminal, a second (right) input terminal, a control terminal, and an output terminal.

[0206] The control terminals of MUX circuits 1312a to 1312e are coupled to the input signal U16 at the second input terminal 1306. If U16 = 1 (e.g., the received input data is UTF-16 encoded data), the first (left) input terminal of each of MUX circuits 1312a to 1312e is coupled to the corresponding output terminal of each of MUX circuits 1312a to 1312e. If U16 = 0 (e.g., the received input data is UTF-8 encoded data), the second (right) input terminal of each of MUX circuits 1312a to 1312e is coupled to the corresponding output terminal of each of MUX circuits 1312a to 1312e.

[0207] In the implementation, MUX circuits 1312a to 1312d selectively couple the output signals of comparator circuits 1310a to 1310f to decoder circuit 1314a based on the value of input signal U16, and MUX circuit 1312e selectively couples the output signals of comparator circuits 1310g to 1310h to OR gate 1316a based on the value of input signal U16.

[0208] Specifically, if U16 = 0 (e.g., the received input data is UTF-8 encoded data), then MUX circuits 1312a to 1312d couple the output signals of comparator circuits 1310a, 1310c, 1310d, and 1310f as input signals to decoder circuit 1314a, MUX circuit 1312e couples the output signal of comparator circuit 1310h to OR gate 1316a, and comparator circuits 1310a, 1310c, 1310d, 1310f, and 1310h determine the associated byte length value of the high-order byte of each received input word.

[0209] If U16 = 1 (for example, the received input data is UTF-16 encoded data), then MUX circuits 1312a to 1312d couple the output signals of comparator circuits 1310b and 1310e as input signals to decoder circuit 1314a, MUX circuit 1312e couples the output signal of comparator circuit 1310g to OR gate 1316a, and comparator circuits 1310b, 1310e, and 1310g determine the associated byte length value for each received input word.

[0210] Specifically, the comparator circuit 1310a compares bit 15 of the received input word with a 1-bit binary value "0". If bit 15 = 0, the comparator circuit 1310a provides an output of 1 (indicating that the associated byte length of the high-order byte is 1). Otherwise, if bit 15 ≠ 0, the comparator circuit 1310a provides an output of 0.

[0211] Comparator circuit 1310b compares bits 15 through 10 of the received input word with two 6-bit binary values, “110110” and “110111”. If bits 15 through 10 ≠ 110110 and bits 15 through 10 ≠ 110111, comparator circuit 1310b provides an output of 1 (indicating that the associated byte length of the received input word is 2). Otherwise, if bits 15 through 10 = 110110 or bits 15 through 10 = 110111, comparator circuit 1310b provides an output of 0.

[0212] Comparator circuit 1310c compares bits 15 through 13 of the received input word with the 3-bit binary value "110". If bits 15 through 13 = 110, comparator circuit 1310c provides an output of 1 (indicating that the associated byte length of the high-order byte is 2). Otherwise, if bits 15 through 13 ≠ 110, comparator circuit 1310c provides an output of 0.

[0213] Comparator circuit 1310d compares bits 15 through 12 of the received input word with the 4-bit binary value "1110". If bits 15 through 12 = 1110, comparator circuit 1310d provides an output of 1 (indicating that the associated byte length of the high-order byte is 3). Otherwise, if bits 15 through 12 ≠ 1110, comparator circuit 1310d provides an output of 0.

[0214] Comparator circuit 1310e compares bits 15 through 10 of the received input word with the 6-bit binary value "110110". If bits 15 through 10 = 110110, comparator circuit 1310e provides an output of 1 (indicating that the associated byte length of the received input word is 4). Otherwise, if bits 15 through 10 ≠ 110110, comparator circuit 1310e provides an output of 0.

[0215] Comparator circuit 1310f compares bits 15 through 11 of the received input word with the 5-bit binary value "11110". If bits 15 through 11 = 11110, comparator circuit 1310f provides an output of 1 (indicating that the associated byte length of the high-order byte is 4). Otherwise, if bits 15 through 11 ≠ 11110, comparator circuit 1310f provides an output of 0.

[0216] The comparator circuit 1310g compares bits 15 through 10 of the received input word with the 6-bit binary value "110111". If bits 15 through 10 = 110111, the comparator circuit 1310g provides an output of 1 (indicating the associated byte length value S of the received input word). Otherwise, if bits 15 through 10 ≠ 110111, the comparator circuit 1310g provides an output of 0.

[0217] Comparison circuit 1310h compares bits 15 to 14 of the received input word with the 2-bit binary value "10". If bits 15 to 14 = 10, comparison circuit 1310h provides an output of 1 (indicating that the high-order byte is a jump byte, and the associated byte length is S). Otherwise, if bits 15 to 14 ≠ 10, comparison circuit 1310h provides an output of 0.

[0218] Decoder circuit 1314a receives the 1-bit output of input MUX circuits 1312a to 1312d as a four-bit input, generates a 2-bit decoder output based on the input to decoder circuit 1314a, and also generates a 1-bit jump output based on the input to decoder circuit 1314a. In an embodiment, the outputs of MUX circuits 1312a to 1312d constitute bits 3 to 0 of the input to decoder circuit 1314a, respectively.

[0219] like Figure 13B As depicted, decoder circuit 1314a provides a 2-bit decoder output corresponding to the input to decoder circuit 1314a, which is shown in Table 9 below:

[0220] Table 9

[0221] Decoder input Decoder output byte length value 1000 00 1 0100 01 2 0010 10 3 0001 11 4 0000 00 S

[0222] The last row of Table 9 contains unresolved codes for the case where all comparators in comparator circuits 1310a to 1310d generate zero output. As mentioned above, such unresolved codes are treated as jump bytes, causing these error symbols to be ignored. Figure 13B As depicted, decoder circuit 1314a provides a 1-bit jump output corresponding to the input to decoder circuit 1314a, which is shown in Table 10 below:

[0223] Table 10

[0224] Decoder input Decoder jump output 1000 0 0100 0 0010 0 0001 0 0000 1

[0225] The jump output of decoder circuit 1314a and the output of MUX circuit 1312e are coupled to the input of OR gate 1316a, which provides a 1-bit output to combinational circuit 1318a. Therefore, if the output of MUX circuit 1312e or the jump output of decoder circuit 1314a is equal to 1 (which indicates the jump byte), then the output of OR gate 1316a is equal to 1, otherwise it is equal to 0.

[0226] The combinational circuit 1318 receives the output of the decoder circuit 1314a and the output of the OR gate 1316a as inputs, and provides a first encoded byte length value EBLVH at the first output terminal 1308 corresponding to the determined first byte length value BLVH for each received input word. As described above, for UTF-8 encoded data, the first byte length value BLVH is the byte length value associated with the received high-order byte, while for UTF-16 encoded data, the first byte length value BLVH is the byte length value associated with the received input word.

[0227] In this implementation, the first encoded byte length value EBLVH is a 3-bit signal, wherein the 2-bit decoder output of decoder circuit 1314a forms bits 1 and 0, and the output of OR gate 1316a forms bit 2 of the first encoded byte length value EBLVH. Table 11 below depicts exemplary first encoded byte length values ​​of UTF-8 / 16 high-byte length decoder circuit 1302a:

[0228] Table 11

[0229]

[0230]

[0231] In this implementation, a byte length value of 1 is encoded as 000, a byte length value of 2 is encoded as 001, a byte length value of 3 is encoded as 011, a byte length value of 4 is encoded as 011, and a byte length value of S is encoded as 100. Those skilled in the art will understand that other byte length values ​​can be used for encoding.

[0232] Figure 13C This is a diagram illustrating an embodiment of the UTF-8 / 16 low bit length decoder circuit 1302b. In this embodiment, the UTF-8 / 16 low bit length decoder circuit 1302b can be implemented in one or more memory dies 106 ( Figure 1 ), controller 104 ( Figure 1 Implemented on () or a combination thereof. In the implementation, the UTF-8 / 16 low-bit-length decoder circuit 1302b can be implemented on the regular expression filter circuit 222 ( Figure 2 In other embodiments, the UTF-8 / 16 low-bit length decoder circuit 1302b may be included in the memory system 100. Figure 1 It is implemented in or coupled to other hardware circuits of the memory system. The UTF-8 / 16 low-length decoder circuit 1302b includes comparator circuits 1310i to 1310m, decoder circuit 1314b, OR gate 1316b and combinational circuit 1318b.

[0233] Each of the comparator circuits 1310i to 1310m has an input terminal coupled to the first input terminal 1304 and provides a single bit output value (0 or 1) for each received input word to the decoder circuit 1314b. In an embodiment, the comparator circuits 1310i to 1310m determine the associated byte length value of the least significant byte of each received input word.

[0234] Specifically, the comparator circuit 1310i compares bit 7 of the received input word with the binary value "0". If bit 7 = 0, the comparator circuit 1310i provides an output of 1 (indicating that the associated byte length of the lower-order byte is 1 byte). Otherwise, if bit 7 ≠ 0, the comparator circuit 1310i provides an output of 0.

[0235] Comparator circuit 1310j compares bits 7 through 5 of the received input word with the 3-bit binary value "110". If bits 7 through 5 = 110, comparator circuit 1310j provides an output of 1 (indicating that the associated byte length of the lower-order byte is 2 bytes). Otherwise, if bits 7 through 5 ≠ 110, comparator circuit 1310j provides an output of 0.

[0236] The comparator circuit 1310k compares bits 7 through 4 of the received input byte with the 4-bit binary value "1110". If bits 7 through 4 = 1110, the comparator circuit 1310k provides an output of 1 (indicating that the associated byte length of the lower-order byte is 3 bytes). Otherwise, if bits 7 through 4 ≠ 1110, the comparator circuit 1310k provides an output of 0.

[0237] Comparison circuit 13101 compares bits 7 through 3 of the received input byte with the 5-bit binary value "11110". If bits 7 through 3 = 11110, comparison circuit 13101 provides an output of 1 (indicating that the associated byte length of the lower-order byte is 4 bytes). Otherwise, if bits 7 through 3 ≠ 11110, comparison circuit 13101 provides an output of 0.

[0238] The comparator circuit 1310m compares bits 7 through 6 of the received input byte with the 2-bit binary value "10". If bits 7 through 6 = 10, the comparator circuit 1310m provides an output of 1 (indicating that the lower-order byte is a jump byte, and the associated byte length is S). Otherwise, if bits 7 through 6 ≠ 10, the comparator circuit 1310m provides an output of 0.

[0239] Decoder circuit 1314b receives the 1-bit outputs of comparator circuits 1310i to 1310l as four-bit inputs, generates a 2-bit decoder output based on the inputs to decoder circuit 1314b, and also generates a 1-bit jump output based on the inputs to decoder circuit 1314b. In an embodiment, the outputs of comparator circuits 1310i to 1310l constitute bits 3 to 0 of the inputs to decoder circuit 1314b, respectively.

[0240] like Figure 13C As depicted, decoder circuit 1314b provides a 2-bit decoder output corresponding to the input to decoder circuit 1314b, which is shown in Table 12 below:

[0241] Table 12

[0242] Decoder input Decoder output byte length value 1000 00 1 0100 01 2 0010 10 3 0001 11 4 0000 00 S

[0243] The last row of Table 12 contains unresolved codes for the case where all comparators 1310i to 1310l generate zero output. As mentioned above, such unresolved codes are treated as jump bytes, causing these error symbols to be ignored. Figure 13C As depicted, decoder circuit 1314b provides a 1-bit jump output corresponding to the input to decoder circuit 1314b, which is shown in Table 13 below:

[0244] Table 13

[0245]

[0246]

[0247] The jump output of decoder circuit 1314b, the output of comparator circuit 1310m, and input signal U16 are coupled as inputs to OR gate 1316b, which provides a 1-bit output to combination circuit 1318b. Therefore, if the value of any one of the output of comparator circuit 1310m, jump output of decoder circuit 1314b, or input signal U16 is 1 (which indicates the jump byte), then the output of OR gate 1316b is equal to 1; otherwise, it is equal to 0.

[0248] The combinational circuit 1318b receives the output of the decoder circuit 1314b and the output of the OR gate 1316b as inputs, and provides a second encoded byte length value EBLVL at the second output terminal 1308b corresponding to the determined second byte length value BLVL for each received input word. As described above, for UTF-8 encoded data, the second byte length value BLVL is the associated byte length value of the received low-order byte, while for UTF-16 encoded data, the second byte length value BLVL is always S.

[0249] In this implementation, the second encoded byte length value EBLVL is a 3-bit signal, wherein the 2-bit decoder output of decoder circuit 1314b forms bits 1 and 0, and the output of OR gate 1316b forms bit 2 of the second encoded byte length value EBLVL. Table 14 below depicts exemplary second encoded byte length values ​​of UTF-8 / 16 low-byte length decoder circuit 1302b:

[0250] Table 14

[0251] Low-bit encoded byte length value Low-order byte length value 000 1 001 2 010 3 011 4 100 S

[0252] In this implementation, a byte length value of 1 is encoded as 000, a byte length value of 2 is encoded as 001, a byte length value of 3 is encoded as 011, a byte length value of 4 is encoded as 011, and a byte length value of S is encoded as 100. Those skilled in the art will understand that other byte length values ​​can be used for encoding.

[0253] Figure 14A Symbols for depicting UTF-8 encoding An illustration of an exemplary word stream of §@. As described above and as depicted in Table 2, the symbols... The UTF-8 symbol byte length is 4 bytes. The UTF-8 symbol byte length of the symbol § is 3 bytes, the symbol byte length of the symbol § is 2 bytes, and the UTF-8 symbol byte length of the symbol @ is 1 byte. Therefore, the exemplary UTF-8 word stream includes a total of 10 bytes / 5 words: W0, W1, W2, W3, W4.

[0254] Figure 14A An exemplary UTF-8 word stream can be used as a... Figures 13A to 13C The input of the UTF-8 / 16 length decoder circuit 1300 is applied. The received input data is UTF-8 encoded data, and therefore U16 = 0. (Reference) Figure 13B MUX circuits 1312a to 1312d couple the output signals of comparator circuits 1310a, 1310c, 1310d, and 1310f as input signals to decoder circuit 1314a. MUX circuit 1312e couples the output signal of comparator circuit 1310h to OR gate 1316a. Comparator circuits 1310a, 1310c, 1310d, 1310f, and 1310h determine the associated first byte length value BLVH for each received input word. (Reference) Figure 13C The comparator circuits 1310i to 1310m determine the associated second byte length value (BLVL) for each received input word.

[0255] Figure 14A The diagram depicts the corresponding outputs of the comparator circuits 1310a, 1310c, 1310d, 1310f, and 1310h, as well as the associated first byte length value BLVH and first encoded byte length value EBLVH of the high-order byte of each received input word. Figure 14A The corresponding outputs of the comparator circuits 1310i to 1310m are also shown, as well as the associated second byte length value BLVL and second encoded byte length value EBLVL of the low-order byte of each received input word.

[0256] Specifically, the character W0 (11110000 10011111) includes the encoded symbol. The first two bytes. The high-order byte of word W0 is 11110000. The outputs of comparator circuits 1310a, 1310c, 1310d, 1310f (and their corresponding MUX circuits 1312a to 1312d) are 0001, corresponding to the associated first byte length value BLVH = 4. Decoder circuit 1314a provides a decoder output of 11 and a jump output of 0. The outputs of comparator circuit 1310h (and MUX circuit 1312e) are 0, and the output of OR gate 1316a is 0. Therefore, for word W0, combinational circuit 1318a provides the first encoded byte length value EBLVH = 011.

[0257] The least significant byte of word W0 is 10011111. The outputs of comparator circuits 1310i to 1310l are 0000, corresponding to the associated second byte length value BLVL = S. Decoder circuit 1314b provides a decoder output of 00 and a jump output of 1. The output of comparator circuit 1310m is 1, and the output of OR gate 1316b is 1. Therefore, for word W0, combinational circuit 1318b provides the second encoded byte length value EBLVL = 100.

[0258] The character W1 (10000000 10011001) includes the encoded symbol The second two bytes. The high-order byte of word W1 is 10000000. The outputs of comparator circuits 1310a, 1310c, 1310d, 1310f (and their corresponding MUX circuits 1312a to 1312d) are 0000, corresponding to the associated first byte length value BLVH = S. Decoder circuit 1314a provides a decoder output of 00 and a jump output of 1. The outputs of comparator circuit 1310h (and MUX circuit 1312e) are 1, and the output of OR gate 1316a is 1. Therefore, for word W1, combinational circuit 1318a provides the first encoded byte length value EBLVH = 100.

[0259] The least significant byte of word W1 is 10011001. The outputs of comparator circuits 1310i to 1310l are 0000, corresponding to the associated second byte length value BLVL = S. Decoder circuit 1314b provides a decoder output of 00 and a jump output of 1. The output of comparator circuit 1310m is 1, and the output of OR gate 1316b is 1. Therefore, for word W1, combinational circuit 1318b provides the second encoded byte length value EBLVL = 100.

[0260] The character W2 (11100001 10111010) includes the encoded symbol The first two bytes. The high-order byte of word W2 is 11100001. The outputs of comparator circuits 1310a, 1310c, 1310d, 1310f (and their corresponding MUX circuits 1312a to 1312d) are 0010, corresponding to the associated first byte length value BLVH = 3. Decoder circuit 1314a provides a decoder output of 10 and a jump output of 0. The outputs of comparator circuit 1310h (and MUX circuit 1312e) are 0, and the output of OR gate 1316a is 0. Therefore, for word W2, combinational circuit 1318a provides the first encoded byte length value EBLVH = 010.

[0261] The least significant byte of word W2 is 10111010. The outputs of comparator circuits 1310i to 1310l are 0000, corresponding to the associated second byte length value BLVL = S. Decoder circuit 1314b provides a decoder output of 00 and a jump output of 1. The output of comparator circuit 1310m is 1, and the output of OR gate 1316b is 1. Therefore, for word W2, combinational circuit 1318b provides the second encoded byte length value EBLVL = 100.

[0262] The character W3 (10001010 11000010) includes the encoded symbol The third byte, and the first byte of the encoded symbol §. The high-order byte of word W3 is 10001010. The outputs of comparator circuits 1310a, 1310c, 1310d, 1310f (and the corresponding MUX circuits 1312a to 1312d) are 0000, which corresponds to the associated first byte length value BLVH = S. Decoder circuit 1314a provides a decoder output of 00 and a jump output of 1. The outputs of comparator circuit 1310h (and MUX circuit 1312e) are 1, and the output of OR gate 1316a is 1. Therefore, for word W3, combinational circuit 1318a provides the first encoded byte length value EBLVH = 100.

[0263] The least significant byte of word W3 is 11000010. The outputs of comparator circuits 1310i to 1310l are 0100, corresponding to the associated second byte length value BLVL = 2. Decoder circuit 1314b provides a decoder output of 01 and a jump output of 0. The output of comparator circuit 1310m is 0, and the output of OR gate 1316b is 0. Therefore, for word W3, combinational circuit 1318b provides the second encoded byte length value EBLVL = 001.

[0264] Word W4 (10100111 01000000) includes the second byte of the encoded symbol § and the first byte of the encoded symbol @. The high-order byte of word W4 is 10100111. The outputs of comparator circuits 1310a, 1310c, 1310d, 1310f (and their corresponding MUX circuits 1312a to 1312d) are 0000, corresponding to the associated first byte length value BLVH = S. Decoder circuit 1314a provides a decoder output of 00 and a jump output of 1. The outputs of comparator circuit 1310h (and MUX circuit 1312e) are 1, and the output of OR gate 1316a is 1. Therefore, for word W4, combinational circuit 1318a provides the first encoded byte length value EBLVH = 100.

[0265] The least significant byte of word W4 is 01000000. The outputs of comparator circuits 1310i to 1310l are 1000, corresponding to the associated second byte length value BLVL = 1. Decoder circuit 1314b provides a decoder output of 00 and a jump output of 0. The output of comparator circuit 1310m is 0, and the output of OR gate 1316b is 0. Therefore, for word W4, combinational circuit 1318b provides the second encoded byte length value EBLVL = 000.

[0266] Therefore, as Figure 14A The byte length values ​​associated with words W0, W1, W2, W3, and W4, as described, are 4, S, S, S, 3, S, S, 2, S, and 1, respectively. As mentioned above, the jump bytes do not include the sign byte length information. This can be seen from... Figures 13A to 13C The UTF-8 / 16 length decoder circuit 1300 determines the stream filtering of the byte length value (e.g., using...). Figure 12A The exemplary hardware jump filter 1200 is modified to include two associated byte length values ​​(S byte length values) per word to obtain the symbol byte length of the encoded symbol. Therefore, in this example, the encoded symbol... The UTF-8 symbol byte lengths for § and @ are 4, 3, 2, and 1, respectively.

[0267] Figure 14B Symbols for depicting UTF-16 encoding An illustration of an exemplary word flow of §@. As described above and as depicted in Table 7, the symbols... The UTF-16 symbol byte length is 4 bytes. The UTF-16 symbol byte length of the symbol § is 2 bytes, and the UTF-16 symbol byte length of the symbol @ is 2 bytes. Therefore, the exemplary UTF-16 word stream includes a total of 5 words: W0, W1, W2, W3, and W4.

[0268] Figure 14B An exemplary UTF-16 byte stream can be used as a... Figures 13A to 13C The input of the UTF-8 / 16 length decoder circuit 1300 is applied. The received input data is UTF-16 encoded data, and therefore U16 = 1. (Reference) Figure 13BMUX circuits 1312a and 1312c are coupled to decoder circuit 1314a with a fixed value of 0 as input signal. MUX circuits 1312b and 1312d are coupled to decoder circuit 1314a with the output signals of comparator circuits 1310b and 1310e as input signals, respectively. MUX circuit 1312e is coupled to OR gate 1316a with the output signal of comparator circuit 1310g. Comparator circuits 1310b, 1310e and 1310g determine the associated first byte length value BLVH of each received input word.

[0269] refer to Figure 13C Since U16 = 1, the output of OR gate 1316b is always 1, and the associated second byte length value BLVL = S for each received input word. Therefore, for UTF-16 encoded data, the associated second byte length value BLVL and the second encoded byte length value EBLVL for each received input word can be ignored.

[0270] Figure 14B The diagram depicts the corresponding outputs of the comparator circuits 1310b, 1310e, and 1310g, as well as the associated first byte length value BLVH and second encoded byte length value EBLVH for each received input word.

[0271] Specifically, the character W0 (11011000 00111100) is the encoding symbol. The first word. The outputs of comparator circuits 1310b and 1310e are 01, corresponding to the associated first byte length value BLVH = 4. Decoder circuit 1314a provides a decoder output of 11 and a jump output of 0. The outputs of comparator circuit 1310g (and MUX circuit 1312e) are 0, and the output of OR gate 1316a is 0. Therefore, for word W0, combination circuit 1318a provides the first encoded byte length value EBLVH = 011.

[0272] The character W1 (11011100 00000100) is the encoded symbol. The second word. The outputs of comparator circuits 1310b and 1310e are 00, corresponding to the associated first byte length value BLVH = S. Decoder circuit 1314a provides a decoder output of 00 and a jump output of 1. The outputs of comparator circuit 1310g (and MUX circuit 1312e) are 1, and the output of OR gate 1316a is 1. Therefore, for word W1, combination circuit 1318a provides the first encoded byte length value EBLVH = 100.

[0273] The character W2 (00011110 10001010) is the encoded symbol. The first (and only) word. The outputs of comparator circuits 1310b and 1310e are 10, corresponding to the associated first byte length value BLVH = 2. Decoder circuit 1314a provides a decoder output of 01 and a jump output of 0. The outputs of comparator circuit 1310g (and MUX circuit 1312e) are 0, and the output of OR gate 1316a is 0. Therefore, for word W2, combination circuit 1318a provides the first encoded byte length value EBLVH = 001.

[0274] Word W3 (00000000 00111100) is the first (and only) word of the encoded symbol §. The outputs of comparator circuits 1310b and 1310e are 10, corresponding to the associated first byte length value BLVH = 2. Decoder circuit 1314a provides a decoder output of 01 and a jump output of 0. The outputs of comparator circuit 1310g (and MUX circuit 1312e) are 0, and the output of OR gate 1316a is 0. Therefore, for word W3, combination circuit 1318a provides the first encoded byte length value EBLVH = 001.

[0275] Word W4 (00000000 01000000) is the first (and only) word of the encoded symbol @. The outputs of comparator circuits 1310b and 1310e are 10, corresponding to the associated first byte length value BLVH = 2. Decoder circuit 1314a provides a decoder output of 01 and a jump output of 0. The outputs of comparator circuit 1310g (and MUX circuit 1312e) are 0, and the output of OR gate 1316a is 0. Therefore, for word W4, combination circuit 1318a provides the first encoded byte length value EBLVH = 001.

[0276] Therefore, as Figure 14B The byte length values ​​corresponding to words W0, W1, W2, W3, and W4, as described, are 4, 5, 2, 2, and 2, respectively. As mentioned above, the jump bytes do not include the sign byte length information. This can be seen from... Figures 13A to 13C The UTF-8 / 16 length decoder circuit 1300 determines the stream filtering of the byte length value (e.g., using...). Figure 12A The exemplary hardware jump filter 1200) uses the S-byte length value to obtain the symbol byte length of the encoded symbol. Therefore, in this example, the encoded symbol... The UTF-16 symbol byte lengths for § and @ are 4, 2, 2, and 2, respectively.

[0277] As mentioned above, Figures 13A to 13CAn exemplary UTF-8 / 16 length decoder circuit 1300 receives input data one word at a time from an input stream and determines an associated first byte length value BLVH, a first encoded byte length value EBLVH, an associated second byte length value BLVL, and a second encoded byte length value EBLVL for each received input word. In embodiments, multiple UTF-8 / 16 length decoder circuits 1300 may be combined, with each UTF-8 / 16 length decoder circuit 1300 processing a corresponding word from the input stream. In embodiments, multiple UTF-8 / 16 length decoder circuits 1300 may be combined to determine the associated byte length value of an N-symbol string of data.

[0278] For example, Figure 15A This illustration shows an embodiment of a hardware circuit (N-symbol UTF-8 / 16 length extractor circuit 1500) used to determine the associated byte length value of an N-symbol string of UTF-8 or UTF-16 encoded symbols. In this embodiment, the N-symbol UTF-8 / 16 length extractor circuit 1500 may be located in one or more memory dies 106 ( Figure 1 ), controller 104 ( Figure 1 Implemented on ( ) or a combination thereof. In the implementation, the N-symbol UTF-8 / 16 length extractor circuit 1500 can be implemented on the regular expression filter circuit 222 ( Figure 2 In other embodiments, the N-symbol UTF-8 / 16 length extractor circuit 1500 may be included in the memory system 100. Figure 1 It can be implemented in or coupled to other hardware circuits of the memory system.

[0279] Because each UTF-8 / 16 length decoder circuit 1300 determines the associated byte length value of an input word, and because each UTF-8 or UTF-16 encoded symbol can include up to 2 words, the N-symbol UTF-8 / 16 length extractor circuit 1500 includes 2N length decoder circuits LD0, LD1, LD2, ..., LD0. 2N-2 LD 2N-1 Each circuit is Figures 13A to 13C An example of a UTF-8 / 16 length decoder circuit 1300.

[0280] Each length decoder circuit has LD0, LD1, LD2, ..., LD 2N-2 LD 2N-1 Receive the corresponding input words W0, W1, W2, ..., W from the input stream (not shown). 2N-2 W 2N-1 It also receives input signal U16 (not shown). In the implementation scheme, input words W0, W1, W2, ..., W2N-2 W 2N-1 The input stream consists of consecutive words. For example, the input stream could be from memory die 106 ( Figure 1 (or other input data sources) are streams of data. Figure 15A As depicted, each input word comprises 2 bytes of data. For example, input word W0 includes bytes B0 and B1, input word W1 includes bytes B2 and B3, ..., input word W... 2N-1 Including byte B 4N-2 and B 4N-1 .

[0281] As mentioned above, regarding Figures 13A to 13C The UTF-8 / 16 length decoder circuit 1300, with each length decoder circuit having LD0, LD1, LD2, ..., LD... 2N-2 LD 2N-1 Determine the associated first byte length value BLVH and second byte length value BLVL respectively, and output the corresponding first encoded byte length value EBLVH and second encoded byte length value EBLVL respectively.

[0282] To avoid overcrowding the attached diagram, the associated first encoded byte length value EBLVH and second encoded byte length value EBLVL are placed in... Figure 15A The values ​​in the middle are respectively designated as L x0 and L x1 x = 0, 1, 2, ..., 2N-1. Therefore, the N-symbol UTF-8 / 16 length extractor circuit 1500 receives 2N input words: W0, W1, W2, ..., W 2N-2 W 2N-1 (4N input bytes: B0, B1, B2, ..., B) 4N-2 B 4N-1 And generate 2N pairs of encoded byte length values ​​L respectively. x0 and L x1 ,x=0,1,2,...,2N-1.

[0283] As in Figure 14A and Figure 14B In the example, it can be obtained from the length decoder circuits LD0, LD1, LD2, ..., LD 2N-2 LD 2N-1 Stream filtering with a defined byte length value (e.g., using) Figure 12A An exemplary hardware jump filter 1200) uses S byte length values ​​to obtain the symbol byte length of N symbols. The N filtered first byte length values ​​constitute the symbol byte length of the first N symbols of the input stream.

[0284] Figure 15BThis illustration depicts an exemplary 4-symbol UTF-8 / 16 length extractor circuit 1500a (where N=4 is an embodiment of the N-symbol UTF-8 / 16 length extractor circuit 1500). The 4-symbol UTF-8 / 16 length extractor circuit 1500a includes 2N=8 length decoder circuits LD0, LD1, LD2, ..., LD7. These length decoder circuits respectively receive corresponding input data words W0, W1, W2, ..., W7, and respectively generate multiple pairs of encoded byte length values ​​L. 00 and L 01 L 10 and L 11 L 20 and L 21 ... L 70 and L 71 .

[0285] Stream filtering can be performed by analyzing the byte length values ​​determined by the length decoder circuits LD0, LD1, LD2, ..., LD7 (e.g., using...). Figure 12A An exemplary hardware jump filter (1200) uses the S-byte length value to obtain the symbol byte length of N = 4 encoded symbols. The four filtered first byte length values ​​constitute the symbol byte length of the first four symbols in the input stream.

[0286] As described above, the N-symbol UTF-8 / 16 length extractor circuit 1500 can be used to determine the byte length value of an N-symbol string of UTF-8 or UTF-16 encoded symbols. Because the N-symbol string can appear anywhere in the input data stream, to determine the byte length value of each N-symbol string in the input data, the input to the N-symbol UTF-8 / 16 length extractor circuit 1500 can be shifted by one byte for each N-symbol string. If the input data rate is R bytes per second, the N-symbol UTF-8 / 16 length extractor circuit 1500 will need to operate at the same data rate: R bytes per second. In some systems, such as computing storage devices, the data rate R may be very high (e.g., 50 GB / s). Operating the N-symbol UTF-8 / 16 length extractor circuit 1500 at such a high rate can be very challenging.

[0287] One way to reduce the data rate required by the N-symbol UTF-8 / 16 length extractor circuit 1500 is to implement multiple instances of the N-symbol UTF-8 / 16 length extractor circuit 1500 in parallel, offset by one byte of input data. Since each UTF-8 and UTF-16 symbol can comprise 4 bytes, 4N instances of the N-symbol UTF-8 / 16 length extractor circuit 1500 can be operated in parallel, each instance offset by one byte of input data. Therefore, if the input data rate is R bytes per second, each N-symbol UTF-8 / 16 length extractor circuit 1500 can operate at R / 4N Hz, which significantly simplifies the bandwidth requirements of the N-symbol UTF-8 / 16 length extractor circuit 1500.

[0288] For example, Figure 16 This illustration shows an embodiment of a hardware circuit (N-symbol UTF-8 / 16 length extractor array 1600) used to determine the byte length value of 4N consecutive N symbol strings of UTF-8 or UTF-16 encoded symbols. In this embodiment, the N-symbol UTF-8 / 16 length extractor array 1600 can be located in one or more memory dies 106 (… Figure 1 ), controller 104 ( Figure 1 Implemented on ( ) or a combination thereof. In the implementation, the N-symbol UTF-8 / 16 length extractor array 1600 can be implemented on the regular expression filter circuit 222 ( Figure 2 In other embodiments, the N-symbol UTF-8 / 16 length extractor array 1600 may be included in the memory system 100. Figure 1 It can be implemented in or coupled to other hardware circuits of the memory system.

[0289] The N-symbol UTF-8 / 16 length extractor array 1600 includes 4N UTF-8 / 16 length extractor circuits: LE0, LE1, LE2, ..., LE 4N-1 Each circuit is Figure 15A An example of an N-symbol UTF-8 / 16 length extractor circuit 1500 is provided, with each circuit offset by one byte of input data. For example, the N-symbol UTF-8 / 16 length extractor circuit LE0 receives 4N input bytes B0, B1, B2, ..., B... 4N-2 B 4N-1 And generate 2N pairs of encoded byte length values ​​L(0). 00 and L(0) 11 L(0) 00 and L(0) 11 ... ... L(0) (2N-1)0 and L(0) (2N-1)1The N-symbol UTF-8 / 16 length extractor circuit LE1 receives 4N input bytes B1, B2, B3, ..., B 4N-1 B 4N And generate 2N pairs of encoded byte length values ​​L(1) 00 and L(1) 11 L(1) 00 and L(1) 11 ... ... L(1) (2N-1)0 and L(1) (2N-1)1 Therefore, the N-symbol UTF-8 / 16 length extractor array 1600 determines the byte length value of 4N consecutive N-symbol strings of UTF-8 or UTF-16 encoded symbols.

[0290] Figure 17 This illustration shows an embodiment of a hardware circuit (N-symbol regular expression filter 1700) used to determine the byte length value of a 4N consecutive N-symbol string of symbols encoded in UTF-8 or UTF-16. In this embodiment, the N-symbol regular expression filter 1700 can be located in one or more memory dies 106 ( Figure 1 ), controller 104 ( Figure 1 ) or a combination thereof. In the implementation, the N-symbol regular expression filter 1700 may be implemented on the regular expression filter circuit 222 ( Figure 2 In other embodiments, the N-symbol regular expression filter 1700 may be included in the memory system 100. Figure 1 It can be implemented in or coupled to other hardware circuits of the memory system.

[0291] In one implementation, the N-symbol regular expression filter 1700 includes an N-symbol UTF-8 / 16 length extractor array 1702, an N-symbol jump filter array 1704, an N-symbol trigger comparison array 1706, and an OR gate 1708. In another implementation, the N-symbol regular expression filter 1700 receives input data from an input stream at a first input terminal 1710, receives the symbol byte length of the N-symbol trigger at a second input terminal 1712, and provides a trigger match indication at an output terminal 1714.

[0292] In the implementation plan, the following can be used: Figure 16 The N-symbol UTF-8 / 16 length extractor array 1600 is used to implement the N-symbol UTF-8 / 16 length extractor array 1702. The N-symbol UTF-8 / 16 length extractor array 1702 includes 4N UTF-8 / 16 length extractor circuits: LE0, LE1, LE2, ..., LE 4N-1 Each circuit is Figure 15AAn example of an N-symbol UTF-8 / 16 length extractor circuit 1500, each circuit being coupled to the first input terminal 1710, and each circuit being offset by one byte of input data.

[0293] For example, the N-symbol UTF-8 / 16 length extractor circuit LE0 receives 4N input bytes (0 to 4N-1) at the first input terminal 1710, the N-symbol UTF-8 / 16 length extractor circuit LE1 receives 4N input bytes (1 to 4N) at the first input terminal 1710, ..., the N-symbol UTF-8 / 16 length extractor circuit LE... 4N-2 Receives 4N input bytes (4N-2 to 2(4N-1)-1) and an N-symbol UTF-8 / 16 length extractor circuit LE at the first input terminal 1710. 4N-1 4N input bytes (4N-1 to 2(4N-1)) are received at the first input terminal 1710.

[0294] In the implementation scheme, each N-symbol UTF-8 / 16 long extractor circuit LE0, LE1, ..., LE N-2 LE N-1 Generate 2N pairs of encoded byte length values, such as those mentioned above. Figure 16 The N-symbol UTF-8 / 16 length extractor array 1600 is described. To avoid overcrowding the figures, the N-symbol UTF-8 / 16 length extractor circuitry LE0, LE1, ..., LE2 is shown. 4N-2 LE 4N-1 Each of the following is a single output arrow. Each output arrow represents a 2N pair of encoded byte length values ​​generated by the corresponding length extractor circuit.

[0295] The multiple pairs of encoded byte length values ​​generated by the N-symbol UTF-8 / 16 length extractor array 1702 are provided as input to the N-symbol jump filter array 1704. In the embodiment, the N-symbol jump filter array 1704 includes 4N jump filters: SF0, SF1, ..., SF 4N-2 SF 4N-1 Each jump filter is Figure 12A An instance of the hardware jump filter 1200 is modified to include two associated byte length values ​​per word. In the implementation, each jump filter SF0, SF1, ..., SF... 4N-2 SF 4N-1 The N-symbol UTF-8 / 16 length extractor circuits LE0, LE1, ..., LE1 are respectively used to extract the length of the N-symbol UTF-8 / 16. 4N-2 LE 4N-1 Receive 2N pairs of encoded byte length values. Each jump filter SF0, SF1, ..., SF 4N-2 SF4N-1 The N filtered first byte length values ​​constitute the symbol byte lengths of the N symbols in the associated N symbol strings of a UTF-8 or UTF-16 encoded symbol. Therefore, the N symbol jump filter array 1704 outputs the symbol byte lengths of the N symbols in the 4N consecutive N symbol strings of a UTF-8 or UTF-16 encoded symbol.

[0296] In the implementation plan, each jump filter SF0, SF1, ..., SF... 4N-2 SF 4N-1 The length of the output N sign bytes is provided as input to the N-sign flip-flop comparison array 1706. In this embodiment, the N-sign flip-flop comparison array 1706 includes 4N flip-flop comparison circuits TC0, TC1, ..., TC 4N-2 TC 4N-1 Each TC circuit is coupled to the corresponding jumper filters SF0, SF1, ..., SF2, respectively. 4N-2 SF 4N-1 It receives N symbol byte lengths, and each TC circuit is coupled to the second input terminal 1712 to receive the symbol byte length of the N-symbol flip-flop (referred to herein as the "flip-flop symbol byte length").

[0297] In the implementation scheme, each flip-flop comparison circuit TC0, TC1, ..., TC 4N-2 TC 4N-1 Includes circuitry (e.g., logic gates) to receive signals from the corresponding jump filters SF0, SF1, ..., SF2. 4N-2 SF 4N-1 The length of the N symbol bytes is compared with the length of the flip-flop symbol bytes. In the implementation, if the flip-flop comparison circuits TC0, TC1, ..., TC 4N-2 TC 4N-1 If a match is detected, the flip-flop comparator circuit generates a first 1-bit output value (e.g., 1). Conversely, if the flip-flop comparator circuits TC0, TC1, ..., TC... 4N-2 TC 4N-1 If no match is detected, the flip-flop comparator circuit generates a second 1-bit output value (e.g., 0). Therefore, the N-symbol flip-flop comparator array 1706 outputs a match / no-match indication for 4N consecutive N-symbol strings of UTF-8 or UTF-16 encoded symbols.

[0298] In the implementation scheme, OR gate 1708 receives the flip-flop comparison circuits TC0, TC1, ..., TC 4N-2 TC 4N-1The output has a 1-bit value, and provides a 1-bit output value at output terminal 1714, which indicates whether there is a symbol byte length match in any of the 4N consecutive N symbol strings of UTF-8 or UTF-16 encoded symbols.

[0299] Those skilled in the art will understand that the outputs of the flip-flop comparator circuits TC(0) through TC(4N-1) can be used to determine which byte(s) of the first 4N bytes of the input stream a set of matching N symbols begins with. Furthermore, those skilled in the art will understand that the 4N consecutive bytes in an N-symbol flip-flop represent a worst-case assumption. For flip-flops using mixed symbols (none of which are 4 bytes each), i.e., larger than N-symbol flip-flops (up to 4N), this can be applied to flip-flops consisting only of 1-byte symbols in UTF-8.

[0300] In the implementation, if the output of the N-symbol regular expression filter 1700 is 0, then a string of 4N consecutive N symbols encoded in UTF-8 or UTF-16 cannot match the regular expression, and therefore no memory device (e.g., Figure 1 The memory system 100) and the host device (e.g., Figure 1 Bytes (0 to 4N) are transferred between hosts 102 for regular expression matching.

[0301] In the implementation, if the output of the N-symbol regular expression filter 1700 is 1, then one or more of the 4N consecutive N-symbol strings of UTF-8 or UTF-16 encoded symbols match the regular expression, and therefore can be stored in a memory device (e.g., Figure 1 The memory system 100) and the host device (e.g., Figure 1 One or more of the bytes (0 to 4N) are transmitted between the hosts (102) for use in regular expression matching.

[0302] Without being bound by any particular theory, the hardware regular expression filters described are believed to reduce the amount of data transferred from the memory device to the host device for regular expression matching.

[0303] Without being bound by any particular theory, it is believed that the described hardware regular expression filter (such as the N-symbol regular expression filter 1700) can be a feasible circuit because the described hardware regular expression filter recognizes matching strings but not symbols in the string or symbols in the triggers.

[0304] Without being bound by any particular theory, it is believed that the described hardware regular expression filter (such as the N-symbol regular expression filter 1700) can be a feasible circuit because the described hardware regular expression filter recognizes matching strings without comparing the actual symbols in the string with the actual symbols in the flip-flops.

[0305] One embodiment includes an apparatus comprising a memory system having a memory die, the memory system including circuitry configured to receive a plurality of bytes at a first input terminal, the plurality of bytes comprising an input stream of a plurality of first variable-length coded symbols, and the circuitry being configured to determine a sequence of symbol byte lengths for each of the plurality of first variable-length coded symbols.

[0306] One embodiment includes an apparatus comprising a memory system including a memory die and a regular expression filter circuit. The regular expression filter circuit is configured to identify strings in a data stream read from the memory die that match triggers comprising a plurality of variable-length encoded symbols.

[0307] One embodiment includes a regular expression filter circuit comprising a first length decoder circuit and a second length decoder circuit. The first length decoder circuit is coupled to receive first byte data and second byte data at input terminals, and is configured to determine the symbol byte length of the first byte data. The second length decoder circuit is coupled to receive the first byte data and second byte data at input terminals, and is configured to determine the symbol byte length of the second byte data. The first byte data and the second byte data include variable-length encoded symbols.

[0308] For the purposes of this document, the terms “implementation scheme,” “one implementation scheme,” “some implementation schemes,” or “another implementation scheme” used in the specification may be used to describe different implementation schemes or the same implementation scheme.

[0309] For the purposes of this document, a connection may be a direct connection or an indirect connection (e.g., via one or more other components). In some cases, when an element is mentioned as being connected or coupled to another element, that element may be directly connected to the other element or indirectly connected to the other element via an intermediary element. When an element is mentioned as being directly connected to another element, there is no intermediary element between that element and the other element. If two devices are directly or indirectly connected, the two devices are “communicating”, enabling them to communicate electronic signals between them.

[0310] For the purposes of this document, the term “based on” may be understood as “at least partially based on”.

[0311] For the purposes of this document, the use of numerical terms such as “first” object, “second” object, and “third” object without additional context may not imply an ordering of objects, but may be used for identification purposes to distinguish different objects.

[0312] For the purposes of this document, the term "group" of objects may refer to a "group" of one or more objects.

[0313] The detailed description above has been provided for purposes of illustration and description. It is not intended to be exhaustive or to limit the precise forms disclosed in the invention. Many modifications and variations are possible based on the teachings above. The described embodiments were chosen to best explain the principles of the proposed technology and its practical application, thereby enabling others skilled in the art to best utilize it in various embodiments and various modifications suitable for the specific intended use. The scope of the invention is intended to be defined by the appended claims.

Claims

1. An apparatus comprising: A memory system, the memory system including circuitry, the circuitry being configured to: Receive multiple bytes of an input stream comprising multiple first variable-length encoded symbols at the first input terminal; Determine the sequence of symbol byte lengths for each of the plurality of first variable-length coded symbols; At the second input terminal, a sequence of symbol byte lengths of each of a plurality of second variable-length encoded symbols is received; as well as Determine whether the sequence of symbol byte lengths of the plurality of first variable-length coded symbols matches the sequence of symbol byte lengths of the plurality of second variable-length coded symbols; and in: The memory system is coupled to the host device; and The circuitry is further configured to reduce the amount of data transferred from the memory system to the host device.

2. The apparatus of claim 1, wherein the circuit is further configured to: The second input terminal receives a sequence of N symbol bytes for each of N variable-length coded symbols; and Determine whether any sequence of N symbol byte lengths of the plurality of first variable-length coded symbols matches the sequence of N symbol byte lengths of the N variable-length coded symbols.

3. The apparatus according to claim 1, wherein: The memory system is coupled to the host device; and The memory system is configured to selectively transmit the input stream to the host device based on the determined matching result.

4. The apparatus of claim 1, wherein the circuitry includes a regular expression filter.

5. The apparatus of claim 1, wherein the plurality of second variable-length encoded symbols comprises a trigger for a regular expression filter.

6. The apparatus of claim 1, wherein the first variable-length encoded symbol comprises a Unicode transform format encoding scheme.

7. The apparatus of claim 6, wherein the Unicode transformation format encoding scheme includes either the UTF-8 encoding scheme or the UTF-16 encoding scheme.

8. The apparatus of claim 1, wherein the input stream comprises a data stream read from the memory system.

9. An apparatus comprising: The memory system includes: Memory die; and Regular expression filter circuit, wherein the regular expression filter circuit is configured as follows: Identify a string in a data stream read from the memory die, the string being matched against a trigger comprising a plurality of variable-length coded symbols; and Identify matching strings without identifying actual symbols in the strings or actual symbols in the triggers. The string mentioned above includes multiple variable-length encoded symbols; and in: The memory system is coupled to the host device; and The regular expression filter circuit is further configured to reduce the amount of data transferred from the memory system to the host device.

10. The apparatus of claim 9, wherein the regular expression filter circuit is configured to recognize a matching string without comparing the actual symbol in the string with the actual symbol in the trigger.

11. The apparatus of claim 9, wherein the data stream is read from the memory die at a first rate, and the regular expression filter circuit is configured to operate at a second rate lower than the first rate.

12. The apparatus of claim 9, wherein the string comprises a plurality of symbols encoded using a Unicode transform format encoding scheme.

13. The apparatus of claim 12, wherein the Unicode transformation format encoding scheme includes either the UTF-8 encoding scheme or the UTF-16 encoding scheme.

14. A regular expression filter circuit, comprising: A first length decoder circuit is configured to receive a first byte of data and a second byte of data at an input terminal, and the first length decoder circuit is configured to determine the symbol byte length of the first byte of data. and A second length decoder circuit is configured to receive the first byte of data and the second byte of data at the input terminal, and the second length decoder circuit is configured to determine the symbol byte length of the second byte of data. The first byte of data and the second byte of data include variable-length encoded symbols; and in: The memory system, including the regular expression filter circuit, is coupled to the host device; and The regular expression filter circuit is further configured to reduce the amount of data transferred from the memory system to the host device.

15. The regular expression filter circuit of claim 14, wherein the first byte data and the second byte data include Unicode transformation format encoded symbols.

16. The regular expression filter circuit of claim 14, wherein the first byte data and the second byte data comprise symbols encoded in UTF-8 or UTF-16.

Images