High performance microcode text parser

Abstract

The performance of a text parser implemented using state machines is improved by reducing critical dependency paths. In one aspect, all possible current states for a given text input are read from a state table circuit, then the correct next state and output are selected from among them by an output multiplexer based on the current state, removing the dependency on the current state from the read table and allowing the read to be pipelined. In addition, multiple input units are configured to operate on multiple text characters in parallel, each input unit propagating the output of its state table circuit to the next downstream input unit. Each downstream input unit is configured to use the propagated state to provide the appropriate output to the appropriate multiplexer input. The number of possible output states can be dynamically reduced, reducing the size of the output multiplexer required to select the next state.

Description

Background Technology

[0001] Microcode text parsers allow different text formats to be parsed by microcode programs that describe how to handle different characters. For example, data can be arranged in a text file (such files are often called CSV files, or simply CSV format) with comma-separated values, so that tabular data is stored in plain text, with each row including the same number of fields. Files in CSV format use plain text commas as boundary characters between fields. When reading a CSV file, each character must be checked to determine if it is a comma, and if so, the input of a field begins or ends. Therefore, the operating state of the CSV parser changes over time depending on the characters being read or have been read. Furthermore, the fields in a CSV file may themselves include commas as part of the data for that field, and the parser needs to handle such commas specially to correctly parse the file. For example, consider the following two comma-separated values: abc, "def,hij". The first comma encountered when reading these characters sequentially starting from 'a' will cause the parser to change state (i.e., finish inputting the field 'abc' and then begin inputting the next field). Similarly, when the first "" character is encountered, the second field is referenced and its state changes similarly. However, in this case, the parser changes to text input state so that subsequent characters can be read as is before the next "" character is encountered, thus allowing other special characters (e.g., fields def, hij, etc.) to be included in the field. Similar text parsing requirements are also necessary for parsing other text formats, such as JavaScript Object Notation ("JSON") or Hypertext Markup Language ("HTML").

[0002] A parser can be implemented using a table-based state machine by describing parser state transitions in a table-based format and providing a means to look up the table based on the current input and state to obtain the appropriate output and next state. The performance of state machines is typically limited by the dependency of each new input on previous states. For table-based state machines, this performance limitation is exacerbated by the table lookup in each iteration. Furthermore, this dependency makes parallelization across inputs difficult. Summary of the Invention

[0003] This summary is provided to introduce a selection of concepts in a simplified form, which will be further described in the detailed description below. This summary is not intended to identify key or essential features of the claimed subject matter, nor is it intended to limit the scope of the claimed subject matter.

[0004] This paper describes a method and system for implementing an improved text parser that includes a table-based state machine to reduce critical dependency paths. In one aspect, all possible current states for a given input are read from a state table circuit, and then an output multiplexer selects the correct next state and output from these states based on the current states. This removes any dependencies on the current states from the read table and allows the reads to be pipelined.

[0005] On the other hand, multiple input units are configured to operate in parallel on multiple inputs, with each input unit propagating its current input's state table circuitry to the next downstream input unit. Each downstream input unit is configured to use the propagated state table circuitry to provide a state table circuitry read to the appropriate output multiplexer input.

[0006] On the other hand, embodiments can dynamically reduce the number of possible output states for a given input, thereby reducing the size of the output multiplexer required to select the next state. A further extension dynamically reduces the number of states for a given input.

[0007] The further features and advantages of various examples, as well as their structure and operation, will be described in detail below with reference to the accompanying drawings. It should be noted that the ideas and techniques are not limited to the specific examples described herein. The examples given herein are for illustrative purposes only. Other examples will be apparent to those skilled in the art based on the teachings contained herein. Attached Figure Description

[0008] The accompanying drawings, which are incorporated herein and form a part of the specification, illustrate embodiments of the present application and, together with the specification, further serve to illustrate the principles of the embodiments and enable those skilled in the art to make and use the embodiments.

[0009] Figure 1 An example table-based state machine according to an embodiment is shown.

[0010] Figure 2 An example table-based state machine with reduced iteration intervals is shown according to an embodiment.

[0011] Figure 3 A simplified example implementation of a state machine configured to process inputs in parallel, according to an embodiment, is shown.

[0012] Figure 4 An example state machine configured to process inputs in parallel is shown according to an embodiment, which improves upon this by eliminating its state dependencies. Figure 3 The state machine.

[0013] Figure 5 An example is shown. Figure 4The excerpt from the state machine illustrates aspects of the state propagator's operation.

[0014] Figure 6 A block diagram of an example state propagator according to an embodiment is shown.

[0015] Figure 7 A block diagram of an example CSV file parser according to an embodiment is shown.

[0016] Figure 8 A flowchart is shown according to an embodiment of a method for operating a text parser configured to process input in parallel.

[0017] Figure 9 This is a block diagram of an example computer system that can implement the embodiments.

[0018] The features and advantages of the embodiments will become more apparent when the following detailed description is taken in conjunction with the accompanying drawings, in which the same reference numerals identify corresponding elements throughout. In the drawings, the same reference numerals generally denote the same, functionally similar, and / or structurally similar elements. The first appearance of an element in a graphic is indicated by the leftmost digit of the corresponding reference numeral. Detailed Implementation I. Introduction

[0019] This specification and accompanying drawings disclose one or more embodiments incorporating features of the invention. The scope of the invention is not limited to the disclosed embodiments. The disclosed embodiments are merely illustrative, and modifications of the disclosed embodiments are also included within the scope of the invention. The embodiments of the invention are defined by the appended claims.

[0020] References to "an embodiment," "embodiment," and "example embodiment," etc., in the specification indicate that the described embodiment may include specific features, structures, or characteristics, but each embodiment does not necessarily include specific features, structures, or characteristics. Furthermore, such phrases do not necessarily refer to the same embodiment. Moreover, when a specific feature, structure, or characteristic is described in connection with an embodiment, it is assumed that implementing such a feature, structure, or characteristic in conjunction with other embodiments (whether explicitly described or not) is within the knowledge of those skilled in the art.

[0021] In the discussion, unless otherwise stated, the use of adjectives such as “substantially” and “about” to modify the conditional or relational characteristics of one or more features of embodiments of this disclosure is to be understood as meaning that the condition or characteristic is defined within the operational tolerances acceptable in the embodiments to which it is intended to be applied.

[0022] Numerous exemplary embodiments are described below. It should be noted that any section / section headings provided herein are not intended to be limiting. Embodiments are described in this document, and embodiments of any type may be included in any section / section. Furthermore, embodiments disclosed in any section / section may be combined in any manner with any other embodiments described in the same section / section and / or different sections / sections. II. Example Implementation

[0023] The embodiments disclosed herein provide a high-performance text parser that incorporates a table-based state machine, which implements extensive pipelining to hide latency and allows for high clock frequencies. An overview of parsing comma-separated value (“CSV”) files within a state machine context will now be described.

[0024] You can refer to the following example table to understand the example CSV file: John Doyle 800-555-1212 simple Doyle 987-654-3210 Example table data table

[0025] Below is a sample CSV text file corresponding to the tabular data shown in the table above: Example CSV file

[0026] When the example CSV file shown above is opened for reading, the embodiment of the CSV parser discussed below accepts the text shown above (one character or multiple character blocks at a time) upon input and parses the text to place values ​​into table fields of, for example, the example tabular data table shown above (note that the CR-LF at the end of each line in the example CSV file is implicit and not shown). Several things can be observed by comparing the example CSV file and the corresponding example tabular data table. First, the double quotes (i.e., "characters") appearing in the example CSV file are not displayed in the data fields of the example tabular data table. In this example, double quotes are used to encapsulate data that may include spaces or other special characters (such as commas). Second, consistent with the definition of the CSV file format, commas are used as field separators by which the parser understands the start and end times of fields. Below is a qualitative functional description of the table-based state machine CSV parser.

[0027] In an embodiment, the CSV parser reads one text character at a time from, for example, the example CSV file described above. Each text character may include, for example, one or more Unicode code points. In an embodiment, the first character read from the file forms the beginning of the characters for the first field to be parsed from the file. However, in other embodiments, the CSV file may include a header or control field that includes metadata that can also be parsed and subsequently used as control metadata. For example, such metadata may include information indicating comma-separated values ​​subsequently included in the CSV file, which is encoded according to a Unicode encoding format (e.g., UTF-8 or UTF-16, etc.). For the remainder of the description herein, it is assumed that the disclosed embodiments are operating to parse a CSV file encoded using UTF-8 characters. It should be further assumed that the UTF-8 characters of the CSV text file are decoded into a Unicode code point stream before being parsed by the disclosed embodiments, and details of decoding UTF-8 encoded characters into such code points are omitted.

[0028] Continuing with the example tabular data table and example CSV file shown above, and assuming a Unicode code point stream corresponding to the example CSV file is being provided, the embodiment can transition the state machine to a character checking state, whereby the Unicode code point corresponding to the first character is compared with a double quote. If the character is a double quote, the parser is signaled that each character following the next double quote should be included verbatim in the parsed data field output by the parser. If the character is a comma, the parser is signaled that the end of the current field has been reached, and the set of characters parsed so far includes the entire data field, and should be saved or otherwise output accordingly.

[0029] From the qualitative functional description of the parser embodiments described above, it can be understood that the parser can be implemented as a Mealy-type state machine, whereby the state machine accepts input data in the current state and is caused to generate output data / signals and a new state. For example, Figure 1 An example table-based state machine 100 according to an embodiment is shown. The table-based state machine 100 includes a state table circuit 102. The state table circuit 102 may be implemented in hardware (e.g., a circuit including transistors, logic gates, electrical components, etc.) and is configured to receive / accept code point 104 and the current state 106 (also referred to herein as the "current operating state") as input. The state table circuit 102 is configured to subsequently generate an output 108 and a new state 110 (also referred to herein as the "new operating state"). The table-based state machine includes hardware that implements a programmable state machine by encoding state transitions and outputs through a lookup table. For example, consider the following example state transition table, State Transition Table A: State transition table A State transition table A lists all possible state transitions in this example. That is, state transition table A enumerates a new state (i.e., new state 110) for each possible combination of the input (i.e., code point 104) and the current state (i.e., current state 106). Such a state transition table can be called a dense state transition table. A dense state transition table can be used whenever the total number of states is relatively small. However, in other embodiments, and as described further in detail below, a sparse state transition table can be used.

[0030] State table circuit 102 can be implemented in hardware in various ways known to those skilled in the art to provide a lookup table mechanism in which, given code point 104 and current state 106 as input, state table circuit 102 generates output 108 and a new state 110 as outputs, the output values ​​of which are specified by its state transition table (e.g., state transition table A). For example, state table circuit 102 can be implemented as configurable circuits such as transistors, logic gates, application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), complex programmable logic devices (CPLDs), and / or fabricated directly on silicon or other semiconductor materials using photolithography techniques known in the art. Thus, state table circuit 102 can be implemented using microprogramming techniques also known in the art, and one or more state transition tables associated with state table circuit 102 can also include microprogramming instructions, microcode, one or more addresses for in-memory microcode subroutines, etc. Therefore, state table circuit 102 is not merely an arrangement of data, but exposes a lookup table function, thereby retrieving outputs and new states from corresponding state transition tables stored in memory. For example, referring to state transition table A shown above, when code point 104 is 1 and the current state 106 is 1 and is provided to state table circuit 102 as input, state table circuit 102 will provide 0 for output 108 and provide 0 as output for the new state 110. Note that in the context of the Unicode CSV parser embodiment discussed herein, although inputs such as code point 104 include Unicode code points, such code points are described herein in simplified forms such as 0 or 1.

[0031] The performance of the table-based state machine 100 is limited in at least two ways. First, each input is processed sequentially because a new state depends on previous inputs. Second, the processing time of each iteration of the state machine (i.e., the “iteration interval”) is limited by the relatively slow memory access required to look up an entry in the state table circuitry and output a new state 110. That is, no matter how quickly the next code point 104 is provided to the state machine, the generation of the next output 108 is blocked until the new state 110 is generated and fed back as the current state 106. Furthermore, an increase in the time spent computing the output 108 does not need to affect the iteration time, since computing the output 108 can be done in parallel with starting to process the next code point 104. Based on these facts, it can be understood that the iteration interval of the table-based state machine 100 is determined by the amount of time required to look up and output a new state 110. Based on these observations, we now turn to a description of an embodiment configured to reduce the iteration interval by removing any dependency on the current state or previous inputs from the state table lookup.

[0032] Figure 2 An example table-based state machine 200 with reduced iteration intervals according to an embodiment is shown. The table-based state machine 200 includes state table circuits 202A-202N and a multiplexer (“MUX”) 204. Each state table circuit 202A-202N corresponds to a sub-table of a complete state transition table (e.g., state transition table A as shown above), where each sub-table includes information for only one possible current state value. For example, state table circuit 202A includes state transition information corresponding only to the current state 0, state table module 202B includes state transition information corresponding only to the current state 1, and so on. When a new value for code point 104 is received and provided to each of the state table circuits 202A-202N, each such table looks up and provides the state and output corresponding to code point 104 for each table as input to the MUX 204. Note that in order to... Figure 2 For clarity, the states and outputs of each state table circuit in state table circuits 202A-202N are described as signal lines represented as outputs 206A-206N. However, it should be understood that each of outputs 206A-206N includes two independent signals (i.e., a state and output based on a lookup in the table corresponding to a fixed state of a particular state table circuit 202A-202N, and code point 104). Furthermore, although each state table circuit in state table circuits 202A-202N is depicted as an independent circuit, it should be understood that the embodiments are not limited thereto. For example, state table circuits 202A-202N may include a static random access memory (“SRAM”) containing all state transition tables for each state table circuit 202 to 202N.

[0033] As described above, MUX 204 accepts outputs 206A-206N from state table circuits 202A-202N, and is also configured to receive the current state 106 as an input. MUX 204 (or multiplexer 204), also referred to as a data selector, is a device composed of circuits (e.g., transistors, logic gates such as AND, NAND, OR, XOR, etc.) that selects among several analog or digital input signals and forwards the selected input to a single output line. Furthermore, although MUX 204 is depicted as an N-input MUX, since each output of 206A-206N includes two signals, MUX 204 can also be configured as two independent N-input MUXs. One MUX is used to select the correct outputs 206A-206N based on the current state 106 to route to output 108, and the other MUX is used to select the correct input to route to the new state 110. It should be noted that after each iteration, the current state 106 will be updated to reflect the new state 110.

[0034] like Figure 2 The table-based state machine 200 shown eliminates state table lookups / reads from the dependency path because each table lookup depends only on the current input, and the memory reads corresponding to state table reads can be pipelined. Therefore, the iteration interval of the table-based state machine 200 depends only on the time it takes for the MUX 204 to correctly route the output and state from the state table output 108 and the new state 110, respectively. In the embodiment, and depending on the size of the state machine (i.e., the number of possible states of the machine), the operation of the MUX 204 can be significantly faster than table reads, which, when combined with pipelined reads, allow the embodiment to switch at a very high frequency. The embodiment can implement further performance enhancement techniques described below.

[0035] As mentioned above, the table-based state machine 100 has another drawback: each input is processed sequentially because the new state depends on the previous input. That is, because each input depends on the new state generated by the previous input, there is a dependency that prevents full parallelization. The above embodiment reduces the dependency on the current state of the most expensive part of the lookup process (i.e., state table lookup), but this state independence is not necessarily achieved when used as a parallel input state machine.

[0036] For example, consider Figure 3This illustrates a simplified implementation of a state machine 300 configured to process inputs in parallel according to an embodiment. State machine 300 includes four instances of a table-based state machine 200, each instance including a corresponding instance of state table circuits 202A-202N. Each instance of the table-based state machine 200 receives a corresponding input from inputs 302-308 (represented as code points [0]-[3], respectively) and generates a corresponding output from outputs 310-316 (represented as outputs [0]-[3], respectively). Furthermore, each new state from new states 318-324 is fed forward to the MUX 204 of the next instance of the table-based state machine 200, and the new state 324 is ultimately fed back and becomes the current state 326 of the MUX 204 controlling the first instance of the table-based state machine 200.

[0037] exist Figure 3 It is evident that a simple implementation of state machine 300 comprises a chain of state dependencies, as indicated by the thick dashed lines representing the new states 318-322 fed forward and the new state 324 fed back. That is, each of the outputs [1] 310, [2] 312, and [3] 314 is not generated until the new state of the previous stage (e.g., each of the new states in 318-322) is computed, because such a new state drives the corresponding MUX 204 at each stage. In other words, each of the outputs 310-316 becomes valid only after the delay imposed by the delay of its corresponding MUX 204, thus obtaining the four outputs including outputs 310-316 requires a minimum of four delays of the MUX 204. Unfortunately, due to this dependency, this MUX operation cannot be piped. In the embodiment, this dependency is removed to achieve a full pipeline that generates multiple outputs per clock cycle.

[0038] For example, consider Figure 4 It illustrates an example state machine 400 configured to process inputs in parallel according to an embodiment, and improves upon it by eliminating its state dependencies. Figure 3 State machine 300. State machine 400 includes a table lookup stage 428, a state propagation stage 430, and an output selection stage 432. Table lookup stage 428 includes state table circuits 414-420. State propagation stage 430 includes state propagators 408-412, and output selection stage 432 includes MUX 420-426. Each state table circuit in state table circuits 414-420 is connected to... Figure 3 The state table circuits 202A-202N of the table-based state machine 200 shown correspond to those circuits. Each MUX in MUX 420-426 is similarly associated with... Figure 3 The example shown corresponds to the MUX 204 of the table-based state machine 200. Based on the following... Figure 4The discussion of state machine 400, other structural and operational embodiments will be obvious to those skilled in the art.

[0039] like Figure 4 As shown, state table circuits 414-420 each receive one corresponding input from inputs [0]-[3] 302-308 and generate the corresponding output state. State table circuit 414 generates the first output state 402 received by state propagator 408 and MUX 420. Each state table circuit in state table circuits 416-420 generates the corresponding output state received by one of the state propagators 408-412. State propagator 408 generates the output state 404 received by MUX 422 and state propagator 410. State propagator 410 generates the output state 406 received by MUX 424 and state propagator 412. State propagator 412 generates the output state 434 received by MUX 426. MUX 420 receives the output state 402 and the current state 326, and generates output [0] 310 and the new state 318. MUX 422 receives output state 404 and current state 326, and generates output [1] 312 and new state 320. MUX 424 receives output state 406 and current state 326, and generates output [2] 314 and new state 322. MUX 426 receives output state 434 and current state 326, and generates output [3] 316 and new state 324.

[0040] The state machine 400 uses the state transition outputs from each set of state table circuits to determine the state transition table for later inputs, thereby eliminating, for example, Figure 3 The state machine 300 shown has inherent state dependencies. The process of determining later state transitions can be described as state tracking, which can be understood through the following example.

[0041] Assume that code point [0]302 and code point [1]304 are equal to arbitrary values ​​x and y, respectively, and that these values ​​and the state table lookup for each possible current state are shown in Table B: 0 1 2 1 3 3 2 2 0 3 0 1 Table B

[0042] Each entry in Table B indicates, for example, the new state indicated by the corresponding values ​​of code point [0] 302 and code point [1] 304 for each possible current state, as provided by the state table circuit 414. The idea behind state tracking is to determine what the current state corresponding to code point [1] 304 will be given a specific current state corresponding to code point [0] 302 (i.e., the new state generated in response to code point [0]). If the current state to code point [0] is 0, the new state of code point [1] 302 can be found by tracking table B.

[0043] For example, suppose the current state corresponding to code point [0] is 0. The table above shows that for the current input at code point [0] and the current state 0, the new state will be 1. Next, given the current state 1 304 corresponding to code point [1] (i.e., the new state generated from code point [0] 302 and its current state 0), the table shows that the new state of code point [1] is 3. The table below shows a tracking and resulting state transition table for each possible current state given specific values ​​of code point [0] 302 and code point [1] 304. 0 1 0→1→3＝3 1 3 1→3→1＝1 2 2 2→2→0＝0 3 0 3→0→2＝2 Tracking State Transition Table C For example, Table C above shows that for the current value and current state of code point [0]302 and code point [1]304, which is 2, the appropriate state corresponding to code point [1]304 is 0. The operation of state machine 400 and state propagator 408 will now be described in more detail with reference to the state propagation table above.

[0044] State machine 400 is very similar to state machine 300, but they differ in several key aspects. First, referencing... Figure 2 and Figure 3 As described above, each output in outputs 206A-206N includes both possible states and possible outputs that will subsequently be selected by the corresponding MUX 204. For clarity, state machine 400 only describes the states provided by a given set of state tables and omits the outputs. However, it should be understood that, as described above with respect to table-based state machine 200, state table circuits 414-420 provide states and outputs to MUXs 420-426. Second, state machine 400 includes state propagators 408-412, the operation of which will be described in further detail below. Third, each MUX in MUXs 420-426 corresponds to... Figure 3 Each MUX 204 shown is configured to be controlled by the same signal. That is, each of the MUXs 420-426 simultaneously generates an output for its corresponding input based on the current state 326. Therefore, each output of outputs 310-316 becomes active at the same time (i.e., in the same clock cycle), and Figure 3 The aforementioned state dependencies present in state machine 300 are removed. State dependencies are removed by using state propagators 408-412. Although there are dependency chains that propagate logic through states (e.g., a series of logic gates, flip-flops, transistors, etc.), the logic itself does not depend on anything other than the current state and inputs, and therefore can be fully piped.

[0045] For example, such as Figure 4As shown in state machine 400, state table circuits 416-420 operate in parallel to perform their respective table lookups simultaneously in a pipeline stage denoted as table lookup stage 428. Similarly, state propagation, as described below, is performed in a pipeline stage denoted as state propagation stage 430. Finally, in output selection stage 432, the final output and the state corresponding to each input are simultaneously selected by the respective MUXs in MUXs 420-426. Furthermore, although the layout area and pipeline depth of state machine 400 are increased, the additional time to fill the pipeline is negligible, and thereafter state machine 400 can deliver multiple outputs per clock cycle at a clock cycle shorter than the sum of the computational delays of each stage (i.e., the clock cycle only needs to be as long as the time required to complete the slowest stage).

[0046] Figure 5 An example is shown. Figure 4 An excerpt of state machine 400, portion 500, illustrates various aspects of the operation of state propagator 408. Portion 500 includes state table circuitry 414 and 416, MUX 420, MUX 422, and state propagator 408. (As...) Figure 5 Each of the state table circuits 414 and 416 shown includes a corresponding example of circuits 202A-202D with a state table corresponding to each possible state (as further described above). Based on the following regarding... Figure 5 The discussion of part 500 of the state machine 400 shown herein, as well as other structural and operational embodiments, will be obvious to those skilled in the art.

[0047] As mentioned above, Figure 5 A state machine 400 in the context of a state tracking example is shown, along with its corresponding table B and tracking state transition table C. Specifically, for illustrative purposes, it is assumed that code point [0] 302 and code point [1] 304 have values ​​x and y, respectively. Furthermore, for each possible state, the state output from state table circuit 414 corresponds to the value of the given code point [0] 302 = x. Similarly, the state output from state table circuit 416 corresponds to the value of the given code point [1] 304 = y for each possible state.

[0048] State propagator 408 is configured to remap the state outputs of a given set of state table circuits to interpret the tracking states reflected in the tracking state transition table C as shown above. For example, assume the current state 326 = 0. The tracking state transition table C indicates that the next state 320 is expected to have a value of 3. Therefore, state propagator 408 is configured to route the correct next state output from state table circuit 416 to input 0 of MUX 422 (input 0 is appropriate since the current state 326 = 0). More specifically, and according to the example state table circuit described above in the context of this example, state table circuit 416 indicates the state transitions indicated in Table B above, which indicate that for the value y at code point [1] 304, the next state is 2, 3, 0, and 1 of each of the current states 0, 1, 2, and 3. Recalling that MUX 422 selects a signal line with a value of 3 at input 0 (because the current state is 0), state propagator 408 selects the next state from state table circuit 416 corresponding to the current state = 1, since the next state is equal to 3.

[0049] Continuing this example, assume the current state 326 is 3. According to the tracking state transition table C shown above, when the current state 326 is 3, the next state 320 selected by the MUX 422 is 2. Therefore, the state propagator 408 selects the next state corresponding to the current state = 0 in the state table circuit 416, because this next state equals 2. Through a similar process, it can be seen that in this example, the state propagator 408 routes the outputs corresponding to the current states 2 and 3 of the state table circuit 416 to inputs 2 and 1 of the MUX422, respectively.

[0050] Considering these aspects, it can be understood that the next state 402 output from the state table circuit 414 provides the precise mapping described above. That is, in this example, the next state 402 = [1, 3, 2, 0] can be directly used by the state propagator 408 to route the correct next state to the correct input of the MUX 422, because the value of the next state 402 corresponds one-to-one with the inputs 0 to 3 on the MUX 422, and such a value indicates which of the outputs 0 to 3 of the state table circuit 416 should be selected. In other words, the state value of the next state 402 can be used to index the next state output of the state table circuit 416.

[0051] For example, the value of the first element of the next state 402 (i.e., value 1) corresponds to input 0 of the MUX 422 and indicates that the output of the state table circuit 416, whose current state is 1, is routed by the state propagator 408 to input 0. Similarly, the second element of the next state 402 (i.e., value 3) corresponds to input 1 of the MUX 422, which receives the value of the state table circuit, whose current state is 3, from the state table circuit 416, etc. More simply, the value of the next state 402 indicates which of the next states output by the state propagator 408 from the state table circuit 416 should be routed to each of the inputs 0 to 3 of the MUX 422. The state propagator 408 can perform this routing in various ways. For example, the state propagator 408 can be implemented as a crossbar switch, or as will be described now. Figure 6 The MUX shown.

[0052] Figure 6 A block diagram of an example system 600 for a state propagator 408 according to an embodiment is shown. Figure 6 As shown, system 600 includes a state propagator 408 and a multiplexer 422. Furthermore, state propagator 408 includes first MUX 602 through fourth MUX 608, each MUX accepting a new state from state table circuit 416 as a multiplexed input. Each of MUX 602-608 also receives a new state from the output of state table circuit 414 and is configured to switch the multiplexed input from state table circuit 416 to the MUX output. Such outputs are then delivered to, for example, MUX 422 for ultimately selecting and outputting the next state 320 based on the current state as described above.

[0053] Although the above figures and corresponding descriptions illustrate and describe the embodiments from the perspective of state output, those skilled in the art will understand that these principles are equally applicable to routing and selecting appropriate output values ​​based on state table circuitry and state propagation logic. For example, although the state propagator 408 and MUX 422 in Figure 6 The diagrams and descriptions shown above are applicable to other state propagators and MUXs disclosed herein.

[0054] Implementations of text parsers can be carried out in various ways. For example, Figure 7 A block diagram 700 of an example CSV file parser 702 according to an embodiment is shown. The CSV file parser 702 includes an instance of a Unicode parser 706 and a state machine 400. Based on the following... Figure 7 The discussion of the CSV file parser 702 shown, other structural and operational embodiments will be obvious to those skilled in the art.

[0055] CSV file parser 702 is configured to accept a bitstream / bytestream 708 as input, comprising a stream of bits or 8-bit bytes read from a Unicode CSV file 704. For the purposes of this description, it is assumed that the Unicode CSV file 704 is encoded in UTF-8 as described above. In an embodiment, Unicode parser 706 is configured to accept eight-bit bytes from bitstream / bytestream 708 and output Unicode code points [0] 302 to code point [3] 308 from them according to Unicode decoding methods known to those skilled in the art.

[0056] The four code points, including code point [0] 302 to code point [3] 308, are provided in parallel to the state machine 400, which is configured as described above to process these code points in parallel and output table data 710.

[0057] Other operational embodiments will be apparent to those skilled in the art. It should also be noted that the foregoing general description of the operation of the CSV file parser 702 is for illustrative purposes only, and embodiments of the CSV file parser 702 may include different hardware and / or software and may operate in a different manner than described above. Indeed, as stated above, the use of UTF-8 is by no means limiting, and embodiments can also be implemented to handle other text encoding formats, whether encoded in Unicode or any other character encoding. Similarly, although described from the perspective of parsing CSV files, those skilled in the art will understand that embodiments are not limited thereto and can be readily extended to parse text formatted according to other conventions or standards. For example, JSON and HTML (as described above) or Extensible Markup Language (“XML”) all have parsing requirements that can be at least partially met by the embodiments described herein. For example, JSON has more delimiters, and correctly parsing nested delimiters may require a state machine with too many states, thus efficient implementation may be challenging. XML is even more complex because XML delimiters can include any valid start and end tags (and these tags can have arbitrary names). However, embodiments can be used to perform top-level parsing before passing the results to a parser of a different embodiment or, if necessary, to a more complex parser.

[0058] For example, a JSON file parser can be based on, such as Figure 7 Instead of the tabular data 710 output by state machine 400, the CSV file parser 702 shown can provide one or more outputs of state machine 400 (e.g., tagged text or some other intermediate parsing result) to a second parser or parser level (e.g., an instance of state machine 400) to perform additional parsing of the intermediate results.

[0059] In the embodiments, it can be used in various ways. Figure 2 Status table circuits 202A-202N and Figure 4 A state machine 400 is used to process the characters of the stream in parallel. For example, Figure 8 A flowchart 800 is shown, according to an embodiment, of a method for operating a text parser configured to process input in parallel. (Continuing to refer to...) Figure 2 and Figure 4 Flowchart 800 is described. However, based on the following discussion of flowchart 800, other structural and operational embodiments will be apparent to those skilled in the art.

[0060] Flowchart 800 begins at step 802. In step 802, a stream of text characters, including the first character and the second character, is received sequentially. For example, refer to... Figure 7 The CSV file parser 702 and the Unicode parser 706 are configured to accept a byte stream 708 including a Unicode CSV file 704 and parse the byte stream to generate a text character stream including Unicode code points [0-3] 302-308, which is then provided to the state machine 400. Flowchart 800 continues at step 804.

[0061] In step 804, a first state table circuit output is generated based on the first character and a set of predetermined states corresponding to the predetermined states, wherein the first state table circuit output corresponds to a first set of state table circuits and collectively includes the first set of state table circuit outputs. For example, refer to... Figure 4 The state machine 400 and state table circuits 414-420 each include a set of state tables, where each state table corresponds to a state in which the state machine 400 can operate. For example, Figure 2 State table circuits 202A-202N correspond to states 0-N (i.e., predetermined states) of state machine 400, respectively. As described above, the output and next state of state machine 400 depend on the input and current state of the state machine. By maintaining a lookup table for each of the N possible states of state machine 400 (i.e., instances of which are incorporated into each state table circuit of state table circuits 414-420 in the form of state table circuits 202A-202N), the table lookup no longer depends on the current state, because all possible output and next state values ​​are looked up simultaneously. The output of instances of state table circuits 202A-202N includes "first state table circuit output [.]". Although Figure 4 The state machine 400 is shown generating states 402-406 and state 434, and as described above, it should be understood that the state table circuits 414-420 not only output possible next states, but also possible outputs, for example, via... Figure 4 The MUX 420 can be selected later.

[0062] Figure 8Flowchart 800 continues in step 806. In step 806, a second state table circuit output is generated based on the second character and the corresponding predetermined state in a set of predetermined states, wherein the second state table circuit output corresponds to the second set of state table circuits and collectively includes the second set of state table circuit outputs. For example, and continuing to refer to... Figure 4 The state machine 400, the second state table circuit output, is, for example, the output generated by any one of the state table circuits 416, 418, or 420. For example, suppose the second input includes input [1] 304. In this case, the second state table circuit output is those outputs generated by the state table circuit 416. Figure 8 Flowchart 800 continues in step 808.

[0063] In step 808, a first text parser output is selected from the outputs of the first set of state table circuits based on the current state of the text parser. For example, continue to refer to Figure 4 The state machine 400 and MUX 420 receive possible next states and possible outputs from the state table circuit 414, and select appropriate states and outputs based on the current state 326, where the first state machine output corresponds to output [0] 310.

[0064] Figure 8 Flowchart 800 continues in step 810. In step 810, based on the outputs of the first set of state table circuits, it is selected which output from the corresponding output of the second set of state table circuits will be routed as a set of state propagator outputs. For example, continue to refer to Figure 4 The state machine 400 and state propagator 408 are configured to receive the output of the state table circuit 416 and select an appropriate set of state propagator outputs based on state 402 to route to the MUX 422. For example, and referring to... Figure 6 The state propagator 408 may include Figure 6 The MUXs 602-608 are configured such that each MUX receives a corresponding state / output from the possible states / outputs from the state table circuit 414, and then uses these state table circuits 416 to select the appropriate state of the new state [0-3] to route to the MUX 422, as shown. Figure 6 As shown. This operation of state propagator 408 is used to perform the state tracking operation described in more detail above, thereby removing any dependency of state machine 400 on the current state of state machine 400.

[0065] Figure 8 Flowchart 800 ends at step 812. In step 812, a second text parser output is selected from a set of state propagator outputs based on the current state. For example, continue to refer to... Figure 4The state machine 400, wherein the output of the state propagator 408 selected from the output of the state table circuit 416 is received by the MUX 422, and then according to the selection signal provided to the MUX 422, the corresponding output and the appropriate output in the new state corresponding to the input are selected respectively (i.e., the current state 326 of the state machine 400, including the feedback new state 324 of the output of the MUX 426).

[0066] In the preceding discussion of steps 804-812 of flowchart 800, it should be understood that sometimes these steps can be performed in a different order or even simultaneously with other steps. For example, the selection of step 808 can be performed after the selection of steps 810 and / or 812, or can be performed at least partially in parallel. Similarly, it should be understood that although flowchart 800 describes a method of operating state machine 400 in parallel processing of only two inputs (i.e., the first input and the second input), it is understood that... Figure 8 The method described in flowchart 800 can be extended to handle any number of inputs. For example, such as Figure 4 As shown, state machine 400 can be configured to process four inputs in parallel (i.e., inputs [0]-inputs [3] 302-308). Other embodiments of operation will be apparent to those skilled in the art. It should also be noted that the foregoing general description of the operation of state machine 400 is for illustrative purposes only, and embodiments of state machine 400 may include different hardware and / or software and may operate in a different manner than described above.

[0067] Similarly, it is understood that the maximum clock frequency is limited by the speed of MUX operation, which in turn is limited by the size of the MUX. As mentioned above, the size of the MUX is determined one-to-one by the number of states in the machine. However, the embodiments described herein can employ further optimization techniques to keep the size of the MUX manageable for state machines with a large number of possible states. More specifically, and as stated above, when the number of unique transitions for a given input is small for a particular application, the embodiments can use sparse state transition tables. That is, the embodiments can store state transitions sparsely, thereby limiting the required size of the MUX.

[0068] For example, consider the following partial state transition table, dense state transition table D: Dense state transition table D

[0069] The dense state transition table D is "dense" because, as mentioned above, it enumerates the 'next state' for every possible combination of input and current state. However, note that while there are 4 possible current states, there are only two unique state transitions. That is, for any given state and input, the 'next state' has only two values. For input 'A', regardless of the current state, the 'next state' can only be 0 or 1. Similarly, for input 'B', regardless of the current state, the 'next state' can only be 1 or 2. Therefore, the dense table can be simplified to obtain the sparse state transition table E: Sparse state transition table E

[0070] The sparse state transition table E shown above is a "sparse" transition table because it explicitly enumerates the 'next state' for each unique state transition and at least one 'next state' common to multiple non-unique state transitions. Specifically, the sparse state transition table E only enumerates the 'next state' 1 that occurs uniquely for input A and the current state 3. Given the same input A and any other state, the next state is always 0. Therefore, such as Figure 4 A MUX like the MUX 420 shown only needs to multiplex the inputs between two possible next states. In other words, the state table circuit 414 in the diagram, which implements or uses a state transition table (such as a sparse state transition table E), has only two state outputs, 1 or 0, where the former is selected by the MUX 420 for the current state 3, and has a default output of 0 for any other current state. Therefore, the MUX 420 only needs to switch between two inputs instead of four. In this example, the overall reduction is small compared to the original values ​​shown above. However, it should be understood that these tables are merely exemplary, and the reduction may be more significant for larger tables.

[0071] In this example, the state table circuit (e.g., such as...) Figure 4 The outputs of the state table circuit 414 shown are now paired (current state, next state). To multiplex between next states, the actual current state must be compared with these possible current states to determine which next state to use. If it does not match the specified current state, a default transition is used.

[0072] In the case of processing multiple inputs in parallel, for example using Figure 4The state machine 400 shown can sometimes be further sparsified in the number of effective state transitions because, for a particular input considered together, only a particular state may be possible. For example, using the dense state transition table D shown above, consider input A, followed by B. Input A can only lead to a next state of 1 or 0. Therefore, the next state of input B can only be 1.

[0073] In the examples above, we assumed that all possible input states are valid. However, this is not always the case. For some state machines, certain combinations of inputs and the current state are invalid. For example, consider this new transition table, E, with invalid inputs: State transition table E with invalid input The state transition table E with invalid inputs can be sparsified similarly to produce the following sparse state transition table F with invalid inputs: Sparse state transition table F with invalid input

[0074] In the case of a sparse state transition table F with invalid inputs, dynamic information can be used to reduce the number of possible initial states in the input sequence. Considering the inputs to A and B, the only possible initial states for the input to B are 0 and 3. If we observe the possible next states of A, the only valid state is 0, which corresponds to the current state 1. In this case, the state transition (3,1) corresponding to input A can be removed from the state transition table.

[0075] At runtime, the implementation can use the two techniques described above to reduce the number of sparse transitions. Further improvements can be made by analyzing the state machine during typical workloads to determine the choice of hardware MUX size based on the average or typical number of transitions after sparsification. Optimizing the MUX size in this way allows for further increases in clock frequency due to reduced MUX latency. Of course, a slower fallback path is needed to handle any cases where the number of transitions exceeds the optimized MUX size. III. Computer System Implementation Examples

[0076] Each of the state table circuits 102, 202A-202N, MUX 204, 414-420, state propagators 408-412, MUX 420-426, MUX 602-608, CSV file parser 702, Unicode parser 706, and / or flowchart 800 can be implemented in hardware or in hardware combined with software and / or firmware. For example, state table circuits 102, 202A-202N, MUX 204, 414-420, state propagators 408-412, MUX 420-426, MUX 602-608, CSV file parser 702, Unicode parser 706, and / or flowchart 800 can be implemented as computer program code / instructions configured to execute in one or more processors and stored in a computer-readable storage medium. Alternatively, state table circuit 102, state table circuit 202A-202N, MUX 204, state table circuit 414-420, state propagator 408-412, MUX 420-426, MUX 602-608, CSV file parser 702, Unicode parser 706 and / or flowchart 800 can be implemented as hardware logic / circuits.

[0077] For example, in an embodiment, one or more of the following can be implemented together in the SoC: state table circuit 102, state table circuits 202A-202N, MUX 204, state table circuits 414-420, state propagators 408-412, MUX 420-426, MUX 602-608, CSV file parser 702, Unicode parser 706, and / or flowchart 800. The SoC may include an integrated circuit chip that includes a processor (e.g., a central processing unit (CPU), microcontroller, microprocessor, digital signal processor (DSP), etc.), memory, one or more communication interfaces, and / or other circuitry, and may optionally execute received program code and / or include embedded firmware to perform functions.

[0078] Figure 9Exemplary implementations of a computing device 900, in which embodiments may be implemented, are depicted. For example, in embodiments, one or more of any combination of state table circuitry 102, state table circuitry 202A-202N, MUX 204, state table circuitry 414-420, state propagators 408-412, MUX 420-426, MUX 602-608, CSV file parser 702, Unicode parser 706, and / or flowchart 800 may be implemented in one or more computing devices similar to the computing device 900 in fixed or mobile computer embodiments, including one or more features and / or alternative features of the computing device 900. The description of the computing device 900 provided herein is for illustrative purposes and is not intended to be limiting. As will be known to those skilled in the art, embodiments may be implemented in other types of computer systems.

[0079] like Figure 9 As shown, computing device 900 includes one or more processors (referred to as processor circuitry 902), system memory 904, and bus 906, which couples various system components, including system memory 904, to processor circuitry 902. Processor circuitry 902 is electrical and / or optical circuitry implemented in one or more physical hardware circuitry elements and / or integrated circuit devices (semiconductor chip or die) as a central processing unit (CPU), microcontroller, microprocessor, and / or other physical hardware processor circuitry. Processor circuitry 902 can execute program code stored in a computer-readable medium, such as program code in an operating system 930, application program 932, other programs 934, etc. Bus 906 represents one or more of several bus architectures, including a memory bus or memory controller, a peripheral bus, an accelerated graphics port, and a processor or local bus using various bus architectures. System memory 904 includes read-only memory (ROM) 908 and random access memory (RAM) 910. Basic input / output system 912 (BIOS) is stored in ROM 908.

[0080] The computing device 900 also includes one or more of the following drives: a hard disk drive 914 for reading from and writing to a hard disk, a disk drive 916 for reading from or writing to a removable disk 918, and an optical disc drive 920 for reading from or writing to a removable optical disc 922 such as a CD-ROM, DVD-ROM, or other optical media. The hard disk drive 914, disk drive 916, and optical disc drive 920 are connected to the bus 906 via a hard disk drive interface 924, a disk drive interface 926, and an optical disc drive interface 928, respectively. The drives and their associated computer-readable media provide the computer with non-volatile storage of computer-readable instructions, data structures, program modules, and other data. Although hard disks, removable disks, and removable optical discs have been described, other types of hardware-based computer-readable storage media, such as flash memory cards, digital video discs, RAM, ROM, and other hardware storage media, can be used to store data.

[0081] Many program modules may be stored on a hard disk, magnetic disk, optical disk, ROM, or RAM. These programs include an operating system 930, one or more application programs 932, other programs 934, and program data 936. Application programs 932 or other programs 934 may include, for example, those for implementing state table circuits 102, state table circuits 202A-202N, MUX 204, state table circuits 414-420, state propagators 408-412, MUX 420-426, MUX 602-608, CSV file parser 702, Unicode parser 706, and / or flowchart 800 and / or other embodiments described herein.

[0082] Users can input commands and information into computing device 900 through input devices such as keyboard 938 and pointing device 940. Other input devices (not shown) may include microphone, joystick, gamepad, satellite antenna, scanner, touchscreen and / or touchpad, voice recognition system for receiving voice input, gesture recognition system for receiving gesture input, and so on. These and other input devices are typically connected to processor circuitry 902 via serial port interface 942 coupled to bus 906, but may be connected via other interfaces such as parallel port, game port, or Universal Serial Bus (USB).

[0083] Display screen 944 is also connected to bus 906 via an interface such as video adapter 946. Display screen 944 may be external to or incorporated into computing device 900. Display screen 944 may display information and is a user interface for receiving user commands and / or other information (e.g., via touch, finger gestures, virtual keyboard, etc.). In addition to display screen 944, computing device 900 may include other peripheral output devices (not shown), such as speakers and printers.

[0084] Computing device 900 is connected to network 948 (e.g., the Internet) via an adapter or network interface 950, modem 952, or other means for establishing communication on the network. Modem 952 can be internal or external, such as... Figure 9 As shown, it can be connected to bus 906 via serial port interface 942, or it can be connected to bus 906 via another interface type that includes a parallel interface.

[0085] As used herein, the terms “computer program medium,” “computer-readable medium,” and “computer-readable storage medium” refer to physical hardware media, such as hard disks associated with hard disk drive 914, removable disks 918, removable optical disks 922, other physical hardware media such as RAM, ROM, flash memory cards, digital video disks, compressed disks, MEM, nanotechnology-based storage devices, and other types of physical / tangible hardware storage media. Such computer-readable storage media are distinct from and do not overlap with communication media. Communication media contain computer-readable instructions, data structures, program modules, or other data in modulated data signals (such as carrier waves). The term “modulated data signal” refers to a signal whose one or more characteristics are set or altered in a manner coded information in the signal. For example, and not limitingly, communication media include wireless media such as acoustic, radio frequency, infrared, and other wireless media, as well as wired media. Embodiments also relate to such communication media that are separate from and do not overlap with embodiments relating to computer-readable storage media.

[0086] As described above, computer programs and modules (including application program 932 and other programs 934) can be stored on a hard disk, magnetic disk, optical disk, ROM, RAM, or other hardware storage medium. Such computer programs can also be received via a network interface 950, a serial port interface 942, or any other interface type. When executed or loaded by an application, such computer programs enable the computing device 900 to implement the features of the embodiments described herein. Therefore, such computer programs represent the controller of the computing device 900.

[0087] The embodiments also relate to computer program products that include computer code or instructions stored on any computer-readable medium. Such computer program products include hard disk drives, optical disk drives, storage device packages, memory sticks, memory cards, and other types of physical storage hardware. IV. Additional Example Implementations

[0088] This paper provides a text parser configured to process multiple input characters from a character stream in parallel. The first text parser includes: a first set of state table circuits, each state table circuit in the first set corresponding to a predetermined state in a set of predetermined states, and configured to generate a first state table circuit output based on a first input character from the character stream and the corresponding predetermined state, wherein the first state table circuit outputs corresponding to each state table circuit in the first set collectively include the output of the first set of state table circuits; and a second set of state table circuits, each state table circuit in the second set corresponding to a predetermined state in a set of predetermined states, and configured to generate a second state table circuit output based on a second input character from the character stream and the corresponding predetermined state, wherein the second state table circuit outputs corresponding to each state table circuit in the second set collectively include the output of the second set of state table circuits. The system comprises: a first output multiplexer (MUX) configured to receive the outputs of a first set of state table circuits and the current state of a first text parser, and select a first text parser output from the first set of state table circuit outputs based on the current state; a state propagator configured to receive the outputs of the first set of state table circuits and a second set of state table circuit outputs, and select which output of the corresponding output of the second set of state table circuits to route to the corresponding output of the state propagator based on the outputs of the first set of state table circuits, the corresponding output including a set of state propagator outputs; and a second output MUX configured to receive a set of state propagator outputs and the current state of the first text parser, and select a second text parser output from the set of state propagator outputs based on the current state.

[0089] In the aforementioned embodiments of the text parser, the text parser further includes a second text parser configured to accept at least one of the first text parser output and the second text parser output, and to parse at least one of the first text parser output and the second text parser output to generate a third text parser output.

[0090] In the aforementioned embodiment of the text parser, the output of each state table circuit in the first set of state table circuit outputs and the second set of state table circuit outputs includes an output value and a state value, and the first text parser output and the second text parser output include an output value.

[0091] In the aforementioned embodiment of the text parser, the state propagator is further configured to select which state table circuit output in the second set of state table circuit outputs to route to the corresponding output of the state propagator based on the state value corresponding to each state table circuit output in the first set of state table circuit outputs.

[0092] In the aforementioned embodiment of the text parser, the text parser further includes: one or more additional input stages, each of the one or more additional input stages being configured to receive one or more additional input characters from a character stream, the one or more additional input characters being time-sequentially located between a first input character and a second input character; the one or more additional input stages include: an additional set of state table circuits, each of the additional set of state table circuits corresponding to a predetermined state in a set of predetermined states, and configured to generate a state table circuit output based on a corresponding additional input and a corresponding predetermined state from the one or more additional inputs, wherein the output is consistent with the output of the additional set of state table circuits. Each state table circuit's corresponding state table circuit output collectively includes an additional set of state table circuit outputs; an additional state propagator, configured to receive the additional set of state table circuit outputs and route each state table circuit output in the additional set of state table circuit outputs to the corresponding output of the additional state propagator based on the state value corresponding to the state table circuit output received from the state table circuit, the state table circuit corresponding to the next input character of the character stream in time; and an additional output MUX, configured to receive the additional set of state table circuit outputs and the current state of the text parser, and select an additional text parser output from the additional set of state table circuit outputs based on the current state of the text parser.

[0093] In the aforementioned embodiments of the text parser, at least one of the first set of state table circuits, the second set of state table circuits, or an additional set of state table circuits includes a sparse state transition table.

[0094] This paper provides a method for a text parser configured to process input in parallel. The method includes: sequentially receiving a stream of text characters including a first character and a second character; generating a first state table circuit output based on the first character and a corresponding predetermined state in a set of predetermined states, wherein the first state table circuit output corresponds to and collectively includes the outputs of the first set of state table circuits; generating a second state table circuit output based on a second character and a corresponding predetermined state in a set of predetermined states, wherein the second state table circuit output corresponds to and collectively includes the outputs of the second set of state table circuits; selecting a first text parser output from the first set of state table circuit outputs based on the current state of the text parser; selecting, based on the first set of state table circuit outputs, which output from the corresponding output of the second set of state table circuits will be routed as a set of state propagator outputs; and selecting a second text parser output from the set of state propagator outputs based on the current state.

[0095] In an embodiment of the foregoing method, the method further includes parsing at least one of the first text parser output and the second text parser output to generate a third text parser output.

[0096] In the embodiments of the foregoing method, each state table circuit output in the first group of state table circuit outputs and the second group of state table circuit outputs includes an output value and a state value, and wherein the first text parser output and the second text parser output include an output value. In an embodiment of the aforementioned method, selecting the corresponding output in the second group of state table circuit outputs further includes: based on the state value corresponding to each state table circuit output of the first group of state table circuit outputs, selecting the corresponding output of the second group of state table circuit outputs to route to the corresponding output of the state propagator.

[0097] In an embodiment of the foregoing method, the method further includes: generating one or more additional text parser outputs, each of the one or more additional text parser outputs corresponding to a corresponding additional character in one or more additional characters in a text character stream, the one or more additional characters being located between a first character and a second character in time; generating a set of circuit outputs of an additional state table for each of the one or more additional characters, based on a corresponding additional character in one or more additional characters and a corresponding predetermined state in a set of predetermined states; selecting which output of the corresponding output of the additional set of state table circuit outputs to route as an additional set of state propagator outputs based on the value of the set of state table circuit outputs, the set of state table circuit outputs corresponding to the next character in time in the text character stream; and selecting a corresponding additional text parser output from the one or more additional text parser outputs based on the current state.

[0098] In an embodiment of the aforementioned method, the method further includes: generating at least one of the following based on a sparse state transition table: a first set of state table circuit outputs, a second set of state table circuit outputs, or an additional set of state table circuit outputs.

[0099] In an embodiment of the foregoing method, the method further includes: operating the text parser by pipelined steps of: generating an additional set of state table circuit outputs, selecting which output from the additional set of state table circuit outputs, and selecting a corresponding additional text parser output from one or more additional text parser outputs.

[0100] This document provides a computer program product including a computer-readable storage device having computer program logic recorded thereon. When executed by at least one processor of a computing device, the computer program logic causes the at least one processor to perform operations implementing a text parser configured to process text characters in parallel. The operations include: sequentially receiving a stream of text characters including a first character and a second character; generating a first state table circuit output based on the first character and a corresponding predetermined state in a set of predetermined states, wherein the first state table circuit output corresponds to and collectively includes the outputs of the first set of state table circuits; generating a second state table circuit output based on a second character and a corresponding predetermined state in a set of predetermined states, wherein the second state table circuit output corresponds to and collectively includes the outputs of the second set of state table circuits; selecting a first text parser output from the first set of state table circuit outputs based on the current state of the text parser; selecting, based on the first set of state table circuit outputs, which output from the corresponding output of the second set of state table circuits will be routed as a set of state propagator outputs; and selecting a second text parser output from the set of state propagator outputs based on the current state.

[0101] In the aforementioned embodiments of the computer program product, the operation further includes: parsing at least one of the first text parser output and the second text parser output to generate a third text parser output.

[0102] In the aforementioned embodiments of the computer program product, each of the first set of state table circuit outputs and the second set of state table circuit outputs includes an output value and a state value, and wherein the first text parser output and the second text parser output include an output value.

[0103] In the aforementioned embodiment of the computer program product, selecting the first text parser output includes: selecting an output value from the output values ​​corresponding to each state table circuit output in the first group of state table circuit outputs.

[0104] In the aforementioned embodiments of the computer program product, the operation further includes: generating one or more additional text parser outputs, each of the one or more additional text parser outputs corresponding to a corresponding additional character in one or more additional characters in the text character stream, the one or more additional characters being located between a first character and a second character in time; generating a set of circuit outputs of an additional state table for each of the one or more additional characters, based on a corresponding additional character in one or more additional characters and a corresponding predetermined state in a set of predetermined states; selecting which output of the corresponding output of the additional set of state table circuit outputs to route as an additional set of state propagator outputs based on the value of the set of state table circuit outputs, the set of state table circuits corresponding to the next character in the text character stream in time; and selecting a corresponding additional text parser output from the one or more additional text parser outputs from the additional set of state propagator outputs based on the current state of the text parser.

[0105] In the aforementioned embodiment of the computer program product, the operation further includes: operating the text parser by pipelined steps of: generating an additional set of state table circuit outputs, selecting which output from the additional set of state table circuit outputs, and selecting a corresponding additional text parser output from one or more additional text parser outputs. V. Conclusion

[0106] While various embodiments of the disclosed subject matter have been described above, it should be understood that they are presented merely by way of example and not by way of limitation. Those skilled in the art will understand that various changes in form and detail may be made therein without departing from the spirit and scope of the embodiments as defined in the appended claims. Therefore, the breadth and scope of the disclosed subject matter should not be limited to any of the exemplary embodiments described above, but should be defined solely by the following claims and their equivalents.

Claims

1. A first text parser configured to process multiple input characters from a character stream in parallel, the first text parser comprising: A first group of state table circuits, each state table circuit in the first group corresponds to a predetermined state in a set of predetermined states, and is configured to generate a first state table circuit output based on a first input character of the character stream and the corresponding predetermined state, wherein the first state table circuit outputs corresponding to each state table circuit in the first group of state table circuits collectively include the first group of state table circuit outputs. The second group of state table circuits, each state table circuit in the second group corresponds to a predetermined state in the group of predetermined states, and is configured to generate a second state table circuit output based on the second input character of the character stream and the corresponding predetermined state, wherein the second state table circuit output corresponding to each state table circuit in the second group of state table circuits includes the second group of state table circuit output; A first output multiplexer MUX is configured to receive the output of the first set of state table circuits and the current state of the first text parser, and select the first text parser output from the output of the first set of state table circuits based on the current state. A state propagator is configured to receive the outputs of the first set of state table circuits and the outputs of the second set of state table circuits, and to select, based on the outputs of the first set of state table circuits, which output of the second set of state table circuits should be routed to the corresponding output of the state propagator, the corresponding output including a set of state propagator outputs; as well as The second output MUX is configured to receive the set of state propagator outputs and the current state of the first text parser, and select a second text parser output from the set of state propagator outputs based on the current state.

2. The first text parser according to claim 1, further comprising: A second text parser is configured to accept at least one of the first text parser output and the second text parser output, and parse the at least one of the first text parser output and the second text parser output to generate a third text parser output.

3. The first text parser according to claim 1, wherein each state table circuit output of the first group of state table circuit outputs and the second group of state table circuit outputs includes an output value and a state value, and wherein the first text parser output and the second text parser output include an output value.

4. The first text parser of claim 3, wherein the state propagator is further configured to select, based on the state value corresponding to each state table circuit output in the first set of state table circuit outputs, which state table circuit output in the second set of state table circuit outputs will be routed to the corresponding output of the state propagator.

5. The first text parser according to claim 3, further comprising: One or more additional input levels, each of the one or more additional input levels being configured to receive one or more additional input characters from the character stream, the one or more additional input characters being located in time between the first input character and the second input character, the one or more additional input levels comprising: An additional set of state table circuits, each of the additional set of state table circuits corresponding to a predetermined state in the set of predetermined states, and configured to generate a state table circuit output based on a corresponding additional input in one or more of the additional inputs and the corresponding predetermined state, wherein the state table circuit outputs corresponding to each of the additional set of state table circuits collectively include the additional set of state table circuit outputs. An additional state propagator is configured to receive the additional set of state table circuit outputs and route each state table circuit output in the additional set of state table circuit outputs to a corresponding output of the additional state propagator based on the state value corresponding to the state table circuit output received from the state table circuit, the state table circuit corresponding to the next input character of the character stream in time; and An additional output MUX is configured to receive the additional set of state table circuit outputs and the current state of the text parser, and to select an additional text parser output from the additional set of state table circuit outputs based on the current state of the text parser.

6. The first text parser according to claim 5, wherein at least one of the first set of state table circuits, the second set of state table circuits, or the additional set of state table circuits comprises a sparse state transition table.

7. The first text parser of claim 5, wherein an additional set of state table circuits, an additional state propagator, and an additional output MUX corresponding to one of the one or more additional input stages operate in a pipeline such that each of the first state table circuit output, the second state table circuit output, and the additional set of state table circuit outputs is valid on the same clock cycle.

8. A method for a text parser, the text parser being configured to process input in parallel, the method comprising: Receive the text character stream, including the first and second characters, in sequence; Independent of the current state of the text parser and through a first set of state table circuits corresponding to a predetermined state in a set of predetermined states, a first state table circuit output is generated based on the first character and the corresponding predetermined state in the set of predetermined states, wherein the first state table circuit output corresponding to each state table circuit in the first set of state table circuits includes the first set of state table circuit outputs. Independent of the current state and through a second set of state table circuits corresponding to the predetermined states in the set of predetermined states respectively, a second state table circuit output is generated based on the second character and the corresponding predetermined state in the set of predetermined states, wherein the second state table circuit output corresponding to each state table circuit in the second set of state table circuits includes the second set of state table circuit outputs; Based on the current state, select the first text parser output from the outputs of the first set of state table circuits; Based on the output of the first set of state table circuits, select which output from the corresponding output of the second set of state table circuits to route as a set of state propagator outputs; and Based on the current state, a second text parser is selected from the outputs of the set of state propagators.

9. The method according to claim 8, further comprising: Parse at least one of the first text parser output and the second text parser output to generate a third text parser output.

10. The method of claim 8, wherein each of the first set of state table circuit outputs and the second set of state table circuit outputs includes an output value and a state value, and wherein the first text parser output and the second text parser output include an output value.

11. The method of claim 10, wherein selecting the corresponding output from the second group of state table circuit outputs further comprises: Based on the state value corresponding to each state table circuit output in the first group of state table circuit outputs, the corresponding output in the second group of state table circuit outputs is selected to be routed to the corresponding output of the state propagator.

12. The method of claim 10, further comprising: Generate one or more additional text parser outputs, each of the one or more additional text parser outputs corresponding to a corresponding additional character in one or more additional characters in the text character stream, the one or more additional characters being located between the first character and the second character in time, the generation including for each of the one or more additional characters: Based on the corresponding additional character in the one or more additional characters and the corresponding predetermined state in a set of predetermined states, a set of circuit outputs for the additional state table is generated. Based on the value output by a set of state table circuits, select which output from the corresponding output of the additional set of state table circuits to route as the output of the additional set of state propagators, the output of the set of state table circuits corresponding to the next character in time of the text character stream. as well as Based on the current state, select the corresponding additional text parser output from the one or more additional text parser outputs from the additional set of state propagator outputs.

13. The method of claim 12, further comprising: Based on the sparse state transition table, generate at least one of the following: the output of the first set of state table circuits, the output of the second set of state table circuits, or the output of the additional set of state table circuits.

14. The method of claim 12, further comprising: The text parser is operated by pipelined steps including: generating an additional set of state table circuit outputs, selecting which output from the additional set of state table circuit outputs, and selecting the corresponding additional text parser output from the one or more additional text parser outputs.

15. A computer program product comprising a computer-readable storage device having computer program logic recorded thereon, the computer program logic, when executed by at least one processor of a computing device, causing the at least one processor to perform operations implementing a text parser, the text parser being configured to process input in parallel, the operations including: Receive the text character stream, including the first and second characters, in sequence; Independent of the current state of the text parser and through the first set of state table circuits corresponding to the predetermined states in a set of predetermined states respectively, a first state table circuit output is generated based on the first character and the corresponding predetermined states in the set of predetermined states, wherein the first state table circuit output corresponding to each state table circuit in the first set of state table circuits includes the first set of state table circuit output. Independent of the current state and through the text parser, a second set of state table circuits corresponding to the predetermined states in the set of predetermined states respectively generate a second state table circuit output based on the second character and the corresponding predetermined state in the set of predetermined states, wherein the second state table circuit output corresponding to each state table circuit in the second set of state table circuits includes the second set of state table circuit output; Based on the current state, select the first text parser output from the outputs of the first set of state table circuits; Based on the output of the first set of state table circuits, select which output from the corresponding output of the second set of state table circuits to route as a set of state propagator outputs; and Based on the current state, a second text parser is selected from the outputs of the set of state propagators.

16. The computer program product of claim 15, wherein the operation further comprises: Parse at least one of the first text parser output and the second text parser output to generate a third text parser output.

17. The computer program product of claim 15, wherein each of the first set of state table circuit outputs and the second set of state table circuit outputs includes an output value and a state value, and wherein the first text parser output and the second text parser output include an output value.

18. The computer program product of claim 17, wherein selecting the first text parser output comprises: Select an output value from the output values ​​corresponding to the outputs of each state table circuit in the first group of state table circuits.

19. The computer program product of claim 17, wherein the operation further comprises: Generate one or more additional text parser outputs, each of the one or more additional text parser outputs corresponding to a corresponding additional character in one or more additional characters in the text character stream, the one or more additional characters being located between the first character and the second character in time, the generation including for each of the one or more additional characters: Based on the corresponding additional character in the one or more additional characters and the corresponding predetermined state in a set of predetermined states, an additional set of state table circuit outputs is generated. Based on the value output by a set of state table circuits, select which output from the corresponding output of the additional set of state table circuits to route as the output of the additional set of state propagators, the output of the set of state table circuits corresponding to the next character in time of the text character stream. as well as Based on the current state of the text parser, select the corresponding additional text parser output from the one or more additional text parser outputs from the additional set of state propagator outputs.

20. The computer program product of claim 19, wherein the operation further comprises: The pipelined steps are: generating an additional set of state table circuit outputs, selecting which output from the additional set of state table circuit outputs, and selecting the corresponding additional text parser output from the one or more additional text parser outputs.

Images