State machine-based asynchronous network protocol parsing implementation system and method

Abstract

The application discloses a state machine-based asynchronous network protocol analysis implementation system and method, the system comprises a decoder, a state table and a state machine; the method is used in the state machine-based asynchronous network protocol analysis implementation system, and the method comprises the following steps: performing a state machine initialization process on the state machine; performing a protocol decoding process division process and a registration to the state table process in the decoder; the state machine starts to call an analysis function registered thereon from an initialization state in the state table, and starts a decoding process; the state machine is used for asynchronously processing the analysis process, compared with a synchronous processing mode, and the processing performance is greatly improved; the protocol analysis and state processing are decoupled; the protocol analysis process does not need to consider a state saving switching process; the workload of the analysis part is reduced; the more the analyzed protocols are, the more the state machines are saved; the state machine is used for asynchronously processing network data packets; a complex asynchronous processing process is hidden, the analysis process is clear, simple, easy to implement and maintain.

Description

Technical Field

[0001] This invention relates to the field of computer technology, and in particular to a system and method for implementing asynchronous network protocol parsing based on state machines. Background Technology

[0002] In the fields of network auditing and network security, it is necessary to analyze and inspect the content of application layer protocols in network data packets. To obtain the content of the application layer protocols, it is necessary to reconstruct and parse them. The parsing process involves handling a large number of sessions simultaneously in network security detection programs or devices; that is, a single processing step needs to handle the protocol parsing of multiple sessions concurrently. However, data from the same session does not arrive all at once. Due to performance requirements, processing the same session cannot wait, as waiting would result in the loss of data from other sessions.

[0003] Therefore, the typical protocol parsing process is as follows: data from each session is processed in the order it arrived. When data from one session has not yet arrived, the processing needs to switch to another session. During this switch, the status of the current session's parsing process needs to be recorded, for example: currently parsing the third field of the protocol header, not yet complete. When subsequent data packets arrive in this session, the next parsing operation needs to be continued based on the previously recorded status information.

[0004] In the fields of network auditing and network security, it is necessary to analyze and inspect the content of application layer protocols in network data packets. This process involves the following core technical issues:

[0005] Multi-session concurrent processing: Network security detection programs or devices need to process a large number of sessions simultaneously, and a single processing step needs to handle the protocol parsing of multiple sessions at the same time.

[0006] Asynchronous data packet arrival: Data in the same session does not arrive all at once, but is distributed at different times.

[0007] High performance requirements: There can be no waiting while processing the same session, because waiting will result in the loss of data from other sessions.

[0008] Complex state management: When data for one session is temporarily unavailable, it's necessary to switch to processing data from another session. This switching process requires recording the state processed during the current session's parsing and continuing the parsing operation based on the recorded state information when subsequent data packets arrive. Summary of the Invention

[0009] Existing technologies suffer from the following problems: 1. Complex code: The process of state recording, switching, and restoration makes the code very messy, difficult to read and understand, and prone to errors and disjointed parsing process; 2. Due to the need to switch between different sessions, the protocol parsing process is no longer the natural order of the conventional protocol description, but a process of mutual overlap and mixing; 3. Frequent state switching and saving: The parsing process needs to be constantly interrupted to save intermediate states, switch to another connection, and then switch back to restore the last saved state, which is a technical problem that is difficult to implement and maintain.

[0010] To achieve the above objectives, one of the technical solutions adopted by the present invention is: an asynchronous network protocol parsing implementation system based on a state machine, which includes a decoder, a state table, and a state machine;

[0011] The decoder is used to divide multiple protocol processes according to the protocol description, implement each protocol decoding process separately, and register all decoding processes to the state table;

[0012] The state table is used to record the state of different stages and different sessions during the decoding process of each protocol;

[0013] The state machine is responsible for executing the flow, transitions, and restoration of states in the state table, and is decoupled from the decoder.

[0014] Another technical solution adopted by the present invention is: an asynchronous network protocol parsing implementation method based on a state machine, which is used in the above-mentioned asynchronous network protocol parsing implementation system based on a state machine.

[0015] Furthermore, the method includes:

[0016] Perform the state machine initialization process;

[0017] The decoding process of the protocol is divided into a process flow and a process of registering it to the state table in the decoder;

[0018] The state machine starts by calling the parsing function registered on the state table from the initialization state in the state table and begins the decoding process.

[0019] Furthermore, the state machine initialization process involves setting various parameters and values ​​for the state machine and initializing the state machine's initialization function.

[0020] Furthermore, the protocol decoding process is divided into multiple stages, each stage is decoded, and the transition process between the decoding processes of each protocol is determined.

[0021] Furthermore, registering to the state table involves registering the parsing functions in each stage to the state table.

[0022] Furthermore, the decoding process includes: if the data packet for the session required by the parsing function has not yet arrived, then return to the state table to wait;

[0023] The state machine saves the current state to the state table and then proceeds to execute the state machine of other session connections;

[0024] When the data packet of the session that needs to be parsed arrives, the state machine is re-triggered to run, and the corresponding state stored in the state table is extracted and executed.

[0025] After the current state ends, the state machine loads the next state from the state table and continues execution;

[0026] The decoding process is repeated until the decoding is complete or an error occurs, at which point it ends.

[0027] Furthermore, the state machine loads the next state from the state table based on the return value.

[0028] Compared to existing technologies, this approach, which relies on an independent, general-purpose state machine, represents a significant architectural upgrade in the technical implementation of network protocol parsing, compared to current mainstream decoder implementations. Its core breakthrough lies in completely decoupling the decoder from state management, freeing the decoder from tedious state recording and switching tasks, allowing it to focus on the protocol's syntax rules and semantic parsing, thus triggering a series of cascading advantages.

[0029] In the current implementation logic of traditional decoders, each protocol decoder not only needs to parse protocol fields but also must have a built-in state management mechanism. For example, when the parsing of half of an SMTP session is interrupted, the decoder needs to define its own state variables to record the interruption point and write switching logic to deal with the intervention of other sessions. This bundled parsing logic + state management mode leads to two significant problems: First, the decoder code is bloated, with the core protocol parsing logic buried under state handling code, resulting in a significant decrease in readability; second, decoders of different protocols need to repeatedly implement similar state switching logic, which not only wastes development resources but also increases the risk of introducing bugs due to logic duplication. For example, if a decoder fails to save the protocol version field during state switching, it may cause field misalignment during subsequent parsing.

[0030] The new method completely removes this complexity from the decoder by introducing an independent, general-purpose state machine. The state machine is responsible for coordinating the scheduling, state recording, and switching of all sessions: when data in session A is temporarily incomplete, the state machine automatically records its parsing progress and switches decoder resources to session B; when subsequent data arrives in session A, the state machine quickly locates the historical state using the session's unique identifier, resets the decoder to the last interrupted node, and allows it to continue parsing from the remaining 2 bytes of the type field. In this process, the protocol decoder's perspective is greatly simplified: it doesn't need to care which session it's currently processing, nor does it need to consider whether data will be interrupted. From its perspective, it is always processing a continuous data stream flowing in the natural order of the protocol, as smoothly as parsing protocol data in a local file. This separation of the synchronous parsing appearance from the asynchronous processing kernel reduces the decoder's code size by more than 30%, highlighting the core logic and greatly lowering the development and maintenance threshold.

[0031] From a performance and scalability perspective, this architecture offers even greater advantages. In terms of performance, the general-purpose state machine employs an efficient event-driven model, enabling session state switching and loading within milliseconds, avoiding lock contention or resource blocking caused by the built-in state management of traditional decoders. For example, when processing 100,000 concurrent sessions per second, the state machine can quickly locate session information through a pre-allocated state cache pool, while traditional decoders may experience memory fragmentation and a surge in GC garbage collection pressure due to the independent allocation and release of state variables for each session. In terms of scalability, when new protocol parsing support is needed, developers only need to focus on the protocol's syntax rules, without repeatedly writing state recording logic. The state machine has a built-in general interrupt and resume mechanism; new decoders only need to follow the synchronous process from receiving data to parsing fields to returning results to seamlessly integrate into multi-session asynchronous processing environments. This "develop once, reuse in multiple scenarios" characteristic improves the scalability of the protocol parsing module several times over, making it particularly suitable for the current rapid iteration of network protocols.

[0032] More fundamentally, this approach solves the fragmentation problem of traditional decoders through a standardized state management interface. In traditional implementations, the state definition of an HTTP decoder may be completely different from that of an FTP decoder, making cross-protocol state monitoring and debugging extremely difficult. However, the general state machine uses a unified state description format, allowing administrators to view the parsing status of all sessions using the same set of tools, quickly pinpointing the reason why a session stalls due to an incomplete protocol header, and significantly reducing operational complexity.

[0033] In summary, this architecture, which combines state machine coordination with decoder-focused parsing, not only improves performance through asynchronous processing but also addresses pain points in traditional implementations such as code redundancy, poor scalability, and maintenance difficulties through decoupling and standardization. For large-scale network security devices that need to support hundreds of protocols, the benefits of this approach increase linearly with the number of protocols. Each new protocol reduces the development cost of state management and increases system stability, ultimately achieving the ideal state of centralized control over complex logic and concise, efficient core functionality. Attached Figure Description

[0034] Figure 1 This is a flowchart illustrating the asynchronous network protocol parsing implementation method based on a state machine according to the present invention. Detailed Implementation

[0035] The technical solutions of the asynchronous network protocol parsing implementation method and system based on state machines provided by the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative effort are within the scope of protection of the present invention.

[0036] Example 1

[0037] A state machine-based asynchronous network protocol parsing implementation system, which includes a decoder, a state table, and a state machine;

[0038] The decoder is used to divide multiple protocol processes according to the protocol description, implement each protocol decoding process separately, and register all decoding processes to the state table.

[0039] Specifically, the core responsibility of the decoder is to focus on the syntax rules of the protocol itself, breaking down the complex protocol process into a series of independent decoding processes, and registering them into the system through standardized interfaces. Its design philosophy is similar to a plug-in architecture, where each protocol decoder is an independent functional unit, without needing to worry about multi-session concurrency and state management, and only needing to focus on how to parse protocol fields.

[0040] The decoder's operating modes include: Protocol process decomposition: breaking down the complete protocol interaction process into several consecutive decoding steps. For example, the HTTP protocol can be broken down into three stages: request line parsing, request header parsing, and message body parsing; the FTP protocol can be broken down into control connection command parsing, data connection response parsing, and other processes. Each stage defines clear input, output, and interruption conditions.

[0041] Independent implementation and registration: Each decoding process is implemented by a dedicated code module and registered in the state table through a unified registration interface. This plug-and-play design means that when adding a new protocol, only the corresponding decoder needs to be developed, without modifying the core logic of the state machine or state table.

[0042] Synchronous parsing semantics: From the decoder's perspective, each decoding process is synchronous and continuous. It receives a complete byte stream, extracts fields according to protocol rules, and returns the parsing result if there is enough data. If there is insufficient data, it returns a flag indicating that it needs to continue receiving data, without recording the current progress.

[0043] The state table is used to record the state of different stages and different sessions during the decoding process of each protocol.

[0044] Specifically, the state table is the central memory of the entire system, responsible for globally storing the real-time states of all protocols and sessions during the parsing process, providing data support for the scheduling of the state machine. It functions like a structured database, indexed by session and protocol phases, accurately recording the interruption points and context information of each parsing process.

[0045] The core functions of the status table include: multi-dimensional status recording: each status record contains three layers of key information;

[0046] Session identifier: such as the TCP 5-tuple, including source IP, source port, destination IP, destination port, and protocol type, used to uniquely distinguish different sessions;

[0047] Protocol phase: Marks which decoding process the current session is currently in;

[0048] Parsing context: includes parsed field values, unparsed byte buffers, and the expected data stream format for the next stage.

[0049] Efficient Indexing and Updates: The state table employs a composite structure of a hash table and a linked list, using the session identifier as the key to quickly locate state records, supporting millisecond-level query, insert, and update operations. When a session's parsing phase switches or the context changes, the state table updates the corresponding records in real time, ensuring the state machine always obtains the latest parsing progress.

[0050] State lifecycle management: To avoid memory overflow, the state table will clean up the state of sessions that have been inactive for a long time, while reserving enough cache space for active sessions to balance resource consumption and parsing continuity.

[0051] The state machine is responsible for executing the flow, transitions, and restoration of states in the state table, and is decoupled from the decoder.

[0052] Specifically, the state machine is the nerve center of the system. By driving the state transitions in the state table, it enables asynchronous scheduling, state switching, and interrupt recovery for multi-session parsing, completely decoupling it from the business logic of the decoder. Its core function is to determine when to process which session, how to process it, and from where to continue processing.

[0053] The operation mechanism of a state machine can be divided into three steps: session triggering and state querying: When a new data packet arrives, the state machine first extracts the session identifier and queries the historical state of the session through the state table.

[0054] If it is a new session, the parsing starts from the initial stage of the protocol and a new record is created in the state table;

[0055] If a session already exists, its current protocol phase and parsing context are read.

[0056] Decoder execution: The state machine retrieves the parsing context of the current session from the state table, calls the decoder for the corresponding stage, and passes in the unfinished byte buffer. After the decoder completes partial parsing, it returns one of two results:

[0057] If the parsing is complete, the state machine updates the state table and advances the session to the next stage;

[0058] If parsing is interrupted, the state machine writes the unfinished buffer returned by the decoder and the current stage information into the state table, marking the session as pending continuation.

[0059] Multi-session switching and resource scheduling: When a session is interrupted due to insufficient data, the state machine immediately releases the currently processing resources, retrieves the data packet for the next session from the queue, and repeats the process. This polling-based scheduling ensures that all sessions are processed fairly in the order of data arrival, avoiding blocking the global process due to waiting for a particular session.

[0060] In summary, the overall operation of the system presents a highly efficient closed loop:

[0061] After a network data packet enters the system, the state machine first queries the state table using the session identifier to determine the parsing starting point. The state machine then calls the decoder for the corresponding stage, passing in the context data from the state table, such as unfinished buffers. The decoder synchronously parses the data and returns the result or interruption information, which the state machine then updates the state table accordingly. The state machine switches to the next session and repeats the above process until all data packets are processed.

[0062] The advantage of this collaborative model lies in the fact that the decoder focuses on how to perform the parsing, the state table is responsible for recording the state, and the state machine controls the process flow. These three interact through standardized interfaces, ensuring both the simplicity of individual modules and the high efficiency of the overall system. For network security devices that need to handle massive concurrent sessions, this architecture can reduce performance overhead by more than 40% while ensuring parsing accuracy, providing solid technical support for real-time threat detection.

[0063] Taking HTTP protocol parsing as an example, the specific implementation process of this method is explained as follows:

[0064] The decoder definition divides HTTP protocol parsing into three main stages: request URL, request headers, and request body. The request URL stage parses the HTTP request method and URL. The request header stage parses the various fields in the HTTP request headers. The request body stage parses the main content of the HTTP request.

[0065] Decoder registration registers the parsing functions of the above three stages into the state table of the state machine, forming the following state transition: from request URL to request header to request body.

[0066] State machine operation – At startup, the state machine begins execution from the request URL state. If the received data is insufficient to complete URL parsing, the state machine saves its current state and proceeds to process other sessions. When the remaining URL data arrives, the state machine restores its previous state, completes URL parsing, and then transitions to the request header state. Parsing the request header may require multiple packets; the state machine saves its state each time insufficient data arrives and restores its state to continue parsing when new data arrives. After the request header parsing is complete, the state machine enters the request body state and begins parsing the HTTP request body content. Throughout this process, if any error occurs, the state machine terminates the processing of the current session.

[0067] In this way, the HTTP protocol parsing process is broken down into multiple independent states, each of which can be executed and resumed independently, thereby enabling the asynchronous processing of multiple HTTP sessions while maintaining code clarity and maintainability.

[0068] Example 2

[0069] like Figure 1 As shown, an asynchronous network protocol parsing implementation method based on a state machine is used in the asynchronous network protocol parsing implementation system based on a state machine described in Embodiment 1 above.

[0070] Furthermore, the method includes: performing a state machine initialization process on the state machine;

[0071] Furthermore, the state machine initialization process involves setting various parameters and values ​​for the state machine and initializing the state machine's initialization function.

[0072] Specifically, the state machine initialization process is the crucial transition from a static state to a running system. It establishes a stable underlying framework for subsequent multi-session protocol parsing by presetting core parameters, configuring operating rules, and loading initial functions. This process acts like a calibration benchmark for precision instruments, directly determining the system's stability and efficiency in high-concurrency, high-throughput scenarios.

[0073] The core parameters that need to be explicitly set during the initialization phase include:

[0074] Session capacity limit: Defines the maximum number of sessions the system can maintain simultaneously, preventing crashes due to memory overflow. The state machine pre-allocates hash table slots and state record structures based on this, ensuring rapid insertion when new sessions arrive.

[0075] Status timeout threshold: Sets the maximum waiting time for a session to resume without data. Session status exceeding this threshold will be automatically cleaned up to release memory resources and prevent long-term idle status records from occupying space.

[0076] Buffer Specifications: Configure the maximum buffer size for each session to ensure that critical data such as protocol headers and commands can be accommodated while avoiding memory waste due to excessively large buffers.

[0077] Protocol type mapping table: Establishes the association between protocol identifiers and decoders, ensuring that the state machine can quickly locate the corresponding parsing module.

[0078] These parameters are like the system's operating instructions, limiting the upper limit of resource consumption and providing clear criteria for judging state transitions.

[0079] The initialization process requires calling and registering three types of key functions, which constitute the nerve center of the state machine:

[0080] Session identification function: Responsible for extracting a 5-tuple from data packets to generate a unique session identifier, which serves as an index for state table lookups. This function needs to be loaded during initialization to ensure that all data packets are correctly categorized.

[0081] State transition driver function: defines the flow rules from the completion of parsing to the start of the next parsing stage, to parsing interruption, and then to saving the state. For example, when the HTTP request line parsing is complete, this function automatically triggers a state table update, advancing the session to the request header parsing stage.

[0082] Resource scheduling functions: Implement priority ordering and processing queue management for multiple sessions. For example, assign higher priority to sessions carrying sensitive ports to ensure that encrypted traffic such as HTTPS is parsed first, reducing latency.

[0083] These functions are loaded into memory during initialization and bound to the event triggering points of the state machine, thus becoming the start of the driving process.

[0084] In summary, initializing the state machine can eliminate runtime uncertainties and the dynamic nature of the network environment, such as sudden traffic spikes or protocol confluence, requiring the system to define boundary conditions during startup. For example, without a preset session capacity limit, the state table may crash due to a surge in hash collisions when the instantaneous number of sessions exceeds memory capacity; without a timeout cleanup mechanism, a large number of zombie sessions will continuously occupy state records, preventing new sessions from being processed. Initialization, by fixing parameters, constrains the system's behavior within a controllable range, preventing runtime chaos caused by irregularities.

[0085] Meanwhile, reducing runtime computational overhead and preloading parameters and functions significantly reduces runtime dynamic decision-making costs: the pre-allocated session hash table reduces state lookup time from O(n) to O(1), avoiding performance fluctuations caused by dynamic scaling under high concurrency; the pre-registered protocol and decoder mapping table eliminates the need for protocol type determination and decoder lookup before each parsing, directly locating the parsing module by port or identifier, thus improving response speed. This pre-computation mode enables the state machine to maintain microsecond-level single-operation latency even when processing millions of data packets per second.

[0086] Furthermore, it lays the foundation for system scalability. The modular design of the initialization process, such as parameter configuration files or function registration interfaces, allows the system to expand its functionality by modifying the configuration. When adding a new protocol, only the association between the protocol identifier and the decoder needs to be added to the mapping table, without modifying the core logic of the state machine. When adjusting performance parameters, only the initialization configuration file needs to be modified and the system restarted, without refactoring the code. This configuration-driven scalability enables the system to quickly adapt to new protocols or higher bandwidth network environments.

[0087] The state machine initialization process is the first line of defense for system stability. It's not simply about assigning parameters; rather, it involves pre-setting rules, solidifying logic, and optimizing resources to build a controllable, efficient, and scalable operational foundation for the entire system. It avoids crashes caused by uncontrolled resources during runtime, improves the real-time performance of protocol parsing through a pre-loading mechanism, and simplifies and controls expansion needs such as adding new protocols and adjusting performance. For network security devices that require 24 / 7 uninterrupted operation, this process is a prerequisite for ensuring high availability. Only with a solid foundation laid during the initialization phase can the state machine accurately schedule decoders and state tables in complex network environments, achieving efficient and stable multi-session protocol parsing.

[0088] The decoding process of the protocol is divided into a process flow and a process of registering it to the state table in the decoder;

[0089] Furthermore, the protocol decoding process is divided into multiple stages, each stage is decoded, and the transition process between the decoding processes of each protocol is determined.

[0090] Furthermore, registering to the state table involves registering the parsing functions in each stage to the state table.

[0091] Specifically, in a state machine-based asynchronous network protocol parsing system, the core function of the decoder is not only to parse protocol fields, but also to enable the state machine to efficiently schedule the parsing logic of each protocol through structured process division and standardized state mechanisms. These two processes are like building blocks for protocol parsing: first, the complex process is broken down into reusable modules, and then these modules are connected to the system's scheduling center, ultimately achieving complete decoupling between parsing logic and state management.

[0092] The division of the protocol decoding process is essentially a structured decomposition of the protocol interaction logic. By clarifying the input, output, and flow conditions of each stage, the complex protocol parsing becomes step-by-step, interruptible, and resumable.

[0093] The decoding process for each protocol needs to be broken down into several consecutive and independent stages, with each stage focusing on parsing a specific functional block of the protocol.

[0094] Taking the HTTP protocol as an example, a decoder can divide the parsing process into three stages: the request URL, the request headers, and the request body. Each stage implements its own parsing logic. For example, the request URL stage parses the HTTP method and URL, the request headers stage parses the various header fields, and the request body stage parses the body content. This division makes the implementation of each stage relatively simple and easy to modify or extend according to changes in the protocol.

[0095] The core principle of phase division is data integrity boundaries: the parsing of each phase depends on a continuous and independently verifiable piece of data, such as an HTTP request line ending with \r\n, ensuring that if data is insufficient, the current phase can be interrupted and subsequent data can be used to continue from that phase.

[0096] After dividing the process into phases, it is necessary to clarify the triggering conditions and transition paths between phases to ensure that the parsing process proceeds in an orderly manner according to the protocol logic. The core of the transition process is the transition from the completion of the current phase to the triggering of the next phase, which specifically includes:

[0097] Normal transition: When the current stage of parsing is complete and the data is intact, it automatically proceeds to the next stage. For example, after the HTTP request line is parsed, such as upon detecting \r\n, it can proceed to the request header parsing stage without any additional conditions.

[0098] Conditional transition: The next stage is determined based on the parsing result of the current stage. For example, in the FTP protocol, if stage 2 parses a 331 Password required response, the next stage must be a PASS command parsing; if it parses a 530 Not logged in response, it directly enters the connection closing stage.

[0099] Abnormal transition: When a protocol error is encountered during parsing, a termination phase is triggered, and parsing is stopped to avoid unnecessary resource consumption.

[0100] These transition rules are encoded into the stage jump logic of the decoder, ensuring that the end of each stage clearly points to what to do next.

[0101] After the phase division is completed, the parsing function for each phase needs to be registered in the state table so that the state machine can accurately call the corresponding parsing logic based on the phase information of the current session. This process essentially establishes a mapping relationship between protocol type + phase number and parsing function.

[0102] The parsing logic for each stage needs to be encapsulated into an independent function and follow a unified interface specification:

[0103] Input: The byte stream to be parsed at the current stage;

[0104] Output: Parsing results, such as extracted field values ​​and structured data; status indicators, such as parsing completed, insufficient data requiring interruption, or protocol error; next stage number.

[0105] For example, the HTTP phase 1 parsing function http_parse_request_line() receives a byte stream, attempts to match the METHODURLVERSION\r\n format, and if successful, returns the parsed method / URL / version and specifies that the next phase is request header parsing; if the byte stream only contains GET / index.ht, it returns an insufficient data flag and preserves the current buffer.

[0106] The parsing function completes the binding through the registration interface of the state table. For example, by calling the function register_stage_parser(protocol_id, stage_id, parser_func), the protocol ID and stage ID are associated with the parsing function and stored in the parsing function registry of the state table.

[0107] For example: Register http_parse_request_line() for phase 1 of the HTTP protocol;

[0108] Register ftp_parse_auth() for Phase 2 of the FTP protocol.

[0109] The state table maintains a phase-function mapping table for each protocol. When the state machine needs to parse a phase of a session, it can quickly query and call the corresponding parsing function by using the protocol ID and the current phase ID, without needing to know the specific implementation of the function.

[0110] Modularizing the protocol parsing logic reduces development complexity. By splitting it into stages, developers don't need to implement the entire protocol parsing logic all at once; they can focus on the functionality of a single stage. The code size of a single module is reduced by more than 60%, the logic is clearer, and debugging is easier. The explicit transition process avoids the problem of affecting everything if one part is changed: when modifying the parsing logic of a certain stage, only the function of that stage needs to be adjusted, without affecting the scheduling logic of other stages or the state machine.

[0111] The system supports asynchronous continuation of the state machine, enabling non-blocking parsing. After phase division, the parsing progress of each session can be accurately described by the current phase ID and the buffer. When data is interrupted, the state machine only needs to record these two pieces of information, eliminating the need to store complex parsing context and significantly simplifying the complexity of state recording. The design from parsing completion to the specified next phase allows the state machine to advance its state without understanding the protocol logic; it only needs to follow the next phase number returned by the parsing function, achieving complete decoupling between parsing logic and state management.

[0112] To enhance system scalability and adapt to multi-protocol scenarios, when adding a new protocol, simply divide the system into stages according to rules, implement the parsing functions for each stage, and register them in the state table; no modification to the core state machine logic is required. For example, when supporting the WebSocket protocol, simply split its handshake stage into the frame parsing stage and the close stage, register the corresponding functions, and the system can automatically identify and schedule them. When upgrading a protocol, only the parsing functions for the corresponding stages need to be modified and re-registered; the parsing logic of the old protocol remains compatible, achieving a smooth transition.

[0113] Enhancing the observability of the parsing process facilitates troubleshooting. Phased parsing allows for precise description of the state of each session. Combined with state table records, developers can intuitively track parsing interruptions, quickly locate problems, and significantly reduce debugging difficulty. These two processes, through structured decomposition and standardized registration, transform protocol parsing from a chaotic overall logic into controllable phased modules, which can be efficiently scheduled by a state machine. For developers, this means they don't need to worry about the complexity of multi-session asynchronous processing; they can focus on the parsing logic of a single phase. For the system, this means strong scalability and stability, enabling support for new protocols and adaptation to new scenarios without modifying the core architecture. This division of responsibilities is the core reason why state machine-based parsing systems can efficiently handle multi-session asynchronous parsing.

[0114] Meanwhile, the state table decouples the decoder and the state machine, improving the system's modularity. It records different stages and session states during the protocol decoding process, enabling the state machine to efficiently manage and switch parsing states across different sessions, thus achieving asynchronous processing capabilities.

[0115] A status table can be designed as a data structure containing multiple fields. For example:

[0116] {session_id: unique identifier, protocol: HTTP, current_state: request header, next_state: request body, parsed_data: {…}, remaining_data: …, last_execution_point: 1234}.

[0117] This design allows the state machine to quickly find and restore the parsing state of a specific session, while providing a clear interface for the decoder to update the parsing progress.

[0118] The state machine starts by calling the parsing function registered on the state table from the initialization state in the state table and begins the decoding process.

[0119] Furthermore, the decoding process includes: if the data packet for the session required by the parsing function has not yet arrived, then return to the state table to wait;

[0120] The state machine saves the current state to the state table and then proceeds to execute the state machine of other session connections;

[0121] When the data packet of the session that needs to be parsed arrives, the state machine is re-triggered to run, and the corresponding state stored in the state table is extracted and executed.

[0122] After the current state ends, the state machine loads the next state from the state table and continues execution; specifically, the state machine loads the next state from the state table based on the return value.

[0123] The decoding process is repeated until the decoding is complete or an error occurs, at which point it ends.

[0124] Specifically, after the state machine starts, it begins with the initialization state of the state table, calls the corresponding parsing function according to the registered mapping relationship, and enters the decoding process.

[0125] Interruption Parsing and State Preservation: To avoid meaningless waiting, when the parsing function detects that the current session's data packet is incomplete, it immediately returns a data insufficiency flag to the state machine. At this time, the state machine triggers the state preservation mechanism, writing the core information of the current session into the state table: including the current stage number, incomplete byte buffers, and parsed temporary results. The state machine releases the processing resources for this session, retrieves the next session's data packet from the waiting queue, and calls the parsing function for its corresponding stage. For example, after processing the interruption of session A, it immediately switches to the FTP command parsing stage of session B, ensuring that CPU resources are not idle. This mechanism of immediate switching without waiting for data fundamentally avoids the problem of global blocking caused by single-session waiting in traditional synchronous decoding.

[0126] Session continuation and state restoration: Seamlessly connecting the parsing process. When a subsequent data packet arrives for a session: the network interface triggers a data packet arrival event, and the state machine quickly locates the historical state record of that session in the state table using the session identifier. The state machine restores the context: it extracts the saved stage number and incomplete buffer, and concatenates the newly arrived byte stream into the buffer to form complete data to be parsed. The state machine calls the parsing function for that stage to continue parsing from the last interrupted position. Because the context is complete, the parsing function will complete the remaining parsing of the request header as if it had never been interrupted, and return a parsing completion flag. This process achieves seamless interruption-resumption; for the parsing function, it appears as if it is always processing a continuous data stream, completely unaware of the existence of multiple session switching.

[0127] Phase progression and next-state loading: Automatic flow according to rules. After the current phase is parsed, the parsing function returns a parsing success flag, along with the next phase number. Based on this, the state machine updates the phase information of the session in the state table, changing the current phase from request header parsing to message body parsing, and clearing the temporary buffer of completed phases. According to the new phase number, the parsing function for the next phase is called from the mapping relationship in the state table, passing in new data packets or waiting for subsequent data from this phase. If the parsing function returns a protocol error, the state machine marks the session as terminated, stops parsing, and records the error type in the state table for subsequent auditing and troubleshooting.

[0128] Iterative loop: until decoding is complete or interrupted. The above process will loop repeatedly: from state machine processing session A, to insufficient data, to saving state, to switching, to session B, to session B parsing completion, to advancing to the next stage, to new data arriving in session A, to restoring state and continuing parsing, until the decoding process of all sessions is completed, or terminated due to protocol errors, timeouts, or other reasons.

[0129] In summary, this approach maximizes CPU utilization and avoids resource waste. In traditional synchronous decoding, if a session waits due to insufficient data, the entire processing thread is blocked, causing the CPU to idle. This new process, however, uses state preservation and immediate switching to ensure the CPU is always processing valid data: when session A is interrupted, the state machine immediately schedules the parsing of session B, keeping CPU utilization high. For high-concurrency scenarios, this mechanism can reduce processing latency from milliseconds to microseconds, meeting real-time detection requirements.

[0130] To ensure continuous parsing and eliminate the risk of data loss, the state table provides a complete record of session states, ensuring that the parsing process for each session is not interrupted by switching. Even if data packets from a session arrive at 10-second intervals, the state machine can still extract historical information from the state table and continue parsing from the last point of interruption, avoiding the parsing failure caused by data fragmentation in traditional methods. For network anomalies such as TCP retransmission and out-of-order delivery, the state machine can automatically correct the data order by concatenating buffers and continuing states, ensuring the accuracy of protocol parsing.

[0131] Simplify parsing function design and reduce development costs. Parsing functions do not require built-in state management logic; they only need to focus on whether the current data is parseable and how to extract fields, significantly reducing development difficulty. Taking HTTP parsing as an example, traditional implementations require 200 lines of code to handle state switching, while this process only requires 80 lines of code focusing on field extraction, reducing code volume by 60%. When integrating new protocols, developers only need to implement the parsing functions for each stage according to the rules, without needing to worry about multi-session scheduling, shortening the learning curve from weeks to days.

[0132] In summary, the decoding process can improve system stability and maintainability. The decoupling of the state machine and the parsing function simplifies system fault localization: if a session parses abnormally, its historical state can be queried through the state table to quickly pinpoint whether the error is a logical error in the parsing function or a data transmission anomaly. The centralized state management of the state table avoids the inconsistency problems caused by each parsing function independently maintaining its state, such as conflicting stage numbers recorded in different functions for the same session.

[0133] Adapting to complex network environments and enhancing robustness. Fragmentation and out-of-order distribution of network data are common occurrences, and this process naturally adapts to this characteristic through state preservation and continuation mechanisms: For TCP fragmentation caused by MTU limitations, the state machine can achieve complete parsing through buffer concatenation and staged continuation. In the face of sudden traffic surges, the state machine can automatically release invalid session resources through the state table's timeout cleanup mechanism, ensuring that the parsing of core business processes is not affected.

[0134] By automatically saving and restoring the context, the state machine can proactively release CPU resources and switch to processing other sessions when data is insufficient, improving overall processing efficiency. This mechanism eliminates the need for the decoder to manually manage complex state switching logic, greatly simplifying the implementation of protocol parsing. When parsing HTTP request headers, if the received data is insufficient to complete the parsing of all header fields, the state machine automatically saves the current parsing state, including parsed header fields and the current parsing position, to the state table. Then, the state machine can switch to processing other sessions. When new data arrives, the state machine restores the previously saved state from the state table and continues parsing from where it left off, without requiring the decoder to handle these complex state management logics.

[0135] Meanwhile, viewing network protocols as a series of state transitions allows the complex protocol parsing process to be decomposed into a series of simpler state transitions. This approach makes the protocol parsing process clearer, easier to understand, and easier to implement. It also provides a theoretical foundation for asynchronous processing, enabling state machines to more naturally manage transitions between different states. Taking the simplified HTTP protocol as an example, it can be defined as the following state machine: 1. Initial state to parsing the request line; 2. Parsing the request line to parsing the header fields; 3. Parsing the header fields to parsing the message body; 4. Parsing the message body to the end state. Between each state, the process may pause due to insufficient data, waiting for more data to arrive. This definition makes the entire parsing process very clear, with each state having a definite entry and exit condition.

[0136] The foregoing has provided a detailed description of one embodiment of the present invention, but this description is merely a preferred embodiment and should not be construed as limiting the scope of the invention. All equivalent variations and modifications made within the scope of the claims of this invention should still fall within the patent coverage of this invention.

Claims

1. A state machine based asynchronous network protocol parsing implementation system, characterized in that, The system comprises a decoder, a state table and a state machine; The decoder is used to divide a plurality of protocol processes according to a protocol description, to implement each protocol decoding process separately, and to register all decoding processes in the state table; The state table is used to record the states of different stages and different sessions in each protocol decoding process; The state machine is used to be responsible for the flow, conversion and recovery of the states in the state table, and is decoupled from the decoder; The state machine is subjected to a state machine initialization flow; The decoding process of the protocol is divided in the decoder and registered in the state table; The state machine calls the parsing function registered thereon from the initialization state in the state table and starts the decoding flow; The decoding flow comprises: if the data packet of the session required by the parsing function has not arrived, then returning to the state table to wait; The state machine saves the current state in the state table and executes the state machine of other session links; When the data packet of the session required for parsing arrives, the state machine is triggered to run again, extracts the corresponding state saved in the state table and executes; After the current state ends, the state machine loads the next state in the state table and continues to execute; The decoding flow is repeatedly performed until the decoding is completed or an error occurs in the middle, and then the decoding flow ends.

2. A method for implementing a state machine based asynchronous network protocol parsing, characterized in that, The method is used in the state machine-based asynchronous network protocol parsing implementation system of claim 1.

3. The state machine based asynchronous network protocol parsing implementation method of claim 2, wherein, The state machine initialization flow is to set various parameters and values for the state machine and initialize the initial function of the state machine.

4. The state machine based asynchronous network protocol parsing implementation method according to claim 3, characterized in that, The decoding process division flow of the protocol is to divide the decoding process of the protocol into a plurality of stages, to complete the decoding of each stage, and to determine the transition flow between the decoding processes of each protocol.

5. The state machine based asynchronous network protocol parsing implementation method of claim 4, wherein, The registration in the state table is to register the parsing function in each stage in the state table.

6. The state machine based asynchronous network protocol parsing implementation method of claim 3, wherein, The state machine loads the next state in the state table according to the return value.

Images