A state machine-based http fragmented message restoration method and system

Abstract

This application relates to a state machine-based method and system for restoring fragmented HTTP messages. By performing session cohesion on network traffic and initializing a message restoration state machine, different termination methods based on length identifiers, chunk transmission identifiers, and connection closures are mapped to the state machine for processing, thus solving the problem of poor scalability caused by multi-branch code processing. During the reassembly process, subsequent data fragments are validated for legality and then written to external storage media in segments, mitigating the risk of resource exhaustion and system blocking caused by relying on memory to assemble the complete message body. Furthermore, a dynamic control strategy is introduced, including restoration timeout, maximum allowed restoration length, and message content type restrictions. When a termination condition is detected, the restoration process is terminated promptly and resources are released, preventing problems caused by abnormal traffic, black hole messages, or malicious attacks. This method is suitable for processing continuously generated HTTP message bodies in generative AI scenarios.

Description

Technical Field

[0001] This application relates to the field of network data analysis and security supervision, and in particular to a method, apparatus, system, computer device and computer-readable storage medium for restoring HTTP fragmented message based on a state machine. Background Technology

[0002] With the widespread adoption of artificial intelligence technology, AI question answering has become a fundamental means of problem-solving in the digital field. In areas such as data supervision and information leakage prevention, it is necessary to use specific methods to capture and decrypt raw HTTP traffic in the intranet environment to obtain AI dialogue information in HTTP sessions, and then perform operations such as security supervision and risk event reconstruction.

[0003] According to the HTTP protocol standard, there are three main ways to identify the end of a message body: the Content-Length field, chunked transmission, and implicit identification based on connection closure. In related technologies, such as traditional network protocol analysis tools, when handling TCP fragment reassembly and HTTP segment reconstruction, the core implementation mainly relies on caching all fragmented data in system memory, performing protocol parsing only after complete reconstruction.

[0004] Currently, no effective solution has been proposed for the problems of HTTP message bodies relying on memory reorganization, easily leading to resource exhaustion and abnormal blocking when the data volume is large, the length is unpredictable, or the network is abnormal. Summary of the Invention

[0005] This application provides a state machine-based method, system, computer device, and computer-readable storage medium for restoring fragmented HTTP messages, which at least solves the problem in related technologies that HTTP message bodies are prone to resource exhaustion and abnormal blocking when the data volume is large, the length is unpredictable, or the network is abnormal.

[0006] In a first aspect, embodiments of this application provide a method for restoring fragmented HTTP messages based on a state machine, the method comprising: Data packets in network traffic are acquired, and session cohesion is performed on the data packets based on connection identification information to establish corresponding HTTP sessions. A message restoration state machine is initialized for the HTTP sessions. Based on the message header field of the first HTTP message in the HTTP session, determine whether the HTTP message contains an HTTP message body. If so, map the end mode of the HTTP message body to the message restoration state machine for processing based on the message header field. Determine whether the end condition of the HTTP message body is met. If not, submit the corresponding message restoration task to the fragmentation and reassembly processing unit for asynchronous processing. The fragmentation and reassembly processing unit is independent of the HTTP protocol parsing unit. During the fragmentation and reassembly process, subsequent data fragments belonging to the HTTP session are received and their validity is verified. If the validity verification passes, the subsequent data fragments are written to the external storage medium in a segmented manner. During the HTTP message body restoration process, the restoration process is dynamically judged according to the preset message restoration control strategy. When the termination condition is met or the termination condition in the message restoration control strategy is triggered, the restoration process of the current HTTP message body ends and the corresponding HTTP session is output.

[0007] In some embodiments, mapping the end mode of the HTTP message body to the message restoration state machine based on the message header field includes the following: If the Content-Length field value of the first HTTP message is 0, then the message restoration state machine is set to the completed state; If the Content-Length field value of the first HTTP message is greater than 0, it is mapped to an end mode based on the length identifier, and the target restored length is set to the Content-Length field value. If the Transfer-Encoding field of the first HTTP message contains a chunk transfer identifier, then it is mapped to an end mode based on the chunk transfer identifier, and the target restore length is set to the preset maximum allowed restore length; If the first HTTP message is a response message and does not contain a Content-Length field greater than 0 or a Transfer-Encoding field indicating chunked, then it is mapped to the connection-closed termination method, and the target restore length is set to the preset maximum allowed restore length.

[0008] In some embodiments, determining whether the end condition of the HTTP message body is met includes: In the length-based termination method, it is determined whether the length of the received and disked message body data is equal to the expected message body length in the message header field. If so, the termination condition is met. In the case of termination based on chunk transmission identifier, it is determined whether the received data contains a chunk transmission termination identifier. If so, it is determined that the termination condition is met. In the connection-closed termination method, the connection termination event corresponding to the HTTP session is detected. If a disconnection message is received, it is determined that the termination condition is met.

[0009] In some embodiments, after receiving subsequent data fragments belonging to the HTTP session, the method further includes: Identify subsequent data fragments that do not contain HTTP message headers as non-first packet messages, and set the non-first packet messages to stream mode in the message restoration state machine; The corresponding hash node is located based on the session identifier, and the non-first packet is associated with the established message restoration task.

[0010] In some embodiments, the subsequent data fragments are validated for legality, including: Based on transport layer sequence information, determine whether the sequence number of the subsequent data fragment is within the valid data range calculated from the initial sequence number and the expected message body length, and whether it is compatible with sequence number flipping scenarios; In addition, it determines whether the current data length written to the external storage medium and the total file size after writing do not exceed the pre-configured maximum allowable restore length threshold or the field length declared in the message header.

[0011] In some embodiments, the message restoration control strategy includes a restoration timeout threshold, a maximum allowed restoration length threshold, and restrictions on supported message content types; The termination conditions for triggering the message restoration control strategy include: the size of the file written to the external storage medium reaches the maximum allowed restoration length threshold, or the time waiting to receive subsequent data fragments exceeds the restoration timeout threshold. When any of the termination conditions is triggered, the resources corresponding to the message restoration state machine are released, and the data reassembly and export are terminated.

[0012] In some embodiments, the process of parsing the first HTTP message in the HTTP session further includes: Parse the message content type field in the first HTTP message; If the message content type field does not belong to the pre-configured set of supported message content type restrictions, then the message restoration state machine is changed to the completed state.

[0013] Secondly, embodiments of this application provide a state machine-based HTTP fragmented message restoration system, the system comprising: an acquisition module, a parsing module, a judgment module, a verification module, and a restoration module, wherein: The acquisition module is used to acquire data packets in network traffic, perform session cohesion on the data packets based on connection identification information to establish a corresponding HTTP session, and initialize a message restoration state machine for the HTTP session. The parsing module is used to determine whether the HTTP message contains an HTTP message body based on the message header field of the first HTTP message in the HTTP session. If so, it maps the end mode of the HTTP message body to the message restoration state machine for processing based on the message header field. The judgment module is used to determine whether the end condition of the HTTP message body is met. If not, the corresponding message restoration task is submitted to the fragmentation and reassembly processing unit for asynchronous processing. The fragmentation and reassembly processing unit is independent of the HTTP protocol parsing unit. The verification module is used to receive subsequent data fragments belonging to the HTTP session and perform legality verification during the fragmentation and reassembly process. If the legality verification passes, the subsequent data fragments are written to the external storage medium in a segmented manner. The restoration module is used to dynamically judge the restoration process according to a preset message restoration control strategy during the HTTP message body restoration process. When the end condition is detected or the termination condition in the message restoration control strategy is triggered, the current HTTP message body restoration process is terminated and the corresponding HTTP session is output.

[0014] Thirdly, embodiments of this application provide a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the method described in the first aspect above.

[0015] Fourthly, embodiments of this application provide a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the method described in the first aspect above.

[0016] Compared to related technologies, the HTTP fragmented message restoration method and system based on a state machine provided in this application have the following advantages: 1. This application achieves a unified and compatible processing architecture, resolving the scalability bottleneck of multi-branch processing. Existing technologies typically implement three independent processing flows for three termination modes: Content-Length, chunked, and close-end. Furthermore, support for chunked and close-end is often non-standard or patched (hacked), leading to code redundancy and poor compatibility. This application proposes a unified message restoration state machine based on message semantics and TCP states. It abstracts and maps different termination methods based on length identifiers, chunked transmission identifiers, and connection closure to a single type of state machine for flow processing. The state transition mechanism completely replaces complex multi-branch judgment logic, significantly improving the execution efficiency of protocol parsing and system scalability.

[0017] 2. This application's solution exhibits strong system robustness against abnormal traffic and malicious attacks, resolving the memory resource exhaustion problem. Most existing network protocol analysis solutions heavily rely on system memory to cache and assemble complete fragmented data. When facing abnormal network environments such as malicious forgery of excessively large Content-Length messages, black-hole packet attacks, and loss of critical packets, their defense capabilities are extremely weak, easily leading to memory exhaustion or prolonged blocking and deadlock states. Compared to traditional pure memory splicing solutions, this application employs a mechanism of receiving data while simultaneously verifying its validity and segmenting it for disk storage on external media; it also introduces a dynamic control strategy including restoration timeout, maximum allowed restoration length, and message content type restrictions. Upon detecting any abnormal termination condition, it can promptly circuit break the restoration process and forcibly release resources. This design makes the system robust against attacks, particularly suitable for high-availability security monitoring of high-volume, long-running HTTP message bodies in generative AI scenarios. Attached Figure Description

[0018] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings: Figure 1 This is a flowchart of a state machine-based HTTP fragmented message restoration method according to an embodiment of this application; Figure 2 This is a schematic diagram of the state transition and processing flow of a state machine-based HTTP fragmented message restoration method according to an embodiment of this application; Figure 3 This is a structural block diagram of a state machine-based HTTP fragmented message restoration system according to an embodiment of this application; Figure 4 This is a schematic diagram of the internal structure of an electronic device according to an embodiment of this application. Detailed Implementation

[0019] To make the objectives, technical solutions, and advantages of this application clearer, the application is described and illustrated below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application. All other embodiments obtained by those skilled in the art based on the embodiments provided in this application without inventive effort are within the scope of protection of this application.

[0020] Obviously, the accompanying drawings described below are merely some examples or embodiments of this application. Those skilled in the art can apply this application to other similar scenarios based on these drawings without any inventive effort. Furthermore, it is understood that although the efforts made in this development process may be complex and lengthy, for those skilled in the art related to the content disclosed in this application, any changes to design, manufacturing, or production based on the technical content disclosed in this application are merely conventional technical means and should not be construed as insufficient disclosure of the content of this application.

[0021] In this application, the reference to "embodiment" means that a specific feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of this application. The appearance of this phrase in various places in the specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment that is mutually exclusive with other embodiments. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described in this application may be combined with other embodiments without conflict.

[0022] Unless otherwise defined, the technical or scientific terms used in this application shall have the ordinary meaning understood by one of ordinary skill in the art to which this application pertains. The terms “a,” “an,” “an,” “the,” and similar words used in this application do not indicate quantity limitation and may indicate singular or plural. The terms “comprising,” “including,” “having,” and any variations thereof used in this application are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or device that includes a series of steps or modules (units) is not limited to the listed steps or units, but may also include steps or units not listed, or may include other steps or units inherent to these processes, methods, products, or devices. The terms “connected,” “linked,” “coupled,” and similar words used in this application are not limited to physical or mechanical connections, but may include electrical connections, whether direct or indirect. “Multiple” used in this application refers to two or more. “And / or” describes the relationship between related objects, indicating that three relationships may exist; for example, “A and / or B” can represent: A alone, A and B simultaneously, and B alone. The character " / " generally indicates that the preceding and following objects are in an "or" relationship. The terms "first," "second," and "third" used in this application are merely to distinguish similar objects and do not represent a specific ordering of the objects.

[0023] In generative AI scenarios, AI models output data by reasoning and responding simultaneously. Even when carrying a Content-Length field, its value can be extremely large, theoretically reaching infinity. Furthermore, when using chunked transmission, the time waiting for the end character to arrive is unpredictable; and in Close-end mode, the proportion of data in the response message that does not contain a Content-Length field is extremely high. Existing technologies mostly implement branch processing for the three ending methods mentioned above, relying on memory concatenation. When faced with malicious forgery of excessively large Content-Length values, black hole messages, or loss of critical messages, these technologies are vulnerable to attacks that could lead to memory exhaustion or a deadlock in the analysis system, failing to meet the real-time processing requirements of engineered security systems.

[0024] In view of this, this embodiment provides a state machine-based method for restoring fragmented HTTP messages. This method is suitable for scenarios involving the restoration of HTTP message bodies where the message body length cannot be predetermined and the messages are continuously generated. Figure 1 This is a flowchart of a state machine-based HTTP fragmented message restoration method according to an embodiment of this application. The process includes the following steps: Step S101: Configure message restoration control parameters according to requirements, obtain the original data packets in the network traffic, and perform session cohesion on the data packets based on the connection identification information to establish the corresponding HTTP session and initialize the message restoration state machine for the HTTP session.

[0025] In the implementation process, key parameters of the message restoration control policy are pre-configured based on resource availability and security requirements. Configuration items include the maximum allowed restoration length (MAX-CONTENT_LENGTH), timeout period (WAIT-TIME-OUT), and the set of supported message content types (CONTENT-TYPE). After configuration, raw data packets from the network environment are continuously acquired through the network interface.

[0026] Subsequently, the five-tuple information of the data packet (i.e., source IP, destination IP, source port, destination port, and transport layer protocol type) is extracted as connection identification information. Based on this five-tuple, the traffic is coheded and reassembled to establish the corresponding HTTP session, and the message restoration state machine is initialized for the newly established HTTP session.

[0027] During the initialization phase, the initial message state of the session is set to INVALID MODE, pending state updates in the subsequent protocol parsing phase.

[0028] Step S101 establishes the boundaries of resource usage from the source by pre-configuring multi-dimensional dynamic control parameters (such as maximum length and timeout) and implementing precise session cohesion and state machine initialization based on the quintuple.

[0029] Step S102: Parse the first HTTP message in the HTTP session, identify the message header field of the first HTTP message to determine whether it contains an HTTP message body, and if it does, map the end method of the HTTP message body to the message restoration state machine for processing based on the message header field.

[0030] The first HTTP message in the session is parsed to determine whether it is a request (HTTP REQUEST) or a response. The message header fields are then parsed to determine the existence of the message body and its termination semantics, and processed directly through a state transition mechanism instead of separate processing via branching logic. Specifically, the following decision logic is included: First, parse the Content-Type field in the first HTTP message. If the type indicated by this field does not belong to the pre-configured set of supported CONTENT-TYPE types, actively filter the message and change the message restoration state machine to the complete status, without performing reassembly and restoration.

[0031] Secondly, the message body length and transmission encoding fields are identified. If the message header contains a Content-Length field with a value of 0, it indicates that there is no message body data, and the message restoration state machine is directly set to the complete status.

[0032] If the Content-Length field value is greater than 0, it indicates that the message body length is explicitly specified. Map it to a length-based end mode (CONTENT-LENGTH MODE) and set the target restored length (content-len) to the Content-Length field value.

[0033] If the message header contains a Transfer-Encoding field and includes a chunked transfer identifier, it indicates that chunked transfer encoding is used. This is mapped to a CHUNKED MODE based on the chunked transfer identifier. Since the overall length is unknown, the target restored length (content-len) is set to the pre-configured MAX-CONTENT_LENGTH.

[0034] If the message is identified as a response message and contains neither a Content-Length field greater than 0 nor a Transfer-Encoding field indicating chunked data, it is implicitly determined to be a connection-closed termination mode (CLOSE-END MODE). In this mode, the target restore length (content-len) is also set to the pre-configured MAX-CONTENT_LENGTH.

[0035] Step S102 involves deep analysis of key fields in the first HTTP header, mapping the semantic termination of the three protocols to a single state machine pattern. Compared to the complex branching decisions of traditional solutions, this improves the execution efficiency of protocol parsing and enhances the system's compatibility with diverse and continuously generated AI traffic.

[0036] Step S103: Determine whether the HTTP message body termination condition is met. If it is not in a completed state, submit the corresponding message restoration task to the fragment reassembly processing unit for asynchronous processing.

[0037] Under the control of a unified message restoration state machine, it checks whether the current processing stage has met the end conditions of the HTTP message body. Specific judgment rules vary depending on the different state machine modes. In the CONTENT-LENGTH MODE, the length of the received and written message body data is compared with the expected message body length (content-len) in the message header field. If they are equal, the state machine is marked as complete, and the termination condition is met.

[0038] In the CHUNKED MODE, the end of the HTTP message body is determined by identifying whether the received data stream contains a last-chunk identifier (consisting of a specific length of "0"s and CRLF). If the last-chunk identifier is detected, the end condition is considered met.

[0039] In the CLOSE-END MODE, the connection termination event corresponding to the HTTP session is continuously monitored. If a FIN packet (disconnection packet) is received from the sender, the message transmission is considered terminated, and the termination condition is met.

[0040] If, after the above checks, the HTTP session status is neither complete nor INVALID MODE, it indicates that the HTTP message body has not been fully received. The message restoration task corresponding to this message is then submitted to a separate fragmentation and reassembly unit for asynchronous processing. By employing a multi-threaded mechanism, the fragmentation and reassembly unit and the HTTP protocol parsing unit operate independently, avoiding data waiting that could block the parsing process.

[0041] Step S103 employs a multi-threaded asynchronous processing architecture, physically isolating the HTTP protocol parsing unit from the high-volume data fragmentation and reassembly unit. This decoupling mechanism eliminates the blocking of the core parsing link by I / O wait during the reassembly of massive data fragments, improving throughput and real-time performance in high-concurrency security monitoring scenarios.

[0042] Step S104: During the fragment reassembly process, receive subsequent data fragments belonging to the HTTP session, set the streaming mode, and associate them with the established message restoration task.

[0043] After the fragmentation and reassembly process is initiated, it continuously listens for and receives data fragments belonging to the current HTTP session. Upon arrival, the system identifies the data fragments; if a subsequent data fragment does not contain a valid HTTP header, it is identified as a non-first-packet message. This non-first-packet message is then set to stream mode in the message restoration state machine.

[0044] Subsequently, the corresponding hash node is located based on the session identifier carried by the current data fragment. If a corresponding hash node is found, the non-first packet data is appended to the established message restoration task, preparing for the validity verification and storage process. Non-first packets must have a matching hash node to proceed to subsequent restoration steps.

[0045] Step S104 marks subsequent fragments in the session as stream mode and performs source verification based on a strict hash node association mechanism. This ensures that massive fragmented packets can be accurately collected into the corresponding session tasks, preventing out-of-order or cross-session data pollution and guaranteeing the accuracy of session reconstruction.

[0046] Step S105: Perform a validity check on the received subsequent data fragments. If the check passes, write the data fragments to the external storage medium in a segmented manner.

[0047] To prevent abnormal data contamination and alleviate load, a validity check mechanism is introduced during the data fragmentation and reassembly phase. The received data fragments are validated in the following ways: Verification Dimension 1: Sequence range check based on transport layer sequence information. Extract the TCP sequence number (TcpSeqNum) of subsequent data fragments and determine if this sequence number is within the expected valid data range. The calculation formula is to verify whether the sequence number belongs to the closed interval [InitSeq, MaxSeq], where MaxSeq is calculated by adding the initial sequence number InitSeq to the expected message body length content-len. Simultaneously, the judgment logic includes a processing mechanism to accommodate transport layer sequence number reversal scenarios.

[0048] Verification Dimension Two: Length Limit Check Based on Configuration Policy and Message Declaration. The length of the current data fragment is summed with the historical length of the written files to determine whether the total file size after disk writes does not exceed the smaller of the pre-configured maximum allowable restore length threshold (MAX-CONTENT_LENGTH) and the length declared in the Content-Length field of the message.

[0049] If the data fragment passes the aforementioned validity verification, the fragment reassembly processing unit, carrying an identifier file (FileId), writes the data fragment to the external storage medium in a segmented append manner. Through this on-demand write-to-disk mechanism, the complete HTTP message body data is not stored in the working memory. In this embodiment, the external storage medium includes, but is not limited to, at least one of a file system, database storage, or distributed storage system.

[0050] Step S105 introduces a mechanism for rigorous verification of the serial number range and segmented disk partitioning of the external file system, replacing the traditional high-risk pure memory splicing scheme. This immunizes against memory exhaustion attacks targeting large files or infinitely segmented transmissions, enabling the engineering system to remain stable under extreme loads.

[0051] Step S106: Dynamically judge the restoration process according to the preset message restoration control strategy. When the end condition is met or the termination condition is triggered, end the restoration process and output the corresponding HTTP session content.

[0052] During the HTTP message body restoration process, the current session is dynamically assessed and monitored periodically based on a preset message restoration control strategy. This strategy includes a restoration timeout threshold, a maximum allowed restoration length threshold, and message content type restrictions.

[0053] During the reassembly phase, continuous evaluation is conducted to determine if termination conditions are triggered. Triggering conditions include the following two: First, due to network interruption or packet loss causing congestion, the reassembly thread is detected to have waited longer than the configured restore timeout threshold (WAIT-TIME-OUT) for receiving subsequent data fragments. Second, due to receiving abnormal packets or malicious long-connection data, the total size of files written to external storage media has been detected to have reached the preset maximum allowable restore length threshold.

[0054] When the normal state machine logic is detected to meet the HTTP message body termination condition, or when any control policy termination condition is triggered due to an abnormal situation, a termination response instruction is executed, ending the current HTTP message body restoration process. Immediately afterwards, the memory and internal resources such as descriptors occupied by the message restoration state machine are released, and the reassembly ends to export the relevant data. In the event of an abnormal termination condition, the HTTP session will be marked with the corresponding abnormal status for use in security audit analysis.

[0055] Finally, the corresponding complete or partially restored HTTP session content is output for subsequent risk event reconstruction and other analysis and processing. Furthermore, after completing the segmented storage of the HTTP message body, the design supports re-triggering the HTTP message body restoration process based on message characteristic data stored on external storage media, thereby adapting to traffic replay requirements in the network environment.

[0056] This step employs a dual circuit breaker mechanism of timeout and capacity to implement dynamic control, and forcibly reclaims system resources upon termination. This strategy endows the system with strong self-protection and anti-attack robustness, enabling it to promptly stop losses and output critical tracing data in the face of malicious behaviors such as black hole packets or persistent connection crashes.

[0057] also, Figure 2This is a schematic diagram illustrating the state transition and processing flow of a state machine-based HTTP fragmented message restoration method according to an embodiment of this application. Figure 2 As shown, the overall processing architecture is logically divided into two parts: the upper part is the HTTP protocol parsing and state machine transition process, and the lower part is an independent asynchronous fragment reassembly process. In the parsing process, by parsing the HTTPHEAD and CONTENT of the first packet and subsequent packets, state transitions are triggered, including INVALID MODE, CONTENT-LENGTH MODE (highlighted in yellow), CHUNKED MODE (highlighted in yellow), CLOSE-END MODE (highlighted in yellow), and STREAM MODE (highlighted in yellow). When the termination semantics or filtering conditions are met, the status is set to complete or not complete (highlighted in green). After confirming the status and finding or creating a HASHNODE, the data flows to the reassembly thread in the lower part. The reassembly process executes waiting and verification (file size and timeout judgment) in an independent thread, and finally segments the data and disks it to the file medium associated with the FileId, forming a closed-loop secure reassembly logic.

[0058] Steps S101 to S103 above establish a restoration state machine compatible with three termination modes through a unified message restoration design based on HTTP message semantics and TCP states. Different termination methods based on length identifiers, chunked transmission identifiers, and connection closures are uniformly mapped to the message restoration state machine for processing, and a state transition mechanism replaces complex branch judgment code. This mechanism solves the system redundancy and compatibility problems caused by existing technologies implementing independent processes for different termination semantics.

[0059] Meanwhile, in the fragment reassembly process, this embodiment continuously receives data fragments belonging to the same HTTP session and verifies their validity. If the verification passes, the data fragments are written to external storage media in segments. By replacing the traditional memory caching scheme with file-to-disk or external storage segment reassembly, the complete HTTP message body data is not stored in memory, thus solving the memory resource exhaustion problem easily induced in scenarios with large message bodies or chunked transmissions, and improving the system's real-time processing capabilities in engineering environments. Combining a multi-threaded asynchronous mechanism to make the protocol parsing unit and the fragment reassembly processing unit independent effectively improves reassembly performance.

[0060] Furthermore, this application introduces a dynamic message restoration control strategy during the HTTP message body restoration process, comprehensively judging restoration timeout thresholds, maximum allowed restoration length thresholds, and supported message content type restrictions. When the HTTP message body termination condition is met or the termination condition in the aforementioned strategy is triggered, the restoration process ends and resources are released. This control strategy prevents attackers from forging excessively large Content-Length messages, constructing endless chunked data streams, and causing prolonged blocking due to network packet loss. It endows the system with strong anti-attack capabilities and robustness in abnormal network environments such as untrusted black hole packets, critical packet loss, and traffic replay, and possesses targeted processing capabilities for long-running AI-generated traffic characteristics.

[0061] This embodiment also provides a state machine-based HTTP fragmented message restoration system, which is used to implement the above embodiments and preferred embodiments. Figure 3 This is a structural block diagram of a state machine-based HTTP fragmented message restoration system according to an embodiment of this application, as shown below. Figure 3 As shown, the system includes: an acquisition module 30, a parsing module 31, a judgment module 32, a verification module 33, and a restoration module 34.

[0062] The acquisition module 30 is used to acquire data packets in network traffic, perform session cohesion on the data packets based on connection identification information to establish corresponding HTTP sessions, and restore the state machine for HTTP session initialization messages. The parsing module 31 is used to determine whether the HTTP message contains an HTTP message body based on the message header field of the first HTTP message in the HTTP session. If so, it maps the end mode of the HTTP message body to the message restoration state machine for processing based on the message header field. The judgment module 32 is used to determine whether the end condition of the HTTP message body is met. If not, the corresponding message restoration task is submitted to the fragmentation and reassembly processing unit for asynchronous processing. The fragmentation and reassembly processing unit is independent of the HTTP protocol parsing unit. The verification module 33 is used to receive subsequent data fragments belonging to the HTTP session and perform legality verification during the fragmentation and reassembly process. If the legality verification passes, the subsequent data fragments are written to the external storage medium in a segmented manner. The restoration module 34 is used to dynamically judge the restoration process according to the preset message restoration control strategy during the HTTP message body restoration process. When the end condition is detected or the termination condition in the message restoration control strategy is triggered, the current HTTP message body restoration process is terminated and the corresponding HTTP session is output.

[0063] This system decouples protocol parsing and fragmentation / reassembly using multi-threaded asynchronous methods. Utilizing a unified state machine model, it is compatible with complex HTTP message body termination semantics, including explicit fixed-length, chunked transmission, and implicit disconnection, significantly improving system compatibility and processing efficiency. Simultaneously, relying on a mechanism of simultaneous reception and disk write-to-repository and dynamic policy supervision by the restoration module, it abandons the traditional full cache splicing mode, which is prone to memory exhaustion. Even when facing massive long connections, abnormal black hole packets, or malicious ultra-long payload attacks in generative AI scenarios, this system maintains extremely low resource consumption and excellent anti-blocking capabilities. While ensuring high availability in engineering practice environments, it provides an accurate and reliable foundation for subsequent data leakage prevention analysis, security monitoring, and threat tracing.

[0064] In one embodiment, Figure 4 This is a schematic diagram of the internal structure of an electronic device according to an embodiment of this application, such as... Figure 4 As shown, an electronic device is provided, which can be a server, and its internal structure diagram can be as follows. Figure 4 As shown, the electronic device includes a processor, a network interface, internal memory, and non-volatile memory connected via an internal bus. The non-volatile memory stores the operating system, computer programs, and a database. The processor provides computing and control capabilities, the network interface communicates with external terminals via a network connection, the internal memory provides the environment for the operating system, the computer programs are executed by the processor to implement a state machine-based HTTP fragmented message reconstruction method, and the database stores data.

[0065] Those skilled in the art will understand that Figure 4 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the electronic device to which the present application is applied. The specific electronic device may include more or fewer components than shown in the figure, or combine certain components, or have different component arrangements.

[0066] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by hardware related to computer program instructions. This computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments of the above methods. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), Rambus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0067] The technical features of the above embodiments can be combined arbitrarily. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as the combination of these technical features does not contradict each other, it should be considered within the scope of this specification. The above embodiments only illustrate several implementation methods of this application, and their descriptions are relatively specific and detailed, but they should not be construed as limiting the scope of the invention patent. It should be noted that for those skilled in the art, several modifications and improvements can be made without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this patent application should be determined by the appended claims.

Claims

1. A state machine based HTTP fragmented message restoration method, characterized in that, The method includes: Data packets in network traffic are acquired, and session cohesion is performed on the data packets based on connection identification information to establish corresponding HTTP sessions. A message restoration state machine is initialized for the HTTP sessions. Based on the message header field of the first HTTP message in the HTTP session, determine whether the HTTP message contains an HTTP message body. If so, map the end mode of the HTTP message body to the message restoration state machine for processing based on the message header field. Determine whether the end condition of the HTTP message body is met. If not, submit the corresponding message restoration task to the fragmentation and reassembly processing unit for asynchronous processing. The fragmentation and reassembly processing unit is independent of the HTTP protocol parsing unit. During the fragmentation and reassembly process, subsequent data fragments belonging to the HTTP session are received and their validity is verified. If the validity verification passes, the subsequent data fragments are written to the external storage medium in a segmented manner. During the HTTP message body restoration process, the restoration process is dynamically judged according to the preset message restoration control strategy. When the termination condition is met or the termination condition in the message restoration control strategy is triggered, the restoration process of the current HTTP message body ends and the corresponding HTTP session is output.

2. The method of claim 1, wherein, Mapping the HTTP message body's end mode to the message restoration state machine based on the message header fields includes: If the Content-Length field value of the first HTTP message is 0, then the message restoration state machine is set to the completed state; If the Content-Length field value of the first HTTP message is greater than 0, it is mapped to an end mode based on the length identifier, and the target restored length is set to the Content-Length field value. If the Transfer-Encoding field of the first HTTP message contains a chunk transfer identifier, then it is mapped to an end mode based on the chunk transfer identifier, and the target restore length is set to the preset maximum allowed restore length; If the first HTTP message is a response message and does not contain a Content-Length field greater than 0 or a Transfer-Encoding field indicating chunked, then it is mapped to the connection-closed termination method, and the target restore length is set to the preset maximum allowed restore length.

3. The method of claim 2, wherein, Determining whether the HTTP message body's termination condition is met includes: In the length-based termination method, it is determined whether the length of the received and disked message body data is equal to the expected message body length in the message header field. If so, it is determined that the termination condition is met. In the case of termination based on chunk transmission identifier, it is determined whether the received data contains a chunk transmission termination identifier. If so, it is determined that the termination condition is met. In the connection-closed termination method, the connection termination event corresponding to the HTTP session is detected. If a disconnection message is received, it is determined that the termination condition is met.

4. The method of claim 1, wherein, After receiving subsequent data fragments belonging to the HTTP session, the method further includes: Identify subsequent data fragments that do not contain HTTP message headers as non-first packet messages, and set the non-first packet messages to stream mode in the message restoration state machine; The corresponding hash node is located based on the session identifier, and the non-first packet is associated with the established message restoration task.

5. The method of claim 4, wherein, The subsequent data shards are validated for legality, including: Based on transport layer sequence information, determine whether the sequence number of the subsequent data fragment is within the valid data range calculated from the initial sequence number and the expected message body length, and whether it is compatible with sequence number flipping scenarios; In addition, it determines whether the current data length written to the external storage medium and the total file size after writing do not exceed the pre-configured maximum allowable restore length threshold or the field length declared in the message header.

6. The method of claim 1, wherein, The message restoration control strategy includes a restoration timeout threshold, a maximum allowed restoration length threshold, and restrictions on supported message content types. The termination conditions for triggering the message restoration control strategy include: the size of the file written to the external storage medium reaches the maximum allowed restoration length threshold, or the time waiting to receive subsequent data fragments exceeds the restoration timeout threshold. When any of the termination conditions is triggered, the resources corresponding to the message restoration state machine are released, and the data reassembly and export are terminated.

7. The method of claim 6, wherein, The method further includes the process of parsing the first HTTP message in the HTTP session: Parse the message content type field in the first HTTP message; If the message content type field does not belong to the pre-configured set of supported message content type restrictions, then the message restoration state machine is changed to the completed state.

8. A state machine based HTTP fragmented message restoration system, comprising: The system includes: an acquisition module, a parsing module, a judgment module, a verification module, and a restoration module, wherein: The acquisition module is used to acquire data packets in network traffic, perform session cohesion on the data packets based on connection identification information to establish a corresponding HTTP session, and initialize a message restoration state machine for the HTTP session. The parsing module is used to determine whether the HTTP message contains an HTTP message body based on the message header field of the first HTTP message in the HTTP session. If so, it maps the end mode of the HTTP message body to the message restoration state machine for processing based on the message header field. The judgment module is used to determine whether the end condition of the HTTP message body is met. If not, the corresponding message restoration task is submitted to the fragmentation and reassembly processing unit for asynchronous processing. The fragmentation and reassembly processing unit is independent of the HTTP protocol parsing unit. The verification module is used to receive subsequent data fragments belonging to the HTTP session and perform legality verification during the fragmentation and reassembly process. If the legality verification passes, the subsequent data fragments are written to the external storage medium in a segmented manner. The restoration module is used to dynamically judge the restoration process according to a preset message restoration control strategy during the HTTP message body restoration process. When the end condition is detected or the termination condition in the message restoration control strategy is triggered, the current HTTP message body restoration process is terminated and the corresponding HTTP session is output.

9. A computer device comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the method as described in any one of claims 1 to 7.

10. A computer-readable storage medium having stored thereon a computer program, characterized in that, When the program is executed by the processor, it implements the method as described in any one of claims 1 to 7.

Images